Unstructured-IO/irs-manual-demo

Chat with IRS Manuals

This directory contains an application for chatting with IRS manuals. Once data is available, the chat application only uses self-hosted models and can be run in a disconnected environment. Here's how to get started with the chatbot:

Installation

pip install -r requirements.txt

Environment Variables

Note there are other options for these connections, but these are the ones referenced in this implementation

OpenAI

Pinecone

PINECONE_API_KEY
PINECONE_API_ENV
OPENAI_API_KEY
PINECONE_INDEX_NAME

Download PDFs from IRS website

python download_data.py <Base URL> <Page Start> <Page End> <Target Directory>

Run PDFs against unstructured-ingest

PYTHONPATH=. ./unstructured/ingest/main.py \
  --local-input-path <ingest-input-dir> \
  --structured-output-dir <ingest-output-dir> \
  # optional parameter -> this will hit the *NEW* API vs. processing locally
  --partition-by-api

Here's an example of the structured json output

Seed and utilize vector db

python ingest_data.py <path-to-structured-json-file-directory>

Run the chat CLI

python cli_app.py

irs-manual-demo

Commits

docs: add -r flag to install command

Update download_data.py

Update README.md

whoops removing commented out logic

fixing content length error

adding json files

README