Two years ago I refactored this code to fix a Chroma issue on Windows, add argument passing, and clean up the structure. Turns out two years is a long time in Langchain years. The code still runs, but it now throws deprecation warnings at you the whole way through, and there’s a newer, cleaner pattern for the actual question-answering part that’s worth switching to.

The code is being updated for three main reasons:

  • RetrievalQA is deprecated and will eventually be removed.
  • The completion-style model I was using (gpt-3.5-turbo-instruct) has been superseded by chat models.
  • A bit of defensive code in extract_data was never actually doing anything useful.

You can find the updated code for this post on GitHub (coming soon!).

The RetrievalQA Problem

RetrievalQA.from_chain_type() was the easy button for question-answering when I wrote the last version of this post. It’s still the easy button, but Langchain has been deprecating it since version 0.1.17, and the old chains have since been shuffled off into a langchain_classic package for backward compatibility. That’s your sign it’s time to move on.

The replacement is create_retrieval_chain paired with create_stuff_documents_chain. It’s a couple more lines than RetrievalQA, but you get an explicit prompt template instead of a black box, which I actually prefer once you get past the extra typing.

Swapping the Model

While I was in there, I also swapped OpenAI(model_name="gpt-3.5-turbo-instruct", ...) for ChatOpenAI(). The old completions-style model still works, but chat models are where the ecosystem’s attention (and the model improvements) have been going, and the new chain helpers are built around chat models anyway.

Cleaning Up extract_data

The old extract_data function had an isinstance check for whether each loaded document was a dict or a string:

if isinstance(doc, dict) and 'text' in doc:
    policy_text += doc['text']
elif isinstance(doc, str):
    policy_text += doc
else:
    policy_text += repr(doc)

I honestly don’t remember why I wrote it that way. PyPDFLoader.load() returns a list of Document objects with a .page_content attribute, full stop. So that got simplified down to one line.

Updated Version

Here is the updated script:

import os
import sys
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import FAISS
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
load_dotenv()


def main():
    question = sys.argv[1]
    pdf_name = sys.argv[2]
    api_key = os.getenv("OPENAI_API_KEY")

    text = extract_data(pdf_name)
    docs = split_text(text)

    docstorage = vectorize_and_store(docs, api_key)
    response = answer_question(question, api_key, docstorage)

    print(response['answer'])
    # return response

def extract_data(pdf_name):
    loader = PyPDFLoader(pdf_name)
    data = loader.load()
    policy_text = ""
    for doc in data:
        policy_text += doc.page_content
    return policy_text

def split_text(text):
    ct_splitter = CharacterTextSplitter(separator='.', chunk_size=1000, chunk_overlap=200)
    docs = ct_splitter.split_text(text)
    return docs

def vectorize_and_store(docs, api_key):
    embedding_function = OpenAIEmbeddings(openai_api_key=api_key)
    docstorage = FAISS.from_texts(docs, embedding_function)
    return docstorage

def answer_question(question, api_key, docstorage):
    llm = ChatOpenAI(model="gpt-4o-mini", openai_api_key=api_key)

    system_prompt = (
        "Use the given context to answer the question. "
        "If you don't know the answer, say you don't know. "
        "Use three sentences maximum and keep the answer concise.\n\n"
        "Context: {context}"
    )
    prompt = ChatPromptTemplate.from_messages([
        ("system", system_prompt),
        ("human", "{input}"),
    ])

    question_answer_chain = create_stuff_documents_chain(llm, prompt)
    qa = create_retrieval_chain(docstorage.as_retriever(), question_answer_chain)
    response = qa.invoke({"input": question})
    return response

main()

Running the Script

Running it looks exactly the same as before — same two arguments, same idea:

$ python llm_faiss_vectorstore.py "Can I receive a blanket approval for outside work?" "pm-11.pdf"

No, blanket approvals for outside employment will not be granted. Approval for outside work must be obtained through normal administrative channels by the Chancellor or a designated campus administrative officer.

The only real difference under the hood: the response dictionary now comes back with an answer key instead of result, since that’s what create_retrieval_chain hands you. If you’re migrating your own code from the old version of this post, that’s the gotcha most likely to bite you.