Primarily developed in Python, LangChain is a robust framework designed to simplify the development of applications that utilize Large Language Models (LLMs). It enables developers to connect these models with various data sources and external APIs, facilitating the integration of advanced natural language processing capabilities into their applications. LangChain provides tools for constructing LLM-powered workflows, handling memory and state management, and ensuring scalability and efficiency. By offering a streamlined interface for working with LLMs, LangChain empowers developers to create sophisticated AI-driven solutions in areas such as chatbots, data analysis, content generation, and more, without needing extensive expertise in machine learning or AI model development.

One of the most common uses of LangChain is building Retrieval-Augmented Generation (RAG) applications, and that is exactly what we will build in this post. Let’s dive in!

What is RAG?

An LLM only knows what it learned during training. It has never read your university’s policy manual, so if you ask it a question about that policy, it will either admit it doesn’t know or, worse, confidently make something up. Retrieval-Augmented Generation solves this problem by giving the model the relevant source material at the moment it answers.

A RAG application works in four steps:

  1. Load and chunk the source documents into smaller pieces of text.
  2. Embed and store each chunk as a vector (a list of numbers representing its meaning) in a vector database.
  3. Retrieve the chunks whose vectors are most similar to the user’s question.
  4. Generate an answer by passing the question and the retrieved chunks to the LLM.

Think of it as the difference between a closed-book and an open-book exam. Instead of asking the model to answer from memory, we let it look up the right pages first. The model’s answer is grounded in your documents, and updating its knowledge is as simple as updating the documents. No retraining or fine-tuning is required.

Getting Started

In this example, I will create a foundational RAG application that ingests information from a PDF file and enables interactive questioning about its content.

You can find the code for this post on GitHub. Depending on your experience with LangChain, you might need to run a few pip install commands or simply use my requirements.txt file. Feel free to use a PDF file of your choice; I have also included a policy memo, PM-11, from the university where I teach.

The smart play would also be to create a virtual environment for this project. I like pipenv but you are welcome to use something different.

$ pipenv shell
$ pip install -r requirements.txt

Use whatever manner you want to edit & run the code. For projects like this, I usually boot up a session using Jupyter Notebooks.

$ jupyter notebook

First, we need to import the necessary libraries and load environment variables:

# Import necessary libraries
import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain.llms import OpenAI
from langchain_openai import OpenAI

load_dotenv()

In the block above, I import essential modules for each stage of the RAG pipeline: handling PDFs, text processing, embeddings, vector storage, and the language model. The load_dotenv function loads environment variables from a .env file, which is useful for securely managing API keys.

Extract Contents from PDF

The first stage of RAG is loading the source material. Here, I load the PDF and extract its contents:

loader = PyPDFLoader("pm-11.pdf")
data = loader.load()

policy_text = ""
for doc in data:
    if isinstance(doc, dict) and 'text' in doc:
        policy_text += doc['text']
    elif isinstance(doc, str):
        policy_text += doc
    else:
        policy_text += repr(doc)

Here, I use PyPDFLoader to load the PDF file and then iterate through the loaded data, concatenating text from the document into a single string. This step ensures I have all the text content in a manageable format.

Chunk the Text

Next, I split the extracted text into chunks. Retrieval works on chunks rather than whole documents, so the model receives only the passages relevant to the question instead of the entire PDF:

ct_splitter = CharacterTextSplitter(separator='.', chunk_size=1000, chunk_overlap=200)
docs = ct_splitter.split_text(policy_text)

In this section, CharacterTextSplitter is used to split the text into smaller chunks. By specifying a separator, chunk size, and overlap, I ensure that the text is divided into manageable pieces of about 1,000 characters. The 200-character overlap means an idea that spans two chunks isn’t lost at the boundary.

Vectorize and Store in DB

Next, I want to vectorize the text chunks and store them in a database for efficient retrieval:

api_key = os.getenv("OPENAI_API_KEY")
embedding_function = OpenAIEmbeddings(openai_api_key=api_key)

vectordb = Chroma(persist_directory='LSU', embedding_function=embedding_function)

docstorage = Chroma.from_texts(docs, embedding_function)

Here, I retrieve the OpenAI API key from environment variables and initialize the OpenAIEmbeddings function to convert text into embeddings. An embedding is a vector that captures the meaning of a chunk, so chunks about similar topics end up close together. I then use Chroma to create a vector database, storing the embeddings of our text chunks. When a question comes in, the vector database can quickly find the chunks closest in meaning to it, which is the “retrieval” in Retrieval-Augmented Generation.

As a note, Chroma creates a SQLite database file inside your project in a subfolder designated by the persist_directory input. In my case, it would be a subfolder called “LSU”.

Connect the Retriever to the LLM

I then configure the language model to work with our vector database:

llm=OpenAI(model_name="gpt-3.5-turbo-instruct", openai_api_key=api_key)

qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=docstorage.as_retriever())

In this block, I initialize the OpenAI language model (gpt-3.5-turbo-instruct) and set up a retrieval-based question-answering system (RetrievalQA). The docstorage.as_retriever() method allows the chain to fetch relevant chunks from our vector database. The "stuff" chain type then “stuffs” those retrieved chunks into the prompt alongside the question before sending it to the LLM. This is the “augmented generation” half of RAG.

It’s worth pointing out that the model itself is not changed or retrained here. Rather than fine-tuning the model on the policy, RAG hands the model the relevant text each time a question is asked.

Query the LLM

Finally, I can query the language model and get answers based on the document content:

question = "Can I get a blanket approval for work outside of LSU?"
response = qa.invoke(question)

print(response['result'])

No, blanket approvals for outside employment will not be granted.

Here, I pose a question to the RAG application. The retriever finds the relevant sections of PM-11 in the vector database, and the language model uses them to generate a response. The LLM was never trained on this policy, yet it answers correctly because the right passage was included in its prompt.

By following these steps, you can harness the power of LangChain to build a versatile RAG application capable of interactive and dynamic querying of document contents.