After rebuilding this script with LangChain a few times now, I wanted to try something different: skip the framework entirely and call the vendor SDKs directly. Partly out of curiosity, partly because I kept noticing that for a script this small (one PDF, one question, no agents, no multi-step tool calls), LangChain’s chains and retrievers were doing a fairly small amount of actual work under a fairly large amount of abstraction.

Like every earlier version, this script is a Retrieval-Augmented Generation (RAG) application: it answers questions about a PDF by retrieving the most relevant passages and handing them to an LLM along with the question. (The original post has a longer explanation of what RAG is and why it works.) What changes here is that every stage of the RAG pipeline is written by hand instead of assembled from framework components.

The code is being updated for three main reasons:

  • I wanted to see exactly what LangChain’s chains were doing for me.
  • Fewer dependencies means fewer things that break when a package gets refactored again.
  • It’s a decent way to actually understand embeddings and retrieval, the core of RAG, instead of trusting a .as_retriever() call to handle it.

You can find the code for this post on GitHub.

What LangChain Was Actually Doing

Stripped down, the RAG pipeline is four steps: read the PDF, chunk the text, embed the chunks and store them, then embed the question, find the closest chunks, and hand them to Claude. The first three steps build the searchable index; the last one is the retrieval and generation that gives RAG its name. LangChain wraps each of those in its own abstraction (PyPDFLoader, CharacterTextSplitter, VoyageAIEmbeddings, FAISS.as_retriever(), create_retrieval_chain). None of that is wrong, but none of it is strictly necessary either. The underlying voyageai, anthropic, and faiss packages will do all four steps directly, with a bit more code but no chain machinery in between.

Reading and Chunking the PDF

The first stage of RAG is getting the source text into pieces small enough to retrieve individually. Retrieval works on chunks rather than whole documents, so Claude receives only the passages relevant to the question instead of the entire PDF.

I swapped PyPDFLoader for pypdf directly, and CharacterTextSplitter for a plain function that splits on sentence boundaries and groups them into chunks:

pip install pypdf

from pypdf import PdfReader

def extract_data(pdf_name):
    reader = PdfReader(pdf_name)
    policy_text = ""
    for page in reader.pages:
        policy_text += page.extract_text()
    return policy_text

def split_text(text, chunk_size=1000, overlap=200):
    sentences = text.split('.')
    chunks = []
    current = ""
    for sentence in sentences:
        if len(current) + len(sentence) < chunk_size:
            current += sentence + "."
        else:
            chunks.append(current.strip())
            current = current[-overlap:] + sentence + "."
    if current.strip():
        chunks.append(current.strip())
    return chunks

Nothing fancy, and it’s admittedly cruder than CharacterTextSplitter, but it’s also about fifteen lines I can read top to bottom without checking documentation for what a separator argument does. The overlap works the same way it did before: each new chunk starts with the last 200 characters of the previous one, so an idea that spans a chunk boundary isn’t lost.

Embedding and Storing with FAISS

Next, each chunk gets turned into an embedding, a vector that captures its meaning, so chunks about similar topics end up close together. Those vectors go into FAISS, which becomes the searchable index for the retrieval step.

voyageai and faiss both work directly with plain Python lists and numpy arrays, so this part is mostly just removing a wrapper class:

pip install voyageai faiss-cpu numpy

import numpy as np
import faiss
import voyageai

def vectorize_and_store(chunks, voyage_api_key):
    vo = voyageai.Client(api_key=voyage_api_key)
    result = vo.embed(chunks, model="voyage-3-large", input_type="document")
    embeddings = np.array(result.embeddings, dtype="float32")

    index = faiss.IndexFlatL2(embeddings.shape[1])
    index.add(embeddings)
    return index, chunks

I’m keeping the chunks list alongside the FAISS index because FAISS only stores vectors and their positions. It doesn’t know or care what text a vector came from. That mapping is on you now, which is exactly the kind of thing .as_retriever() was quietly handling before. Without it, retrieval would hand back a list of row numbers instead of text Claude can actually read.

Note the input_type="document" argument. Voyage embeds documents and queries slightly differently to improve retrieval, so the chunks are embedded as documents here and the question will be embedded as a query in the next step.

Retrieving and Asking Claude

This is where the “retrieval” and “augmented generation” halves of RAG happen, now as two separate functions you can see:

  • retrieve() embeds the question, asks FAISS for the k=4 closest chunks by index, and uses the chunks list to turn those positions back into text.
  • answer_question() joins the retrieved chunks into a single context string, drops it into the system prompt, and sends the question to Claude.
pip install anthropic

import anthropic

def retrieve(question, voyage_api_key, index, chunks, k=4):
    vo = voyageai.Client(api_key=voyage_api_key)
    q_embedding = vo.embed([question], model="voyage-3-large", input_type="query").embeddings[0]
    q_embedding = np.array([q_embedding], dtype="float32")

    distances, indices = index.search(q_embedding, k)
    return [chunks[i] for i in indices[0]]

def answer_question(question, anthropic_api_key, context_chunks):
    client = anthropic.Anthropic(api_key=anthropic_api_key)
    context = "\n\n".join(context_chunks)

    message = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        system=(
            "Use the given context to answer the question. "
            "If you don't know the answer, say you don't know. "
            "Use three sentences maximum and keep the answer concise.\n\n"
            f"Context: {context}"
        ),
        messages=[{"role": "user", "content": question}]
    )
    return message.content[0].text

The f-string in the system prompt is the “augmented” part of Retrieval-Augmented Generation, doing by hand what create_stuff_documents_chain did before. And the instruction to say “I don’t know” when the context comes up short is what keeps Claude answering from the policy rather than filling gaps on its own.

Putting It All Together

Here it is as one script, all the pieces above plus main(). Reading main() top to bottom is now reading the RAG pipeline in order:

RAG stage Function
Load the document extract_data()
Chunk the text split_text()
Embed and store vectorize_and_store()
Retrieve retrieve()
Generate answer_question() 
import os
import sys
import numpy as np
import faiss
import voyageai
import anthropic
from pypdf import PdfReader
from dotenv import load_dotenv
load_dotenv()


def main():
    question = sys.argv[1]
    pdf_name = sys.argv[2]
    anthropic_api_key = os.getenv("ANTHROPIC_API_KEY")
    voyage_api_key = os.getenv("VOYAGE_API_KEY")

    text = extract_data(pdf_name)
    chunks = split_text(text)

    index, chunks = vectorize_and_store(chunks, voyage_api_key)
    context_chunks = retrieve(question, voyage_api_key, index, chunks)
    answer = answer_question(question, anthropic_api_key, context_chunks)

    print(answer)

def extract_data(pdf_name):
    reader = PdfReader(pdf_name)
    policy_text = ""
    for page in reader.pages:
        policy_text += page.extract_text()
    return policy_text

def split_text(text, chunk_size=1000, overlap=200):
    sentences = text.split('.')
    chunks = []
    current = ""
    for sentence in sentences:
        if len(current) + len(sentence) < chunk_size:
            current += sentence + "."
        else:
            chunks.append(current.strip())
            current = current[-overlap:] + sentence + "."
    if current.strip():
        chunks.append(current.strip())
    return chunks

def vectorize_and_store(chunks, voyage_api_key):
    vo = voyageai.Client(api_key=voyage_api_key)
    result = vo.embed(chunks, model="voyage-3-large", input_type="document")
    embeddings = np.array(result.embeddings, dtype="float32")

    index = faiss.IndexFlatL2(embeddings.shape[1])
    index.add(embeddings)
    return index, chunks

def retrieve(question, voyage_api_key, index, chunks, k=4):
    vo = voyageai.Client(api_key=voyage_api_key)
    q_embedding = vo.embed([question], model="voyage-3-large", input_type="query").embeddings[0]
    q_embedding = np.array([q_embedding], dtype="float32")

    distances, indices = index.search(q_embedding, k)
    return [chunks[i] for i in indices[0]]

def answer_question(question, anthropic_api_key, context_chunks):
    client = anthropic.Anthropic(api_key=anthropic_api_key)
    context = "\n\n".join(context_chunks)

    message = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        system=(
            "Use the given context to answer the question. "
            "If you don't know the answer, say you don't know. "
            "Use three sentences maximum and keep the answer concise.\n\n"
            f"Context: {context}"
        ),
        messages=[{"role": "user", "content": question}]
    )
    return message.content[0].text

main()

Running the Script

Same as every other version of this script by now:

$ python llm_no_framework.py "Can I receive a blanket approval for outside work?" "pm-11.pdf"

No, blanket approvals for outside employment will not be granted. Approval for outside work must be obtained through normal administrative channels by the Chancellor or a designated campus administrative officer.

Same answer as the LangChain versions, which is the point. The RAG pipeline didn’t change, only who writes the plumbing. Claude was never trained on PM-11, yet it answers correctly because retrieve() found the relevant section and answer_question() put it in the prompt.

Was It Worth It?

Honestly, for a script this size, yes. It’s a similar line count to the LangChain version, but every line is doing something I can point to, instead of half of them being import statements for classes that wrap other classes. The trade-off is real, though: if this script grew into something with multiple retrievers, conversation memory, or agent-style tool calling, I’d be rebuilding a worse version of what LangChain already gives you for free. For one PDF and one question, that’s a trade I’m fine making.

The bigger payoff was understanding. Writing RAG by hand shows there’s no magic in it: a basic RAG application is just text splitting, a nearest-neighbor search over vectors, and string formatting in front of an LLM call. Once you’ve seen that, the framework versions make a lot more sense too.