RAG Pipeline Builder

What it does

RAG Pipeline Builder analyses your document structure and outputs a complete RAG scaffold tailored to your content type and framework preference. It reads your source files, determines the optimal chunking strategy based on document characteristics (heading density, average paragraph length, code block frequency), recommends an embedding model appropriate for your use case and budget, and generates a working vector store configuration with example queries to validate retrieval quality immediately after setup.

The output is production-ready Python code — not a tutorial or a pseudocode sketch. Whether you choose LangChain or LlamaIndex, the scaffold includes a document loader, text splitter, embedding model initialisation, vector store creation, retriever chain, and a set of starter questions drawn from the document content. The skill also generates a hybrid search configuration combining BM25 keyword retrieval with dense vector search, which significantly improves recall on technical documentation and structured data sources.

Capabilities

• PDF, Markdown, and HTML ingestion — automatic loader selection based on file type, with encoding detection
• Chunk size optimisation — analyses document structure to recommend chunk size and overlap values
• Embedding model selection — recommends OpenAI text-embedding-3-small, Cohere embed-v3, or local sentence-transformers based on requirements
• Vector store scaffolding — outputs working setup for Chroma (local), Pinecone (cloud), or FAISS (in-memory)
• LangChain and LlamaIndex output — choose your framework; the scaffold matches its conventions exactly
• Hybrid search (BM25 + semantic) — generates an ensemble retriever that combines keyword and dense retrieval for better recall
• Example query generation — produces 5-10 representative queries to validate retrieval quality before deploying

How to install

skills add rag-builder

Configuration

Add the following to your .claude/skills.json to set your target framework, chunk parameters, and vector store backend:

{
  "rag-builder": {
    "framework": "langchain",
    "chunk-size": 512,
    "chunk-overlap": 50,
    "vector-store": "chroma"
  }
}

Example

Point the skill at a PDF document and it outputs a complete LangChain RAG scaffold:

Build a RAG pipeline for this PDF: docs/handbook.pdf

Example output — complete LangChain Python scaffold:

# Generated by RAG Pipeline Builder — docs/handbook.pdf
# Chunk size: 512 | Overlap: 50 | Store: Chroma

from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

loader = PyPDFLoader("docs/handbook.pdf")
docs = loader.load()

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512, chunk_overlap=50
)
chunks = splitter.split_documents(docs)

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Chroma.from_documents(chunks, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

qa_chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model="gpt-4o"),
    retriever=retriever
)

# Starter queries generated from document content:
# - "What is the onboarding process for new employees?"
# - "What is the company policy on remote work?"