Overview
Most LLM deployments hallucinate when asked about proprietary data. This RAG pipeline solves that by giving the model grounded context — relevant chunks from the client's actual documentation — before generating any answer.
Architecture
Ingestion Pipeline
- Document intake — files uploaded to a watched folder or via webhook trigger
- Chunking — documents are split into overlapping 512-token chunks using a custom n8n function node
- Embedding — each chunk is embedded via OpenAI's
text-embedding-3-smallmodel - Storage — vectors + original chunk text stored in a Supabase
pgvectortable with metadata (source file, chunk index, timestamp)
Query Pipeline
- User question arrives via webhook or chat UI integration
- Query embedding — the question is embedded with the same model
- Semantic search — a Supabase RPC call runs a cosine-similarity search, returning the top-k most relevant chunks
- Context injection — chunks are assembled into a structured prompt alongside the user's question
- LLM call — Claude or GPT-4 generates an answer, citing which source chunks it used
- Response delivery — answer returned to the caller with source references
Key Design Decisions
- n8n as orchestrator — no custom server needed; the whole pipeline runs as n8n workflows, fully visible and editable by non-engineers
- pgvector over a dedicated vector DB — Supabase already in the client's stack; one fewer moving part
- Metadata filtering — queries can be scoped to a specific document category before the vector search runs
- Source citation — every answer includes chunk references so users can verify claims
Outcome
Reduced the client's support team's time-to-answer for internal policy questions from 15–20 minutes (manual document search) to under 30 seconds.