← Field Notes
Engineering11 August 2026·9 min read·Chris Ma

RETRIEVE
FIRST.

How RAG works, where it breaks, and why most implementations over-engineer the wrong things

RAGLLMAI EngineeringVector Search

RAG fixes two problems simultaneously: a language model's training data goes stale, and a model under-specified on a fact will confidently generate a plausible wrong answer. Retrieval gives it a source to draw from instead. It introduces one new way to fail: bad retrieval with false confidence attached.

The pipeline is seven steps and conceptually simple. Getting retrieval right is not. Most production mistakes aren't in the LLM call. They're in the three steps before it.

Key Takeaways
  • RAG fixes stale training data and hallucination simultaneously by giving the model an actual source to draw from instead of generating plausible-sounding answers from memory.
  • Bad retrieval is worse than no RAG: it adds latency and cost while producing ungrounded answers, now with a citation attached to give them unearned authority.
  • Most production RAG failures happen in the three steps before the LLM call: chunking strategy, embedding quality, and retrieval precision, not in the generation step itself.
  • Hybrid retrieval (keyword plus semantic search) with a reranking step consistently outperforms pure vector search for production use cases with mixed query types.
01

WHAT RAG ACTUALLY DOES

#

RAG (Retrieval-Augmented Generation) pairs a language model with an external retrieval system. Instead of answering purely from memorised training data, the model first retrieves relevant documents from a knowledge source, then generates its answer using those retrieved documents as grounding context. The answer cites specific sources. You can check it.

What it fixes

Staleness: model training has a cutoff. RAG works with information from after that cutoff, or with private data the model never saw.

Hallucination: without retrieval, a model under-specified on a fact generates a plausible-sounding wrong answer. With retrieval, it has an actual source.

What it introduces

False confidence: a RAG system with poor retrieval is worse than no RAG. It adds latency and cost while producing an ungrounded answer, now with a citation attached to give it unearned authority.

vs. fine-tuning

Fine-tuning changes model behaviour: tone, format, task specialisation. RAG changes what facts the model has access to. If you need the model to know something specific, use RAG. If you need it to behave differently, fine-tune. RAG is also cheaper and faster to update: swap or add documents rather than retraining. This matters most when the underlying information changes frequently.

02

THE PIPELINE

#

Seven steps, two phases. Steps 1–4 run at ingestion time (once per document, or when documents update). Steps 5–7 run at query time for every user request. The quality of the query-time steps depends entirely on how well the ingestion steps were done.

INGESTcollect +clean docs1CHUNKsplit intopassages2EMBEDconvert tovectors3INDEXstore invector DB4RETRIEVEtop-k bysimilarity5RERANKnarrow to3-5 chunks6GENERATELLM withcontext7QUERY TIME ——————query-time pipelineingestion pipeline (run once / on update)
Shaded steps run at query time — everything before the dashed line is ingestion.
1
Ingest

Collect and clean source documents — PDFs, markdown, web pages, database records. Quality here determines quality throughout. Cleaning means removing headers and footers, stripping embedded-image text that won't extract, normalising encoding.

2
Chunk

Split documents into smaller passages. Retrieval works at the chunk level. Chunk too small: context gets severed. Chunk too large: relevance gets diluted and tokens get wasted. Chunk size is a real trade-off — the right answer depends on your document type and query patterns.

3
Embed

Convert each chunk into a vector representation capturing semantic meaning. The embedding model choice affects retrieval quality directly. Domain-specific embedding models often outperform general-purpose ones on specialized content.

4
Index

Store vectors in a vector database for fast similarity search. Choice of database affects latency, scale, and operational complexity — but it's a secondary concern until you've got the first three steps right.

5
Retrieve

At query time, embed the user's query and compare against the index to surface the most relevant chunks. This is where hybrid search pays off: vector search finds conceptually related content, keyword search catches exact terms the embedding might blur.

6
Rerank

A secondary model reorders the retrieved candidates by relevance before the final set is sent to the LLM. Common pattern: retrieve top 20, rerank down to 3–5. First-pass vector similarity is a rough filter, not a precise one.

7
Generate

The query plus the final selected chunks are sent to the LLM, which generates the answer grounded in that context. This is the step most people optimise first. It's usually the wrong place to start.

03

CHUNKING STRATEGY

#

Fixed-size chunking — split every N characters — is a fast start and a low ceiling. The chunk boundary is arbitrary, so it frequently falls mid-sentence, severing context and making the extracted chunk ambiguous without what came before or after.

Semantic chunking computes embeddings sentence-by-sentence and starts a new chunk when semantic similarity between adjacent sentences drops below a threshold. Boundaries correspond to where meaning actually shifts. Retrieving a semantically coherent chunk against a semantically similar query produces meaningfully better results — especially on documents that shift topic mid-section.

FIXED-SIZE CHUNKINGsplit every N characterscontext lostSEMANTIC CHUNKINGsplit where meaning shiftssemantic chunking: embed sentence-by-sentence, break when similarity drops. better signal for retrieval
Fixed-size chunks split at arbitrary boundaries — semantic chunks split where meaning shifts.
04

HYBRID RETRIEVAL

#

Dense vector search and sparse keyword search (BM25) are complementary, not competing. Vector search finds conceptually related content even without matching words — useful for paraphrase, synonyms, and domain inference. Keyword search catches exact terms, proper names, codes, and identifiers that embeddings can blur by collapsing similar-sounding but distinct things.

Hybrid retrieval combined with reranking is the default for production systems in 2026. It's not exotic — it's the sensible baseline. The common reranking pattern: retrieve a broad candidate pool (~20), rerank down to 3–5 strong candidates, and send only those to the LLM. Reranking larger pools (100+) rarely pays off — useful signal concentrates at the head of the distribution.

USER QUERY"hybrid mattress forDENSE VECTORsemantic similarityfinds conceptually relatedSPARSE (BM25)keyword matchingcatches names, codes, termsMERGED + RERANKEDtop 20 → rerank → 3–5sent to LLMhybrid outperforms either alone. default in production systems (2026)
Hybrid retrieval — both branches run on the same query, results merge before reranking.
05

QUERY TRANSFORMATION

#

Raw user queries are often poorly shaped for retrieval. Two techniques address this:

Query expansion

Generate several reformulations of the same question to widen the retrieval net. A user asking "how does this work with large files" might retrieve more with "performance characteristics on large datasets" or "scalability with file size" added as parallel queries.

HyDE — Hypothetical Document Embeddings

Have the model generate a hypothetical answer first, then embed and retrieve using that rather than the raw query. The hypothetical answer contains domain language the user's original question likely lacks. It's counterintuitive — you generate before you retrieve — but it works well for knowledge-intensive queries where the user's vocabulary doesn't match the document vocabulary.

06

ADVANCED PATTERNS

#

Know these exist. Don't reach for them by default.

QUERY CLASSIFIERroutes to cheapest pipelinethat can handle the queryVECTOR RAGfast, cheapAGENTIC RAGiterative retrieval loopsGRAPH RAGgraph traversaladaptive RAG: emerging 2026 default; most queries are simple, route them cheap
Adaptive RAG — a classifier routes each query to the cheapest pipeline that can handle it.
Agentic RAG

The model iteratively decides to run multiple retrieval steps, reformulating its own queries and reasoning across rounds before answering. Strong for multi-step questions where a single retrieve-then-generate pass misses intermediate context.

GraphRAG

Builds a knowledge graph over source data and retrieves via graph traversal rather than similarity search. The right tool when questions are relationship-heavy: 'how do these three entities connect' — where the answer isn't in any single chunk and vector similarity won't find it.

Adaptive RAG

A query classifier routes each incoming question to the cheapest pipeline that can handle it. Simple factual questions go to fast vector RAG. Complex multi-step questions go to agentic RAG. Relationship questions go to GraphRAG. Emerging as the sensible default for production systems in 2026 — most real-world queries are simple and don't need the expensive path.

The field's actual lesson

The most common production mistake is not under-engineering RAG — it's over-engineering it. Start with hybrid retrieval plus a reranker. Measure retrieval quality before adding anything else. Only add query transformation, agentic loops, or graph structures once metrics prove the simpler approach genuinely falls short for a specific, real class of queries — not because it seems more sophisticated.

07

TWO TRACKS

#

The right architecture depends on scale. One setup I use routinely; one for production applications.

TRACK APersonal / research RAG
1.

One markdown file per topic — not one giant document. Retrieval works at the chunk level, and a single well-scoped file chunks more predictably than a sprawling one.

2.

Use headings as natural chunk boundaries. A model retrieving 'just the pricing section' should find it by heading alone.

3.

No exotic formatting — strip tables-as-images, embedded screenshots of text, heavy nested formatting. These are extraction failures identical to the Gate 3 problem in AEO.

4.

Plain files on disk (e.g. Obsidian 30-library/) are sufficient at this scale. Claude's context window plus good file organization functions as your retrieval layer.

5.

Update files in place rather than creating 'v2' copies. Retrieval should never have to guess which version is current.

TRACK BProduction / application RAG
1.

Define the question set first. What will this system actually be asked? Build against real query patterns, not hypothetical ones.

2.

Choose a vector database on latency, pricing, and indexing behavior for your actual data volume — not on whichever is most discussed. Chroma and pgvector for smaller deployments; Pinecone, Weaviate, Qdrant, or Milvus for larger scale.

3.

Build hybrid retrieval + reranking before anything fancier. This alone puts you ahead of most production deployments.

4.

Instrument evaluation from the start — don't bolt it on after launch.

5.

Only add complexity (agentic loops, graph retrieval, query transformation) once evaluation data shows the simple pipeline is genuinely insufficient for a real class of queries.

08

FAILURE MODES

#
0

Bad chunking

Context lost at boundaries, or chunks too large to be precise. The most common retrieval quality problem — usually fixed by moving from fixed-size to semantic chunking.

0

Irrelevant embeddings

Retrieval surfaces topically adjacent content that doesn't actually answer the question. Often a sign of a generic embedding model on domain-specific content, or missing query transformation.

0

Outdated index

The knowledge base goes stale even though the system is technically "using RAG." Retrieval is only as current as the last index update.

0

Ambiguous queries

Vague questions retrieve vague or scattered results. Query expansion and HyDE exist specifically for this — but the first fix is often pushing back on the query design.

0

False confidence

A model will generate a fluent answer from irrelevant retrieved context if not explicitly instructed to say when context doesn't address the question. Faithfulness evaluation catches this. Visual inspection usually doesn't.

09

EVALUATION

#

"It looks like it's working" is not evaluation. Systematic evaluation from day one is becoming standard — a majority of new RAG deployments now build it in from the start. Three things to measure:

Retrieval quality

Are the retrieved chunks actually relevant to the query? Measure precision and recall against a labeled test set. This is the most important metric — if retrieval is bad, generation can't save it.

Faithfulness

Does the generated answer reflect what's in the retrieved chunks, or did the model drift from them? A model that ignores its own retrieved context is producing ungrounded output — retrieval failed to constrain it.

Answer relevance

Does the final answer address what was actually asked? Distinct from faithfulness — an answer can be faithful to the retrieved context while still not answering the question if retrieval surfaced the wrong chunks.

RAGAS is a commonly used open framework for scoring these dimensions systematically.

Core RAG mechanism per Lewis et al. (2020), the original RAG paper. Implementation patterns — hybrid retrieval, reranking ratios, adaptive routing, evaluation-first deployment — drawn from practitioner sources reporting on production trends (2026). Treat specific tool and vendor comparisons as directional rather than definitive; verify against current documentation before committing to a specific vector database or framework.

Recommended Reading

Lewis et al. · NeurIPS 2020

The original RAG paper — establishes the retriever-generator architecture and shows it outperforms parametric models on open-domain QA while remaining updateable without retraining.

Karpukhin et al. · EMNLP 2020

Introduces the bi-encoder retrieval approach that powers most production RAG systems — shows dense retrieval substantially outperforms BM25 on knowledge tasks.

Daniel Jurafsky & James Martin · Stanford University (draft)

The standard NLP textbook, with up-to-date chapters on dense retrieval, vector semantics, and language model architecture — freely available at web.stanford.edu/~jurafsky/slp3.

Continue the conversation

If this changed how you think about it — or you think I'm wrong — I want to know.

Corrections, disagreements, and applications all welcome. Replies go directly to Chris.

Get in touch →
Field Notes · PodcastHost + Expert · Gemini TTS

RETRIEVE FIRST

~6-8 min

1× · Two speakers · tap to play