TinkerLoading the retrieval index…
Chapter 4 of 7
The layers of a RAG system
The simple pipeline works, so the team cut costs: documents are now split into small pieces and only the single best piece is sent. A sales lead asks about refunds again, and the answer goes wrong.
This chapter: What happens between “a pile of documents” and “the passages the model reads”, and where can it break?
You will learn
- How a chunker splits documents, and how a bad cut hides a rule
- How embeddings turn text into numbers that capture meaning
- How the index and the context assembler deliver the final prompt
Words you’ll meet (4)
- Chunk
- A small piece of a document. RAG searches and hands over chunks, not whole files.
- Overlap
- Repeating the end of one chunk at the start of the next, so a sentence near a cut survives whole.
- Embedding
- A list of numbers (here 384) that captures what a piece of text means. Similar meanings get similar numbers.
- Vector index
- Storage for every chunk’s embedding, built so all of them can be compared with the question at once.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 7
To save cost, your team cut documents into small 16-word pieces and now sends the model only the single best match. A sales lead asks the refund question again.