Chapter 2 of 7
Why not stuff it all in the prompt?
In chapter 1 the assistant invented a refund policy, because it had never read Larkspur’s real one. The team’s first idea: skip anything clever and paste the whole company drive into the prompt.
This chapter: Why does “just give it everything” break down, even with today’s huge context windows?
You will learn
- How the size of real company data compares with what a context window holds
- Why text that does fit still gets missed or goes stale
- Why one shared prompt can’t respect who is allowed to see what
Words you’ll meet (4)
- Corpus
- All the documents the assistant could draw on: here, the company drive.
- Context window
- How much text a model can read in one request, counted in tokens (a token is about ¾ of a word).
- Truncation
- Cutting text off when it doesn’t fit. Whatever is cut, the model never sees.
- Lost in the middle
- Models tend to miss facts buried in the middle of a long prompt.
The picture on the left is live. Each step below asks you to do one thing in it; the story continues here as you go.
Step 1 of 7
Your team tried the obvious fix: paste the company drive into the assistant’s prompt. Ask it about enterprise refunds.