Long-context methods

Techniques that help models handle longer inputs—bigger context windows, smarter position schemes, retrieval instead of stuffing, and hierarchical summaries.

“Million-token context” marketing is not the same as reliable use of that context. Builders need methods and evals, not vibes.

Options: train/serve with longer windows; better positional methods; sparse/attention efficiency tricks; RAG so you don’t paste entire corpora; summarize-then-answer; sliding windows for logs.

Reading a whole novel vs using a bookmark + notes index—both valid; index often wins for accuracy.

Stuff a long doc into a prompt and ask a detail from the middle (“needle”). Note failures—then try RAG.

⚠️ Myth: Longer context always beats retrieval.

✓ Reality: Models still miss mid-context facts; cost rises fast.

⚠️ Myth: If the window fits, the model will use it all well.

✓ Reality: Attention and training data limit effective use.

• Course 07 context-windows; Course 09 RAG
• Model cards for long-context models (cite specifically)
• Positional encodings chapter