Prompt caching and cost controls

Practical cost control for LLM apps: cache stable prompt prefixes, trim context, choose model tiers, set budgets/max tokens, and measure $/successful task.

Quality projects die from surprise bills as often as from bad models.

Separate static system instructions from dynamic user content when provider caching allows; retrieve less but better (rerank); log token use per feature; alert on spend.

Add a daily $ budget and max_tokens to one script; confirm it stops cleanly.

⚠️ Myth: The smartest model should handle every request.

✓ Reality: Route by difficulty/risk.

• Course 09 RAG eval; Course 24 serving; Course 21 safe API lab
• Provider pricing/caching docs (cite specifically)