Core Insight

Context engineering supersedes prompt engineering when building agents. The goal shifts from writing good instructions to curating the minimal set of high-signal tokens that maximize desired behavior at each inference step. Context is a finite resource with diminishing marginal returns — treat it like working memory, not a dump.

---

Why Context Degrades (The Attention Budget Model)

• Transformer architecture creates n² pairwise token relationships — as context grows, attention spreads thin
• Context rot: empirically, recall accuracy degrades as context window fills (needle-in-haystack benchmarks confirm this across all models)
• Models are trained on distributions where shorter sequences dominate → less robust attention patterns for long-range dependencies
• Degradation is a gradient, not a cliff — precision erodes before capability collapses
• Position encoding interpolation extends context length but introduces token position understanding loss

Implication: More context ≠ better performance. Every token added depletes the attention budget.

---

Framework: Anatomy of Effective Context

System Prompts — The "Right Altitude" Principle

Two failure modes bracket the optimal zone:

| Too Specific | Too Vague |

|---|---|

| Hardcoded if-else logic | High-level platitudes |

| Brittle, high maintenance | False assumption of shared context |

| Breaks on edge cases | Fails to constrain behavior |

Optimal: Specific enough to guide behavior, flexible enough to provide heuristics for novel situations.

Structural recommendations:

• Use XML tags or Markdown headers to delineate sections (<instructions>, <background>, ## Tool guidance)
• Start with the minimal prompt on the best available model; add instructions only to fix observed failure modes
• Minimal ≠ short — include all information needed for correct behavior, nothing more

Tools — Efficiency as a Design Constraint