The Log Line That Reframed How We Instrument Agents
Vercel's cache-reason header is deceptively simple: a field that logs not just whether a request hit the cache, but why it didn't. MISS, STALE, BYPASS, REVALIDATED — each one tells you something different about what just happened and what it cost. Most teams log tool calls as opaque success/fail spans. The ones who log the reason dimension are the ones still sane at 2am when latency spikes.
We've gotten this wrong before. We shipped agents that logged tool calls with a status field (ok, error) and nothing else. A retrieval returned zero results — was it a query rewrite failure? A threshold filter? An ACL dropping chunks silently? The status was the same either way. We'd burn two hours in Datadog trying to narrow it down, when five characters in the span (reason: low_confidence vs reason: filtered_by_acl) would have collapsed that to thirty seconds.
The mental model shift is small but decisive: log the decision, not just the outcome. A cache-reason log isn't an afterthought — it's a first-class observability primitive that transforms "this is slow" into "this is slow because tool X re-ran despite a 20-second-old cached result, because the schema hash changed."
Why Agents Are Cache Systems in Disguise
Agents are layered caches that nobody treats as caches. The LLM response itself is cached on prompt hash plus model version. Embeddings are cached on chunk content. Tool calls are memoized on argument hash. The context window is a hot cache. The planner's scratchpad gets reused across turns. Each layer has hit/miss/stale semantics, but most teams surface none of it.
When you ship an agent at scale, every redundant layer costs you tokens and wall-clock time. A retrieval that re-runs because you incremented the chunk_id field but didn't change the embedding burns embedding tokens and API calls. A tool that executes twice because the planner saw a different confidence score the second time and didn't recognize the prior invocation burns latency and cost. At scale, orchestration is solved — token routing is the problem. But you can't route what you can't see. Reason codes make the invisible layers visible.
Start simple: if you can describe what the agent did as a flowchart, you can assign a reason code to every decision point. Did the planner decide to call the tool? (reason: initial_call or reason: replan_required.) Did the cache hit? (reason: match, reason: stale, reason: bypass.) Did the retrieval return results? (reason: hit, reason: low_confidence, reason: filtered.) Each reason tag your spans with a single scalar that lets you build histograms, timeseries, and cost breakdowns that actually inform how you tune the system.
The Reason Codes We Now Emit on Every Tool Call
Here's the taxonomy we've converged on. Adapt it to your stack, but the shape matters more than the exact words.
tool.miss: No prior call with this argument hash. Cache was empty.tool.stale: TTL expired. Attach ttl_ms and age_ms to the span so you can histogram when things go stale.tool.bypass: Caller set no_cache=true or the user is an admin re-running a command. Expected, not a problem.tool.revalidated: 304 from upstream; cheap refresh that didn't re-execute the tool.tool.rerun_schema_drift: Input schema hash changed (a field was added, type changed, validation rule tightened). Same logical call, different signature.tool.rerun_planner_forced: LLM asked for the call again despite cache. Attach planner_temp and model to understand why it re-planned.
Each reason gets cost_ms (how long it took) and cost_tokens (if applicable) attached. This is the data that feeds cost-per-successful-action breakdowns — you can ask "which reason codes are burning the most tokens this week?" and actually get an answer backed by data.
Retrieval Misses Lie Louder Than Tool Misses
Tool caches are straightforward because tool calls are deterministic. Retrieval is where reason codes earn their keep, because retrieval failures are silent by default.
We distinguish retrieval.miss (no chunks attempted), retrieval.low_confidence (chunks returned but scored below your threshold), and retrieval.filtered_by_acl (chunks would've been returned but user permissions dropped them). A customer's production system that hits retrieval.low_confidence 40% of the time is a different problem than one hitting retrieval.miss 5% of the time. The first points to embedding quality or chunk boundaries. The second points to query rewrite logic or index gaps.
The gap between eval recall (0.9) and production recall (0.4) is where reason codes do their best work. Log not just whether retrieval hit, but why it didn't: below_threshold, deduped (duplicate chunk already in context), reranker_dropped (passed semantic similarity but failed lexical or domain checks). And log the query rewrite that produced the miss, not just the final query. If your agent rewrote "show me Q3 revenue by region" to "revenue region quarter:Q3" and got zero results, that's the data point that teaches you where the rewrite logic broke.
Run a weekly report: top 10 miss reasons by frequency. The ones that move your needle feed directly into index changes, chunk-size tweaks, and reranker tuning. This is how you diagnose retrieval quality before users file tickets.
The Failure Modes This Catches Before Your Users Do
Silent schema drift is the worst kind. You change an API's input struct, forget to bump a schema_hash field in the cache key, and suddenly a tool that used to cache cleanly re-runs on every turn. 10x cost spike, zero errors, no alerts until someone asks why the bill doubled.
We've shipped a tool with TTL set to 60 seconds on data that changes hourly. Result: 98% of cache hits were stale, and revalidation calls were doubling latency. The dashboard showed "cache hit: 98%" and everyone celebrated until we added reason codes. Then we saw tool.stale dominating the histogram and fixed it in an afternoon.
Planner temperature drift is subtle. One code path sets temperature=0.7 for planning, another doesn't override a global default of 1.0. Same tool call, different LLM behavior, different decision to re-run. The cache looks like it's working until you histogram by reason — then you see tool.rerun_planner_forced spiking in one customer's workflow.
ACL filters that quietly drop 30% of retrieved chunks are invisible without reason codes. The agent still returns an answer (from the 70% that passed), so the call looks successful. But retrieval.filtered_by_acl in your histograms tells you this customer is operating on stale or incomplete data and doesn't know it.
Retry logic can masquerade as a cache miss. A 429 triggers a retry, the retry succeeds, and you log it as tool.miss instead of tool.rate_limited. Over time, your tuning logic can't distinguish between genuine missing data and transient rate limits. Add the reason, and the distinction becomes obvious in a histogram.
Wiring It In Without Rebuilding Your Stack
You don't need a new observability platform. Add a single reason enum to your existing OpenTelemetry spans: agent.cache.reason as a string attribute. Wrap your tool clients with a thin decorator that emits the reason on every return, along with arg_hash and schema_hash so you can group by decision, not by call site.
Dashboards: reason-code cardinality over time (new reasons = new failure modes), cost-per-reason (which decisions burn the most), p95-per-reason (which reasons slow you down). For sampling, keep 100% of non-hit reasons (they're rare and valuable) and sample hits at 1% (they're noise at scale).
Start small. Pick your two most expensive tool calls this week. Add a five-value reason enum to each. Deploy on Tuesday. By Friday, your dashboard will tell you which reason codes are actually firing and which ones you invented. Expand from there. Real agent observability is built on decisions made visible, not outcomes logged after the fact.
Pick your single most expensive tool call this week, add a five-value reason enum to its span, and watch what your dashboard tells you by Friday — if you want a hand wiring it into an existing agent stack, we're at /contact.