The 18x headline hides the number that actually matters
Vercel published a routing latency improvement: 62ms p50 down to 3.4ms p50. Every benchmark headline quoted that 18x figure. It's the wrong number to care about.
For systems where you're chaining four to six synchronous tool calls—agents polling APIs, verifying webhooks, invoking external services in sequence—the median doesn't ship you. The tail does. A 62ms p50 usually means a 200ms p99. A 3.4ms p50 means something closer to 18–22ms p99. That's the difference between feasible and impossible.
Here's why medians lie for agent workloads: one slow call in a chain breaks the entire sequence. If you have six tool invocations and each has a 200ms p99, you're not looking at 6 × 200ms in wall-clock time—you're looking at something closer to 1.2 seconds at the chain level, with the p99 of the sum being the sum of the p99s when they're independent. Your SLA budget might be 1.5 seconds. One tenth of your calls would violate it before you shipped a single feature. Regional routing variance was your SLA blocker, not model quality or throughput.
Vercel's win collapses that variance floor. Variance collapse is the headline that matters.
How regional DNS replicas trade consistency for divergence you can live with
Vercel moved from a centralized routing table to per-region DNS replicas. That's an explicit consistency tradeoff: eventual regional consistency (propagation in the seconds range) instead of strong global consistency.
This is the right shape for agents. Here's why: agents making tool calls tolerate DNS entries that are stale by seconds. They don't tolerate DNS requests that take 62ms p99. A routing table that's 5 seconds behind across regions but answers in 3ms is strictly better than one that's perfectly consistent but makes you wait 62ms p99 for that consistency guarantee.
Cloudflare built this principle into their entire platform—anycast with eventual consistency across 200+ data centers. The consistency window is part of the service definition, not a hidden implementation detail. Vercel made that tradeoff explicit. In CAP theorem terms, they chose availability and partition tolerance over consistency, but they quantified the consistency window so you know what you're running on.
Most platforms hide this decision entirely. You don't know whether your routing is eventually consistent in milliseconds or seconds. You certainly don't know the propagation window. For agents, that opacity is a footgun—you end up tuning retries and timeouts blind.
Where your 6-call agent chain actually spends its wall clock
Walk through a concrete agent chain: the model makes a decision to call Tool A. That decision happens at time T=0. Here's where the time goes:
- T+0–5ms: DNS resolution for Tool A's endpoint. Used to be 10–60ms p99 depending on your region and whether DNS was cached.
- T+5–15ms: TLS handshake (cached session helps, but cold-start handshakes are real).
- T+15–85ms: Tool A executes (Stripe API call, GitHub query, Slack webhook). This is external and you can't optimize it.
- T+85–90ms: Response received and parsed.
- T+90–100ms: Agent reasoning on the result, decision to call Tool B.
- Repeat for Tools B, C, D, E, F.
Add this up on the old routing: 200ms model call + 6 × (40ms DNS + 10ms TLS + 70ms tool execution) = 200 + 6 × 120 = 920ms best case. P99? You're looking at 1.2–1.4 seconds easily. After this week's rollout, DNS is 3–4ms p99 instead of 40ms p99. That's 36ms per hop, times six hops, times the number of calls you make. A single agent run might save 200–400ms in tail latency.
Why async doesn't rescue you: some tool calls need the result to reason forward. Yes, you can parallelize independent calls. But most agent patterns are sequential—each tool result informs the next choice. You're stuck with wall-clock time.
The failure modes this fixes—and the three it doesn't
Routing latency variance is one source of SLA breach. Regional DNS replicas collapse that source. Here's what they don't fix:
Fixed: Cold-start routing jitter, cross-region tool call amplification, sandbox spawn tail. If your DNS lookup was the variance floor, you're done here.
Not fixed: Upstream API rate limits and 429 backoff variance. Stripe rate-limits you; their backoff is exponential and unpredictable. That's upstream of the routing layer.
Not fixed: Model provider queueing under load. Anthropic and OpenAI have request queues. When they're under load, your model call adds latency. Routing doesn't touch this.
Not fixed: Retry-storm amplification when a tool 503s mid-chain. If Tool D fails, your agent typically retries. If that retry also fails, you cascade. Routing speed doesn't prevent cascades; it just makes them tighter and more visible.
The lesson: instrument p99 latency per hop, not per request. If DNS is now 3ms p99 and Tool D's latency is still 200ms p99, you know where your next ship-blocker lives. Most teams measure end-to-end p99 and wonder why agents are slow. Measure the hops.
What we changed in our deploys after the rollout
We tightened per-tool timeouts from 2 seconds to 800ms. When routing jitter was 60ms p99, we padded hard. Now that DNS is 3ms p99, we can be aggressive. If a tool takes longer than 800ms, it's the tool, not the network. Fail fast.
We removed a speculative parallel-call hedge. Before the rollout, we were sometimes making redundant calls to the same tool (both gated on timeout, racing for the first result). The latency variance made that hedge pay for itself. Post-rollout, it's just waste. We deleted the code.
We moved webhook verification back inline instead of deferred. Webhook verification used to be its own async step because we couldn't afford the latency hit during tool invocation. Now we can. Simpler state machine, fewer bugs.
We recalibrated SLA budgets. A six-call agent chain under 1.5s p99 went from "hope and retries" to actually achievable. We now commit to that SLA in contracts and structure our tool timeout ladder around it.
We kept regional pinning for stateful tools. Some tool calls (database writes, payment captures) need to hit the same region twice. We still explicitly pin those to avoid divergence surprises. The regional replicas are eventually consistent, not strongly consistent. For some payloads, that matters.
The design principle: pick platforms that expose the tradeoff
Here's what matters for agents at scale: platforms that hide consistency decisions are platforms you can't instrument. You can't tune what you can't measure.
When you're evaluating a platform for agent routing, ask three questions:
- What's your DNS propagation window? Seconds? Minutes?
- What's your p99 latency floor for a routing lookup?
- Is that floor predictable, or does it jitter based on region or load?
Most vendors won't answer these. That's the signal you need to pass. Vercel published their numbers; so does Cloudflare. If a vendor won't quantify their routing SLA, they're hiding something, and agent workloads will find it at 2am.
This maps upstream too. The Vercel AI Gateway routing rules expose failover and load-balancing explicitly. You can see the tradeoff. That's the pattern to demand from every layer: inference routing, database routing, webhook dispatch. Agents expose hidden latency; platforms that publish their variance budgets will ship you faster than platforms that don't.
Pull your traces and measure the gap
Pull last week's agent traces. Plot p99 minus p50 per tool hop. If that gap is wider than your model call latency (usually 100–150ms), routing variance is your next ship-blocker, not model quality or context window. Fix it first. Alignment and governance follow; observability comes first.