OpenAI's metric answers the second question, not the first
OpenAI's framing of "useful work per dollar" in ChatGPT Work is intuitive: measure how much billable output you get per dollar spent on inference. Optimize that ratio. Ship it. But this metric only works if an agent already exists and you're just tuning it. It sidesteps the harder architectural decision: whether your problem needs an agent loop at all.
The trap is subtle. The metric assumes work cleanly decomposes into billable tasks — draft this email, summarize that doc, route this ticket. But that's not how production agents behave. They emit trajectories: 6 tool calls, 2 retries, 1 self-correction, 1 answer. Which unit is the "work"? The metric collapses the question into the thing you already knew how to price.
An analogy: pre-metric France measured cloth in aunes, a unit that varied by town. You could optimize the price per aune all day. Better looms, cheaper dye, faster shipping. But you'd never know what a meter cost until you sent surveyors to the Equator to measure the Earth's circumference and define a standard. The metric system required an architectural decision first — then optimization within it.
The real ROI decision is architectural: agent loop versus orchestrated workflow. Only after you've made that call do you measure the cost of the thing you're actually shipping.
Workflows are the default; agents are the exception
Most projects sold as "agents" are workflows in disguise. A workflow is a directed acyclic graph with LLM nodes. An agent is a loop with tool choice and a termination condition. The distinction matters because they fail differently and cost different amounts to operate.
You need a loop when: the step count is unknown beforehand, tool selection depends on intermediate results, or recovery requires re-planning. You don't need one when: the schema is fixed end-to-end, you have fewer than five steps, and branching is deterministic. Most ticket-routing, document-summarization, and customer-inquiry problems fall into the second category.
In our experience, a workflow in n8n or Inngest fails in one node — you unit-test that node, fix the prompt, redeploy. An agent fails in a trajectory, and you have to replay the whole loop to see where the decision went wrong. The operational cost delta is real. If you can describe the work as a flowchart, build a workflow. If the flowchart has "wait for tool results, then decide what comes next," you're halfway to an agent — and you should measure whether the loop actually saves you on-call pages before you commit to it.
Anthropic tool-use and OpenAI Assistants are built for agent loops. Temporal and Inngest excel at workflows. The substrate matters, but only after you've chosen your architecture.
What 'useful work' actually means when the loop chooses its own steps
ChatGPT Work assumes tasks arrive pre-labeled. Production agents don't. They choose their own steps, retry them, sometimes fail to converge, sometimes hallucinate work that wasn't needed.
Cost-per-token is even worse as a metric for agents. A single token might be a retry that shouldn't have happened. Cost-per-successful-action is better, but it requires defining success at the trajectory level, not the token level. Was the customer actually satisfied? Did the ticket resolve? Did the downstream system accept the output?
Concrete instrumentation: emit OpenTelemetry spans per tool call, annotate with a success label assigned after human or downstream verification, and track p50 and p95 token counts per successful trajectory. The ratio between your worst and best trajectories is your real waste factor. We've seen agents burning $0.18 per resolved ticket because they were retrying retrieval calls that should have failed fast. The generation cost was $0.04. The waste was the loop, not the model.
The measurement stack we actually ship
Before you touch an agent framework, build the workflow version. Use n8n or Inngest. Measure cost and success rate for two weeks on representative inputs. This is your baseline.
Then A/B the agent variant on the same inputs, same tools, same evaluation set. Anthropic tool-use or OpenAI Assistants, doesn't matter — the comparison is the architecture, not the model. Measure end-to-end latency: if your budget is 8 seconds at p95, and retrieval plus tool calls eat 6 seconds, the LLM is rarely the bottleneck. The slowness lives in your infrastructure.
Instrument retries properly. Idempotency keys on every tool call, exponential backoff with jitter, dead-letter queue in Postgres or Redis Streams for failed tasks. Without this, you're measuring chaos, not agents.
Use LangSmith, Langfuse, or Arize. Tag traces with trajectory_id, tool_name, retry_count, and terminal_status. The honest comparison is total cost of ownership over 90 days — inference spend, on-call pages, engineer hours spent debugging silent failures. Not just the model bill.
Where the metric breaks: three failure modes we've debugged at 2am
Failure one: the agent "succeeds" cheaply because it silently truncates work. The success label was too coarse. You measured the case where the agent gave up early and called it a win. You find this when you compare agent output against human ground truth and realize the agent skipped half the task.
Failure two: the workflow looks cheaper per run, but you're spending 4 hours a week patching prompt branches and routing rules. The hidden ops cost is massive. The agent, for all its retries, doesn't need prompt updates every time the input distribution shifts slightly.
Failure three: your retrieval system hits 0.9 recall in eval and 0.6 in production because the query distribution drifted. The agent burns loops retrying, costs explode, and the metric becomes meaningless. RAG recall lies without production telemetry.
The fix pattern: evaluate per-decision, not per-trajectory. Was this tool call the right one? Did the retrieval return the right document? You can't compare cost-per-task across systems until you've agreed what a task is. Same reason France sent surveyors to Ecuador.
How to make the call before you write the first prompt
Start with a checklist. Fixed steps? Deterministic tools? Bounded latency? Yes to all three — build a workflow. Open-ended planning? Tool choice depends on prior results? Recoverable from partial failure? — agent, but budget 3x the ops cost upfront.
The pilot rule we enforce now: ship the workflow first, measure for 30 days, only escalate to an agent loop if a specific failure mode demands it. No agent goes to production without a per-trajectory success definition written down before the first commit. When you can't write it down, you don't understand the work well enough yet.
Our guide to picking between agents and workflows walks the technical signals. The hidden cost of manual ops shows what happens when you measure only the obvious costs and miss the people-hours. Token routing at scale covers what changes when you're pushing real volume.
If you're picking between an agent loop and a workflow this quarter and the ROI math keeps sliding around, that's the call we take every week. Talk to us at /contact and we'll walk your specific decision in 30 minutes.