Your Agent Task Costs $0.50 in Tokens but $8 in Retries. Where Your LLM Bill Actually Goes.

One task. Dozens of billed calls. The gap between $0.50 and $8 is what this post is about.

TL;DR

• Your LLM invoice shows tokens; it hides outcomes. Failed calls, retries, and context replay bill the same rate as successful work — but produce nothing.
• Productive tokens are ~15–20% of a typical agent bill. Retries, failed-but-billed calls, and context accumulation eat the rest.
• Agentic flows cost 5–25× more than simple chat on the same nominal task (TechAhead, Inference Cost Explosion, 2026).
• A $10 prototype becomes a $1,200/month line item at production load — that's a 120× increase from the same workload (Teamvoy, Hidden Costs of AI Agents, 2026).
• Three levers cut 70–85% of agent cost without touching success rate: bounded retries, context pruning, and model routing.

An engineer we work with budgeted $1/day for their customer-support agent. Clean calculation: ~4,000 input tokens per ticket, 800 output tokens, Claude Sonnet pricing, 15 tickets a day — comes out to roughly $1.05. After three weeks the bill was $12/day. Same tickets. Same model. Same task.

The token math was right. The agent math was wrong.

Vendor invoices itemize tokens. They don't itemize outcomes. Every failed tool call, every retry that replays the full conversation, every task the agent abandoned after four attempts — they all billed full price. The invoice looked like successful work because failed work costs exactly the same as successful work.

This post breaks down where that $11 gap actually came from, and gives you a 5-step framework to find the same gap in your own system.

Why Token Cost Is Only Part of the Bill

The headline number for a frontier-model agent task is real. A Claude Sonnet call with 4,000 input tokens and 800 output tokens costs about $0.02–$0.06 depending on pricing tier. That's the number that shows up in back-of-envelope estimates and product pitches.

What it omits: agents aren't single calls. They're loops.

A typical agent task makes 5–80 LLM calls to complete one unit of work — decomposing the problem, calling tools, reading results, retrying on failures, verifying outputs (LeanLM, 7 Patterns That Blow Up Your Budget, 2025). Each call includes the full conversation history as input. Token consumption scales between 10× and 100× compared to single-turn for the same end result.

Three structural reasons explain the gap:

1. Invoice opacity. Your provider's dashboard shows token counts, not useful work. A call that returns a 500 error from a downstream tool, triggers a retry, and eventually fails still bills at the same rate as a call that succeeds. The invoice doesn't distinguish.

2. Retry loops multiply context. Each retry doesn't just re-send the original prompt — it re-sends the full accumulated conversation history plus tool definitions plus all prior failed attempts. A 3-step task with 2 retries per step bills roughly 6× the happy-path tokens (TechAhead, Inference Cost Explosion, 2026). As an AI infra practitioner put it: "Ten retries on a long conversation can eat $5–$10 in tokens without producing anything useful" (AI infra blog, production incident report, 2025).

3. Context grows quadratically with tool-call count. Each tool call replays most of the prior context. Each parse error retries the call. Each sub-agent spawn carries its own full context. The token bill grows quadratically with tool-call count, not linearly (Zop, Agentic AI FinOps, 2026).

The result: token tsunamis from loops, retries, and context reloads multiply costs 3–7× before optimization (TechAhead, Inference Cost Explosion, 2026). Agentic flows cost 5–25× more than simple chat on the same nominal task.

One task generates 5–80 internal LLM calls. Retries replay the full context each time.

The Three Hidden Cost Drivers

Real agent cost breaks into three buckets — not one. The useful work is the smallest.

Productive tokens are the calls that succeeded and produced value. For a typical production agent, these are 15–20% of total spend.

Retry tokens are the calls re-attempting a step that failed. When a tool returns a timeout, a malformed JSON response, or a rate-limit error, the agent retries. Each retry sends the full context plus the new attempt. If 15% of calls retry, effective cost is already 1.45× the dashboard number (dev.to, The Real Cost of Your AI Agent, 2025). With 20% retry rates, you're at 1.6×. With a flaky external API that fails 40% of the time, one tool can account for 30–40% of the entire bill.

Failed-but-billed tokens are the calls where the agent exhausted retries and gave up. The task failed. The tokens still billed. Typical ranges across observed production systems: 20–30% of spend goes to calls that failed but billed full tokens (dev.to, The Real Cost of Your AI Agent, 2025).

The remaining cost is model over-provisioning — using frontier models for steps that don't need them. That accounts for 30–50% of spend in typical systems.

Here's what the composition looks like for a $5/task workflow:

Retries, failed calls, and tools make up 80% of the bill. Productive tokens are the thin slice.

Cost Amplification at Scale

Per-task numbers look manageable. At scale they compound fast.

Reliability is the most expensive upgrade you can make to an agent. Raising task success rate from 80% to 99.9% requires verification loops, multi-step recovery, and more aggressive retries. That triples spend despite the same task semantics (Teamvoy, Hidden Costs of AI Agents, 2026). You're not doing more work; you're doing the same work but with a safety net that costs 3× more to operate.

Token variability is extreme. Stanford's Digital Economy Lab measured agentic coding tasks and found that runs on the same task can differ by up to 30× in total tokens (Stanford Digital Economy Lab, How Do AI Agents Spend Your Money?, 2025). That's not noise — it's structural variance from stochastic retrieval, tool-call chains, and path-dependent context growth. Your P99 run doesn't just cost more than average; it costs an order of magnitude more.

Multi-step agents compound these effects. An agent making 15–80 LLM calls per task sees retry storms, quadratic context growth, and tool overhead stack simultaneously. Engineers on Reddit report 70–120× cost spikes on multi-step agents versus single-turn estimates (Teamvoy, Hidden Costs of AI Agents, 2026).

The prototype-to-production gap is brutal: a $10 prototype becomes a $1,200/month line item at production load on the same workload (Teamvoy, Hidden Costs of AI Agents, 2026). That's 120×.

The gap between prototype and production isn't just load — it's retry overhead and context growth compounding at scale.

Most teams discover this after the bill arrives. The fix is measurement first.

How to Measure Your Own Cost Breakdown

You can't optimize what you don't see. Most vendor dashboards show token counts — they don't show retry counts, failure rates, or cost per successful outcome. Here's a 5-step framework to get visibility.

Step 1 — Enable full request logging. Capture every LLM call, every tool call, and every failure with timestamp, workflow stage, token counts, and outcome (success / retry / failed). The stage tag is critical: it's how you attribute cost to planning vs. retrieval vs. execution vs. retries.

Most teams already have this infrastructure (CloudWatch, Datadog, or custom middleware). The missing piece is the outcome tag — specifically whether a call was a retry attempt and whether the task ultimately succeeded.

Step 2 — Attribute cost by failure mode. For each call, categorize:

• Success: cost = token_cost + tool_cost
• Retry: cost = (1 + retry_count) × token_cost + tool_cost
• Failed (exhausted retries): cost = token_cost (wasted)

This one change converts your "cost per call" data into "cost per outcome" data. Input tokens drive 60–80% of agentic cost — and retried calls include the full accumulated context every time (LeanLM, 7 Patterns That Blow Up Your Budget, 2025).

Step 3 — Identify your highest-retry tools. Which tools fail most often? Which tools trigger the most expensive retries (because they're called later in a long conversation, when context is largest)? One misbehaving API can account for 30–40% of your bill. Rank tools by (retry_count × avg_context_at_retry_time × cost_per_token).

Step 4 — Calculate cost per successful task. The correct formula:

cost_per_successful_task = (total_run_cost + tool_cost + overhead) / successful_tasks

Compare this to your single-call happy-path estimate. If actual is 3–8× higher, retry overhead dominates. If it's 10–30×, you have either a context growth problem (growing conversations bloating input tokens) or a retry storm on a specific tool.

Step 5 — Build a cost heatmap by workflow step. Graph cost per stage: planning, retrieval, execution, review, retries. Hotspots are obvious. Most teams find that one or two stages account for 60–70% of spend — and at least one of those is retries on a single failing tool.

Once you have this data, optimization priorities become clear.

Optimization Patterns That Actually Move the Number

Once you've measured where cost goes, three techniques cut 70–85% of agent cost without meaningful success-rate regression (Klaws, How to Cut AI Agent Costs by 80%, 2026).

1. Bounded retries and circuit breakers.

Set max retries per tool (3 is empirically reasonable for most external APIs). Set max retries per task (5–10). Set a hard token budget per task — e.g., 3× expected happy-path cost. After N failures or $X spend, the agent reports failure and stops.

The economic effect: prevents rare retry storms from dominating monthly spend. P99 cost drops dramatically. Mean cost drops modestly. It's cheap insurance.

Build in a 15–25% buffer for output-length variation and unexpected spikes (CodeAnt, Why LLM Token Costs Lie, 2026). If your measured retry overhead exceeds 25%, that's your optimization target.

2. Context summarization.

When conversation history exceeds 60K tokens, run a cheap summarization step that compresses prior turns to 5–10K tokens before continuing. This can be done with a fast, cheap model (Haiku, Gemini Flash) — the cost of the summarization call is tiny compared to the reduction in input tokens for every subsequent call.

Effect: 2–5× reduction in input tokens per call after the summarization threshold. Accuracy impact is negligible for most tasks (the model rarely needs verbatim recall of turn 1 to complete turn 15).

3. Model routing by task step.

Frontier models are for frontier problems — complex reasoning, ambiguous tool orchestration, multi-step planning. Parsing structured output, classification, simple retrieval, and formatting don't need Opus-class models.

Route each step to the cheapest model that meets a quality threshold. The economics of this are surprisingly good: if 70% of your agent's calls are "cheap" steps and you route those to a model that costs 10× less, total cost drops 60%+ even if the expensive steps stay unchanged.

The curve is steep early. Bounded retries alone cut ~35% of cost. Add context pruning and you're at ~60% reduction. Model routing closes most of the remaining gap. Caching is incremental after that.

Caching. For deterministic subtasks — same retrieval queries, same tool inputs — cache successful results. Dedup identical queries. Effect: 20–40% reduction in redundant calls, which compounds with the retry reduction.

Real Example: The $12 Task

Here's the specific breakdown for the customer-support agent mentioned at the start.

Setup: Claude Sonnet agent. One ticket per task: classify issue → retrieve relevant docs → generate response. Happy path: 3 LLM calls, 2 tool calls.

Expected cost (happy path):

• Input: ~4,000 tokens, Output: ~800 tokens per task
• At Sonnet pricing (~$3/$15 per 1M tokens): $0.024 in/out
• Tool calls: ~$0.01
• Expected: ~$0.04–$0.07 per task × 15 tickets/day ≈ $1.05/day

Actual cost:

• Vector DB retrieval was timing out ~35% of the time → agent retried 2–3 times per ticket, replaying growing context
• Response generation returned malformed JSON ~15% of the time → parse retry
• In practice: 5–7 LLM calls per ticket (not 3), with expanding context window
• Actual: ~$0.80 per task × 15 tickets/day ≈ $12/day

Breakdown of that $0.80:

The productive tokens were $0.06. Retries cost nearly 6× that.

What fixed it: Set max retries per tool to 2. Added a circuit breaker (stop after 3 total API failures per ticket). Switched to a local embedding cache for frequent retrieval queries. Cost dropped to ~$0.10/task — a 7× reduction, still within the $1.50/day budget.

The fix took one afternoon. The measurement setup took one day. Both were worth it. Practitioners who've traced similar workflows report $5–$8 per task for complex software-engineering agents once you count retries and tool loops (Stanford Digital Economy Lab, How Do AI Agents Spend Your Money?, 2025).

Frequently Asked Questions

My agent costs spiked. Is it because I chose the wrong model?

Probably not. Model price is roughly 10–15% of agent cost in production workflows (AgentPMT, Token Spend vs Tool Spend, 2025). Swapping Opus for Sonnet saves 20–30% of that slice — meaningful, but it leaves 85–90% of the bill untouched. Check retry rate and failed-call percentage first. If those are under control, then model routing is worth optimizing.

How much should I budget for retries as overhead?

Build in 15–25% over your happy-path token estimate (CodeAnt, Why LLM Token Costs Lie, 2026). If measured retry overhead exceeds 25%, that's a signal to dig into which tools are failing and implement bounded retries. Below 15% means you're running a tight system.

Why does adding more reliability make my agent so much more expensive?

Reliability is structurally expensive for agents. Raising success rate from 80% to 99.9% requires retries, verification passes, and fallback strategies. Each of those paths bills tokens. Agents that need high reliability need budget to match — going from 80% to 99.9% success roughly triples spend (Teamvoy, Hidden Costs of AI Agents, 2026).

My costs spike randomly — is that normal?

It's common. Rare retry storms — one tool failing repeatedly at peak load — can account for 60–70% of monthly spend even if they only happen a few times. The distribution of agent costs is heavy-tailed: most tasks are cheap, a few are very expensive. Add per-tool retry counters and circuit breakers. When tool X fails 3 consecutive times, stop retrying and route to a fallback — don't let one flaky API own your bill.

How do I know when an agent is actually cheaper than a human?

Compare cost per successful task to the human alternative. A $30/hour support engineer handles roughly 6–8 tickets per hour — that's $4–$5 per ticket. For an agent to beat that, it needs to succeed at under $4 per ticket fully loaded (retries, tools, infra, human review for edge cases). At $0.80 per task after measurement, the agent wins by 5×. At $12/day for 15 tickets, it's worse than hiring a part-time human.

If your agent costs more than you expected, you're in good company. The standard advice — "optimize token usage" — misses 80% of the bill. The real levers are retry budgets, context pruning, and measuring cost by outcome rather than by call.

If you want to see exactly where your agent's budget is going across multiple providers, Mesh API's cost analytics breaks down spend by workflow stage and retry pattern — the kind of visibility that makes the $0.50/$8 gap visible before the invoice arrives.