Chapter 1: The true cost of an AI task

Overview#

Most comparisons start with input and output token prices. Those numbers price a model call, but customers pay for a processed document, a resolved issue or a finished coding task.

A single operation may involve classification, retrieval, two model calls, an external search and a retry. The price of the final model response covers only one of those steps. A successful API response can also hide a failed task: syntactically valid JSON can contain the wrong invoice total.

Measure cost per accepted task instead. An accepted task meets the requirements your product sets for that operation.

Practical#

Start with:

Include the costs that apply: inference, embeddings and retrieval, external tools, retries, compute, human correction and a share of fixed capacity. Don’t count a shared platform bill twice, and write down how you allocated it.

A hypothetical document workflow:

Halving the first call saves $0.008. The new total is $0.025, about 24% lower, assuming nothing else changes.

Run 10,000 tasks at an attributable cost of $330 and accept 9,000, and your cost per accepted task is about $0.0367. An alternative that costs $280 but accepts only 7,000 comes to $0.04 per accepted task. The invoice is smaller, but each accepted task costs more.

The worksheet has a sheet that does this calculation for each of your operations.

You don’t need a perfect definition of quality to start. Write down what acceptance means for each operation.

Ledgerline. Ledgerline’s provider bill for October was $2,600. Spread over 72,000 accepted tasks, that is about $0.036 per task, and the average says almost nothing. One extracted invoice costs $0.030, one support answer $0.020 and one vendor-risk report $0.40. Each operation needs its own number before anyone can decide what to change.

Deep Dive#

A simple uncached API call costs:

That formula changes when a provider distinguishes cached reads and writes, reasoning tokens, tool fees or batch modes. If reasoning tokens are already included in reported output tokens, don’t count them twice.

For fixed infrastructure, separate marginal from allocated costs. A service costing $1,000 monthly processes 50,000 accepted tasks at an allocated $0.02 per task. While spare capacity remains, one more task that fits into that capacity may cost much less than that at the margin. Once the service is saturated, the next task may require a new capacity tier.

Look beyond averages. A few long-running agents can push the P95 task cost far above the median, so start investigating in the tail.