New guide: AI Cost Optimization. Cut LLM and AI infrastructure costs without losing quality, measured per accepted task. Read the guide →

Chapter 13: The optimization playbook

Six steps for finding the few changes that will lower your bill the most.

Published

Read to 3 min in full

Overview#

We don’t recommend implementing every technique in this guide; it could take months. Some changes remove a lot of unnecessary work. Others save almost nothing or make the system harder to maintain.

Start with the bill, then look at the workflow. Separate inference, retrieval and tools, infrastructure, subscriptions and operational overhead, attribute those costs to features and accepted tasks, and only then choose what to change.

Practical#

Work in this order:

  1. Find the expensive operations. Identify the largest total costs and the worst cost per accepted task. Look at costly outliers as well as the average.
  2. Remove work that shouldn’t happen. Duplicate requests, repeated searches, needless LLM calculations and unused output can often be fixed without changing providers.
  3. Reconsider model and reasoning choices. Use suitable smaller models for constrained tasks; add routing or cascades only when they make economic sense.
  4. Fix context and response shape. Keep necessary evidence, remove irrelevant material, preserve reusable prefixes and compact agent state deliberately.
  5. Choose how the work runs. Batch work that can wait. Compare a metered API, a flat-rate plan and dedicated capacity, managed or self-hosted, against the remaining workload.
  6. Buy for the workload you have measured. Once volume and request shape are stable, look at committed-use or enterprise discounts, volume tiers, provisioned or reserved throughput, priority versus flexible service tiers, and the higher rates some providers charge for long context. Commit only to usage you have already observed, leave headroom for model and traffic changes, and compare the full cost of the commitment with the metered baseline for the same work.
The order matters: the first two steps remove spending without changing providers or models, so they usually pay back fastest.

Prioritize by net economic impact, confidence, implementation effort and risk. For illustration, removing duplicate extraction might save $400 per month with little engineering work, while a GPU migration promises $1,500 but carries a heavy maintenance load and depends on utilization. The larger headline figure alone shouldn’t decide which comes first.

Before approving any optimization, ask:

  • Which work does this change remove or make cheaper?
  • How much of the total workflow does that work represent?
  • What may get worse: correctness, latency, reliability, privacy or maintainability?
  • What new ongoing cost does the change introduce?
  • Are we comparing equivalent accepted outcomes?

Approve it only if the answers show it lowers the cost of equivalent accepted work once its new costs are counted.

The worksheet has a sheet for ranking candidate changes by saving, payback, confidence, effort and risk, with these questions as columns.

Check AI subscriptions outside your inference bill too: developer assistants, writing and research tools, meeting products, automation seats. Cancelling ten unused $30/month seats on a cancellable contract saves $300 a month, sometimes with less effort than redesigning part of the inference stack.

Ledgerline. Ledgerline’s inference lines went from $1,200, $600 and $800 to $567, $330 and $560: $1,457 a month, 44% lower. An observability tool and regular evaluation runs add $140 a month, so the monthly total is $1,597 and the net saving $1,003, or 38.6%. The work took about $8,000 of engineering time, which pays back in eight months. The order followed the playbook: measure, remove duplicate and unnecessary work, right-size models, fix context, then change how work runs.

Deep Dive#

Measure net savings on the total, so a percentage never lands on the wrong denominator:

Payback only makes sense when monthly net savings are positive. Keep one-time costs (migration, engineering time, contract exit fees) out of the monthly figure and show them through payback. If inference falls from $4,000 to $2,800 but the change adds $300 of tools and $400 of operations, monthly spending becomes $3,500. The inference line fell 30%; the total fell 12.5%, a net saving of $500 a month. If the change took $4,000 of engineering work, payback is 8 months.

A budget dashboard can alert you, but agent spending limits have to be enforced in application code, as Agent loops, validation and retries describes.

Keep the routine light. Check unusual runs often, look at cost by feature and provider every month, and reconsider subscriptions, commitments and infrastructure less often. Large changes in traffic, models or product behavior are a reason to revisit your assumptions.