Token economics and model routing

Real-world analogy

What is it?

Optimise cost per accepted task, including retries and human correction. Input, output and cached-token rates can differ by provider. Prices may rise or fall; never build a career plan on a guaranteed direction. Small models are candidates to evaluate, not automatically the right answer.

Real-world relevance

One shared support-ticket product, three optional entry points. Frontend: Compare a deterministic form validator with an unnecessary model call before adding AI. Backend: Compare routing candidates on the same labelled requests, accounting for retries. Mobile: Measure user-visible latency and server inference cost; do not assume on-device inference is free.

Key points

• Start without a model — Use deterministic parsing where the rules are clear.
• Log actual usage — Capture provider-reported input/output/cache usage and the model version. Character count is not actual tokens.
• Record price date — Use the active provider price schedule and preserve units; do not hard-code a forever price.
• Compare accepted outcomes — A low per-call cost can hide expensive retries.
• Route by evidence — Use a cheaper model for validated narrow tasks and escalate on rubric failure or ambiguity.
• Cache safely — Cache only authorised data with correct tenant scope and invalidation.
• Count review time — Record human corrections and latency alongside spend.
• Frontend practice — Compare a deterministic form validator with an unnecessary model call before adding AI.
• Backend practice — Compare routing candidates on the same labelled requests, accounting for retries.
• Mobile practice — Measure user-visible latency and server inference cost; do not assume on-device inference is free.

Code example

def cost_usd(input_tokens, output_tokens, input_per_million, output_per_million):
    values = (input_tokens, output_tokens, input_per_million, output_per_million)
    if any(v < 0 for v in values):
        raise ValueError("negative usage or rate")
    return (input_tokens * input_per_million +
            output_tokens * output_per_million) / 1_000_000

Line-by-line walkthrough

1. 1. Python: rates are explicit inputs rather than guessed current prices. This basic formula excludes cache-specific pricing and other fees; add those categories when your provider reports them.

Spot the bug

We saved 80% because the prompt has 80% fewer characters

Explain like I'm 5

Fun fact

Hands-on challenge

More resources

• Building effective agents (Anthropic)
• Download the free offline lab (Python, tests and instructions) (Teamz Lab)