LLM Context Cost Modeling & Tiered Pricing Over 200k Tokens

Modeling LLM Context Costs and Tiered Pricing Beyond 200k Tokens

SitePoint TeamPublished inAI·Programming·APIs·
September 14, 2026

The AI briefing for Developers

Stay up to date with AI tools, model releases, and developer workflows that matter.

Weekly. Free. One click to leave.

SitePoint Premium
Stay Relevant and Grow Your Career in Tech

• Premium Results
• Publish articles on SitePoint
• Daily curated jobs
• Learning Paths
• Discounts to dev tools

7 Day Free Trial. Cancel Anytime.

Most engineers building on large language model APIs operate under a flawed assumption: that per-token pricing is flat regardless of how many tokens get sent. It is not. LLM context cost modeling reveals a nonlinear reality once prompts exceed certain thresholds. Google applies tiered Gemini pricing across multiple context ranges, and Anthropic introduces significant shifts in prompt caching economics at 200k tokens. A 300k-token prompt on Google Gemini costs up to twice as much per input token as a 100k-token prompt; on Anthropic, raw input rates are flat, but caching economics shift materially above 200k tokens. At 1,000 requests per day, a 300k-token workload on Gemini 1.5 Pro overspends by roughly $125/day compared to what a flat-rate assumption would predict.

This article builds a purpose-built TypeScript library and interactive CLI tool for modeling tiered pricing beyond 200k tokens. The tool calculates exact nonlinear pricing thresholds, prompt cache break-even points, and retrieval-vs-long-context cost trade-offs across Anthropic, OpenAI, and Google APIs in a single unified model. Every cost function runs against real usage patterns so you can calculate costs instead of estimating them.

Pricing disclaimer: All prices in this article reflect published rates as of mid-2025. Verify at anthropic.com/pricing, openai.com/api/pricing, and ai.google.dev/pricing before production budgeting.

Table of Contents

The Non-Linear Cost Reality Across Major Providers

How Anthropic, OpenAI, and Google Structure Token Pricing

Each major provider structures token pricing differently, but all share one trait: costs are not uniform across the full advertised context window once caching and tiering are considered.

Anthropic prices Claude models with a boundary at 200k tokens that affects caching economics. For Claude 3.5 Sonnet, Anthropic charges $3.00 per million input tokens at all context lengths. The 200k boundary affects prompt caching economics only, not base input pricing. On cache writes, Anthropic adds a 25% premium, so the first time tokens enter the cache they cost $3.75 per million rather than the standard $3.00. Claude 3 Opus charges $15.00 per million input tokens, with output tokens at $75.00 per million (verify at anthropic.com/pricing before production budgeting).

OpenAI prices GPT-4o at $2.50 per million input tokens and $10.00 per million output tokens. OpenAI’s automatic prompt caching provides a 50% discount on cached input tokens, with no additional write cost. OpenAI applies a flat rate across the full 128k window, but factoring in caching behavior, effective input cost drops from $2.50 to $1.25 per million on cache hits, a 50% reduction that reshapes budgets at scale.

Google applies the most explicitly tiered structure. Gemini 1.5 Pro charges $1.25 per million input tokens for prompts up to 128k tokens, then jumps to $2.50 per million for prompts exceeding 128k. Gemini 2.5 Pro uses a similar tiered structure. Output pricing follows the same pattern: $5.00 per million tokens under 128k, $10.00 per million above it for Gemini 1.5 Pro.

This table is designed to be referenced and shared. The key insight it surfaces: Google’s pricing doubles at the 128k boundary, making it the provider where tiered modeling matters most for raw input costs. Anthropic’s input pricing is flat across all context lengths, but its caching economics introduce nonlinearity above 200k tokens.

Google’s pricing doubles at the 128k boundary, making it the provider where tiered modeling matters most for raw input costs.

Why Costs Escalate: The Infrastructure Behind Long Context

The pricing tiers reflect real infrastructure costs. Transformer-based models maintain a key-value (KV) cache that grows with sequence length. Attention computation scales quadratically with context length in naive self-attention implementations, though production systems use FlashAttention to reduce memory overhead. While providers like Google have invested in ring attention and other parallelism techniques, serving a 500k-token context requires roughly 10x the KV-cache memory of a 50k-token context. Providers charge more for longer contexts to cover this overhead.

This creates a distinction worth internalizing: the advertised context window is the maximum a model accepts, while the cost-efficient context window is the range where pricing remains economical for a given workload. For Gemini 1.5 Pro, the advertised window is 1 million tokens, but cost-efficient usage stays under 128k, where per-token input cost is $1.25 rather than $2.50. Engineers who conflate these two numbers consistently overshoot budgets.

Designing the Cost Modeling Library in TypeScript

Project Setup and Dependencies

The tool uses TypeScript on Node.js ≥18 (required for ESM and WASM support), tiktoken for token counting on OpenAI models, and commander.js for CLI structure.

Warning:tiktoken is calibrated to OpenAI’s tokenizer only. For Anthropic models, use the @anthropic-ai/sdk‘s countTokens method. For Google Gemini, use the Gemini API’s countTokens endpoint. Tokenizers differ across providers, and counts can vary by 10-20% for non-English or code-heavy content.

• Node.js ≥18 — run node --version to verify before installing dependencies
• npm ≥8 (or pnpm/yarn equivalent)

{"name":"llm-cost-modeler","version":"1.0.0","type":"module","bin":{"llm-cost":"dist/cli.js"},"scripts":{"build":"tsc","start":"node dist/cli.js"},"dependencies":{"commander":"^12.1.0","tiktoken":"^1.0.15"},"devDependencies":{"typescript":"^5.5.0","@types/node":"^20.14.0"}}

{"compilerOptions":{"target":"ES2022","module":"Node16","moduleResolution":"Node16","outDir":"dist","strict":true},"include":["src"]}

Note: With module: Node16, all relative imports must use .js extensions even when importing .tspricing.js’

llm-cost-modeler/├── package.json├── tsconfig.json└── src/├── pricing.ts├── cache.ts├── rag.ts└── cli.ts

Defining Provider Pricing Tiers as Data

Nonlinear pricing rules are best encoded as typed configuration rather than buried in conditional logic. Each provider’s pricing becomes a structured array of tier objects, where each tier defines a token ceiling and its corresponding per-token rate.

exportinterfacePricingTier{readonly maxTokens:number;readonly inputPricePerMillion:number;readonly outputPricePerMillion:number;readonly cachedInputPricePerMillion:number;readonly cacheWritePricePerMillion:number;}exportinterfaceProviderConfig{readonly name:string;readonly model:string;readonly tiers:readonly PricingTier[];}exportconst providers: ProviderConfig[]=[{name:"Google",model:"Gemini 2.5 Pro",tiers:[{maxTokens:128_000,inputPricePerMillion:1.25,outputPricePerMillion:10.0,cachedInputPricePerMillion:0.3125,cacheWritePricePerMillion:1.25},{maxTokens:Infinity,inputPricePerMillion:2.50,outputPricePerMillion:15.0,cachedInputPricePerMillion:0.625,cacheWritePricePerMillion:2.50},],},{name:"Anthropic",model:"Claude 3.5 Sonnet",tiers:[{maxTokens:200_000,inputPricePerMillion:3.0,outputPricePerMillion:15.0,cachedInputPricePerMillion:0.30,cacheWritePricePerMillion:3.75},{maxTokens:Infinity,inputPricePerMillion:3.0,outputPricePerMillion:15.0,cachedInputPricePerMillion:0.30,cacheWritePricePerMillion:3.75},],},{name:"OpenAI",model:"GPT-4o",tiers:[{maxTokens:128_000,inputPricePerMillion:2.50,outputPricePerMillion:10.0,cachedInputPricePerMillion:1.25,cacheWritePricePerMillion:2.50},],},];

Note that Anthropic’s cacheWritePricePerMillion is $3.75, reflecting the 25% premium on the $3.00 base input rate. OpenAI’s cached rate is exactly 50% of standard input, with no separate write premium (the cacheWritePricePerMillion is set equal to the standard input rate because OpenAI does not charge a distinct write fee).

The Core Cost Calculation Function

The algorithm splits the total token count across tiers, multiplies each segment by its applicable rate, and accumulates. Output tokens are priced separately since they typically do not follow the same tiered structure as input tokens (though some providers tier both).

interfaceCostBreakdown{provider:string;model:string;inputTokens:number;outputTokens:number;tierBreakdown:{ tier:number; tokens:number; cost:number}[];totalInputCost:number;totalOutputCost:number;totalCost:number;}exportfunctioncalculateContextCost(config: ProviderConfig,inputTokens:number,outputTokens:number): CostBreakdown {if(inputTokens <0|| outputTokens <0){thrownewError(`Token counts must be non-negative. Got inputTokens=${inputTokens}, outputTokens=${outputTokens}`);}const tierBreakdown:{ tier:number; tokens:number; cost:number}[]=[];let remainingTokens = inputTokens;let totalInputCost =0;let previousMax =0;for(let i =0; i < config.tiers.length; i++){const tier = config.tiers[i];const tierCapacity = tier.maxTokens ===Infinity? remainingTokens: tier.maxTokens - previousMax;const tokensInTier = Math.min(remainingTokens, tierCapacity);if(tokensInTier <=0)break;const cost =(tokensInTier /1_000_000)* tier.inputPricePerMillion;tierBreakdown.push({ tier: i +1, tokens: tokensInTier, cost });totalInputCost += cost;remainingTokens -= tokensInTier;previousMax = tier.maxTokens ===Infinity? previousMax : tier.maxTokens;}if(remainingTokens >0){thrownewError(`${remainingTokens}input tokens could not be assigned to any pricing tier`+`for${config.name}${config.model}.`+`Ensure the final tier has maxTokens: Infinity or that inputTokens does not exceed the provider's context window.`);}const applicableTier = config.tiers.find(t => inputTokens <= t.maxTokens);if(!applicableTier){thrownewError(`Input token count${inputTokens}exceeds all defined tiers for${config.name}${config.model}`);}const totalOutputCost =(outputTokens /1_000_000)* applicableTier.outputPricePerMillion;return{provider: config.name,model: config.model,inputTokens,outputTokens,tierBreakdown,totalInputCost,totalOutputCost,totalCost: totalInputCost + totalOutputCost,};}

The edge case worth noting: batch pricing. Both Anthropic and OpenAI offer 50% discounts on batch API requests submittede for batch pricing. This function handles real-time pricing; extending it for batch mode requires halving the applicable rates, which can be done by adding a mode parameter to the provider config

Prompt Caching Break-Even Analysis

How Prompt Caching Changes the Math

All three providers now offer prompt caching, but the mechanisms differ in ways that affect your break-even math. Anthropic requires explicit cache control headers and charges 25% above base rate on the first write, then discounts cache reads by 90%. Anthropic requires cache-eligible content to appear at the start of the prompt and to exceed a minimum block size (1,024 tokens for Claude 3; 2,048 tokens for Claude 3.5 and later). Content that does not meet these requirements is processed as standard uncached input. OpenAI automatically caches prompts longer than 1,024 tokens with a 50% discount on cached tokens and no write premium. Google’s context caching offers a 75% discount on cached input tokens.

The cache hit ratio — the percentage of requests that reuse cached context — is the single most important variable. A system prompt reused across every request will have a hit ratio near 100%. A per-user document context that changes frequently might sit at 10-20%.

Calculating Your Cache Break-Even Point

The break-even calculation answers: at what cache hit ratio does the total cost (cache writes plus cached reads plus uncached reads) drop below the cost of sending full uncached context every time? The writesCount parameter accounts for cache TTL expiry: if inter-request gaps exceed the provider’s cache TTL (about 5 minutes for Anthropic), the write cost is incurred again. Set writesCount to the expected number of cache population events across the request window.

import{ providers }from"./pricing.js";importtype{ ProviderConfig }from"./pricing.js";interfaceCacheBreakEven{provider:string;model:string;inputTokens:number;breakEvenHitRatio:number;costAtFullCache:number;costWithoutCache:number;}exportfunctioncalculateCacheBreakEven(config: ProviderConfig,inputTokens:number,totalRequests:number,writesCount:number=1): CacheBreakEven {const tier = config.tiers.find(t => inputTokens <= t.maxTokens)?? config.tiers[config.tiers.length -1];const uncachedCostPerReq =(inputTokens /1_000_000)* tier.inputPricePerMillion;const costWithoutCache = uncachedCostPerReq * totalRequests;const writeCostPerEvent =(inputTokens /1_000_000)* tier.cacheWritePricePerMillion;const totalWritesCost = writeCostPerEvent * writesCount;const cachedReadCost =(inputTokens /1_000_000)* tier.cachedInputPricePerMillion;const denominator = totalRequests *(uncachedCostPerReq - cachedReadCost);if(denominator <=0){thrownewError("Cache read cost >= uncached cost; caching never breaks even for this config.");}const breakEvenHitRatio = totalWritesCost / denominator;const costAtFullCache = totalWritesCost +(totalRequests * cachedReadCost);return{provider: config.name,model: config.model,inputTokens,breakEvenHitRatio: Math.min(Math.max(breakEvenHitRatio,0),1),costAtFullCache,costWithoutCache,};}

For Anthropic’s Claude 3.5 Sonnet with 200k input tokens and 100 requests, the break-even hit ratio is 1.4% — a single cache write followed by two cache hits already saves money, because the 90% cached discount is steep. Teams delay implementing caching because they assume they need near-100% hit ratios. The math says otherwise. This formula uses the writesCount parameter to account for TTL-based expiry in sustained workloads. For a single cache population event, the default writesCount = 1 applies. For an hourly workload against Anthropic’s ~5-minute TTL, set writesCount to 12 per hour to model repeated write costs accurately.

Long Context vs. RAG: Modeling the Cost Trade-Off

When Retrieval Is Cheaper Than Stuffing the Context

The long context vs. RAG cost comparison requires accounting for the full retrieval stack: embedding API calls to vectorize queries, vector database hosting costs (Pinecone, Wea, and the reduced LLM context needed per query. Against that, long-context approaches send the entire corpus per request but skip retrieval infrastructure entirely

The trade-off is not purely financial. RAG introduces retrieval latency and the risk of missed relevant chunks — a risk you can tune by adjusting top-k and recall thresholds, but never eliminate entirely. Long context avoids retrieval errors but incurs higher per-request costs. Latency also grows with context length: expect response times to roughly double between 50k and 200k input tokens on most providers (benchmark on your own deployment, as hardware and batching strategies vary). The accuracy implications are workload-dependent, but the cost implications are calculable.

The Crossover Point Calculator

The crossover function compares daily costs for both approaches given a corpus size, average query size, and daily query volume. The retrievedChunkTokens parameter controls the assumed size of retrieved context per RAG query (e.g., top-10 chunks at 200 tokens each = 2,000 tokens).

import{ calculateContextCost }from"./pricing.js";importtype{ ProviderConfig }from"./pricing.js";interfaceCostComparison{provider:string;longContextDailyCost:number;ragDailyCost:number;recommendation:string;}exportfunctionmodelRetrievalVsContext(config: ProviderConfig,corpusTokens:number,avgQueryTokens:number,outputTokens:number,queriesPerDay:number,embeddingCostPerMillion:number,vectorDbDailyCost:number,retrievedChunkTokens:number=2_000): CostComparison {const longCtx =calculateContextCost(config,corpusTokens + avgQueryTokens,outputTokens);const longContextDailyCost = longCtx.totalCost * queriesPerDay;const embeddingCost =(avgQueryTokens /1_000_000)* embeddingCostPerMillion * queriesPerDay;const ragCtx =calculateContextCost(config,retrievedChunkTokens + avgQueryTokens,outputTokens);const ragDailyCost =(ragCtx.totalCost * queriesPerDay)+ embeddingCost + vectorDbDailyCost;return{provider: config.name,longContextDailyCost,ragDailyCost,recommendation:longContextDailyCost < ragDailyCost ?"Long Context":"RAG",};}

The crossover point shifts with query volume. At fewer than 10 queries per day, vector database hosting costs dominate, making long context cheaper. At high volumes with large corpora, RAG wins because each request processes only a few thousand retrieved tokens rather than the full corpus. The sample session below shows the magnitude: at 500k tokens and 100 daily queries on Gemini, RAG costs $3.45/day vs. $140.00/day for long context.

Building the Interactive CLI Tool

CLI Commands and Interactive Prompts

The three library functions map directly to three CLI commands: cost, cache-breakeven, and compare.

import{ Command }from"commander";import{ providers, calculateContextCost }from"./pricing.js";import{ calculateCacheBreakEven }from"./cache.js";import{ modelRetrievalVsContext }from"./rag.js";const program =newCommand();program.name("llm-cost").description("LLM context cost modeling tool").version("1.0.0");functionparsePositiveInt(value:string, fieldName:string):number{const parsed =Number(value);if(!Number.isInteger(parsed)|| parsed <=0){console.error(`Error: --${fieldName}must be a positive integer, got:${value}`);process.exit(1);}return parsed;}functionparseNonNegativeFloat(value:string, fieldName:string):number{const parsed =parseFloat(value);if(isNaN(parsed)|| parsed <0){console.error(`Error: --${fieldName}must be a non-negative number, got:${value}`);process.exit(1);}return parsed;}program.command("cost").description("Calculate tiered context cost").requiredOption("-p, --provider <name>","Provider: Google, Anthropic, OpenAI").requiredOption("-i, --input <tokens>","Input token count",(v:string)=>parsePositiveInt(v,"input")).option("-o, --output <tokens>","Output token count",(v:string)=>parsePositiveInt(v,"output"),1000).action((opts)=>{const config = providers.find(p => p.name.toLowerCase()=== opts.provider.toLowerCase());if(!config){console.error(`Unknown provider:${opts.provider}. Valid options:${providers.map(p => p.name).join(", ")}`);process.exit(1);}const result =calculateContextCost(config, opts.input, opts.output);console.table(result.tierBreakdown);console.log(`Total: $${result.totalCost.toFixed(4)}`);});program.command("cache-breakeven").description("Calculate prompt cache break-even hit ratio").requiredOption("-p, --provider <name>","Provider name").requiredOption("-i, --input <tokens>","Input token count",(v:string)=>parsePositiveInt(v,"input")).option("-r, --requests <count>","Total requests",(v:string)=>parsePositiveInt(v,"requests"),100).option("-w, --writes <count>","Number of cache write events (for TTL expiry modeling)",(v:string)=>parsePositiveInt(v,"writes"),1).action((opts)=>{const config = providers.find(p => p.name.toLowerCase()=== opts.provider.toLowerCase());if(!config){console.error(`Unknown provider:${opts.provider}. Valid options:${providers.map(p => p.name).join(", ")}`);process.exit(1);}const result =calculateCacheBreakEven(config, opts.input, opts.requests, opts.writes);console.log(`Break-even hit ratio:${(result.breakEvenHitRatio *100).toFixed(1)}%`);console.log(`Cost without cache: $${result.costWithoutCache.toFixed(4)}`);console.log(`Cost at 100% cache: $${result.costAtFullCache.toFixed(4)}`);});program.command("compare").description("Compare RAG vs long-context costs").requiredOption("-p, --provider <name>","Provider name").requiredOption("-c, --corpus <tokens>","Corpus size in tokens",(v:string)=>parsePositiveInt(v,"corpus")).requiredOption("-q, --queries <count>","Queries per day",(v:string)=>parsePositiveInt(v,"queries")).option("--query-tokens <n>","Avg query size in tokens",(v:string)=>parsePositiveInt(v,"query-tokens"),500).option("--output-tokens <n>","Output tokens per query",(v:string)=>parsePositiveInt(v,"output-tokens"),1000).option("--vector-db-cost <dollars>","Vector DB daily cost in USD",(v:string)=>parseNonNegativeFloat(v,"vector-db-cost"),1.0).option("--retrieved-chunks <tokens>","Retrieved chunk size in tokens",(v:string)=>parsePositiveInt(v,"retrieved-chunks"),2000).option("--embedding-cost <perMillion>","Embedding cost per 1M tokens",(v:string)=>parseNonNegativeFloat(v,"embedding-cost"),0.10).action((opts)=>{const config = providers.find(p => p.name.toLowerCase()=== opts.provider.toLowerCase());if(!config){console.error(`Unknown provider:${opts.provider}. Valid options:${providers.map(p => p.name).join(", ")}`);process.exit(1);}const result =modelRetrievalVsContext(config,opts.corpus,opts.queryTokens,opts.outputTokens,opts.queries,opts.embeddingCost,opts.vectorDbCost,opts.retrievedChunks,);console.log(`Long Context: $${result.longContextDailyCost.toFixed(4)}/day`);console.log(`RAG:          $${result.ragDailyCost.toFixed(4)}/day`);console.log(`Recommendation:${result.recommendation}`);});program.parse();

Sample CLI Session

$ npx llm-cost cost -p Anthropic -i350000-o2000┌─────────┬──────┬────────┬────────────┐│ (index) │ tier │ tokens │    cost    │├─────────┼──────┼────────┼────────────┤│    0    │  1   │ 200000 │   0.6000   ││    1    │  2   │ 150000 │   0.4500   │└─────────┴──────┴────────┴────────────┘Total: $1.0800

The tier breakdown shows input costs only. The total includes 2,000 output tokens at $15.00/M = $0.03, in addition to the input tier costs shown ($0.60 + $0.45 + $0.03 = $1.08).

$ npx llm-cost cache-breakeven -p Anthropic -i200000-r100Break-even hit ratio: 1.4%Cost without cache: $60.0000Cost at 100% cache: $6.7500$ npx llm-cost compare -p Google -c500000-q100Long Context: $140.0000/dayRAG:          $3.4500/dayRecommendation: RAG

The sample output demonstrates the core insight: at 500k tokens and 100 daily queries on Google’s Gemini pricing, RAG costs $3.45/day versus $140.00/day for long-context processing. The compare command accepts --vector-db-cost, --query-tokens, --output-tokens, --retrieved-chunks, and --embedding-cost flags to model your actual deployment; defaults are $1.00/day for vector DB, 500 query tokens, 1,000 output tokens, 2,000 retrieved chunk tokens, and $0.10/M embedding cost.

At 500k tokens and 100 daily queries on Google’s Gemini pricing, RAG costs $3.45/day versus $140.00/day for long-context processing.

Three Rules for Context-Length Budgeting

1. Model costs at the actual operating context length, not the base tier rate. Google’s pricing doubles at 128k tokens. Using the sub-128k rate to budget a 300k-token workload understates costs by nearly 50%.
2. Prompt caching reaches break-even at lower hit ratios than most teams expect. At 200k tokens across 100 requests, Anthropic’s break-even hit ratio is 1.4% with a single cache write — even sporadic cache reuse pays off against the write premium. This assumes the cache remains populated; use the writesCount parameter (or --writes CLI flag) to account for TTL-based expiry in sustained workloads.
3. Choosing between RAG and long context depends on cost as much as architecture. The crossover point is calculable and shifts with query volume, corpus size, and provider pricing.

To add a new model, append a ProviderConfig entry and every calculation updates automatically. Token pricing engineering and prompt caching cost optimization are not one-time analyses. They save more as usage grows. Re-verify provider pricing constants regularly — LLM pricing changes frequently, and stale constants will silently produce incorrect estimates.

Sharing our passion for building incredible internet things.