The AI pricing paradox unpacked: prices per query have collapsed by hundreds of times, yet business AI bills keep rising. Understanding the training-versus-inference split is the key to making sense of it — and to controlling what you actually spend.

AI-generated (NotebookLM) audio overview. Source: HexLocal in-house research — Why Does AI Cost Money Every Time You Use It? Training vs. Inference Explained (Dr. Priya Nair).

- Training (building the model) is a massive one-time cost paid by the labs — inference (using it) is what businesses actually pay for, every single query

- By common industry estimates, 80–90% of an AI system's lifetime compute cost is inference, not the headline-grabbing training run

- Token length, model size, and whether a model "reasons" before answering are the main levers that determine what any given query costs

- Reasoning models think step by step before responding, generating hidden tokens that make them roughly 8–40x more expensive per query than standard models — and the cost is unpredictable

- AI prices have fallen dramatically (one benchmark: a 75% drop in a single year), but total bills are rising because usage volume has exploded and reasoning models skew costs upward

- Batch processing and shorter prompts are practical tools for trimming inference costs without switching tools or downgrading capability