Learn
What is AI inference?
Last reviewed July 16, 2026
Inference is running a trained AI model to produce output: you send input, the model computes a response, and nothing about the model changes. It is the counterpart of training, which builds the model's weights in the first place. Every chat reply, code completion, and image caption is inference, and it is what per-token prices meter. Where it runs matters: a provider's servers, a gateway that routes across providers, or your own hardware.
Training vs inference
Training sets a model's weights from data: a one-time, compute-heavy build. Inference uses those frozen weights to answer requests: cheap per call, run billions of times. The economics are opposite: training cost is fixed and sunk by the model's maker, while inference cost scales with your usage, which is why per-token pricing, caching, and model choice are inference-side decisions.
Provider, gateway, or your hardware
The same model weights can serve from several places. The maker's own API gets first-party features soonest. Hosting providers serve open-weight models at competing prices. A gateway fronts many providers with one API and failover. And open-weight models can run on your own machine, trading capability ceilings for privacy and zero per-token cost. Serious setups mix these per workload.
idapt routes each request across providers cheapest-first with automatic failover, shows where every reply ran, and can route to your own hardware through local inference.
Frequently asked
See it in practice
The fastest way to understand it is to use it. Start free, no credit card.
Try idapt free