Your Machine Is a Provider Now: Local Inference

Local inference is generally available: pair a computer running Ollama and your machine becomes a provider in your idapt catalog. Requests that route there cost nothing per token and never leave your hardware.

How it works

The idapt daemon on your computer advertises the models Ollama serves. From there, local models behave like catalog models: pick one in any chat, give one to an agent, see it in the model directory with a "runs on my machine" badge.

• Hardware-fit filter: the directory shows which models your machine can actually serve, based on its real memory, so you never pull a model that cannot run.
• Prefer local: one toggle routes what fits to your machine and falls back to the cloud catalog for what does not, in the same conversation.
• Local only: the stricter mode pins requests to your hardware; if the local model cannot take it, the request waits rather than leaking to the cloud.
• Run attribution: every reply shows where it ran, so "this stayed local" is a fact you can check, not a setting you trust.

Why it matters

Privacy stops being a policy and becomes physics: prompts served locally are never transmitted. Cost flips for high-volume small-model work: extraction, classification, and drafting on your own silicon are free per request. And the hybrid routing means choosing local never means giving up the frontier.

Good to know

• Model management is built in: pull, retry, and configure context presets from idapt.
• A workstation GPU or Apple Silicon with 16 GB+ serves the capable mid-sizes; the local models roundup maps picks to hardware tiers.
• Local models work in the Gateway routing too, per workspace preferences.

Get started

Install the daemon on a machine with Ollama and pair it: the feature page has the steps, and Run local models with Ollama is the full guide.