Run Local Models With Ollama in idapt
Local models buy you privacy by construction and zero marginal cost; idapt makes them a routed lane of your normal workspace instead of a separate app. This is the complete setup, from pairing to verification.
1. Pair the machine
Install Ollama on the computer that will serve models, then install the idapt daemon and pair it to your account (the computers feature page covers pairing). The daemon advertises what Ollama serves; from that moment your machine is a provider.
2. Pull models that actually fit
The model directory's hardware-fit filter reads your machine's real memory and shows what it can serve: no more pulling a 70B model into 16 GB of hope. Rules of thumb by tier:
- 8 GB (VRAM or unified): small models for drafting and classification.
- 16 GB: the capable mid-sizes (Mistral Small class).
- 24 GB+: the 27B class (Gemma 4 27B) runs comfortably.
The local models roundup keeps per-tier picks current. Pulling, retrying, and context presets are managed from idapt directly.
3. Choose your routing posture
- Manual: pick the local model per chat like any catalog model.
- Prefer local: requests that fit route to your machine; the rest fall back to the cloud catalog, in the same conversation.
- Local only: the strict mode; requests wait for your hardware rather than ever leaving it.
4. Verify, don't trust
Every reply attributes where it ran. For sensitive work, check the attribution on the first few replies: "this never left my machine" should be an observation, not an assumption.
Good to know
- Local requests cost nothing per token and are not metered against platform usage.
- Agents can use local models too: a free extraction agent over a folder of documents is a genuinely free pipeline.
- Speed depends on your hardware; a workstation GPU feels interactive, integrated graphics do not.
- What is local inference covers the concept; the help article covers troubleshooting.
Pair one machine tonight and route tomorrow's low-stakes drafting locally: the launch post has the short version of why this lane exists.
Found this helpful? Share it: