Run models locally

What you need

You need a computer registered to your account, your laptop, a workstation, or a GPU box. idapt installs and manages the Private AI Engine on it for you, so you do not set anything up by hand.

• Linux and macOS are fully supported.

• Windows works on a best-effort basis.

Enable Private AI on your computer

A machine you pair yourself keeps Private AI off until you turn it on, so pairing a computer never quietly starts running models. Enable it when you pair:

idapt-computer up --enable-local-inference

Or turn it on later on a computer you already paired:

idapt-computer service policy enable local-inference

Find a model that fits

Not every open model fits every machine. idapt helps you find ones that do:

• In the model directory, turn on the Runs on my machine filter to show only models you can host on a computer you own.

• On a computer's own model browser, the Fits this computer toggle narrows the list to models that fit that machine's memory and GPU.

Install a model

1. Open the model you want on its details page (look for the Run locally badge, since only some open models are available to install).

2. In the Self-host section, pick a quantization. idapt shows whether it fits your computer's memory and GPU.

3. Choose a computer and click Install. Download progress streams live.

4. When it finishes, the model shows as installed and is ready to use.

Use models you already have

You do not have to install through idapt to use a local model. idapt reads the models already on your computer and makes them usable in chat.

• A model you pulled with Ollama outside idapt shows up in your installed models with no re-download, and you pick it in chat like any other model.

• If you already run Ollama yourself, idapt uses that engine and the models you already have, so idapt and Ollama never keep separate copies.

Use a local model in chat

Open the model selector and pick your installed model. A monitor icon marks models you can run locally: grey when available to install, and filled green when installed on a reachable computer.

When a response runs locally, the model tab above it shows a green monitor badge with the computer's name, so you always know where an answer came from.

Control where models run

Routing lives in AI Routing, on the Local routing panel. There you decide how idapt uses your machines, with three levels of control:

• Pin or block a computer: prefer a specific machine for local work, or keep one out of the rotation.

• Prefer local, fall back to cloud: a soft setting. idapt runs on your computer when it can, and quietly uses the cloud version when your computer is unreachable. A response can run in the cloud, so use this only when that trade-off is fine.

• Use local only: a hard rule. The model never leaves your machine, and a request fails with a clear message when the computer is offline instead of going to the cloud.

Update or remove models

From a computer's Private AI tab you can update the engine, remove models you no longer need, and see which models are currently loaded in memory.

Check Private AI health

The top of a computer's Private AI tab shows a Health card: whether the engine is up, the recent success rate, and how fast it runs, measured as tokens per second and time to first token. It also lists the categories behind any recent failures.

Below your installed models, Private AI activity lists your most recent local runs on that computer, each with its status, model, speed, and the reason for a failure. Health data starts collecting the first time you run a chat on one of the computer's local models.

Manage local models from the CLI or an agent

You can manage local models without the web UI. An agent, the idapt CLI, and the SDK can install a model on a computer, list what is installed, map a raw model id to a catalog model, and remove one. This is useful for scripting a fresh machine or letting an agent set up its own model.

When something goes wrong

When a local chat cannot run, idapt tells you why in the chat and how to fix it, instead of showing a generic provider error. Common cases are a model that is not installed, a computer that ran out of memory, and a computer that is offline, each with the next step to take.

If a local model is missing, your computer is usually offline. Check that the daemon is running and the computer shows as connected. See Daemon troubleshooting.