Local models
Most terminal coding agents assume a card on file. Kolkrabbi does not: it will run a whole session against a model on your own hardware, with no key, no account and nothing leaving the machine. This page is the overview — how it finds a local server, what it refuses to do to your system, and where the honest limits are.
Every command on this page was checked against kolk help on .
01 / FOUR WAYS IN
One of them needs no configuration at all. The other three are a single flag.
If a server is already listening on 127.0.0.1:11434, Kolkrabbi adopts it. If not, and the ollama binary is on your PATH, it starts one on a port it picks itself. Your pulled models appear in the /model picker either way.
Point --base-url at the server and Kolkrabbi talks to it directly. No key is sent, because there is none to send.
Same flag. LiteLLM is the useful shape when you want one endpoint in front of several backends, local and remote.
--base-url takes any OpenAI-compatible endpoint. If it answers at /v1, Kolkrabbi will use it.
The desktop in the next room, or a GPU box you reach over Tailscale: /localia add rig 100.64.0.7:11434 probes that address, says what runs there, and remembers it. Its models are then rig/<model>, which /model and the effort dial treat like any other. Kolkrabbi tells an Ollama from anything OpenAI-compatible by asking, so Docker Model Runner, llama.cpp and LM Studio need no setting of their own. An address that is not this machine says so before it is saved: your prompts and file contents go there.
02 / THE COMMANDS
Kolkrabbi's outside-a-session surface is four commands. Everything else is a slash command inside the session, including all of the local-model tooling.
| What you want | What you type |
|---|---|
| Start against a local endpoint | kolk --base-url http://localhost:11434/v1 -m qwen2.5-coder:14b |
| See what this machine can run | /localia |
| List local models, with what fits | /localia models |
| Check a model against your hardware | /localia plan <model> |
| Pull one, explicitly | /localia pull <model> |
| Add a machine on the LAN, or one address | /localia add <name> <host:port> |
| Use a model on one of them | /localia use <name> <model> |
| Run your own model runner, then use it | /localia direct docker model run <model> |
| Switch model mid-session | /model |
| See what it cost and how good it was | /stats |
| Check keys, directories, terminal, network | /doctor |
03 / WHAT IT REFUSES
These are deliberate refusals written into the design, not features that are merely missing. They are the reason it is safe to point at a server you care about.
Kolkrabbi's own rules forbid curl | sh and sudo, and a floor the product steps over is not a floor. If the binary is missing, it names the command that installs it and lets you run it.
A server already listening is adopted read-only. It might be your shell, or a systemd service holding your sign-in. Kolkrabbi stops only the process it started itself.
By itself Kolkrabbi probes one address, 127.0.0.1:11434, and never OLLAMA_HOST, which may point at another machine or at ollama.com. Any other address is probed because you typed it into /localia add. There is no scan, no broadcast and no discovery of machines you did not name.
A pull is an explicit approval. Cloud models you have not pulled can appear as rows with the exact command visible, but nothing downloads on its own.
With no model chosen, Kolkrabbi picks a free gateway model — because nothing can tell whether your 1.5B local model is a better choice than a 480B free one. Local is always a choice you make.
--base-url session needs no key at allVerified on a machine with no Kolkrabbi config and no key anywhere: kolk --base-url … -m … goes straight to the endpoint and fails, if it fails, on the connection rather than on a missing credential. Where a key is still expected today is a subscription plan answering the turns; that is a known rough edge and it does not touch this path.
04 / THE POINT
A local model is slower and usually weaker than a frontier one. Three reasons it is still worth having in the same terminal.
Local turns land in the same local stats file as gateway turns, with a real zero next to them. Over a week that is the clearest picture you will get of what your cloud usage is actually buying you.
No telemetry, no analytics, no background version check, no hosted service. With a local model and no gateway turn, nothing goes anywhere at all.
The effort dial chooses the model. Point low at a local model and max at a frontier one, and routine turns stop costing money without you thinking about it. One honest caveat, measured rather than assumed: for agent work that edits files and runs commands, an 8B local model was not enough for any harness we tested — 0 of 18. Chat and completion at that size are fine; the agent loop is where it breaks.
KEEP READING
Ollama coding agent OpenRouter coding CLI Compared with other agents