# Local models in EONYX: fully offline


One of the key differences of EONYX is that it is not tied to the cloud. Since the agent talks to **any OpenAI-compatible API**, you can point it at a local model server and work **fully offline**: the code never leaves the machine, no key is needed, and neither is the internet.

<!--more-->

## Running offline

EONYX sees a local model as an ordinary provider — the whole trick is the `base_url` and an empty key.

### The idea

Local servers (Ollama, llama.cpp, LM Studio) expose an endpoint on your machine that is compatible with OpenAI Chat Completions. EONYX treats it as an ordinary provider — the only differences are `base_url` and an empty `api_key`.

{{< admonition type="tip" title="When you need this" >}}
- The code must not leave the perimeter (NDA, closed network).
- There is no stable internet connection.
- You want zero cost and zero risk of leaks.
{{< /admonition >}}

### Local server

{{< tabs defaultTab="0" >}}
{{< tab title="Ollama" markdown="true" >}}

Start the model:

```bash
ollama pull qwen2.5-coder
ollama serve   # endpoint at http://localhost:11434/v1
```

In `eonyx.json`:

```json
{
  "providers": {
    "local": { "type": "openai", "base_url": "http://localhost:11434/v1", "api_key": "" }
  },
  "models": [
    { "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }
  ],
  "default_model": "qwen2.5-coder"
}
```

{{< /tab >}}
{{< tab title="llama.cpp" markdown="true" >}}

`llama-server` from llama.cpp serves an OpenAI-compatible API:

```bash
llama-server -m model.gguf --port 8080
```

```json
"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:8080/v1", "api_key": "" }
}
```

{{< /tab >}}
{{< tab title="LM Studio" markdown="true" >}}

In LM Studio, enable the local server (the Developer / Local Server tab); it comes up at `http://localhost:1234/v1`:

```json
"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:1234/v1", "api_key": "" }
}
```

{{< /tab >}}
{{< /tabs >}}

After that everything works as usual: `eonyx` (TUI) or `eonyx run "task"`. No key is required — the `Authorization` header is not sent when `api_key` is empty.

## In practice

Two things decide whether a local model will be useful: the right context window and a suitable model.

### Context window

Local models usually have a **small context window** (often 8–32K versus a million for cloud models). Be sure to set the model's real `context_window` — this enables **auto-summarization**: at around 80% fill the older turns get compacted, so the conversation does not hit the limit.

```json
{ "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }
```

{{< admonition type="warning" title="Do not overstate the window" >}}
If you set `context_window` larger than what the model actually holds, summarization kicks in too late and the request is rejected by the server. Use the value from the model's documentation (or slightly less).
{{< /admonition >}}

### Tips

- **A model built for code.** For agentic work take code-oriented models (`qwen2.5-coder` and similar) — they follow tools and call formats better.
- **`max_steps` for a weak model.** A local model can get stuck in a loop — keep `max_steps` reasonable (50 by default) so you do not burn time.
- **Hybrid.** Nothing stops you from keeping both a local and a cloud model in the config and switching on the fly (`Ctrl+N`): rough work locally, hard problems on a powerful cloud model.
- **Verification.** `eonyx config` shows which provider and `base_url` were picked up; the token counter in the TUI header works for local models too (the cost will be `$0`).

## Summary

{{< admonition type="abstract" title="Cheat sheet" >}}
- **Local server**
  - Ollama / llama.cpp / LM Studio — OpenAI-compatible endpoint
  - in EONYX it is a provider with a `base_url` and an empty `api_key`
- **Offline**
  - the code never goes to the cloud
  - no key needed, no internet required
- **Context window**
  - set the real `context_window`
  - enables auto-summarization for small windows
- **Hybrid**
  - keep a local and a cloud model side by side
  - switch with `Ctrl+N`
{{< /admonition >}}