# Local models in EONYX: fully offline


EONYX is not tied to the cloud.
It speaks to **any OpenAI-compatible API**.
Point it at a local model server and your code never leaves the machine.
No model key, no model traffic, no cloud.

<!--more-->

## 🏠 Running against a local model

### The idea

Ollama, llama.cpp and LM Studio all expose an endpoint on your machine.
That endpoint is compatible with OpenAI Chat Completions.
EONYX treats it as an ordinary provider.
Only two things differ: `base_url`, and an empty `api_key`.

{{< admonition type="tip" title="When you need this" >}}
- The code must not leave the perimeter, under an NDA or in a closed network.
- The internet connection is unreliable.
- You want zero cost and zero risk of a leak.
{{< /admonition >}}

### Local server

{{< tabs defaultTab="0" >}}
{{< tab title="Ollama" markdown="true" >}}

Start the model:

```bash
ollama pull qwen2.5-coder
ollama serve   # endpoint at http://localhost:11434/v1
```

In `eonyx.json`:

```json
{
  "providers": {
    "local": { "type": "openai", "base_url": "http://localhost:11434/v1", "api_key": "" }
  },
  "models": [
    { "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }
  ],
  "default_model": "qwen2.5-coder"
}
```

{{< /tab >}}
{{< tab title="llama.cpp" markdown="true" >}}

`llama-server` from llama.cpp serves an OpenAI-compatible API:

```bash
llama-server -m model.gguf --port 8080
```

```json
"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:8080/v1", "api_key": "" }
}
```

{{< /tab >}}
{{< tab title="LM Studio" markdown="true" >}}

Enable the local server in LM Studio, on the Developer / Local Server tab.
It comes up at `http://localhost:1234/v1`:

```json
"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:1234/v1", "api_key": "" }
}
```

{{< /tab >}}
{{< /tabs >}}

Everything then works as usual.
`eonyx` for the TUI, `eonyx run "task"` for a single job.
No key is required: with an empty `api_key` the `Authorization` header is not sent.

{{< admonition type="note" title="One call still goes out" >}}
Model traffic stays on your machine.
Licence verification does not.
EONYX checks the licence against eonyx.tech at every startup, so the machine needs to reach it to launch the agent.
{{< /admonition >}}

## 🔧 In practice

### Context window

Local models usually have a **small context window**.
Often 8–32K, against a million for a large cloud model.
Set the model's real `context_window`.
That switches on **auto-summarization**: at about 80% fill the older turns are compacted, so the conversation never hits the wall.

```json
{ "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }
```

{{< admonition type="warning" title="Do not overstate the window" >}}
Set it larger than the model really holds and summarization starts too late.
The server then rejects the request.
Take the number from the model's documentation, or a little less.
{{< /admonition >}}

### Tips

- **Pick a code model.** Agentic work suits code-oriented models like `qwen2.5-coder`. They follow tool call formats better.
- **Watch `max_steps`.** A weak model can loop. The default of 50 keeps a stuck run from burning your time.
- **Go hybrid.** Keep a local and a cloud model side by side and switch with `Ctrl+N`. Rough work locally, hard problems on the big model.
- **Verify.** `eonyx config` shows the provider and `base_url` that were picked up. The token counter in the TUI header works for local models too, and the cost reads `$0`.

## 📋 Summary

{{< admonition type="abstract" title="Cheat sheet" >}}
- **Local server**
  - Ollama / llama.cpp / LM Studio — OpenAI-compatible endpoint
  - in EONYX it is a provider with a `base_url` and an empty `api_key`
- **Your code stays put**
  - model traffic never goes to the cloud
  - no model key needed
  - licence verification still calls eonyx.tech at startup
- **Context window**
  - set the real `context_window`
  - enables auto-summarization for small windows
- **Hybrid**
  - keep a local and a cloud model side by side
  - switch with `Ctrl+N`
{{< /admonition >}}