Local models in EONYX: fully offline

One of the key differences of EONYX is that it is not tied to the cloud. Since the agent talks to any OpenAI-compatible API, you can point it at a local model server and work fully offline: the code never leaves the machine, no key is needed, and neither is the internet.

1 Running offline

EONYX sees a local model as an ordinary provider — the whole trick is the base_url and an empty key.

1.1 The idea

Local servers (Ollama, llama.cpp, LM Studio) expose an endpoint on your machine that is compatible with OpenAI Chat Completions. EONYX treats it as an ordinary provider — the only differences are base_url and an empty api_key.

1.2 Local server

ollama pull qwen2.5-coder
ollama serve   # endpoint at http://localhost:11434/v1

{
  "providers": {
    "local": { "type": "openai", "base_url": "http://localhost:11434/v1", "api_key": "" }
  },
  "models": [
    { "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }
  ],
  "default_model": "qwen2.5-coder"
}

llama-server -m model.gguf --port 8080

"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:8080/v1", "api_key": "" }
}

"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:1234/v1", "api_key": "" }
}

After that everything works as usual: eonyx (TUI) or eonyx run "task". No key is required — the Authorization header is not sent when api_key is empty.

2 In practice

Two things decide whether a local model will be useful: the right context window and a suitable model.

2.1 Context window

Local models usually have a small context window (often 8–32K versus a million for cloud models). Be sure to set the model’s real context_window — this enables auto-summarization: at around 80% fill the older turns get compacted, so the conversation does not hit the limit.

{ "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }

2.2 Tips

• A model built for code. For agentic work take code-oriented models (qwen2.5-coder and similar) — they follow tools and call formats better.
• max_steps for a weak model. A local model can get stuck in a loop — keep max_steps reasonable (50 by default) so you do not burn time.
• Hybrid. Nothing stops you from keeping both a local and a cloud model in the config and switching on the fly (Ctrl+N): rough work locally, hard problems on a powerful cloud model.
• Verification. eonyx config shows which provider and base_url were picked up; the token counter in the TUI header works for local models too (the cost will be $0).

3 Summary