Local models in EONYX: fully offline

EONYX is not tied to the cloud.
It speaks to any OpenAI-compatible API.
Point it at a local model server and your code never leaves the machine.
No model key, no model traffic, no cloud.

1 🏠 Running against a local model

1.1 The idea

Ollama, llama.cpp and LM Studio all expose an endpoint on your machine.
That endpoint is compatible with OpenAI Chat Completions.
EONYX treats it as an ordinary provider.
Only two things differ: base_url, and an empty api_key.

1.2 Local server

ollama pull qwen2.5-coder
ollama serve   # endpoint at http://localhost:11434/v1

{
  "providers": {
    "local": { "type": "openai", "base_url": "http://localhost:11434/v1", "api_key": "" }
  },
  "models": [
    { "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }
  ],
  "default_model": "qwen2.5-coder"
}

llama-server -m model.gguf --port 8080

"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:8080/v1", "api_key": "" }
}

"providers": {
  "local": { "type": "openai", "base_url": "http://localhost:1234/v1", "api_key": "" }
}

Everything then works as usual.
eonyx for the TUI, eonyx run "task" for a single job.
No key is required: with an empty api_key the Authorization header is not sent.

2 🔧 In practice

2.1 Context window

Local models usually have a small context window.
Often 8–32K, against a million for a large cloud model.
Set the model’s real context_window.
That switches on auto-summarization: at about 80% fill the older turns are compacted, so the conversation never hits the wall.

{ "id": "qwen2.5-coder", "provider": "local", "context_window": 32768 }

2.2 Tips

• Pick a code model. Agentic work suits code-oriented models like qwen2.5-coder. They follow tool call formats better.
• Watch max_steps. A weak model can loop. The default of 50 keeps a stuck run from burning your time.
• Go hybrid. Keep a local and a cloud model side by side and switch with Ctrl+N. Rough work locally, hard problems on the big model.
• Verify. eonyx config shows the provider and base_url that were picked up. The token counter in the TUI header works for local models too, and the cost reads $0.

3 📋 Summary