LLM and harnesses¶
The runner reviews code with an agent harness: a program that lets the model read the repository, run tools and produce findings. Users choose the harness and model per run in Run Analysis; deployments set the defaults.
| Harness | Model access | Status |
|---|---|---|
| OpenCode | Any OpenAI-compatible endpoint, with your own key | Recommended. Enabled by default |
| GitHub Copilot | Each user's Copilot subscription | Opt-in, disabled by default. Read the caveat |
OpenCode¶
OpenCode is bundled in the runner image and talks to any OpenAI-compatible API. There is no OpenCode account; the key is your LLM provider's key.
Two sets of variables point at the same provider:
| Variables | Read by | Purpose |
|---|---|---|
OPENCODE_MODELS_URL, OPENCODE_API_KEY |
backend | Lists models (GET {url}/models) for the model picker in Run Analysis |
CLIO_LLM_BASE_URL, CLIO_LLM_API_KEY |
runner | Makes the actual review calls |
# backend
OPENCODE_ENABLED=true
OPENCODE_PROVIDER=openai # prefix for model ids: openai/gpt-4.1
OPENCODE_MODELS_URL=https://api.openai.com/v1
OPENCODE_API_KEY=sk-...
# runner
CLIO_HARNESS=opencode
CLIO_LLM_BASE_URL=https://api.openai.com/v1
CLIO_LLM_API_KEY=sk-...
CLIO_OPENCODE_MODEL=openai/gpt-4.1 # default when a run does not pick one
Set CLIO_HARNESS=opencode
The runner's built-in default harness is copilot. Runs started from the UI always carry the chosen harness,
but anything else (CLI runs, older jobs) falls back to CLIO_HARNESS.
Providers¶
| Provider | Base URL | Key |
|---|---|---|
| OpenAI | https://api.openai.com/v1 |
sk-… |
| OpenRouter | https://openrouter.ai/api/v1 |
sk-or-… |
| Groq | https://api.groq.com/openai/v1 |
Groq key |
| Azure OpenAI, Anthropic, Gemini, Bedrock | Through a LiteLLM proxy or the provider's OpenAI-compatible endpoint | Provider key |
| Ollama (local, free) | http://host.docker.internal:11434/v1 |
Any non-empty value |
| vLLM, LM Studio, other self-hosted | Your server's /v1 URL |
As configured |
Requirements for the model:
- It must support tool calling. Reviews fail or produce nothing otherwise.
- A long context helps (64k+ tokens); large changes are reviewed with repository context.
- Reasoning models work if the endpoint accepts OpenAI's parameters for them. Some gateways reject
max_tokensfor reasoning models; use a model the gateway supports in that case.
Models without a /models endpoint¶
Leave OPENCODE_MODELS_URL empty and list the models in local.yml:
serviceCoreConfiguration:
openCodeConfiguration:
enabled: true
provider: openai
defaultModel: gpt-4.1 # preselected in the picker
models:
- { id: gpt-4.1, name: GPT-4.1 }
- { id: gpt-4.1-mini, name: GPT-4.1 mini }
Model ids are exposed to runners as provider/id, e.g. openai/gpt-4.1.
Runner overrides¶
| Variable | Default | Purpose |
|---|---|---|
CLIO_OPENCODE_API_KEY |
CLIO_LLM_API_KEY |
Different key for OpenCode only |
CLIO_OPENCODE_BASE_URL |
CLIO_LLM_BASE_URL |
Different endpoint for OpenCode only |
CLIO_OPENCODE_TIMEOUT |
2700 |
Session timeout in seconds |
GitHub Copilot (opt-in)¶
The Copilot harness lets each user run reviews on their own Copilot subscription. Users connect their GitHub account with a device code in Settings. Arnold stores the token encrypted, and gives each job a short-lived execution key that the runner exchanges for the user's token just before the session starts.
Caveat
Account connection and model listing use undocumented GitHub endpoints (copilot_internal), and Copilot is
licensed to individual subscribers. Server-side use on behalf of users is a grey area under GitHub's terms.
It is disabled by default. Enable it only after your own legal review. This is not legal advice.
Enable¶
- Create a GitHub OAuth App (Settings → Developer settings → OAuth Apps), separate from the GitHub App. Check Enable Device Flow. The callback URL is unused; any URL works.
-
Configure:
Keep
COPILOT_ENCRYPTION_KEYstable and secret: changing it disconnects every user. -
docker compose up -d --force-recreate server runner. - Each user: Settings → GitHub Copilot → Connect, enter the code on GitHub, then Test connection.
Security notes:
- Execution keys expire after three hours and stop working when the run ends.
- Keep CI debug tracing off in runner projects: job variables include execution keys.
- Only
github.comaccounts are supported.
How a run picks harness and model¶
- The user picks a harness and model in Run Analysis (or Story mode).
- The backend checks the harness is available: OpenCode enabled, or Copilot enabled and connected for that user.
- The job carries
CLIO_HARNESSand the model, and setsCLIO_DISABLE_FALLBACK=true: a failing harness fails the run instead of silently switching providers. - A blank model means the deployment default (
CLIO_OPENCODE_MODEL/CLIO_LLM_MODEL).
If no harness is available, the setup guide shows No review engine is available yet to users.