Baltor Get started

Endpoints

llama.cpp server

A local runtime you install and run on your own hardware. This page lists the addresses it answers, the facts its documentation states, and the setup of each harness.

Addresses and facts

Start it

llama-server -hf {repository}:{quantization} --jinja

The --jinja flag turns on tool calling with the model's own chat template.

Harness setup

Replace <model> with the model you want.

OpenCode

Put this in opencode.json in your project folder:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "llama-cpp": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "llama.cpp server",
      "options": {
        "baseURL": "http://localhost:8080/v1"
      },
      "models": {
        "<model>": {
          "name": "<model>"
        }
      }
    }
  }
}
  • OpenCode reads any OpenAI-compatible address through the @ai-sdk/openai-compatible package, and an address that speaks the Responses API through @ai-sdk/openai.

From OpenCode documentation, read .

Pi

Put this in ~/.pi/agent/models.json:

{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "local",
      "models": [
        {
          "id": "<model>"
        }
      ]
    }
  }
}
  • The apiKey field can name an environment variable as $NAME.

From Pi documentation, read .

Codex

Put this in ~/.codex/config.toml:

model = "<model>"
model_provider = "llama-cpp"

[model_providers.llama-cpp]
name = "llama.cpp server"
base_url = "http://localhost:8080/v1"
  • Codex speaks the Responses API only: responses is the one supported wire API of a custom provider. Ollama and LM Studio are built in and start with --oss.

From Codex documentation, read .

Claude Code

Put this in ~/.claude/settings.json, or variables in your shell:

export ANTHROPIC_BASE_URL="http://localhost:8080"
export ANTHROPIC_AUTH_TOKEN="local"
export ANTHROPIC_MODEL="<model>"
claude
  • Claude Code sends Anthropic Messages requests to ANTHROPIC_BASE_URL. Anthropic says it does not support routing Claude Code to models other than Claude through any gateway, so some features may not work with another model.

From Claude Code documentation, read .