ai-router

A billing proxy that sits in front of LLM providers. It gives every user their own API key, routes requests to an upstream by model name, and charges the request against a budget — so you can hand out access to openrouter / openai / anthropic without handing out the provider key or the bill.

The client API is openrouter-compatible (POST /v1/chat/completions and POST /v1/embeddings, OpenAI schema plus openrouter extensions, streaming included), so any OpenAI SDK works unchanged: point base_url at ai-router and use the sk-eo-... token as the API key.

client (OpenAI SDK)  ──sk-eo-…──▶  ai-router  ──provider key──▶  openrouter / openai / anthropic
                                       │
                                       └─ SQLite: budgets, top-ups, per-request spend

• Per-user tokens. Each user gets sk-eo-<id>-<secret>; revoke one without touching the others.
• Budgets. A budget is a pot of money shared by its users, topped up explicitly, with an admin who can manage it using their own token.
• Spending limits. Per day / ISO week / month / year, on a budget and on a single user, in a timezone you choose.
• Honest accounting. The cost of every request is recorded, including aborted and failed ones. Balance is always SUM(top-ups) − SUM(spends) — there is no stored balance to drift.
• Model routing. Rules map client-visible model names to upstreams, optionally renaming the model on the way out.
• One binary, one SQLite file. No Redis, no Postgres, no sidecars.

Quickstart

The fastest path is Docker Compose; see Deployment for NixOS and plain-binary setups.

git clone https://git.desu.church/eva/ai-router.git && cd ai-router
cp config.example.yaml config.yaml

# Secrets referenced from the config.
mkdir -p secrets
openssl rand -hex 32 > secrets/ai-router-master
printf '%s' "$OPENROUTER_KEY" > secrets/openrouter-key

Now edit config.yaml:

• keep http.listen: "0.0.0.0:8790" and db.path: /var/lib/ai-router/router.db — the compose file expects both;

• delete the upstreams you have no key for (the example declares anthropic too, and a secret file that cannot be read refuses startup), along with any rule pointing at them;

• replace the users and budgets sections with your own:

  budgets:
    home:
      admin: alex
      limits: { day: "5.00" }
  users:
    alex:
      # No dashes in a secret: the token is split at its last dash.
      secret: { plaintext: "s3cretWithoutDashes" }
  

Then:

docker compose up -d --build          # first build takes a while: it compiles Rust under nix
curl -s localhost:8790/health         # {"status":"ok","commit":"…"}

Your token is sk-eo-alex-s3cretWithoutDashes. Try it:

curl -s localhost:8790/v1/chat/completions \
  -H "Authorization: Bearer sk-eo-alex-s3cretWithoutDashes" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'

A config budget has an unlimited balance, so nothing else is needed to start spending. Budgets that should run on real money are created over the management API and topped up explicitly — see Management API.

From Python:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8790/v1",
                api_key="sk-eo-alex-s3cretWithoutDashes")
client.chat.completions.create(model="openai/gpt-4o-mini",
                               messages=[{"role": "user", "content": "hi"}])

Configuration

The full reference — every section, every field, defaults, validation rules and the config↔database relationship — lives in docs/configuration.md. A commented starting point is config.example.yaml.

The short version:

http:   { listen: "127.0.0.1:8790" }
db:     { path: /var/lib/ai-router/router.db }
auth:   { master_secret: { file: /run/secrets/ai-router-master } }
router:
  upstreams:
    openrouter:
      kind: openrouter
      base_url: "https://openrouter.ai/api/v1"
      api_key: { file: /run/secrets/openrouter-key }
  fallback: openrouter

http, db, auth and router are required; billing, users and budgets are optional. Any secret can be given inline ({ plaintext: "…" }) or read from a file ({ file: /run/secrets/… }). Multiple --config files merge in order, later over earlier, which is how secrets stay in a separate file from the rest.

How it works

Routing. The first matching rule top-down wins (prefix is a starts-with match, model is exact); everything unmatched goes to fallback. A rule can strip the prefix or rename the model before it is sent upstream. Three upstream kinds:

• openrouter / openai — the request body passes through byte-for-byte (with usage reporting switched on), and SSE chunks are forwarded verbatim.
• anthropic — full OpenAI chat ⇄ Anthropic Messages translation, tools and streaming included.

Billing. Post-paid, with guard-rails. Before the request: the budget must have money left and no spending window may be exhausted. After it: the actual cost is recorded. Overdraft during the request is bounded by a per-user in-flight limit (429 beyond it) and an optional per-request cost cap. Cost comes from openrouter's usage.cost (with a /generation fallback), or from router.pricing for openai/anthropic upstreams. A client that disconnects mid-stream is still billed: the upstream response is drained in the background and settled, because the provider charged us either way.

Ledger. SQLite, integer nano-USD end to end — no floats anywhere in the money path. Every request lands in the spend journal with its status (ok, upstream_error, client_aborted, settle_failed), so missing money is always traceable.

Management. The master key (X-Master-Key) creates users and budgets. Each budget has an admin user who can top it up, tune its limits and read its journals with their ordinary Bearer token. GET /v1/me lets a user see their own limits and remaining quota.

Deployment

Docker / Compose

The Dockerfile is a two-stage build: stage one builds the flake package with nix, stage two carries only the resulting runtime closure into a slim Debian image. Nothing from your host toolchain leaks in, and the image runs as a non-root user.

docker build -t ai-router:latest .
docker run -d --name ai-router \
  -p 127.0.0.1:8790:8790 \
  -v "$PWD/config.yaml:/etc/ai-router/config.yaml:ro" \
  -v "$PWD/secrets:/run/secrets:ro" \
  -v ai-router-data:/var/lib/ai-router \
  ai-router:latest

docker-compose.yml wires up the same thing plus a named volume for the database and a /health healthcheck. Points worth knowing:

• The container listens on 0.0.0.0:8790; the published port is bound to 127.0.0.1, so put a TLS terminator in front before exposing it.
• The SQLite file lives in the ai-router-data volume — that volume is your balances and journals. Back it up.
• The build compiles the whole Rust workspace under nix and does not reuse your local target/, so the first build takes minutes. Rebuilds after a source change do too.
• /health reports "commit": "dev" in Docker images: .git is not in the build context, so the build cannot stamp a revision.
• The build needs outbound HTTPS (cache.nixos.org, github, crates.io). Behind a VPN whose MTU is below 1500, the default bridge network black-holes large packets and every download times out; give the daemon the tunnel's MTU ({"mtu": 1300} in daemon.json) or build with docker build --network=host.

NixOS

The flake ships a module:

{
  inputs.ai-router.url = "git+https://git.desu.church/eva/ai-router.git";

  # in a host module:
  imports = [ inputs.ai-router.nixosModules.default ];
  services.ai-router = {
    enable = true;
    settings = {
      # non-secret sections, rendered to yaml in the nix store
      billing.timezone = "Europe/Moscow";
      router = { /* upstreams, rules, fallback, pricing */ };
    };
    extraConfigFiles = [ "/run/secrets/ai-router-secrets.yaml" ];
  };
}

settings lands in the world-readable nix store — keep secrets out of it. Put the secret sections in extraConfigFiles (sops-nix / agenix paths work directly, they are read at runtime), or reference individual values as { file = "/run/secrets/…"; }.

The module runs ai-router-server under systemd with StateDirectory=ai-router, so db.path defaults to /var/lib/ai-router/router.db. It does not create the ai-router user and group — declare them on the host, or point user/group at an existing pair.

Plain binary, no nix, no docker

Build it however you prefer:

nix build .#ai-router          # -> ./result/bin/{ai-router-server,ai-router}
# or
cargo build --release          # -> ./target/release/…

Then run it against a config file:

ai-router-server --config /etc/ai-router/config.yaml

Repeat --config to layer files (later files win) — useful for keeping secrets separate. The server needs write access to the directory holding db.path; the database file itself is created on first start.

A minimal systemd unit:

[Unit]
Description=ai-router LLM billing proxy
After=network.target

[Service]
User=ai-router
StateDirectory=ai-router
ExecStart=/usr/local/bin/ai-router-server --config /etc/ai-router/config.yaml --config /run/secrets/ai-router-secrets.yaml
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Outbound TLS uses bundled root certificates, so the host does not need a CA bundle.

Management API

POST   /admin/v1/users                {budget_id?, limits?} -> {id, token}   [master]
GET    /admin/v1/users/{id}                                                  [master | budget admin]
PATCH  /admin/v1/users/{id}           {budget_id?, disabled?}                [master]
PATCH  /admin/v1/users/{id}/limits    {day?, week?, month?, year?}           [master | budget admin]
POST   /admin/v1/budgets              {admin, infinite?, limits?} -> {id}    [master]
GET    /admin/v1/budgets/{id}         balance, totals, windows               [master | budget admin]
PATCH  /admin/v1/budgets/{id}         {infinite?}                            [master]
PATCH  /admin/v1/budgets/{id}/limits  {day?, ...}                            [master | budget admin]
POST   /admin/v1/budgets/{id}/topup   {amount: "10.00", note?}               [master | budget admin]
GET    /admin/v1/budgets/{id}/topups?after_id&limit                          [master | budget admin]
GET    /admin/v1/budgets/{id}/spend?after_id&limit&user=                     [master | budget admin]
GET    /v1/me                                                                [any Bearer]
GET    /health                                                               [no auth]

Onboarding a paying user from scratch:

M="-H x-master-key:$MASTER"
# 1. the user (no budget yet) — the response carries the only copy of the token
curl -s $M -H 'content-type: application/json' -d '{}' localhost:8790/admin/v1/users
# 2. a budget they administer
curl -s $M -H 'content-type: application/json' \
     -d '{"admin":"<user id>","limits":{"day":"2.00"}}' localhost:8790/admin/v1/budgets
# 3. attach the user to it, then top it up
curl -s -X PATCH $M -H 'content-type: application/json' \
     -d '{"budget_id":"<budget id>"}' localhost:8790/admin/v1/users/<user id>
curl -s $M -H 'content-type: application/json' \
     -d '{"amount":"10.00","note":"initial"}' localhost:8790/admin/v1/budgets/<budget id>/topup

PATCH semantics: an absent field is left alone, an explicit null resets a limit to infinity. Amounts are decimal USD strings ("10.00"), never numbers. Errors use the OpenAI {"error": {message, type, code}} shape; window refusals additionally carry dimension and resets_at.

Entities created here are independent of the config file: the startup sync never touches them, and config-declared users and budgets answer 409 config_managed to editing calls.

Development

cargo test                              # unit + integration + e2e
cargo run -p ai-router-cli -- gen-uid   # fresh entity id
cargo run -p ai-router-cli -- gen-secret
nix build .#ai-router

On a host without a global cc, wrap cargo: nix-shell -p gcc --run 'cargo test'.

The workspace is library-first: crates/backend exposes the proxy_router / admin_router axum routers and takes every dependency (db, transport, clock, config) as an argument, so the whole thing can be embedded into a larger service. See AGENTS.md for the crate layout and code rules.