/{mode}/api/{tags,chat,generate,version,show,ps} beside the OpenAI surface.
forge path is a raw proxy (forge is Ollama); the cloud fallback is translated
both ways — NDJSON frames byte-compatible with Ollama's, tool calls round-trip
with synthesised ids, format mapped, think→reasoning_effort (forced to none
with tools: OpenAI rejects the combination on gpt-5.4-mini). Tags never wait
on a model load (HA's list() has a 5 s timeout). local-strict still 503s in
Ollama's error shape; the OpenAI dialect is untouched.
Home Assistant (Lounge, 2026.9.2) configured through its REST config flow and
proven with a real conversation. HA's VLAN cannot reach the k8s ingress, so
it uses forgeway.dmz.lounge.falkvinge.net through the entrance HAProxy
(RFC1918-fenced vhost); the .services. alias needs a pfSense rule from Rick.
22/22 in-cluster checks on both sites. scripts/ha_setup_ollama.py is
idempotent and ends with a real answer; local-smoke-ollama.sh uses the exact
client HA pins.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
forgeway
An LLM router in front of forge's Ollama gpt-oss:120b, speaking both the OpenAI
API and Ollama's native API on the same two endpoints. One instance runs per
cluster, not one central service.
Names (confirmed by Rick, 2026-09-18). Explicit per-site names
forgeway.lounge.falkvinge.netandforgeway.serenity.falkvinge.net(Chalet later:forgeway.chalet.falkvinge.net), plus the split-horizon local-first aliasforgeway.services.falkvinge.net, which resolves to whichever instance is local to the querying client's site..services.names are internal-only by convention: no entrance SAN, no external exposure, ever.
The two endpoints
| Endpoint | Behaviour when forge is available | Behaviour when it is not |
|---|---|---|
/local-optimistic/v1 |
serves from forge | falls back to OpenAI gpt-5.4-mini |
/local-strict/v1 |
serves from forge | clean HTTP 503, no fallback, by design |
Each endpoint speaks two dialects:
| Dialect | Paths | Typical caller |
|---|---|---|
| OpenAI | /{endpoint}/v1/chat/completions, /{endpoint}/v1/models |
scripts, the OpenAI SDK, OpenClaw |
| Ollama (native) | /{endpoint}/api/chat, /api/generate, /api/tags, /api/version, /api/show, /api/ps |
Home Assistant's Ollama integration, any Ollama client |
Streaming works in both: SSE for the OpenAI dialect, newline-delimited JSON for the
Ollama one (where stream defaults to true, as in Ollama itself).
Where they live:
| Site | Cluster | Use this | Raw LB IP (DNS-independent) |
|---|---|---|---|
| any | — | https://forgeway.services.falkvinge.net → the instance local to you |
— |
| Lounge | Rocinante | http://forgeway.lounge.falkvinge.net |
http://172.19.10.90 |
| Serenity | Niðavellir | http://forgeway.serenity.falkvinge.net |
http://172.24.10.21 |
| Chalet | — | when the cluster exists | — |
The alias is the front door. Use the per-site names only when you deliberately want a
specific site's instance (for example, asking the Lounge one from Serenity). HTTPS with
a valid certificate is available on the alias at both sites; the per-site names are
http:// only — the internal ingress certificates carry *.services.falkvinge.net but
not *.lounge. / *.serenity. (adding those SANs to the two internal ingress
certificates is a one-line cert-manager change if ever wanted; the entrance bundle is
not the place).
Calling them
local-strict — do not retry a 503 blindly. A 503 means the local model is
unavailable right now: forge is off, Ollama is stopped, or the model is evicted for
a training run. None of those clear in the next second, and retrying just adds load
to something already struggling. Handle the error, or degrade to your own fallback.
# strict: one attempt, handle the 503 yourself
curl -sS -X POST https://forgeway.services.falkvinge.net/local-strict/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-oss:120b","messages":[{"role":"user","content":"hello"}]}' \
-w '\n%{http_code}\n'
import httpx
# strict — no retries on 503; that status is an answer, not a hiccup
with httpx.Client(base_url="https://forgeway.services.falkvinge.net/local-strict/v1", timeout=600) as c:
r = c.post("/chat/completions", json={
"model": "gpt-oss:120b",
"messages": [{"role": "user", "content": "hello"}],
})
if r.status_code == 503:
... # local model is down: skip, queue, or use your own fallback
else:
r.raise_for_status()
print(r.json()["choices"][0]["message"]["content"])
# optimistic — fallback is already handled inside the router.
# Do NOT add your own retry loop: a failure here means forge AND the cloud both failed.
with httpx.Client(base_url="https://forgeway.services.falkvinge.net/local-optimistic/v1", timeout=600) as c:
r = c.post("/chat/completions", json={
"model": "gpt-oss:120b",
"messages": [{"role": "user", "content": "hello"}],
})
r.raise_for_status()
print(r.headers["x-forgeway-backend"]) # "forge" or "cloud"
With the OpenAI SDK, point base_url at one of the two prefixes and use any
non-empty api key (the router does not authenticate callers):
from openai import OpenAI
client = OpenAI(base_url="https://forgeway.services.falkvinge.net/local-optimistic/v1", api_key="unused")
Every response carries x-forgeway-backend: forge|cloud, x-forgeway-upstream,
x-forgeway-site and x-forgeway-mode, so a caller can always tell what answered.
Model selection. The model field in a request is ignored — each endpoint routes
to its own backend. Send anything; gpt-oss:120b is conventional. max_tokens and
max_completion_tokens are interchangeable: the router translates between them,
because Ollama only accepts the first and gpt-5.4-mini rejects it outright.
From OpenClaw agents
The same endpoints are registered as model providers (base URL = the .services.
alias over HTTPS, so agents get the local instance too), so agents and scripts use one
backend:
forgeway-strict/gpt-oss-120b # forge or an error
forgeway-optimistic/gpt-oss-120b # forge, else gpt-5.4-mini
Home Assistant
HA's core OpenAI Conversation integration only talks to the official OpenAI endpoint, so it cannot use forgeway. HA's core Ollama integration can: it takes an arbitrary server URL, and its client keeps the URL path, so an endpoint prefix is a valid server.
Lounge HA — configured and proven 2026-09-18. Integration entry
https://forgeway.dmz.lounge.falkvinge.net/local-optimistic, conversation agent
conversation.forgeway_optimistic (model gpt-oss:120b, think off, no device
control). A real question through HA's own conversation API came back from forge:
"Home Assistant is an open-source platform for local home automation and smart-device
control."
Why that URL and not the .services. alias: HA lives on the sandbox VLAN
(192.168.120.0/22), which cannot reach the k8s ingress 172.19.10.11 — the alias
resolves correctly there and then fails to connect. What that VLAN can reach is the
entrance HAProxy, whose certificate covers *.dmz.lounge.falkvinge.net and which already
has firewall clearance into the 172.19.10.80-98:80 band. So:
forgeway.dmz.lounge.falkvinge.net → entrance HAProxy 172.20.0.101 → 172.19.10.90:80 (HA, and any client HAProxy reaches)
forgeway.services.falkvinge.net → ingress-nginx 172.19.10.11 → forgeway Service (DMZ-side clients, local-first alias)
forgeway.lounge.falkvinge.net → ingress-nginx 172.19.10.11 (explicit per-site name, http)
The entrance is a public edge and the router has no authentication, so the HAProxy vhost
is fenced to RFC1918 sources (http-request deny for anything else); the router's
own raw Service IP stays unreachable from outside as before. If you would rather HA use
the .services. alias, the one-line pfSense change is: pass 192.168.120.12 (or the
sandbox net) → 172.19.10.11 TCP 443. Then re-run scripts/ha_setup_ollama.py with the
alias URL.
What HA sends and how it is handled. list() → /api/tags with a 5 s timeout, so
tags never wait on a model load (a cold gpt-oss:120b takes ~25–30 s); the list is
exactly one model, the local one. chat(stream=True) with keep_alive: -1,
options: {num_ctx: 8192}, think: false, and tools when the Assist API is enabled —
all passed to forge untouched. HA reads message.content, message.thinking,
message.tool_calls, done, done_reason; gpt-oss returns content and thinking
separately regardless of think, so a normal answer is never empty. HA shows the
thinking as a collapsible block.
To make it the default assistant / let it control the house (your call, in the HA UI): Settings → Voice assistants → Assist → Conversation agent → "forgeway optimistic"; and on the integration's conversation subentry, tick Control Home Assistant to grant the Assist API (tools). Both were left off deliberately.
Re-run the setup at any time — it is idempotent and ends with a real conversation:
python3 scripts/ha_setup_ollama.py http://192.168.120.12:8123 \
https://forgeway.dmz.lounge.falkvinge.net/local-optimistic --name "forgeway optimistic" [--assist]
The Ollama dialect, and the cloud fallback through it
forge is Ollama, so on the forge path the router is a raw proxy: the response —
including every streamed NDJSON frame — is forge's own, with only the model pinned.
Everything below is about the cloud fallback (local-optimistic only; local-strict
returns Ollama's error shape, {"error": "..."}, with status 503):
- The Ollama request is translated to an OpenAI chat completion and the answer is
translated back into Ollama's shape, non-streamed or as NDJSON frames byte-compatible
with Ollama's:
model,created_at(nanosecond RFC 3339),message.role/content,done,done_reason(stop/length), and the*_duration/*_countstats (token counts from usage; durations that OpenAI does not report are 0). - Tool calls round-trip. Ollama messages carry no tool-call ids and OpenAI insists on
them, so ids are synthesised in order and handed to the following
toolmessages in order; OpenAI's JSON-string arguments become Ollama's objects. On the way out, OpenAI streams tool calls as fragments; they are reassembled and emitted as one frame. format: "json"→response_format: json_object; a schema →json_schema.think→reasoning_effort:false→low,true→medium, a level → itself. With tools present it is forced tonone: gpt-5.4-mini rejects any other value together with function tools on/v1/chat/completions, and HA's Assist API always sends tools. The cloud model's reasoning text is not exposed by that API, so cloud answers carry nothinking.options.num_predict→max_completion_tokens;num_ctxandkeep_aliveare Ollama-only and dropped./api/generateis served the same way (prompt/system/imagesbecome messages)./api/pull,/api/push,/api/create,/api/delete,/api/embedreturn 400: this endpoint serves exactly one model and manages none.
The reasoning trap, decided. A thinking model with a tiny output budget spends it all
thinking and returns empty content — that is what bit the OpenAI dialect at
max_tokens: 10. The router does not silently rewrite budgets; instead it makes the
failure visible: a non-streamed answer with thinking but no content gets the header
x-forgeway-warning: content-starved-by-reasoning and a log line. HA never sets a
budget, so it never hits this.
DNS: how the alias finds the local instance
Every .services. name in the estate is a CNAME chain ending at a site's ingress-nginx,
and the authoritative services.falkvinge.net zone lives on the Lounge Technitium. So
at the Lounge nothing special is needed:
forgeway.services.falkvinge.net → ingress.dmz.lounge.falkvinge.net → 172.19.10.11
forgeway.lounge.falkvinge.net → ingress.dmz.lounge.falkvinge.net → 172.19.10.11
forgeway.dmz.lounge.falkvinge.net → entrance.dmz.falkvinge.net → 172.20.0.101 (HAProxy path, see Home Assistant)
forgeway.serenity.falkvinge.net → forgeway.dmz.serenity.falkvinge.net → ingress.dmz.serenity.falkvinge.net → 172.24.10.11
Serenity's Pi-holes forward everything under falkvinge.net (except serenity.) to the
Lounge — so without an override, a Serenity client would silently get the Lounge
ingress and the remote instance, defeating the whole multi-site design. Both Serenity
Pi-holes therefore pin the alias to the Serenity ingress:
| Resolver | Where the override lives |
|---|---|
172.24.10.201 (k8s, two pods) |
serenity-cluster-config/apps/pihole/pihole.yaml → 03-serenity-forwards.conf |
172.24.1.201 (LXC) |
Pi-hole v6 API: dns.hosts entry + misc.dnsmasq_lines |
Both carry host-record=forgeway.services.falkvinge.net,172.24.10.11 and
local=/forgeway.services.falkvinge.net/. The local= line matters: without it an AAAA
query is still forwarded to the Lounge, and the day the Lounge ingress gains an IPv6
address every dual-stack Serenity client would prefer it and quietly reach the remote
instance — the same dual-stack bypass that defeated the Lounge overrides on 2026-08-24.
Each site's ingress serves forgeway.services.falkvinge.net for its own instance, so the
DNS answer alone decides which site you land on. Verified 2026-09-18 from real clients:
Lounge (OpenClaw VM, Lounge Kuma host) → 172.19.10.11, x-forgeway-site: lounge;
Serenity (Niðavellir pod, both Serenity Pi-holes) → 172.24.10.11,
x-forgeway-site: serenity. Never the reverse.
Why one instance per cluster
A single central router would sit on one site. The cross-site link to forge is exactly
the kind of thing that goes down — and if the router lived on the far side of it, the
router would be the outage rather than the thing that reports it. So each cluster
runs its own, and each one answers for itself: when Serenity cannot reach forge,
Serenity's local-optimistic still serves from the cloud and its local-strict still
returns a clean 503, both without depending on anything at the Lounge.
It does not run on forge itself, for the same reason: a router that dies with its upstream cannot report on it.
Discovery — and what happens when silicon moves to Serenity
No instance is told which site forge is at. Each one holds an ordered candidate list
(FORGEWAY_UPSTREAMS) and finds forge at request time:
http://forge.dmz.falkvinge.net:11434 the stable name — DNS follows a host move
http://172.19.0.130:11434 forge at the Lounge (today)
http://172.24.0.130:11434 forge at Serenity (after the move)
Candidates are probed concurrently but awaited in priority order, so the happy path costs one fast round trip and dead candidates never slow it down. The winner is sticky and tried first next time.
When silicon relocates to Serenity, nothing here needs rewriting. Re-point
forge.dmz.falkvinge.net at its new address and both instances follow on their next
health refresh (seconds). If DNS is stale or broken, discovery still finds forge at the
Serenity literal already in the list. The only thing that changes materially is which
site pays the cross-site latency — and that flips automatically, because each instance
independently discovers the nearest working answer.
Two things do need a human when that happens:
- forge's ufw — the allow rules are scoped to source ranges. Serenity-local
traffic will then arrive from
172.24.9.0/24(already allowed); Lounge traffic will arrive SNAT'd from the Serenity edge and will need its own rule. - Kuma — the monitors point at each local router by IP, so they keep working, but the Lounge monitor will start reporting the slower cross-site path.
Health and monitoring
/healthz performs a real 1-token completion against forge, never a ping. A ping
cannot tell the difference between a healthy forge and a forge that is up with Ollama
stopped — which was literally its state on the morning this was built.
| Result | Meaning |
|---|---|
200 |
forge answered a real completion |
503 |
forge did not |
| no response at all | the router itself is gone |
Verdicts are cached for a few seconds (FORGEWAY_HEALTH_TTL), so a burst of traffic
does not become a burst of probe completions against forge.
/healthz/live and /healthz/ready are deliberately not gated on forge. If
readiness were, Kubernetes would pull every pod out of the Service the moment forge
died — callers would get a connection failure instead of the 503 local-strict
promises, and Kuma could not distinguish "forge down" from "router gone".
Monitoring is active HTTP probing, not push: a push heartbeat cannot tell "router dead" from "idle". One monitor per Kuma instance, each watching its own local router:
| Kuma | Monitor | Target |
|---|---|---|
Lounge 192.168.81.98:3001 |
#307 forgeway — local LLM router (Lounge) |
http://172.19.10.90/healthz |
Serenity 192.168.24.98:3001 |
#57 forgeway — local LLM router (Serenity) |
http://172.24.10.21/healthz |
Both: 60 s interval, 48 s timeout, 2 retries, accepted status 200-299. The timeout is
wide because a cold gpt-oss:120b load takes ~25–30 s and is a legitimate success,
not a failure.
The one operational conflict: monitoring vs. LoRA training
A 60 s probe interval keeps gpt-oss:120b (~67 GB) permanently resident on forge's
card 1. An ai-toolkit LoRA run needs ~88 GB on the same card, and the two do not fit —
ollama stop gpt-oss:120b is the documented pre-training step, but with monitoring
running, the next probe simply loads it again.
Before a training run, do one of:
- pause the Kuma monitor for that site (simplest), or
- set
FORGEWAY_HEALTH_COLD_LOAD=falsein the ConfigMap. The probe then checks Ollama's/api/psand reports DOWN instead of triggering a load. During training that reading is correct:local-strictgenuinely cannot serve, andlocal-optimisticcorrectly falls through to the cloud.
Configuration
Everything is environment-driven; one image serves every site. See
apps/forgeway/manifests/configmap.yaml in each cluster repo.
| Variable | Default | Notes |
|---|---|---|
FORGEWAY_NAME |
forgeway |
the single place the service's name lives |
FORGEWAY_SITE |
unknown |
lounge / serenity |
FORGEWAY_UPSTREAMS |
forge DNS name | comma-separated, priority order |
FORGEWAY_LOCAL_MODEL |
gpt-oss:120b |
|
FORGEWAY_CLOUD_MODEL |
gpt-5.4-mini |
optimistic only |
FORGEWAY_HEALTH_TTL |
5 |
cache for a healthy verdict |
FORGEWAY_HEALTH_FAIL_TTL |
3 |
cache for an unhealthy verdict |
FORGEWAY_HEALTH_TIMEOUT |
45 |
must exceed a ~25–30 s cold load |
FORGEWAY_DISCOVERY_TIMEOUT |
4 |
6 at Serenity — the mesh hop |
FORGEWAY_HEALTH_COLD_LOAD |
true |
see the training conflict above |
FORGEWAY_REQUEST_TIMEOUT |
600 |
proxied completions |
OPENAI_API_KEY |
— | from the forgeway-cloud Secret, applied out of band |
The cloud credential is never in git. It is applied with
scripts/apply_cloud_secret.py, which reads the value from the process environment and
hands it to kubectl on stdin. The secretKeyRef is optional: true, so the Deployment
still runs without it — local-strict is unaffected and local-optimistic reports
that no fallback credential is configured.
Deploying
./scripts/build-push.sh 0.1.1 # multi-arch; both clusters are arm64-majority
Then bump the image tag in each cluster repo and push — ArgoCD syncs:
- Lounge:
lounge-cluster-config/apps/forgeway/manifests/+base/apps/forgeway/ - Serenity:
serenity-cluster-config/apps/forgeway/+base/apps/forgeway-app.yaml
Verifying
./scripts/local-smoke.sh # OpenAI dialect, against a real forge
./scripts/local-smoke-ollama.sh # Ollama dialect, incl. the exact client HA pins
./scripts/verify-cluster.sh ~/.kube/rocinante-admin.yaml lounge
./scripts/verify-cluster.sh ~/.kube/nidavellir.yaml serenity
verify-cluster.sh runs 22 checks from inside the cluster (13 OpenAI, 9 Ollama). It produces the
"forge unavailable" case with a throwaway Deployment pointed at a dead upstream —
it never stops Ollama on forge.