14 Commits
Author SHA1 Message Date
Dolon (Agent) 2db73ecd08 ollama: translate STREAMED generate->chat chunks back to generate shape (0.2.6)
0.2.5 rewrote /api/generate to /api/chat upstream when a format is set,
but only converted the response shape on the NON-streaming path.
Streamed chunks passed through in chat shape (message.content) while the
caller read generate shape (response) -- arriving as a run of literal
'undefined'. Karakeep tags via a streamed generate call and reported
exactly that: 'didn't respond with the expected JSON ...
undefinedundefinedun'. Summaries were unaffected because summarisation
is not streamed, which is why they started working and tags did not.

Now every NDJSON line is translated: message.content -> response,
thinking preserved unless the caller asked for none, context added on
the terminal chunk.
2026-09-19 18:23:25 +00:00
Dolon (Agent) a8b6a3ee6e ollama: run /api/generate as /api/chat when a format is set (0.2.5)
gpt-oss:120b returns an EMPTY response on /api/generate whenever a JSON
format is requested. Verified against forge directly, bypassing this
router, so it is upstream behaviour and not ours:

  /api/generate + format:'json'  -> response ''   done_reason stop
  /api/generate, no format       -> response 'OK'
  /api/chat     + format:'json'  -> content '{"tags":["x"]}'

Karakeep tags via /api/generate WITH format, so every job returned
empty and it reported 'the model ignored our prompt'.

Fix: when a format is set, run the request upstream as /api/chat and
hand the answer back in generate's shape (response/context). Callers
change nothing.
2026-09-19 13:47:21 +00:00
Dolon (Agent) a5bd550971 ollama: never honour think=false literally (0.2.4)
gpt-oss:120b generates its answer AFTER the reasoning pass. With
think=false AND a JSON format set it returns EMPTY content and no
thinking at all. Karakeep sends exactly that, so every tagging job
failed in under a second with 'the model ignored our prompt and
didn't respond with the expected JSON: {}'.

Verified by curl through the router:
  think=false -> content ''
  think=true  -> content '{"tags":["x"]}'

Fix: keep reasoning ON upstream and strip the 'thinking' field from
the reply instead, which is what a caller asking for no thinking
actually wants. Applies to the native forge path (which forwards the
body as-is) and the cloud mapping. Streaming is untouched: HA streams
and already handles the separate field.
2026-09-19 13:26:55 +00:00
Dolon (Agent)andClaude Opus 5 820bcc8e4e checks: cover residency, and prove a short-keep_alive caller is overridden
24 asserts forge's runner is permanently resident, not merely loaded at the
moment of the check — a cold load is 21-66 s and lands in front of the caller,
so "loaded right now" is not the property that matters.

25 is the regression that would otherwise be silent: an explicit finite
keep_alive from a caller BEATS forge's OLLAMA_KEEP_ALIVE default, so the router
override is the only thing stopping one such caller evicting the 64 GB model for
everyone else. The check sends keep_alive:"30s" natively and asserts residency
survives it.

Both pass on lounge and nidavellir, 25/25.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 20:10:39 +00:00
Dolon (Agent)andClaude Opus 5 f7cd9b6690 0.2.3: keep the model resident — pin keep_alive on the native dialect
A cold load of gpt-oss:120b is 21-66 s and lands entirely in front of the
caller, so every idle spell put that pause on Rick's next Home Assistant turn.
The 0.3-0.6 s figures from 2026-09-18 were warm-model numbers.

Measured on Ollama 0.34.0, and NOT symmetrical with the num_ctx pin:

  native /api/chat, keep_alive:-1     -> Forever
  native /api/chat, explicit finite   -> that duration, and it BEATS the
                                         server-side OLLAMA_KEEP_ALIVE default
  native /api/chat, absent            -> does NOT reset an existing value;
                                         the duration is fixed AT LOAD TIME
  /v1/chat/completions, keep_alive    -> IGNORED, like options.num_ctx
                                         (confirmed in openai/openai.go: the
                                         request struct has no such field)

So the router alone can never carry residency: an OpenAI-dialect caller cannot
set it, and whichever request happens to trigger the cold load fixes the
duration. forge therefore also gets OLLAMA_KEEP_ALIVE=-1 in
/etc/systemd/system/ollama.service.d/30-keepalive.conf. The two are
complementary rather than redundant — the server default covers dialects and
callers we never proxy, this pin defeats a native caller that sends a short
keep_alive of its own.

The health probe asserts residency rather than inheriting it: it runs before
every proxied request, so it is the most frequent caller forge has and the one
most likely to be what triggers a cold load.

/status gains keep_alive_pin, runner_resident and runner_expires_at, and the
locator logs a WARNING if forge's runner stops being permanently resident —
the same visibility pattern as num_ctx_pin vs runner_context.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 20:02:03 +00:00
Dolon (Agent)andClaude Fable 5.1 2d3c548542 0.2.2: pin num_ctx on every forge-bound request and on the health probe
Ollama keeps one runner config per model and reloads the 64 GB model on
every request whose num_ctx differs from the loaded runner's. The router
now sets options.num_ctx = FORGEWAY_NUM_CTX (default 131072) on the Ollama
dialect and carries the same value in the probe (native /api/chat; the
OpenAI /v1 endpoint ignores options, verified). /status exposes the pin and
the observed runner context and warns on mismatch; check 23 asserts it.

Verified by behaviour: an 8192 caller, a 4096 caller, probes and 40k/100k-
token retrieval prompts in one window -> 0 reloads, 0 timeouts, HA turns
0.3-0.6 s; before, 3 rounds of an 8192 caller = 7 loads and two probe
timeouts. 23/23 acceptance on both clusters.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 19:43:13 +00:00
Dolon (Agent)andClaude Fable 5.1 297b1c421b REPORT: spurious cloud fallback at 19:10:53 traced to num_ctx runner thrash
Forge journal shows four 64 GB reloads in two minutes alternating between
HA's num_ctx 8192 and the probes' default 131072; one pushed a health
probe past 45 s while forge was answering, so an optimistic request went to
the cloud. Recommended router-side fix recorded; nothing changed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 19:16:56 +00:00
Dolon (Agent)andClaude Fable 5.1 846940e1d0 REPORT: sequencing check after the fact; resolution paths recorded
Timeline from HA's created_at and the ingress log shows the required order
(rule proven from HA -> entries on the alias -> vhost removed) held, and that
no HA entry referenced forgeway.dmz.lounge when the vhost went. All four
names' resolution paths are written down explicitly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 19:13:38 +00:00
Dolon (Agent)andClaude Fable 5.1 f1e0103b5e Converge on the approved design: pfSense rule for HA, edge vhost removed
Rick added the pfSense rule (HA IP -> ingress); the entrance HAProxy vhost,
its DNS record and the Ingress host for forgeway.dmz.lounge are removed and
verified gone from outside. Both HA agents are on the .services. alias and
re-proven by real conversations attributed in the ingress log, plus the
dead-upstream difference test through HA. Docs state who did what.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 19:10:48 +00:00
Dolon (Agent)andClaude Fable 5.1 cac83dff55 Home Assistant: both privacy flavours on the .services. alias, proven
Two Ollama entries — "Local only — private (fails if forge is down)"
(local-strict) and "Local preferred — falls back to OpenAI cloud"
(local-optimistic) — created/renamed headlessly with scripts/ha_ollama.py
(REST config flows + WebSocket renames; wait for the entry to be loaded
before opening a subentry flow; entities matched by subentry id).

Each proven by a real conversation through HA, and the difference proven
through HA against a throwaway dead-upstream instance published under a
temporary .services. name: strict → HA error, no cloud; optimistic → cloud
answer in ~1 s. All temporaries removed. Rick's pfSense rule (HA IP →
ingress) verified from HA's side; alias is NXDOMAIN publicly, so a loaded
entry proves internal resolution.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 18:54:05 +00:00
Dolon (Agent)andClaude Fable 5.1 c37e54c5c2 0.2.1: native Ollama dialect on both modes; Home Assistant proven end-to-end
/{mode}/api/{tags,chat,generate,version,show,ps} beside the OpenAI surface.
forge path is a raw proxy (forge is Ollama); the cloud fallback is translated
both ways — NDJSON frames byte-compatible with Ollama's, tool calls round-trip
with synthesised ids, format mapped, think→reasoning_effort (forced to none
with tools: OpenAI rejects the combination on gpt-5.4-mini). Tags never wait
on a model load (HA's list() has a 5 s timeout). local-strict still 503s in
Ollama's error shape; the OpenAI dialect is untouched.

Home Assistant (Lounge, 2026.9.2) configured through its REST config flow and
proven with a real conversation. HA's VLAN cannot reach the k8s ingress, so
it uses forgeway.dmz.lounge.falkvinge.net through the entrance HAProxy
(RFC1918-fenced vhost); the .services. alias needs a pfSense rule from Rick.

22/22 in-cluster checks on both sites. scripts/ha_setup_ollama.py is
idempotent and ends with a real answer; local-smoke-ollama.sh uses the exact
client HA pins.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 18:39:31 +00:00
Dolon (Agent)andClaude Fable 5.1 1bec7c784c DNS + ingress: the confirmed names, split-horizon alias, providers on https
Rick confirmed forgeway.<site>.falkvinge.net and the local-first alias
forgeway.services.falkvinge.net. README documents the alias as the front
door, the per-site names as explicit handles, and exactly how the Serenity
override keeps the alias LOCAL (host-record + local= on both Serenity
Pi-holes; the local= line closes the dual-stack AAAA bypass).

OpenClaw providers now point at the https alias with a placeholder apiKey:
OpenClaw exempts plain-IP http base URLs from needing a key but not https
hostnames. scripts/technitium.py ships the Technitium API helper used for
the records (zones / records / add / resync / logtail).

REPORT.md addendum carries the verification table from real clients at both
sites and the findings made on the way.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 14:56:12 +00:00
Dolon (Agent)andClaude Opus 5 b81d39fdd4 scripts: add the Gitea repo bootstrap helper
Keeps the PAT out of the stored remote URL and out of every command line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:30:31 +00:00
Dolon (Agent)andClaude Opus 5 3155d4a230 forgeway 0.1.1 — per-cluster OpenAI-compatible LLM router
Two endpoints over forge's Ollama gpt-oss:120b:
  /local-optimistic/v1  forge, else cloud gpt-5.4-mini
  /local-strict/v1      forge, else a clean 503 the caller handles

Discovery, not configuration: each instance probes an ordered candidate list
(stable DNS name, then the Lounge and future Serenity literals) concurrently but
in priority order, so the happy path is one round trip and silicon's move to
Serenity needs no rewrite.

/healthz runs a real 1-token completion — forge can be up with Ollama stopped.
Liveness/readiness are deliberately NOT forge-gated: otherwise k8s would evict
the pods during an outage and callers would get a connection failure instead of
the promised 503, and Kuma could not tell "forge down" from "router gone".

max_tokens/max_completion_tokens are translated per backend: Ollama accepts only
the former, gpt-5.4-mini rejects it outright.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:30:14 +00:00