Files
Dolon (Agent)andClaude Opus 5 3155d4a230 forgeway 0.1.1 — per-cluster OpenAI-compatible LLM router
Two endpoints over forge's Ollama gpt-oss:120b:
  /local-optimistic/v1  forge, else cloud gpt-5.4-mini
  /local-strict/v1      forge, else a clean 503 the caller handles

Discovery, not configuration: each instance probes an ordered candidate list
(stable DNS name, then the Lounge and future Serenity literals) concurrently but
in priority order, so the happy path is one round trip and silicon's move to
Serenity needs no rewrite.

/healthz runs a real 1-token completion — forge can be up with Ollama stopped.
Liveness/readiness are deliberately NOT forge-gated: otherwise k8s would evict
the pods during an outage and callers would get a connection failure instead of
the promised 503, and Kuma could not tell "forge down" from "router gone".

max_tokens/max_completion_tokens are translated per backend: Ollama accepts only
the former, gpt-5.4-mini rejects it outright.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:30:14 +00:00

65 lines
2.7 KiB
Bash
Executable File

#!/usr/bin/env bash
# Local smoke test: proves both endpoints against a real forge, and proves the
# strict-503 path by pointing a second instance at a deliberately bad upstream.
# Never stops Ollama on forge to produce the failure.
set -uo pipefail
BASE_GOOD="http://127.0.0.1:18080"
BASE_BAD="http://127.0.0.1:18081"
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
PY="$ROOT/.venv/bin/python"
start() { # name port upstreams
FORGEWAY_NAME=forgeway FORGEWAY_SITE="$1" FORGEWAY_UPSTREAMS="$3" \
FORGEWAY_DISCOVERY_TIMEOUT=3 FORGEWAY_HEALTH_TIMEOUT=60 \
"$PY" -m uvicorn app.main:app --host 127.0.0.1 --port "$2" --no-access-log \
>"/tmp/forgeway-$1.log" 2>&1 &
echo $!
}
cd "$ROOT"
PID_GOOD=$(start smoke-good 18080 "http://forge.dmz.falkvinge.net:11434")
PID_BAD=$(start smoke-bad 18081 "http://127.0.0.1:1")
trap 'kill $PID_GOOD $PID_BAD 2>/dev/null' EXIT
sleep 4
hdr() { printf '\n\033[1m== %s ==\033[0m\n' "$1"; }
probe() { curl -s -o /tmp/fw-body -m 120 -w "%{http_code}" "$@"; }
hdr "healthz (forge reachable)"
echo "HTTP $(probe "$BASE_GOOD/healthz")"; cat /tmp/fw-body; echo
hdr "local-strict chat (forge reachable)"
echo "HTTP $(probe -X POST "$BASE_GOOD/local-strict/v1/chat/completions" \
-H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Reply with exactly: PONG"}],"max_tokens":10}')"
head -c 400 /tmp/fw-body; echo
hdr "local-optimistic chat (forge reachable -> should use forge)"
echo "HTTP $(probe -X POST "$BASE_GOOD/local-optimistic/v1/chat/completions" \
-H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Reply with exactly: PONG"}],"max_tokens":10}')"
head -c 300 /tmp/fw-body; echo
hdr "models"
echo "strict: $(curl -s "$BASE_GOOD/local-strict/v1/models")"
echo "optimistic: $(curl -s "$BASE_GOOD/local-optimistic/v1/models")"
hdr "BAD UPSTREAM: healthz must be 503"
echo "HTTP $(probe "$BASE_BAD/healthz")"; cat /tmp/fw-body; echo
hdr "BAD UPSTREAM: local-strict must be a clean 503"
echo "HTTP $(probe -X POST "$BASE_BAD/local-strict/v1/chat/completions" \
-H 'Content-Type: application/json' -d '{"messages":[{"role":"user","content":"hi"}]}')"
cat /tmp/fw-body; echo
hdr "BAD UPSTREAM: local-optimistic must fall back to cloud"
echo "HTTP $(probe -X POST "$BASE_BAD/local-optimistic/v1/chat/completions" \
-H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Reply with exactly: PONG"}],"max_tokens":10}')"
head -c 400 /tmp/fw-body; echo
hdr "BAD UPSTREAM: liveness/readiness must stay 200 (so 503s are deliverable)"
echo "live: $(curl -s -o /dev/null -w '%{http_code}' "$BASE_BAD/healthz/live")"
echo "ready: $(curl -s -o /dev/null -w '%{http_code}' "$BASE_BAD/healthz/ready")"