fix(deploy): close container final review

This commit is contained in:
2026-07-12 00:44:39 +02:00
parent 0ca6346f75
commit a3a266fd81
30 changed files with 526 additions and 37 deletions
+34
View File
@@ -0,0 +1,34 @@
name: Container multi-architecture gate
on:
pull_request:
paths:
- "backend/**"
- "frontend/**"
- "harness/**"
- "docker/**"
- "scripts/verify-container-images.sh"
- ".github/workflows/container-multiarch.yml"
workflow_dispatch:
jobs:
verify:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
platform: [linux/amd64, linux/arm64]
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
with:
driver: docker
- name: Build, smoke, security-check, and inventory
env:
PLATFORM: ${{ matrix.platform }}
run: ./scripts/verify-container-images.sh
- uses: actions/upload-artifact@v4
with:
name: container-inventory-${{ strategy.job-index }}
path: .artifacts/container-images/
+3
View File
@@ -53,3 +53,6 @@ htmlcov/
# === MkDocs build output === # === MkDocs build output ===
site/ site/
# Generated container inventory / SBOM-equivalent verification artifacts
.artifacts/
+55 -6
View File
@@ -9,19 +9,19 @@ external in this profile.
Requirements: Docker Engine with Compose v2 and reachable DWH, vector, and embeddings Requirements: Docker Engine with Compose v2 and reachable DWH, vector, and embeddings
services. services.
1. Copy `deploy/env.example` to `deploy/.env` and fill in runtime credentials. The `.env` 1. For local development only, copy `deploy/env.example` to `deploy/.env` and fill in runtime
file is gitignored and is read only when the container starts; secrets are never copied credentials. The file is gitignored and is never copied into either image.
into either image.
2. Add or edit YAML workspace descriptors under `deploy/workspaces/`. These files are mounted 2. Add or edit YAML workspace descriptors under `deploy/workspaces/`. These files are mounted
read-only. Use relative `roots`; they resolve beneath `/data/workspaces/<workspace-name>`. read-only. Use relative `roots`; they resolve beneath `/data/workspaces/<workspace-name>`.
3. Start the external-service profile: 3. Start the external-service profile:
```sh ```sh
docker compose --profile external up --build --wait docker compose -f compose.yaml -f deploy/compose.local.yaml \
--profile external up --build --wait
``` ```
4. Open <http://localhost:8080>. Set `THOTH_HTTP_PORT` before starting to use another host 4. Open <http://127.0.0.1:8080>. The published port is loopback-only. Set `THOTH_HTTP_PORT`
port. before starting to use another loopback port.
Application state, including settings, sessions, artifacts, and indexes, lives in the named Application state, including settings, sessions, artifacts, and indexes, lives in the named
`thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an `thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an
@@ -44,3 +44,52 @@ volume afterward. It never targets the fixed `thothii` operator project or its v
`SMOKE_PROJECT` to a different explicit project name for reproducible debugging, and set `SMOKE_PROJECT` to a different explicit project name for reproducible debugging, and set
`KEEP_SMOKE_RESOURCES=1` to retain that smoke project's resources for inspection; remove them `KEEP_SMOKE_RESOURCES=1` to retain that smoke project's resources for inspection; remove them
later with `docker compose --project-name "$SMOKE_PROJECT" --profile external down --volumes`. later with `docker compose --project-name "$SMOKE_PROJECT" --profile external down --volumes`.
## Production trust boundary and secrets
ThothII does not implement OIDC. Do not expose its application port directly to a network.
The production pattern is an authenticated host reverse proxy that:
- terminates TLS and authenticates every request;
- removes any client-supplied identity header;
- injects one trusted `X-Authenticated-User` value;
- proxies to the loopback-only ThothII frontend.
[`deploy/nginx-authenticated-proxy.conf.example`](deploy/nginx-authenticated-proxy.conf.example)
shows the contract using nginx `auth_request`; replace the placeholder authentication gateway
with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts this boundary and
rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other
auth mode fails during core startup.
Production credentials use Compose secrets, not `deploy/.env`. Create four files outside the
repository, restrict their host permissions, and point these variables to them:
```sh
export THT_DWH_API_KEY_SECRET_FILE=/secure/thoth/dwh-api-key
export THT_VEC_API_KEY_SECRET_FILE=/secure/thoth/vector-reader-api-key
export THT_VEC_WRITE_API_KEY_SECRET_FILE=/secure/thoth/vector-writer-api-key
export THT_CA_SECRET_FILE=/secure/thoth/ca-chain.pem
export THT_DB_NAME=warehouse
export THT_DWH_REST_URL=https://dwh.example.test
export THT_VEC_REST_URL=https://vectors.example.test
export THT_OLLAMA_URL=https://embeddings.example.test
docker compose -f compose.yaml -f deploy/compose.production.yaml \
--profile external up --build --wait
```
The secrets and public CA chain are mounted read-only under `/run/secrets` and must be readable by
the core's UID 10001. See [`deploy/secrets/README.md`](deploy/secrets/README.md) for the verification
command. The frontend remains on loopback; the authenticated host proxy is the only public listener.
## Reproducible image verification
Base images use exact tags and immutable multi-platform manifest digests. Dependency update and
residual OS-repository limitations are documented in [`docker/LOCKS.md`](docker/LOCKS.md).
Run the shared architecture gate with `PLATFORM=linux/amd64` or `PLATFORM=linux/arm64`:
```sh
PLATFORM=linux/arm64 ./scripts/verify-container-images.sh
```
It builds both images, runs common version/runtime/security smokes, and emits an image/package
inventory beneath `.artifacts/container-images/`. CI runs the same script for both architectures.
+6 -1
View File
@@ -42,7 +42,12 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
const listModels = deps?.listModels ?? createPiModelLister(config); const listModels = deps?.listModels ?? createPiModelLister(config);
const getSettings = deps?.getSettings ?? (() => effectiveSettings(config, loadSettings(config))); const getSettings = deps?.getSettings ?? (() => effectiveSettings(config, loadSettings(config)));
app.addHook("preHandler", authPreHandler(config.authMode)); const authenticate = authPreHandler(config.authMode);
app.addHook("preHandler", async (req, reply) => {
// Process readiness is intentionally unauthenticated for local container/proxy probes.
if (req.url === "/health") return;
return authenticate(req, reply);
});
app.get("/health", async () => ({ status: "ok" })); app.get("/health", async () => ({ status: "ok" }));
sessionRoutes(app, { sessionRoutes(app, {
mgr, tht: tht as ThtRunner, hub, getSettings, mgr, tht: tht as ThtRunner, hub, getSettings,
+6 -3
View File
@@ -1,6 +1,6 @@
import type { FastifyRequest, FastifyReply } from "fastify"; import type { FastifyRequest, FastifyReply } from "fastify";
export function authPreHandler(mode: "none" | "mock" | "oidc") { export function authPreHandler(mode: "none" | "mock" | "upstream") {
return async (req: FastifyRequest, reply: FastifyReply) => { return async (req: FastifyRequest, reply: FastifyReply) => {
if (mode === "none") { if (mode === "none") {
(req as any).user = { id: "dev@local" }; (req as any).user = { id: "dev@local" };
@@ -9,8 +9,11 @@ export function authPreHandler(mode: "none" | "mock" | "oidc") {
id: (req.headers["x-mock-user"] as string) ?? "mock", id: (req.headers["x-mock-user"] as string) ?? "mock",
}; };
} else { } else {
reply.code(501); const id = req.headers["x-authenticated-user"];
throw new Error("OIDC non configurato (MVP: usa none/mock)"); if (typeof id !== "string" || id.trim() === "") {
return reply.code(401).send({ error: "authenticated upstream identity required" });
}
(req as any).user = { id };
} }
}; };
} }
+9 -2
View File
@@ -1,6 +1,6 @@
export interface AppConfig { export interface AppConfig {
host: string; port: number; harnessDir: string; thtBin: string; piBin: string; host: string; port: number; harnessDir: string; thtBin: string; piBin: string;
authMode: "none" | "mock" | "oidc"; authMode: "none" | "mock" | "upstream";
defaults: { provider?: string; model?: string; thinking?: string }; defaults: { provider?: string; model?: string; thinking?: string };
maxPiProcesses: number; maxPiProcesses: number;
settingsFile: string; settingsFile: string;
@@ -8,13 +8,20 @@ export interface AppConfig {
ollamaEnsureTimeoutMs: number; ollamaEnsureTimeoutMs: number;
} }
export function loadConfig(env: Record<string, string | undefined>): AppConfig { export function loadConfig(env: Record<string, string | undefined>): AppConfig {
const authMode = env.AUTH_MODE ?? "none";
if (!(["none", "mock", "upstream"] as const).includes(authMode as AppConfig["authMode"])) {
throw new Error(`unsupported AUTH_MODE=${authMode}; use none, mock, or upstream`);
}
if (env.THOTH_PUBLIC_EXPOSURE === "true" && authMode !== "upstream") {
throw new Error("public exposure requires AUTH_MODE=upstream behind a trusted proxy");
}
return { return {
host: env.HOST ?? "127.0.0.1", host: env.HOST ?? "127.0.0.1",
port: Number(env.PORT ?? 8787), port: Number(env.PORT ?? 8787),
harnessDir: env.THT_HARNESS_DIR ?? "../harness", harnessDir: env.THT_HARNESS_DIR ?? "../harness",
thtBin: env.THT_BIN ?? "tht", thtBin: env.THT_BIN ?? "tht",
piBin: env.PI_BIN ?? "pi", piBin: env.PI_BIN ?? "pi",
authMode: (env.AUTH_MODE as AppConfig["authMode"]) ?? "none", authMode: authMode as AppConfig["authMode"],
defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING }, defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING },
maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4), maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4),
settingsFile: env.SETTINGS_FILE ?? "data/settings.json", settingsFile: env.SETTINGS_FILE ?? "data/settings.json",
+14
View File
@@ -22,3 +22,17 @@ test("mode mock legge l'header", async () => {
}); });
expect(res.json()).toEqual({ id: "alice" }); expect(res.json()).toEqual({ id: "alice" });
}); });
test("upstream mode requires the authenticated proxy identity header", async () => {
const app = Fastify();
app.addHook("preHandler", authPreHandler("upstream"));
app.get("/me", async (req) => getUser(req));
expect((await app.inject({ method: "GET", url: "/me" })).statusCode).toBe(401);
const authenticated = await app.inject({
method: "GET",
url: "/me",
headers: { "x-authenticated-user": "alice@example.test" },
});
expect(authenticated.json()).toEqual({ id: "alice@example.test" });
});
+14
View File
@@ -32,3 +32,17 @@ test("loadConfig keeps local development defaults", () => {
}); });
expect(loadConfig({}).dataRoot).toBeUndefined(); expect(loadConfig({}).dataRoot).toBeUndefined();
}); });
test("loadConfig rejects unauthenticated public exposure", () => {
expect(() => loadConfig({
THOTH_PUBLIC_EXPOSURE: "true",
AUTH_MODE: "none",
})).toThrow(/public exposure requires AUTH_MODE=upstream/);
});
test("loadConfig accepts an authenticated upstream trust boundary", () => {
expect(loadConfig({
THOTH_PUBLIC_EXPOSURE: "true",
AUTH_MODE: "upstream",
}).authMode).toBe("upstream");
});
+11
View File
@@ -10,6 +10,17 @@ test("GET /health reports process readiness without external services", async ()
expect(res.json()).toEqual({ status: "ok" }); expect(res.json()).toEqual({ status: "ok" });
}); });
test("GET /health remains available to container probes in upstream auth mode", async () => {
const app = buildApp(loadConfig({
THT_HARNESS_DIR: "/tmp/h",
AUTH_MODE: "upstream",
THOTH_PUBLIC_EXPOSURE: "true",
}));
const res = await app.inject({ method: "GET", url: "/health" });
expect(res.statusCode).toBe(200);
expect(res.json()).toEqual({ status: "ok" });
});
test("SSE response headers are flushed before the first event", async () => { test("SSE response headers are flushed before the first event", async () => {
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/tmp/h" })); const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/tmp/h" }));
await app.listen({ port: 0, host: "127.0.0.1" }); await app.listen({ port: 0, host: "127.0.0.1" });
+3 -4
View File
@@ -6,10 +6,9 @@ services:
build: build:
context: . context: .
dockerfile: docker/core.Dockerfile dockerfile: docker/core.Dockerfile
env_file:
- path: deploy/.env
required: false
environment: environment:
AUTH_MODE: "${AUTH_MODE:-none}"
THOTH_PUBLIC_EXPOSURE: "${THOTH_PUBLIC_EXPOSURE:-false}"
THT_DATA_ROOT: /data THT_DATA_ROOT: /data
SETTINGS_FILE: /data/settings/settings.json SETTINGS_FILE: /data/settings/settings.json
volumes: volumes:
@@ -31,7 +30,7 @@ services:
environment: environment:
BACKEND_BASE_URL: /api BACKEND_BASE_URL: /api
ports: ports:
- "${THOTH_HTTP_PORT:-8080}:8080" - "127.0.0.1:${THOTH_HTTP_PORT:-8080}:8080"
depends_on: depends_on:
core: core:
condition: service_healthy condition: service_healthy
+5
View File
@@ -0,0 +1,5 @@
services:
core:
env_file:
- path: ./deploy/.env
required: false
+33
View File
@@ -0,0 +1,33 @@
services:
core:
environment:
AUTH_MODE: upstream
THOTH_PUBLIC_EXPOSURE: "true"
THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME}
THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL}
THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL}
THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL}
THT_DOCS_ROOT: ${THT_DOCS_ROOT:-/data/workspaces/example/evidence-source}
THT_DWH_API_KEY_FILE: /run/secrets/dwh_api_key
THT_VEC_API_KEY_FILE: /run/secrets/vector_reader_api_key
THT_VEC_WRITE_API_KEY_FILE: /run/secrets/vector_writer_api_key
THT_SSL_CA: /run/secrets/thoth_ca.pem
secrets:
- source: dwh_api_key
target: dwh_api_key
- source: vector_reader_api_key
target: vector_reader_api_key
- source: vector_writer_api_key
target: vector_writer_api_key
- source: thoth_ca
target: thoth_ca.pem
secrets:
dwh_api_key:
file: ${THT_DWH_API_KEY_SECRET_FILE:?set THT_DWH_API_KEY_SECRET_FILE}
vector_reader_api_key:
file: ${THT_VEC_API_KEY_SECRET_FILE:?set THT_VEC_API_KEY_SECRET_FILE}
vector_writer_api_key:
file: ${THT_VEC_WRITE_API_KEY_SECRET_FILE:?set THT_VEC_WRITE_API_KEY_SECRET_FILE}
thoth_ca:
file: ${THT_CA_SECRET_FILE:?set THT_CA_SECRET_FILE}
+2 -2
View File
@@ -1,5 +1,5 @@
# Copy this file to deploy/.env. Never commit deploy/.env or real credentials. # LOCAL DEVELOPMENT ONLY. Copy to deploy/.env and use deploy/compose.local.yaml.
# Compose passes these values to the core container at runtime; images contain no secrets. # Never commit deploy/.env or real credentials. Production uses Compose secrets instead.
# Optional application defaults # Optional application defaults
PI_PROVIDER= PI_PROVIDER=
@@ -0,0 +1,26 @@
# Host nginx example. The auth service MUST authenticate every request and return a stable
# identity in X-Authenticated-User. ThothII itself remains on 127.0.0.1:8080.
server {
listen 443 ssl;
server_name thoth.example.test;
ssl_certificate /etc/nginx/tls/fullchain.pem;
ssl_certificate_key /etc/nginx/tls/privkey.pem;
location = /_authenticate {
internal;
proxy_pass http://authentication-gateway/verify;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
}
location / {
auth_request /_authenticate;
auth_request_set $authenticated_user $upstream_http_x_authenticated_user;
proxy_set_header X-Authenticated-User $authenticated_user;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
}
+16
View File
@@ -0,0 +1,16 @@
# Runtime secrets and private CA
Do not put secret values in this directory or in Git. For production, create files outside the
repository and point the `*_SECRET_FILE` variables documented in the root README at them.
Compose mounts each file read-only beneath `/run/secrets`. The core process runs as UID 10001;
the mounted files must be readable by that UID. Docker Compose file-backed secrets are normally
mounted read-only with mode `0444`; verify with:
```sh
docker compose -f compose.yaml -f deploy/compose.production.yaml \
--profile external run --rm core sh -c 'id && test -r /run/secrets/thoth_ca.pem'
```
The CA file should contain only the public PEM certificate chain. API-key files should contain
one value with no surrounding quotes.
+14
View File
@@ -36,3 +36,17 @@ The small input file pins the harness's PEP 517 build backend as well; it is not
host environment. The image installs the resulting lock with pip's `--require-hashes`, then host environment. The image installs the resulting lock with pip's `--require-hashes`, then
installs the local `tht` project with `--no-deps --no-build-isolation`. This prevents both project installs the local `tht` project with `--no-deps --no-build-isolation`. This prevents both project
metadata and an isolated build environment from resolving unpinned packages. metadata and an isolated build environment from resolving unpinned packages.
## Base images
Every `FROM` uses an exact tag plus a multi-platform manifest-list digest. To update one:
1. Choose an exact patch tag that publishes both `linux/amd64` and `linux/arm64`.
2. Inspect it with `docker buildx imagetools inspect <tag>`.
3. Replace both the human-readable tag and `@sha256:...` digest.
4. Run `./scripts/verify-container-images.sh` and the Compose smoke.
5. Review the generated inventory under `.artifacts/container-images/`.
The digest freezes image layers, but `apt-get update` and `apk add` still consume mutable package
repositories during a no-cache rebuild. Full OS-package immutability would require Debian/Alpine
snapshot repositories and is not claimed by this deployment.
+18
View File
@@ -1,6 +1,24 @@
#!/bin/sh #!/bin/sh
set -eu set -eu
load_secret() {
value_name=$1
file_name=$2
secret_file=$(printenv "$file_name" 2>/dev/null || true)
if [ -n "$secret_file" ]; then
if [ ! -r "$secret_file" ]; then
echo "$file_name is not readable: $secret_file" >&2
exit 2
fi
secret_value=$(cat "$secret_file")
export "$value_name=$secret_value"
fi
}
load_secret THT_DWH_API_KEY THT_DWH_API_KEY_FILE
load_secret THT_VEC_API_KEY THT_VEC_API_KEY_FILE
load_secret THT_VEC_WRITE_API_KEY THT_VEC_WRITE_API_KEY_FILE
case "${1:-server}" in case "${1:-server}" in
server) server)
shift || true shift || true
+4 -4
View File
@@ -1,11 +1,11 @@
# syntax=docker/dockerfile:1 # syntax=docker/dockerfile:1
FROM node:22.19.0-bookworm-slim AS node-runtime FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS node-runtime
WORKDIR /opt/pi-runtime WORKDIR /opt/pi-runtime
COPY docker/pi-runtime/package.json docker/pi-runtime/package-lock.json ./ COPY docker/pi-runtime/package.json docker/pi-runtime/package-lock.json ./
RUN npm ci --omit=dev --ignore-scripts --no-audit --no-fund \ RUN npm ci --omit=dev --ignore-scripts --no-audit --no-fund \
&& ./node_modules/.bin/pi --version && ./node_modules/.bin/pi --version
FROM node:22.19.0-bookworm-slim AS backend-build FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS backend-build
WORKDIR /src/backend WORKDIR /src/backend
COPY backend/package.json backend/package-lock.json ./ COPY backend/package.json backend/package-lock.json ./
RUN npm ci --no-audit --no-fund RUN npm ci --no-audit --no-fund
@@ -13,12 +13,12 @@ COPY backend/ ./
RUN npm run build \ RUN npm run build \
&& npm prune --omit=dev && npm prune --omit=dev
FROM node:22.19.0-bookworm-slim AS gate-deps FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS gate-deps
WORKDIR /src/harness WORKDIR /src/harness
COPY harness/package.json harness/package-lock.json ./ COPY harness/package.json harness/package-lock.json ./
RUN npm ci --no-audit --no-fund RUN npm ci --no-audit --no-fund
FROM python:3.12-slim-bookworm AS runtime FROM python:3.12.11-slim-bookworm@sha256:519591d6871b7bc437060736b9f7456b8731f1499a57e22e6c285135ae657bf7 AS runtime
RUN apt-get update \ RUN apt-get update \
&& apt-get install --yes --no-install-recommends ca-certificates curl \ && apt-get install --yes --no-install-recommends ca-certificates curl \
+14 -1
View File
@@ -1,7 +1,20 @@
#!/bin/sh #!/bin/sh
set -eu set -eu
backend_base_url=${BACKEND_BASE_URL:-/api} backend_base_url=${BACKEND_BASE_URL-/api}
case "$backend_base_url" in
""|/|/api|/api/) ;;
http://*|https://*)
if printf '%s' "$backend_base_url" | grep -Eq '[[:space:]]|^https?://[^/]*@'; then
echo "Invalid BACKEND_BASE_URL: credentials and whitespace are not allowed" >&2
exit 2
fi
;;
*)
echo "Invalid BACKEND_BASE_URL: use empty/root, /api, or an absolute http(s) URL" >&2
exit 2
;;
esac
runtime_config=$(jq -cn --arg backend_base_url "$backend_base_url" \ runtime_config=$(jq -cn --arg backend_base_url "$backend_base_url" \
'{backendBaseUrl: $backend_base_url}') '{backendBaseUrl: $backend_base_url}')
printf 'window.__THOTHII_CONFIG__ = %s;\n' "$runtime_config" \ printf 'window.__THOTHII_CONFIG__ = %s;\n' "$runtime_config" \
+2 -2
View File
@@ -1,12 +1,12 @@
# syntax=docker/dockerfile:1 # syntax=docker/dockerfile:1
FROM node:22.19.0-bookworm-slim AS build FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS build
WORKDIR /src/frontend WORKDIR /src/frontend
COPY frontend/package.json frontend/package-lock.json ./ COPY frontend/package.json frontend/package-lock.json ./
RUN npm ci --no-audit --no-fund RUN npm ci --no-audit --no-fund
COPY frontend/ ./ COPY frontend/ ./
RUN npm run build RUN npm run build
FROM nginxinc/nginx-unprivileged:1.27-alpine FROM nginxinc/nginx-unprivileged:1.27.5-alpine@sha256:65e3e85dbaed8ba248841d9d58a899b6197106c23cb0ff1a132b7bfe0547e4c0
USER root USER root
RUN apk add --no-cache jq RUN apk add --no-cache jq
COPY --from=build /src/frontend/dist /usr/share/nginx/html COPY --from=build /src/frontend/dist /usr/share/nginx/html
+10
View File
@@ -8,6 +8,13 @@ server {
try_files $uri =404; try_files $uri =404;
} }
location = /health {
proxy_pass http://core:8787/health;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_cache off;
}
location /api/ { location /api/ {
proxy_pass http://core:8787/; proxy_pass http://core:8787/;
proxy_http_version 1.1; proxy_http_version 1.1;
@@ -15,6 +22,9 @@ server {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
# Trusted only when AUTH_MODE=upstream and this frontend port is reachable solely
# from the authenticated host proxy documented in deploy/.
proxy_set_header X-Authenticated-User $http_x_authenticated_user;
proxy_buffering off; proxy_buffering off;
proxy_cache off; proxy_cache off;
proxy_read_timeout 1h; proxy_read_timeout 1h;
+1 -1
View File
@@ -7,7 +7,7 @@ assignment=$(sed \
/usr/share/nginx/html/config.js) /usr/share/nginx/html/config.js)
printf '%s\n' "$assignment" \ printf '%s\n' "$assignment" \
| jq -e --arg expected "${BACKEND_BASE_URL:-/api}" \ | jq -e --arg expected "${BACKEND_BASE_URL-/api}" \
'type == "object" and keys == ["backendBaseUrl"] and .backendBaseUrl == $expected' \ 'type == "object" and keys == ["backendBaseUrl"] and .backendBaseUrl == $expected' \
>/dev/null >/dev/null
@@ -236,6 +236,13 @@ git commit -m "build(docker): add runtime-configured frontend image"
### Task 5: Compose external profile and end-to-end smoke gate ### Task 5: Compose external profile and end-to-end smoke gate
> **Final-review security amendment (2026-07-12):** the frontend port binds to `127.0.0.1` by
> default. Public deployment uses an authenticated upstream proxy with `AUTH_MODE=upstream`;
> `THOTH_PUBLIC_EXPOSURE=true` plus `AUTH_MODE=none` is invalid. Local env files are development
> only; production uses read-only Compose secrets. Image gates pin exact tags and multi-platform
> digests and verify both linux/amd64 and linux/arm64 using the shared container verification
> script.
**Files:** **Files:**
- Create: `compose.yaml` - Create: `compose.yaml`
- Create: `deploy/env.example` - Create: `deploy/env.example`
@@ -36,6 +36,7 @@ Non sono presenti Dockerfile o file Compose. Il processo di sviluppo assume Pi e
6. **Configurazione dichiarativa e validata.** I workspace contengono riferimenti logici e configurazioni non segrete; i segreti sono in environment variables o secret store. 6. **Configurazione dichiarativa e validata.** I workspace contengono riferimenti logici e configurazioni non segrete; i segreti sono in environment variables o secret store.
7. **Capability esplicite.** Un adapter dichiara ciò che supporta. Le funzioni mancanti producono degradazione o blocco comprensibile, non emulazioni implicite. 7. **Capability esplicite.** Un adapter dichiara ciò che supporta. Le funzioni mancanti producono degradazione o blocco comprensibile, non emulazioni implicite.
8. **Read-only by construction sul DWH.** Credenziali, API e guard client-side mantengono la separazione dall'autorità di scrittura. 8. **Read-only by construction sul DWH.** Credenziali, API e guard client-side mantengono la separazione dall'autorità di scrittura.
9. **Esposizione sicura per default.** La porta applicativa pubblicata è vincolata a loopback. Un'esposizione pubblica richiede un reverse proxy autenticante esterno e `AUTH_MODE=upstream`; la combinazione pubblico + `none` viene rifiutata all'avvio. OIDC interno non fa parte di questa fase.
## 4. Packaging e runtime ## 4. Packaging e runtime
@@ -143,6 +144,11 @@ La configurazione si divide in:
- **workspace:** lingua, adapter, namespace, collezioni e policy di preprocessing; - **workspace:** lingua, adapter, namespace, collezioni e policy di preprocessing;
- **segreti:** password, token, certificati e chiavi reader/writer. - **segreti:** password, token, certificati e chiavi reader/writer.
Nel profilo locale i segreti possono provenire da un env-file non versionato. In produzione sono
file read-only sotto `/run/secrets`, leggibili dall'UID 10001. Il reverse proxy autenticante è un
confine fidato: rimuove header identità forniti dal client e inserisce
`X-Authenticated-User` soltanto dopo autenticazione.
Deve esistere un comando di diagnostica che produca sia output umano sia JSON pristino, rispettando il contratto CLI corrente. Deve esistere un comando di diagnostica che produca sia output umano sia JSON pristino, rispettando il contratto CLI corrente.
## 7. Pipeline di preprocessing ## 7. Pipeline di preprocessing
+14
View File
@@ -14,6 +14,20 @@ describe("resolveBackendUrl", () => {
it("preserves the client default when Vite has no configured backend", () => { it("preserves the client default when Vite has no configured backend", () => {
expect(backendBaseUrl).toBe(import.meta.env.VITE_BACKEND_URL ?? "http://localhost:8787"); expect(backendBaseUrl).toBe(import.meta.env.VITE_BACKEND_URL ?? "http://localhost:8787");
}); });
it.each(["/backend", "api", "//evil.test", "ftp://example.test", "https://user:pass@example.test"])(
"rejects unsupported backend URL %j",
(backendBaseUrl) => {
expect(() => resolveBackendUrl({ backendBaseUrl })).toThrow(/BACKEND_BASE_URL/);
},
);
it.each(["", "/", "/api", "/api/", "http://localhost:8787", "https://api.example.test/v1"])(
"accepts supported backend URL %j",
(backendBaseUrl) => {
expect(resolveBackendUrl({ backendBaseUrl })).toBe(backendBaseUrl);
},
);
}); });
describe("joinBackendPath", () => { describe("joinBackendPath", () => {
+13 -1
View File
@@ -9,7 +9,19 @@ declare global {
} }
export function resolveBackendUrl(config: RuntimeConfig | undefined): string { export function resolveBackendUrl(config: RuntimeConfig | undefined): string {
return config?.backendBaseUrl ?? import.meta.env.VITE_BACKEND_URL ?? ""; const value = config?.backendBaseUrl ?? import.meta.env.VITE_BACKEND_URL ?? "";
if (value === "" || value === "/" || value === "/api" || value === "/api/") return value;
try {
const url = new URL(value);
if ((url.protocol === "http:" || url.protocol === "https:") && !url.username && !url.password) {
return value;
}
} catch {
// Fall through to the single actionable runtime error below.
}
throw new Error(
"Invalid BACKEND_BASE_URL: use empty/root, /api, or an absolute http(s) URL without credentials",
);
} }
export function joinBackendPath(base: string, path: string): string { export function joinBackendPath(base: string, path: string): string {
+19 -2
View File
@@ -43,7 +43,24 @@ compose up --build --wait core frontend
published=$(compose port frontend 8080) published=$(compose port frontend 8080)
http_port=${published##*:} http_port=${published##*:}
curl --fail --silent --show-error "http://127.0.0.1:$http_port/health" >/dev/null assert_health() {
health_headers=$(mktemp)
health_body=$(mktemp)
if ! curl --fail --silent --show-error --dump-header "$health_headers" \
"http://127.0.0.1:$http_port/health" >"$health_body"; then
rm -f "$health_headers" "$health_body"
return 1
fi
if ! grep -qi '^content-type: application/json' "$health_headers" ||
! jq -e 'type == "object" and keys == ["status"] and .status == "ok"' \
"$health_body" >/dev/null; then
rm -f "$health_headers" "$health_body"
return 1
fi
rm -f "$health_headers" "$health_body"
}
assert_health
# The nginx proxy must preserve streaming semantics for the backend SSE endpoint. # The nginx proxy must preserve streaming semantics for the backend SSE endpoint.
headers=$(mktemp) headers=$(mktemp)
@@ -68,5 +85,5 @@ persisted=$(compose exec -T core sh -c 'cat /data/.compose-smoke-marker')
[ "$persisted" = "$marker" ] [ "$persisted" = "$marker" ]
compose exec -T core rm -f /data/.compose-smoke-marker compose exec -T core rm -f /data/.compose-smoke-marker
curl --fail --silent --show-error "http://127.0.0.1:$http_port/health" >/dev/null assert_health
echo "Compose health, SSE proxy, and restart persistence checks passed." echo "Compose health, SSE proxy, and restart persistence checks passed."
+42
View File
@@ -0,0 +1,42 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT HUP INT TERM
docker compose --profile external config >"$tmp/base.yaml"
grep -q 'host_ip: 127.0.0.1' "$tmp/base.yaml"
grep -q 'AUTH_MODE: none' "$tmp/base.yaml"
grep -q 'THOTH_PUBLIC_EXPOSURE: "false"' "$tmp/base.yaml"
if grep -q 'env_file:' "$tmp/base.yaml"; then
echo "base/production-neutral Compose must not load the local env file" >&2
exit 1
fi
docker compose -f compose.yaml -f deploy/compose.local.yaml \
--profile external config >"$tmp/local.yaml"
grep -q 'env_file:' deploy/compose.local.yaml
for secret in dwh reader writer ca; do printf '%s\n' "test-$secret" >"$tmp/$secret"; done
THT_DWH_API_KEY_SECRET_FILE="$tmp/dwh" \
THT_VEC_API_KEY_SECRET_FILE="$tmp/reader" \
THT_VEC_WRITE_API_KEY_SECRET_FILE="$tmp/writer" \
THT_CA_SECRET_FILE="$tmp/ca" \
THT_DB_NAME=test THT_DWH_REST_URL=https://dwh.example.test \
THT_VEC_REST_URL=https://vector.example.test THT_OLLAMA_URL=https://embed.example.test \
docker compose -f compose.yaml -f deploy/compose.production.yaml \
--profile external config >"$tmp/production.yaml"
grep -q 'AUTH_MODE: upstream' "$tmp/production.yaml"
grep -q 'THOTH_PUBLIC_EXPOSURE: "true"' "$tmp/production.yaml"
grep -q 'target: thoth_ca.pem' "$tmp/production.yaml"
grep -q 'THT_DWH_API_KEY_FILE: /run/secrets/dwh_api_key' "$tmp/production.yaml"
if awk '/^FROM / && $2 !~ /@sha256:/ { found=1 } END { exit !found }' \
docker/core.Dockerfile docker/frontend.Dockerfile; then
echo "every Dockerfile base must include an immutable digest" >&2
exit 1
fi
echo "container deployment security contract passed."
+75 -8
View File
@@ -3,17 +3,84 @@ set -eu
cd "$(dirname "$0")/.." cd "$(dirname "$0")/.."
script=scripts/docker-smoke.sh tmp=$(mktemp -d)
sh -n "$script" trap 'rm -rf "$tmp"' EXIT HUP INT TERM
log="$tmp/docker.log"
marker_file="$tmp/marker"
grep -q 'SMOKE_PROJECT' "$script" mkdir -p "$tmp/bin"
grep -q -- '--project-name' "$script" cat >"$tmp/bin/docker" <<'EOF'
grep -q 'KEEP_SMOKE_RESOURCES' "$script" #!/bin/sh
grep -q 'down --volumes' "$script" set -eu
printf '%s\n' "$*" >>"$FAKE_DOCKER_LOG"
if grep -q -- 'down --remove-orphans' "$script"; then case " $* " in
*" port frontend 8080 "*) printf '%s\n' '127.0.0.1:49152' ;;
*" exec -T core sh -c "*"printf"*)
for last do :; done
printf '%s\n' "$last" >"$FAKE_MARKER_FILE"
;;
*" exec -T core sh -c "*"cat /data/.compose-smoke-marker"*)
cat "$FAKE_MARKER_FILE"
;;
esac
EOF
cat >"$tmp/bin/curl" <<'EOF'
#!/bin/sh
set -eu
header_file=""
for arg do
if [ "${previous:-}" = "--dump-header" ]; then header_file=$arg; fi
previous=$arg
done
if [ -n "$header_file" ]; then
case "$*" in
*"/events"*) printf 'HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nCache-Control: no-cache\r\n\r\n' >"$header_file" ;;
*) printf 'HTTP/1.1 200 OK\r\nContent-Type: application/json; charset=utf-8\r\n\r\n' >"$header_file" ;;
esac
fi
case "$*" in
*"/health"*) printf '%s\n' '{"status":"ok"}' ;;
esac
EOF
chmod +x "$tmp/bin/docker" "$tmp/bin/curl"
run_smoke() {
PATH="$tmp/bin:$PATH" \
FAKE_DOCKER_LOG="$log" \
FAKE_MARKER_FILE="$marker_file" \
SMOKE_PROJECT="$1" \
KEEP_SMOKE_RESOURCES="${2:-0}" \
./scripts/docker-smoke.sh
}
run_smoke thothii-smoke-dynamic
while IFS= read -r invocation; do
case "$invocation" in
"compose --project-name thothii-smoke-dynamic --profile external "*) ;;
*) echo "Compose invocation escaped the smoke project/profile: $invocation" >&2; exit 1 ;;
esac
done <"$log"
grep -q ' down --volumes$' "$log"
if grep -q -- '--remove-orphans' "$log"; then
echo "smoke cleanup must not remove operator orphans" >&2 echo "smoke cleanup must not remove operator orphans" >&2
exit 1 exit 1
fi fi
echo "docker-smoke isolation contract passed." : >"$log"
run_smoke thothii-smoke-kept 1
if grep -q ' down ' "$log"; then
echo "KEEP_SMOKE_RESOURCES=1 unexpectedly cleaned the project" >&2
exit 1
fi
: >"$log"
if PATH="$tmp/bin:$PATH" FAKE_DOCKER_LOG="$log" FAKE_MARKER_FILE="$marker_file" \
SMOKE_PROJECT=thothii ./scripts/docker-smoke.sh >/dev/null 2>&1; then
echo "reserved operator project was accepted" >&2
exit 1
fi
test ! -s "$log"
echo "docker-smoke dynamic isolation contract passed."
+50
View File
@@ -0,0 +1,50 @@
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
platform=${PLATFORM:-linux/arm64}
slug=$(printf '%s' "$platform" | tr '/:' '--')
core_image="thothii-core:verify-$slug"
frontend_image="thothii-frontend:verify-$slug"
inventory_dir=${CONTAINER_INVENTORY_DIR:-.artifacts/container-images/$slug}
mkdir -p "$inventory_dir"
docker buildx build --platform "$platform" --load \
-f docker/core.Dockerfile -t "$core_image" .
docker buildx build --platform "$platform" --load \
-f docker/frontend.Dockerfile -t "$frontend_image" .
docker run --rm --platform "$platform" --entrypoint /app/docker/smoke/core-smoke.sh \
"$core_image"
docker run --rm --platform "$platform" -e BACKEND_BASE_URL=/api \
"$frontend_image" frontend-config-smoke
docker run --rm --platform "$platform" -e BACKEND_BASE_URL= \
"$frontend_image" frontend-config-smoke
if docker run --rm --platform "$platform" -e BACKEND_BASE_URL=/backend \
"$frontend_image" frontend-config-smoke >/dev/null 2>&1; then
echo "frontend accepted an unsupported BACKEND_BASE_URL" >&2
exit 1
fi
if docker run --rm --platform "$platform" -e THOTH_PUBLIC_EXPOSURE=true -e AUTH_MODE=none \
"$core_image" server >/dev/null 2>&1; then
echo "core accepted public exposure without upstream authentication" >&2
exit 1
fi
docker image inspect "$core_image" >"$inventory_dir/core-image-inspect.json"
docker image inspect "$frontend_image" >"$inventory_dir/frontend-image-inspect.json"
docker run --rm --platform "$platform" --entrypoint sh "$core_image" -c \
'dpkg-query -W; /opt/venv/bin/pip freeze; /opt/venv/bin/python -c '"'"'import glob,json; rows=set();
for path in glob.glob("/app/backend/node_modules/**/package.json", recursive=True):
try:
package=json.load(open(path)); rows.add((package.get("name","?"), package.get("version","?")))
except (OSError, ValueError): pass
print("\n".join(f"{name}=={version}" for name,version in sorted(rows)))'"'"'' \
>"$inventory_dir/core-packages.txt"
docker run --rm --platform "$platform" --entrypoint sh "$frontend_image" -c 'apk info -vv' \
>"$inventory_dir/frontend-packages.txt"
echo "container verification and inventory complete for $platform"