fix(deploy): close container final review
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
name: Container multi-architecture gate
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- "backend/**"
|
||||
- "frontend/**"
|
||||
- "harness/**"
|
||||
- "docker/**"
|
||||
- "scripts/verify-container-images.sh"
|
||||
- ".github/workflows/container-multiarch.yml"
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
runs-on: ubuntu-24.04
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
platform: [linux/amd64, linux/arm64]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: docker/setup-qemu-action@v3
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
with:
|
||||
driver: docker
|
||||
- name: Build, smoke, security-check, and inventory
|
||||
env:
|
||||
PLATFORM: ${{ matrix.platform }}
|
||||
run: ./scripts/verify-container-images.sh
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: container-inventory-${{ strategy.job-index }}
|
||||
path: .artifacts/container-images/
|
||||
@@ -53,3 +53,6 @@ htmlcov/
|
||||
|
||||
# === MkDocs build output ===
|
||||
site/
|
||||
|
||||
# Generated container inventory / SBOM-equivalent verification artifacts
|
||||
.artifacts/
|
||||
|
||||
@@ -9,19 +9,19 @@ external in this profile.
|
||||
Requirements: Docker Engine with Compose v2 and reachable DWH, vector, and embeddings
|
||||
services.
|
||||
|
||||
1. Copy `deploy/env.example` to `deploy/.env` and fill in runtime credentials. The `.env`
|
||||
file is gitignored and is read only when the container starts; secrets are never copied
|
||||
into either image.
|
||||
1. For local development only, copy `deploy/env.example` to `deploy/.env` and fill in runtime
|
||||
credentials. The file is gitignored and is never copied into either image.
|
||||
2. Add or edit YAML workspace descriptors under `deploy/workspaces/`. These files are mounted
|
||||
read-only. Use relative `roots`; they resolve beneath `/data/workspaces/<workspace-name>`.
|
||||
3. Start the external-service profile:
|
||||
|
||||
```sh
|
||||
docker compose --profile external up --build --wait
|
||||
docker compose -f compose.yaml -f deploy/compose.local.yaml \
|
||||
--profile external up --build --wait
|
||||
```
|
||||
|
||||
4. Open <http://localhost:8080>. Set `THOTH_HTTP_PORT` before starting to use another host
|
||||
port.
|
||||
4. Open <http://127.0.0.1:8080>. The published port is loopback-only. Set `THOTH_HTTP_PORT`
|
||||
before starting to use another loopback port.
|
||||
|
||||
Application state, including settings, sessions, artifacts, and indexes, lives in the named
|
||||
`thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an
|
||||
@@ -44,3 +44,52 @@ volume afterward. It never targets the fixed `thothii` operator project or its v
|
||||
`SMOKE_PROJECT` to a different explicit project name for reproducible debugging, and set
|
||||
`KEEP_SMOKE_RESOURCES=1` to retain that smoke project's resources for inspection; remove them
|
||||
later with `docker compose --project-name "$SMOKE_PROJECT" --profile external down --volumes`.
|
||||
|
||||
## Production trust boundary and secrets
|
||||
|
||||
ThothII does not implement OIDC. Do not expose its application port directly to a network.
|
||||
The production pattern is an authenticated host reverse proxy that:
|
||||
|
||||
- terminates TLS and authenticates every request;
|
||||
- removes any client-supplied identity header;
|
||||
- injects one trusted `X-Authenticated-User` value;
|
||||
- proxies to the loopback-only ThothII frontend.
|
||||
|
||||
[`deploy/nginx-authenticated-proxy.conf.example`](deploy/nginx-authenticated-proxy.conf.example)
|
||||
shows the contract using nginx `auth_request`; replace the placeholder authentication gateway
|
||||
with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts this boundary and
|
||||
rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other
|
||||
auth mode fails during core startup.
|
||||
|
||||
Production credentials use Compose secrets, not `deploy/.env`. Create four files outside the
|
||||
repository, restrict their host permissions, and point these variables to them:
|
||||
|
||||
```sh
|
||||
export THT_DWH_API_KEY_SECRET_FILE=/secure/thoth/dwh-api-key
|
||||
export THT_VEC_API_KEY_SECRET_FILE=/secure/thoth/vector-reader-api-key
|
||||
export THT_VEC_WRITE_API_KEY_SECRET_FILE=/secure/thoth/vector-writer-api-key
|
||||
export THT_CA_SECRET_FILE=/secure/thoth/ca-chain.pem
|
||||
export THT_DB_NAME=warehouse
|
||||
export THT_DWH_REST_URL=https://dwh.example.test
|
||||
export THT_VEC_REST_URL=https://vectors.example.test
|
||||
export THT_OLLAMA_URL=https://embeddings.example.test
|
||||
docker compose -f compose.yaml -f deploy/compose.production.yaml \
|
||||
--profile external up --build --wait
|
||||
```
|
||||
|
||||
The secrets and public CA chain are mounted read-only under `/run/secrets` and must be readable by
|
||||
the core's UID 10001. See [`deploy/secrets/README.md`](deploy/secrets/README.md) for the verification
|
||||
command. The frontend remains on loopback; the authenticated host proxy is the only public listener.
|
||||
|
||||
## Reproducible image verification
|
||||
|
||||
Base images use exact tags and immutable multi-platform manifest digests. Dependency update and
|
||||
residual OS-repository limitations are documented in [`docker/LOCKS.md`](docker/LOCKS.md).
|
||||
Run the shared architecture gate with `PLATFORM=linux/amd64` or `PLATFORM=linux/arm64`:
|
||||
|
||||
```sh
|
||||
PLATFORM=linux/arm64 ./scripts/verify-container-images.sh
|
||||
```
|
||||
|
||||
It builds both images, runs common version/runtime/security smokes, and emits an image/package
|
||||
inventory beneath `.artifacts/container-images/`. CI runs the same script for both architectures.
|
||||
|
||||
+6
-1
@@ -42,7 +42,12 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
const listModels = deps?.listModels ?? createPiModelLister(config);
|
||||
const getSettings = deps?.getSettings ?? (() => effectiveSettings(config, loadSettings(config)));
|
||||
|
||||
app.addHook("preHandler", authPreHandler(config.authMode));
|
||||
const authenticate = authPreHandler(config.authMode);
|
||||
app.addHook("preHandler", async (req, reply) => {
|
||||
// Process readiness is intentionally unauthenticated for local container/proxy probes.
|
||||
if (req.url === "/health") return;
|
||||
return authenticate(req, reply);
|
||||
});
|
||||
app.get("/health", async () => ({ status: "ok" }));
|
||||
sessionRoutes(app, {
|
||||
mgr, tht: tht as ThtRunner, hub, getSettings,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { FastifyRequest, FastifyReply } from "fastify";
|
||||
|
||||
export function authPreHandler(mode: "none" | "mock" | "oidc") {
|
||||
export function authPreHandler(mode: "none" | "mock" | "upstream") {
|
||||
return async (req: FastifyRequest, reply: FastifyReply) => {
|
||||
if (mode === "none") {
|
||||
(req as any).user = { id: "dev@local" };
|
||||
@@ -9,8 +9,11 @@ export function authPreHandler(mode: "none" | "mock" | "oidc") {
|
||||
id: (req.headers["x-mock-user"] as string) ?? "mock",
|
||||
};
|
||||
} else {
|
||||
reply.code(501);
|
||||
throw new Error("OIDC non configurato (MVP: usa none/mock)");
|
||||
const id = req.headers["x-authenticated-user"];
|
||||
if (typeof id !== "string" || id.trim() === "") {
|
||||
return reply.code(401).send({ error: "authenticated upstream identity required" });
|
||||
}
|
||||
(req as any).user = { id };
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
export interface AppConfig {
|
||||
host: string; port: number; harnessDir: string; thtBin: string; piBin: string;
|
||||
authMode: "none" | "mock" | "oidc";
|
||||
authMode: "none" | "mock" | "upstream";
|
||||
defaults: { provider?: string; model?: string; thinking?: string };
|
||||
maxPiProcesses: number;
|
||||
settingsFile: string;
|
||||
@@ -8,13 +8,20 @@ export interface AppConfig {
|
||||
ollamaEnsureTimeoutMs: number;
|
||||
}
|
||||
export function loadConfig(env: Record<string, string | undefined>): AppConfig {
|
||||
const authMode = env.AUTH_MODE ?? "none";
|
||||
if (!(["none", "mock", "upstream"] as const).includes(authMode as AppConfig["authMode"])) {
|
||||
throw new Error(`unsupported AUTH_MODE=${authMode}; use none, mock, or upstream`);
|
||||
}
|
||||
if (env.THOTH_PUBLIC_EXPOSURE === "true" && authMode !== "upstream") {
|
||||
throw new Error("public exposure requires AUTH_MODE=upstream behind a trusted proxy");
|
||||
}
|
||||
return {
|
||||
host: env.HOST ?? "127.0.0.1",
|
||||
port: Number(env.PORT ?? 8787),
|
||||
harnessDir: env.THT_HARNESS_DIR ?? "../harness",
|
||||
thtBin: env.THT_BIN ?? "tht",
|
||||
piBin: env.PI_BIN ?? "pi",
|
||||
authMode: (env.AUTH_MODE as AppConfig["authMode"]) ?? "none",
|
||||
authMode: authMode as AppConfig["authMode"],
|
||||
defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING },
|
||||
maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4),
|
||||
settingsFile: env.SETTINGS_FILE ?? "data/settings.json",
|
||||
|
||||
@@ -22,3 +22,17 @@ test("mode mock legge l'header", async () => {
|
||||
});
|
||||
expect(res.json()).toEqual({ id: "alice" });
|
||||
});
|
||||
|
||||
test("upstream mode requires the authenticated proxy identity header", async () => {
|
||||
const app = Fastify();
|
||||
app.addHook("preHandler", authPreHandler("upstream"));
|
||||
app.get("/me", async (req) => getUser(req));
|
||||
|
||||
expect((await app.inject({ method: "GET", url: "/me" })).statusCode).toBe(401);
|
||||
const authenticated = await app.inject({
|
||||
method: "GET",
|
||||
url: "/me",
|
||||
headers: { "x-authenticated-user": "alice@example.test" },
|
||||
});
|
||||
expect(authenticated.json()).toEqual({ id: "alice@example.test" });
|
||||
});
|
||||
|
||||
@@ -32,3 +32,17 @@ test("loadConfig keeps local development defaults", () => {
|
||||
});
|
||||
expect(loadConfig({}).dataRoot).toBeUndefined();
|
||||
});
|
||||
|
||||
test("loadConfig rejects unauthenticated public exposure", () => {
|
||||
expect(() => loadConfig({
|
||||
THOTH_PUBLIC_EXPOSURE: "true",
|
||||
AUTH_MODE: "none",
|
||||
})).toThrow(/public exposure requires AUTH_MODE=upstream/);
|
||||
});
|
||||
|
||||
test("loadConfig accepts an authenticated upstream trust boundary", () => {
|
||||
expect(loadConfig({
|
||||
THOTH_PUBLIC_EXPOSURE: "true",
|
||||
AUTH_MODE: "upstream",
|
||||
}).authMode).toBe("upstream");
|
||||
});
|
||||
|
||||
@@ -10,6 +10,17 @@ test("GET /health reports process readiness without external services", async ()
|
||||
expect(res.json()).toEqual({ status: "ok" });
|
||||
});
|
||||
|
||||
test("GET /health remains available to container probes in upstream auth mode", async () => {
|
||||
const app = buildApp(loadConfig({
|
||||
THT_HARNESS_DIR: "/tmp/h",
|
||||
AUTH_MODE: "upstream",
|
||||
THOTH_PUBLIC_EXPOSURE: "true",
|
||||
}));
|
||||
const res = await app.inject({ method: "GET", url: "/health" });
|
||||
expect(res.statusCode).toBe(200);
|
||||
expect(res.json()).toEqual({ status: "ok" });
|
||||
});
|
||||
|
||||
test("SSE response headers are flushed before the first event", async () => {
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/tmp/h" }));
|
||||
await app.listen({ port: 0, host: "127.0.0.1" });
|
||||
|
||||
+3
-4
@@ -6,10 +6,9 @@ services:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: docker/core.Dockerfile
|
||||
env_file:
|
||||
- path: deploy/.env
|
||||
required: false
|
||||
environment:
|
||||
AUTH_MODE: "${AUTH_MODE:-none}"
|
||||
THOTH_PUBLIC_EXPOSURE: "${THOTH_PUBLIC_EXPOSURE:-false}"
|
||||
THT_DATA_ROOT: /data
|
||||
SETTINGS_FILE: /data/settings/settings.json
|
||||
volumes:
|
||||
@@ -31,7 +30,7 @@ services:
|
||||
environment:
|
||||
BACKEND_BASE_URL: /api
|
||||
ports:
|
||||
- "${THOTH_HTTP_PORT:-8080}:8080"
|
||||
- "127.0.0.1:${THOTH_HTTP_PORT:-8080}:8080"
|
||||
depends_on:
|
||||
core:
|
||||
condition: service_healthy
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
services:
|
||||
core:
|
||||
env_file:
|
||||
- path: ./deploy/.env
|
||||
required: false
|
||||
@@ -0,0 +1,33 @@
|
||||
services:
|
||||
core:
|
||||
environment:
|
||||
AUTH_MODE: upstream
|
||||
THOTH_PUBLIC_EXPOSURE: "true"
|
||||
THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME}
|
||||
THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL}
|
||||
THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL}
|
||||
THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL}
|
||||
THT_DOCS_ROOT: ${THT_DOCS_ROOT:-/data/workspaces/example/evidence-source}
|
||||
THT_DWH_API_KEY_FILE: /run/secrets/dwh_api_key
|
||||
THT_VEC_API_KEY_FILE: /run/secrets/vector_reader_api_key
|
||||
THT_VEC_WRITE_API_KEY_FILE: /run/secrets/vector_writer_api_key
|
||||
THT_SSL_CA: /run/secrets/thoth_ca.pem
|
||||
secrets:
|
||||
- source: dwh_api_key
|
||||
target: dwh_api_key
|
||||
- source: vector_reader_api_key
|
||||
target: vector_reader_api_key
|
||||
- source: vector_writer_api_key
|
||||
target: vector_writer_api_key
|
||||
- source: thoth_ca
|
||||
target: thoth_ca.pem
|
||||
|
||||
secrets:
|
||||
dwh_api_key:
|
||||
file: ${THT_DWH_API_KEY_SECRET_FILE:?set THT_DWH_API_KEY_SECRET_FILE}
|
||||
vector_reader_api_key:
|
||||
file: ${THT_VEC_API_KEY_SECRET_FILE:?set THT_VEC_API_KEY_SECRET_FILE}
|
||||
vector_writer_api_key:
|
||||
file: ${THT_VEC_WRITE_API_KEY_SECRET_FILE:?set THT_VEC_WRITE_API_KEY_SECRET_FILE}
|
||||
thoth_ca:
|
||||
file: ${THT_CA_SECRET_FILE:?set THT_CA_SECRET_FILE}
|
||||
+2
-2
@@ -1,5 +1,5 @@
|
||||
# Copy this file to deploy/.env. Never commit deploy/.env or real credentials.
|
||||
# Compose passes these values to the core container at runtime; images contain no secrets.
|
||||
# LOCAL DEVELOPMENT ONLY. Copy to deploy/.env and use deploy/compose.local.yaml.
|
||||
# Never commit deploy/.env or real credentials. Production uses Compose secrets instead.
|
||||
|
||||
# Optional application defaults
|
||||
PI_PROVIDER=
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# Host nginx example. The auth service MUST authenticate every request and return a stable
|
||||
# identity in X-Authenticated-User. ThothII itself remains on 127.0.0.1:8080.
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name thoth.example.test;
|
||||
|
||||
ssl_certificate /etc/nginx/tls/fullchain.pem;
|
||||
ssl_certificate_key /etc/nginx/tls/privkey.pem;
|
||||
|
||||
location = /_authenticate {
|
||||
internal;
|
||||
proxy_pass http://authentication-gateway/verify;
|
||||
proxy_pass_request_body off;
|
||||
proxy_set_header Content-Length "";
|
||||
proxy_set_header X-Original-URI $request_uri;
|
||||
}
|
||||
|
||||
location / {
|
||||
auth_request /_authenticate;
|
||||
auth_request_set $authenticated_user $upstream_http_x_authenticated_user;
|
||||
proxy_set_header X-Authenticated-User $authenticated_user;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
proxy_set_header Host $host;
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
# Runtime secrets and private CA
|
||||
|
||||
Do not put secret values in this directory or in Git. For production, create files outside the
|
||||
repository and point the `*_SECRET_FILE` variables documented in the root README at them.
|
||||
|
||||
Compose mounts each file read-only beneath `/run/secrets`. The core process runs as UID 10001;
|
||||
the mounted files must be readable by that UID. Docker Compose file-backed secrets are normally
|
||||
mounted read-only with mode `0444`; verify with:
|
||||
|
||||
```sh
|
||||
docker compose -f compose.yaml -f deploy/compose.production.yaml \
|
||||
--profile external run --rm core sh -c 'id && test -r /run/secrets/thoth_ca.pem'
|
||||
```
|
||||
|
||||
The CA file should contain only the public PEM certificate chain. API-key files should contain
|
||||
one value with no surrounding quotes.
|
||||
@@ -36,3 +36,17 @@ The small input file pins the harness's PEP 517 build backend as well; it is not
|
||||
host environment. The image installs the resulting lock with pip's `--require-hashes`, then
|
||||
installs the local `tht` project with `--no-deps --no-build-isolation`. This prevents both project
|
||||
metadata and an isolated build environment from resolving unpinned packages.
|
||||
|
||||
## Base images
|
||||
|
||||
Every `FROM` uses an exact tag plus a multi-platform manifest-list digest. To update one:
|
||||
|
||||
1. Choose an exact patch tag that publishes both `linux/amd64` and `linux/arm64`.
|
||||
2. Inspect it with `docker buildx imagetools inspect <tag>`.
|
||||
3. Replace both the human-readable tag and `@sha256:...` digest.
|
||||
4. Run `./scripts/verify-container-images.sh` and the Compose smoke.
|
||||
5. Review the generated inventory under `.artifacts/container-images/`.
|
||||
|
||||
The digest freezes image layers, but `apt-get update` and `apk add` still consume mutable package
|
||||
repositories during a no-cache rebuild. Full OS-package immutability would require Debian/Alpine
|
||||
snapshot repositories and is not claimed by this deployment.
|
||||
|
||||
@@ -1,6 +1,24 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
load_secret() {
|
||||
value_name=$1
|
||||
file_name=$2
|
||||
secret_file=$(printenv "$file_name" 2>/dev/null || true)
|
||||
if [ -n "$secret_file" ]; then
|
||||
if [ ! -r "$secret_file" ]; then
|
||||
echo "$file_name is not readable: $secret_file" >&2
|
||||
exit 2
|
||||
fi
|
||||
secret_value=$(cat "$secret_file")
|
||||
export "$value_name=$secret_value"
|
||||
fi
|
||||
}
|
||||
|
||||
load_secret THT_DWH_API_KEY THT_DWH_API_KEY_FILE
|
||||
load_secret THT_VEC_API_KEY THT_VEC_API_KEY_FILE
|
||||
load_secret THT_VEC_WRITE_API_KEY THT_VEC_WRITE_API_KEY_FILE
|
||||
|
||||
case "${1:-server}" in
|
||||
server)
|
||||
shift || true
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
FROM node:22.19.0-bookworm-slim AS node-runtime
|
||||
FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS node-runtime
|
||||
WORKDIR /opt/pi-runtime
|
||||
COPY docker/pi-runtime/package.json docker/pi-runtime/package-lock.json ./
|
||||
RUN npm ci --omit=dev --ignore-scripts --no-audit --no-fund \
|
||||
&& ./node_modules/.bin/pi --version
|
||||
|
||||
FROM node:22.19.0-bookworm-slim AS backend-build
|
||||
FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS backend-build
|
||||
WORKDIR /src/backend
|
||||
COPY backend/package.json backend/package-lock.json ./
|
||||
RUN npm ci --no-audit --no-fund
|
||||
@@ -13,12 +13,12 @@ COPY backend/ ./
|
||||
RUN npm run build \
|
||||
&& npm prune --omit=dev
|
||||
|
||||
FROM node:22.19.0-bookworm-slim AS gate-deps
|
||||
FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS gate-deps
|
||||
WORKDIR /src/harness
|
||||
COPY harness/package.json harness/package-lock.json ./
|
||||
RUN npm ci --no-audit --no-fund
|
||||
|
||||
FROM python:3.12-slim-bookworm AS runtime
|
||||
FROM python:3.12.11-slim-bookworm@sha256:519591d6871b7bc437060736b9f7456b8731f1499a57e22e6c285135ae657bf7 AS runtime
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install --yes --no-install-recommends ca-certificates curl \
|
||||
|
||||
@@ -1,7 +1,20 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
backend_base_url=${BACKEND_BASE_URL:-/api}
|
||||
backend_base_url=${BACKEND_BASE_URL-/api}
|
||||
case "$backend_base_url" in
|
||||
""|/|/api|/api/) ;;
|
||||
http://*|https://*)
|
||||
if printf '%s' "$backend_base_url" | grep -Eq '[[:space:]]|^https?://[^/]*@'; then
|
||||
echo "Invalid BACKEND_BASE_URL: credentials and whitespace are not allowed" >&2
|
||||
exit 2
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "Invalid BACKEND_BASE_URL: use empty/root, /api, or an absolute http(s) URL" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
runtime_config=$(jq -cn --arg backend_base_url "$backend_base_url" \
|
||||
'{backendBaseUrl: $backend_base_url}')
|
||||
printf 'window.__THOTHII_CONFIG__ = %s;\n' "$runtime_config" \
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
FROM node:22.19.0-bookworm-slim AS build
|
||||
FROM node:22.19.0-bookworm-slim@sha256:4a4884e8a44826194dff92ba316264f392056cbe243dcc9fd3551e71cea02b90 AS build
|
||||
WORKDIR /src/frontend
|
||||
COPY frontend/package.json frontend/package-lock.json ./
|
||||
RUN npm ci --no-audit --no-fund
|
||||
COPY frontend/ ./
|
||||
RUN npm run build
|
||||
|
||||
FROM nginxinc/nginx-unprivileged:1.27-alpine
|
||||
FROM nginxinc/nginx-unprivileged:1.27.5-alpine@sha256:65e3e85dbaed8ba248841d9d58a899b6197106c23cb0ff1a132b7bfe0547e4c0
|
||||
USER root
|
||||
RUN apk add --no-cache jq
|
||||
COPY --from=build /src/frontend/dist /usr/share/nginx/html
|
||||
|
||||
@@ -8,6 +8,13 @@ server {
|
||||
try_files $uri =404;
|
||||
}
|
||||
|
||||
location = /health {
|
||||
proxy_pass http://core:8787/health;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_cache off;
|
||||
}
|
||||
|
||||
location /api/ {
|
||||
proxy_pass http://core:8787/;
|
||||
proxy_http_version 1.1;
|
||||
@@ -15,6 +22,9 @@ server {
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
# Trusted only when AUTH_MODE=upstream and this frontend port is reachable solely
|
||||
# from the authenticated host proxy documented in deploy/.
|
||||
proxy_set_header X-Authenticated-User $http_x_authenticated_user;
|
||||
proxy_buffering off;
|
||||
proxy_cache off;
|
||||
proxy_read_timeout 1h;
|
||||
|
||||
@@ -7,7 +7,7 @@ assignment=$(sed \
|
||||
/usr/share/nginx/html/config.js)
|
||||
|
||||
printf '%s\n' "$assignment" \
|
||||
| jq -e --arg expected "${BACKEND_BASE_URL:-/api}" \
|
||||
| jq -e --arg expected "${BACKEND_BASE_URL-/api}" \
|
||||
'type == "object" and keys == ["backendBaseUrl"] and .backendBaseUrl == $expected' \
|
||||
>/dev/null
|
||||
|
||||
|
||||
@@ -236,6 +236,13 @@ git commit -m "build(docker): add runtime-configured frontend image"
|
||||
|
||||
### Task 5: Compose external profile and end-to-end smoke gate
|
||||
|
||||
> **Final-review security amendment (2026-07-12):** the frontend port binds to `127.0.0.1` by
|
||||
> default. Public deployment uses an authenticated upstream proxy with `AUTH_MODE=upstream`;
|
||||
> `THOTH_PUBLIC_EXPOSURE=true` plus `AUTH_MODE=none` is invalid. Local env files are development
|
||||
> only; production uses read-only Compose secrets. Image gates pin exact tags and multi-platform
|
||||
> digests and verify both linux/amd64 and linux/arm64 using the shared container verification
|
||||
> script.
|
||||
|
||||
**Files:**
|
||||
- Create: `compose.yaml`
|
||||
- Create: `deploy/env.example`
|
||||
|
||||
@@ -36,6 +36,7 @@ Non sono presenti Dockerfile o file Compose. Il processo di sviluppo assume Pi e
|
||||
6. **Configurazione dichiarativa e validata.** I workspace contengono riferimenti logici e configurazioni non segrete; i segreti sono in environment variables o secret store.
|
||||
7. **Capability esplicite.** Un adapter dichiara ciò che supporta. Le funzioni mancanti producono degradazione o blocco comprensibile, non emulazioni implicite.
|
||||
8. **Read-only by construction sul DWH.** Credenziali, API e guard client-side mantengono la separazione dall'autorità di scrittura.
|
||||
9. **Esposizione sicura per default.** La porta applicativa pubblicata è vincolata a loopback. Un'esposizione pubblica richiede un reverse proxy autenticante esterno e `AUTH_MODE=upstream`; la combinazione pubblico + `none` viene rifiutata all'avvio. OIDC interno non fa parte di questa fase.
|
||||
|
||||
## 4. Packaging e runtime
|
||||
|
||||
@@ -143,6 +144,11 @@ La configurazione si divide in:
|
||||
- **workspace:** lingua, adapter, namespace, collezioni e policy di preprocessing;
|
||||
- **segreti:** password, token, certificati e chiavi reader/writer.
|
||||
|
||||
Nel profilo locale i segreti possono provenire da un env-file non versionato. In produzione sono
|
||||
file read-only sotto `/run/secrets`, leggibili dall'UID 10001. Il reverse proxy autenticante è un
|
||||
confine fidato: rimuove header identità forniti dal client e inserisce
|
||||
`X-Authenticated-User` soltanto dopo autenticazione.
|
||||
|
||||
Deve esistere un comando di diagnostica che produca sia output umano sia JSON pristino, rispettando il contratto CLI corrente.
|
||||
|
||||
## 7. Pipeline di preprocessing
|
||||
|
||||
@@ -14,6 +14,20 @@ describe("resolveBackendUrl", () => {
|
||||
it("preserves the client default when Vite has no configured backend", () => {
|
||||
expect(backendBaseUrl).toBe(import.meta.env.VITE_BACKEND_URL ?? "http://localhost:8787");
|
||||
});
|
||||
|
||||
it.each(["/backend", "api", "//evil.test", "ftp://example.test", "https://user:pass@example.test"])(
|
||||
"rejects unsupported backend URL %j",
|
||||
(backendBaseUrl) => {
|
||||
expect(() => resolveBackendUrl({ backendBaseUrl })).toThrow(/BACKEND_BASE_URL/);
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["", "/", "/api", "/api/", "http://localhost:8787", "https://api.example.test/v1"])(
|
||||
"accepts supported backend URL %j",
|
||||
(backendBaseUrl) => {
|
||||
expect(resolveBackendUrl({ backendBaseUrl })).toBe(backendBaseUrl);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("joinBackendPath", () => {
|
||||
|
||||
@@ -9,7 +9,19 @@ declare global {
|
||||
}
|
||||
|
||||
export function resolveBackendUrl(config: RuntimeConfig | undefined): string {
|
||||
return config?.backendBaseUrl ?? import.meta.env.VITE_BACKEND_URL ?? "";
|
||||
const value = config?.backendBaseUrl ?? import.meta.env.VITE_BACKEND_URL ?? "";
|
||||
if (value === "" || value === "/" || value === "/api" || value === "/api/") return value;
|
||||
try {
|
||||
const url = new URL(value);
|
||||
if ((url.protocol === "http:" || url.protocol === "https:") && !url.username && !url.password) {
|
||||
return value;
|
||||
}
|
||||
} catch {
|
||||
// Fall through to the single actionable runtime error below.
|
||||
}
|
||||
throw new Error(
|
||||
"Invalid BACKEND_BASE_URL: use empty/root, /api, or an absolute http(s) URL without credentials",
|
||||
);
|
||||
}
|
||||
|
||||
export function joinBackendPath(base: string, path: string): string {
|
||||
|
||||
+19
-2
@@ -43,7 +43,24 @@ compose up --build --wait core frontend
|
||||
published=$(compose port frontend 8080)
|
||||
http_port=${published##*:}
|
||||
|
||||
curl --fail --silent --show-error "http://127.0.0.1:$http_port/health" >/dev/null
|
||||
assert_health() {
|
||||
health_headers=$(mktemp)
|
||||
health_body=$(mktemp)
|
||||
if ! curl --fail --silent --show-error --dump-header "$health_headers" \
|
||||
"http://127.0.0.1:$http_port/health" >"$health_body"; then
|
||||
rm -f "$health_headers" "$health_body"
|
||||
return 1
|
||||
fi
|
||||
if ! grep -qi '^content-type: application/json' "$health_headers" ||
|
||||
! jq -e 'type == "object" and keys == ["status"] and .status == "ok"' \
|
||||
"$health_body" >/dev/null; then
|
||||
rm -f "$health_headers" "$health_body"
|
||||
return 1
|
||||
fi
|
||||
rm -f "$health_headers" "$health_body"
|
||||
}
|
||||
|
||||
assert_health
|
||||
|
||||
# The nginx proxy must preserve streaming semantics for the backend SSE endpoint.
|
||||
headers=$(mktemp)
|
||||
@@ -68,5 +85,5 @@ persisted=$(compose exec -T core sh -c 'cat /data/.compose-smoke-marker')
|
||||
[ "$persisted" = "$marker" ]
|
||||
compose exec -T core rm -f /data/.compose-smoke-marker
|
||||
|
||||
curl --fail --silent --show-error "http://127.0.0.1:$http_port/health" >/dev/null
|
||||
assert_health
|
||||
echo "Compose health, SSE proxy, and restart persistence checks passed."
|
||||
|
||||
Executable
+42
@@ -0,0 +1,42 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT HUP INT TERM
|
||||
|
||||
docker compose --profile external config >"$tmp/base.yaml"
|
||||
grep -q 'host_ip: 127.0.0.1' "$tmp/base.yaml"
|
||||
grep -q 'AUTH_MODE: none' "$tmp/base.yaml"
|
||||
grep -q 'THOTH_PUBLIC_EXPOSURE: "false"' "$tmp/base.yaml"
|
||||
if grep -q 'env_file:' "$tmp/base.yaml"; then
|
||||
echo "base/production-neutral Compose must not load the local env file" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
docker compose -f compose.yaml -f deploy/compose.local.yaml \
|
||||
--profile external config >"$tmp/local.yaml"
|
||||
grep -q 'env_file:' deploy/compose.local.yaml
|
||||
|
||||
for secret in dwh reader writer ca; do printf '%s\n' "test-$secret" >"$tmp/$secret"; done
|
||||
THT_DWH_API_KEY_SECRET_FILE="$tmp/dwh" \
|
||||
THT_VEC_API_KEY_SECRET_FILE="$tmp/reader" \
|
||||
THT_VEC_WRITE_API_KEY_SECRET_FILE="$tmp/writer" \
|
||||
THT_CA_SECRET_FILE="$tmp/ca" \
|
||||
THT_DB_NAME=test THT_DWH_REST_URL=https://dwh.example.test \
|
||||
THT_VEC_REST_URL=https://vector.example.test THT_OLLAMA_URL=https://embed.example.test \
|
||||
docker compose -f compose.yaml -f deploy/compose.production.yaml \
|
||||
--profile external config >"$tmp/production.yaml"
|
||||
grep -q 'AUTH_MODE: upstream' "$tmp/production.yaml"
|
||||
grep -q 'THOTH_PUBLIC_EXPOSURE: "true"' "$tmp/production.yaml"
|
||||
grep -q 'target: thoth_ca.pem' "$tmp/production.yaml"
|
||||
grep -q 'THT_DWH_API_KEY_FILE: /run/secrets/dwh_api_key' "$tmp/production.yaml"
|
||||
|
||||
if awk '/^FROM / && $2 !~ /@sha256:/ { found=1 } END { exit !found }' \
|
||||
docker/core.Dockerfile docker/frontend.Dockerfile; then
|
||||
echo "every Dockerfile base must include an immutable digest" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "container deployment security contract passed."
|
||||
@@ -3,17 +3,84 @@ set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
script=scripts/docker-smoke.sh
|
||||
sh -n "$script"
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT HUP INT TERM
|
||||
log="$tmp/docker.log"
|
||||
marker_file="$tmp/marker"
|
||||
|
||||
grep -q 'SMOKE_PROJECT' "$script"
|
||||
grep -q -- '--project-name' "$script"
|
||||
grep -q 'KEEP_SMOKE_RESOURCES' "$script"
|
||||
grep -q 'down --volumes' "$script"
|
||||
mkdir -p "$tmp/bin"
|
||||
cat >"$tmp/bin/docker" <<'EOF'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
printf '%s\n' "$*" >>"$FAKE_DOCKER_LOG"
|
||||
|
||||
if grep -q -- 'down --remove-orphans' "$script"; then
|
||||
case " $* " in
|
||||
*" port frontend 8080 "*) printf '%s\n' '127.0.0.1:49152' ;;
|
||||
*" exec -T core sh -c "*"printf"*)
|
||||
for last do :; done
|
||||
printf '%s\n' "$last" >"$FAKE_MARKER_FILE"
|
||||
;;
|
||||
*" exec -T core sh -c "*"cat /data/.compose-smoke-marker"*)
|
||||
cat "$FAKE_MARKER_FILE"
|
||||
;;
|
||||
esac
|
||||
EOF
|
||||
cat >"$tmp/bin/curl" <<'EOF'
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
header_file=""
|
||||
for arg do
|
||||
if [ "${previous:-}" = "--dump-header" ]; then header_file=$arg; fi
|
||||
previous=$arg
|
||||
done
|
||||
if [ -n "$header_file" ]; then
|
||||
case "$*" in
|
||||
*"/events"*) printf 'HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nCache-Control: no-cache\r\n\r\n' >"$header_file" ;;
|
||||
*) printf 'HTTP/1.1 200 OK\r\nContent-Type: application/json; charset=utf-8\r\n\r\n' >"$header_file" ;;
|
||||
esac
|
||||
fi
|
||||
case "$*" in
|
||||
*"/health"*) printf '%s\n' '{"status":"ok"}' ;;
|
||||
esac
|
||||
EOF
|
||||
chmod +x "$tmp/bin/docker" "$tmp/bin/curl"
|
||||
|
||||
run_smoke() {
|
||||
PATH="$tmp/bin:$PATH" \
|
||||
FAKE_DOCKER_LOG="$log" \
|
||||
FAKE_MARKER_FILE="$marker_file" \
|
||||
SMOKE_PROJECT="$1" \
|
||||
KEEP_SMOKE_RESOURCES="${2:-0}" \
|
||||
./scripts/docker-smoke.sh
|
||||
}
|
||||
|
||||
run_smoke thothii-smoke-dynamic
|
||||
|
||||
while IFS= read -r invocation; do
|
||||
case "$invocation" in
|
||||
"compose --project-name thothii-smoke-dynamic --profile external "*) ;;
|
||||
*) echo "Compose invocation escaped the smoke project/profile: $invocation" >&2; exit 1 ;;
|
||||
esac
|
||||
done <"$log"
|
||||
grep -q ' down --volumes$' "$log"
|
||||
if grep -q -- '--remove-orphans' "$log"; then
|
||||
echo "smoke cleanup must not remove operator orphans" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "docker-smoke isolation contract passed."
|
||||
: >"$log"
|
||||
run_smoke thothii-smoke-kept 1
|
||||
if grep -q ' down ' "$log"; then
|
||||
echo "KEEP_SMOKE_RESOURCES=1 unexpectedly cleaned the project" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
: >"$log"
|
||||
if PATH="$tmp/bin:$PATH" FAKE_DOCKER_LOG="$log" FAKE_MARKER_FILE="$marker_file" \
|
||||
SMOKE_PROJECT=thothii ./scripts/docker-smoke.sh >/dev/null 2>&1; then
|
||||
echo "reserved operator project was accepted" >&2
|
||||
exit 1
|
||||
fi
|
||||
test ! -s "$log"
|
||||
|
||||
echo "docker-smoke dynamic isolation contract passed."
|
||||
|
||||
Executable
+50
@@ -0,0 +1,50 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
platform=${PLATFORM:-linux/arm64}
|
||||
slug=$(printf '%s' "$platform" | tr '/:' '--')
|
||||
core_image="thothii-core:verify-$slug"
|
||||
frontend_image="thothii-frontend:verify-$slug"
|
||||
inventory_dir=${CONTAINER_INVENTORY_DIR:-.artifacts/container-images/$slug}
|
||||
|
||||
mkdir -p "$inventory_dir"
|
||||
|
||||
docker buildx build --platform "$platform" --load \
|
||||
-f docker/core.Dockerfile -t "$core_image" .
|
||||
docker buildx build --platform "$platform" --load \
|
||||
-f docker/frontend.Dockerfile -t "$frontend_image" .
|
||||
|
||||
docker run --rm --platform "$platform" --entrypoint /app/docker/smoke/core-smoke.sh \
|
||||
"$core_image"
|
||||
docker run --rm --platform "$platform" -e BACKEND_BASE_URL=/api \
|
||||
"$frontend_image" frontend-config-smoke
|
||||
docker run --rm --platform "$platform" -e BACKEND_BASE_URL= \
|
||||
"$frontend_image" frontend-config-smoke
|
||||
|
||||
if docker run --rm --platform "$platform" -e BACKEND_BASE_URL=/backend \
|
||||
"$frontend_image" frontend-config-smoke >/dev/null 2>&1; then
|
||||
echo "frontend accepted an unsupported BACKEND_BASE_URL" >&2
|
||||
exit 1
|
||||
fi
|
||||
if docker run --rm --platform "$platform" -e THOTH_PUBLIC_EXPOSURE=true -e AUTH_MODE=none \
|
||||
"$core_image" server >/dev/null 2>&1; then
|
||||
echo "core accepted public exposure without upstream authentication" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
docker image inspect "$core_image" >"$inventory_dir/core-image-inspect.json"
|
||||
docker image inspect "$frontend_image" >"$inventory_dir/frontend-image-inspect.json"
|
||||
docker run --rm --platform "$platform" --entrypoint sh "$core_image" -c \
|
||||
'dpkg-query -W; /opt/venv/bin/pip freeze; /opt/venv/bin/python -c '"'"'import glob,json; rows=set();
|
||||
for path in glob.glob("/app/backend/node_modules/**/package.json", recursive=True):
|
||||
try:
|
||||
package=json.load(open(path)); rows.add((package.get("name","?"), package.get("version","?")))
|
||||
except (OSError, ValueError): pass
|
||||
print("\n".join(f"{name}=={version}" for name,version in sorted(rows)))'"'"'' \
|
||||
>"$inventory_dir/core-packages.txt"
|
||||
docker run --rm --platform "$platform" --entrypoint sh "$frontend_image" -c 'apk info -vv' \
|
||||
>"$inventory_dir/frontend-packages.txt"
|
||||
|
||||
echo "container verification and inventory complete for $platform"
|
||||
Reference in New Issue
Block a user