docs(deploy): document user-owned session cutover
This commit is contained in:
@@ -0,0 +1,68 @@
|
|||||||
|
# Task 7 report — deployment contract and user-owned-session cutover
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
Implemented the deployment contract only. No Supabase migration, portal change, live-stack
|
||||||
|
restart, session archive, or deletion was run.
|
||||||
|
|
||||||
|
- `backend/src/config.ts` now makes the session-store deployment mode explicit. `local` is the
|
||||||
|
default and cannot be publicly exposed. `postgres` requires `AUTH_MODE=upstream`, direct DB
|
||||||
|
host/name/runtime user, an absolute runtime-password file, `verify-ca` or `verify-full`, and an
|
||||||
|
absolute CA path.
|
||||||
|
- `docker-compose.dev.yml` now publishes only loopback ports and explicitly selects local
|
||||||
|
session storage rooted at `/data/local-home`.
|
||||||
|
- `deploy/compose.session-server.yaml.example` separates the runtime and one-shot migrator
|
||||||
|
secrets. The core gets only `session_runtime_password` and the CA; the profile-gated
|
||||||
|
`session-migrate` service gets only `session_migrator_password` and the CA.
|
||||||
|
- `deploy/workspaces/server-sessions.yaml.example` binds the runtime repository to the
|
||||||
|
TLS-verified direct PostgreSQL configuration. The runtime password remains a file reference.
|
||||||
|
- `docker/cutover-legacy-sessions.sh` archives/checksums exactly three reviewed legacy sessions
|
||||||
|
and requires an explicit `--delete` rerun before deleting them.
|
||||||
|
- README, secret guidance, environment examples, and PROJECT_STATE describe the maintenance
|
||||||
|
sequence, Task 4+5 coordinated rollout, liveness vs storage 503 behavior, and the no-dual-write
|
||||||
|
rollback rule.
|
||||||
|
|
||||||
|
## TDD evidence
|
||||||
|
|
||||||
|
RED was established with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd backend && npx vitest run test/config.test.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
The new tests failed because `sessionStorage` did not exist and public/local and unauthenticated
|
||||||
|
server combinations were accepted. After implementing the minimal configuration contract, the
|
||||||
|
same focused suite passed (7 tests). Updating the existing upstream-health fixture to supply the
|
||||||
|
now-required server inputs confirmed that `/health` remains an unauthenticated `200` liveness
|
||||||
|
endpoint under the valid server contract.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```text
|
||||||
|
cd harness && .venv/bin/pytest -q
|
||||||
|
826 passed, 5 deselected, 67 warnings in 63.01s
|
||||||
|
|
||||||
|
cd backend && npx vitest run && npx tsc --noEmit -p . && npm run build
|
||||||
|
22 files / 215 tests passed; TypeScript check and production build passed
|
||||||
|
|
||||||
|
cd frontend && npx vitest run && npx tsc -b && npm run build
|
||||||
|
full Vitest suite, TypeScript build, and Vite production build passed
|
||||||
|
```
|
||||||
|
|
||||||
|
The frontend gate retained its pre-existing React-ref/MSW/act warnings and Vite chunk-size warning;
|
||||||
|
none caused a test or build failure.
|
||||||
|
|
||||||
|
Additional static validation passed:
|
||||||
|
|
||||||
|
```text
|
||||||
|
docker compose config --quiet (base plus copied session-server overlay with temporary empty secrets)
|
||||||
|
bash -n docker/cutover-legacy-sessions.sh
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
## Manual gate remaining
|
||||||
|
|
||||||
|
An operator must still choose the three reviewed legacy IDs, materialize real runtime/migrator/CA
|
||||||
|
secrets, deploy Task 4 and Task 5 together in a maintenance window, apply the one-shot migrator,
|
||||||
|
and run the documented authenticated smoke. The guarded helper has not been invoked with
|
||||||
|
`--delete`.
|
||||||
@@ -3,6 +3,28 @@
|
|||||||
> Starting-point snapshot for new sessions. Last updated: 2026-07-15 (central live log and compact CTE density live).
|
> Starting-point snapshot for new sessions. Last updated: 2026-07-15 (central live log and compact CTE density live).
|
||||||
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
||||||
|
|
||||||
|
## User-owned sessions cutover — prepared, manual gate pending (2026-07-16)
|
||||||
|
|
||||||
|
- **Target contract:** the public server runs `AUTH_MODE=upstream` with Task 4 portal identity
|
||||||
|
forwarding and Task 5 principal enforcement deployed together. Its session source of truth is
|
||||||
|
direct TLS-verified PostgreSQL `thoth_sessions`; local development remains loopback-only with
|
||||||
|
filesystem sessions under `THT_HOME`. The core never receives the migrator credential.
|
||||||
|
- **Deployment material:** copy `deploy/compose.session-server.yaml.example` and
|
||||||
|
`deploy/workspaces/server-sessions.yaml.example` into reviewed, untracked operator files. The
|
||||||
|
runtime password, migrator password, and CA are three separate Docker secret mounts; server
|
||||||
|
startup rejects public/local storage and incomplete server DB/TLS configuration.
|
||||||
|
- **Readiness behavior:** `/health` remains the unauthenticated process liveness endpoint. Any
|
||||||
|
route requiring unavailable session/preferences storage returns fixed HTTP 503 before starting
|
||||||
|
Pi; this is intentional and must not be hidden by changing liveness to a database check.
|
||||||
|
- **Manual cutover only:** schedule maintenance, drain Pi work, run the one-shot migrator and
|
||||||
|
require `pending=[]` and `drifted=[]`, then replace core and perform an authenticated storage
|
||||||
|
smoke. Archive/checksum the three reviewed legacy filesystem session directories before deleting
|
||||||
|
exactly those three with `docker/cutover-legacy-sessions.sh --delete`; no deletion has been run
|
||||||
|
from this repository task. Do not import their untrusted ownership.
|
||||||
|
- **Rollback:** PostgreSQL remains the single source of truth. Revert only to a compatible fixed
|
||||||
|
release; never re-enable filesystem persistence, restore the archive into production, or
|
||||||
|
dual-write during rollback.
|
||||||
|
|
||||||
## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12
|
## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12
|
||||||
|
|
||||||
ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale).
|
ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale).
|
||||||
|
|||||||
@@ -60,6 +60,11 @@ The frontend depends on the core health check and proxies `/health` and `/api/*`
|
|||||||
application health endpoint intentionally checks process readiness only; external dependency
|
application health endpoint intentionally checks process readiness only; external dependency
|
||||||
diagnostics are exposed by `tht doctor` and do not prevent the UI from starting.
|
diagnostics are exposed by `tht doctor` and do not prevent the UI from starting.
|
||||||
|
|
||||||
|
`docker-compose.dev.yml` is deliberately local: both published ports bind to `127.0.0.1`,
|
||||||
|
`THT_SESSION_STORAGE=local`, and `THT_HOME=/data/local-home`. Do not set
|
||||||
|
`THOTH_PUBLIC_EXPOSURE=true` for that profile; the backend rejects that public/local combination
|
||||||
|
at startup.
|
||||||
|
|
||||||
Run the end-to-end packaging check with:
|
Run the end-to-end packaging check with:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
@@ -192,6 +197,76 @@ Compound providers are deliberately unsupported: `amazon-bedrock`, `azure-openai
|
|||||||
values. Selecting one fails before Pi starts; ambient AWS, Azure, and Cloudflare credentials are
|
values. Selecting one fails before Pi starts; ambient AWS, Azure, and Cloudflare credentials are
|
||||||
still scrubbed. Supporting them requires a future dedicated provider-specific configuration.
|
still scrubbed. Supporting them requires a future dedicated provider-specific configuration.
|
||||||
|
|
||||||
|
## User-owned session server cutover
|
||||||
|
|
||||||
|
The server profile stores sessions and per-user preferences directly in PostgreSQL schema
|
||||||
|
`thoth_sessions`; it does not use PostgREST, browser storage, a shared session directory, or a
|
||||||
|
dual write. Start from [`deploy/compose.session-server.yaml.example`](deploy/compose.session-server.yaml.example)
|
||||||
|
and copy [`deploy/workspaces/server-sessions.yaml.example`](deploy/workspaces/server-sessions.yaml.example)
|
||||||
|
to the untracked `deploy/workspaces/server-sessions.yaml` mounted into the core container.
|
||||||
|
|
||||||
|
The runtime login needs membership in the no-login database role `thoth_sessions_runtime` only.
|
||||||
|
The distinct, one-shot migrator login needs migration authority and uses
|
||||||
|
`thoth_sessions_migrator`; it must never be mounted into `core`. Set the non-secret endpoint and
|
||||||
|
role fields in the protected deployment environment:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
AUTH_MODE=upstream
|
||||||
|
THOTH_PUBLIC_EXPOSURE=true
|
||||||
|
THT_SESSION_STORAGE=postgres
|
||||||
|
THT_SESSION_DB_HOST=sessions-db.internal
|
||||||
|
THT_SESSION_DB_PORT=5432
|
||||||
|
THT_SESSION_DB_NAME=thoth
|
||||||
|
THT_SESSION_RUNTIME_USER=thoth_sessions_app
|
||||||
|
THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate
|
||||||
|
THT_SESSION_DB_SSLMODE=verify-full
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_SOURCE=/secure/thoth/session-runtime-password
|
||||||
|
THT_SESSION_MIGRATOR_PASSWORD_SOURCE=/secure/thoth/session-migrator-password
|
||||||
|
THT_SESSION_CA_SOURCE=/secure/thoth/session-ca.pem
|
||||||
|
```
|
||||||
|
|
||||||
|
The overlay mounts the runtime password at `/run/secrets/session_runtime_password`, the CA at
|
||||||
|
`/run/secrets/session_ca.pem`, and passes those paths—not their contents—to the server workspace.
|
||||||
|
It mounts `session_migrator_password` only to `session-migrate`. The backend refuses a server
|
||||||
|
session store without upstream authentication, direct DB host/name/runtime user/password-file,
|
||||||
|
`verify-ca` or `verify-full`, and an absolute CA path.
|
||||||
|
|
||||||
|
Perform the cutover in one maintenance window, with the Task 4 portal proxy headers and Task 5
|
||||||
|
backend principal parser deployed together. Neither change is safe to deploy independently: Task
|
||||||
|
4 clears the legacy identity header and Task 5 rejects it. Drain/stop active Pi work, enable a
|
||||||
|
maintenance response at the portal, then run the migrator once and inspect its pristine JSON:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose -f compose.yaml -f deploy/compose.session-server.yaml \
|
||||||
|
--profile session-migrate run --rm session-migrate
|
||||||
|
```
|
||||||
|
|
||||||
|
It must report no pending or drifted migrations before starting the replacement core. `/health`
|
||||||
|
is a liveness probe and remains `200`; any request that needs unavailable repository storage
|
||||||
|
returns a fixed `503` before a Pi process starts. Verify this with an authenticated request after
|
||||||
|
the replacement core is healthy, then remove maintenance mode.
|
||||||
|
|
||||||
|
Do not import the three legacy server filesystem sessions: they have no trusted owner binding.
|
||||||
|
During the same maintenance window, archive the exact three reviewed IDs, verify the generated
|
||||||
|
archive and `.sha256`, then rerun the command with `--delete` to remove only those three source
|
||||||
|
directories:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./docker/cutover-legacy-sessions.sh \
|
||||||
|
/secure/thoth/legacy-sessions /secure/backups/thoth-legacy-sessions-2026-07-16.tar \
|
||||||
|
SESSION_ID_1 SESSION_ID_2 SESSION_ID_3
|
||||||
|
# After independent archive review, use a new backup filename:
|
||||||
|
./docker/cutover-legacy-sessions.sh --delete \
|
||||||
|
/secure/thoth/legacy-sessions /secure/backups/thoth-legacy-sessions-2026-07-16-delete.tar \
|
||||||
|
SESSION_ID_1 SESSION_ID_2 SESSION_ID_3
|
||||||
|
```
|
||||||
|
|
||||||
|
The helper refuses to overwrite an existing backup and refuses any count other than three
|
||||||
|
distinct IDs. Never run it against a live path without the maintenance gate. Roll back application
|
||||||
|
code only by keeping PostgreSQL as the single source of truth and deploying a compatible fixed
|
||||||
|
release. Do not restore filesystem persistence, do not re-import the archive, and never dual-write
|
||||||
|
sessions to database and files.
|
||||||
|
|
||||||
## Reproducible image verification
|
## Reproducible image verification
|
||||||
|
|
||||||
Base images use exact tags and immutable multi-platform manifest digests. Dependency update and
|
Base images use exact tags and immutable multi-platform manifest digests. Dependency update and
|
||||||
|
|||||||
@@ -3,6 +3,11 @@ import path from "node:path";
|
|||||||
export interface AppConfig {
|
export interface AppConfig {
|
||||||
host: string; port: number; harnessDir: string; thtBin: string; piBin: string;
|
host: string; port: number; harnessDir: string; thtBin: string; piBin: string;
|
||||||
authMode: "none" | "mock" | "upstream";
|
authMode: "none" | "mock" | "upstream";
|
||||||
|
sessionStorage: {
|
||||||
|
mode: "local" | "postgres";
|
||||||
|
host?: string; port?: number; database?: string; runtimeUser?: string;
|
||||||
|
runtimePasswordFile?: string; sslmode?: "verify-ca" | "verify-full"; sslrootcert?: string;
|
||||||
|
};
|
||||||
defaults: { provider?: string; model?: string; thinking?: string };
|
defaults: { provider?: string; model?: string; thinking?: string };
|
||||||
maxPiProcesses: number;
|
maxPiProcesses: number;
|
||||||
settingsFile: string;
|
settingsFile: string;
|
||||||
@@ -20,6 +25,43 @@ export function loadConfig(env: Record<string, string | undefined>): AppConfig {
|
|||||||
if (env.THOTH_PUBLIC_EXPOSURE === "true" && authMode !== "upstream") {
|
if (env.THOTH_PUBLIC_EXPOSURE === "true" && authMode !== "upstream") {
|
||||||
throw new Error("public exposure requires AUTH_MODE=upstream behind a trusted proxy");
|
throw new Error("public exposure requires AUTH_MODE=upstream behind a trusted proxy");
|
||||||
}
|
}
|
||||||
|
const sessionStorageMode = env.THT_SESSION_STORAGE ?? "local";
|
||||||
|
if (sessionStorageMode !== "local" && sessionStorageMode !== "postgres") {
|
||||||
|
throw new Error("session storage configuration is invalid");
|
||||||
|
}
|
||||||
|
if (sessionStorageMode === "local" && env.THOTH_PUBLIC_EXPOSURE === "true") {
|
||||||
|
throw new Error("local session storage requires loopback-only deployment");
|
||||||
|
}
|
||||||
|
const sessionStorage: AppConfig["sessionStorage"] = { mode: sessionStorageMode };
|
||||||
|
if (sessionStorageMode === "postgres") {
|
||||||
|
const host = env.THT_SESSION_DB_HOST;
|
||||||
|
const database = env.THT_SESSION_DB_NAME;
|
||||||
|
const runtimeUser = env.THT_SESSION_RUNTIME_USER;
|
||||||
|
const runtimePasswordFile = env.THT_SESSION_RUNTIME_PASSWORD_FILE;
|
||||||
|
const sslmode = env.THT_SESSION_DB_SSLMODE;
|
||||||
|
const sslrootcert = env.THT_SESSION_DB_SSLROOTCERT;
|
||||||
|
const port = Number(env.THT_SESSION_DB_PORT ?? 5432);
|
||||||
|
if (
|
||||||
|
authMode !== "upstream"
|
||||||
|
|| !host || !database || !runtimeUser
|
||||||
|
|| !runtimePasswordFile || !path.isAbsolute(runtimePasswordFile)
|
||||||
|
|| (sslmode !== "verify-ca" && sslmode !== "verify-full")
|
||||||
|
|| !sslrootcert || !path.isAbsolute(sslrootcert)
|
||||||
|
|| !Number.isInteger(port) || port < 1 || port > 65535
|
||||||
|
) {
|
||||||
|
if (authMode !== "upstream") {
|
||||||
|
throw new Error("server session storage requires AUTH_MODE=upstream");
|
||||||
|
}
|
||||||
|
throw new Error("server session storage configuration is invalid");
|
||||||
|
}
|
||||||
|
sessionStorage.host = host;
|
||||||
|
sessionStorage.port = port;
|
||||||
|
sessionStorage.database = database;
|
||||||
|
sessionStorage.runtimeUser = runtimeUser;
|
||||||
|
sessionStorage.runtimePasswordFile = runtimePasswordFile;
|
||||||
|
sessionStorage.sslmode = sslmode;
|
||||||
|
sessionStorage.sslrootcert = sslrootcert;
|
||||||
|
}
|
||||||
const modelApiKeyFile = env.THT_MODEL_API_KEY_FILE;
|
const modelApiKeyFile = env.THT_MODEL_API_KEY_FILE;
|
||||||
if (modelApiKeyFile !== undefined && (
|
if (modelApiKeyFile !== undefined && (
|
||||||
modelApiKeyFile.trim() !== modelApiKeyFile
|
modelApiKeyFile.trim() !== modelApiKeyFile
|
||||||
@@ -48,6 +90,7 @@ export function loadConfig(env: Record<string, string | undefined>): AppConfig {
|
|||||||
thtBin: env.THT_BIN ?? "tht",
|
thtBin: env.THT_BIN ?? "tht",
|
||||||
piBin: env.PI_BIN ?? "pi",
|
piBin: env.PI_BIN ?? "pi",
|
||||||
authMode: authMode as AppConfig["authMode"],
|
authMode: authMode as AppConfig["authMode"],
|
||||||
|
sessionStorage,
|
||||||
defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING },
|
defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING },
|
||||||
maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4),
|
maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4),
|
||||||
settingsFile: env.SETTINGS_FILE ?? "data/settings.json",
|
settingsFile: env.SETTINGS_FILE ?? "data/settings.json",
|
||||||
|
|||||||
@@ -44,9 +44,60 @@ test("loadConfig accepts an authenticated upstream trust boundary", () => {
|
|||||||
expect(loadConfig({
|
expect(loadConfig({
|
||||||
THOTH_PUBLIC_EXPOSURE: "true",
|
THOTH_PUBLIC_EXPOSURE: "true",
|
||||||
AUTH_MODE: "upstream",
|
AUTH_MODE: "upstream",
|
||||||
|
THT_SESSION_STORAGE: "postgres",
|
||||||
|
THT_SESSION_DB_HOST: "db.internal",
|
||||||
|
THT_SESSION_DB_NAME: "thoth",
|
||||||
|
THT_SESSION_RUNTIME_USER: "thoth_sessions_app",
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password",
|
||||||
|
THT_SESSION_DB_SSLMODE: "verify-full",
|
||||||
|
THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem",
|
||||||
}).authMode).toBe("upstream");
|
}).authMode).toBe("upstream");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("loadConfig requires direct PostgreSQL TLS inputs for the public server session store", () => {
|
||||||
|
const env = {
|
||||||
|
THOTH_PUBLIC_EXPOSURE: "true",
|
||||||
|
AUTH_MODE: "upstream",
|
||||||
|
THT_SESSION_STORAGE: "postgres",
|
||||||
|
THT_SESSION_DB_HOST: "db.internal",
|
||||||
|
THT_SESSION_DB_NAME: "thoth",
|
||||||
|
THT_SESSION_RUNTIME_USER: "thoth_sessions_app",
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password",
|
||||||
|
THT_SESSION_DB_SSLMODE: "verify-full",
|
||||||
|
THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem",
|
||||||
|
};
|
||||||
|
|
||||||
|
expect(loadConfig(env).sessionStorage).toMatchObject({
|
||||||
|
mode: "postgres",
|
||||||
|
host: "db.internal",
|
||||||
|
port: 5432,
|
||||||
|
database: "thoth",
|
||||||
|
runtimeUser: "thoth_sessions_app",
|
||||||
|
runtimePasswordFile: "/run/secrets/session_runtime_password",
|
||||||
|
sslmode: "verify-full",
|
||||||
|
sslrootcert: "/run/secrets/session_ca.pem",
|
||||||
|
});
|
||||||
|
for (const required of [
|
||||||
|
"THT_SESSION_DB_HOST", "THT_SESSION_DB_NAME", "THT_SESSION_RUNTIME_USER",
|
||||||
|
"THT_SESSION_RUNTIME_PASSWORD_FILE", "THT_SESSION_DB_SSLMODE", "THT_SESSION_DB_SSLROOTCERT",
|
||||||
|
]) {
|
||||||
|
const missing = { ...env, [required]: undefined };
|
||||||
|
expect(() => loadConfig(missing)).toThrow(/server session storage configuration is invalid/);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test("loadConfig rejects public local storage and server storage without upstream auth", () => {
|
||||||
|
expect(() => loadConfig({
|
||||||
|
THOTH_PUBLIC_EXPOSURE: "true",
|
||||||
|
AUTH_MODE: "upstream",
|
||||||
|
THT_SESSION_STORAGE: "local",
|
||||||
|
})).toThrow(/local session storage requires loopback-only deployment/);
|
||||||
|
expect(() => loadConfig({
|
||||||
|
THT_SESSION_STORAGE: "postgres",
|
||||||
|
AUTH_MODE: "none",
|
||||||
|
})).toThrow(/server session storage requires AUTH_MODE=upstream/);
|
||||||
|
});
|
||||||
|
|
||||||
test("loadConfig accepts only an absolute generic model key file", () => {
|
test("loadConfig accepts only an absolute generic model key file", () => {
|
||||||
expect(loadConfig({ THT_MODEL_API_KEY_FILE: "/run/secrets/model_api_key" }).modelApiKeyFile)
|
expect(loadConfig({ THT_MODEL_API_KEY_FILE: "/run/secrets/model_api_key" }).modelApiKeyFile)
|
||||||
.toBe("/run/secrets/model_api_key");
|
.toBe("/run/secrets/model_api_key");
|
||||||
|
|||||||
@@ -15,6 +15,13 @@ test("GET /health remains available to container probes in upstream auth mode",
|
|||||||
THT_HARNESS_DIR: "/tmp/h",
|
THT_HARNESS_DIR: "/tmp/h",
|
||||||
AUTH_MODE: "upstream",
|
AUTH_MODE: "upstream",
|
||||||
THOTH_PUBLIC_EXPOSURE: "true",
|
THOTH_PUBLIC_EXPOSURE: "true",
|
||||||
|
THT_SESSION_STORAGE: "postgres",
|
||||||
|
THT_SESSION_DB_HOST: "db.internal",
|
||||||
|
THT_SESSION_DB_NAME: "thoth",
|
||||||
|
THT_SESSION_RUNTIME_USER: "thoth_sessions_app",
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password",
|
||||||
|
THT_SESSION_DB_SSLMODE: "verify-full",
|
||||||
|
THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem",
|
||||||
}));
|
}));
|
||||||
const res = await app.inject({ method: "GET", url: "/health" });
|
const res = await app.inject({ method: "GET", url: "/health" });
|
||||||
expect(res.statusCode).toBe(200);
|
expect(res.statusCode).toBe(200);
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Opt-in server overlay for user-owned PostgreSQL sessions. Copy this file to a
|
||||||
|
# reviewed local override; it is intentionally not loaded by default Compose.
|
||||||
|
services:
|
||||||
|
core:
|
||||||
|
environment:
|
||||||
|
AUTH_MODE: upstream
|
||||||
|
THOTH_PUBLIC_EXPOSURE: "true"
|
||||||
|
THT_SESSION_STORAGE: postgres
|
||||||
|
THT_CONFIG: /app/harness/workspaces/server-sessions.yaml
|
||||||
|
THT_SESSION_DB_HOST: ${THT_SESSION_DB_HOST:?set THT_SESSION_DB_HOST}
|
||||||
|
THT_SESSION_DB_PORT: ${THT_SESSION_DB_PORT:-5432}
|
||||||
|
THT_SESSION_DB_NAME: ${THT_SESSION_DB_NAME:?set THT_SESSION_DB_NAME}
|
||||||
|
THT_SESSION_RUNTIME_USER: ${THT_SESSION_RUNTIME_USER:?set THT_SESSION_RUNTIME_USER}
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_FILE: /run/secrets/session_runtime_password
|
||||||
|
THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full}
|
||||||
|
THT_SESSION_DB_SSLROOTCERT: /run/secrets/session_ca.pem
|
||||||
|
secrets:
|
||||||
|
- source: session_runtime_password
|
||||||
|
target: session_runtime_password
|
||||||
|
- source: session_ca
|
||||||
|
target: session_ca.pem
|
||||||
|
volumes:
|
||||||
|
- ./deploy/workspaces:/app/harness/workspaces:ro
|
||||||
|
|
||||||
|
# Run manually during the maintenance window. It is not a dependency of core,
|
||||||
|
# so the application never gains the schema-changing migrator credential.
|
||||||
|
session-migrate:
|
||||||
|
image: thothii-core:local
|
||||||
|
profiles: [session-migrate]
|
||||||
|
entrypoint: [/bin/sh, -ec]
|
||||||
|
command: >-
|
||||||
|
export PGPASSWORD="$$(cat /run/secrets/session_migrator_password)";
|
||||||
|
exec /opt/venv/bin/tht session migrate --database-url
|
||||||
|
"postgresql+psycopg2://$${THT_SESSION_MIGRATOR_USER}@$${THT_SESSION_DB_HOST}:$${THT_SESSION_DB_PORT}/$${THT_SESSION_DB_NAME}?sslmode=$${THT_SESSION_DB_SSLMODE}&sslrootcert=/run/secrets/session_ca.pem"
|
||||||
|
--json
|
||||||
|
environment:
|
||||||
|
THT_SESSION_DB_HOST: ${THT_SESSION_DB_HOST:?set THT_SESSION_DB_HOST}
|
||||||
|
THT_SESSION_DB_PORT: ${THT_SESSION_DB_PORT:-5432}
|
||||||
|
THT_SESSION_DB_NAME: ${THT_SESSION_DB_NAME:?set THT_SESSION_DB_NAME}
|
||||||
|
THT_SESSION_MIGRATOR_USER: ${THT_SESSION_MIGRATOR_USER:?set THT_SESSION_MIGRATOR_USER}
|
||||||
|
THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full}
|
||||||
|
secrets:
|
||||||
|
- source: session_migrator_password
|
||||||
|
target: session_migrator_password
|
||||||
|
- source: session_ca
|
||||||
|
target: session_ca.pem
|
||||||
|
restart: "no"
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
session_runtime_password:
|
||||||
|
file: ${THT_SESSION_RUNTIME_PASSWORD_SOURCE:?set THT_SESSION_RUNTIME_PASSWORD_SOURCE}
|
||||||
|
session_migrator_password:
|
||||||
|
file: ${THT_SESSION_MIGRATOR_PASSWORD_SOURCE:?set THT_SESSION_MIGRATOR_PASSWORD_SOURCE}
|
||||||
|
session_ca:
|
||||||
|
file: ${THT_SESSION_CA_SOURCE:?set THT_SESSION_CA_SOURCE}
|
||||||
@@ -14,6 +14,19 @@ PI_THINKING=
|
|||||||
MAX_PI_PROCESSES=4
|
MAX_PI_PROCESSES=4
|
||||||
AUTH_MODE=none
|
AUTH_MODE=none
|
||||||
|
|
||||||
|
# User-owned session storage. Keep local for the loopback-only development stack.
|
||||||
|
# The server-session overlay requires every THT_SESSION_* value below.
|
||||||
|
THT_SESSION_STORAGE=local
|
||||||
|
THT_SESSION_DB_HOST=
|
||||||
|
THT_SESSION_DB_PORT=5432
|
||||||
|
THT_SESSION_DB_NAME=
|
||||||
|
THT_SESSION_RUNTIME_USER=
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_SOURCE=
|
||||||
|
THT_SESSION_MIGRATOR_USER=
|
||||||
|
THT_SESSION_MIGRATOR_PASSWORD_SOURCE=
|
||||||
|
THT_SESSION_DB_SSLMODE=verify-full
|
||||||
|
THT_SESSION_CA_SOURCE=
|
||||||
|
|
||||||
THT_DB_NAME=
|
THT_DB_NAME=
|
||||||
THT_DWH_REST_URL=
|
THT_DWH_REST_URL=
|
||||||
THT_VEC_REST_URL=
|
THT_VEC_REST_URL=
|
||||||
|
|||||||
@@ -43,3 +43,19 @@ password into `THT_VECTOR_BOOTSTRAP_PASSWORD` in the bundle before restarting
|
|||||||
Hosted Pi providers must use a single provider key. Compound providers (Bedrock, Azure OpenAI
|
Hosted Pi providers must use a single provider key. Compound providers (Bedrock, Azure OpenAI
|
||||||
Responses, Cloudflare Workers AI/Gateway) fail closed until a provider-specific credential
|
Responses, Cloudflare Workers AI/Gateway) fail closed until a provider-specific credential
|
||||||
adapter is implemented.
|
adapter is implemented.
|
||||||
|
|
||||||
|
## User-owned session database secrets
|
||||||
|
|
||||||
|
The server-session overlay deliberately does **not** add session database credentials to the
|
||||||
|
shared bundle. Materialize three distinct Docker secrets from protected host or secret-manager
|
||||||
|
files: `session_runtime_password`, `session_migrator_password`, and `session_ca.pem`. Their host
|
||||||
|
source paths are respectively `THT_SESSION_RUNTIME_PASSWORD_SOURCE`,
|
||||||
|
`THT_SESSION_MIGRATOR_PASSWORD_SOURCE`, and `THT_SESSION_CA_SOURCE`; all must be absolute paths
|
||||||
|
outside the repository. The runtime password is mounted only into `core`; the migrator password
|
||||||
|
is mounted only into the one-shot `session-migrate` service. Do not reuse either login for the
|
||||||
|
other role.
|
||||||
|
|
||||||
|
`session_ca.pem` is a PEM file rather than a bundle value because the bundle rejects whitespace.
|
||||||
|
The server workspace receives only its mount path through `THT_SESSION_DB_SSLROOTCERT`; it uses
|
||||||
|
`THT_SESSION_DB_SSLMODE=verify-ca` or, normally, `verify-full`. TLS disable/prefer/require modes
|
||||||
|
are unsupported for session storage.
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Copy to deploy/workspaces/server-sessions.yaml for the user-owned-session server profile.
|
||||||
|
# The runtime login is intentionally separate from the one-shot migrator login.
|
||||||
|
language: en
|
||||||
|
|
||||||
|
dwh:
|
||||||
|
type: thoth_rest
|
||||||
|
database:
|
||||||
|
database: ${THT_DB_NAME}
|
||||||
|
schema: datawarehouse
|
||||||
|
endpoint:
|
||||||
|
base_url: ${THT_DWH_REST_URL}
|
||||||
|
api_key: ${THT_DWH_API_KEY}
|
||||||
|
ssl_ca: ${THT_SSL_CA}
|
||||||
|
|
||||||
|
session_storage:
|
||||||
|
type: postgres_direct
|
||||||
|
connection:
|
||||||
|
host: ${THT_SESSION_DB_HOST}
|
||||||
|
port: ${THT_SESSION_DB_PORT}
|
||||||
|
database: ${THT_SESSION_DB_NAME}
|
||||||
|
schema: thoth_sessions
|
||||||
|
user: ${THT_SESSION_RUNTIME_USER}
|
||||||
|
password_file: ${THT_SESSION_RUNTIME_PASSWORD_FILE}
|
||||||
|
sslmode: ${THT_SESSION_DB_SSLMODE}
|
||||||
|
sslrootcert: ${THT_SESSION_DB_SSLROOTCERT}
|
||||||
|
|
||||||
|
roots:
|
||||||
|
artifacts: artifacts
|
||||||
|
indexes: indexes
|
||||||
|
sessions: sessions
|
||||||
|
|
||||||
|
embeddings:
|
||||||
|
base_url: ${THT_OLLAMA_URL}
|
||||||
|
model: nomic-embed-text-v2-moe
|
||||||
|
dim: 768
|
||||||
|
batch_size: 32
|
||||||
@@ -18,6 +18,8 @@ services:
|
|||||||
THT_BIN: /opt/venv/bin/tht
|
THT_BIN: /opt/venv/bin/tht
|
||||||
PI_BIN: pi
|
PI_BIN: pi
|
||||||
AUTH_MODE: ${AUTH_MODE:-none}
|
AUTH_MODE: ${AUTH_MODE:-none}
|
||||||
|
THT_SESSION_STORAGE: local
|
||||||
|
THT_HOME: /data/local-home
|
||||||
SETTINGS_FILE: /data/settings/settings.json
|
SETTINGS_FILE: /data/settings/settings.json
|
||||||
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
||||||
extra_hosts:
|
extra_hosts:
|
||||||
@@ -27,7 +29,7 @@ services:
|
|||||||
- /home/chirone/thothii-data/pi-config:/home/thoth/.pi
|
- /home/chirone/thothii-data/pi-config:/home/thoth/.pi
|
||||||
- /home/chirone/chirone/etl/docs/evidence:/data/evidence:ro
|
- /home/chirone/chirone/etl/docs/evidence:/data/evidence:ro
|
||||||
ports:
|
ports:
|
||||||
- "8787:8787"
|
- "127.0.0.1:8787:8787"
|
||||||
restart: "no"
|
restart: "no"
|
||||||
networks: [thothii-net]
|
networks: [thothii-net]
|
||||||
|
|
||||||
@@ -40,7 +42,7 @@ services:
|
|||||||
VITE_BACKEND_URL: /api
|
VITE_BACKEND_URL: /api
|
||||||
image: thothii-frontend:local
|
image: thothii-frontend:local
|
||||||
ports:
|
ports:
|
||||||
- "8090:8080"
|
- "127.0.0.1:8090:8080"
|
||||||
depends_on:
|
depends_on:
|
||||||
- core
|
- core
|
||||||
restart: "no"
|
restart: "no"
|
||||||
|
|||||||
Executable
+51
@@ -0,0 +1,51 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Archive exactly three legacy filesystem sessions before optionally deleting them.
|
||||||
|
# This helper is deliberately inert unless --delete is supplied after archive verification.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
echo "usage: $0 [--delete] SOURCE_SESSIONS_DIR BACKUP.tar SESSION_ID SESSION_ID SESSION_ID" >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
|
||||||
|
delete_after_backup=false
|
||||||
|
if [[ ${1:-} == "--delete" ]]; then
|
||||||
|
delete_after_backup=true
|
||||||
|
shift
|
||||||
|
fi
|
||||||
|
[[ $# -eq 5 ]] || usage
|
||||||
|
|
||||||
|
source_dir=$1
|
||||||
|
backup=$2
|
||||||
|
shift 2
|
||||||
|
ids=("$@")
|
||||||
|
|
||||||
|
[[ -d $source_dir ]] || { echo "legacy session root is not a directory" >&2; exit 1; }
|
||||||
|
[[ ! -e $backup ]] || { echo "backup already exists; refusing to overwrite it" >&2; exit 1; }
|
||||||
|
[[ ${ids[0]} != "${ids[1]}" && ${ids[0]} != "${ids[2]}" && ${ids[1]} != "${ids[2]}" ]] \
|
||||||
|
|| { echo "exactly three distinct legacy session IDs are required" >&2; exit 1; }
|
||||||
|
|
||||||
|
paths=()
|
||||||
|
for id in "${ids[@]}"; do
|
||||||
|
[[ $id =~ ^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$ ]] \
|
||||||
|
|| { echo "invalid legacy session ID: $id" >&2; exit 1; }
|
||||||
|
[[ -f "$source_dir/$id/session_manifest.yaml" ]] \
|
||||||
|
|| { echo "legacy session manifest is missing: $id" >&2; exit 1; }
|
||||||
|
paths+=("$id")
|
||||||
|
done
|
||||||
|
|
||||||
|
tar --create --file "$backup" --directory "$source_dir" -- "${paths[@]}"
|
||||||
|
tar --list --file "$backup" >/dev/null
|
||||||
|
sha256sum "$backup" >"$backup.sha256"
|
||||||
|
echo "verified backup: $backup"
|
||||||
|
echo "checksum: $backup.sha256"
|
||||||
|
|
||||||
|
if ! $delete_after_backup; then
|
||||||
|
echo "no legacy sessions deleted; review the archive, then rerun with --delete during maintenance" >&2
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
for id in "${ids[@]}"; do
|
||||||
|
rm -rf -- "$source_dir/$id"
|
||||||
|
done
|
||||||
|
echo "deleted exactly three archived legacy sessions"
|
||||||
Reference in New Issue
Block a user