# ThothII ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fastify/Pi/`tht` core. The portable deployment runs exactly two application services; data services remain external in this profile. ## Docker Compose: external services Requirements: Docker Engine with Compose v2 and reachable DWH, vector, and embeddings services. 1. For local development only, copy `deploy/env.example` to `deploy/.env` and fill in runtime credentials. The file is gitignored and is never copied into either image. 2. Add or edit YAML workspace descriptors under `deploy/workspaces/`. These files are mounted read-only. Use relative `roots`; they resolve beneath `/data/workspaces/`. 3. Start the external-service profile: ```sh docker compose -f compose.yaml -f deploy/compose.local.yaml \ --profile external up --build --wait ``` 4. Open . The published port is loopback-only. Set `THOTH_HTTP_PORT` before starting to use another loopback port. Application state, including settings, sessions, artifacts, and indexes, lives in the named `thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an explicit destructive command such as `docker compose down --volumes` removes it. The frontend depends on the core health check and proxies `/health` and `/api/*` to it. The application health endpoint intentionally checks process readiness only; external dependency diagnostics are exposed by `tht doctor` and do not prevent the UI from starting. Run the end-to-end packaging check with: ```sh ./scripts/docker-smoke.sh ``` The smoke script validates Compose, builds and waits for both services, checks health through the frontend, verifies SSE response headers, restarts the core, and confirms `/data` survives. Each run uses a unique Compose project and removes that project's containers, network, and test volume afterward. It never targets the fixed `thothii` operator project or its volume. Set `SMOKE_PROJECT` to a different explicit project name for reproducible debugging, and set `KEEP_SMOKE_RESOURCES=1` to retain that smoke project's resources for inspection; remove them later with `docker compose --project-name "$SMOKE_PROJECT" --profile external down --volumes`. ## Optional local pgvector and recovery Start the persistent local vector profile with `docker compose -f compose.yaml -f deploy/compose.local-vector.yaml --profile local-vector up --build --wait`. Its `vector_data` volume is independent of application state. Reader, writer, migrator, and bootstrap credentials remain separate; password files must be mode `0600` and must not be passed as URL arguments. ## Preprocessing jobs and S3 Evidence The included job workspaces target the local-vector profile. Point the four `THT_VECTOR_*_PASSWORD_SECRET_FILE` variables at owner-only files, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then run the explicit overlays (which are inert for normal runtime): ```sh docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \ -f deploy/compose.preprocess.yaml --profile local-vector --profile preprocess \ run --rm preprocess-evidence docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \ -f deploy/compose.preprocess.yaml --profile local-vector --profile preprocess \ run --rm preprocess-dwh ``` S3 Evidence uses the optional `tht[s3]` dependency and canonical `s3://bucket/key` provenance. AWS endpoints are used when no custom URL is supplied. Every custom endpoint is an explicit egress trust-boundary opt-in and uses path-style addressing; private and HTTP endpoints require additional independent opt-ins. Literal non-global IPv4/IPv6 addresses are classified locally; hostnames are not DNS-pinned, so trusted custom-endpoint deployments must enforce their destination with network egress policy. Store access key, secret key, and session token as secret references in deployment configuration—never in Compose environment values or source URIs. Discovery and reads are bounded by configured page, object, and byte limits. Create a versioned PostgreSQL custom-format backup (the filename is operator-controlled, so use an immutable timestamp or release identifier): ```sh ./scripts/vector-backup.sh \ --host 127.0.0.1 --port 5432 --database thoth --user thoth_backup \ --password-file /secure/thoth/vector-backup-password \ --output /secure/backups/thoth-vectors-2026-07-12.dump ``` The dump contains the three allowlisted `vectors` tables, their data and ACLs, plus the `public.tht_vector_migrations` ledger. Login roles and passwords are deliberately not copied: provision/reconcile the approved role names on the target first, and install the `vector` extension in its `vectors` schema. The target must otherwise contain no vector tables or ledger. Restore always names both the currently active source and a target on a physically distinct PostgreSQL cluster. The script compares PostgreSQL system identity, so host aliases or a different database in the active cluster cannot bypass the guard. It refuses a non-empty target unless `--force-nonempty` is explicit, and the clean restore is one transaction: ```sh ./scripts/vector-restore.sh \ --active-host vector-db --active-database thoth --active-user thoth_backup \ --active-password-file /secure/thoth/vector-active-password \ --target-host vector-db-restore --target-database thoth --target-user thoth_restore \ --target-password-file /secure/thoth/vector-restore-password \ --input /secure/backups/thoth-vectors-2026-07-12.dump ``` After restore, run `tht vector migrate --status --json`, adapter health, and a known retrieval query against the target before changing any deployment endpoint. Never test recovery against the active `vector_data` volume. `./scripts/local-vector-smoke.sh --backup-restore` performs this drill with disposable source and target volumes. ## Production trust boundary and secrets ThothII does not implement OIDC. Do not expose its application port directly to a network. The production pattern is an authenticated host reverse proxy that: - terminates TLS and authenticates every request; - removes any client-supplied identity header; - injects one trusted `X-Authenticated-User` value; - proxies to the loopback-only ThothII frontend. [`deploy/nginx-authenticated-proxy.conf.example`](deploy/nginx-authenticated-proxy.conf.example) shows the contract using nginx `auth_request`; replace the placeholder authentication gateway with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts this boundary and rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other auth mode fails during core startup. Production credentials use Compose secrets, not `deploy/.env`. Create four files outside the repository, restrict their host permissions, and point these variables to them: ```sh export THT_DWH_API_KEY_SECRET_FILE=/secure/thoth/dwh-api-key export THT_VEC_API_KEY_SECRET_FILE=/secure/thoth/vector-reader-api-key export THT_VEC_WRITE_API_KEY_SECRET_FILE=/secure/thoth/vector-writer-api-key export THT_CA_SECRET_FILE=/secure/thoth/ca-chain.pem export THT_DB_NAME=warehouse export THT_DWH_REST_URL=https://dwh.example.test export THT_VEC_REST_URL=https://vectors.example.test export THT_OLLAMA_URL=https://embeddings.example.test docker compose -f compose.yaml -f deploy/compose.production.yaml \ --profile external up --build --wait ``` The secrets and public CA chain are mounted read-only under `/run/secrets` and must be readable by the core's UID 10001. Host secret files must be `0600` or `0400`; Docker's runtime `0444` mount is accepted only beneath `/run/secrets`. See [`deploy/secrets/README.md`](deploy/secrets/README.md) for the verification command. The frontend remains on loopback; the authenticated host proxy is the only public listener. ## Reproducible image verification Base images use exact tags and immutable multi-platform manifest digests. Dependency update and residual OS-repository limitations are documented in [`docker/LOCKS.md`](docker/LOCKS.md). Run the shared architecture gate with `PLATFORM=linux/amd64` or `PLATFORM=linux/arm64`: ```sh PLATFORM=linux/arm64 ./scripts/verify-container-images.sh ``` It builds both images, runs common version/runtime/security smokes, and emits an image/package inventory beneath `.artifacts/container-images/`. CI runs the same script for both architectures.