# Unified Docker Compose Deployment Design **Date:** 2026-08-04 **Status:** approved in conversation, pending written-spec review ## Objective ThothII ships as one autonomous Docker Compose application that runs unchanged on a developer PC/Mac or on a server. ThothII has no runtime, build, network, path, proxy, configuration, or documentation dependency on PSD, Chirone, `omics_portal`, or any other application that happens to provide databases or vector services. ## Architecture The distribution contains the same `frontend` and `core` images in every environment. A portable base Compose file defines services, health checks, internal networking, named volumes, registry storage, and configuration contracts. Small local and server overrides select host exposure, storage bindings, authentication policy, restart policy, and operational limits without copying the complete service definitions. ```text Browser -> frontend container -> core container |-> Git workspace registry |-> configured DWH |-> configured VectorDB |-> configured embedding/LLM services `-> configured session persistence ``` The frontend calls the core over the private Compose network through same-origin proxying. The browser never needs the core port in the server profile. On a workstation, the frontend is bound to loopback and the core may be bound to loopback for diagnostics. On a server, a generic external reverse proxy forwards to the frontend host port; that proxy is an operator concern and is not a ThothII dependency. ## Compose layout - `compose.yaml`: portable base stack and the only complete service definition. - `deploy/compose.local.yaml`: loopback ports, local named volumes, `AUTH_MODE=none`, workstation defaults, and local installation identity. - `deploy/compose.server.yaml`: frontend host binding suitable for a reverse proxy, no public core binding, server storage policy, configurable authentication, restart policy, and resource limits. - `deploy/compose.git-ssh.yaml` and `deploy/compose.git-https.yaml`: mutually exclusive Git secret mounts and trust configuration. - `deploy/compose.connector-secrets.yaml`: explicit connector secret mounts selected by an installation. - `.env.example`: non-secret common variables and documented absolute host paths. - `deploy/env/local.env.example` and `deploy/env/server.env.example`: profile-specific examples containing names and safe defaults, never credentials. The standard commands are intentionally symmetric: ```sh docker compose -f compose.yaml -f deploy/compose.local.yaml build docker compose -f compose.yaml -f deploy/compose.local.yaml up -d ``` ```sh docker compose -f compose.yaml -f deploy/compose.server.yaml build docker compose -f compose.yaml -f deploy/compose.server.yaml up -d ``` Published images remain optional. A server can build from a release checkout or consume pinned images through an operator override without changing the application architecture. ## Configuration and secrets Portable workspace descriptors remain in the Git workspace registry. Installation-specific endpoints, ports, usernames, CA paths, and secret-file bindings remain local. Secret contents are mounted as files and never enter Git, browser drafts, image layers, Compose output, or generated workspace artifacts. The same deterministic naming contract continues to apply: `THT_WS___` for bindings and `_FILE`/`_SOURCE` for secret paths. The base Compose accepts generic DWH, vector, embedding, LLM, Git, and session-storage endpoints; none has a default hostname, path, or network associated with PSD or `omics_portal`. ## Networking and exposure The base stack owns a private Compose network. `frontend` reaches `core` by service name. External DWH, vector, Git, embedding, and LLM services are reached through operator-configured DNS names or URLs. `host.docker.internal` may be documented as a workstation option but is not hard-coded as a product dependency. The local profile binds user-facing ports to `127.0.0.1`. The server profile exposes only the frontend host port needed by a generic reverse proxy. Examples for Nginx and Caddy document TLS, forwarded identity, websocket/SSE behavior, and timeouts, but neither proxy is embedded into or required by the core architecture. ## Persistence Named volumes are the portable default for workstation installations. Server documentation shows explicit bind mounts under an operator-selected root such as `/srv/thothii`, with UID/GID and backup requirements. The same container paths are used in both profiles: - `/data/workspace-registry` for Git checkout, immutable snapshots, leases, state, and locks; - `/data/settings` for application settings; - `/data/sessions` for filesystem sessions when selected; - `/home/thoth/.pi` for Pi runtime state; - `/run/secrets` for read-only secret files. Shared PostgreSQL session storage remains optional. It is required only when multiple ThothII installations must see and resume the same sessions. ## Cross-platform source and line endings The repository gains a root `.gitattributes` that makes line endings deterministic independently of a developer's global Git configuration: ```gitattributes * text=auto *.sh text eol=lf Dockerfile* text eol=lf *.Dockerfile text eol=lf *.yml text eol=lf *.yaml text eol=lf *.json text eol=lf *.ts text eol=lf *.tsx text eol=lf *.py text eol=lf *.md text eol=lf *.ps1 text eol=crlf ``` Executable shell scripts keep their executable bit and LF bytes. CI and a local verification script scan Docker entrypoints, shell scripts, Compose/YAML, and Dockerfiles for carriage returns. The Docker build also fails early with a clear message if an executable copied into an image has CRLF. Documentation covers Git for Windows and WSL2, recommends cloning inside the WSL filesystem for Linux-container work, and provides a safe one-time renormalization procedure for existing clones. The documented process does not require changing global `core.autocrlf`. ## Build and release behavior The local build uses the repository checkout as a BuildKit context and builds both images with blocking TypeScript checks. The current non-blocking frontend typecheck is changed into a build gate. A validation command renders each supported Compose combination before build. Image tags include a local default and may be overridden with a release version or digest. Build inputs exclude `.git`, worktrees, local `.env` files, secrets, test output, caches, and workspace runtime data through `.dockerignore`. Builds must work from macOS, Windows/WSL2, and Linux without host-language runtimes beyond Docker, Compose, and Git. ## Migration from the current deployment The current PSD/portal-oriented root Compose is replaced by the portable base. Reusable settings from existing local and production overrides are folded into the new local/server overrides. Portal network aliases, absolute Chirone paths, external `localllm_default`, PSD evidence mounts, and `/datamart-builder` build arguments are removed from the product defaults. Existing operators migrate by copying only intentional values into the new environment and secret files, rendering Compose, backing up volumes, building the new stack, and validating health and workspace-registry status before switching the proxy. Legacy Compose examples remain in an archive or are removed only after their replacement documentation and migration checks exist. ## Error handling and operability Compose rendering fails when required non-secret values are absent. Entrypoints report stable, sanitized errors for unreadable secret files, invalid registry configuration, incompatible line endings, and storage permissions. Health checks distinguish process liveness from registry and connector readiness. Remote outages preserve the last valid workspace snapshot as already specified by the Git registry design. ## Documentation Two complete guides are maintained against the same architecture: - local PC/Mac installation: Docker Desktop/Engine prerequisites, Windows/WSL2 line endings, clone, environment creation, local build, first start, browser URL, update, backup, and recovery; - server installation: service account, directories, firewall, generic reverse proxy, environment and secrets, local image build or pinned image use, startup, health, upgrade, rollback, backup, and recovery. Both guides use copy-pastable commands validated by scripts. They explain which configuration is shared in Git, which is installation-local, and how a server that only provides DWH/VectorDB is consumed without installing ThothII there. ## Verification strategy Automated gates cover Compose rendering for local/server plus each Git transport, LF enforcement, Docker builds, container health, frontend-to-core same-origin routing, registry bootstrap and offline fallback, secret non-disclosure, and absence of PSD/Chirone/portal dependencies in active deployment files. Existing backend, frontend, harness, registry, and installation-document tests remain required. Manual acceptance covers a clean macOS build, a clean Windows Docker Desktop/WSL2 build from a GitHub clone, a Linux server deployment behind a generic proxy, an update after `git pull`, volume persistence, and restoration from backup. ## Non-goals - Bundling a DWH, production VectorDB, embedding server, LLM server, or reverse proxy into the mandatory ThothII stack. - Making PSD, Chirone, or `omics_portal` supported product profiles. - Synchronizing local filesystem sessions between installations without shared session storage. - Implementing persistent runtime SSH tunnels for workspace connectors in this migration.