# Unified Docker Compose Deployment Design **Date:** 2026-08-04 **Status:** approved in conversation, pending written-spec review ## Objective ThothII ships as one autonomous Docker Compose application that runs unchanged on a developer PC/Mac or on a server. ThothII has no runtime, build, network, path, proxy, configuration, or documentation dependency on PSD, Chirone, `omics_portal`, or any other application that happens to provide databases or vector services. ## Architecture The distribution contains the same `frontend` and `core` images in every environment. A portable base Compose file defines services, health checks, internal networking, named volumes, registry storage, and configuration contracts. Small local and server overrides select host exposure, storage bindings, authentication policy, restart policy, and operational limits without copying the complete service definitions. ```text Browser -> frontend container -> core container |-> Git workspace registry |-> configured external DWH endpoint |-> configured external VectorDB endpoint |-> configured external embedding/LLM endpoints `-> configured session persistence ``` The frontend calls the core over the private Compose network through same-origin proxying. The browser never needs the core port in the server profile. On a workstation, the frontend is bound to loopback and the core may be bound to loopback for diagnostics. On a server, a generic external reverse proxy forwards to the frontend host port; that proxy is an operator concern and is not a ThothII dependency. ## Compose layout - `compose.yaml`: portable base stack and the only complete service definition. - `deploy/compose.local.yaml`: loopback ports, local named volumes, `AUTH_MODE=none`, workstation defaults, and local installation identity. - `deploy/compose.server.yaml`: frontend host binding suitable for a reverse proxy, no public core binding, server storage policy, configurable authentication, restart policy, and resource limits. - `deploy/compose.git-ssh.yaml` and `deploy/compose.git-https.yaml`: mutually exclusive Git secret mounts and trust configuration. - `deploy/compose.connector-secrets.yaml`: explicit connector secret mounts selected by an installation. - `.env.example`: non-secret common variables and documented absolute host paths. - `deploy/env/local.env.example` and `deploy/env/server.env.example`: profile-specific examples containing names and safe defaults, never credentials. The standard commands are intentionally symmetric: ```sh docker compose -f compose.yaml -f deploy/compose.local.yaml build docker compose -f compose.yaml -f deploy/compose.local.yaml up -d ``` ```sh docker compose -f compose.yaml -f deploy/compose.server.yaml build docker compose -f compose.yaml -f deploy/compose.server.yaml up -d ``` Published images remain optional. A server can build from a release checkout or consume pinned images through an operator override without changing the application architecture. ## Configuration and secrets Portable workspace descriptors remain in the Git workspace registry. Installation-specific endpoints, ports, usernames, CA paths, and secret-file bindings remain local. Secret contents are mounted as files and never enter Git, browser drafts, image layers, Compose output, or generated workspace artifacts. The same deterministic naming contract continues to apply: `THT_WS___` for bindings and `_FILE`/`_SOURCE` for secret paths. The base Compose accepts generic DWH, vector, embedding, LLM, Git, and session-storage endpoints; none has a default hostname, path, or network associated with PSD or `omics_portal`. ## Networking and exposure The base stack owns a private Compose network. `frontend` reaches `core` by service name. External DWH, vector, Git, embedding, and LLM services are reached through operator-configured DNS names or URLs. `host.docker.internal` may be documented as a workstation option but is not hard-coded as a product dependency. The local profile binds user-facing ports to `127.0.0.1`. The server profile exposes only the frontend host port needed by a generic reverse proxy. Examples for Nginx and Caddy document TLS, forwarded identity, websocket/SSE behavior, and timeouts, but neither proxy is embedded into or required by the core architecture. ## Product boundary and external services DWH, VectorDB, and embedding services are always external to the ThothII product boundary and Compose lifecycle. The mandatory ThothII stack neither defines nor starts them, never uses `depends_on` for them, and does not assume their implementation, installation path, container name, Docker network, or host. The same rule applies when all services happen to run on the same physical server: from ThothII's perspective they remain independently operated services reached through configurable addresses, ports, URLs, TLS settings, credentials, database/schema names, and vector collections. An operator may address a co-resident service through a routable host address, a DNS name, a documented host-gateway alias, or an explicitly configured external Docker network. None of these becomes a product default. In particular, `127.0.0.1` inside `core` always means the core container, not the Docker host; the installation guides must show the correct Mac/Windows Docker Desktop and Linux server alternatives. LLM endpoints follow the same configurable external-service model, while the Pi coding-agent runtime itself is part of ThothII as described below. ## Embedded Pi runtime Pi is an internal runtime dependency of ThothII and is installed at a pinned version while building the `core` image. The backend starts the image-bundled Pi executable; it never searches for or bind-mounts a Pi installation from the host. Therefore a clean Windows PC, Mac, Linux workstation, or server needs only Docker, Docker Compose, and Git to build and run ThothII. Pi configuration and provider credentials are supplied to the container through the documented ThothII configuration and secret-file contracts. Pi writable state may use the ThothII-managed `/home/thoth/.pi` volume, but the binary and package installation remain immutable image content. Upgrading Pi requires changing the pinned build argument, rebuilding the image, and passing the normal ThothII regression and image-smoke gates. The container health/smoke test verifies that Pi exists in the image and can be invoked without any host executable. ## Persistence Named volumes are the portable default for workstation installations. Server documentation shows explicit bind mounts under an operator-selected root such as `/srv/thothii`, with UID/GID and backup requirements. The same container paths are used in both profiles: - `/data/workspace-registry` for Git checkout, immutable snapshots, leases, state, and locks; - `/data/settings` for application settings; - `/data/sessions` for filesystem sessions when selected; - `/home/thoth/.pi` for Pi runtime state; - `/run/secrets` for read-only secret files. Shared PostgreSQL session storage remains optional. It is required only when multiple ThothII installations must see and resume the same sessions. ## Cross-platform source and line endings The repository gains a root `.gitattributes` that makes line endings deterministic independently of a developer's global Git configuration: ```gitattributes * text=auto *.sh text eol=lf Dockerfile* text eol=lf *.Dockerfile text eol=lf *.yml text eol=lf *.yaml text eol=lf *.json text eol=lf *.ts text eol=lf *.tsx text eol=lf *.py text eol=lf *.md text eol=lf *.ps1 text eol=crlf ``` Executable shell scripts keep their executable bit and LF bytes. CI and a local verification script scan Docker entrypoints, shell scripts, Compose/YAML, and Dockerfiles for carriage returns. The Docker build also fails early with a clear message if an executable copied into an image has CRLF. Documentation covers Git for Windows and WSL2, recommends cloning inside the WSL filesystem for Linux-container work, and provides a safe one-time renormalization procedure for existing clones. The documented process does not require changing global `core.autocrlf`. ## Build and release behavior The local build uses the repository checkout as a BuildKit context and builds both images with blocking TypeScript checks. The current non-blocking frontend typecheck is changed into a build gate. A validation command renders each supported Compose combination before build. Image tags include a local default and may be overridden with a release version or digest. Build inputs exclude `.git`, worktrees, local `.env` files, secrets, test output, caches, and workspace runtime data through `.dockerignore`. Builds must work from macOS, Windows/WSL2, and Linux without host-language runtimes or a host Pi installation beyond Docker, Compose, and Git. ## Migration from the current deployment The current PSD/portal-oriented root Compose is replaced by the portable base. Reusable settings from existing local and production overrides are folded into the new local/server overrides. Portal network aliases, absolute Chirone paths, external `localllm_default`, PSD evidence mounts, and `/datamart-builder` build arguments are removed from the product defaults. Existing operators migrate by copying only intentional values into the new environment and secret files, rendering Compose, backing up volumes, building the new stack, and validating health and workspace-registry status before switching the proxy. Legacy Compose examples remain in an archive or are removed only after their replacement documentation and migration checks exist. ## Error handling and operability Compose rendering fails when required non-secret values are absent. Entrypoints report stable, sanitized errors for unreadable secret files, invalid registry configuration, incompatible line endings, and storage permissions. Health checks distinguish process liveness from registry and connector readiness. Remote outages preserve the last valid workspace snapshot as already specified by the Git registry design. ## Documentation Two complete guides are maintained against the same architecture: - local PC/Mac installation: Docker Desktop/Engine prerequisites, Windows/WSL2 line endings, clone, environment creation, local build, first start, browser URL, update, backup, and recovery; - server installation: service account, directories, firewall, generic reverse proxy, environment and secrets, local image build or pinned image use, startup, health, upgrade, rollback, backup, and recovery. Both guides use copy-pastable commands validated by scripts. They explain which configuration is shared in Git, which is installation-local, and how a server that only provides DWH/VectorDB is consumed without installing ThothII there. ## Verification strategy Automated gates cover Compose rendering for local/server plus each Git transport, LF enforcement, Docker builds, container health, frontend-to-core same-origin routing, registry bootstrap and offline fallback, secret non-disclosure, and absence of PSD/Chirone/portal dependencies in active deployment files. Image tests also prove that the pinned Pi executable is available inside `core` and that no host Pi path is mounted or required. Existing backend, frontend, harness, registry, and installation-document tests remain required. Manual acceptance covers a clean macOS build, a clean Windows Docker Desktop/WSL2 build from a GitHub clone, a Linux server deployment behind a generic proxy, an update after `git pull`, volume persistence, and restoration from backup. ## Non-goals - Bundling a DWH, production VectorDB, embedding server, LLM server, or reverse proxy into the mandatory ThothII stack, even when those services are co-resident on the same physical host. - Making PSD, Chirone, or `omics_portal` supported product profiles. - Synchronizing local filesystem sessions between installations without shared session storage. - Implementing persistent runtime SSH tunnels for workspace connectors in this migration.