docs: plan unified compose deployment

2026-08-04 14:26:28 +02:00
parent d82ebece4d
commit efda41aaa4
1 changed files with 687 additions and 0 deletions
@@ -0,0 +1,687 @@
+# Unified Docker Compose Deployment Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Convert ThothII into one autonomous Docker Compose distribution for Windows, macOS, and Linux servers, with external configurable data/AI services, embedded Pi, guided Pi management, reproducible local builds, and deterministic line endings.
+
+**Architecture:** `compose.yaml` is the sole complete stack definition; local and server files are small overrides. The mandatory stack contains only `frontend` and `core`; DWH, VectorDB, embedding, and LLM remain independently operated endpoints even when co-resident. Pi is pinned inside `core`; a Go `thothctl` executable wraps host-side Compose operations, while a web Pi Management page handles safe configuration and diagnostics.
+
+**Tech Stack:** Docker BuildKit/Compose v2, Node.js 22, TypeScript/Fastify, React/Vite, nginx-unprivileged, Python 3.12 harness, Go 1.24 for `thothctl`, Vitest, Playwright, shell/PowerShell verification.
+
+**Design:** `docs/superpowers/specs/2026-08-04-unified-compose-deployment-design.md`
+
+## Global Constraints
+
+- ThothII has no active build/runtime dependency on PSD, Chirone, or `omics_portal`.
+- The mandatory stack contains only `frontend` and `core`; it never starts DWH, VectorDB, embedding, LLM, or a reverse proxy.
+- External services use installation/workspace addresses even when they run on the Docker host.
+- Pi is pinned inside `core`; no host Pi or host Node/Python/Go runtime is required.
+- `core` never mounts a Docker daemon endpoint.
+- Secrets remain file-mounted under `/run/secrets` and never enter Git, images, browser storage, rendered Compose, or logs.
+- Local ports bind to `127.0.0.1`; server deployment publishes only the frontend port.
+- Shell, YAML, Dockerfile, JSON, TypeScript, Python, and Markdown use LF; PowerShell uses CRLF.
+- Every task follows red-green TDD and ends with a reviewable commit.
+
+## Target file map
+
+- `.gitattributes`, `.editorconfig`, `scripts/verify-line-endings.sh`: line-ending contract.
+- `compose.yaml`, `deploy/compose.local.yaml`, `deploy/compose.server.yaml`: portable stack and profiles.
+- `deploy/compose.git-*.yaml`, `deploy/compose.connector-secrets.yaml`: optional secret mounts.
+- `.env.example`, `deploy/env/*.env.example`: non-secret operator contracts.
+- `docker/core.Dockerfile`, `docker/frontend.Dockerfile`, `docker/nginx.conf.template`: reproducible images and same-origin routing.
+- `tools/thothctl/`: cross-platform host control executable.
+- `backend/src/pi/management.ts`, `backend/src/routes/pi-management.ts`: sanitized Pi APIs.
+- `frontend/src/api/pi-management.ts`, `frontend/src/shell/PiManagement.tsx`: Pi operator UI.
+- `docs/install/local.md`, `docs/install/server.md`, `docs/install/pi-management.md`: installation manuals.
+
+---
+
+### Task 1: Enforce deterministic line endings
+
+**Files:**
+- Create: `.gitattributes`
+- Create: `.editorconfig`
+- Create: `scripts/verify-line-endings.sh`
+- Create: `scripts/test-verify-line-endings.sh`
+- Modify: `docker/core.Dockerfile`
+- Test: `scripts/test-verify-line-endings.sh`
+
+**Interfaces:**
+- Produces: `scripts/verify-line-endings.sh [root]`, exit `0` when compliant and `1` with offending paths for CRLF.
+- Consumes: tracked files at repository root or an explicit fixture root; never scans volumes or secrets.
+
+- [ ] **Step 1: Write the failing verifier test**
+
+Create a temporary fixture with LF `ok.sh` and CRLF `bad.sh`, `compose.yaml`, and `Dockerfile`. Assert all bad paths are reported, `ok.sh` is absent, and LF-only input exits `0`.
+
+```sh
+fixture_root="$(mktemp -d)"
+trap 'rm -rf "$fixture_root"' EXIT
+printf '#!/bin/sh\r\nexit 0\r\n' > "$fixture_root/bad.sh"
+if ./scripts/verify-line-endings.sh "$fixture_root"; then
+  echo "expected CRLF rejection" >&2
+  exit 1
+fi
+```
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/test-verify-line-endings.sh`
+
+Expected: failure because the verifier does not exist.
+
+- [ ] **Step 3: Add attributes, editor settings, and verifier**
+
+Use this `.gitattributes` contract:
+
+```gitattributes
+* text=auto
+*.sh text eol=lf
+Dockerfile* text eol=lf
+*.Dockerfile text eol=lf
+*.yml text eol=lf
+*.yaml text eol=lf
+*.json text eol=lf
+*.ts text eol=lf
+*.tsx text eol=lf
+*.py text eol=lf
+*.md text eol=lf
+*.ps1 text eol=crlf
+```
+
+Use `git ls-files` for the repository and `find` only for fixture mode. Detect carriage returns with `LC_ALL=C grep -Il $'\r'`. Add an image-build check before `chmod` of Docker scripts.
+
+- [ ] **Step 4: Renormalize and verify**
+
+Run:
+
+```sh
+git add --renormalize .
+bash scripts/test-verify-line-endings.sh
+bash scripts/verify-line-endings.sh
+git diff --check
+```
+
+Expected: both scripts pass; review renormalized files to confirm line-ending-only changes.
+
+- [ ] **Step 5: Commit**
+
+```sh
+git add .gitattributes .editorconfig scripts/verify-line-endings.sh scripts/test-verify-line-endings.sh docker/core.Dockerfile
+git commit -m "build: enforce portable line endings"
+```
+
+### Task 2: Define the portable Compose contract
+
+**Files:**
+- Modify: `compose.yaml`
+- Modify: `deploy/compose.local.yaml`
+- Create: `deploy/compose.server.yaml`
+- Create: `deploy/env/local.env.example`
+- Create: `deploy/env/server.env.example`
+- Create: `.env.example`
+- Modify: `scripts/test-default-compose.sh`
+- Create: `scripts/test-unified-compose.sh`
+
+**Interfaces:**
+- Produces: base services `core` and `frontend`, network `thothii`, and volumes `settings`, `pi-state`, `workspace-registry`, `sessions`.
+- Produces: supported pairs base+local and base+server.
+
+- [ ] **Step 1: Write failing structural assertions**
+
+Render both profiles as JSON and assert:
+
+```js
+const services = Object.keys(config.services).sort();
+if (services.join(",") !== "core,frontend") throw new Error("mandatory stack must be core,frontend");
+if (/omics_portal|chirone|localllm_default|\/home\/chirone/i.test(JSON.stringify(config))) {
+  throw new Error("forbidden application coupling");
+}
+```
+
+Assert local publishes loopback frontend and optional loopback core; server publishes frontend only.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/test-unified-compose.sh`
+
+Expected: failure on current portal networks and host paths.
+
+- [ ] **Step 3: Replace root Compose with the portable base**
+
+Define only `core`, `frontend`, private network, health checks, and named volumes. `depends_on` may connect frontend to healthy core but never external services. Keep endpoint variables generic and secret-free.
+
+- [ ] **Step 4: Add local/server overrides**
+
+Local: `AUTH_MODE=none`, loopback ports, named volumes, installation ID `local`. Server: configurable frontend bind, `AUTH_MODE=upstream`, no core host port, `THT_DATA_ROOT` mounts, installation ID `server`.
+
+- [ ] **Step 5: Add environment examples**
+
+Use documentation domains such as `https://dwh.example.invalid`. Assert absent `THT_WORKSPACE_GIT_REMOTE` fails rendering with that exact variable name.
+
+- [ ] **Step 6: Verify and commit**
+
+```sh
+bash scripts/test-default-compose.sh
+bash scripts/test-unified-compose.sh
+docker compose -f compose.yaml -f deploy/compose.local.yaml config --quiet
+docker compose -f compose.yaml -f deploy/compose.server.yaml config --quiet
+git add compose.yaml deploy/compose.local.yaml deploy/compose.server.yaml deploy/env .env.example scripts/test-default-compose.sh scripts/test-unified-compose.sh
+git commit -m "deploy: unify local and server compose stack"
+```
+
+### Task 3: Isolate Git and connector secrets
+
+**Files:**
+- Create: `deploy/compose.git-ssh.yaml`
+- Create: `deploy/compose.git-https.yaml`
+- Create: `deploy/compose.connector-secrets.yaml`
+- Modify: `deploy/workspace-registry.env.example`
+- Create: `scripts/test-compose-secret-policy.sh`
+- Modify: `scripts/verify-workspace-install-docs.sh`
+
+**Interfaces:**
+- Produces: mutually exclusive Git overrides and explicit connector targets under `/run/secrets`.
+- Consumes: `THT_WS_*_FILE` and host-only `*_SOURCE` variables.
+
+- [ ] **Step 1: Write failing rendered-secret tests**
+
+Assert base has no `/dev/null` mounts; SSH mounts only key/known-hosts; HTTPS mounts only credentials/CA; connector mounts match declared `_FILE` targets; rendered output never contains fixture secret values.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/test-compose-secret-policy.sh`
+
+Expected: failure because secret mounts currently live in the portal-oriented base.
+
+- [ ] **Step 3: Implement overrides**
+
+Use read-only mounts and `${VAR:?message}` only in selected overrides. Keep strict SSH host checking and HTTPS CA verification. Reject relative/non-normalized `_SOURCE` paths.
+
+- [ ] **Step 4: Verify and commit**
+
+```sh
+bash scripts/test-compose-secret-policy.sh
+bash scripts/test-verify-workspace-install-docs.sh
+bash scripts/workspace-registry-smoke.sh
+git add deploy/compose.git-ssh.yaml deploy/compose.git-https.yaml deploy/compose.connector-secrets.yaml deploy/workspace-registry.env.example scripts/test-compose-secret-policy.sh scripts/verify-workspace-install-docs.sh
+git commit -m "deploy: isolate git and connector secrets"
+```
+
+### Task 4: Make frontend-to-core routing same-origin
+
+**Files:**
+- Modify: `docker/nginx.conf.template`
+- Modify: `docker/frontend-entrypoint.sh`
+- Modify: `docker/frontend.Dockerfile`
+- Modify: `frontend/src/api/runtime-config.ts`
+- Test: `frontend/src/api/runtime-config.test.ts`
+- Modify: `docker/smoke/frontend-policy-smoke.sh`
+- Modify: `scripts/test-backend-url-policy.sh`
+
+**Interfaces:**
+- Produces: browser base `/api`; nginx proxies to `http://core:8787` privately.
+- Consumes: optional internal `THT_FRONTEND_API_UPSTREAM` only.
+
+- [ ] **Step 1: Write failing routing tests**
+
+Assert `/api` default, rejection of browser-facing absolute production URLs, and nginx SSE settings:
+
+```nginx
+proxy_http_version 1.1;
+proxy_buffering off;
+proxy_read_timeout 3600s;
+```
+
+- [ ] **Step 2: Confirm red**
+
+```sh
+npm --prefix frontend test -- --run src/api/runtime-config.test.ts
+bash scripts/test-backend-url-policy.sh
+```
+
+Expected: failure because deployment-specific build arguments remain.
+
+- [ ] **Step 3: Implement runtime routing and blocking frontend build**
+
+Build once with `/` assets and `/api`. Render private upstream at startup, strip `/api`, preserve SSE. Replace non-blocking typecheck with `RUN npm run build`. Remove `/datamart-builder` assumptions.
+
+- [ ] **Step 4: Verify and commit**
+
+```sh
+npm --prefix frontend test -- --run src/api/runtime-config.test.ts
+npm --prefix frontend run build
+bash scripts/test-backend-url-policy.sh
+bash docker/smoke/frontend-policy-smoke.sh
+git add docker/nginx.conf.template docker/frontend-entrypoint.sh docker/frontend.Dockerfile docker/smoke/frontend-policy-smoke.sh frontend/src/api/runtime-config.ts frontend/src/api/runtime-config.test.ts scripts/test-backend-url-policy.sh
+git commit -m "deploy: route frontend and core through one origin"
+```
+
+### Task 5: Harden local image builds and embedded Pi
+
+**Files:**
+- Modify: `docker/core.Dockerfile`
+- Modify: `docker/pi-runtime/package.json`
+- Modify: `docker/pi-runtime/package-lock.json`
+- Create: `.dockerignore`
+- Modify: `docker/smoke/core-smoke.sh`
+- Modify: `scripts/verify-container-images.sh`
+- Create: `scripts/build-local.sh`
+- Create: `scripts/build-local.ps1`
+- Test: `scripts/test-container-deployment.sh`
+
+**Interfaces:**
+- Produces: `core` containing the pinned `/usr/local/bin/pi` and a standalone `frontend` image.
+- Produces: local build launchers requiring only Docker, Compose, and Git.
+
+- [ ] **Step 1: Write failing image-contract assertions**
+
+Inside `core`, assert `pi --version` matches `PI_VERSION`, UID is `10001`, Docker socket is absent, `/data` is writable, and no portal/Chirone path exists. Assert frontend health and `/api` routing.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/test-container-deployment.sh`
+
+Expected: failure on at least one old deployment contract.
+
+- [ ] **Step 3: Pin Pi from its lock input**
+
+Make `docker/pi-runtime/package-lock.json` the sole Pi dependency lock. Install with `npm ci --omit=dev` in a build stage, copy into `core`, and fail build when `pi --version` differs from `PI_VERSION`.
+
+- [ ] **Step 4: Add exclusions and platform launchers**
+
+Exclude `.git`, `.worktrees`, `.env`, secrets, dependencies, virtual environments, coverage, and runtime data. Both launchers run:
+
+```text
+docker compose -f compose.yaml -f deploy/compose.local.yaml build --pull
+```
+
+They print the same next command and preserve Docker's exit code.
+
+- [ ] **Step 5: Verify and commit**
+
+```sh
+bash scripts/build-local.sh
+bash scripts/test-container-deployment.sh
+bash scripts/verify-container-images.sh
+git add .dockerignore docker/core.Dockerfile docker/pi-runtime docker/smoke/core-smoke.sh scripts/build-local.sh scripts/build-local.ps1 scripts/test-container-deployment.sh scripts/verify-container-images.sh
+git commit -m "build: make embedded pi images reproducible"
+```
+
+Windows gate: `powershell -ExecutionPolicy Bypass -File scripts/build-local.ps1`.
+
+### Task 6: Build the cross-platform `thothctl` foundation
+
+**Files:**
+- Create: `tools/thothctl/go.mod`
+- Create: `tools/thothctl/cmd/thothctl/main.go`
+- Create: `tools/thothctl/internal/compose/runner.go`
+- Create: `tools/thothctl/internal/config/installation.go`
+- Create: `tools/thothctl/internal/output/sanitize.go`
+- Test: `tools/thothctl/internal/compose/runner_test.go`
+- Test: `tools/thothctl/internal/config/installation_test.go`
+- Test: `tools/thothctl/internal/output/sanitize_test.go`
+- Create: `docker/thothctl.Dockerfile`
+- Create: `scripts/build-thothctl.sh`
+
+**Interfaces:**
+- Produces: `thothctl --installation <absolute-path> <command>` binaries for Windows amd64, macOS amd64/arm64, Linux amd64/arm64.
+- Produces: `Runner.Run(ctx, args, stdin) (Result, error)` using argument arrays, never shell concatenation.
+
+- [ ] **Step 1: Write failing tests**
+
+Cover local/server Compose selection, paths containing spaces, missing Docker, propagated exit codes, and replacement of values matching password/token/key fields or secret-file contents with `[REDACTED]`.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./...`
+
+Expected: failure because packages do not exist.
+
+- [ ] **Step 3: Implement installation discovery and safe runner**
+
+Read `thothii-installation.yaml` fields `profile`, `projectDirectory`, `envFile`, and `overrides`. Resolve and validate absolute paths. Invoke `docker compose` with `exec.CommandContext` argument slices.
+
+- [ ] **Step 4: Implement base commands**
+
+Add `status`, `doctor`, `logs`, `start`, `stop`, and `update --check-only`. `doctor` validates Docker/Compose versions, rendered config, LF, volumes, and frontend/core health without printing environment values.
+
+- [ ] **Step 5: Cross-compile with Docker and verify**
+
+Produce `dist/thothctl/thothctl-windows-amd64.exe`, Darwin amd64/arm64, and Linux amd64/arm64 from `docker/thothctl.Dockerfile`.
+
+```sh
+docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./...
+bash scripts/build-thothctl.sh
+```
+
+Run the host-matching binary with `--help`, then commit:
+
+```sh
+git add tools/thothctl docker/thothctl.Dockerfile scripts/build-thothctl.sh
+git commit -m "feat: add cross-platform thothctl"
+```
+
+### Task 7: Implement safe Pi lifecycle commands in `thothctl`
+
+**Files:**
+- Create: `tools/thothctl/internal/pi/commands.go`
+- Create: `tools/thothctl/internal/pi/update.go`
+- Create: `tools/thothctl/internal/pi/state.go`
+- Test: `tools/thothctl/internal/pi/commands_test.go`
+- Test: `tools/thothctl/internal/pi/update_test.go`
+- Modify: `tools/thothctl/cmd/thothctl/main.go`
+- Create: `docs/contracts/thothctl-pi.md`
+
+**Interfaces:**
+- Produces: `pi status|doctor|configure|test|update|logs`.
+- Produces: `.thothctl/update-state.json` with previous/new image references and phase, never credentials.
+- Consumes: existing `/health`, `/models`, and `/settings` APIs plus `docker compose exec core pi --version`; Task 8 replaces the temporary composite checks with the dedicated Pi Management API.
+
+- [ ] **Step 1: Write failing update-state tests**
+
+With a fake Compose runner cover success and failure during build/pull, recreate, health, version, and smoke. Assert every post-recreate failure restores the prior image and leaves volume names unchanged.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./internal/pi -v`
+
+Expected: failure because Pi commands are absent.
+
+- [ ] **Step 3: Implement read-only commands**
+
+`status` runs `core pi --version`; `doctor` verifies image version, writable Pi volume, provider configuration presence, and `/health`; `test` combines `/models`, `/settings`, and a fixed `core pi --version` probe until Task 8 supplies the dedicated smoke endpoint; `logs` uses the shared sanitizer.
+
+- [ ] **Step 4: Implement guided configuration**
+
+Offer provider/model/reasoning choices returned by the backend. Atomically write non-secret defaults. For credentials print the expected secret filename and permissions; never accept secret text as an argument.
+
+- [ ] **Step 5: Implement transactional update**
+
+Record current image ID, pull/build requested pinned version, recreate only `core`, verify health/version/test, and roll back on failure. Refuse update while sessions are active unless `--drain` completes.
+
+- [ ] **Step 6: Verify and commit**
+
+```sh
+docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./internal/pi -v
+docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./...
+git add tools/thothctl docs/contracts/thothctl-pi.md
+git commit -m "feat: manage embedded pi with thothctl"
+```
+
+### Task 8: Add sanitized Pi Management backend APIs
+
+**Files:**
+- Create: `backend/src/pi/management.ts`
+- Create: `backend/src/routes/pi-management.ts`
+- Modify: `backend/src/app.ts`
+- Modify: `backend/src/config.ts`
+- Modify: `tools/thothctl/internal/pi/commands.go`
+- Test: `backend/test/pi-management.test.ts`
+- Test: `backend/test/routes-pi-management.test.ts`
+- Test: `tools/thothctl/internal/pi/commands_test.go`
+
+**Interfaces:**
+- Produces: `GET /pi-management/status`, `GET /pi-management/options`, `PUT /pi-management/config`, `POST /pi-management/test`, `GET /pi-management/logs`.
+- Produces: sanitized `PiStatus`, `PiOptions`, `PiInstallationConfig`, and stable errors.
+
+- [ ] **Step 1: Write failing service/route tests**
+
+Mock process execution and settings. Assert version parsing, closed choices, free-field validation, atomic writes, smoke timeout, 200-line log limit, redaction, and `403 pi_management_forbidden` in exposed server mode without trusted admin identity.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `cd backend && npx vitest run test/pi-management.test.ts test/routes-pi-management.test.ts`
+
+Expected: module-not-found failure.
+
+- [ ] **Step 3: Implement service and authorization**
+
+Use `execFile` with fixed argument arrays. Return only version, readiness, provider names, model IDs, reasoning choices, timestamps, and sanitized messages. Allow `AUTH_MODE=none` only when public exposure is false; upstream mode requires the documented admin claim. Expose no image-update API.
+
+- [ ] **Step 4: Switch `thothctl` to the dedicated API**
+
+Replace the Task 7 composite `/health`/`/models`/`/settings` test with `POST /pi-management/test`; load closed configuration choices from `GET /pi-management/options`. Preserve the direct in-container `pi --version` check as an independent image-integrity signal.
+
+- [ ] **Step 5: Verify and commit**
+
+```sh
+cd backend
+npx vitest run test/pi-management.test.ts test/routes-pi-management.test.ts
+npx vitest run
+npx tsc --noEmit -p .
+cd ..
+docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./internal/pi -v
+git add backend/src/pi/management.ts backend/src/routes/pi-management.ts backend/src/app.ts backend/src/config.ts backend/test/pi-management.test.ts backend/test/routes-pi-management.test.ts tools/thothctl/internal/pi/commands.go tools/thothctl/internal/pi/commands_test.go
+git commit -m "feat: expose safe pi management api"
+```
+
+### Task 9: Add the Pi Management interface
+
+**Files:**
+- Create: `frontend/src/api/pi-management.ts`
+- Create: `frontend/src/api/pi-management.test.ts`
+- Create: `frontend/src/shell/PiManagement.tsx`
+- Create: `frontend/src/shell/PiManagement.test.tsx`
+- Modify: `frontend/src/shell/AppShell.tsx`
+- Modify: `frontend/src/shell/AppShell.session-mgmt.test.tsx`
+
+**Interfaces:**
+- Consumes: Task 8 Pi APIs.
+- Produces: right-sidebar Pi Management panel without image-update execution or browser terminal.
+
+- [ ] **Step 1: Write failing UI tests**
+
+Cover loading/version, closed selects, validation, save, smoke test, sanitized logs, forbidden state, and copyable `thothctl pi update` instruction. Assert no secret input or browser terminal.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `cd frontend && npx vitest run src/shell/PiManagement.test.tsx src/api/pi-management.test.ts`
+
+Expected: module-not-found failure.
+
+- [ ] **Step 3: Implement API and panel**
+
+Use query keys `['pi-management','status']` and `['pi-management','options']`. Render provider/model/reasoning as closed choices, validate numeric fields before PUT, and show credentials only as present/missing.
+
+- [ ] **Step 4: Attach to the right sidebar**
+
+Add Pi next to Workspace Management, preserve current session/activity panels and accessibility, and remove AppShell comments/layout assumptions about Omics Portal chrome.
+
+- [ ] **Step 5: Verify and commit**
+
+```sh
+cd frontend
+npx vitest run src/shell/PiManagement.test.tsx src/api/pi-management.test.ts src/shell/AppShell.session-mgmt.test.tsx
+npx vitest run
+npx tsc -b
+git add src/api/pi-management.ts src/api/pi-management.test.ts src/shell/PiManagement.tsx src/shell/PiManagement.test.tsx src/shell/AppShell.tsx src/shell/AppShell.session-mgmt.test.tsx
+git commit -m "feat: add pi management interface"
+```
+
+### Task 10: Remove active PSD and portal deployment coupling
+
+**Files:**
+- Delete: `deploy/compose.production.yaml`
+- Delete: `deploy/compose.psd-local.yaml.example`
+- Modify: `README.md`
+- Modify: `PROJECT_STATE.md`
+- Modify: `scripts/run-stack.sh`
+- Create: `scripts/test-no-deployment-coupling.sh`
+- Modify: `scripts/test-external-compose-lifecycle.sh`
+
+**Interfaces:**
+- Produces: active deployment files free of PSD/Chirone/portal networks, paths, and prefixes.
+- Preserves: generic external endpoint support and historical design documents.
+
+- [ ] **Step 1: Write the failing coupling scan**
+
+Scan active Compose, Docker, root README, install guides, env examples, and run scripts for `omics_portal`, `/home/chirone`, `localllm_default`, `datamart-builder`, and PSD deployment filenames. Exclude historical specs/plans and canonical workspace content.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/test-no-deployment-coupling.sh`
+
+Expected: failure listing current portal-oriented files.
+
+- [ ] **Step 3: Remove superseded files and genericize launch behavior**
+
+Delete only deployment files replaced by Tasks 2–5. Make `run-stack.sh` call base+local or direct users to `thothctl start`. Preserve data migration utilities that do not affect runtime coupling.
+
+- [ ] **Step 4: Update root documentation and verify**
+
+State that co-location never puts DWH/vector/embedding inside ThothII.
+
+```sh
+bash scripts/test-no-deployment-coupling.sh
+bash scripts/test-unified-compose.sh
+bash scripts/test-external-compose-lifecycle.sh
+git add -A deploy/compose.production.yaml deploy/compose.psd-local.yaml.example README.md PROJECT_STATE.md scripts/run-stack.sh scripts/test-no-deployment-coupling.sh scripts/test-external-compose-lifecycle.sh
+git commit -m "refactor: remove portal deployment coupling"
+```
+
+### Task 11: Write and verify the PC/Mac installation manual
+
+**Files:**
+- Create: `docs/install/local.md`
+- Create: `docs/install/windows-line-endings.md`
+- Create: `docs/install/pi-management.md`
+- Create: `docs/install/examples/thothii-installation.local.yaml`
+- Modify: `docs/install/local-workspace-registry.md`
+- Modify: `scripts/verify-workspace-install-docs.sh`
+- Modify: `scripts/test-verify-workspace-install-docs.sh`
+
+**Interfaces:**
+- Produces: complete clean-install, build, update, backup, restore, Pi, and CRLF instructions for Windows/WSL2, macOS, and Linux PC.
+
+- [ ] **Step 1: Extend the verifier first**
+
+Require prerequisites, Git clone, LF verification, `.env`, external-service addressing, build, start, health, browser URL, `thothctl`, Pi update/rollback, backup/restore, pull update, and data-preserving uninstall. Require `pi-management.md` to cover UI permissions, every `thothctl pi` command, secret handling, direct support access, and failed-update recovery. Copy examples to a temporary path containing spaces and render there.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/test-verify-workspace-install-docs.sh`
+
+Expected: failure naming missing local-guide sections.
+
+- [ ] **Step 3: Write platform-specific instructions**
+
+Document macOS, Windows PowerShell, Windows WSL2, and Linux separately. Recommend cloning inside the WSL filesystem. Include LF checks after clone/pull. For an existing CRLF clone, prefer recloning; describe repository-local `core.autocrlf=false` and renormalization. If mentioning `git reset --hard`, require explicit backup/commit and a destructive-action warning immediately before it.
+
+- [ ] **Step 4: Document external services on the host**
+
+Show `host.docker.internal` for Docker Desktop and `extra_hosts: host.docker.internal:host-gateway` for Linux. Explain that container `127.0.0.1` is not the host. All addresses remain configurable.
+
+- [ ] **Step 5: Verify and commit**
+
+```sh
+bash scripts/test-verify-workspace-install-docs.sh
+bash scripts/verify-workspace-install-docs.sh --profile local
+bash scripts/docker-smoke.sh
+git add docs/install/local.md docs/install/windows-line-endings.md docs/install/pi-management.md docs/install/examples/thothii-installation.local.yaml docs/install/local-workspace-registry.md scripts/verify-workspace-install-docs.sh scripts/test-verify-workspace-install-docs.sh
+git commit -m "docs: add autonomous local installation guide"
+```
+
+### Task 12: Write and verify the server installation manual
+
+**Files:**
+- Create: `docs/install/server.md`
+- Create: `docs/install/reverse-proxy-nginx.md`
+- Create: `docs/install/reverse-proxy-caddy.md`
+- Create: `docs/install/examples/thothii-installation.server.yaml`
+- Modify: `docs/install/server-workspace-registry.md`
+- Modify: `scripts/verify-workspace-install-docs.sh`
+- Test: `scripts/test-verify-workspace-install-docs.sh`
+
+**Interfaces:**
+- Produces: generic Linux server deployment independent of another application's network.
+- Consumes: server profile, secret overrides, `thothctl`, and same-origin frontend.
+
+- [ ] **Step 1: Add failing server-guide assertions**
+
+Require service account/UID, directories, firewall, co-resident external endpoints, DNS/host-gateway choices, generic TLS proxy, upstream auth, local build or pinned images, startup, readiness, Pi management, drain, rollback, backup, restore, and diagnostics.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/test-verify-workspace-install-docs.sh`
+
+Expected: failure naming missing server sections.
+
+- [ ] **Step 3: Write the server and proxy guides**
+
+Use `/srv/thothii` only as an example operator root. State that DWH/vector/embedding on the same physical machine remain independently addressed services. Nginx/Caddy proxy only to frontend, preserve SSE, terminate TLS, and forward identity only after authentication.
+
+- [ ] **Step 4: Verify and commit**
+
+```sh
+bash scripts/test-verify-workspace-install-docs.sh
+bash scripts/verify-workspace-install-docs.sh --profile server
+bash scripts/test-unified-compose.sh
+git add docs/install/server.md docs/install/reverse-proxy-nginx.md docs/install/reverse-proxy-caddy.md docs/install/examples/thothii-installation.server.yaml docs/install/server-workspace-registry.md scripts/verify-workspace-install-docs.sh scripts/test-verify-workspace-install-docs.sh
+git commit -m "docs: add autonomous server installation guide"
+```
+
+### Task 13: Add end-to-end deployment and update gates
+
+**Files:**
+- Create: `scripts/unified-deployment-smoke.sh`
+- Create: `scripts/thothctl-update-smoke.sh`
+- Create: `scripts/test-windows-clone-contract.ps1`
+- Create: `.github/workflows/deployment.yml`
+- Modify: `PROJECT_STATE.md`
+- Modify: `README.md`
+
+**Interfaces:**
+- Produces: release gate for rendering, image build, embedded Pi, registry persistence, offline recovery, and update rollback.
+
+- [ ] **Step 1: Write failing orchestration smoke**
+
+Create an isolated Compose project and bare Git registry, build, start local, verify frontend/core/Pi/registry, recreate offline, perform a valid Git update, and prove volumes survive. Inject a bad Pi image/version and prove rollback.
+
+- [ ] **Step 2: Confirm red**
+
+Run: `bash scripts/unified-deployment-smoke.sh`
+
+Expected: failure because the script is absent.
+
+- [ ] **Step 3: Implement exact-resource cleanup**
+
+Generate a unique project name and temporary directory, label resources, and remove only those exact resources on exit. Never prune global Docker state. Sanitize captured logs.
+
+- [ ] **Step 4: Add Windows and CI gates**
+
+PowerShell scans tracked shell/YAML/Docker files for byte `0x0D`, renders Compose, and invokes Windows `thothctl`. CI runs LF and TypeScript gates everywhere, Docker smoke on Linux, and clone/Compose contract on Windows.
+
+- [ ] **Step 5: Run final verification**
+
+```sh
+bash scripts/verify-line-endings.sh
+bash scripts/test-unified-compose.sh
+bash scripts/test-compose-secret-policy.sh
+bash scripts/unified-deployment-smoke.sh
+bash scripts/thothctl-update-smoke.sh
+bash scripts/test-verify-workspace-install-docs.sh
+cd backend && npx vitest run && npx tsc --noEmit -p .
+cd ../frontend && npx vitest run && npx tsc -b
+cd ../harness && .venv/bin/pytest -q
+```
+
+Expected: all non-L2 tests pass; Docker-dependent harness tests run on a Docker-capable host.
+
+- [ ] **Step 6: Commit**
+
+```sh
+git add scripts/unified-deployment-smoke.sh scripts/thothctl-update-smoke.sh scripts/test-windows-clone-contract.ps1 .github/workflows/deployment.yml PROJECT_STATE.md README.md
+git commit -m "test: gate unified compose deployment"
+```
+
+## Completion criteria
+
+- Clean GitHub clones build/run on Windows Docker Desktop/WSL2 and macOS using only Docker, Compose, and Git.
+- The same source/images deploy on Linux through the server override.
+- Active deployment files contain no PSD, Chirone, or `omics_portal` dependency.
+- DWH, VectorDB, embedding, and LLM are always configurable external endpoints.
+- Pi exists in `core`, is configurable through Pi Management, and is safely updated by `thothctl`.
+- Git attributes prevent CRLF and verification catches corruption before image startup.
+- Local/server guides pass executable documentation checks.
+- Failed updates restore the previous image while preserving registry, sessions, settings, and Pi state.