docs: plan simplified Docker configuration implementation
This commit is contained in:
@@ -0,0 +1,156 @@
|
||||
# Simple Docker Configuration Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task with review checkpoints.
|
||||
|
||||
**Goal:** Make a fresh ThothII clone runnable with `docker compose up --build -d`, using one
|
||||
`deploy/secrets/thothii.secrets` bundle while preserving a tested legacy fallback.
|
||||
|
||||
**Architecture:** A strict Python secret-bundle loader becomes the single in-process source of
|
||||
secret values. Compose mounts the one bundle only where needed; the core converts values to
|
||||
provider/database runtime interfaces without logging or placing them in argv. The root `.env`
|
||||
is the default Compose interpolation file and selects the appropriate overlay through
|
||||
`COMPOSE_FILE`/`COMPOSE_PROFILES`; legacy `THT_*_SECRET_FILE` installations remain supported.
|
||||
|
||||
**Tech Stack:** Docker Compose v2, YAML, Python 3.12/Pydantic, Fastify/TypeScript, shell smoke
|
||||
tests, pytest, Vitest.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- The normal command must be exactly `docker compose up --build -d` from `ThothII/`.
|
||||
- The canonical secret bundle is `deploy/secrets/thothii.secrets`, key/value syntax, mode `0600`,
|
||||
ignored by Git and excluded from image build contexts.
|
||||
- Secret values must never appear in Compose config output, logs, argv, settings, health, or
|
||||
committed workspace files.
|
||||
- Existing `THT_*_SECRET_FILE` variables remain a documented compatibility path until removed by
|
||||
a later migration.
|
||||
- External, local-vector, and preprocess overlays must remain independently renderable.
|
||||
- Provider compound credentials remain fail-closed; only supported single-key providers are
|
||||
restored from the bundle.
|
||||
- Every task starts with a failing regression test and ends with focused tests, diff checks, and
|
||||
a small commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add the strict secret-bundle loader and compatibility adapter
|
||||
|
||||
**Files:**
|
||||
- Create: `backend/src/config/secret-bundle.ts`
|
||||
- Modify: `backend/src/config.ts`
|
||||
- Modify: `backend/src/pi/provider-credentials.ts`
|
||||
- Modify: `backend/src/pi/pi-process-manager.ts`
|
||||
- Modify: `backend/src/pi/list-models.ts`
|
||||
- Create: `backend/test/secret-bundle.test.ts`
|
||||
- Modify: `backend/test/provider-credentials.test.ts`
|
||||
- Modify: `backend/test/pi-process-manager.test.ts`
|
||||
- Modify: `backend/test/list-models.test.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- `loadSecretBundle(file: string): ReadonlyMap<string, string>` validates `NAME=VALUE` lines,
|
||||
duplicate/unknown/empty keys, `lstat`/`open(O_NOFOLLOW)`/`fstat` identity, owner and mode.
|
||||
- `secretValue(config, key)` first reads `THT_SECRETS_FILE`, then falls back to the existing
|
||||
`THT_*_SECRET_FILE` variable for compatibility.
|
||||
- The existing provider environment builder consumes a value map, so session and model-listing
|
||||
children share identical scrubbing and canonical-provider mapping.
|
||||
|
||||
- [ ] **Step 1: Write failing tests** for valid bundle parsing, comments/blank lines, duplicate
|
||||
keys, unknown keys, missing file, mode/owner failure, inode replacement, and secret redaction.
|
||||
- [ ] **Step 2: Run** `cd backend && npx vitest run test/secret-bundle.test.ts`; expected failure
|
||||
because the loader does not exist.
|
||||
- [ ] **Step 3: Implement** the loader with bounded line lengths, strict key allowlist, no shell
|
||||
evaluation, sanitized errors, and legacy adapter lookup.
|
||||
- [ ] **Step 4: Add tests** proving session spawn and model listing use the same bundle values and
|
||||
do not inherit bundle path or unselected provider credentials.
|
||||
- [ ] **Step 5: Run** `cd backend && npm run build && npx tsc --noEmit -p . && npx vitest run`;
|
||||
expected all backend tests pass.
|
||||
- [ ] **Step 6: Commit** `git commit -m "feat(config): load one validated secret bundle"`.
|
||||
|
||||
### Task 2: Make the root Compose command the default
|
||||
|
||||
**Files:**
|
||||
- Create: `.env.example`
|
||||
- Modify: `.gitignore`
|
||||
- Modify: `compose.yaml`
|
||||
- Modify: `deploy/compose.production.yaml`
|
||||
- Modify: `deploy/compose.local.yaml`
|
||||
- Modify: `deploy/env.example`
|
||||
- Create: `deploy/secrets/thothii.secrets.example`
|
||||
- Create: `scripts/test-default-compose.sh`
|
||||
- Modify: `scripts/test-container-deployment.sh`
|
||||
|
||||
**Interfaces:**
|
||||
- Root `.env` is Compose's automatic interpolation file; `.env.example` contains relative
|
||||
`THT_SECRETS_FILE=deploy/secrets/thothii.secrets`, default `COMPOSE_FILE=compose.yaml`, and
|
||||
the selected overlay/profile values.
|
||||
- `compose.yaml` starts `core` and `frontend` without requiring a profile; overlays extend it.
|
||||
- Core receives one `/run/secrets/thothii.secrets` mount and `THT_SECRETS_FILE` path.
|
||||
|
||||
- [ ] **Step 1: Write failing static tests** that run `docker compose config --quiet` from a
|
||||
temporary clone with `.env` and assert the default services are `core` and `frontend`, one
|
||||
bundle is declared, and no legacy secret file is required.
|
||||
- [ ] **Step 2: Run** `./scripts/test-default-compose.sh`; expected failure because root defaults
|
||||
still require profiles/separate secret files.
|
||||
- [ ] **Step 3: Implement** `.env.example`, `.gitignore`, Compose defaults and one secret mount.
|
||||
Preserve `deploy/compose.production.yaml` as an optional authenticated production override.
|
||||
- [ ] **Step 4: Run** `docker compose --env-file .env.example config --quiet` and the existing
|
||||
deployment/security scripts; expected no secret values in rendered YAML.
|
||||
- [ ] **Step 5: Commit** `git commit -m "build(compose): make root startup the default"`.
|
||||
|
||||
### Task 3: Convert local-vector and preprocess services to the bundle
|
||||
|
||||
**Files:**
|
||||
- Modify: `deploy/compose.local-vector.yaml`
|
||||
- Modify: `deploy/compose.preprocess-local-vector.yaml`
|
||||
- Modify: `deploy/compose.preprocess.yaml`
|
||||
- Modify: `deploy/workspaces/local-vector.yaml`
|
||||
- Modify: `deploy/workspaces/preprocess-evidence.yaml`
|
||||
- Modify: `deploy/workspaces/preprocess-dwh.yaml`
|
||||
- Modify: `scripts/local-vector-smoke.sh`
|
||||
- Modify: `scripts/preprocess-smoke.sh`
|
||||
- Modify: `scripts/test-preprocess-compose-config.sh`
|
||||
- Modify: `scripts/test-vector-backup-restore-safety.sh`
|
||||
|
||||
**Interfaces:**
|
||||
- Every local-vector/preprocess service reads the same mounted bundle path and selects only the
|
||||
named value through the shared loader/helper.
|
||||
- No service declares four file-backed Compose secrets after this task.
|
||||
|
||||
- [ ] **Step 1: Add failing tests** asserting one bundle mount, no `vector_*_password` secret
|
||||
declarations, and valid local-vector workspace resolution.
|
||||
- [ ] **Step 2: Run** focused Compose config and smoke tests; expected failure with current
|
||||
separate-secret declarations.
|
||||
- [ ] **Step 3: Implement** bundle mounts and helper invocations for bootstrap/migrator/reader/
|
||||
writer operations, keeping passwords out of URLs and shell logs.
|
||||
- [ ] **Step 4: Run** `./scripts/test-preprocess-compose-config.sh`, local-vector smoke and
|
||||
preprocess smoke with a clean generated project; expected all pass.
|
||||
- [ ] **Step 5: Commit** `git commit -m "feat(compose): use one secret bundle for local services"`.
|
||||
|
||||
### Task 4: Finish documentation and end-to-end default verification
|
||||
|
||||
**Files:**
|
||||
- Modify: `README.md`
|
||||
- Modify: `docs/installazione-docker-4-contesti.md`
|
||||
- Modify: `docs/index.md`
|
||||
- Modify: `deploy/secrets/README.md`
|
||||
- Modify: `scripts/docker-smoke.sh`
|
||||
- Modify: `scripts/test-default-compose.sh`
|
||||
|
||||
**Interfaces:**
|
||||
- Installation docs show only `cp .env.example .env`, create/fill one bundle, then
|
||||
`docker compose up --build -d`.
|
||||
- Advanced overlays are shown as optional `.env` presets, not mandatory command-line flags.
|
||||
|
||||
- [ ] **Step 1: Add failing documentation/smoke assertions** for the exact command and default
|
||||
files.
|
||||
- [ ] **Step 2: Implement** concise context-specific instructions and migration notes for old
|
||||
separate secret files.
|
||||
- [ ] **Step 3: Run** all shell syntax/config gates, backend/frontend builds/tests, full harness,
|
||||
default Docker smoke, local-vector smoke, preprocess smoke and `git diff --check`.
|
||||
- [ ] **Step 4: Commit** `git commit -m "docs: document one-command Docker installation"`.
|
||||
|
||||
### Task 5: Whole-plan review and handoff
|
||||
|
||||
- [ ] Review `a6b195b..HEAD` against this plan and confirm no secret leakage, profile regression,
|
||||
or legacy fallback bypass.
|
||||
- [ ] Run the complete verification matrix and report exact counts, skipped L2 tests, and any
|
||||
unavailable Docker/registry prerequisites.
|
||||
- [ ] Keep the branch/worktree intact for the user's integration choice.
|
||||
Reference in New Issue
Block a user