From 7118950416b3008a8182825de027c7f8b235de57 Mon Sep 17 00:00:00 2001 From: User Date: Fri, 21 Aug 2026 13:34:36 +0200 Subject: [PATCH] docs: record DWH auth implementation evidence --- .superpowers/sdd/progress.md | 39 ++- .superpowers/sdd/task-2-report.md | 468 ++++++++++++++++++++---------- .superpowers/sdd/task-3-report.md | 165 ++++++++--- .superpowers/sdd/task-4-report.md | 91 +++--- .superpowers/sdd/task-6-report.md | 233 +++++++++++++++ .superpowers/sdd/task-7-report.md | 133 +++++---- 6 files changed, 807 insertions(+), 322 deletions(-) diff --git a/.superpowers/sdd/progress.md b/.superpowers/sdd/progress.md index c6b9d14a..732ac907 100644 --- a/.superpowers/sdd/progress.md +++ b/.superpowers/sdd/progress.md @@ -1,14 +1,29 @@ -# Portable deployment SDD progress +# DWH REST per-installation authentication SDD progress -Plan: `docs/superpowers/plans/2026-07-11-adapter-foundations.md` -Branch: `codex/portable-deployment` -Worktree: `/Users/mp/projects/ThothII/.worktrees/portable-deployment` +Plan: `docs/superpowers/plans/2026-08-20-dwh-rest-per-installation-auth.md` +Branch: `feat/dwh-rest-installation-auth` +Worktree: `/home/chirone/ThothII-next/.worktrees/dwh-rest-installation-auth` +Baseline: workspace docs PASS; Go unavailable on host (use containerized Go 1.26.5); default Compose pre-existing path-sensitive false positive under `/home/chirone`. -Task 1: complete (commits e02e61e..a4eb6cc, review clean) -Task 1 final-review follow-up: public exports and frozen capability records now have explicit regressions. -Task 2: complete (commits a4eb6cc..f6302b3, review clean after authorized contract correction) -Task 3: complete (commits f6302b3..fe8d70d, review clean after authorized write-envelope correction) -Task 4: complete (commits fe8d70d..1e0911b, review clean) -Task 4 final-review follow-up: a real `tht` subprocess now proves exactly one legacy warning on stderr and pristine JSON stdout. -Task 5: complete (commits 1e0911b..dbbab6d, review clean after two fix waves) -Final adapter review fix wave: complete (`fix(adapter): close final foundation review`). HTTP vector reader/writer endpoints are independently optional; writer-only targeted memory/solved writes are supported. Vector health reports each side separately plus configured/observed embedding dimensions. `build_vector_loader` remains an explicitly tracked bulk-sync-only exception scheduled for the local pgvector migration plan; it is not used by interactive/targeted writes. +Task 1: complete (commits 3bc84b0..1e82fd3, review clean after bounded-digest fix wave). +Task 1 plan note: the dotted-import grep was resolved by the exact module assertion in `09290a0`; `go list -m all` confirms the DWH module only. + +Task 2: complete (commits 541ef45, 971a0e6, e90a1a1; independent vet fix d2415b5; review clean after bounded-read, expiry/JSON, same-Store, and cross-Store synchronization fix waves). + +Task 3: complete (commits ebb360f, 055dcab, b1079bd; independent review clean after CLI grammar, metadata validation, and ambiguous-publication cleanup fix waves). + +Task 4: complete (commits 943f809, 419c344, 134dc19; independent review PASS after legacy multiplicity, fail-closed handler/socket, SGID 2750, O_RDONLY shared lock, OpenReadOnly, and safe socket-parent waves). Task 5 must precreate `.writer.lock` as `0640 root:dwh-auth`; add a non-owner group/cross-process integration proof when packaging permits. + +Task 5: complete (commits 87606c7, 62ec29f; independent review PASS after auth-subrequest header isolation and target-Nginx duplicate-header verification). No runtime installation or service/Nginx mutation performed. + +Task 6: complete (commits 1b18a0f, d0f7e04; independent review PASS after regex/duplicate bypass, exact-PID TCP, and bounded-cleanup hardening). Minor for final review: remove or rename the redundant legacy `negative_postgrest_bypass` fixture and align the historical fixture-count prose if useful. No active Nginx/runtime mutation performed. + +Task 7: complete (commits 7b9b8b3, 707c13d, f616aab, 7b86ea9, 7fe5316; Terra review PASS). Documentation and rollout-contract alignment completed; no runtime mutation performed. + +Task 8: complete at frozen SHA `6499d24892b4383ac492579303e766cfb51fe44e` (fix commits `09290a0`, `6499d24`; Terra review PASS). Focused, portability, scanner, DWH Go, and two full tools/tht matrix runs PASS; evidence recorded at `.artifacts/dwh-auth/source-verification.md`. Broad coupling remains `BASELINE_RED` debt; immutable paths remain unchanged. + +Task 9: PASS at frozen SHA `0c4ff3750d3ecd3fc514e50e511cf7475fbe0446`. Built and installed the exact local candidate, enabled and started `dwh-auth`, imported the protected legacy credential under public ID `legacy-shared`, and created `psd-mac-primary` with public key ID `oNPdOfoH7ypLtVb1`. Registry check, AF_UNIX-only listener, v1/legacy `204`, random/missing `401`, bounded journal scan, installed-file hashes, and `nginx -t` all PASS. Protected report: `/root/dwh-auth-provision/gate9-20260821T054514Z.report.md`, SHA-256 `65ce1e0d8be5f74355eca2b1dca901da16f2864f68eafb7dede9d23ef36b82d5`. Nginx was not changed or reloaded; the legacy key remains active; the old stack was not changed or stopped. Terra final review: PASS with no Critical or Important findings. + +Task 10: NOT STARTED and requires a second explicit authorization. Public Nginx cutover, Mac-key delivery/configuration, and legacy revocation have not occurred. Activity 1 remains `IN_DISCUSSION`; external deployment remains `SURVEY_NO_GO`. + +Final clarification (bookkeeping): initial authorization at `6499d24` stopped before installation because the protected legacy file was missing and a journal-scan finding remained. A secret-safe legacy file was prepared without emit/hash; Nginx metadata remained unchanged and `nginx -t` PASS. Fix commits `6fb4886`, `dee0f9c`, `0c4ff37` received Terra PASS, followed by a detached complete re-freeze PASS at full `0c4ff3750d3ecd3fc514e50e511cf7475fbe0446`. The owner then explicitly authorized Gate 9 at that exact SHA; Gate 9 completed as recorded above. Task 10 remains a separate gate. diff --git a/.superpowers/sdd/task-2-report.md b/.superpowers/sdd/task-2-report.md index 404a87ab..8de7fa04 100644 --- a/.superpowers/sdd/task-2-report.md +++ b/.superpowers/sdd/task-2-report.md @@ -1,177 +1,325 @@ -# Task 2 report — root Compose startup +# Task 2 report — protected atomic registry -Status: DONE +## Scope and commit -Implemented the root Compose defaults and the single bundle declaration: - -- added `.env.example` with automatic Compose defaults (`COMPOSE_FILE=compose.yaml`, an empty - profile, and the relative `THT_SECRETS_FILE` path); -- removed the mandatory `external` profile from `core` and `frontend`; -- mounted `deploy/secrets/thothii.secrets` at `/run/secrets/thothii.secrets` and passed only the - mounted path into the core container; -- changed the production overlay to inherit that bundle instead of declaring per-secret mounts; -- removed the local overlay's legacy `env_file` dependency; -- added the versioned bundle template and `.gitignore` exception; -- updated deployment security checks and added `scripts/test-default-compose.sh`. - -Focused verification: - -```text -./scripts/test-default-compose.sh # default Compose contract passed. -./scripts/test-container-deployment.sh # container deployment security contract passed. -./scripts/test-preprocess-compose-config.sh # preprocess compose config: ok -docker compose --env-file .env.example config --quiet - (with a temporary mode-0600 bundle via THT_SECRETS_FILE) -git diff --check -``` - -The local-vector and preprocess service secret declarations remain for Task 3, which converts -those services to the same bundle helper. Documentation and smoke command migration is reserved -for Task 4. - ---- - -# Task 2 report — PostgreSQL session repository - -## Scope delivered - -- Added `PostgresSessionRepository`, implementing the Task 1 repository contract with a - direct PostgreSQL SQLAlchemy connection, transaction-local RLS context, UUIDv4 validation, - current artifacts (including `cte_sql:`), append-only decisions, preferences, and - content-free deletion tombstones. -- Added `tht session migrate --database-url URL [--status] --json` and a checksum-protected, - advisory-transaction-locked migration runner. -- Added server session configuration selection. `session_storage.connection` uses direct - PostgreSQL TLS modes `verify-ca` or `verify-full`; it does not use PostgREST. -- Updated packaging and `.gitignore` so session migrations are present in the built wheel. -- Did not alter Task 3 workflow commands, Pi gate code, or backend code. +- Commit: `541ef45 feat: add protected DWH credential registry` +- Committed files only: + - `tools/dwh-auth/internal/securefile/securefile_linux.go` + - `tools/dwh-auth/internal/securefile/securefile_linux_test.go` + - `tools/dwh-auth/internal/registry/store.go` + - `tools/dwh-auth/internal/registry/store_test.go` +- No server, Nginx, systemd, Docker stack, real registry, secrets, or legacy ThothII files were + read or changed. Tests use `t.TempDir` and synthetic record digests only. ## TDD evidence -### RED - -Command: - -```sh -cd harness && .venv/bin/pytest tests/test_postgres_session_repository.py tests/test_session_migrate_cmd.py -q -``` - -Result before production implementation: `1 failed, 4 errors in 3.89s`. - -- Four setup errors were `ModuleNotFoundError: No module named - 'tht.session.postgres_repository'`. -- The migration CLI test failed because `tht session migrate` did not exist (`No such command - 'migrate'`). - -### GREEN - -Initial focused suite after implementation: `5 passed in 4.18s`. - -Final focused verification: - -```sh -cd harness && .venv/bin/pytest \ - tests/test_session_repository.py \ - tests/test_postgres_session_repository.py \ - tests/test_session_migrate_cmd.py \ - tests/test_vector_migration_packaging.py -q -``` - -Result: `12 passed in 5.80s`. - -Changed-file lint verification: - -```sh -cd harness && .venv/bin/ruff check \ - tht/session/postgres_repository.py tht/migrations/sessions tht/config.py \ - tht/session/repository.py tht/cli/session_cmd.py \ - tests/test_postgres_session_repository.py tests/test_session_migrate_cmd.py \ - tests/test_vector_migration_packaging.py -``` - -Result: `All checks passed!`. - -## Migration and role policy choices - -`001_schema.sql` creates only private `thoth_sessions` tables: - -- `principals` and `principal_preferences`; -- `sessions`, with `session_artifacts` and `review_decisions` cascading on session deletion; -- `audit_log`, which deliberately has no content/detail/metadata column and keeps only action, - session UUID, actor identity, owner identity, and timestamp. - -`002_security.sql` creates separate `thoth_sessions_runtime` and -`thoth_sessions_migrator` group roles, explicitly `NOLOGIN NOBYPASSRLS NOSUPERUSER`, revokes -public access, gives the runtime role only the operations required by the adapter, and enables -and forces RLS on every table. Owner/admin policies read only transaction-local settings: -`thoth_sessions.actor_issuer`, `thoth_sessions.actor_subject`, and -`thoth_sessions.is_admin`. The adapter starts every operation in a transaction, switches to the -restricted runtime role, sets those settings with `set_config(..., true)`, and uses advisory -transaction locks for migrations and per-session mutations. - -The runtime role remains a `NOLOGIN` group role by design. Deployment must provision a dedicated -non-superuser LOGIN role and grant it membership, for example: - -```sql -CREATE ROLE thoth_sessions_app LOGIN NOINHERIT PASSWORD ''; -GRANT thoth_sessions_runtime TO thoth_sessions_app; -``` - -This avoids embedding an environment-specific login name or credential in versioned SQL. The -new integration test proves that this non-superuser membership path can create and read a -session while the adapter executes as `thoth_sessions_runtime`. - -## Security/self-review - -- Owner isolation and admin cross-owner reads run against disposable PostgreSQL containers, - not Supabase. -- No table or column includes `embedding`; repository code imports no embedding/vector code; - the regression test writes a session artifact under a monkeypatched embedding sentinel. -- An unauthorized owner receives the same `SessionError` as an absent session, preserving the - future backend's 404 mapping boundary. -- The audit row is inserted before deleting the parent session, so cascades remove all artifact - and decision content while the tombstone survives. -- A security review found and this task fixed the initial `.gitignore` rule that would have - excluded `migrations/sessions/*.sql` from Git/wheels. The wheel test now asserts both session - migration files and checks both the existing vector CLI and the new session CLI. -- The review also highlighted runtime login provisioning. It is covered by a non-superuser - regression test and documented above; concrete credential/role deployment belongs to Task 7. - -## Remaining concerns - -- Full `harness/.venv/bin/pytest -q` could not complete in this execution environment: the - runner terminated the command after roughly 30 seconds. Captured output reached 44% with no - failures before termination; `pgrep` confirmed no pytest process remained. The Task 2 focused - suites above completed successfully. -- `harness/.venv/bin/ruff check .` currently reports 34 pre-existing violations in unrelated - test files (for example unused imports in `tests/l0/test_db_connection.py` and semicolon style - in `tests/test_phase_effective.py`). The changed-file Ruff command is clean. -- Task 7 must safely provision the dedicated runtime login/membership and inject its TLS - credentials/CA; this task intentionally does not create a deployment-specific LOGIN role or - password. - -## Review follow-up — unavailable migration database JSON contract +All Go commands ran in the required official `golang:1.26.5` container with only this linked +worktree bind-mounted at `/work`. The container image reports `go version go1.26.5 linux/amd64`. ### RED -Command: +Before either Task 2 production file existed, the focused command was run inside the container: -```sh -cd harness && .venv/bin/pytest \ - tests/test_session_migrate_cmd.py::test_session_migrate_status_database_failure_is_pristine_json -q +```text +go test ./internal/securefile ./internal/registry -count=1 ``` -Result: `1 failed in 0.46s`. The unreachable direct PostgreSQL URL exited with code 1 but left -stdout empty, so `json.loads(result.stdout)` raised `JSONDecodeError`. +It failed non-zero for the expected absent implementation symbols, including `undefined: OpenDir`, +`undefined: ReadSecret`, `undefined: Open`, `undefined: State`, `undefined: PublicRecord`, and +`undefined: Store`. ### GREEN -The session migration CLI now catches `SQLAlchemyError` at the same command boundary as its -migration/domain errors and emits only `{"error": ...}` on stdout for `--json`. +After the minimal implementation and formatting: -```sh -cd harness && .venv/bin/pytest tests/test_session_migrate_cmd.py -q -cd harness && .venv/bin/ruff check tht/cli/session_cmd.py tests/test_session_migrate_cmd.py +```text +go test ./internal/securefile ./internal/registry -count=1 ``` -Result: `2 passed in 3.60s`; Ruff: `All checks passed!`. +Result: + +```text +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/securefile +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry +``` + +### Race verification + +The required race command completed successfully: + +```text +go test -race ./internal/securefile ./internal/registry -count=1 +``` + +Result: + +```text +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/securefile 1.027s +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry 1.179s +``` + +Additional scoped verification: + +```text +go vet ./internal/securefile ./internal/registry +go test ./... -count=1 +git diff --cached --check +``` + +The Task 2 vet command completed with no findings; all four DWH-auth packages passed the full +module test run; the staged-diff check completed with no output. + +## Delivered behavior + +- `securefile` is Linux-only and traverses absolute paths through descriptor-anchored + `syscall.Open`/`Openat` calls with `O_NOFOLLOW|O_CLOEXEC`; protected roots, child directories, + records, and secret files are regular/directories only and are checked against `Lstat` after + `Fstat`. +- Protected reads reject special, group-writable, or world-writable modes, cap record reads at + 4096 bytes, read at most one extra byte, and reject file-size changes or short/partial reads. + Secret ingress additionally requires exact `0600`. +- Secret output uses `O_CREAT|O_EXCL|O_NOFOLLOW`, exact `0600`, and an absolute protected parent. +- `registry.Open` creates protected `active` and `revoked` subdirectories under an existing safe + root. Record enumeration rejects unexpected entries, unsafe files, symlinks, oversized files, + bad filenames, malformed JSON, unknown JSON fields, duplicate JSON fields, and trailing JSON. +- `Add` validates Task 1 records, writes canonical JSON plus one newline through an exclusive + temporary file, sets final mode `0640`, syncs the file, renames under a protected per-root writer + lock, then syncs the directory. +- `Revoke` writes and syncs a valid revoked record before unlinking and syncing the active record. + `Find` checks revoked first; `List` resolves an active/revoked overlap to the revoked public + record. `FindLegacy` scans fail-closed and permits only the reserved legacy record state. +- `PublicRecord` deliberately omits `secret_sha256`; the redaction is regression-tested. + +## Security-test coverage + +- protected normal files and canonical record publication; +- symlinked roots, registry directories, records, and secret input; +- unsafe root/directory/record/secret modes; +- bounded/oversized record input; +- unknown, duplicate, trailing, and partial JSON; +- filename mismatch and multiple legacy-record integrity failures; +- revoked-state precedence when both active and revoked files exist; +- concurrent adds and concurrent reads during revocation, including the race detector. + +## Self-review + +Reviewed all syscall, path, mode, and error paths after the final race run: + +- Directory traversal never follows a supplied component; later operations use retained directory + descriptors, not re-opened untrusted prefixes. +- `Fstat` validates the opened object and `Lstat` must identify the same inode/device; the direct + child name grammar refuses separators, dot components, and NUL. +- File validation occurs before and after reads; mode/type/size checks fail closed. Directory + listing obtains a fresh `openat(dirfd, ".")` descriptor so scans do not share a mutable directory + offset. +- Writer serialization protects the check-then-rename no-replace sequence. Failed temporary + cleanup leaves an unexpected entry that later scans reject rather than silently accepting it. +- State-specific validation rejects revocation metadata in active records and requires it in + revoked records. Revoked files are consulted before active files so interruption after revoked + publication cannot reactivate a credential. +- All functionality uses only Go standard-library packages and Linux `syscall`; no CGO, SQLite, + or third-party module was added. + +## Concerns + +- The optional whole-module `go vet ./...` reports a pre-existing Task 1 test warning at + `internal/credential/credential_test.go:86` (`append` with no variadic values). The identical + line is present in approved HEAD `1e82fd3`, outside this task’s authorized files. Focused Task 2 + vet passes, and all module tests pass. +- The official image's login shell resets `PATH` and hides `/usr/local/go/bin`; all evidence uses + direct `go`/`gofmt` container entrypoints, which preserves the image’s Go 1.26.5 environment. +- The generic `apply_patch` helper intermittently failed before file access with a sandbox network + namespace error. Exact scoped corrections were applied through the shared worktree workflow; + this did not affect the final staged file set or verification evidence. + +## Review remediation — 2026-08-21 + +### Scope and fix commit + +- Review-fix commit: `971a0e6 fix: harden DWH credential registry reads`. +- Committed files only: + - `tools/dwh-auth/internal/registry/store.go` + - `tools/dwh-auth/internal/registry/store_test.go` +- The separate Task 1 vet correction is the independent preceding commit `d2415b5`; it is not + included in this Task 2 fix commit. No filesystem primitive, server, Nginx, service, registry, + secret, Docker stack, or legacy ThothII file was changed. + +### Strict TDD evidence + +All commands again used the official `golang:1.26.5` image with only this linked worktree mounted +at `/work`. + +#### RED + +The first focused command was run after the new regression tests and before production changes: + +```text +go test ./internal/securefile ./internal/registry -count=1 +``` + +It failed as intended. The three case-variant aliases (`SECRET_SHA256`, `Secret_SHA256`, and +`Schema_Version`) were accepted; past expiry returned active records from both `Find` and +`FindLegacy`; a revocation snapshot let readers return active data before publication; a temporary +file let `List`, `Check`, and `FindLegacy` observe false integrity failures; and the original +concurrent-read regression observed `ErrNotFound` during revocation. + +The deterministic exact-expiry test was then added before the clock implementation. Its focused +run failed as intended with: + +```text +internal/registry/store_test.go:572:10: store.now undefined +``` + +#### GREEN and verification + +After the minimum implementation and `gofmt`: + +```text +go test ./internal/securefile ./internal/registry -count=1 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/securefile 0.014s +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry 0.684s + +go test -race ./internal/securefile ./internal/registry -count=1 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/securefile 1.022s +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry 1.711s + +go vet ./internal/securefile ./internal/registry +``` + +The focused vet output was empty (success). Additional final checks passed: + +```text +go test ./... -count=1 +ok internal/credential +ok internal/record +ok internal/registry +ok internal/securefile + +go vet ./... +go test -race ./internal/registry -count=10 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry 8.127s +git diff --cached --check +``` + +### Remediated security invariants + +- `Find` and `FindLegacy` now deny an active record when `ExpiresAt <= now.UTC()`, returning the + existing non-disclosing `ErrNotFound`. The unexported per-Store `now` function is the minimal + deterministic clock seam; past, exact-equality, and future cases are covered for v1 and legacy + records. A revoked record is still consulted before expiry and therefore remains authoritative. +- One per-Store `sync.RWMutex` creates an in-process consistent snapshot. `Add` and `Revoke` hold + it exclusively for their full writer-lock lifetime, including temporary-file publication and + revoked-then-active removal. `Find`, `FindLegacy`, `List`, and `Check` hold a shared lock; their + bodies delegate only to unlocked helpers, preventing nested-lock deadlocks. `Close` also takes + the exclusive lock before closing descriptors. +- Deterministic regression tests hold the writer path at the revocation publication/unlink and + temporary-file stages. They prove public readers wait, then see either the final revoked state or + a clean directory, eliminating the `Names`-to-load/unlink and temporary-entry false failures + within the Store contract. +- Before struct decoding, the outer record JSON object now requires exactly spelled keys from the + schema allowlist and rejects duplicate literal keys. `Decoder.DisallowUnknownFields`, recursive + duplicate detection, trailing-value rejection, record validation, filename matching, no-follow + reads, modes, and durability ordering remain intact. + +### Self-review and concerns + +- Reviewed the new lock boundaries, error returns, revoked-first ordering, clock fallback, + JSON-token consumption, and every unchanged `securefile` syscall/path/mode boundary. The change + adds only standard-library `sync`; it does not relax existing fail-closed behavior. +- The synchronized snapshot is intentionally per `Store`, matching the requested in-process + contract. The existing protected advisory lock continues to serialize writers across Store + instances/processes; no cross-process reader snapshot is claimed by this fix. +- The historical whole-module vet concern in the original Task 2 report is now resolved by the + independent Task 1 commit `d2415b5`; complete module vet passes in the final evidence above. + +## Cross-Store snapshot remediation — 2026-08-21 + +### Scope and TDD evidence + +This third Task 2 fix wave changes only the protected lock primitive and registry snapshot code: + +- `tools/dwh-auth/internal/securefile/securefile_linux.go` +- `tools/dwh-auth/internal/securefile/securefile_linux_test.go` +- `tools/dwh-auth/internal/registry/store.go` +- `tools/dwh-auth/internal/registry/store_test.go` + +All commands used the official `golang:1.26.5` image with only this linked worktree mounted at +`/work`. + +The test-only red patch initially tried to inspect the unexported `securefile.Dir.fd` through the +registry package and therefore did not compile. That assertion was removed without production +changes: the registry tests still create writer Store A and reader Store B through two independent +`Open(root)` calls, while the securefile test proves separate descriptors directly in its own +package. The subsequent behavioral RED run, before the production change, was: + +```text +go test ./internal/securefile ./internal/registry -count=1 +FAIL TestLockSharedAllowsReadersAndBlocksExclusiveWriter: Dir lacks shared advisory locking +FAIL TestCrossStoreReadersWaitAcrossRevokePublicationAndUnlink: + Find, List, Check, and FindLegacy completed during Store A's revocation snapshot +FAIL TestCrossStoreScanReadersWaitForWriterTemporaryFile: + Store B's List, Check, and FindLegacy observed `.tmp-regression` +``` + +### Delivered synchronization contract + +- `securefile.Dir.LockShared` now acquires `LOCK_SH` on the same protected, no-follow, exact-0600 + root lock file used by `Lock`, which continues to acquire `LOCK_EX`. The lock file is still + opened/created, mode-validated, inode-checked, and closed through the existing Linux syscall + path. +- Every public snapshot reader (`Find`, `FindLegacy`, `List`, and `Check`) takes its Store + `RLock`, then a shared advisory lock on root `.writer.lock`, and retains both through the whole + revoked/active lookup or directory scan/load. `Add` and `Revoke` retain Store `Lock`, then the + same root lock under `LOCK_EX`, over their full operation. +- The lock order is universally Store mutex then root advisory lock. Public methods delegate only + to unlocked helpers, so neither reader nor writer paths recursively acquire the Store mutex. + `Close` retains its exclusive Store mutex, preventing descriptor closure from racing any locked + reader or writer. +- Revoked-first precedence, expiry denial, exact JSON validation, no-follow checks, record modes, + temporary-file durability, and all previous behavior remain unchanged. + +### GREEN and repeated verification + +```text +go test ./internal/securefile ./internal/registry -count=1 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/securefile 0.019s +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry 0.711s + +go test -race ./internal/securefile ./internal/registry -count=1 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/securefile 1.032s +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry 1.748s + +go vet ./internal/securefile ./internal/registry +go test ./... -count=1 +ok internal/credential +ok internal/record +ok internal/registry +ok internal/securefile +go vet ./... + +go test -race ./internal/registry \ + -run 'TestCrossStoreReadersWaitAcrossRevokePublicationAndUnlink|TestCrossStoreScanReadersWaitForWriterTemporaryFile' -count=20 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/registry 9.880s + +go test -race ./internal/securefile \ + -run TestLockSharedAllowsReadersAndBlocksExclusiveWriter -count=20 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/securefile 1.063s + +git diff --check +``` + +Both vet commands and the whitespace check produced no output. The securefile regression opens +three protected directory descriptors, proves they are distinct, permits two independent shared +holders, proves a third descriptor cannot take `LOCK_EX|LOCK_NB`, then proves exclusive acquisition +succeeds after shared release. The registry regressions deterministically block Store B readers +while Store A holds the exclusive root lock and verify only final revoked/clean states afterward. + +### Self-review and concerns + +- Reviewed lock creation/reopen races, no-follow flags, exact lock-file mode validation, lock + release, descriptor lifetime, lock ordering, error wrapping, and the unlocked-helper call graph. + No public reader invokes another public reader or writer while holding a Store lock. +- Advisory synchronization necessarily covers cooperating registry Store instances/processes; + arbitrary external filesystem mutation remains fail-closed through the existing integrity + checks rather than being silently accepted. +- No known concerns within the registry's cooperating-process contract. diff --git a/.superpowers/sdd/task-3-report.md b/.superpowers/sdd/task-3-report.md index 5fb7ae6d..52d8a10f 100644 --- a/.superpowers/sdd/task-3-report.md +++ b/.superpowers/sdd/task-3-report.md @@ -1,56 +1,129 @@ -# Task 3 report — workflow repository migration +# Task 3 report — secret-safe dwh-auth administrative CLI -## RED +## Scope -- `harness/tests/test_session_repository_workflow.py` initially failed at collection: - `persist_verified_finalization` did not exist. -- The new gate test initially failed because `write_cte_sql` and `write_final_sql` - were not registered. Its first run also exposed the worktree-local missing - Node dependency (`typebox`); `npm ci` installed the lockfile dependency. -- After the principal/legacy policy was clarified, the resolver tests initially - failed because `resolve_principal` did not exist. +- Added `tools/dwh-auth/internal/command/command.go`, its command tests, and + `tools/dwh-auth/cmd/dwh-auth/main.go`. +- The CLI accepts only the frozen Task 3 grammar: key create/import/list/status/revoke, + registry check, and the reserved serve invocation. +- Existing Task 1–2 APIs are consumed without modifying their files. +- No server, Nginx, systemd, Compose, portable `tht`, real registry, real secret, or legacy + stack was accessed or changed. -## GREEN evidence +## TDD evidence -- Focused Python regression set: `66 passed`: - `test_session_repository_workflow`, `test_session_repository`, session mutation/list/ - documents/schema-linking, CTE plan/next, decision phase gate, and phase requirement tests. -- Gate suite: `127 passed`, including - `session-repository-writes.test.js`. -- Changed-source Ruff checks pass. `git diff --check` passes. +Tests were written before `Run` existed. In the official `golang:1.26.5` container, mounted +against only the dedicated worktree, the focused RED run was: -## Implemented boundary +```text +go test ./internal/command -count=1 +internal/command/command_test.go:214:10: undefined: Run +FAIL +``` -- Added `resolve_principal`: PostgreSQL session storage requires trusted - `THT_PRINCIPAL_ISSUER` and `THT_PRINCIPAL_SUBJECT`, optional display name, and - strict admin parsing (`1`/`true`). It fails closed and never substitutes a local - identity. Filesystem storage uses `local_principal()`. -- Filesystem repository creates UUIDv4 sessions only and permits safe historical - timestamp IDs (`YYYY-MM-DD-HHMMSS`) for read/mutate compatibility. PostgreSQL - remains UUIDv4 only. -- Phase helpers fold `SessionSnapshot` ledger/artifacts; decision, phase, CTE, - session mutation/list/document paths, retrieval-pack persistence, SQL promotion - lookup, and task-doc/CTE test helpers gained repository/snapshot paths. -- Finalization now publishes report, evidence, and finalized manifest through - `repository.finalize`: one PostgreSQL transaction; filesystem writes artifacts - before the finalized manifest commit marker. Solved-question indexing stays - best-effort after this durable write. -- Added `tht cte save --session --name --file -` and - `tht sql set-final --session --file -`; Pi tools and SKILL.md now use them. +After implementation and formatting: -## Outstanding in-scope migration work +```text +go test ./internal/command -count=1 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/command -Do not treat this task as complete yet. Remaining direct session path consumers are: +go test -race ./internal/command -count=1 +ok github.com/aritmolab/thothii/tools/dwh-auth/internal/command -- `harness/tht/cli/memory_cmd.py`: lines 60, 93, 165, 400, 458. -- `harness/tht/cli/sql_cmd.py`: `_session_sql_file` at line 254 remains a legacy - Path-returning bridge for preview/save/export. -- `harness/tht/cli/session_cmd.py:session_dir` remains only as a compatibility - bridge for the out-of-scope datamart command and the still-unmigrated memory/ - SQL consumers; workflow mutations in session_cmd do not call it. +go test ./... -count=1 +ok internal/command, internal/credential, internal/record, internal/registry, internal/securefile -The full Python suite has not been conclusively re-run to completion after the -latest changes. An earlier root-directory invocation failed only because a -pre-existing test expects `workflow.yaml` relative to `harness/`. Full gate tests -are green. Full-repo Ruff currently fails on pre-existing test-file lint findings; -changed-source Ruff passes. +go test -race ./... -count=1 +ok internal/command, internal/credential, internal/record, internal/registry, internal/securefile + +go vet ./... +``` + +## Contract coverage + +- Create generates the Task 1 canonical credential, writes it once through the protected + exclusive `0600` output primitive, syncs/closes it before registry publication, and emits + only `created key_id=... installation_id=... output=...`. +- Existing output is never overwritten. Publication failure attempts compensating removal; + cleanup uncertainty returns exit 4 and reports only the output path. +- Legacy import requires `--legacy-raw`, the reserved `legacy-shared` installation ID, and an + absolute exact-`0600` source. It verifies the opaque value, never changes its source, and + stores only its digest. +- List/status expose `PublicRecord` data only; JSON is written as pristine JSON with no digest. + Revoke requires a non-empty reason and reports only its public key ID. +- Relative paths, malformed/unknown flags, duplicate options, invalid IDs/metadata/expiry, + and missing required values return exit 2. Missing status/revoke keys return exit 3. + Registry/filesystem/integrity failures return exit 4. +- Diagnostics are fixed redacted strings. Tests use a sentinel secret and assert it is absent + from stdout/stderr, list/status/check JSON, import output, and unsafe/integrity failures. + +## Secret-redaction evidence + +The command never prints credential contents or digests. It does not use environment fallback, +interactive stdin, or `flag` diagnostics that echo argument values. The sentinel appears only +in synthetic temporary test input and an integrity-fixture file; all command output assertions +confirm it is absent. The registry’s existing `PublicRecord` contract omits `secret_sha256`. + +## Concerns + +- `serve` is grammar-reserved and returns a redacted exit-4 unavailable response; Task 4 owns + the Unix-socket service implementation and will wire this dispatch. +- The earlier concern about UTF-8 metadata hardening is superseded by `055dcab`: metadata now + rejects invalid UTF-8 and Unicode controls before generation/output. The nil-safe cleanup note + remains non-blocking and outside this review wave. + +## Review-fix wave + +Review findings were addressed in separate commit `055dcab`. Regression tests were added first. The focused RED run in the official Go 1.26.5 container failed on intentionally absent seams: + +```text +undefined: nowUTC +undefined: addRecord +undefined: closeStore +FAIL github.com/aritmolab/thothii/tools/dwh-auth/internal/command +``` + +The fix rejects embedded canonical v1 credentials in description/revocation reason without echoing metadata, validates UTF-8/Unicode controls and expiry against one captured UTC creation time before generation/output, reserves exactly `serve --registry-root ABS --socket ABS`, and makes publication cleanup depend on a definitive registry lookup. Output is retained after publication or close ambiguity, with path-only recovery guidance. + +Review-fix verification in Go 1.26.5: + +```text +go test ./internal/command -count=1 PASS +go test -race ./internal/command -count=1 PASS +go test ./... -count=1 PASS +go test -race ./... -count=1 PASS +go vet ./... PASS +git diff --check PASS +``` + +New tests cover synthetic canonical credentials embedded with prefix/suffix, invalid UTF-8, C1 Unicode controls, past/equal/future expiry, exact serve ordering, deterministic pre-/post-publication and close-failure seams, and sentinel absence from stdout/stderr/list/status JSON. + +## Cleanup snapshot review-fix wave + +The second re-review added two regression tests before implementation. The RED run in the +official Go 1.26.5 container showed the old `Find` proof incorrectly treated both cases as +cleanup-safe: + +```text +FAIL TestCreateRetainsOutputWhenSnapshotFindsUnrelatedIntegrityFailure + corrupt snapshot result = (4, "", "integrity failure\n") +FAIL TestCreateRetainsOutputWhenFailedPublicationRecordIsExpired + expired publication result = (4, "", "integrity failure\n") +``` + +Commit `b1079bd fix: retain DWH key output on ambiguous publication` replaces the `Find` proof +with a complete `Store.List()` snapshot. It removes generated output only when the snapshot +succeeds, the generated key ID is absent, and `Store.Close()` succeeds. Any unrelated integrity +error, active/revoked/expired record, or close error retains the output and emits only path-based +recovery guidance. The clean pre-publication failure path still removes the output. + +Final cleanup-wave verification in Go 1.26.5: + +```text +go test ./internal/command -count=1 PASS +go test -race ./internal/command -count=1 PASS +go test ./... -count=1 PASS +go test -race ./... -count=1 PASS +go vet ./... PASS +git diff --check PASS +``` diff --git a/.superpowers/sdd/task-4-report.md b/.superpowers/sdd/task-4-report.md index 0ac3a0ad..a7f811b0 100644 --- a/.superpowers/sdd/task-4-report.md +++ b/.superpowers/sdd/task-4-report.md @@ -1,55 +1,60 @@ -# Task 4 report — one-command Docker documentation +# Task 4 — Unix-socket DWH verification report -## Status +## Scope -Implemented. The installation documentation now uses the canonical flow: +Implemented the standalone Linux verifier at `tools/dwh-auth/internal/service` and wired the exact command: -```sh -cp .env.example .env -cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets -chmod 600 deploy/secrets/thothii.secrets -docker compose up --build -d +```text +dwh-auth serve --registry-root ABSOLUTE_CANONICAL --socket ABSOLUTE_CANONICAL ``` -Updated: +The service accepts only `GET /verify`. It returns empty `204` responses with `X-DWH-Key-ID` for verified v1 or reserved legacy credentials; credential failures are generic empty `401` responses, and registry/integrity faults are empty `503` responses. Other paths/methods return empty `404`/`405`. -- `README.md` with root `.env` defaults, one bundle, optional overlay presets, CA limitation, - preprocessing, and migration notes. -- `docs/installazione-docker-4-contesti.md` rewritten with exact files to create/edit and the - four requested contexts (co-located DB/vector, Mac, Windows, and remote DB/Evidence server). -- `docs/index.md` link text for the one-command installation. -- `deploy/secrets/README.md` bundle syntax, permissions, runtime mount verification, CA handling, - and migration guidance. -- `scripts/docker-smoke.sh` now creates a disposable mode-0600 bundle and exercises the default - Compose services without the legacy `external` profile. -- `scripts/test-default-compose.sh` asserts the exact installation command, tracked templates, - and absence of the legacy setup in the guide. -- `scripts/test-container-deployment.sh` now validates the bundle mount and rejects legacy - per-secret references; `.dockerignore` explicitly re-includes only the required vector policy - helper so the Docker build context remains safe. -- The Mac/Windows/local-vector and remote-server snippets now include required DWH/database and - Evidence-root settings. `deploy/env.example` is explicitly deprecated and no longer selects a - different Compose overlay. +## Security decisions -The docs explicitly state that a PEM CA chain cannot be put in the strict single-line bundle. A -reviewed Compose override/secret-manager mount is required for `THT_SSL_CA`. Direct PostgreSQL -workspace examples are marked as advanced and require a separate reviewed runtime password mount; -the base bundle mount is the only default mount. +- Exactly one `X-API-Key` header, maximum 128 bytes. +- Strict `thtdwh_v1.` parsing precedes legacy lookup; non-v1 values alone may use the reserved legacy record. +- Registry integrity is checked before every verification request, so unrelated malformed/unsafe records fail closed with `503`. +- Logs emit only timestamp, decision code, and (when safely parsed or verified) public key ID; test sentinels prove no key, digest, description, or query value is emitted. +- `serve` validates canonical absolute paths, performs startup `Store.Check`, and reports service startup errors as non-secret `integrity failure`. +- Socket collisions that are regular files, directories, symlinks, live sockets, or foreign-owned stale sockets are refused. Only an owned stale Unix socket after `ECONNREFUSED` can be reclaimed. +- Published sockets are mode `0660`; cancellation calls graceful shutdown and removes only a revalidated same-device/same-inode owned socket. A test seam proves a changed path is retained rather than unlinked. + +## Required supporting security fix + +Commit `943f809` (`fix: reject duplicate legacy DWH records`) tightens the Task 2 registry contract: a synthetically valid active plus revoked legacy pair is now an integrity failure. It is intentionally separate from the Task 4 commit. + +## TDD evidence + +RED was observed for the missing handler, listener/configuration API, CLI wiring, unrelated-registry corruption, active+revoked legacy state, and cleanup replacement race. Each increment was then implemented minimally and rerun GREEN. ## Verification -- `sh -n scripts/docker-smoke.sh scripts/test-default-compose.sh` — passed. -- `./scripts/test-default-compose.sh` — passed. -- `./scripts/test-container-deployment.sh` — passed after migrating its local-vector assertions - to the single bundle and checking the `.dockerignore` deployment allowlist. -- `git diff --check` — passed. -- `./scripts/test-docker-smoke.sh` — passed after updating its static assertion to the default - no-profile invocation. -- `docker buildx build --file docker/core.Dockerfile --check .` — passed; BuildKit reported no - warnings after the `.dockerignore` parent-directory fix. +All commands were executed in official `golang:1.26.5`, with only this worktree mounted: -## Concerns +```text +gofmt -w cmd internal/command internal/service +go test ./internal/service ./internal/command -count=1 +go test ./... -count=1 +go test -race ./... -count=1 +go vet ./... +git diff --check +``` -The legacy `scripts/vector-rotate-bootstrap-password.sh` maintenance helper still accepts -old/new standalone files. Its output is intentionally documented as a transitional interface; -the resulting value must be copied into the bundle before restarting local-vector services. +All passed. A dependency scan also found no third-party Go dependencies. + +## Scope boundary + +No Nginx, systemd, real Unix socket, real registry, credential, legacy stack, or external service was changed. All test data was synthetic and temporary. + +## Follow-up hardening: runtime read-only registry and socket parent + +The Task 5 storage contract uses `root:dwh-auth` SGID directories (`2750`) and a service account with read-only group access. The original registry reader path was incompatible because shared locks were opened `O_RDWR` and lazily created as `0600`; secure-directory validation also rejected SGID. + +The runtime path now uses `registry.OpenReadOnly`: it opens only preprovisioned root, `active`, `revoked`, and `.writer.lock` paths, and rejects `Add`/`Revoke`. The administrative `Open` path bootstraps the lock through the exclusive writer path. Shared lock acquisition opens the existing `root:dwh-auth 0640` lock `O_RDONLY` with `LOCK_SH`; writer acquisition remains `O_RDWR` with `LOCK_EX`, preserving cross-process snapshot exclusion. Secure directories allow SGID but still reject setuid, sticky, group-write, and world-write bits. + +Task 5 must create `.writer.lock` as `0640 root:dwh-auth` alongside the `2750 root:dwh-auth` registry directories before the service starts. + +The socket parent must be a canonical non-symlink directory owned by the service EUID and not group/world writable. This removes the bind-to-chmod and path-replacement exposure from other principals. The remaining POSIX path race is bounded to trusted processes sharing the service EUID inside that non-contendible parent. + +Additional verification (official `golang:1.26.5`, worktree only): focused securefile/registry/service/command tests, full tests, full race tests, vet, plus ten race repetitions each for cross-store snapshot readers, `OpenReadOnly`, and listener tests: all PASS. diff --git a/.superpowers/sdd/task-6-report.md b/.superpowers/sdd/task-6-report.md index 88b7af75..38267610 100644 --- a/.superpowers/sdd/task-6-report.md +++ b/.superpowers/sdd/task-6-report.md @@ -63,3 +63,236 @@ before and after switching. - Remediation verification: focused regressions passed; full frontend Vitest (44 files / 305 tests), `npx tsc -b`, `npm run build`, and `git diff --check` all passed. + + +--- + +# DWH authentication Task 6 — Nginx and CI gate report + +## Scope + +Added only the two DWH-auth Nginx gates and the `dwh-auth-linux` deployment workflow job: + +- `scripts/test-dwh-auth-nginx-contract.sh` +- `scripts/test-dwh-auth-nginx-integration.sh` +- `.github/workflows/deployment.yml` + +This report deliberately remains unstaged. The pre-existing frontend Task 6 report above is +preserved rather than overwritten. + +## TDD RED + +The structural gate was written before any Task 5 template change. Those templates already met +the approved contract, so the behavioral RED was obtained by copying them into one exact temporary +root and removing only the effective `/dwh/` `auth_request` directive. The new checker failed as +required, with no credential material in output: + +```text +case=source_contract status=FAIL +``` + +The runtime gate was also first invoked before its file existed: + +```text +bash: scripts/test-dwh-auth-nginx-integration.sh: No such file or directory +``` + +The CI-job RED check found no `dwh-auth-linux` job in `deployment.yml`. No production template was +modified: the tests prove the existing Task 5 template contract instead of weakening it. + +## GREEN + +Shell syntax and workflow YAML were checked with: + +```text +bash -n scripts/test-dwh-auth-nginx-contract.sh scripts/test-dwh-auth-nginx-integration.sh +python3 -c import-yaml-and-safe-load +``` + +The structural gate passed its source contract plus these 13 real copied-and-mutated Nginx fixtures: + +```text +missing_auth_request +missing_proxy_method +missing_proxy_body +missing_proxy_header_isolation +missing_content_length_clear +missing_verifier_key_forward +missing_upstream_key_clear +missing_failure_mapping +public_verifier +tcp_authenticator +postgrest_bypass +failure_mapped_to_success +full_secret_rate_key +``` + +Each test mutates an effective, not comment-only, directive and requires the checker to reject it. +The source test and all 13 fixture tests emitted `case=... status=PASS`, followed by +`case=summary status=PASS`. + +The isolated Nginx 1.24 smoke passed these sanitized cases: + +```text +nginx_1_24 +build_dwh_auth +registry_setup +verifier_start +synthetic_upstreams +composite_nginx_config +nginx_start +auth_socket_unix_only +verifier_not_public +valid_v1 +valid_legacy +invalid_key +revoked_key +expired_key +duplicate_v1 +duplicate_legacy +stopped_verifier +header_and_path_isolation +summary +``` + +It builds with the pinned official Go 1.26.5 image when the host Go binary is absent, creates only +synthetic v1, legacy, revoked, and expired credentials in a `0700` `/tmp` root, runs both Nginx and +the verifier on explicit temporary Unix sockets, and uses a loopback-only marker backend. Its output +is strictly `case` and `status`; keys, values, and digests remain only in the exact temporary root +and are removed by the trap. + +`nginx -t` passed against the complete generated configuration. The marker proves that successful +`/dwh/?keep=exact&second=two` reaches the upstream unchanged, while neither the client API key nor +client or verifier `X-DWH-Key-ID` reaches it. A Unix forwarding probe proves that the verifier sees +only `X-API-Key`, with Cookie, Authorization, and spoofed audit ID absent. Duplicate v1 and ordinary +legacy headers return 401 through Nginx; a stopped verifier returns 503. + +The final local equivalent of the four CI commands passed: + +```text +Docker Go 1.26.5: go test -race ./... -count=1 and go vet ./... +bash scripts/test-dwh-auth-build-contract.sh +bash scripts/test-dwh-auth-nginx-contract.sh +bash scripts/test-dwh-auth-nginx-integration.sh +``` + +The Go race suite passed for command, credential, record, registry, securefile, and service; +`go vet` was silent; the build contract passed; both Nginx gates reached their summaries. + +## CI contract + +The new job uses `actions/checkout` with `persist-credentials: false`, pins Go 1.26.5 with cache +keyed on `tools/dwh-auth/go.mod`, installs `nginx-light`, and runs exactly the four required commands. +Existing jobs were not altered. + +## Self-review + +- The template tests parse normalized effective directives, so commented-out declarations cannot + satisfy the gate. +- The authentication socket is configured as `http://unix:...:/verify`, is observed by `ss -xl`, + and Nginx itself listens only on a temporary Unix socket; neither test starts a public listener. +- All spawned processes are registered by PID; cleanup signals only those PIDs and deletes only the + exact `mktemp` root after a guarded path check. +- The verifier, marker, registry, Nginx prefix, PID, logs, config, and sockets all reside beneath + that root. No `/etc`, systemd, active Nginx config, stack, legacy route, or real registry/key is + read or changed. +- Task 5 templates were not modified because the structural and runtime tests passed unchanged. + +## Concern + +The sandbox `apply_patch` helper repeatedly failed with `bwrap: loopback: Failed RTM_NEWADDR: +Operation not permitted`. A narrowly scoped fallback editor was used only for the workflow and the +Nginx-version assertion. Its first workflow insertion interpreted the action-reference at signs; +the two malformed values were immediately corrected and all final YAML, exact-string, syntax, and +four-command checks were rerun. No remaining product concern is known; the integration gate requires +Nginx 1.24 and Python 3, both supplied by the specified Ubuntu CI runner. + + +--- + +# DWH authentication Task 6 — review remediation wave + +## Review findings and RED evidence + +The three review findings were reproduced against the Task 6 commit before their corresponding +hardening was accepted. + +1. The contract checker originally selected only the first matching `/dwh/` location. A real copied + fixture appended this competing location without authentication: + +```nginx +location ~ ^/dwh/ { + proxy_pass http://127.0.0.1:3001; +} +``` + + The first run reached the new check and failed as required: + +```text +case=negative_postgrest_regex_bypass status=FAIL +``` + +2. The previous process stop sent TERM and immediately used an unbounded `wait`. A synthetic Python + child ignored TERM; the RED run used one exact short-lived watchdog only to prevent a test hang and + produced: + +```text +case=cleanup_term_ignored_bounded status=FAIL +``` + +3. The TCP detector has a positive-control regression. A scratch copy of the integration script + replaced its `ss -ltnpH` detector with `return 1`; its known loopback listener was then not + detected and the run failed with: + +```text +case=tcp_listener_detector_positive status=FAIL +``` + +All RED fixtures and the scratch script used an exact temporary path and were removed. No template, +service, workflow, key, or active Nginx configuration was changed. + +## GREEN changes + +- `location_declarations` consumes normalized, comment-stripped effective lines and `check_templates` + requires exactly one each of the only approved locations: verifier, unavailable named location, and + `/dwh/`. It therefore rejects both any extra intercepting location and a duplicate. The real regex + bypass and a new real duplicate `/dwh/` bypass fixture both pass by being rejected. +- `tcp_listener_for_pid` uses `ss -ltnpH` and a PID-bound match. The integration gate starts a + loopback-only synthetic listener, proves the detector sees that exact PID, stops and deregisters it, + then proves the verifier PID has no TCP listener while its Unix socket remains present. +- `stop_registered_pid` now sends TERM, polls for exit or zombie for a bounded deadline, sends KILL + if required, polls a second bounded deadline, and only reaps a direct child after terminal state is + proved. Explicit stops deregister their PID. The cleanup loop invokes that bounded operation only + for recorded PIDs and removes only its guarded temporary root. +- The synthetic child that ignores TERM is killed by the bounded path, must no longer answer to + `kill -0`, must not remain registered, and must finish within three seconds. Final gate output is + restricted to `case` and `status` lines. + +## GREEN verification + +```text +bash -n scripts/test-dwh-auth-nginx-contract.sh scripts/test-dwh-auth-nginx-integration.sh +Docker Go 1.26.5: go test -race ./... -count=1 and go vet ./... +bash scripts/test-dwh-auth-build-contract.sh +bash scripts/test-dwh-auth-nginx-contract.sh +gate contract: source plus 15 negative fixtures PASS, then summary PASS +bash scripts/test-dwh-auth-nginx-integration.sh +gate integration: 20 named cases PASS, then summary PASS +git diff --check +``` + +The integration cases include `cleanup_term_ignored_bounded`, +`tcp_listener_detector_positive`, `auth_socket_unix_only`, all existing credential decisions, +composite Nginx syntax, and stopped-verifier 503 behavior. Go race tests passed for command, +credential, record, registry, securefile, and service; vet and both diff checks were silent. + +## Self-review and concern + +The new location parser rejects comment-only and non-exact declarations because it operates on the +same normalized effective representation used by the rest of the contract. The TCP positive control +binds only `127.0.0.1` on a kernel-selected temporary port and is stopped through the same exact-PID +path under test. The bounded cleanup avoids arbitrary process lookup or broad signaling. + +The environment still intermittently rejects `apply_patch` with the sandbox loopback error noted in +the original report; only narrowly scoped fallback edits to the two authorized scripts were used and +all final gates were rerun. No remaining review concern is known. diff --git a/.superpowers/sdd/task-7-report.md b/.superpowers/sdd/task-7-report.md index ca384282..7ebb7998 100644 --- a/.superpowers/sdd/task-7-report.md +++ b/.superpowers/sdd/task-7-report.md @@ -1,79 +1,90 @@ -# Task 7 report — deployment contract and user-owned-session cutover +# Task 7 — report -## Scope +## RED -Implemented the deployment contract only. No Supabase migration, portal change, live-stack -restart, session archive, or deletion was run. +- Creato `scripts/test-verify-dwh-auth-docs.sh` con fixture positiva e fixture negative per + credenziale/digest sintetici, TLS insicuro, segreto in env/argv, mode world-readable, cattura + Nginx e coupling Compose. +- Eseguito `bash scripts/test-verify-dwh-auth-docs.sh` prima del verificatore: `case=verifier_missing status=FAIL`. -- `backend/src/config.ts` now makes the session-store deployment mode explicit. `local` is the - default and cannot be publicly exposed. `postgres` requires `AUTH_MODE=upstream`, direct DB - host/name/runtime user, an absolute runtime-password file, `verify-ca` or `verify-full`, and an - absolute CA path. -- `docker-compose.dev.yml` now publishes only loopback ports and explicitly selects local - session storage rooted at `/data/local-home`. -- `deploy/compose.session-server.yaml.example` separates the runtime and one-shot migrator - secrets. The core gets only `session_runtime_password` and the CA; the profile-gated - `session-migrate` service gets only `session_migrator_password` and the CA. -- `deploy/workspaces/server-sessions.yaml.example` binds the runtime repository to the - TLS-verified direct PostgreSQL configuration. The runtime password remains a file reference. -- `docker/cutover-legacy-sessions.sh` archives/checksums exactly three reviewed legacy sessions - and requires an explicit `--delete` rerun before deleting them. -- README, secret guidance, environment examples, and PROJECT_STATE describe the maintenance - sequence, Task 4+5 coordinated rollout, liveness vs storage 503 behavior, and the no-dual-write - rollback rule. +## GREEN -## TDD evidence +- Aggiunti manuali server, client, TLS, runbook PSD, collaudo ed evidenza sanitizzata; collegati + manuali locali/server, setup PSD, guida, indice e nav MkDocs. +- Eseguiti: `bash -n scripts/verify-dwh-auth-docs.sh scripts/test-verify-dwh-auth-docs.sh`, + `bash scripts/test-verify-dwh-auth-docs.sh`, `bash scripts/verify-dwh-auth-docs.sh`, + `bash scripts/test-verify-workspace-install-docs.sh`, `bash scripts/auth-docs-smoke.sh`. +- Tutti gli output finali sono PASS; il nuovo gate esercita una fixture positiva e nove negative. -RED was established with: +## Self-review -```sh -cd backend && npx vitest run test/config.test.ts -``` +- Verificati path/owner/mode: registry 2750, lock/record 0640, socket 0660. +- Verificata separazione: chiavi solo `rest_api`; PSD server `postgres_direct`; Mac/remoti REST; + nessun lifecycle Compose per `dwh-auth`. +- Verificati TLS `.it`/SAN, `.com` non coperto, `TLS_CA_FILE`, fingerprint fuori banda, rinnovo e + assenza di bypass. +- Verificati due gate Task 9–10, evidenze solo metadati e nessuna migrazione di sessioni/index/cache legacy. -The new tests failed because `sessionStorage` did not exist and public/local and unauthenticated -server combinations were accepted. After implementing the minimal configuration contract, the -same focused suite passed (7 tests). Updating the existing upstream-health fixture to supply the -now-required server inputs confirmed that `/health` remains an unauthenticated `200` liveness -endpoint under the valid server contract. +## Concern -## Verification +- Nessuna mutazione PSD/Nginx/systemd/registry o lettura di segreti è stata eseguita. I comandi del + runbook restano condizionati alle autorizzazioni separate dei Task 9 e 10. -```text -cd harness && .venv/bin/pytest -q -826 passed, 5 deselected, 67 warnings in 63.01s -cd backend && npx vitest run && npx tsc --noEmit -p . && npm run build -22 files / 215 tests passed; TypeScript check and production build passed +## Review fix — RED/GREEN -cd frontend && npx vitest run && npx tsc -b && npm run build -full Vitest suite, TypeScript build, and Vite production build passed -``` +### RED review -The frontend gate retained its pre-existing React-ref/MSW/act warnings and Vite chunk-size warning; -none caused a test or build failure. +- La fixture `sudo nginx -T` ha prodotto il rifiuto `case=sudo_raw_nginx_capture status=FAIL` prima della correzione del gate. +- La fixture header legacy opaco ha prodotto `case=opaque_legacy_header_literal status=FAIL` prima della correzione del gate. +- Dopo avere riallineato le label UI nei manuali, `bash scripts/test-verify-workspace-install-docs.sh` ha prodotto `server-workspace-registry.md: curator flow missing registry rule`: il verifier cercava ancora le due label precedenti. Il test sulla base HEAD e il diff hanno confermato la causa. -Additional static validation passed: +### GREEN review -```text -docker compose config --quiet (base plus copied session-server overlay with temporary empty secrets) -bash -n docker/cutover-legacy-sessions.sh -git diff --check -``` +- Il gate DWH ora rifiuta anche header opaco, digest JSON quotato, `export` di API key, `curl --header` e `-H`, `sudo nginx -T`, raw diff e Compose; le mutation fixture coprono label, PSD direct/Mac REST/CA, socket e flag REST. +- Il runbook non prescrive raw diff o dump: solo checker strutturale e secret scan con metadati e PASS/FAIL. Il piano Task 10 adotta la stessa regola. +- Il template `psd-local` resta `rest_api` solo Mac/local/remota; il server PSD Project A resta `postgres_direct` con binding separato. La CA privata e `TLS_CA_FILE` sono obbligatori salvo trust approvato equivalente. +- Le procedure server ora coprono backup manifest protetto, restore, curl config 0600 senza segreto in argv/env/output, Unix 204/401, HTTPS 2xx/401, 503 bounded con trap, journal PASS/FAIL e retention alla disinstallazione. +- Il verifier workspace-install e entrambi i manuali registry usano ora le quattro label effettive: `Validate workspace source`, `Test workspace connections`, `Save entered secrets`, `Forget stored value`. -## Manual gate remaining +### Final verification review -An operator must still choose the three reviewed legacy IDs, materialize real runtime/migrator/CA -secrets, deploy Task 4 and Task 5 together in a maintenance window, apply the one-shot migrator, -and run the documented authenticated smoke. The guarded helper has not been invoked with -`--delete`. +- PASS: `bash scripts/test-verify-dwh-auth-docs.sh`. +- PASS: `bash scripts/verify-dwh-auth-docs.sh`. +- PASS: `bash scripts/test-verify-workspace-install-docs.sh` (fixture complete). +- PASS: `bash scripts/auth-docs-smoke.sh`. +- PASS: `bash -n scripts/verify-dwh-auth-docs.sh scripts/test-verify-dwh-auth-docs.sh` e `git diff --check`. -## P1 correction — migrator TLS validation +### Review concern -The original migrator Compose command interpolated `THT_SESSION_DB_SSLMODE` into its URL without -checking it. `docker/session-migrate.sh` now rejects every value except `verify-ca` and -`verify-full` before reading the password file or building that URL; the Compose service invokes -this helper. `docker/session-migrate.test.sh` first established RED because the helper did not -exist, then verified that `prefer` is rejected before `tht` can run and that `verify-full` reaches -a fake `tht` binary with the expected TLS URL. The helper and test pass `bash -n`; the focused -backend config/health suite remains green, and the base-plus-overlay Compose configuration renders -with temporary empty secret files. +- Nessuna mutazione runtime e nessun segreto reale sono stati letti. I soli comandi server documentati restano soggetti ai gate autorizzativi Task 9 e Task 10. + +## Review fix wave 2 — RED/GREEN + +### RED wave 2 + +- Prima della correzione del proxy, `bash scripts/test-dwh-auth-build-contract.sh` ha fallito il contratto di preservazione path e `bash scripts/test-dwh-auth-nginx-integration.sh` ha chiuso con `case=header_and_path_isolation status=FAIL`: il prefisso `/dwh` arrivava a PostgREST invece di essere rimosso. +- Prima delle procedure finali, il gate docs ha rifiutato il path chiave non deterministico e la fixture curl con header legacy opaco ha dato `case=header_file_curl_synthetic status=FAIL` perché il valore non veniva confrontato esattamente. +- Le mutation fixture hanno catturato l'estrazione tar sul registro attivo e i rename non protetti. Dopo l'inasprimento finale del gate, la sorgente ha dato `dwh-auth docs: restore must stage/check then use guarded same-filesystem renames` finché mancava il controllo fail-closed del candidato. +- Il RED finale dello scanner journal è stato `dwh-auth docs: docs/install/dwh-auth-server.md lacks required topic: sys.argv[2:]`: il gate esige la lettura byte-esatta di v1 e legacy e un `journalctl` che fallisca chiuso. + +### GREEN wave 2 + +- Commit `f616aab fix: preserve PostgREST RPC path through DWH proxy`: `proxy_pass` termina con `/`; il contratto e l'integrazione verificano `/dwh/rpc/ping?x` verso `/rpc/ping?x`. +- Il runbook usa un singolo file chiave v1, header file `0600` passati solo con `curl --header @file`, socket 204 dual-key, HTTPS 2xx pre/post per v1 e 401 post-revoca per legacy `legacy-shared`. +- Restore protetto: staging sul filesystem `/var/lib`, check candidato, `mv -T --` guardato per ogni publish/rollback e pre-restore conservato. Backup/manifest restano root-only `0600` su storage cifrato approvato. +- Lo scanner journal esegue `journalctl` in un unico processo Python root, sopprime stderr, controlla return code e bytes esatti di entrambe le chiavi senza emettere journal o segreti; la shell mostra solo PASS/FAIL. +- Il verifier rifiuta `curl --config`, header in argv, raw Nginx/diff, TLS insicuro, segreti env, mode insicuri e Compose. Le fixture mutano path chiave, ID legacy, header/legacy probes, restore, journal, codici HTTPS e label UI. + +### Final verification wave 2 + +- PASS: `bash scripts/test-dwh-auth-build-contract.sh`. +- PASS: `bash scripts/test-dwh-auth-nginx-contract.sh`. +- PASS: `bash scripts/test-dwh-auth-nginx-integration.sh`. +- PASS: `bash scripts/test-verify-dwh-auth-docs.sh` e `bash scripts/verify-dwh-auth-docs.sh`. +- PASS: `bash scripts/test-verify-workspace-install-docs.sh` e `bash scripts/auth-docs-smoke.sh`. +- PASS: `bash -n` sugli otto gate shell e `git diff --check`. + +### Review concern wave 2 + +- Nessuna configurazione protetta, chiave reale, Nginx, systemd o stack PSD è stata letta o mutata. Le procedure privilegiate restano istruzioni condizionate ai Gate 9–10; la verifica degli owner/mode reali è un'attività del rollout autorizzato, non di questo task documentale.