docs: define staged Datamart Builder cutover

docs/superpowers/specs/2026-08-22-datamart-builder-cutover-design.md

@@ -0,0 +1,148 @@
+# Datamart Builder — staged cutover to the accepted ThothII release
+
+Date: 2026-08-22
+
+## Outcome
+
+Promote the ThothII release already accepted at `/datamart-builder-test/` to the canonical
+`/datamart-builder/` entry point, retain a reversible checkpoint for the first visual test, then
+remove the legacy release and the temporary test route only after the two explicit operator gates.
+
+The completed topology has one portal menu entry (`Datamart Builder`), one canonical route
+(`/datamart-builder/`), and one ThothII Compose application running the accepted release.
+
+## Preconditions and authority
+
+- The new release has already passed browser login through the portal/Authentik path.
+- Workspace discovery, `psd-clinical`, connector diagnostics, DeepSeek, Pi, and session creation
+  have been exercised successfully on the test route.
+- The operator has explicitly authorized the staged cutover and, after the first visual approval,
+  deletion of the disconnected legacy containers and images because no legacy state must be
+  retained.
+- The operator has chosen to keep `/datamart-builder-test/` active during the first visual
+  checkpoint as a rollback route.
+
+This authority does not extend to shared infrastructure: Omics/LocalLLM networks, DWH,
+`dwh-auth`, Supabase, Authentik, Superset, Aritmolab, portal secrets, and unrelated containers or
+images are never cleanup targets.
+
+## Current topology
+
+The portal currently exposes two independent paths:
+
+- `/datamart-builder/` reaches the legacy `thothii` Compose project (`thothii-core` and
+  `thothii-frontend`).
+- `/datamart-builder-test/` reaches the accepted `thothii-test` project
+  (`thothii-test-core` and `thothii-test-frontend`).
+
+Both paths use the portal's existing Authentik admission flow. Nginx forwards the normalized
+identity headers to the selected backend. The accepted release uses the server workspace and
+secret layout under the protected `/srv/thothii` installation; those runtime values are not copied
+into Git.
+
+## Phase 1 — reversible canonical switch
+
+Phase 1 changes routing only; it does not delete runtime resources.
+
+1. Update the canonical Django page and asset selection so `Datamart Builder` renders the accepted
+   frontend while retaining the canonical browser URL.
+2. Point the canonical Nginx API, static asset, runtime-config, and streaming paths to the accepted
+   `thothii-test` frontend/core upstreams.
+3. Preserve the existing Authentik subrequest and the exact normalized principal-header contract.
+4. Keep `/datamart-builder-test/` and its `Datamart Builder Test` menu entry operational.
+5. Leave the legacy `thothii` containers running but disconnected from canonical portal traffic.
+
+The runtime browser configuration must be path-aware during this temporary dual-route phase: the
+canonical page uses `/datamart-builder/api`, while the test page uses
+`/datamart-builder-test/api`. No frontend build may hard-code a server hostname or credential.
+
+### Phase 1 verification
+
+Before asking for the visual test:
+
+- run the portal Django tests covering the two pages, menu visibility, page titles, asset variants,
+  and Authentik behavior;
+- run the Nginx routing tests and validate the live Nginx configuration before reload;
+- validate the accepted Compose configuration and confirm frontend/core health;
+- probe both public paths without credentials and confirm that admission still redirects through
+  Authentik rather than bypassing it;
+- verify that the canonical API and asset requests now reach the accepted containers;
+- verify that the test route remains a working rollback path;
+- verify that the legacy containers receive no canonical application traffic after the switch.
+
+The operator then performs the first visual test at
+`https://aritmolab.policlinicosandonato.it/datamart-builder/`, including login, workspace selection,
+connector check, and creation of a short disposable session.
+
+### Phase 1 rollback
+
+Until the first visual approval, rollback consists only of restoring the prior Django/Nginx route
+selection and reloading validated configuration. The legacy containers remain intact, so no image
+rebuild or data recovery is required.
+
+## Phase 2 — canonicalize the accepted stack and remove legacy/test topology
+
+Phase 2 starts only after the operator explicitly approves the first visual test.
+
+1. Capture an exact inventory of the legacy `thothii` project resources, image IDs, mounts,
+   networks, and labels. Resolve cleanup targets by Compose project labels and exact IDs, never by
+   a broad name pattern or recursive directory deletion.
+2. Stop and remove only the legacy `thothii` core/frontend containers and their obsolete images.
+   Remove no shared volume or network. Remove an old source/config directory only if the inventory
+   proves that the accepted installation does not mount or reference it and the target is entirely
+   legacy.
+3. Recreate the accepted release under the canonical `thothii` Compose identity. It continues to
+   consume the approved `/srv/thothii` installation, workspace registry, authentication material,
+   and external service configuration; it must not inherit old `/home/chirone/ThothII` runtime
+   state.
+4. Keep the accepted `thothii-test` project available until the canonical replacement is healthy
+   and the canonical Nginx route has passed automated probes. This bounds the promotion downtime
+   and preserves a known-good recovery target during recreation.
+5. Point canonical Nginx upstreams at the canonical accepted containers, validate and reload.
+6. Remove the `Datamart Builder Test` menu item, Django URL/view/asset variant, Nginx test routes
+   and upstreams, test-only runtime configuration, and test Compose overlay.
+7. Stop and remove the now-redundant `thothii-test` containers and test-tagged images only after
+   confirming that canonical core/frontend are healthy and no longer depend on them.
+
+If canonical recreation or verification fails, leave the test project running and restore the
+canonical Nginx route to that accepted upstream. Do not continue cleanup while the canonical route
+is unhealthy.
+
+### Phase 2 verification
+
+Before asking for the second visual test:
+
+- confirm there is exactly one ThothII application project serving the portal;
+- confirm `/datamart-builder/` authenticates through Authentik and reaches the accepted release;
+- confirm `/datamart-builder-test/` and its menu entry are absent;
+- confirm workspace discovery, connector diagnostics, SSE/session creation, and model/provider
+  configuration still work;
+- confirm no legacy ThothII container or obsolete image remains;
+- confirm all shared services and networks are unchanged and healthy;
+- rerun the portal, Nginx, backend/frontend, installation, and focused acceptance gates appropriate
+  to the changed files.
+
+The operator then performs the second visual test on the canonical URL. No commit or push follows
+until that approval.
+
+## Source control and evidence
+
+After the second visual approval:
+
+- inspect both repositories independently (`ThothII-next` and `omics_portal`);
+- stage only the reviewed source/config/test/documentation files belonging to this work;
+- exclude protected environment files, `/srv` runtime state, API keys, generated session data,
+  local logs, temporary evidence, `.superpowers/sdd/progress.md`, and unrelated user changes;
+- run final tests and secret/diff checks on the exact staged sets;
+- create clear repository-specific commits and push only their current intended branches;
+- report commit hashes, pushed branches, verification evidence, and the exact runtime resources
+  removed.
+
+## Manual gates
+
+The execution intentionally pauses twice:
+
+1. after the reversible canonical route switch and before any legacy deletion;
+2. after legacy/test cleanup and before commit/push.
+
+Silence or a partial test is not approval. Each continuation requires an explicit operator result.