docs: plan PSD server deployment program
This commit is contained in:
@@ -0,0 +1,220 @@
|
||||
# PSD Server Deployment Program Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Replace the legacy PSD ThothII installation, prove the replacement with local authentication, and then integrate the accepted release with Supabase, Authentik, Nginx, the load balancer, and the Aritmolab sidebar.
|
||||
|
||||
**Architecture:** A read-only common survey freezes the actual server topology before any mutation. Project A installs a clean private stack and proves a complete PSD workflow; Project B begins only after a signed Project A PASS and performs the public OIDC/SSO cutover. Each project has an independent rollback boundary, human test guide, and evidence report.
|
||||
|
||||
**Tech Stack:** Linux, Docker Engine, Docker Compose v2, native `tht`, Fastify/React/Pi, PostgreSQL/Supabase, Qdrant, Ollama, Nginx, Authentik OIDC, Aritmolab, load balancer.
|
||||
|
||||
---
|
||||
|
||||
## Required reading and authority
|
||||
|
||||
Read these files completely before starting:
|
||||
|
||||
- `AGENTS.md`
|
||||
- `PROJECT_STATE.md`
|
||||
- `docs/plans/2026-08-20-psd-server-deployment-program-design.md`
|
||||
- `docs/install/server.md`
|
||||
- `docs/install/server-workspace-registry.md`
|
||||
- `docs/install/authentication-local.md`
|
||||
- `docs/install/authentication-oidc.md`
|
||||
- `docs/install/authentik.md`
|
||||
- `docs/contracts/workspace-preprocessing-cli.md`
|
||||
- `docs/testing/authentication-manual-acceptance.md`
|
||||
|
||||
The current `compose.yaml`, `deploy/compose.server.yaml`, repository instructions, and design are
|
||||
authoritative where older server prose still describes Qdrant or Ollama as external.
|
||||
|
||||
Run only from a terminal local to the server. Do not require SSH port forwarding. Do not print or
|
||||
paste passwords, tokens, cookies, private keys, hashes, or raw identity claims. Commands that need
|
||||
a credential must read a protected file or use an echo-free prompt.
|
||||
|
||||
## Documents used during execution
|
||||
|
||||
- Survey plan: `docs/plans/2026-08-20-psd-server-survey.md`
|
||||
- Survey report: `docs/testing/evidence/psd-server-survey-report-template.md`
|
||||
- Project A plan: `docs/plans/2026-08-20-psd-server-project-a-standalone.md`
|
||||
- Project A human guide: `docs/testing/psd-server-project-a-manual.md`
|
||||
- Project A report: `docs/testing/evidence/psd-server-project-a-report-template.md`
|
||||
- Project B plan: `docs/plans/2026-08-20-psd-server-project-b-authentik.md`
|
||||
- Project B human guide: `docs/testing/psd-server-project-b-manual.md`
|
||||
- Project B report: `docs/testing/evidence/psd-server-project-b-report-template.md`
|
||||
|
||||
The detailed evidence directory is a protected path on the server selected during the survey. The
|
||||
repository receives only redacted reports after explicit owner review.
|
||||
|
||||
### Task 1: Freeze the planning source
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/plans/2026-08-20-psd-server-deployment-program-design.md`
|
||||
- Record: protected server execution journal selected during the survey
|
||||
|
||||
**Step 1: Verify the application checkout**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git status --short --branch
|
||||
git rev-parse HEAD
|
||||
git rev-parse origin/main
|
||||
git show -s --format='%H%n%P%n%s' HEAD
|
||||
```
|
||||
|
||||
Expected: the tree is clean and `HEAD` is the explicitly approved `origin/main` SHA. A newer SHA
|
||||
than the design-time `5c0dc8c` is allowed only after recording and reviewing the intervening commits.
|
||||
|
||||
**Step 2: Verify the plan files exist at that SHA**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
test -f docs/plans/2026-08-20-psd-server-survey.md
|
||||
test -f docs/plans/2026-08-20-psd-server-project-a-standalone.md
|
||||
test -f docs/plans/2026-08-20-psd-server-project-b-authentik.md
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: every command exits zero.
|
||||
|
||||
**Step 3: Record the immutable planning identity**
|
||||
|
||||
Record the application SHA, plan commit, UTC timestamp, operator identity, and terminal-local access
|
||||
method in the protected journal. Do not record CyberArk session secrets or screenshots.
|
||||
|
||||
### Task 2: Execute and approve the common survey
|
||||
|
||||
**Files:**
|
||||
- Execute: `docs/plans/2026-08-20-psd-server-survey.md`
|
||||
- Create from: `docs/testing/evidence/psd-server-survey-report-template.md`
|
||||
|
||||
**Step 1: Execute every survey task without mutation**
|
||||
|
||||
Expected: the survey identifies exact paths and owners for the old and new installations, Nginx,
|
||||
load balancer, Aritmolab, Authentik, Supabase, the DWH, and protected credentials.
|
||||
|
||||
**Step 2: Resolve every unknown**
|
||||
|
||||
If an Authentik credential cannot be located, stop and ask the owner. If a configuration owner or
|
||||
rollback boundary is unclear, stop; do not infer authority from file readability.
|
||||
|
||||
**Step 3: Review the survey GO/NO-GO**
|
||||
|
||||
Expected: GO requires a verified old-stack recovery path, a new-installation root, enough resources,
|
||||
a read-only DWH path, and no unresolved shared-infrastructure mutation.
|
||||
|
||||
**Step 4: Checkpoint the survey**
|
||||
|
||||
Hash the protected report and record only its path, SHA-256, timestamp, and GO result in the journal.
|
||||
|
||||
### Task 3: Execute Project A
|
||||
|
||||
**Files:**
|
||||
- Execute: `docs/plans/2026-08-20-psd-server-project-a-standalone.md`
|
||||
- Complete: `docs/testing/evidence/psd-server-project-a-report-template.md`
|
||||
|
||||
**Step 1: Confirm the survey is GO**
|
||||
|
||||
Expected: the survey report hash matches the journal and no unresolved blocker remains.
|
||||
|
||||
**Step 2: Execute Project A task-by-task**
|
||||
|
||||
Do not configure Authentik, change the production Aritmolab sidebar, or open the production route.
|
||||
|
||||
**Step 3: Run the Project A human guide**
|
||||
|
||||
Follow `docs/testing/psd-server-project-a-manual.md`. Record PASS/FAIL for every case; do not infer
|
||||
manual PASS from automated output.
|
||||
|
||||
**Step 4: Close the Project A report**
|
||||
|
||||
Expected: automated gates and the human guide are PASS; one harmless PSD session reached F8 and
|
||||
produced validated read-only SQL; rollback remains available.
|
||||
|
||||
**Step 5: Obtain explicit owner approval**
|
||||
|
||||
Record the approval and report digest. Project B remains forbidden without it.
|
||||
|
||||
### Task 4: Freeze the Project B candidate
|
||||
|
||||
**Files:**
|
||||
- Read: accepted Project A report
|
||||
- Record: protected server execution journal
|
||||
|
||||
**Step 1: Recheck source and running images**
|
||||
|
||||
Run the Project A plan's identity commands again. Record application SHA, workspace SHA, core image
|
||||
ID, frontend image ID, Qdrant image digest, Ollama image digest, and local-auth configuration revision.
|
||||
|
||||
Expected: all values match the accepted Project A report.
|
||||
|
||||
**Step 2: Recheck rollback**
|
||||
|
||||
Prove that the public route is still closed and the protected Project A configuration can be
|
||||
selected without reconstructing it from memory.
|
||||
|
||||
### Task 5: Execute Project B
|
||||
|
||||
**Files:**
|
||||
- Execute: `docs/plans/2026-08-20-psd-server-project-b-authentik.md`
|
||||
- Complete: `docs/testing/evidence/psd-server-project-b-report-template.md`
|
||||
|
||||
**Step 1: Execute Project B task-by-task**
|
||||
|
||||
Keep public traffic closed until Authentik, Supabase migrations, ThothII diagnostics, Nginx, TLS,
|
||||
and load-balancer preflight all pass.
|
||||
|
||||
**Step 2: Run the Project B human guide**
|
||||
|
||||
Follow `docs/testing/psd-server-project-b-manual.md` using approved ordinary and administrator
|
||||
identities. The final path begins at the Aritmolab homepage and uses its existing sidebar link.
|
||||
|
||||
**Step 3: Close the Project B report**
|
||||
|
||||
Expected: SSO, roles, PostgreSQL ownership, the F1-F8 session, rollback rehearsal, and cleanup of the
|
||||
temporary Project A endpoint all pass.
|
||||
|
||||
### Task 6: Close the program
|
||||
|
||||
**Files:**
|
||||
- Modify: `PROJECT_STATE.md`
|
||||
- Optionally create: reviewed redacted acceptance reports under `docs/testing/evidence/`
|
||||
|
||||
**Step 1: Reconcile final state**
|
||||
|
||||
Record final SHAs, image identities, workspace revision, Authentik object names/IDs (never secrets),
|
||||
Supabase database/schema names, public origin, sidebar source revision, Nginx configuration identity,
|
||||
and both report digests.
|
||||
|
||||
**Step 2: Verify final negative boundaries**
|
||||
|
||||
Expected: old stack stopped; Project A private endpoint removed; core/Qdrant/Ollama not externally
|
||||
published; `thoth_sessions` absent from PostgREST exposed schemas; no secret appears in reports.
|
||||
|
||||
**Step 3: Update project state**
|
||||
|
||||
Add a dated factual section to `PROJECT_STATE.md`. Mark anything not actually run as PENDING.
|
||||
|
||||
**Step 4: Run documentation checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff --check
|
||||
bash scripts/auth-docs-smoke.sh
|
||||
bash scripts/verify-workspace-install-docs.sh --fixtures-only
|
||||
```
|
||||
|
||||
Expected: all checks pass.
|
||||
|
||||
**Step 5: Commit only reviewed redacted documentation**
|
||||
|
||||
```bash
|
||||
git add PROJECT_STATE.md docs/testing/evidence
|
||||
git diff --cached --check
|
||||
git commit -m "docs: record PSD server deployment acceptance"
|
||||
```
|
||||
|
||||
Expected: the commit contains no raw server inventory or secret material.
|
||||
@@ -0,0 +1,500 @@
|
||||
# PSD Server Project A Standalone Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Install a clean PSD ThothII stack with local authentication, direct read-only DWH access, internal Qdrant/Ollama, rebuilt preprocessing, and one completed F1-F8 work session.
|
||||
|
||||
**Architecture:** Preserve the stopped legacy installation and deploy the current canonical five-service Compose stack from an adjacent clean clone. A reviewed server-local override disables public exposure and uses filesystem work sessions; an optional load-balancer route is permitted only when it is operator-only and its denial boundary is proved first.
|
||||
|
||||
**Tech Stack:** Git, Docker/Compose, native `tht`, local Argon2id authentication, PSD Supabase PostgreSQL direct transport, Qdrant, Ollama, Pi, Nginx/load-balancer test route where safe.
|
||||
|
||||
---
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Common survey result is GO and its digest is recorded.
|
||||
- Every path below is replaced by the exact survey result before execution.
|
||||
- No production Nginx/load-balancer/sidebar/Authentik change is in scope.
|
||||
- The old stack remains running only until backup verification finishes; old and new stacks never
|
||||
run together.
|
||||
- The server's workspace deploy credential remains read-only. A curator with write access publishes
|
||||
the workspace change.
|
||||
|
||||
### Task 1: Freeze exact inputs
|
||||
|
||||
**Files:**
|
||||
- Read: protected survey report
|
||||
- Read: `docs/plans/2026-08-20-psd-server-deployment-program-design.md`
|
||||
- Record: protected Project A journal
|
||||
|
||||
**Step 1: Record application identity**
|
||||
|
||||
Run in the new planning checkout:
|
||||
|
||||
```bash
|
||||
git status --short --branch
|
||||
git rev-parse HEAD
|
||||
git rev-parse origin/main
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: clean and explicitly approved SHA.
|
||||
|
||||
**Step 2: Record workspace remote identity**
|
||||
|
||||
Use the surveyed read-only credential and run:
|
||||
|
||||
```bash
|
||||
git ls-remote <workspace-remote> refs/heads/main
|
||||
```
|
||||
|
||||
Expected: one SHA recorded as the pre-change workspace revision.
|
||||
|
||||
**Step 3: Check old-stack recoverability**
|
||||
|
||||
Expected: exact old start/stop procedure, source SHA, Compose identity, volumes/binds, proxy closure
|
||||
procedure, and backup destination are present in the survey. Stop if any is missing.
|
||||
|
||||
### Task 2: Publish the multi-transport workspace revision
|
||||
|
||||
**Files:**
|
||||
- Modify in authorized curator clone: `psd-clinical/workspace.yaml`
|
||||
- Verify: `thoth-workspaces.yaml`
|
||||
|
||||
**Step 1: Create a clean curator branch**
|
||||
|
||||
Run in a write-authorized clone, never in the application-managed registry checkout:
|
||||
|
||||
```bash
|
||||
git status --short --branch
|
||||
git fetch origin main
|
||||
git switch --create codex/psd-direct-transport origin/main
|
||||
```
|
||||
|
||||
Expected: clean branch at the recorded remote SHA.
|
||||
|
||||
**Step 2: Make the minimal descriptor change**
|
||||
|
||||
Change exactly:
|
||||
|
||||
```yaml
|
||||
supported_transports: [rest_api]
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```yaml
|
||||
supported_transports: [rest_api, postgres_direct]
|
||||
```
|
||||
|
||||
Do not duplicate the workspace or change its ID, collection, Evidence, annotations, model policy,
|
||||
database, or schema.
|
||||
|
||||
**Step 3: Review the descriptor-only diff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff --check
|
||||
git diff -- psd-clinical/workspace.yaml thoth-workspaces.yaml
|
||||
```
|
||||
|
||||
Expected: one semantic line changed; catalog metadata remains identical.
|
||||
|
||||
**Step 4: Validate with the current ThothII contract**
|
||||
|
||||
Use a disposable installation/registry or the repository's current registry validation harness to
|
||||
activate the candidate commit before publication. Expected: schema v3 accepts both transports,
|
||||
Evidence and annotations materialize, and no secret is required for source validation.
|
||||
|
||||
If no supported validator can be run in the curator environment, stop and request the owner to run
|
||||
the established Mac validation; do not publish based only on YAML parsing.
|
||||
|
||||
**Step 5: Commit and publish through curator review**
|
||||
|
||||
```bash
|
||||
git add psd-clinical/workspace.yaml
|
||||
git diff --cached --check
|
||||
git commit -m "feat: support direct PSD DWH transport"
|
||||
git push --set-upstream origin codex/psd-direct-transport
|
||||
```
|
||||
|
||||
Merge through the repository's normal review path. Record the resulting `main` SHA.
|
||||
|
||||
**Step 6: Prove the Mac REST installation is unchanged**
|
||||
|
||||
The owner pulls/activates the new workspace commit on the Mac, confirms selected transport
|
||||
`rest_api`, runs workspace inspection/connection diagnostics, and records PASS. Project A server
|
||||
deployment stops if this cross-installation proof is not available.
|
||||
|
||||
### Task 3: Back up and stop the legacy installation
|
||||
|
||||
**Files:**
|
||||
- Create: surveyed protected legacy backup directory
|
||||
- Record: Project A journal
|
||||
|
||||
**Step 1: Capture final legacy state**
|
||||
|
||||
Run the surveyed legacy status/doctor commands, record source SHA and image IDs, and confirm no
|
||||
active user work. Do not use the new `tht` against an incompatible old descriptor.
|
||||
|
||||
**Step 2: Close or maintenance-gate the old ThothII route**
|
||||
|
||||
Change only the surveyed ThothII-specific route using its established mechanism. Validate Nginx and
|
||||
load-balancer configuration before applying. Confirm external requests no longer reach the app.
|
||||
|
||||
**Step 3: Create the legacy backup**
|
||||
|
||||
Use the surveyed, version-compatible backup procedure. Include source/config metadata and all old
|
||||
runtime volumes/binds needed to restart; store credentials separately under existing protected
|
||||
custody. Create SHA-256 checksums and verify them.
|
||||
|
||||
**Step 4: Stop the old stack**
|
||||
|
||||
Use its own supported controller. Expected: old containers stopped, not removed; volumes and bind
|
||||
trees unchanged.
|
||||
|
||||
**Step 5: Rehearse the restart command without executing it**
|
||||
|
||||
Record the exact command, preconditions, port ownership, and route-restoration order. If it cannot
|
||||
be stated unambiguously, stop before creating the new stack.
|
||||
|
||||
### Task 4: Prepare the adjacent clean installation
|
||||
|
||||
**Files:**
|
||||
- Create: survey-selected new source root
|
||||
- Create: survey-selected operator, secret, data, Pi-state, registry, and backup roots
|
||||
|
||||
**Step 1: Create dedicated identities and paths**
|
||||
|
||||
Follow `docs/install/server.md` ownership rules using the surveyed available UID/GID. Do not reuse a
|
||||
UID already owned by another service and do not change image UID 10001 without a reviewed mapping.
|
||||
|
||||
**Step 2: Clone the frozen application source**
|
||||
|
||||
```bash
|
||||
git -c core.autocrlf=false clone <thothii-remote> <new-source-root>/ThothII
|
||||
git -C <new-source-root>/ThothII config --local core.autocrlf false
|
||||
git -C <new-source-root>/ThothII switch --detach <approved-application-sha>
|
||||
git -C <new-source-root>/ThothII status --short --branch
|
||||
```
|
||||
|
||||
Expected: detached exact SHA, clean tree.
|
||||
|
||||
**Step 3: Verify source and platform**
|
||||
|
||||
```bash
|
||||
cd <new-source-root>/ThothII
|
||||
bash scripts/verify-line-endings.sh
|
||||
docker version
|
||||
docker compose version
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
**Step 4: Prepare Pi state and build the operator**
|
||||
|
||||
```bash
|
||||
sudo scripts/prepare-server-pi-state.sh <new-pi-state-root> 10001 10001
|
||||
THT_THT_OUTPUT_DIRECTORY=<protected-build-output> bash scripts/build-tht.sh
|
||||
```
|
||||
|
||||
Install only the binary matching the surveyed server architecture. Run `tht version --json` and
|
||||
record its source identity.
|
||||
|
||||
### Task 5: Create the protected Project A configuration
|
||||
|
||||
**Files:**
|
||||
- Create outside Git: `<project-a-operator-root>/server.env`
|
||||
- Create outside Git: `<project-a-operator-root>/thothii-installation.yaml`
|
||||
- Create outside Git: `<project-a-operator-root>/project-a-private.yaml`
|
||||
- Create outside Git: `<project-a-auth-root>/auth.yaml` through `tht`
|
||||
|
||||
**Step 1: Start from current examples**
|
||||
|
||||
Copy `deploy/env/server.env.example` and `docs/install/examples/thothii-installation.server.yaml`
|
||||
to the protected Project A operator root. Replace every placeholder with surveyed absolute paths.
|
||||
Never source `server.env` as shell code.
|
||||
|
||||
**Step 2: Add the private/local-session override**
|
||||
|
||||
Create this reviewed override:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
core:
|
||||
environment:
|
||||
THOTH_PUBLIC_EXPOSURE: "false"
|
||||
THT_SESSION_STORAGE: local
|
||||
frontend:
|
||||
ports: !override
|
||||
- "127.0.0.1:<project-a-port>:8080"
|
||||
```
|
||||
|
||||
Select an unused loopback port proved by `ss -lntp`. Do not publish core, Qdrant, or Ollama.
|
||||
|
||||
**Step 3: Compose the installation descriptor**
|
||||
|
||||
Use `profile: server`, the exact new source root/env/auth root, workspace remote/branch/read-only
|
||||
access, Project A override, and exactly one Git transport override. Do not include the public
|
||||
session-server overlay in Project A.
|
||||
|
||||
**Step 4: Validate permissions and render**
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> update --check-only
|
||||
```
|
||||
|
||||
Expected: Compose validates; only frontend has a loopback port; core declares public exposure false
|
||||
and local session storage; Qdrant/Ollama are internal.
|
||||
|
||||
**Step 5: Configure the local administrator**
|
||||
|
||||
Create a temporary mode-0600 password file using an echo-free prompt, then run:
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> auth configure \
|
||||
--mode local --public-url <project-a-origin> \
|
||||
--admin-user <test-admin> --admin-display-name <display-name> \
|
||||
--password-file <protected-temporary-password-file>
|
||||
```
|
||||
|
||||
Remove the temporary input file after success and record that removal. Do not delete generated
|
||||
`auth.yaml` or `users.yaml`.
|
||||
|
||||
### Task 6: Build and start the clean stack
|
||||
|
||||
**Files:**
|
||||
- Record: Project A evidence directory
|
||||
|
||||
**Step 1: Build current images**
|
||||
|
||||
```bash
|
||||
cd <new-source-root>/ThothII
|
||||
bash scripts/build-local.sh
|
||||
```
|
||||
|
||||
Expected: current core/frontend images build; pinned Qdrant/Ollama references resolve.
|
||||
|
||||
**Step 2: Run preflight**
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> update --check-only
|
||||
<new-tht> --installation <project-a-installation> pi doctor
|
||||
```
|
||||
|
||||
Expected: no mutation error and no secret in output.
|
||||
|
||||
**Step 3: Start through `tht`**
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> start --build
|
||||
<new-tht> --installation <project-a-installation> status
|
||||
<new-tht> --installation <project-a-installation> doctor --json
|
||||
<new-tht> --installation <project-a-installation> pi test
|
||||
```
|
||||
|
||||
Expected: frontend, core, Qdrant, embedding healthy and model initializer completed. Doctor has the
|
||||
documented ordered checks and authentication PASS.
|
||||
|
||||
**Step 4: Verify listener boundaries**
|
||||
|
||||
Use `ss -lntp` and bounded Docker inspection. Expected: only the selected frontend loopback port is
|
||||
host-published; no external core, Qdrant, or Ollama listener.
|
||||
|
||||
### Task 7: Activate the workspace and direct DWH binding
|
||||
|
||||
**Files:**
|
||||
- Modify only through authenticated Workspace Management: encrypted workspace secret store
|
||||
|
||||
**Step 1: Pull and inspect the reviewed workspace revision**
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> \
|
||||
workspace inspect --workspace psd-clinical --json
|
||||
```
|
||||
|
||||
Expected: active workspace SHA equals the approved multi-transport revision.
|
||||
|
||||
**Step 2: Configure runtime bindings**
|
||||
|
||||
Through the authenticated Workspace Management API/UI, select `postgres_direct` and provide the
|
||||
surveyed host, port, runtime user, password, and optional TLS CA. Secret values go to the encrypted
|
||||
vault; they do not enter `server.env`, Git, shell arguments, or evidence.
|
||||
|
||||
The generated contract names are:
|
||||
|
||||
```text
|
||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT
|
||||
THT_WS_PSD_CLINICAL_DWH_HOST
|
||||
THT_WS_PSD_CLINICAL_DWH_PORT
|
||||
THT_WS_PSD_CLINICAL_DWH_USER
|
||||
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE
|
||||
THT_WS_PSD_CLINICAL_DWH_TLS_CA_FILE
|
||||
```
|
||||
|
||||
**Step 3: Validate and test connections**
|
||||
|
||||
Run static validation, live connection test, workspace inspect, and `doctor --json`. Expected: DWH,
|
||||
workspace, internal embedding, and Qdrant checks pass with redacted output.
|
||||
|
||||
**Step 4: Re-prove read-only grants**
|
||||
|
||||
Use the survey's catalog query through the exact configured identity. Expected: no DML/DDL grant on
|
||||
`datawarehouse`. Stop if the runtime user is an owner, superuser, or write-capable role.
|
||||
|
||||
### Task 8: Rebuild and verify semantic preprocessing
|
||||
|
||||
**Files:**
|
||||
- Create: Project A preprocessing evidence
|
||||
|
||||
**Step 1: Inspect the empty/new collection state**
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> \
|
||||
workspace vector inspect --workspace psd-clinical --json
|
||||
```
|
||||
|
||||
Expected: either a compatible empty collection or the documented missing-collection state.
|
||||
|
||||
**Step 2: Create the descriptor-owned collection when missing**
|
||||
|
||||
Use the guarded vector rebuild only for collection `psd-clinical`, with exact repeated confirmation
|
||||
and `--destroy`. Do not run it against any other collection.
|
||||
|
||||
**Step 3: Run complete preprocessing**
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> \
|
||||
workspace preprocess run --workspace psd-clinical --json
|
||||
```
|
||||
|
||||
If it returns `manual_review_required`, inspect the exact run and curated annotations, obtain the
|
||||
required human decision, run `workspace schema accept --workspace psd-clinical --run <run-id> --yes`,
|
||||
then resume the same run. Never auto-approve unknown FK changes.
|
||||
|
||||
**Step 4: Verify collection contract and counts**
|
||||
|
||||
Run vector inspection and record dimensions, cosine distance, required keyword indexes, and bounded
|
||||
counts by payload kind/revision. Expected: all points carry the active workspace revision.
|
||||
|
||||
**Step 5: Prove idempotency**
|
||||
|
||||
Run the complete preprocessing command again. Expected: no new review, no duplicate logical points,
|
||||
unchanged Evidence reported as unchanged, and the same effective configuration identity.
|
||||
|
||||
### Task 9: Configure and test local users
|
||||
|
||||
**Files:**
|
||||
- Modify through `tht auth user`: protected local user registry
|
||||
|
||||
**Step 1: Add an ordinary test user**
|
||||
|
||||
Use an echo-free prompt or protected temporary password file:
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> auth user add <test-user> \
|
||||
--role user --display-name <display-name> --password-file <protected-temporary-password-file>
|
||||
```
|
||||
|
||||
Remove the temporary input file after success.
|
||||
|
||||
**Step 2: Run authentication diagnostics**
|
||||
|
||||
```bash
|
||||
<new-tht> --installation <project-a-installation> auth status --json
|
||||
<new-tht> --installation <project-a-installation> auth check --json
|
||||
```
|
||||
|
||||
Expected: pristine redacted JSON and PASS.
|
||||
|
||||
**Step 3: Execute automated local-auth cases**
|
||||
|
||||
Use same-origin requests or a local headless browser to prove ordinary/admin authorization, generic
|
||||
wrong-password failure, disable/enable, password/role revision invalidation, logout-all, CSRF, and
|
||||
remembered-session survival after core restart. Do not retain cookie jars after the test.
|
||||
|
||||
### Task 10: Optionally add the private network-path test
|
||||
|
||||
**Files:**
|
||||
- Modify only surveyed test-specific load-balancer/Nginx files
|
||||
- Create: test certificate through the existing managed mechanism
|
||||
|
||||
**Step 1: Prove allowlist capability before proxying**
|
||||
|
||||
Create a temporary hostname that returns a fixed maintenance response. From an approved operator
|
||||
source expect success; from an unapproved source expect denial. Do not point it at ThothII yet.
|
||||
|
||||
**Step 2: Validate and activate the test proxy**
|
||||
|
||||
Configure Nginx with the same Host/HTTPS forwarding and SSE settings intended for production. Run
|
||||
`nginx -t`, validate the load balancer, then reload through the established mechanism.
|
||||
|
||||
**Step 3: Reconfigure local-auth public URL transactionally**
|
||||
|
||||
If the exact private HTTPS origin differs from the loopback origin, use the supported authentication
|
||||
configuration workflow and invalidate prior test sessions. Re-run auth and doctor checks.
|
||||
|
||||
**Step 4: Prove both sides**
|
||||
|
||||
Expected: authorized operator reaches the local login; unauthorized source remains denied before
|
||||
ThothII. If this cannot be demonstrated, remove the test route and continue on loopback.
|
||||
|
||||
### Task 11: Complete the F1-F8 acceptance session
|
||||
|
||||
**Files:**
|
||||
- Complete: `docs/testing/psd-server-project-a-manual.md`
|
||||
- Create: protected session evidence
|
||||
|
||||
**Step 1: Select the approved harmless question**
|
||||
|
||||
Use a known read-only PSD question agreed by the owner. Record the wording in the protected report;
|
||||
do not include patient-identifying values.
|
||||
|
||||
**Step 2: Create the session as the ordinary local user**
|
||||
|
||||
Use the private browser route when present; otherwise drive the same-origin frontend/API from the
|
||||
server-local terminal/headless browser. Record only session ID and sanitized milestones.
|
||||
|
||||
**Step 3: Review every gate**
|
||||
|
||||
Complete F1-F8 without auto-confirming human decisions. Confirm persisted phase/artifact state after
|
||||
each gate and resume once to prove recovery.
|
||||
|
||||
**Step 4: Validate final SQL**
|
||||
|
||||
Expected: finalized session, DWH validation PASS, SQL is read-only, and no clinical mutation occurs.
|
||||
|
||||
**Step 5: Inspect persisted state**
|
||||
|
||||
Confirm manifest, question, schema linking, Evidence, CTE plan/tests, final SQL, validation report,
|
||||
and decision ledger exist in local filesystem session storage. Chat/SSE need not persist.
|
||||
|
||||
### Task 12: Close Project A and preserve rollback
|
||||
|
||||
**Files:**
|
||||
- Complete: `docs/testing/evidence/psd-server-project-a-report-template.md`
|
||||
|
||||
**Step 1: Run final diagnostics**
|
||||
|
||||
Run status, doctor, auth check, workspace inspect, vector inspect, Pi test, and a bounded secret scan
|
||||
of the intended report.
|
||||
|
||||
**Step 2: Create a transactional new-installation backup**
|
||||
|
||||
Use `tht backup --drain` with a protected explicit output. Verify its checksum. Do not include
|
||||
secrets in the ordinary evidence archive.
|
||||
|
||||
**Step 3: Complete human acceptance**
|
||||
|
||||
Every row in `docs/testing/psd-server-project-a-manual.md` must be PASS or explicitly blocking.
|
||||
|
||||
**Step 4: Record the gate**
|
||||
|
||||
Record exact SHAs/images, workspace revision, preprocessing identity/counts, session ID, report
|
||||
digest, rollback status, and explicit `PROJECT_A_PASS` or `PROJECT_A_FAIL`.
|
||||
|
||||
**Step 5: Stop on FAIL**
|
||||
|
||||
On FAIL, stop the new stack and use the surveyed old-stack recovery plan if service restoration is
|
||||
desired. Do not start Project B.
|
||||
@@ -0,0 +1,437 @@
|
||||
# PSD Server Project B Authentik Integration Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Convert the accepted Project A installation to public OIDC mode, store owned work sessions in the existing Supabase database's `thoth_sessions` schema, and restore the established Aritmolab-sidebar user journey through the load balancer and Nginx.
|
||||
|
||||
**Architecture:** Keep the same installation-descriptor path and Compose project so Project A Qdrant/Ollama volumes and bind state remain authoritative. Close ingress, snapshot Project A configuration, configure Authentik and Supabase, replace the protected installation configuration transactionally at the same paths, validate privately, then open the production route and sidebar link.
|
||||
|
||||
**Tech Stack:** Authentik OAuth2/OIDC Authorization Code + PKCE, ThothII OIDC/session store, Supabase PostgreSQL schema migrations/RLS, Docker Compose, Nginx, load balancer, Aritmolab.
|
||||
|
||||
---
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Project A automated and human reports are PASS and explicitly owner-approved.
|
||||
- Application SHA, workspace SHA, images, Project A report digest, and rollback configuration match
|
||||
the accepted evidence.
|
||||
- The production route is closed before authentication/session-storage changes.
|
||||
- All Authentik operations use the installed version's API/OpenAPI contract. Official current
|
||||
references include [OAuth2/OIDC providers](https://docs.goauthentik.io/add-secure-apps/providers/oauth2/),
|
||||
[provider property mappings](https://docs.goauthentik.io/add-secure-apps/providers/property-mappings/),
|
||||
[application bindings](https://docs.goauthentik.io/add-secure-apps/applications/manage_apps/), and
|
||||
[blueprint export](https://docs.goauthentik.io/customize/blueprints/export); installed-version
|
||||
behavior wins over newer documentation.
|
||||
|
||||
### Task 1: Freeze Project A and close ingress
|
||||
|
||||
**Files:**
|
||||
- Read: accepted Project A report
|
||||
- Create: protected Project B transaction root
|
||||
|
||||
**Step 1: Verify exact Project A state**
|
||||
|
||||
Run status, doctor, auth check, workspace inspect, vector inspect, Pi status/test, Git SHA, and image
|
||||
identity checks from Project A. Expected: all match the accepted report.
|
||||
|
||||
**Step 2: Create a protected transaction root**
|
||||
|
||||
Use `mktemp -d` under the survey-approved protected parent, mode `0700`. Record its path and do not
|
||||
place it in Git.
|
||||
|
||||
**Step 3: Close production and temporary ingress**
|
||||
|
||||
Keep or restore a maintenance response at the production ThothII route. Disable the optional
|
||||
Project A test route before changing authentication unless it is needed for a separately approved
|
||||
private preflight. Confirm neither route reaches ThothII.
|
||||
|
||||
**Step 4: Stop and back up Project A**
|
||||
|
||||
```bash
|
||||
<tht> --installation <stable-installation-path> backup \
|
||||
--output <project-b-transaction-root>/project-a-backup.tar --drain
|
||||
<tht> --installation <stable-installation-path> stop
|
||||
```
|
||||
|
||||
Verify the archive using the controller's manifest/checksum contract. Preserve a copy of the exact
|
||||
installation descriptor, env file, override files, auth directory, Nginx fragment, load-balancer
|
||||
route, Aritmolab sidebar file/revision, and relevant Authentik export metadata.
|
||||
|
||||
### Task 2: Decide exact Authentik names and roles
|
||||
|
||||
**Files:**
|
||||
- Create: protected `authentik-change-manifest.yaml`
|
||||
|
||||
**Step 1: Select the final public origin**
|
||||
|
||||
Use the live survey result, not historical `.it`/`.com` assumptions. Record exactly one HTTPS origin
|
||||
and callback `<origin>/api/auth/oidc/callback`.
|
||||
|
||||
**Step 2: Select exact groups**
|
||||
|
||||
Default to dedicated `TOT Users` and `TOT Admin`. Reuse existing groups only if their membership
|
||||
semantics match and the owner approves. Record exact case-sensitive names.
|
||||
|
||||
**Step 3: Define least privilege**
|
||||
|
||||
Map user group → `user`, admin group → `admin`. Define a separate service account/token with only
|
||||
the installed Authentik permission needed to view exact group objects. No write, user-management,
|
||||
directory-administration, or superuser permission.
|
||||
|
||||
**Step 4: Obtain owner approval of the manifest**
|
||||
|
||||
The manifest contains object names, slugs, intended bindings, callback, scopes, grant types,
|
||||
credential destinations, and rollback action—but no secret values. Do not mutate Authentik before
|
||||
approval.
|
||||
|
||||
### Task 3: Export and prepare Authentik
|
||||
|
||||
**Files:**
|
||||
- Create: protected pre-change Authentik export
|
||||
- Modify: Authentik objects named in the approved manifest
|
||||
|
||||
**Step 1: Export relevant configuration**
|
||||
|
||||
Use the installed version's supported blueprint/API export. A worker command such as
|
||||
`ak export_blueprint` is valid only if present in that version. Protect export mode `0600`; remember
|
||||
write-only provider secrets are not included, so backup their custody separately without printing.
|
||||
|
||||
**Step 2: Verify API credential scope**
|
||||
|
||||
Use a read-only call to list relevant groups/applications. Expected: administrative creation access
|
||||
for the setup identity and a distinct path for the future group-view service account. Stop if the
|
||||
credential is missing or ambiguous.
|
||||
|
||||
**Step 3: Create or confirm exact groups**
|
||||
|
||||
Create missing dedicated groups, or record approved existing group IDs. Do not bulk-copy LDAP or
|
||||
unrelated Authentik memberships.
|
||||
|
||||
**Step 4: Create the group-catalog service account**
|
||||
|
||||
Grant only exact group-view permission. Create its token through the approved protected-secret
|
||||
mechanism; write it directly to the ThothII secret destination without displaying it.
|
||||
|
||||
**Step 5: Create the OIDC provider**
|
||||
|
||||
Configure a confidential OAuth2/OIDC provider with the exact callback, issuer mode observed as
|
||||
appropriate, Authorization Code, PKCE support, and Device Code only when required for
|
||||
`tht auth check --interactive` and supported by the installed release. Do not enable implicit flow.
|
||||
|
||||
**Step 6: Configure scopes and direct groups claim**
|
||||
|
||||
Select `openid`, `profile`, and `email`. Inspect a disposable identity's decoded claim keys through
|
||||
a protected verifier; retain only a redacted shape. Expected: ID token includes direct non-empty
|
||||
`groups: [string, ...]`.
|
||||
|
||||
If the installed default profile mapping already provides that exact claim, reuse it. Otherwise add
|
||||
a provider scope/property mapping under the requested `profile` scope that returns:
|
||||
|
||||
```python
|
||||
return {"groups": [group.name for group in request.user.ak_groups.all()]}
|
||||
```
|
||||
|
||||
Verify the installed mapping merge semantics before activation. Do not add a custom unrequested
|
||||
scope because ThothII requests only `openid`, `profile`, and `email`.
|
||||
|
||||
**Step 7: Create the Authentik application**
|
||||
|
||||
Bind it to the provider. Configure display metadata according to local Aritmolab conventions. Do not
|
||||
use an Authentik proxy provider or Nginx forward-auth for ThothII.
|
||||
|
||||
**Step 8: Create and store the client secret**
|
||||
|
||||
Write the client secret directly into the protected ThothII secret bundle key
|
||||
`THT_OIDC_CLIENT_SECRET`. Store the service-account token as `THT_AUTHENTIK_API_TOKEN`. Never place
|
||||
either value in the change manifest, shell history, Compose environment, or evidence.
|
||||
|
||||
### Task 4: Prepare Supabase schema roles and backup
|
||||
|
||||
**Files:**
|
||||
- Read: `harness/tht/migrations/sessions/001_schema.sql`
|
||||
- Read: `harness/tht/migrations/sessions/002_security.sql`
|
||||
- Create: protected Supabase backup/evidence
|
||||
- Create: runtime and migrator credential files
|
||||
|
||||
**Step 1: Confirm the database/schema boundary**
|
||||
|
||||
Expected: use the surveyed existing Supabase PostgreSQL database; clinical data remains in
|
||||
`datawarehouse`; application sessions use schema `thoth_sessions`; no new database is created.
|
||||
|
||||
**Step 2: Back up database metadata/data consistently**
|
||||
|
||||
Use the existing Supabase/PostgreSQL backup procedure before DDL. Record backup ID, timestamp,
|
||||
checksum, and restore command. Do not put a dump in the Git repository.
|
||||
|
||||
**Step 3: Create or validate dedicated roles**
|
||||
|
||||
Create one migrator login and one runtime login according to the migration contract. The runtime
|
||||
role must not be superuser, owner, BYPASSRLS, CREATEROLE, CREATEDB, or a member of DWH write roles.
|
||||
The migrator credential remains unavailable to core.
|
||||
|
||||
**Step 4: Write protected credential files**
|
||||
|
||||
Create separate mode-0640 runtime-password, migrator-password, and session-CA files with surveyed
|
||||
ownership. Do not use command-line password arguments.
|
||||
|
||||
**Step 5: Confirm PostgREST exclusion before migration**
|
||||
|
||||
Record the exact exposed schema list. Expected: `thoth_sessions` absent. If the system exposes all
|
||||
schemas implicitly, stop and resolve the boundary before migration.
|
||||
|
||||
### Task 5: Prepare the stable Project B installation configuration
|
||||
|
||||
**Files:**
|
||||
- Modify at the same stable paths: operator env, installation descriptor, authentication directory
|
||||
- Create: reviewed session-server override copied from `deploy/compose.session-server.yaml.example`
|
||||
- Create: protected server-session workspace config copied from `deploy/workspaces/server-sessions.yaml.example`
|
||||
|
||||
**Step 1: Preserve the Compose project name**
|
||||
|
||||
The native controller derives the project name from the absolute installation-descriptor path.
|
||||
Keep that exact path. Do not point Project B at a second descriptor path, because that would create
|
||||
new Qdrant/Ollama named volumes instead of using the Project A accepted state.
|
||||
|
||||
**Step 2: Stage Project B files beside the live files**
|
||||
|
||||
Prepare new env/descriptor/override/auth inputs in the protected transaction root. Add the session
|
||||
DB host/port/existing database name/runtime role/migrator role/TLS mode and three secret source
|
||||
paths. Use `verify-full` where hostname/SAN permits; any `verify-ca` exception requires explicit
|
||||
survey evidence and owner approval.
|
||||
|
||||
**Step 3: Add the server-session override**
|
||||
|
||||
Copy the current example to a reviewed local file and add it to the existing stable descriptor's
|
||||
overrides before the Git transport override ordering required by the installation. Do not edit the
|
||||
tracked example.
|
||||
|
||||
**Step 4: Replace local auth state transactionally**
|
||||
|
||||
With the stack stopped, move the complete Project A auth directory into the protected transaction
|
||||
root, recreate an empty private directory at the same path/ownership/mode, and configure OIDC:
|
||||
|
||||
```bash
|
||||
<tht> --installation <stable-installation-path> auth configure \
|
||||
--mode oidc --public-url <final-https-origin> \
|
||||
--issuer <authentik-issuer> --client-id <oidc-client-id> \
|
||||
--authentik-base-url <authentik-base-url> \
|
||||
--user-group '<exact-user-group>' --admin-group '<exact-admin-group>'
|
||||
```
|
||||
|
||||
Expected: non-secret `auth.yaml` only; secrets resolved from the protected bundle.
|
||||
|
||||
**Step 5: Atomically install staged path-only files**
|
||||
|
||||
Use same-filesystem rename and preserve required ownership/mode. Keep the Project A originals in
|
||||
the transaction root. Run `update --check-only`; on failure restore the originals immediately.
|
||||
|
||||
### Task 6: Run and verify session migrations
|
||||
|
||||
**Files:**
|
||||
- Modify through one-shot migrator: existing database schema `thoth_sessions`
|
||||
|
||||
**Step 1: Validate migration rendering**
|
||||
|
||||
```bash
|
||||
<tht> --installation <stable-installation-path> update --check-only
|
||||
```
|
||||
|
||||
Expected: core and `session-migrate` resolve the same core image; core lacks migrator password;
|
||||
only the one-shot service sees it.
|
||||
|
||||
**Step 2: Run migrations once**
|
||||
|
||||
```bash
|
||||
<tht> --installation <stable-installation-path> sessions migrate --yes
|
||||
```
|
||||
|
||||
Expected JSON: `"pending":[]` and `"drifted":[]`; `applied` may list `001` and `002` on first use.
|
||||
|
||||
**Step 3: Run migration status/idempotency again**
|
||||
|
||||
Run the same command. Expected: no new application and both pending/drifted remain empty.
|
||||
|
||||
**Step 4: Verify database security**
|
||||
|
||||
Through bounded catalog queries, prove forced RLS, policies on session tables, runtime role without
|
||||
BYPASSRLS/ownership/DDL, migrator absent from core, and no runtime privileges on unrelated schemas.
|
||||
|
||||
**Step 5: Recheck PostgREST exclusion**
|
||||
|
||||
Expected: `thoth_sessions` still absent from exposed schemas and REST endpoints cannot address it.
|
||||
|
||||
### Task 7: Validate Authentik and start privately
|
||||
|
||||
**Files:**
|
||||
- Record: Project B protected evidence
|
||||
|
||||
**Step 1: Run static configuration validation**
|
||||
|
||||
Run `update --check-only` and redacted `auth status --json`. Expected: mode OIDC, exact public origin,
|
||||
issuer/client ID/group names, and no secret values.
|
||||
|
||||
**Step 2: Start while public ingress remains closed**
|
||||
|
||||
```bash
|
||||
<tht> --installation <stable-installation-path> start
|
||||
<tht> --installation <stable-installation-path> status
|
||||
<tht> --installation <stable-installation-path> auth check --json
|
||||
<tht> --installation <stable-installation-path> doctor --json
|
||||
<tht> --installation <stable-installation-path> pi test
|
||||
```
|
||||
|
||||
Expected: OIDC discovery/issuer/JWKS, client-secret access, Authentik catalog token, exact mapped
|
||||
groups, PostgreSQL session storage, workspace, services, workflow, and Pi pass.
|
||||
|
||||
**Step 3: Run interactive device check when supported**
|
||||
|
||||
```bash
|
||||
<tht> --installation <stable-installation-path> auth check --interactive
|
||||
```
|
||||
|
||||
Expected: approved identity completes Device Authorization and direct groups claim validates. If
|
||||
the installed provider does not support device flow, record PENDING rather than substituting a token.
|
||||
|
||||
### Task 8: Prepare Nginx, TLS, load balancer, and sidebar
|
||||
|
||||
**Files:**
|
||||
- Modify only survey-approved ThothII Nginx fragment
|
||||
- Modify only survey-approved load-balancer route
|
||||
- Modify only exact Aritmolab sidebar source when its target must change
|
||||
|
||||
**Step 1: Prepare direct-OIDC Nginx configuration**
|
||||
|
||||
Follow `docs/install/reverse-proxy-nginx.md`, direct OIDC section. Required behavior: no
|
||||
`auth_request`, no callback rewrite, frontend loopback upstream, Host and HTTPS forwarded headers,
|
||||
HTTP/1.1, buffering/cache off, long SSE read timeout, and `X-Accel-Buffering: no`.
|
||||
|
||||
**Step 2: Validate the managed certificate**
|
||||
|
||||
Expected: SAN matches final hostname, validity is current, chain is trusted by approved clients,
|
||||
private-key permissions match local policy, and renewal/generation ownership is recorded. Never
|
||||
copy the key into ThothII.
|
||||
|
||||
**Step 3: Validate Nginx without opening traffic**
|
||||
|
||||
```bash
|
||||
sudo nginx -t
|
||||
curl --fail http://127.0.0.1:<frontend-port>/health
|
||||
```
|
||||
|
||||
Use local `--resolve`/Host tests only when they do not bypass the identity behavior being tested.
|
||||
|
||||
**Step 4: Prepare the load-balancer route**
|
||||
|
||||
Configure backend/health/TLS according to the surveyed owner procedure, initially disabled or
|
||||
operator-only. Confirm it targets Nginx, never core/Qdrant/Ollama directly.
|
||||
|
||||
**Step 5: Preserve the Aritmolab link contract**
|
||||
|
||||
If the existing sidebar target already equals the final origin/path, leave source unchanged and
|
||||
record proof. Otherwise make the smallest reviewed change, test it in Aritmolab's own test/build
|
||||
system, and commit in that repository before deployment.
|
||||
|
||||
### Task 9: Open ingress and run OIDC acceptance
|
||||
|
||||
**Files:**
|
||||
- Complete: `docs/testing/psd-server-project-b-manual.md`
|
||||
|
||||
**Step 1: Reload Nginx through the established mechanism**
|
||||
|
||||
Run `nginx -t` immediately before reload. Expected: reload succeeds and unrelated virtual hosts
|
||||
remain healthy.
|
||||
|
||||
**Step 2: Enable the final load-balancer route**
|
||||
|
||||
Expected: HTTP redirects to HTTPS; TLS is valid; `/api/auth/oidc/login` redirects to the correct
|
||||
Authentik provider; callback returns to the exact public origin.
|
||||
|
||||
**Step 3: Test ordinary and admin identities**
|
||||
|
||||
Start at Aritmolab, authenticate once, then use the sidebar. Expected: no second credential prompt;
|
||||
ordinary user can use sessions but receives 403 for admin operations; admin has only documented
|
||||
permissions.
|
||||
|
||||
**Step 4: Test no-role and malformed cases**
|
||||
|
||||
An identity with no mapped group authenticates but receives no application role/403. Missing,
|
||||
malformed, indirect, or ambiguous group claims fail closed with generic browser errors and redacted
|
||||
diagnostics. Do not retain raw claims.
|
||||
|
||||
**Step 5: Test logout and restart**
|
||||
|
||||
Verify ThothII logout revokes its own cookie. Document whether the Authentik SSO session remains and
|
||||
therefore allows immediate re-login without credentials; do not claim global logout unless
|
||||
configured and tested. Restart core and verify expected OIDC session behavior.
|
||||
|
||||
### Task 10: Verify PostgreSQL ownership and complete F1-F8
|
||||
|
||||
**Files:**
|
||||
- Create: protected Project B session evidence
|
||||
|
||||
**Step 1: Create sessions under two identities**
|
||||
|
||||
Expected: ordinary users see only their own sessions; cross-user access returns the documented
|
||||
not-found boundary; admin behavior matches `session.read_all/manage_all` permissions.
|
||||
|
||||
**Step 2: Verify RLS with the runtime path**
|
||||
|
||||
Use application/API tests and bounded catalog evidence. Never disable RLS for diagnosis.
|
||||
|
||||
**Step 3: Complete one harmless OIDC PSD session**
|
||||
|
||||
Use the same approved read-only question or another owner-approved one. Complete F1-F8, validate
|
||||
final SQL, resume once, and confirm session/artifacts/decisions are stored in `thoth_sessions`.
|
||||
|
||||
**Step 4: Verify ephemeral boundaries**
|
||||
|
||||
Expected: no chat transcript or SSE stream stored as session artifacts; no vectors in PostgreSQL;
|
||||
Qdrant remains the semantic store.
|
||||
|
||||
### Task 11: Test controlled failures and rollback
|
||||
|
||||
**Files:**
|
||||
- Record: protected rollback evidence
|
||||
|
||||
**Step 1: Test a reversible provider/catalog failure**
|
||||
|
||||
Use a controlled, owner-approved method such as a temporary disabled test credential or test object.
|
||||
Expected: auth diagnostics and browser login fail closed, redacted, then pass after restoration.
|
||||
Never break unrelated Authentik applications.
|
||||
|
||||
**Step 2: Rehearse ingress-first rollback**
|
||||
|
||||
Close the production route, validate Nginx restoration commands, and prove the protected Project A
|
||||
configuration snapshot is complete. A full rollback need not destroy `thoth_sessions`.
|
||||
|
||||
**Step 3: Verify additive database rollback boundary**
|
||||
|
||||
Expected: rollback leaves schema/data intact for evidence and future recovery. No automatic DROP
|
||||
SCHEMA or role deletion.
|
||||
|
||||
### Task 12: Close Project B
|
||||
|
||||
**Files:**
|
||||
- Complete: `docs/testing/evidence/psd-server-project-b-report-template.md`
|
||||
|
||||
**Step 1: Run final diagnostics**
|
||||
|
||||
Run status, doctor, auth check, interactive check when supported, workspace inspect, vector inspect,
|
||||
Pi test, Nginx validation, load-balancer health, Supabase migration/security checks, and sidebar test.
|
||||
|
||||
**Step 2: Remove the Project A temporary endpoint**
|
||||
|
||||
Remove only its load-balancer route, Nginx fragment, and managed certificate reference according to
|
||||
their owners. Validate/reload and prove the hostname no longer routes.
|
||||
|
||||
**Step 3: Complete the human guide and report**
|
||||
|
||||
Every mandatory row must be PASS. Record exact source/image/workspace identities, Authentik object
|
||||
names/IDs, Supabase database plus `thoth_sessions`, public origin, Aritmolab revision, report digest,
|
||||
and `PROJECT_B_PASS` or `PROJECT_B_FAIL`.
|
||||
|
||||
**Step 4: Handle FAIL safely**
|
||||
|
||||
On FAIL, close ingress first. Restore Project A files at the same stable paths, move OIDC auth state
|
||||
to protected evidence, restore the local-auth directory, validate, and start Project A privately.
|
||||
Disable new Authentik objects; do not delete them or drop the session schema automatically.
|
||||
@@ -0,0 +1,308 @@
|
||||
# PSD Server Survey Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Produce a non-mutating, redacted survey of the PSD server that resolves every path, owner, network boundary, credential location, and rollback prerequisite needed by Projects A and B.
|
||||
|
||||
**Architecture:** Collect bounded metadata from the terminal local to the server, retain raw output only in a protected directory, and summarize it in a redacted report. The survey makes no service, file, database, proxy, Authentik, or Git mutation.
|
||||
|
||||
**Tech Stack:** Linux utilities, Docker/Compose inspection, Git, Nginx, OpenSSL, PostgreSQL/Supabase metadata queries, Authentik metadata/API discovery, Aritmolab source inspection.
|
||||
|
||||
---
|
||||
|
||||
## Safety contract
|
||||
|
||||
- Do not run `docker inspect` without a restrictive Go template; its default output can contain secrets.
|
||||
- Do not run `docker compose config` into chat or a public log. Store raw output mode `0600`, then create a redacted derivative.
|
||||
- Do not print process environments, secret-file contents, private keys, cookies, tokens, password hashes, or raw OIDC claims.
|
||||
- Do not reload/restart services, fetch/pull Git, log in interactively, change file modes, or make API mutations.
|
||||
- When a command needs privilege, use the server's approved CyberArk/local-terminal procedure.
|
||||
|
||||
### Task 1: Create the protected survey workspace
|
||||
|
||||
**Files:**
|
||||
- Create: `/var/tmp/thothii-psd-survey.<random>/`
|
||||
- Create: protected `survey-report.md`
|
||||
|
||||
**Step 1: Create a private temporary root**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
umask 0077
|
||||
PSD_SURVEY_ROOT="$(mktemp -d /var/tmp/thothii-psd-survey.XXXXXX)"
|
||||
test -d "$PSD_SURVEY_ROOT"
|
||||
chmod 0700 "$PSD_SURVEY_ROOT"
|
||||
printf '%s\n' "$PSD_SURVEY_ROOT"
|
||||
```
|
||||
|
||||
Expected: one new mode-0700 directory whose exact path is recorded in the operator journal.
|
||||
|
||||
**Step 2: Copy the report template**
|
||||
|
||||
Create `$PSD_SURVEY_ROOT/survey-report.md` from the headings in the design's Common Survey section.
|
||||
Record only findings and references to protected raw files.
|
||||
|
||||
### Task 2: Record host and Docker facts
|
||||
|
||||
**Files:**
|
||||
- Create: `$PSD_SURVEY_ROOT/host.txt`
|
||||
- Create: `$PSD_SURVEY_ROOT/docker-projects.json`
|
||||
- Create: `$PSD_SURVEY_ROOT/docker-containers.txt`
|
||||
|
||||
**Step 1: Record bounded host metadata**
|
||||
|
||||
Run each command with output redirected to `$PSD_SURVEY_ROOT/host.txt`:
|
||||
|
||||
```bash
|
||||
date -u '+%Y-%m-%dT%H:%M:%SZ'
|
||||
uname -a
|
||||
cat /etc/os-release
|
||||
getconf LONG_BIT
|
||||
nproc
|
||||
free -h
|
||||
df -hT
|
||||
docker version
|
||||
docker compose version
|
||||
```
|
||||
|
||||
Expected: no credential content and enough capacity information to judge a parallel source tree and
|
||||
a new five-service stack.
|
||||
|
||||
**Step 2: Record Compose projects and bounded container identity**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
docker compose ls --format json > "$PSD_SURVEY_ROOT/docker-projects.json"
|
||||
docker ps -a --no-trunc --format '{{json .}}' > "$PSD_SURVEY_ROOT/docker-containers.txt"
|
||||
docker network ls --format '{{json .}}' > "$PSD_SURVEY_ROOT/docker-networks.txt"
|
||||
docker volume ls --format '{{json .}}' > "$PSD_SURVEY_ROOT/docker-volumes.txt"
|
||||
```
|
||||
|
||||
Expected: inventory only. Do not inspect full container JSON.
|
||||
|
||||
**Step 3: Identify candidate legacy ThothII containers**
|
||||
|
||||
Use names, images, Compose project labels, published ports, and health from the bounded inventory.
|
||||
For each candidate, query only these templates:
|
||||
|
||||
```bash
|
||||
docker inspect --format '{{.Name}} {{.Config.Image}} {{index .Config.Labels "com.docker.compose.project"}} {{index .Config.Labels "com.docker.compose.project.working_dir"}}' <container>
|
||||
docker inspect --format '{{json .NetworkSettings.Networks}}' <container>
|
||||
docker inspect --format '{{range .Mounts}}{{.Type}} {{.Source}} -> {{.Destination}} rw={{.RW}}{{println}}{{end}}' <container>
|
||||
```
|
||||
|
||||
Expected: exact source/Compose ownership and mounts without environment values.
|
||||
|
||||
### Task 3: Survey the legacy ThothII installation
|
||||
|
||||
**Files:**
|
||||
- Create: `$PSD_SURVEY_ROOT/legacy-thothii.txt`
|
||||
- Create: `$PSD_SURVEY_ROOT/legacy-compose.redacted.yaml`
|
||||
|
||||
**Step 1: Resolve source and operator paths from evidence**
|
||||
|
||||
Do not search broad filesystem roots. Derive paths from Compose labels, systemd units, Nginx
|
||||
upstreams, and known operator documentation. Record uncertainty rather than guessing.
|
||||
|
||||
**Step 2: Record source identity without fetching**
|
||||
|
||||
Run in the identified source tree:
|
||||
|
||||
```bash
|
||||
git status --short --branch
|
||||
git rev-parse HEAD
|
||||
git remote -v
|
||||
git log -5 --oneline --decorate
|
||||
```
|
||||
|
||||
Expected: no mutation. Mark a dirty tree as NO-GO until the owner decides how to preserve it.
|
||||
|
||||
**Step 3: Record lifecycle state with the legacy controller**
|
||||
|
||||
If the old installation has a supported `tht`, run its bounded `status`, `doctor`, `pi status`, and
|
||||
`pi maintenance status` commands. Otherwise record exact read-only Docker health and identify the
|
||||
old lifecycle mechanism. Do not substitute current `tht` against an incompatible descriptor.
|
||||
|
||||
**Step 4: Render and redact Compose safely**
|
||||
|
||||
Store the raw render as mode `0600`. Replace secret-bearing scalar values with `[redacted]` before
|
||||
using the derivative in analysis. Confirm the redacted render still shows service names, networks,
|
||||
ports, volumes, image/build identities, and config file paths.
|
||||
|
||||
### Task 4: Survey Nginx, TLS, and the load balancer
|
||||
|
||||
**Files:**
|
||||
- Create: `$PSD_SURVEY_ROOT/nginx.raw.txt` (protected)
|
||||
- Create: `$PSD_SURVEY_ROOT/nginx-thothii.redacted.txt`
|
||||
- Create: `$PSD_SURVEY_ROOT/tls-metadata.txt`
|
||||
- Create: `$PSD_SURVEY_ROOT/load-balancer.md`
|
||||
|
||||
**Step 1: Validate and capture Nginx without reload**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
sudo nginx -t
|
||||
sudo nginx -T > "$PSD_SURVEY_ROOT/nginx.raw.txt" 2>&1
|
||||
chmod 0600 "$PSD_SURVEY_ROOT/nginx.raw.txt"
|
||||
```
|
||||
|
||||
Expected: configuration test passes. Do not reload Nginx.
|
||||
|
||||
**Step 2: Extract only relevant directives**
|
||||
|
||||
Create the redacted derivative containing the ThothII/Aritmolab `server_name`, `listen`, `location`,
|
||||
`proxy_pass`, `proxy_set_header`, `proxy_buffering`, timeout, certificate path, and include-file
|
||||
directives. Exclude unrelated virtual hosts and all authorization values.
|
||||
|
||||
**Step 3: Record certificate metadata only**
|
||||
|
||||
For each relevant public certificate—not its key—run:
|
||||
|
||||
```bash
|
||||
openssl x509 -in <certificate-path> -noout -subject -issuer -serial -dates -ext subjectAltName
|
||||
```
|
||||
|
||||
Expected: exact SAN/expiry/issuer and the observed generation/renewal mechanism.
|
||||
|
||||
**Step 4: Map the load balancer**
|
||||
|
||||
Record its owner, configuration surface, current Aritmolab backend, health check, TLS boundary,
|
||||
source addresses seen by Nginx, and whether it can enforce a temporary hostname allowlist. Do not
|
||||
create a route. If Sol cannot inspect it, name the human/team required for Project A/B gates.
|
||||
|
||||
### Task 5: Survey Aritmolab and the sidebar integration
|
||||
|
||||
**Files:**
|
||||
- Create: `$PSD_SURVEY_ROOT/aritmolab.md`
|
||||
|
||||
**Step 1: Resolve the Aritmolab source/deployment**
|
||||
|
||||
Use Compose labels, Nginx paths, or the documented service unit. Record repository path, SHA, dirty
|
||||
state, deployment command, container/network identity, and configuration owner.
|
||||
|
||||
**Step 2: Locate the sidebar link**
|
||||
|
||||
Use `rg` in the resolved source tree for the current ThothII URL, label, historical
|
||||
`datamart-builder`, and sidebar/navigation definitions. Record exact files and line numbers.
|
||||
|
||||
**Step 3: Resolve the real public origin**
|
||||
|
||||
The owner reports `aritmolab.policlinicosandonato.com`; historical project state mentions a `.it`
|
||||
origin and `/datamart-builder`. Record the live browser-visible origin and path from deployed
|
||||
configuration. Do not choose between them without evidence.
|
||||
|
||||
### Task 6: Survey Authentik
|
||||
|
||||
**Files:**
|
||||
- Create: `$PSD_SURVEY_ROOT/authentik.md`
|
||||
|
||||
**Step 1: Identify deployment and version**
|
||||
|
||||
Record Authentik containers/services, immutable image reference, version, base URL, database/Redis
|
||||
dependencies, configuration owner, and backup/export procedure. Do not print container environments.
|
||||
|
||||
**Step 2: Locate credential references**
|
||||
|
||||
Record only file/secret-object paths, ownership, mode, and whether the local operator can use them.
|
||||
If no usable administrative/API credential is found, stop and request owner help.
|
||||
|
||||
**Step 3: Inventory relevant objects read-only**
|
||||
|
||||
Using the installed version's API schema or admin interface, list only names/IDs for current
|
||||
Aritmolab applications/providers, authorization flows, property mappings, groups, service accounts,
|
||||
and policies that establish local conventions. Do not retrieve write-only secrets or raw tokens.
|
||||
|
||||
**Step 4: Record version-specific constraints**
|
||||
|
||||
Consult the official documentation matching the installed release for OAuth2/OIDC providers,
|
||||
scope/property mappings, application bindings, blueprints/export, and API permission semantics.
|
||||
Do not copy examples from a newer release without comparing the installed OpenAPI schema.
|
||||
|
||||
### Task 7: Survey Supabase and the PSD DWH
|
||||
|
||||
**Files:**
|
||||
- Create: `$PSD_SURVEY_ROOT/supabase.md`
|
||||
- Create: `$PSD_SURVEY_ROOT/dwh-readonly.txt`
|
||||
|
||||
**Step 1: Map Supabase services without environments**
|
||||
|
||||
Record PostgreSQL, pooler, PostgREST, gateway, and backup components; container networks and local
|
||||
listeners; database name; TLS listener/CA; and the approved direct-connect route from ThothII core.
|
||||
|
||||
**Step 2: Record exposed PostgREST schemas**
|
||||
|
||||
Query only the explicit PostgREST schema setting through its known configuration mechanism. Do not
|
||||
dump the whole environment. Confirm whether `thoth_sessions` already exists or is exposed.
|
||||
|
||||
**Step 3: Inspect schemas and migration state**
|
||||
|
||||
Through an approved administrative connection, run bounded catalog queries for existing schemas,
|
||||
owners, and any `thoth_sessions` tables/migration records. Do not change them.
|
||||
|
||||
**Step 4: Prove the intended DWH runtime identity is read-only**
|
||||
|
||||
Connect using the protected runtime credential mechanism and query `current_database()`,
|
||||
`current_user`, and grants for schema `datawarehouse`. Expected: USAGE/SELECT as required and no
|
||||
INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER, CREATE, or ownership privileges. Do not run a
|
||||
write probe against clinical tables.
|
||||
|
||||
**Step 5: Identify session-schema roles**
|
||||
|
||||
Record names or naming rules for a future migrator and runtime role. Do not create them. The final
|
||||
design uses the existing database plus schema `thoth_sessions`, never a new database.
|
||||
|
||||
### Task 8: Survey Git workspace and model boundaries
|
||||
|
||||
**Files:**
|
||||
- Create: `$PSD_SURVEY_ROOT/workspace-and-models.md`
|
||||
|
||||
**Step 1: Inspect the workspace remote read-only**
|
||||
|
||||
Record remote URL, branch, fetch credential path, current remote SHA, catalog entry, descriptor
|
||||
schema version, current `supported_transports`, Evidence tree, annotations blob, and deploy-key
|
||||
permissions. Do not push with the server's read-only deployment credential.
|
||||
|
||||
**Step 2: Inspect Pi/LLM policy**
|
||||
|
||||
Record selected provider/model/thinking level and credential references. Use bounded `tht pi status`
|
||||
and `pi doctor` where compatible. Do not output provider keys.
|
||||
|
||||
**Step 3: Check internal semantic capacity**
|
||||
|
||||
Record CPU/GPU availability, free storage, and whether Docker can run the pinned Qdrant and Ollama
|
||||
architectures. Do not pull images or models during the survey.
|
||||
|
||||
### Task 9: Produce the survey decision
|
||||
|
||||
**Files:**
|
||||
- Modify: `$PSD_SURVEY_ROOT/survey-report.md`
|
||||
|
||||
**Step 1: Complete the topology**
|
||||
|
||||
Include exact component owners and flows for user → load balancer → Nginx → Aritmolab/sidebar →
|
||||
ThothII, and core → Supabase DWH/auth session schema/Qdrant/Ollama/LLM/Authentik.
|
||||
|
||||
**Step 2: List exact intended change files**
|
||||
|
||||
Separate files owned by the new ThothII installation, workspace curator, Nginx, load balancer,
|
||||
Aritmolab, Authentik, and Supabase. Mark shared files as owner-gated.
|
||||
|
||||
**Step 3: State GO or NO-GO**
|
||||
|
||||
GO requires all mandatory paths, permissions, backup owners, and rollback boundaries. NO-GO must
|
||||
name concrete missing facts and the person/system needed to resolve them.
|
||||
|
||||
**Step 4: Hash and retain the report**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
sha256sum "$PSD_SURVEY_ROOT/survey-report.md" > "$PSD_SURVEY_ROOT/survey-report.sha256"
|
||||
sha256sum --check "$PSD_SURVEY_ROOT/survey-report.sha256"
|
||||
```
|
||||
|
||||
Expected: checksum passes. Move the complete mode-0700 survey directory to the approved protected
|
||||
evidence root without changing its contents; record the final path and digest in the journal.
|
||||
Reference in New Issue
Block a user