ThothII/docs/plans/2026-08-20-psd-server-project-a-standalone.md

# PSD Server Project A Standalone Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Install a clean PSD ThothII stack with local authentication, direct read-only DWH access, internal Qdrant/Ollama, rebuilt preprocessing, and one completed F1-F8 work session.

**Architecture:** Keep the stopped legacy installation intact only as a temporary rollback boundary and deploy the current canonical five-service Compose stack from an adjacent clean clone. Create no host service identity: the image's UID/GID 10001 remains numeric and unmapped, confined to the new writable bind roots. A reviewed server-local override disables public exposure and uses filesystem work sessions; an optional load-balancer route is permitted only when it is operator-only and its denial boundary is proved first.

**Tech Stack:** Git, Docker/Compose, native `tht`, local Argon2id authentication, PSD Supabase PostgreSQL direct transport, Qdrant, Ollama, Pi, Nginx/load-balancer test route where safe.

---

## Preconditions

- Common survey result is `SURVEY_GO_PROJECT_A_PRIVATE` and its digest is recorded.
- Every path below is replaced by the exact survey result before execution.
- No production Nginx/load-balancer/sidebar/Authentik change is in scope.
- The old stack remains running until its exact inventory and restart recipe are verified; old and
  new stacks never run together.
- The server's workspace deploy credential remains read-only. A curator with write access publishes
  the workspace change.
- Owner amendment 2026-08-21 declares legacy sessions/configuration disposable. Retain old resources
  only until Project B proves the production Aritmolab journey, then delete them under a separate
  exact cleanup authorization. No legacy backup is required.
- Do not create a host user or group for 10001. Immediately before preparing `/srv/thothii`, both
  `getent passwd 10001` and `getent group 10001` must return no match.
- Owner amendment 2026-08-21 defers live Mac `rest_api` acceptance and `legacy-shared` revocation to
  the mandatory pre-Project-B gate. It does not authorize stop/start; those require a later explicit
  owner gate even after private preparation is complete.

### Task 1: Freeze exact inputs

**Files:**
- Read: protected survey report
- Read: `docs/plans/2026-08-20-psd-server-deployment-program-design.md`
- Record: protected Project A journal

**Step 1: Record application identity**

Run in the new planning checkout:

```bash
git status --short --branch
git rev-parse HEAD
git rev-parse origin/main
git diff --check
```

Expected: clean and explicitly approved SHA.

**Step 2: Record workspace remote identity**

Use the surveyed read-only credential and run:

```bash
git ls-remote <workspace-remote> refs/heads/main
```

Expected: one SHA recorded as the pre-change workspace revision.

**Step 3: Check old-stack recoverability**

Expected: exact old start/stop procedure, source SHA, Compose identity, volumes/binds, proxy closure
procedure, restart recipe, and shared-resource exclusions are present in the survey. Stop if any is
missing.

### Task 2: Publish the multi-transport workspace revision

**Files:**
- Modify in authorized curator clone: `psd-clinical/workspace.yaml`
- Verify: `thoth-workspaces.yaml`

**Step 1: Create a clean curator branch**

Run in a write-authorized clone, never in the application-managed registry checkout:

```bash
git status --short --branch
git fetch origin main
git switch --create codex/psd-direct-transport origin/main
```

Expected: clean branch at the recorded remote SHA.

**Step 2: Make the minimal descriptor change**

Change exactly:

```yaml
supported_transports: [rest_api]
```

to:

```yaml
supported_transports: [rest_api, postgres_direct]
```

Do not duplicate the workspace or change its ID, collection, Evidence, annotations, model policy,
database, or schema.

**Step 3: Review the descriptor-only diff**

Run:

```bash
git diff --check
git diff -- psd-clinical/workspace.yaml thoth-workspaces.yaml
```

Expected: one semantic line changed; catalog metadata remains identical.

**Step 4: Validate with the current ThothII contract**

Use a disposable installation/registry or the repository's current registry validation harness to
activate the candidate commit before publication. Expected: schema v3 accepts both transports,
Evidence and annotations materialize, and no secret is required for source validation.

If no supported validator can be run in the curator environment, stop and request the owner to run
the established Mac validation; do not publish based only on YAML parsing.

**Step 5: Commit and publish through curator review**

```bash
git add psd-clinical/workspace.yaml
git diff --cached --check
git commit -m "feat: support direct PSD DWH transport"
git push --set-upstream origin codex/psd-direct-transport
```

Merge through the repository's normal review path. Record the resulting `main` SHA.

**Step 6: Record the deferred Mac REST proof**

Do not change the Mac during Project A. Record `DEFERRED_PRE_PROJECT_B`, the unchanged expected
transport `rest_api`, and the exact future diagnostics. The proof must become PASS before Project B,
after protected delivery/configuration of the per-installation key.

### Task 3: Back up and stop the legacy installation

**Files:**
- Create: surveyed protected legacy backup directory
- Record: Project A journal

**Step 1: Capture final legacy state**

Run the surveyed legacy status/doctor commands, record source SHA and image IDs, and confirm no
active user work. Do not use the new `tht` against an incompatible old descriptor.

**Step 2: Close or maintenance-gate the old ThothII route**

Change only the surveyed ThothII-specific route using its established mechanism. Validate Nginx and
load-balancer configuration before applying. Confirm external requests no longer reach the app.

**Step 3: Record the disposable legacy boundary**

Record exact container and image IDs plus filesystem device/inode/ownership/size for
`/home/chirone/ThothII` and `/home/chirone/thothii-data`. Do not archive their contents: the owner
declared them disposable. Prove that the external Evidence bind and both shared Docker networks
are excluded from any later cleanup manifest.

**Step 4: Stop the old stack**

Use its own supported controller. Expected: old containers stopped, not removed; volumes and bind
trees unchanged.

**Step 5: Rehearse the restart command without executing it**

Record the exact command, preconditions, port ownership, and route-restoration order. If it cannot
be stated unambiguously, stop before creating the new stack.

### Task 4: Prepare the adjacent clean installation

**Files:**
- Create: survey-selected new source root
- Create: survey-selected operator, secret, data, Pi-state, registry, and backup roots

**Step 1: Create dedicated paths without creating identities**

Follow `docs/install/server.md` numeric-ownership rules. Do not call `useradd`, `groupadd`, or
`usermod`. The existing operator owns source/operator files; only the new data, Pi-state, and
workspace-registry roots use unmapped numeric `10001:10001`. Stop if UID or GID 10001 resolves to a
host account, and do not change the image identity without a reviewed design amendment.

**Step 2: Clone the frozen application source**

```bash
git -c core.autocrlf=false clone <thothii-remote> <new-source-root>/ThothII
git -C <new-source-root>/ThothII config --local core.autocrlf false
git -C <new-source-root>/ThothII switch --detach <approved-application-sha>
git -C <new-source-root>/ThothII status --short --branch
```

Expected: detached exact SHA, clean tree.

**Step 3: Verify source and platform**

```bash
cd <new-source-root>/ThothII
bash scripts/verify-line-endings.sh
docker version
docker compose version
```

Expected: all pass.

**Step 4: Prepare Pi state and build the operator**

```bash
sudo scripts/prepare-server-pi-state.sh <new-pi-state-root> 10001 10001
THT_THT_OUTPUT_DIRECTORY=<protected-build-output> bash scripts/build-tht.sh
```

Install only the binary matching the surveyed server architecture. Run `tht version --json` and
record its source identity.

### Task 5: Create the protected Project A configuration

**Files:**
- Create outside Git: `<project-a-operator-root>/server.env`
- Create outside Git: `<project-a-operator-root>/thothii-installation.yaml`
- Create outside Git: `<project-a-operator-root>/project-a-private.yaml`
- Create outside Git: `<project-a-auth-root>/auth.yaml` through `tht`

**Step 1: Start from current examples**

Copy `deploy/env/server.env.example` and `docs/install/examples/thothii-installation.server.yaml`
to the protected Project A operator root. Replace every placeholder with surveyed absolute paths.
Never source `server.env` as shell code.

**Step 2: Add the private/local-session override**

Create this reviewed override:

```yaml
services:
  core:
    environment:
      THOTH_PUBLIC_EXPOSURE: "false"
      THT_SESSION_STORAGE: local
  frontend:
    ports: !override
      - "127.0.0.1:<project-a-port>:8080"
```

Select an unused loopback port proved by `ss -lntp`. Do not publish core, Qdrant, or Ollama.

**Step 3: Compose the installation descriptor**

Use `profile: server`, the exact new source root/env/auth root, workspace remote/branch/read-only
access, Project A override, and exactly one Git transport override. Do not include the public
session-server overlay in Project A.

**Step 4: Validate permissions and render**

```bash
<new-tht> --installation <project-a-installation> update --check-only
```

Expected: Compose validates; only frontend has a loopback port; core declares public exposure false
and local session storage; Qdrant/Ollama are internal.

**Step 5: Configure the local administrator**

Create a temporary mode-0600 password file using an echo-free prompt, then run:

```bash
<new-tht> --installation <project-a-installation> auth configure \
  --mode local --public-url <project-a-origin> \
  --admin-user <test-admin> --admin-display-name <display-name> \
  --password-file <protected-temporary-password-file>
```

Remove the temporary input file after success and record that removal. Do not delete generated
`auth.yaml` or `users.yaml`.

### Task 6: Build and start the clean stack

**Files:**
- Record: Project A evidence directory

**Step 1: Build current images**

```bash
cd <new-source-root>/ThothII
bash scripts/build-local.sh
```

Expected: current core/frontend images build; pinned Qdrant/Ollama references resolve.

**Step 2: Run preflight**

```bash
<new-tht> --installation <project-a-installation> update --check-only
<new-tht> --installation <project-a-installation> pi doctor
```

Expected: no mutation error and no secret in output.

**Step 3: Start through `tht`**

```bash
<new-tht> --installation <project-a-installation> start --build
<new-tht> --installation <project-a-installation> status
<new-tht> --installation <project-a-installation> doctor --json
<new-tht> --installation <project-a-installation> pi test
```

Expected: frontend, core, Qdrant, embedding healthy and model initializer completed. Doctor has the
documented ordered checks and authentication PASS.

**Step 4: Verify listener boundaries**

Use `ss -lntp` and bounded Docker inspection. Expected: only the selected frontend loopback port is
host-published; no external core, Qdrant, or Ollama listener.

### Task 7: Activate the workspace and direct DWH binding

**Files:**
- Modify only through authenticated Workspace Management: encrypted workspace secret store

**Step 1: Pull and inspect the reviewed workspace revision**

```bash
<new-tht> --installation <project-a-installation> \
  workspace inspect --workspace psd-clinical --json
```

Expected: active workspace SHA equals the approved multi-transport revision.

**Step 2: Configure runtime bindings**

Through the authenticated Workspace Management API/UI, select `postgres_direct` and provide the
surveyed host, port, runtime user, password, and optional TLS CA. Secret values go to the encrypted
vault; they do not enter `server.env`, Git, shell arguments, or evidence.

The generated contract names are:

```text
THT_WS_PSD_CLINICAL_DWH_TRANSPORT
THT_WS_PSD_CLINICAL_DWH_HOST
THT_WS_PSD_CLINICAL_DWH_PORT
THT_WS_PSD_CLINICAL_DWH_USER
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE
THT_WS_PSD_CLINICAL_DWH_TLS_CA_FILE
```

**Step 3: Validate and test connections**

Run static validation, live connection test, workspace inspect, and `doctor --json`. Expected: DWH,
workspace, internal embedding, and Qdrant checks pass with redacted output.

**Step 4: Re-prove read-only grants**

Use the survey's catalog query through the exact configured identity. Expected: no DML/DDL grant on
`datawarehouse`. Stop if the runtime user is an owner, superuser, or write-capable role.

### Task 8: Rebuild and verify semantic preprocessing

**Files:**
- Create: Project A preprocessing evidence

**Step 1: Inspect the empty/new collection state**

```bash
<new-tht> --installation <project-a-installation> \
  workspace vector inspect --workspace psd-clinical --json
```

Expected: either a compatible empty collection or the documented missing-collection state.

**Step 2: Create the descriptor-owned collection when missing**

Use the guarded vector rebuild only for collection `psd-clinical`, with exact repeated confirmation
and `--destroy`. Do not run it against any other collection.

**Step 3: Run complete preprocessing**

```bash
<new-tht> --installation <project-a-installation> \
  workspace preprocess run --workspace psd-clinical --json
```

If it returns `manual_review_required`, inspect the exact run and curated annotations, obtain the
required human decision, run `workspace schema accept --workspace psd-clinical --run <run-id> --yes`,
then resume the same run. Never auto-approve unknown FK changes.

**Step 4: Verify collection contract and counts**

Run vector inspection and record dimensions, cosine distance, required keyword indexes, and bounded
counts by payload kind/revision. Expected: all points carry the active workspace revision.

**Step 5: Prove idempotency**

Run the complete preprocessing command again. Expected: no new review, no duplicate logical points,
unchanged Evidence reported as unchanged, and the same effective configuration identity.

### Task 9: Configure and test local users

**Files:**
- Modify through `tht auth user`: protected local user registry

**Step 1: Add an ordinary test user**

Use an echo-free prompt or protected temporary password file:

```bash
<new-tht> --installation <project-a-installation> auth user add <test-user> \
  --role user --display-name <display-name> --password-file <protected-temporary-password-file>
```

Remove the temporary input file after success.

**Step 2: Run authentication diagnostics**

```bash
<new-tht> --installation <project-a-installation> auth status --json
<new-tht> --installation <project-a-installation> auth check --json
```

Expected: pristine redacted JSON and PASS.

**Step 3: Execute automated local-auth cases**

Use same-origin requests or a local headless browser to prove ordinary/admin authorization, generic
wrong-password failure, disable/enable, password/role revision invalidation, logout-all, CSRF, and
remembered-session survival after core restart. Do not retain cookie jars after the test.

### Task 10: Optionally add the private network-path test

**Current scope boundary (owner, 2026-08-21):** omit this entire task and keep Project A
loopback-only. Any future use requires a separate shared-infrastructure authorization after the
public-origin and load-balancer activities pass; the Project A private survey decision alone is
insufficient.

**Files:**
- Modify only surveyed test-specific load-balancer/Nginx files
- Create: test certificate through the existing managed mechanism

**Step 1: Prove allowlist capability before proxying**

Create a temporary hostname that returns a fixed maintenance response. From an approved operator
source expect success; from an unapproved source expect denial. Do not point it at ThothII yet.

**Step 2: Validate and activate the test proxy**

Configure Nginx with the same Host/HTTPS forwarding and SSE settings intended for production. Run
`nginx -t`, validate the load balancer, then reload through the established mechanism.

**Step 3: Reconfigure local-auth public URL transactionally**

If the exact private HTTPS origin differs from the loopback origin, use the supported authentication
configuration workflow and invalidate prior test sessions. Re-run auth and doctor checks.

**Step 4: Prove both sides**

Expected: authorized operator reaches the local login; unauthorized source remains denied before
ThothII. If this cannot be demonstrated, remove the test route and continue on loopback.

### Task 11: Complete the F1-F8 acceptance session

**Files:**
- Complete: `docs/testing/psd-server-project-a-manual.md`
- Create: protected session evidence

**Step 1: Select the approved harmless question**

Use a known read-only PSD question agreed by the owner. Record the wording in the protected report;
do not include patient-identifying values.

**Step 2: Create the session as the ordinary local user**

Use the private browser route when present; otherwise drive the same-origin frontend/API from the
server-local terminal/headless browser. Record only session ID and sanitized milestones.

**Step 3: Review every gate**

Complete F1-F8 without auto-confirming human decisions. Confirm persisted phase/artifact state after
each gate and resume once to prove recovery.

**Step 4: Validate final SQL**

Expected: finalized session, DWH validation PASS, SQL is read-only, and no clinical mutation occurs.

**Step 5: Inspect persisted state**

Confirm manifest, question, schema linking, Evidence, CTE plan/tests, final SQL, validation report,
and decision ledger exist in local filesystem session storage. Chat/SSE need not persist.

### Task 12: Close Project A and preserve rollback

**Files:**
- Complete: `docs/testing/evidence/psd-server-project-a-report-template.md`

**Step 1: Run final diagnostics**

Run status, doctor, auth check, workspace inspect, vector inspect, Pi test, and a bounded secret scan
of the intended report.

**Step 2: Create a transactional new-installation backup**

Use `tht backup --drain` with a protected explicit output. Verify its checksum. Do not include
secrets in the ordinary evidence archive.

**Step 3: Complete human acceptance**

Every private-server row in `docs/testing/psd-server-project-a-manual.md` must be PASS or explicitly
blocking. Only the Mac REST row may be `DEFERRED_PRE_PROJECT_B` under the dated owner amendment.

**Step 4: Record the gate**

Record exact SHAs/images, workspace revision, preprocessing identity/counts, session ID, report
digest, rollback status, and explicit `PROJECT_A_PRIVATE_PASS` or `PROJECT_A_FAIL`. A private PASS
does not authorize Project B while the deferred gate remains open.

**Step 5: Stop on FAIL**

On FAIL, stop the new stack and use the surveyed old-stack recovery plan if service restoration is
desired. Do not start Project B.