mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: adopt clean PSD replacement model

Browse Source
This commit is contained in:
User
2026-08-21 16:05:11 +02:00
parent 9974fb4bc0
commit 042af932ee
10 changed files with 527 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/plans/2026-08-21-psd-clean-replacement.md
+259
View File
@@ -0,0 +1,259 @@
# PSD Clean ThothII Replacement Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the disposable PSD ThothII installation without creating host identities and remove only its exclusive resources after the new Aritmolab journey passes.
**Architecture:** Project A builds a separate `/srv/thothii` installation while the old resources remain an inert rollback boundary. The container keeps numeric UID/GID 10001 without a host account. Project B moves the Aritmolab integration to the accepted stack; exact legacy deletion is a final post-acceptance operation that excludes shared Chirone resources.
**Tech Stack:** Linux ownership and identity checks, Docker Compose, native `tht`, Nginx/Aritmolab integration, protected evidence journals.
## Global Constraints
- Never call `useradd`, `groupadd`, `usermod`, or edit `/etc/passwd`, `/etc/group`, `/etc/shadow`, or `/etc/gshadow`.
- Treat UID/GID `10001:10001` as an unmapped numeric container identity only.
- Never reuse `/home/chirone/thothii-data` for the replacement.
- Never remove `omics_portal_omics_network`, `localllm_default`, `/home/chirone/chirone/etl/docs/evidence`, or any Omics Portal, LocalLLM, DWH, `dwh-auth`, Supabase, Authentik, ETL, or Superset resource.
- Never run `docker system prune`, `docker network prune`, `docker volume prune`, or `docker compose down --volumes`.
- Do not stop the old stack or start the new stack without the separate owner mutation gate already required by Project A.
- Do not delete any legacy target until Project B automated PASS, human PASS, and explicit owner PASS are all recorded.
---
### Task 1: Bind the clean-replacement decision to the surveyed host
**Files:**
- Read: `/etc/passwd`
- Read: `/etc/group`
- Read: `/home/chirone/ThothII/compose.yaml`
- Read: `/home/chirone/omics_portal/nginx/nginx.conf`
- Record: protected Project A survey journal
**Interfaces:**
- Consumes: approved design `docs/superpowers/specs/2026-08-21-psd-clean-replacement-design.md`.
- Produces: a redacted exact-target inventory and a `numeric_identity_unmapped=PASS` decision.
- [ ] **Step 1: Prove that 10001 is not a host identity**
```bash
if getent passwd 10001 >/dev/null || getent group 10001 >/dev/null; then
printf '%s\n' 'numeric_identity_unmapped=FAIL'
exit 1
fi
printf '%s\n' 'numeric_identity_unmapped=PASS'
```
Expected: one PASS line. Do not continue if either lookup succeeds.
- [ ] **Step 2: Revalidate the exclusive legacy containers and images**
```bash
docker inspect --format '{{.Name}} project={{index .Config.Labels "com.docker.compose.project"}} image={{.Image}}' \
thothii-core-1 thothii-frontend-1
docker image inspect --format '{{.Id}} {{join .RepoTags ","}}' \
thothii-core:local thothii-frontend:local
```
Expected: exactly the two `thothii` project containers and two immutable image IDs. Record IDs,
not environment variables or container configuration values.
- [ ] **Step 3: Revalidate shared exclusions**
```bash
docker network inspect --format '{{.Name}} project={{index .Labels "com.docker.compose.project"}}' \
omics_portal_omics_network localllm_default
stat -c '%u:%g %a %n' /home/chirone/chirone/etl/docs/evidence
```
Expected: both networks exist and the external Evidence path remains outside the legacy data tree.
- [ ] **Step 4: Record the survey checkpoint**
Record only names, IDs, modes, ownership numbers, PASS/FAIL decisions, and timestamps in the
protected journal. Never record container environment, auth files, API keys, or raw Nginx output.
### Task 2: Prepare the replacement roots without host accounts
**Files:**
- Create: `/srv/thothii/source`
- Create: `/srv/thothii/operator`
- Create: `/srv/thothii/data`
- Create: `/srv/thothii/secrets`
- Create: `/srv/thothii/pi-state`
- Create: `/srv/thothii/workspace-registry`
- Create: `/srv/thothii-backups`
- Test: exact ownership/mode checks below
**Interfaces:**
- Consumes: `numeric_identity_unmapped=PASS` from Task 1.
- Produces: isolated bind roots compatible with container UID/GID 10001 and operator UID/GID 1013:1006.
- [ ] **Step 1: Create only the reviewed roots**
```bash
sudo install -d -o 1013 -g 10001 -m 0750 /srv/thothii
sudo install -d -o 1013 -g 1006 -m 0750 /srv/thothii/source
sudo install -d -o 1013 -g 1006 -m 0750 /srv/thothii/operator
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/data
sudo install -d -o 10001 -g 1006 -m 0750 /srv/thothii/secrets
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/pi-state
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/workspace-registry
sudo install -d -o root -g root -m 0700 /srv/thothii-backups
```
Expected: commands create directories only. They do not create identities.
- [ ] **Step 2: Verify identity databases are unchanged and modes are exact**
```bash
if getent passwd 10001 >/dev/null || getent group 10001 >/dev/null; then exit 1; fi
stat -c '%u:%g %a %n' \
/srv/thothii /srv/thothii/source /srv/thothii/operator /srv/thothii/secrets \
/srv/thothii/data /srv/thothii/pi-state /srv/thothii/workspace-registry \
/srv/thothii-backups
```
Expected ownership/modes, in order: `1013:10001 750`, twice `1013:1006 750`,
`10001:1006 750`, three times `10001:10001 750`, and `0:0 700`.
- [ ] **Step 3: Commit documentation/code changes before runtime use**
Replace the account-creation section of `docs/install/server.md` with the numeric ownership model:
the invoking operator UID/GID owns source and operator paths, numeric 10001 owns only secrets and
writable runtime paths, and the manual verifies both `getent` lookups remain empty. Remove every
`useradd`, `groupadd`, `usermod`, `sudo -u thothii`, and `thothii-ops` instruction. Run the existing
documentation gates, obtain review, commit, and push before Project A uses these roots. Runtime
state under `/srv` is never committed.
### Task 3: Execute Project A with a stopped-but-intact legacy boundary
**Files:**
- Read: `docs/plans/2026-08-20-psd-server-project-a-standalone.md`
- Record: protected Project A report and journal
**Interfaces:**
- Consumes: Task 2 roots and a separately approved Project A mutation gate.
- Produces: Project A automated PASS and explicit human PASS while the old resources remain intact.
- [ ] **Step 1: Close the old ThothII route using the separately approved exact route operation**
Validate configuration before applying it. Do not change `/dwh/` or unrelated Omics routes.
- [ ] **Step 2: Stop only the two old ThothII containers**
Use the surveyed legacy Compose controller. Do not remove containers, images, networks, source, or
data. Confirm Omics Portal, LocalLLM, DWH, `dwh-auth`, Supabase, Authentik, ETL, and Superset remain
running.
- [ ] **Step 3: Execute and close Project A**
Follow the Project A plan and manual through its automated and human PASS decisions. On failure,
stop the new stack and restart the still-present old containers; do not delete either installation.
### Task 4: Cut Aritmolab over in Project B
**Files:**
- Read: `docs/plans/2026-08-20-psd-server-project-b-authentik.md`
- Read: `docs/testing/psd-server-project-b-manual.md`
- Record: protected Project B report and journal
**Interfaces:**
- Consumes: accepted Project A candidate and the independent pre-Project-B gate.
- Produces: a production route that no longer depends on the old container names.
- [ ] **Step 1: Complete the mandatory pre-Project-B gates**
Mac REST acceptance, the 48-hour/two-ETL observation, and `legacy-shared` revocation must be PASS.
These DWH-key gates are independent from deleting the old ThothII application.
- [ ] **Step 2: Execute Project B without deleting legacy resources**
Follow the Project B plan. Keep the old containers stopped and intact until all automated checks
pass.
- [ ] **Step 3: Run the real Aritmolab acceptance**
Prove the complete path from Aritmolab home and sidebar through authentication to frontend assets,
API, SSE, and one completed workflow. Confirm the active route no longer resolves the legacy
`thothii-core` or `thothii-frontend` services.
- [ ] **Step 4: Record human and owner PASS**
No cleanup command is authorized by technical success alone. Record the Project B human PASS and a
separate owner decision explicitly authorizing the exact cleanup inventory from Task 5.
### Task 5: Delete only exclusive legacy ThothII resources
**Files:**
- Delete: `/home/chirone/ThothII`
- Delete: `/home/chirone/thothii-data`
- Record: protected final cleanup report
**Interfaces:**
- Consumes: Project B automated PASS, human PASS, owner PASS, and immutable IDs from Task 1.
- Produces: no legacy ThothII application resources and no change to shared Chirone resources.
- [ ] **Step 1: Revalidate the cleanup manifest immediately before deletion**
Re-run Task 1. Stop if an image has another consumer, a legacy container is running, the production
route still mentions a legacy service, or either filesystem path resolves outside its expected
exact target. Record device/inode, ownership, and size; never record file contents.
- [ ] **Step 2: Remove only the stopped legacy containers through their Compose project**
Remove `thothii-core-1` and `thothii-frontend-1` without `--volumes`. Do not remove either shared
network.
- [ ] **Step 3: Remove only the two unreferenced legacy image IDs**
Resolve tags to the immutable IDs recorded in Task 1 and prove no container references them before
removal. Do not prune images globally.
- [ ] **Step 4: Remove the two exact legacy filesystem trees**
Delete only `/home/chirone/ThothII` and `/home/chirone/thothii-data` after their exact identities
match the approved manifest. This operation is irreversible by design; no legacy session or state
restore is promised after this point.
- [ ] **Step 5: Verify shared resources and the production journey**
Confirm both shared Docker networks, the external Evidence path, and every out-of-scope service
still exist. Repeat the Aritmolab sidebar, authentication, frontend, API/SSE, and completed-workflow
checks. Record `CLEAN_REPLACEMENT_PASS` only if all checks pass.
### Task 6: Commit the durable evidence references
**Files:**
- Modify: `PROJECT_STATE.md`
- Modify: approved redacted report index only
**Interfaces:**
- Consumes: Project A, Project B, and cleanup evidence digests.
- Produces: a secret-free durable state summary.
- [ ] **Step 1: Update the state without secret-bearing evidence**
Record candidate SHAs, image IDs, workspace SHA, report digests, gate decisions, deleted exact
targets, and preserved shared resources. Protected journals and credentials stay outside Git.
- [ ] **Step 2: Run documentation and secret gates**
```bash
bash scripts/test-verify-workspace-install-docs.sh
bash scripts/test-verify-dwh-auth-docs.sh
bash scripts/auth-docs-smoke.sh
git diff --check
```
Expected: every script exits zero and the diff check has no output.
- [ ] **Step 3: Commit and push the secret-free state update**
```bash
git add PROJECT_STATE.md
git commit -m "docs: record PSD clean replacement completion"
git push origin feat/dwh-rest-installation-auth
```
Expected: the push contains no `/srv` state, credential, protected journal, raw Nginx output, or
legacy data.
docs/superpowers/specs/2026-08-21-psd-clean-replacement-design.md
+86
View File
@@ -0,0 +1,86 @@
# PSD Clean ThothII Replacement Design
**Date:** 2026-08-21
**Status:** Approved by the owner
## Decision
The existing PSD ThothII installation is disposable. Its sessions, settings, Pi state, derived
artifacts, indexes, images, source checkout, and application data are not migration inputs for the
new installation. They remain present only until the replacement has passed the real Aritmolab
journey; this temporary retention is a cutover safeguard, not a legacy-support requirement.
The replacement does not create a host `thothii` user or group. The image keeps its internal
UID/GID `10001:10001`. Host bind trees that the container must write may use that numeric ownership
without corresponding `/etc/passwd` or `/etc/group` entries. Source and operator-controlled files
remain owned by the existing operator `admlocforn1` (UID 1013) and the existing `chirone` group
(GID 1006).
## Boundaries
The following old ThothII resources become deletion candidates only after Project B human
acceptance proves the production Aritmolab route:
- containers `thothii-core-1` and `thothii-frontend-1`;
- images `thothii-core:local` and `thothii-frontend:local`, after proving no other container uses
their immutable IDs;
- checkout `/home/chirone/ThothII`;
- application bind tree `/home/chirone/thothii-data`.
The following resources are shared and are never deletion candidates in the ThothII cleanup:
- Docker networks `omics_portal_omics_network` and `localllm_default`;
- `/home/chirone/chirone/etl/docs/evidence` and the ETL project;
- Omics Portal/Aritmolab source, Nginx, web, worker, sidebar, and capability configuration;
- LocalLLM API and vLLM services;
- `dwh-auth`, its credential registry, and the `/dwh/` Nginx route;
- Supabase/DWH, Authentik, Superset, and their data or configuration.
No global Docker prune, network removal, broad recursive deletion, or Compose volume deletion is
permitted. Cleanup resolves and revalidates every exact target immediately before removal.
## Sequence
1. Project A preparation creates a distinct installation under `/srv/thothii`; it does not reuse
`/home/chirone/thothii-data`.
2. Immediately before creating bind roots, verify that UID and GID 10001 still have no host account
mapping. A newly observed mapping is a stop condition requiring owner review.
3. Project A may stop the old containers only under its separate mutation authorization. The old
source, data, containers, and images remain intact while the new private stack is tested.
4. Project B changes the production integration and proves the complete path:
`Aritmolab -> sidebar -> Nginx/load balancer -> ThothII`, including frontend assets, API, SSE,
authentication, and a completed workflow.
5. Only after the Project B automated report, human report, and owner decision are PASS may the
exact old ThothII resources be deleted.
If Project A fails, stop the new private stack and restart the still-present old containers. If
Project B fails, close the new ingress and restore the prior route while the old resources still
exist. After Project B PASS and legacy deletion there is deliberately no promise to restore old
ThothII sessions or state.
## Host ownership model
No command may call `useradd`, `groupadd`, `usermod`, or modify the host identity databases.
- `/srv/thothii/source` and `/srv/thothii/operator`: `1013:1006`.
- `/srv/thothii/data`, `/srv/thothii/pi-state`, and `/srv/thothii/workspace-registry`:
numeric `10001:10001`.
- protected files mounted read-only by the core: operator-owned with a narrowly selected numeric
group/owner mode that permits UID/GID 10001 to read only the required file.
- `/srv/thothii-backups`: operator/root protected and outside the runtime write boundary.
Numeric ownership does not create or preserve a host user. It is confined to the new installation
tree and exists solely to match the non-root identity already embedded in the container image.
## Acceptance
The design is satisfied only when evidence proves all of the following:
- no host account or group was created for 10001;
- the new core runs non-root and can write only its intended runtime trees;
- the shared networks and external Evidence tree remain unchanged;
- Project A and Project B pass their independent automated and human gates;
- Aritmolab no longer resolves production traffic to `thothii-core` or `thothii-frontend` legacy
containers before those containers are removed;
- the cleanup inventory contains only exact legacy ThothII targets and excludes every shared
resource listed above.