docs: adopt clean PSD replacement model
This commit is contained in:
User
+48
-48
@@ -40,53 +40,53 @@ Read [server workspace-registry installation](server-workspace-registry.md),
|
||||
|
||||
## Service account and directories
|
||||
|
||||
The container runtime identity is fixed at UID/GID 10001. Reserve the same host ID for a dedicated
|
||||
non-login `thothii` account so bind-mounted ownership is obvious. Stop if either ID is already used
|
||||
by another account; choose a reviewed host mapping instead of changing the image identity.
|
||||
The container runtime identity is fixed at UID/GID 10001. It does not require or permit creation of
|
||||
a matching host account or group. Keep the number unmapped and use numeric ownership only for the
|
||||
dedicated bind paths that the non-root container must read or write. If either lookup below finds a
|
||||
host identity, stop and design an explicit remapping before installation.
|
||||
|
||||
```sh
|
||||
getent passwd 10001
|
||||
getent group 10001
|
||||
sudo useradd --system --uid 10001 --user-group --home-dir /srv/thothii \
|
||||
--create-home --shell /usr/sbin/nologin thothii
|
||||
if getent passwd 10001 >/dev/null || getent group 10001 >/dev/null; then
|
||||
printf '%s\n' 'UID/GID 10001 is already mapped; stop' >&2
|
||||
exit 1
|
||||
fi
|
||||
operator_uid="$(id -u)"
|
||||
operator_gid="$(id -g)"
|
||||
test "$operator_uid" -ne 0
|
||||
id -nG | tr ' ' '\n' | grep -Fx docker >/dev/null
|
||||
```
|
||||
|
||||
Use a dedicated `thothii-ops` group for the small set of human operators. A human who runs
|
||||
`tht` must be in both `thothii-ops` (to traverse operator paths and read declared secret files
|
||||
for output redaction) and the host `docker` group (to invoke Docker). Docker-group membership is
|
||||
effectively host-root access; grant both memberships only to reviewed administrators. The
|
||||
non-login `thothii` account owns files and writable data but does not need Docker access.
|
||||
|
||||
```sh
|
||||
sudo groupadd --system thothii-ops
|
||||
sudo usermod --append --groups thothii-ops,docker "$USER"
|
||||
```
|
||||
|
||||
Log out and back in before continuing; `id` must show both groups. Do not run `tht` through
|
||||
`sudo -u thothii`: that account deliberately lacks Docker access. Do not grant the human direct
|
||||
write access to runtime bind trees.
|
||||
The invoking, pre-existing administrator owns source and operator files. It must already have the
|
||||
site-approved Docker access required to run `tht`; this guide never changes group membership.
|
||||
Docker-group membership is effectively host-root access and must remain limited to reviewed
|
||||
administrators. Do not grant the operator direct write access to container runtime trees.
|
||||
|
||||
Create explicit directories. `source` contains the clone; `operator` contains untracked path-only
|
||||
configuration; the three writable trees are bind-mounted into `core`; `secrets` contains regular
|
||||
files only. Backups are separate from live data. Reset the account home explicitly because
|
||||
distribution `useradd` defaults may otherwise leave `/srv/thothii` non-traversable by
|
||||
`thothii-ops`.
|
||||
files only. The parent is owned by the operator with numeric group 10001 so both the operator and
|
||||
container can traverse it. Numeric ownership does not add entries to `/etc/passwd` or `/etc/group`.
|
||||
|
||||
```sh
|
||||
sudo install -d -o 10001 -g thothii-ops -m 2750 /srv/thothii
|
||||
sudo install -d -o 10001 -g thothii-ops -m 2750 /srv/thothii/source
|
||||
sudo install -d -o 10001 -g thothii-ops -m 2770 /srv/thothii/operator
|
||||
sudo install -d -o 10001 -g thothii-ops -m 2750 /srv/thothii/secrets
|
||||
operator_uid="$(id -u)"
|
||||
operator_gid="$(id -g)"
|
||||
sudo install -d -o "$operator_uid" -g 10001 -m 0750 /srv/thothii
|
||||
sudo install -d -o "$operator_uid" -g "$operator_gid" -m 0750 /srv/thothii/source
|
||||
sudo install -d -o "$operator_uid" -g "$operator_gid" -m 0750 /srv/thothii/operator
|
||||
sudo install -d -o 10001 -g "$operator_gid" -m 0750 /srv/thothii/secrets
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/data
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/pi-state
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/workspace-registry
|
||||
sudo install -d -o root -g root -m 0700 /srv/thothii-backups
|
||||
stat -c '%u:%g %a %n' \
|
||||
/srv/thothii /srv/thothii/source /srv/thothii/operator /srv/thothii/secrets \
|
||||
/srv/thothii/data /srv/thothii/pi-state /srv/thothii/workspace-registry \
|
||||
/srv/thothii-backups
|
||||
```
|
||||
|
||||
Verify `/srv/thothii` is owned by `10001:thothii-ops` with mode `2750`. The human operator can
|
||||
traverse the parent but can write only `operator`; setgid keeps generated files in `thothii-ops`.
|
||||
`source`, `secrets`, and all runtime bind trees remain non-group-writable. Do not make
|
||||
`/srv/thothii` a shared application directory.
|
||||
Expected: the parent is `operator_uid:10001 750`; source/operator are
|
||||
`operator_uid:operator_gid 750`; secrets are `10001:operator_gid 750`; the three runtime trees are
|
||||
`10001:10001 750`; backups are `0:0 700`. Re-run the empty `getent` checks after creation. Do not
|
||||
make `/srv/thothii` a shared application directory.
|
||||
|
||||
## Firewall and network boundaries
|
||||
|
||||
@@ -187,10 +187,10 @@ Never add those services to ThothII's mandatory Compose files. Follow
|
||||
Clone with LF line endings, then verify before every build:
|
||||
|
||||
```sh
|
||||
sudo -u thothii git -c core.autocrlf=false clone \
|
||||
git -c core.autocrlf=false clone \
|
||||
https://github.example.invalid/your-org/ThothII.git /srv/thothii/source/ThothII
|
||||
cd /srv/thothii/source/ThothII
|
||||
sudo -u thothii git config --local core.autocrlf false
|
||||
git config --local core.autocrlf false
|
||||
bash scripts/verify-line-endings.sh
|
||||
sudo /srv/thothii/source/ThothII/scripts/prepare-server-pi-state.sh \
|
||||
/srv/thothii/pi-state 10001 10001
|
||||
@@ -206,17 +206,15 @@ after restoring `pi-state`; run it before any `tht start`, Compose render/start,
|
||||
Copy the path-only server environment and installation descriptor:
|
||||
|
||||
```sh
|
||||
sudo -u thothii cp deploy/env/server.env.example /srv/thothii/operator/server.env
|
||||
sudo -u thothii cp docs/install/examples/thothii-installation.server.yaml \
|
||||
cp deploy/env/server.env.example /srv/thothii/operator/server.env
|
||||
cp docs/install/examples/thothii-installation.server.yaml \
|
||||
/srv/thothii/operator/thothii-installation.yaml
|
||||
sudo chown 10001:thothii-ops /srv/thothii/operator/server.env \
|
||||
/srv/thothii/operator/thothii-installation.yaml
|
||||
sudo chmod 0660 /srv/thothii/operator/server.env \
|
||||
chmod 0600 /srv/thothii/operator/server.env \
|
||||
/srv/thothii/operator/thothii-installation.yaml
|
||||
```
|
||||
|
||||
The named human operator can now edit both placeholder files without `sudo`; use an editor that
|
||||
preserves the group, or create replacements under `umask 0007` in the setgid operator directory.
|
||||
The invoking operator owns both placeholder files; use an editor that preserves ownership and mode,
|
||||
or create replacements under `umask 0077` in the operator directory.
|
||||
Replace every placeholder with an absolute path. Use exactly one Git transport override. For
|
||||
HTTPS, replace `deploy/compose.git-ssh.yaml` with `deploy/compose.git-https.yaml`. Keep the required
|
||||
session-server overlay. Optional host-gateway or pinned
|
||||
@@ -224,17 +222,18 @@ image overrides go after them.
|
||||
|
||||
Create each installation credential (Pi/application, Git, and session storage) as an independent
|
||||
regular file in `/srv/thothii/secrets`, owned by
|
||||
UID 10001, group `thothii-ops`, and mode `0640`. Owner access lets the UID 10001 container read a
|
||||
file mounted under `/run/secrets`; group access lets the reviewed human run `tht`. The
|
||||
UID 10001, the invoking operator's numeric primary GID, and mode `0640`. Owner access lets the UID
|
||||
10001 container read a file mounted under `/run/secrets`; group access lets the operator run `tht`. The
|
||||
operator environment records only absolute `*_FILE` or `*_SOURCE` paths for those installation
|
||||
credentials. DWH and Evidence values are entered later through Workspace management and persist
|
||||
as ciphertext under `/data/workspace-secrets`; the frontend receives no secret values. Do not
|
||||
print file contents while testing permissions.
|
||||
|
||||
```sh
|
||||
sudo find /srv/thothii/secrets -type f -exec chown 10001:thothii-ops {} +
|
||||
operator_gid="$(id -g)"
|
||||
sudo find /srv/thothii/secrets -type f -exec chown "10001:$operator_gid" {} +
|
||||
sudo find /srv/thothii/secrets -type f -exec chmod 0640 {} +
|
||||
sudo find /srv/thothii/secrets -type f \( ! -user thothii -o ! -group thothii-ops -o ! -perm 0640 \) -print
|
||||
sudo find /srv/thothii/secrets -type f \( ! -uid 10001 -o ! -gid "$operator_gid" -o ! -perm 0640 \) -print
|
||||
```
|
||||
|
||||
Configure the remote repository and exactly one read-only Git transport as described in
|
||||
@@ -250,7 +249,7 @@ Compose; copy the reviewed path-only server environment to the launcher's untrac
|
||||
|
||||
```sh
|
||||
cd /srv/thothii/source/ThothII
|
||||
sudo -u thothii cp /srv/thothii/operator/server.env deploy/env/local.env
|
||||
cp /srv/thothii/operator/server.env deploy/env/local.env
|
||||
bash scripts/build-local.sh
|
||||
```
|
||||
|
||||
@@ -286,12 +285,13 @@ Build the operator binaries with Docker. No Go installation or Go knowledge is r
|
||||
cd /srv/thothii/source/ThothII
|
||||
THT_THT_OUTPUT_DIRECTORY=/srv/thothii/operator/build-output \
|
||||
bash scripts/build-tht.sh
|
||||
sudo install -o root -g thothii-ops -m 0750 \
|
||||
operator_gid="$(id -g)"
|
||||
sudo install -o root -g "$operator_gid" -m 0750 \
|
||||
/srv/thothii/operator/build-output/tht-linux-amd64 \
|
||||
/srv/thothii/operator/tht
|
||||
```
|
||||
|
||||
The source checkout stays read-only to the human. The explicit output directory is the only build
|
||||
The source checkout remains controlled by the invoking operator. The explicit output directory is the only build
|
||||
write boundary; the build script rejects relative or non-canonical output paths. After installation,
|
||||
remove or retain `build-output` according to the site's reviewed artifact policy.
|
||||
|
||||
|
||||
@@ -58,7 +58,7 @@ Authentik, Aritmolab o repository esterni.
|
||||
| 4 | Topologia e responsabilità del load balancer | `BLOCKED` | Unassigned | Necessario per Project B/route opzionale |
|
||||
| 5 | Accesso read-only protetto ad Authentik | `BLOCKED` | Unassigned | Necessario per Project B, non per A privato |
|
||||
| 6 | Accesso catalog-only protetto a PostgreSQL | `BLOCKED` | Unassigned | DWH direct read-only ancora da provare |
|
||||
| 7 | Backup e rollback del vecchio ThothII | `BLOCKED` | Proprietario del progetto | Procedura e destinazione da approvare |
|
||||
| 7 | Confine temporaneo e cleanup del vecchio ThothII | `PASS` | Proprietario del progetto | Retain fino a Project B PASS; cleanup esatto separato |
|
||||
| 8 | Accesso Git read-only al workspace PSD | `BLOCKED` | Curator da confermare | Checkout/deploy key server mancanti |
|
||||
| 9 | Metadati Pi e LLM verificabili | `BLOCKED` | Unassigned | Policy e reachability redatte mancanti |
|
||||
| 10 | Conservazione evidenze e nuovo survey bounded | `PENDING` | Unassigned | Nuovo report e decisione proprietario |
|
||||
@@ -281,23 +281,22 @@ Authentik, Aritmolab o repository esterni.
|
||||
- Next step: definire con il DBA ruolo, secret-file protetto, TLS/rete e query di grant da ripetere;
|
||||
non creare il ruolo durante il survey.
|
||||
|
||||
## Activity 7: Prove legacy backup and rollback
|
||||
## Activity 7: Bind the disposable legacy boundary and cleanup exclusions
|
||||
|
||||
- Status: `BLOCKED`
|
||||
- Accountable owner: Unassigned
|
||||
- Objective: dimostrare che il vecchio ThothII possa essere preservato e ripristinato prima di
|
||||
qualsiasi stop.
|
||||
- Why this is required: source, container, bind e network sono inventariati, ma backup verificato,
|
||||
controller e restart recipe non sono disponibili.
|
||||
- Status: `PASS`
|
||||
- Accountable owner: Proprietario del progetto (confermato dall'utente)
|
||||
- Objective: mantenere il vecchio ThothII solo come confine temporaneo di cutover e rimuovere
|
||||
esclusivamente le sue risorse dopo il PASS reale di Aritmolab.
|
||||
- Why this is required: Aritmolab usa oggi i container legacy, ma il proprietario ha dichiarato
|
||||
sacrificabili sessioni, configurazione e dati del vecchio ThothII.
|
||||
- Ordered actions:
|
||||
1. identificare owner e meccanismo lifecycle compatibile;
|
||||
2. confermare assenza di lavoro utente da preservare;
|
||||
3. definire contenuti, destinazione e protezione del backup;
|
||||
4. definire checksum e verifica di leggibilità;
|
||||
5. scrivere comandi esatti di stop/start e ordine di chiusura/ripristino route;
|
||||
6. eseguire il backup soltanto nella successiva fase autorizzata, prima dello stop.
|
||||
- Required redacted evidence: inventario, percorso backup, checksum, owner, restart recipe e
|
||||
rollback route; nessun contenuto di secret.
|
||||
1. inventariare nuovamente container, image ID, source e data immediatamente prima dello stop;
|
||||
2. preservare invariati i due network condivisi e l'Evidence ETL esterna;
|
||||
3. chiudere la route e fermare solo i container legacy nel gate Project A autorizzato;
|
||||
4. conservare container, immagini, source e data fino al PASS automatico, umano e owner di B;
|
||||
5. rimuovere poi soltanto gli exact target sotto un'autorizzazione cleanup separata.
|
||||
- Required redacted evidence: inventario esatto, owner, restart recipe, esclusioni shared, decisioni
|
||||
Project A/B e manifest finale di cleanup; nessun contenuto di secret.
|
||||
- Discussion notes: il legacy stack è ancora attivo e invariato. Compose project `thothii` usa
|
||||
`/home/chirone/ThothII/compose.yaml`; i servizi sono `core` e `frontend`, senza named volume.
|
||||
Il solo bind applicativo RW è `/home/chirone/thothii-data` (con i bind Pi annidati); Evidence è
|
||||
@@ -307,13 +306,13 @@ Authentik, Aritmolab o repository esterni.
|
||||
`PI_AUTH_FILE=/home/chirone/thothii-data/pi-config/agent/auth.json` insieme a
|
||||
`--env-file deploy/thothii.env -p thothii -f compose.yaml`; stop individua entrambi i container
|
||||
e start è sintatticamente valido (non trova container arrestati mentre lo stack è ancora attivo).
|
||||
- Decision: il controller e le radici da archiviare sono identificati; nessun backup o stop è stato
|
||||
eseguito. Per un rollback normale i container restano esistenti e il controller usa `start`.
|
||||
- Blockers: scegliere/approvare la destinazione protetta, creare il backup con manifest/checksum,
|
||||
verificare la leggibilità dell'archivio, definire la gestione della route durante lo stop e
|
||||
ottenere il consenso separato prima di eseguire stop/start.
|
||||
- Next step: approvare manifest, destinazione e comandi; eseguire backup e checksum soltanto nel
|
||||
successivo gate di mutazione.
|
||||
- Decision: nessun backup legacy è richiesto. I container, immagini, source e data già presenti
|
||||
restano il solo rollback temporaneo fino al PASS Project B; non si crea alcun utente host. Le
|
||||
risorse esclusive potranno essere cancellate dopo il collaudo Aritmolab, mentre network shared,
|
||||
ETL Evidence, Omics, LocalLLM, DWH, `dwh-auth`, Supabase, Authentik e Superset sono esclusi.
|
||||
- Blockers: nessuno per questa decisione survey. Stop/start, route change e cleanup restano tre
|
||||
autorizzazioni di mutazione separate e non sono autorizzati da questo PASS.
|
||||
- Next step: usare il design e piano clean-replacement approvati; non eseguire ancora mutazioni.
|
||||
|
||||
## Activity 8: Provide read-only PSD workspace Git access
|
||||
|
||||
@@ -375,12 +374,13 @@ Authentik, Aritmolab o repository esterni.
|
||||
`/home/chirone/ThothII` at `6ca4275`; the only RW application bind is
|
||||
`/home/chirone/thothii-data`, plus the nested Pi binds. The source is about 1.05 GB and the data
|
||||
bind about 1.9 MB. No backup was created.
|
||||
- Recovery evidence: existing containers can be inventoried for a stopped-container restart, but
|
||||
backup destination, checksum/restore procedure and route ordering are not yet approved.
|
||||
- Identity collision: legacy data and protected subtrees are numerically owned by UID/GID 10001.
|
||||
The example in `docs/install/server.md` and the Project A Pi preparation use UID 10001 for the
|
||||
new service. Do not create that host
|
||||
identity or grant it access until a reviewed isolation/ownership strategy is approved.
|
||||
- Recovery decision: legacy state is disposable; no backup is required. Existing stopped
|
||||
containers, images, source and data remain only as the temporary Project A/B rollback boundary.
|
||||
- Identity decision: neither UID nor GID 10001 maps to a host account. No host identity will be
|
||||
created. The new image retains numeric `10001:10001`, confined to its distinct writable roots;
|
||||
the existing operator owns source/configuration. Recheck both `getent` lookups before creating
|
||||
paths and stop on any new mapping.
|
||||
- Shared exclusion: never remove the Omics/LocalLLM networks or external ETL Evidence bind.
|
||||
- Candidate paths: documented examples `/srv/thothii` and `/srv/thothii-backups` are absent and
|
||||
therefore only candidates; they have not been created. `127.0.0.1:18080` è il candidato
|
||||
frontend e risultava libero al momento del survey, ma non è riservato e va ricontrollato prima
|
||||
@@ -425,10 +425,10 @@ Authentik, Aritmolab o repository esterni.
|
||||
|
||||
Un nuovo `SURVEY_GO_PROJECT_A_PRIVATE` richiede:
|
||||
|
||||
- owner e autorità di backup/stop/start/rollback identificati per il legacy e Project A;
|
||||
- owner e autorità di stop/start/rollback identificati per il legacy e Project A;
|
||||
- accessi read-only PostgreSQL, workspace Git e Pi/LLM verificati;
|
||||
- identità DWH dimostrata read-only;
|
||||
- backup e restart recipe legacy verificabili;
|
||||
- inventario, restart recipe e cleanup exclusions legacy verificabili;
|
||||
- risorse e percorsi della nuova installazione approvati;
|
||||
- report redatto, secret-scan valido e checksum verificato;
|
||||
- approvazione esplicita del proprietario.
|
||||
@@ -452,3 +452,4 @@ registrata separatamente.
|
||||
|---|---|---|---|
|
||||
| 2026-08-20 | Initial | Creata checklist; Activity 1 aperta, Activity 2–10 pending | Sol |
|
||||
| 2026-08-21 | Sequencing amendment | Activity 1 deferred pre-B; survey ripreso read-only; Project A private resta NO-GO | Owner/Sol |
|
||||
| 2026-08-21 | Clean replacement | Activity 7 PASS; legacy disposable dopo B; nessun account host 10001 | Owner/Sol |
|
||||
|
||||
@@ -36,8 +36,13 @@ manual-test document for each project.
|
||||
- The new source clone is prepared beside the old source directory. The old and new stacks are not
|
||||
kept running simultaneously: inventory and backup happen first, the old stack is stopped, and
|
||||
only then is the new stack started.
|
||||
- Preserve the old directory, configuration, and volumes as a recovery source until both projects
|
||||
pass. Do not migrate legacy application sessions, Qdrant data, Ollama caches, or derived indexes.
|
||||
- Keep the old directory, configuration, containers, images, and data only as a temporary recovery
|
||||
boundary until the real Aritmolab journey passes Project B. They are disposable after Project B
|
||||
automated, human, and owner PASS. Do not migrate legacy application sessions, Qdrant data,
|
||||
Ollama caches, or derived indexes.
|
||||
- Do not create a host `thothii` user or group. Keep UID/GID `10001:10001` as the image's unmapped
|
||||
numeric runtime identity and confine numeric ownership to the new installation's writable bind
|
||||
trees. Stop if either number becomes mapped to a host account before installation.
|
||||
- Recover configuration only: endpoints, non-secret policy, provider/model selection, relevant
|
||||
paths, and references to protected credentials. Never copy an old setting without validating it
|
||||
against the current contract.
|
||||
@@ -149,8 +154,10 @@ proved safe.
|
||||
pre-Project-B gate.
|
||||
4. Prepare the new source clone and protected operator/runtime directories beside the old source.
|
||||
5. Extract only approved configuration facts from the legacy installation.
|
||||
6. Create and verify backups and a restart recipe for the legacy stack.
|
||||
7. Stop the legacy stack without deleting its source, configuration, images, or volumes.
|
||||
6. Record and verify the exact restart recipe for the legacy stack. No data backup is required
|
||||
because the owner declared legacy sessions and configuration disposable; the still-present
|
||||
containers, images, source, and data are the temporary rollback boundary.
|
||||
7. Stop the legacy stack without deleting its source, configuration, images, or data.
|
||||
8. Build/install the current native `tht` and application images from the frozen source.
|
||||
9. Configure local authentication, direct DWH bindings, Pi/provider access, and the Git workspace.
|
||||
10. Start through `tht`, then validate health, authentication, workspace, workflow, and Pi.
|
||||
@@ -313,8 +320,8 @@ current gate is satisfied.
|
||||
|
||||
## Rollback boundaries
|
||||
|
||||
- **Project A:** stop the new stack and restart the preserved legacy installation. No new volume is
|
||||
copied into the legacy installation.
|
||||
- **Project A:** stop the new stack and restart the still-present legacy installation. No new
|
||||
volume is copied into it and no promise is made to retain it after Project B PASS.
|
||||
- **Project B ingress:** close the public route first, then restore the prior Nginx,
|
||||
load-balancer, certificate reference, and sidebar configuration.
|
||||
- **Project B application:** return to the accepted Project A local-auth operator configuration
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
**Goal:** Install a clean PSD ThothII stack with local authentication, direct read-only DWH access, internal Qdrant/Ollama, rebuilt preprocessing, and one completed F1-F8 work session.
|
||||
|
||||
**Architecture:** Preserve the stopped legacy installation and deploy the current canonical five-service Compose stack from an adjacent clean clone. A reviewed server-local override disables public exposure and uses filesystem work sessions; an optional load-balancer route is permitted only when it is operator-only and its denial boundary is proved first.
|
||||
**Architecture:** Keep the stopped legacy installation intact only as a temporary rollback boundary and deploy the current canonical five-service Compose stack from an adjacent clean clone. Create no host service identity: the image's UID/GID 10001 remains numeric and unmapped, confined to the new writable bind roots. A reviewed server-local override disables public exposure and uses filesystem work sessions; an optional load-balancer route is permitted only when it is operator-only and its denial boundary is proved first.
|
||||
|
||||
**Tech Stack:** Git, Docker/Compose, native `tht`, local Argon2id authentication, PSD Supabase PostgreSQL direct transport, Qdrant, Ollama, Pi, Nginx/load-balancer test route where safe.
|
||||
|
||||
@@ -15,10 +15,15 @@
|
||||
- Common survey result is `SURVEY_GO_PROJECT_A_PRIVATE` and its digest is recorded.
|
||||
- Every path below is replaced by the exact survey result before execution.
|
||||
- No production Nginx/load-balancer/sidebar/Authentik change is in scope.
|
||||
- The old stack remains running only until backup verification finishes; old and new stacks never
|
||||
run together.
|
||||
- The old stack remains running until its exact inventory and restart recipe are verified; old and
|
||||
new stacks never run together.
|
||||
- The server's workspace deploy credential remains read-only. A curator with write access publishes
|
||||
the workspace change.
|
||||
- Owner amendment 2026-08-21 declares legacy sessions/configuration disposable. Retain old resources
|
||||
only until Project B proves the production Aritmolab journey, then delete them under a separate
|
||||
exact cleanup authorization. No legacy backup is required.
|
||||
- Do not create a host user or group for 10001. Immediately before preparing `/srv/thothii`, both
|
||||
`getent passwd 10001` and `getent group 10001` must return no match.
|
||||
- Owner amendment 2026-08-21 defers live Mac `rest_api` acceptance and `legacy-shared` revocation to
|
||||
the mandatory pre-Project-B gate. It does not authorize stop/start; those require a later explicit
|
||||
owner gate even after private preparation is complete.
|
||||
@@ -56,7 +61,8 @@ Expected: one SHA recorded as the pre-change workspace revision.
|
||||
**Step 3: Check old-stack recoverability**
|
||||
|
||||
Expected: exact old start/stop procedure, source SHA, Compose identity, volumes/binds, proxy closure
|
||||
procedure, and backup destination are present in the survey. Stop if any is missing.
|
||||
procedure, restart recipe, and shared-resource exclusions are present in the survey. Stop if any is
|
||||
missing.
|
||||
|
||||
### Task 2: Publish the multi-transport workspace revision
|
||||
|
||||
@@ -146,11 +152,12 @@ active user work. Do not use the new `tht` against an incompatible old descripto
|
||||
Change only the surveyed ThothII-specific route using its established mechanism. Validate Nginx and
|
||||
load-balancer configuration before applying. Confirm external requests no longer reach the app.
|
||||
|
||||
**Step 3: Create the legacy backup**
|
||||
**Step 3: Record the disposable legacy boundary**
|
||||
|
||||
Use the surveyed, version-compatible backup procedure. Include source/config metadata and all old
|
||||
runtime volumes/binds needed to restart; store credentials separately under existing protected
|
||||
custody. Create SHA-256 checksums and verify them.
|
||||
Record exact container and image IDs plus filesystem device/inode/ownership/size for
|
||||
`/home/chirone/ThothII` and `/home/chirone/thothii-data`. Do not archive their contents: the owner
|
||||
declared them disposable. Prove that the external Evidence bind and both shared Docker networks
|
||||
are excluded from any later cleanup manifest.
|
||||
|
||||
**Step 4: Stop the old stack**
|
||||
|
||||
@@ -168,10 +175,12 @@ be stated unambiguously, stop before creating the new stack.
|
||||
- Create: survey-selected new source root
|
||||
- Create: survey-selected operator, secret, data, Pi-state, registry, and backup roots
|
||||
|
||||
**Step 1: Create dedicated identities and paths**
|
||||
**Step 1: Create dedicated paths without creating identities**
|
||||
|
||||
Follow `docs/install/server.md` ownership rules using the surveyed available UID/GID. Do not reuse a
|
||||
UID already owned by another service and do not change image UID 10001 without a reviewed mapping.
|
||||
Follow `docs/install/server.md` numeric-ownership rules. Do not call `useradd`, `groupadd`, or
|
||||
`usermod`. The existing operator owns source/operator files; only the new data, Pi-state, and
|
||||
workspace-registry roots use unmapped numeric `10001:10001`. Stop if UID or GID 10001 resolves to a
|
||||
host account, and do not change the image identity without a reviewed design amendment.
|
||||
|
||||
**Step 2: Clone the frozen application source**
|
||||
|
||||
|
||||
@@ -0,0 +1,259 @@
|
||||
# PSD Clean ThothII Replacement Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the disposable PSD ThothII installation without creating host identities and remove only its exclusive resources after the new Aritmolab journey passes.
|
||||
|
||||
**Architecture:** Project A builds a separate `/srv/thothii` installation while the old resources remain an inert rollback boundary. The container keeps numeric UID/GID 10001 without a host account. Project B moves the Aritmolab integration to the accepted stack; exact legacy deletion is a final post-acceptance operation that excludes shared Chirone resources.
|
||||
|
||||
**Tech Stack:** Linux ownership and identity checks, Docker Compose, native `tht`, Nginx/Aritmolab integration, protected evidence journals.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Never call `useradd`, `groupadd`, `usermod`, or edit `/etc/passwd`, `/etc/group`, `/etc/shadow`, or `/etc/gshadow`.
|
||||
- Treat UID/GID `10001:10001` as an unmapped numeric container identity only.
|
||||
- Never reuse `/home/chirone/thothii-data` for the replacement.
|
||||
- Never remove `omics_portal_omics_network`, `localllm_default`, `/home/chirone/chirone/etl/docs/evidence`, or any Omics Portal, LocalLLM, DWH, `dwh-auth`, Supabase, Authentik, ETL, or Superset resource.
|
||||
- Never run `docker system prune`, `docker network prune`, `docker volume prune`, or `docker compose down --volumes`.
|
||||
- Do not stop the old stack or start the new stack without the separate owner mutation gate already required by Project A.
|
||||
- Do not delete any legacy target until Project B automated PASS, human PASS, and explicit owner PASS are all recorded.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Bind the clean-replacement decision to the surveyed host
|
||||
|
||||
**Files:**
|
||||
- Read: `/etc/passwd`
|
||||
- Read: `/etc/group`
|
||||
- Read: `/home/chirone/ThothII/compose.yaml`
|
||||
- Read: `/home/chirone/omics_portal/nginx/nginx.conf`
|
||||
- Record: protected Project A survey journal
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: approved design `docs/superpowers/specs/2026-08-21-psd-clean-replacement-design.md`.
|
||||
- Produces: a redacted exact-target inventory and a `numeric_identity_unmapped=PASS` decision.
|
||||
|
||||
- [ ] **Step 1: Prove that 10001 is not a host identity**
|
||||
|
||||
```bash
|
||||
if getent passwd 10001 >/dev/null || getent group 10001 >/dev/null; then
|
||||
printf '%s\n' 'numeric_identity_unmapped=FAIL'
|
||||
exit 1
|
||||
fi
|
||||
printf '%s\n' 'numeric_identity_unmapped=PASS'
|
||||
```
|
||||
|
||||
Expected: one PASS line. Do not continue if either lookup succeeds.
|
||||
|
||||
- [ ] **Step 2: Revalidate the exclusive legacy containers and images**
|
||||
|
||||
```bash
|
||||
docker inspect --format '{{.Name}} project={{index .Config.Labels "com.docker.compose.project"}} image={{.Image}}' \
|
||||
thothii-core-1 thothii-frontend-1
|
||||
docker image inspect --format '{{.Id}} {{join .RepoTags ","}}' \
|
||||
thothii-core:local thothii-frontend:local
|
||||
```
|
||||
|
||||
Expected: exactly the two `thothii` project containers and two immutable image IDs. Record IDs,
|
||||
not environment variables or container configuration values.
|
||||
|
||||
- [ ] **Step 3: Revalidate shared exclusions**
|
||||
|
||||
```bash
|
||||
docker network inspect --format '{{.Name}} project={{index .Labels "com.docker.compose.project"}}' \
|
||||
omics_portal_omics_network localllm_default
|
||||
stat -c '%u:%g %a %n' /home/chirone/chirone/etl/docs/evidence
|
||||
```
|
||||
|
||||
Expected: both networks exist and the external Evidence path remains outside the legacy data tree.
|
||||
|
||||
- [ ] **Step 4: Record the survey checkpoint**
|
||||
|
||||
Record only names, IDs, modes, ownership numbers, PASS/FAIL decisions, and timestamps in the
|
||||
protected journal. Never record container environment, auth files, API keys, or raw Nginx output.
|
||||
|
||||
### Task 2: Prepare the replacement roots without host accounts
|
||||
|
||||
**Files:**
|
||||
- Create: `/srv/thothii/source`
|
||||
- Create: `/srv/thothii/operator`
|
||||
- Create: `/srv/thothii/data`
|
||||
- Create: `/srv/thothii/secrets`
|
||||
- Create: `/srv/thothii/pi-state`
|
||||
- Create: `/srv/thothii/workspace-registry`
|
||||
- Create: `/srv/thothii-backups`
|
||||
- Test: exact ownership/mode checks below
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `numeric_identity_unmapped=PASS` from Task 1.
|
||||
- Produces: isolated bind roots compatible with container UID/GID 10001 and operator UID/GID 1013:1006.
|
||||
|
||||
- [ ] **Step 1: Create only the reviewed roots**
|
||||
|
||||
```bash
|
||||
sudo install -d -o 1013 -g 10001 -m 0750 /srv/thothii
|
||||
sudo install -d -o 1013 -g 1006 -m 0750 /srv/thothii/source
|
||||
sudo install -d -o 1013 -g 1006 -m 0750 /srv/thothii/operator
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/data
|
||||
sudo install -d -o 10001 -g 1006 -m 0750 /srv/thothii/secrets
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/pi-state
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/workspace-registry
|
||||
sudo install -d -o root -g root -m 0700 /srv/thothii-backups
|
||||
```
|
||||
|
||||
Expected: commands create directories only. They do not create identities.
|
||||
|
||||
- [ ] **Step 2: Verify identity databases are unchanged and modes are exact**
|
||||
|
||||
```bash
|
||||
if getent passwd 10001 >/dev/null || getent group 10001 >/dev/null; then exit 1; fi
|
||||
stat -c '%u:%g %a %n' \
|
||||
/srv/thothii /srv/thothii/source /srv/thothii/operator /srv/thothii/secrets \
|
||||
/srv/thothii/data /srv/thothii/pi-state /srv/thothii/workspace-registry \
|
||||
/srv/thothii-backups
|
||||
```
|
||||
|
||||
Expected ownership/modes, in order: `1013:10001 750`, twice `1013:1006 750`,
|
||||
`10001:1006 750`, three times `10001:10001 750`, and `0:0 700`.
|
||||
|
||||
- [ ] **Step 3: Commit documentation/code changes before runtime use**
|
||||
|
||||
Replace the account-creation section of `docs/install/server.md` with the numeric ownership model:
|
||||
the invoking operator UID/GID owns source and operator paths, numeric 10001 owns only secrets and
|
||||
writable runtime paths, and the manual verifies both `getent` lookups remain empty. Remove every
|
||||
`useradd`, `groupadd`, `usermod`, `sudo -u thothii`, and `thothii-ops` instruction. Run the existing
|
||||
documentation gates, obtain review, commit, and push before Project A uses these roots. Runtime
|
||||
state under `/srv` is never committed.
|
||||
|
||||
### Task 3: Execute Project A with a stopped-but-intact legacy boundary
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/plans/2026-08-20-psd-server-project-a-standalone.md`
|
||||
- Record: protected Project A report and journal
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Task 2 roots and a separately approved Project A mutation gate.
|
||||
- Produces: Project A automated PASS and explicit human PASS while the old resources remain intact.
|
||||
|
||||
- [ ] **Step 1: Close the old ThothII route using the separately approved exact route operation**
|
||||
|
||||
Validate configuration before applying it. Do not change `/dwh/` or unrelated Omics routes.
|
||||
|
||||
- [ ] **Step 2: Stop only the two old ThothII containers**
|
||||
|
||||
Use the surveyed legacy Compose controller. Do not remove containers, images, networks, source, or
|
||||
data. Confirm Omics Portal, LocalLLM, DWH, `dwh-auth`, Supabase, Authentik, ETL, and Superset remain
|
||||
running.
|
||||
|
||||
- [ ] **Step 3: Execute and close Project A**
|
||||
|
||||
Follow the Project A plan and manual through its automated and human PASS decisions. On failure,
|
||||
stop the new stack and restart the still-present old containers; do not delete either installation.
|
||||
|
||||
### Task 4: Cut Aritmolab over in Project B
|
||||
|
||||
**Files:**
|
||||
- Read: `docs/plans/2026-08-20-psd-server-project-b-authentik.md`
|
||||
- Read: `docs/testing/psd-server-project-b-manual.md`
|
||||
- Record: protected Project B report and journal
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: accepted Project A candidate and the independent pre-Project-B gate.
|
||||
- Produces: a production route that no longer depends on the old container names.
|
||||
|
||||
- [ ] **Step 1: Complete the mandatory pre-Project-B gates**
|
||||
|
||||
Mac REST acceptance, the 48-hour/two-ETL observation, and `legacy-shared` revocation must be PASS.
|
||||
These DWH-key gates are independent from deleting the old ThothII application.
|
||||
|
||||
- [ ] **Step 2: Execute Project B without deleting legacy resources**
|
||||
|
||||
Follow the Project B plan. Keep the old containers stopped and intact until all automated checks
|
||||
pass.
|
||||
|
||||
- [ ] **Step 3: Run the real Aritmolab acceptance**
|
||||
|
||||
Prove the complete path from Aritmolab home and sidebar through authentication to frontend assets,
|
||||
API, SSE, and one completed workflow. Confirm the active route no longer resolves the legacy
|
||||
`thothii-core` or `thothii-frontend` services.
|
||||
|
||||
- [ ] **Step 4: Record human and owner PASS**
|
||||
|
||||
No cleanup command is authorized by technical success alone. Record the Project B human PASS and a
|
||||
separate owner decision explicitly authorizing the exact cleanup inventory from Task 5.
|
||||
|
||||
### Task 5: Delete only exclusive legacy ThothII resources
|
||||
|
||||
**Files:**
|
||||
- Delete: `/home/chirone/ThothII`
|
||||
- Delete: `/home/chirone/thothii-data`
|
||||
- Record: protected final cleanup report
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Project B automated PASS, human PASS, owner PASS, and immutable IDs from Task 1.
|
||||
- Produces: no legacy ThothII application resources and no change to shared Chirone resources.
|
||||
|
||||
- [ ] **Step 1: Revalidate the cleanup manifest immediately before deletion**
|
||||
|
||||
Re-run Task 1. Stop if an image has another consumer, a legacy container is running, the production
|
||||
route still mentions a legacy service, or either filesystem path resolves outside its expected
|
||||
exact target. Record device/inode, ownership, and size; never record file contents.
|
||||
|
||||
- [ ] **Step 2: Remove only the stopped legacy containers through their Compose project**
|
||||
|
||||
Remove `thothii-core-1` and `thothii-frontend-1` without `--volumes`. Do not remove either shared
|
||||
network.
|
||||
|
||||
- [ ] **Step 3: Remove only the two unreferenced legacy image IDs**
|
||||
|
||||
Resolve tags to the immutable IDs recorded in Task 1 and prove no container references them before
|
||||
removal. Do not prune images globally.
|
||||
|
||||
- [ ] **Step 4: Remove the two exact legacy filesystem trees**
|
||||
|
||||
Delete only `/home/chirone/ThothII` and `/home/chirone/thothii-data` after their exact identities
|
||||
match the approved manifest. This operation is irreversible by design; no legacy session or state
|
||||
restore is promised after this point.
|
||||
|
||||
- [ ] **Step 5: Verify shared resources and the production journey**
|
||||
|
||||
Confirm both shared Docker networks, the external Evidence path, and every out-of-scope service
|
||||
still exist. Repeat the Aritmolab sidebar, authentication, frontend, API/SSE, and completed-workflow
|
||||
checks. Record `CLEAN_REPLACEMENT_PASS` only if all checks pass.
|
||||
|
||||
### Task 6: Commit the durable evidence references
|
||||
|
||||
**Files:**
|
||||
- Modify: `PROJECT_STATE.md`
|
||||
- Modify: approved redacted report index only
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Project A, Project B, and cleanup evidence digests.
|
||||
- Produces: a secret-free durable state summary.
|
||||
|
||||
- [ ] **Step 1: Update the state without secret-bearing evidence**
|
||||
|
||||
Record candidate SHAs, image IDs, workspace SHA, report digests, gate decisions, deleted exact
|
||||
targets, and preserved shared resources. Protected journals and credentials stay outside Git.
|
||||
|
||||
- [ ] **Step 2: Run documentation and secret gates**
|
||||
|
||||
```bash
|
||||
bash scripts/test-verify-workspace-install-docs.sh
|
||||
bash scripts/test-verify-dwh-auth-docs.sh
|
||||
bash scripts/auth-docs-smoke.sh
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: every script exits zero and the diff check has no output.
|
||||
|
||||
- [ ] **Step 3: Commit and push the secret-free state update**
|
||||
|
||||
```bash
|
||||
git add PROJECT_STATE.md
|
||||
git commit -m "docs: record PSD clean replacement completion"
|
||||
git push origin feat/dwh-rest-installation-auth
|
||||
```
|
||||
|
||||
Expected: the push contains no `/srv` state, credential, protected journal, raw Nginx output, or
|
||||
legacy data.
|
||||
@@ -0,0 +1,86 @@
|
||||
# PSD Clean ThothII Replacement Design
|
||||
|
||||
**Date:** 2026-08-21
|
||||
**Status:** Approved by the owner
|
||||
|
||||
## Decision
|
||||
|
||||
The existing PSD ThothII installation is disposable. Its sessions, settings, Pi state, derived
|
||||
artifacts, indexes, images, source checkout, and application data are not migration inputs for the
|
||||
new installation. They remain present only until the replacement has passed the real Aritmolab
|
||||
journey; this temporary retention is a cutover safeguard, not a legacy-support requirement.
|
||||
|
||||
The replacement does not create a host `thothii` user or group. The image keeps its internal
|
||||
UID/GID `10001:10001`. Host bind trees that the container must write may use that numeric ownership
|
||||
without corresponding `/etc/passwd` or `/etc/group` entries. Source and operator-controlled files
|
||||
remain owned by the existing operator `admlocforn1` (UID 1013) and the existing `chirone` group
|
||||
(GID 1006).
|
||||
|
||||
## Boundaries
|
||||
|
||||
The following old ThothII resources become deletion candidates only after Project B human
|
||||
acceptance proves the production Aritmolab route:
|
||||
|
||||
- containers `thothii-core-1` and `thothii-frontend-1`;
|
||||
- images `thothii-core:local` and `thothii-frontend:local`, after proving no other container uses
|
||||
their immutable IDs;
|
||||
- checkout `/home/chirone/ThothII`;
|
||||
- application bind tree `/home/chirone/thothii-data`.
|
||||
|
||||
The following resources are shared and are never deletion candidates in the ThothII cleanup:
|
||||
|
||||
- Docker networks `omics_portal_omics_network` and `localllm_default`;
|
||||
- `/home/chirone/chirone/etl/docs/evidence` and the ETL project;
|
||||
- Omics Portal/Aritmolab source, Nginx, web, worker, sidebar, and capability configuration;
|
||||
- LocalLLM API and vLLM services;
|
||||
- `dwh-auth`, its credential registry, and the `/dwh/` Nginx route;
|
||||
- Supabase/DWH, Authentik, Superset, and their data or configuration.
|
||||
|
||||
No global Docker prune, network removal, broad recursive deletion, or Compose volume deletion is
|
||||
permitted. Cleanup resolves and revalidates every exact target immediately before removal.
|
||||
|
||||
## Sequence
|
||||
|
||||
1. Project A preparation creates a distinct installation under `/srv/thothii`; it does not reuse
|
||||
`/home/chirone/thothii-data`.
|
||||
2. Immediately before creating bind roots, verify that UID and GID 10001 still have no host account
|
||||
mapping. A newly observed mapping is a stop condition requiring owner review.
|
||||
3. Project A may stop the old containers only under its separate mutation authorization. The old
|
||||
source, data, containers, and images remain intact while the new private stack is tested.
|
||||
4. Project B changes the production integration and proves the complete path:
|
||||
`Aritmolab -> sidebar -> Nginx/load balancer -> ThothII`, including frontend assets, API, SSE,
|
||||
authentication, and a completed workflow.
|
||||
5. Only after the Project B automated report, human report, and owner decision are PASS may the
|
||||
exact old ThothII resources be deleted.
|
||||
|
||||
If Project A fails, stop the new private stack and restart the still-present old containers. If
|
||||
Project B fails, close the new ingress and restore the prior route while the old resources still
|
||||
exist. After Project B PASS and legacy deletion there is deliberately no promise to restore old
|
||||
ThothII sessions or state.
|
||||
|
||||
## Host ownership model
|
||||
|
||||
No command may call `useradd`, `groupadd`, `usermod`, or modify the host identity databases.
|
||||
|
||||
- `/srv/thothii/source` and `/srv/thothii/operator`: `1013:1006`.
|
||||
- `/srv/thothii/data`, `/srv/thothii/pi-state`, and `/srv/thothii/workspace-registry`:
|
||||
numeric `10001:10001`.
|
||||
- protected files mounted read-only by the core: operator-owned with a narrowly selected numeric
|
||||
group/owner mode that permits UID/GID 10001 to read only the required file.
|
||||
- `/srv/thothii-backups`: operator/root protected and outside the runtime write boundary.
|
||||
|
||||
Numeric ownership does not create or preserve a host user. It is confined to the new installation
|
||||
tree and exists solely to match the non-root identity already embedded in the container image.
|
||||
|
||||
## Acceptance
|
||||
|
||||
The design is satisfied only when evidence proves all of the following:
|
||||
|
||||
- no host account or group was created for 10001;
|
||||
- the new core runs non-root and can write only its intended runtime trees;
|
||||
- the shared networks and external Evidence tree remain unchanged;
|
||||
- Project A and Project B pass their independent automated and human gates;
|
||||
- Aritmolab no longer resolves production traffic to `thothii-core` or `thothii-frontend` legacy
|
||||
containers before those containers are removed;
|
||||
- the cleanup inventory contains only exact legacy ThothII targets and excludes every shared
|
||||
resource listed above.
|
||||
Reference in New Issue
Block a user