docs: focus public documentation on product usage
This commit is contained in:
@@ -75,7 +75,6 @@ This section applies only when a Linux `profile: server` descriptor declares a r
|
||||
The canonical authentication root stays root-owned and is the only authority. The container reads
|
||||
only the separate read-only runtime projection selected by `CURRENT`; it never falls back to the
|
||||
canonical files or to a previous generation. Run projected mutations and repairs through the
|
||||
root-operated `tht` commands documented in the [server guide](server.md), and never edit runtime
|
||||
files directly.
|
||||
root-operated `tht` commands, and never edit runtime files directly.
|
||||
|
||||
Mac, Windows, and local direct-file authentication remain unchanged when the projection is absent.
|
||||
|
||||
@@ -71,7 +71,7 @@ The surfaces have distinct semantics and this order is recommended:
|
||||
issuer/JWKS, catalog credentials, and all configured mapped groups.
|
||||
3. `tht auth check --interactive` repeats live diagnosis and additionally validates a device-flow
|
||||
identity and its direct `groups` claim when Device Authorization is available.
|
||||
4. Workspace Test performs aggregate live workspace and authentication validation.
|
||||
4. Installation diagnostics perform aggregate live workspace and authentication validation.
|
||||
|
||||
The live CLI forms are:
|
||||
|
||||
@@ -87,8 +87,8 @@ real ID token including `groups`. It is an operator check, not a replacement for
|
||||
`tht doctor` emits this exact ordered report: `descriptor`, `files`, `docker`, `compose`,
|
||||
`configuration`, `authentication`, `services`, `core-http`, `frontend-http`,
|
||||
`workspace-registry`, `workflow`, `pi`. Its authentication entry is live and non-interactive.
|
||||
Any authentication failure makes Workspace Validate or Workspace Test non-activatable according
|
||||
to that surface's static or live scope.
|
||||
Any authentication failure prevents activation according to the static or live scope of the
|
||||
relevant diagnostic surface.
|
||||
|
||||
The complete closed diagnostic-code union and exact role-to-permission expansion are in the
|
||||
[authentication architecture](../architecture/authentication.md).
|
||||
|
||||
+36
-31
@@ -1,37 +1,42 @@
|
||||
# Authentik provider setup
|
||||
# Configurazione del provider Authentik
|
||||
|
||||
Authentik is the first certified provider for PSD acceptance. The ThothII browser protocol remains
|
||||
generic OIDC; these steps configure the provider-specific group catalog only.
|
||||
Il protocollo browser di ThothII è OIDC generico. Authentik fornisce il catalogo gruppi e il
|
||||
provider di identità senza introdurre un percorso di login proprietario.
|
||||
|
||||
1. Create an OAuth2/OIDC application and provider in Authentik. Register exactly
|
||||
`<publicUrl>/api/auth/oidc/callback` as the callback and enable `openid`, `profile`, and `email`.
|
||||
2. Configure the provider so the ID token contains a direct `groups` array of strings. Verify the
|
||||
claim with a disposable test identity before running acceptance.
|
||||
3. Create a dedicated API service account for the group catalog. Grant group-view-only privilege;
|
||||
do not grant write, user-management, or directory-administration privilege. Put its bearer value
|
||||
in the protected bundle under `THT_AUTHENTIK_API_TOKEN`.
|
||||
4. Create or confirm the exact groups `TOT Users` and `TOT Admin`. Map them explicitly in
|
||||
`auth.yaml` to `user` and `admin`, respectively. Keep other upstream groups out of the mapping.
|
||||
5. Run Workspace Validate for static authentication validation. Then run live non-interactive
|
||||
diagnosis, followed by the optional device-flow identity check:
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Browser
|
||||
participant ThothII
|
||||
participant Authentik
|
||||
Browser->>ThothII: Sign in
|
||||
ThothII->>Authentik: Authorization Code with PKCE
|
||||
Authentik-->>Browser: Login and consent
|
||||
Browser->>ThothII: Callback with code
|
||||
ThothII->>Authentik: Token exchange
|
||||
Authentik-->>ThothII: Identity and groups
|
||||
ThothII-->>Browser: Opaque session
|
||||
```
|
||||
|
||||
```sh
|
||||
tht auth check
|
||||
tht auth check --interactive
|
||||
tht doctor --json
|
||||
```
|
||||
## Provider OIDC
|
||||
|
||||
6. Run Workspace Test for aggregate live workspace and authentication validation. It must prove
|
||||
discovery/JWKS, catalog access, and every configured group. The diagnostic result must contain
|
||||
no secret values. `tht doctor --json` reports `authentication` after `configuration` and before
|
||||
`services` in its exact ordered checklist.
|
||||
1. Creare applicazione e provider OAuth2/OIDC.
|
||||
2. Registrare esattamente `PUBLIC_URL/api/auth/oidc/callback`.
|
||||
3. Abilitare gli scope `openid`, `profile` ed `email`.
|
||||
4. Configurare un claim diretto `groups` come array di stringhe.
|
||||
|
||||
Only configured exact group names are queried. Additional Authentik or directory groups are ignored
|
||||
silently, without a warning. A mapped group absent from Authentik fails closed with
|
||||
`oidc_mapped_group_missing`; an ambiguous exact-name result uses
|
||||
`oidc_mapped_group_ambiguous`. A group visible only in an upstream directory but not represented
|
||||
in Authentik is missing from ThothII’s catalog and must not be treated as present.
|
||||
## Catalogo gruppi
|
||||
|
||||
Rotate the two credentials independently through the protected secret-file procedure, then repeat
|
||||
`tht auth check` and workspace Test. Never put either value in this guide, YAML, shell history,
|
||||
diagnostic output, or acceptance evidence.
|
||||
Creare un account di servizio dedicato con sola lettura dei gruppi. Conservare il token nel
|
||||
bundle protetto come `THT_AUTHENTIK_API_TOKEN`.
|
||||
|
||||
Mappare in `auth.yaml` i nomi esatti dei gruppi aziendali ai ruoli ThothII `user` e `admin`.
|
||||
Gruppi non mappati vengono ignorati; un gruppo configurato ma assente genera un errore chiuso.
|
||||
|
||||
## Diagnostica
|
||||
|
||||
`tht auth check` controlla discovery, issuer, JWKS, accesso al catalogo e presenza dei gruppi
|
||||
configurati. L'opzione `--interactive` aggiunge la verifica dell'identità tramite device flow,
|
||||
quando il provider la supporta.
|
||||
|
||||
Ruotare separatamente secret OIDC e token del catalogo gruppi. Nessuno dei due deve comparire in
|
||||
YAML, cronologia shell, log o output diagnostico.
|
||||
|
||||
@@ -1,70 +1,41 @@
|
||||
# Enrollment client per DWH REST
|
||||
|
||||
La credenziale `dwh-auth` appartiene a una installazione ThothII, non a una persona. Serve solo se
|
||||
il trasporto è `rest_api`; `postgres_direct` e `ssh_tunnel` non la usano.
|
||||
La credenziale `dwh-auth` appartiene a una installazione ThothII e serve soltanto quando il
|
||||
workspace usa il trasporto `rest_api`.
|
||||
|
||||
| Trasporto | Chiave `dwh-auth` | Materiale locale |
|
||||
| --- | --- | --- |
|
||||
| `rest_api` | Sì, una per installazione. | URL HTTPS, `API_KEY_FILE`, eventuale `TLS_CA_FILE`. |
|
||||
| `postgres_direct` | No. | Credenziali PostgreSQL e TLS PostgreSQL. |
|
||||
| `ssh_tunnel` | No. | Credenziali PostgreSQL e materiali SSH; è diagnostico-only nel runtime corrente. |
|
||||
| Trasporto | Materiale richiesto |
|
||||
| --- | --- |
|
||||
| `rest_api` | URL HTTPS, `API_KEY_FILE`, eventuale `TLS_CA_FILE` |
|
||||
| `postgres_direct` | Credenziali PostgreSQL e configurazione TLS PostgreSQL |
|
||||
| `ssh_tunnel` | Credenziali PostgreSQL e materiale SSH |
|
||||
|
||||
Il ThothII server PSD resta `postgres_direct` read-only. Il Mac PSD e le installazioni remote
|
||||
usano `rest_api`; non introdurre un tunnel SSH per aggirare REST.
|
||||
## Consegna e conservazione
|
||||
|
||||
## Prerequisiti
|
||||
Ricevere chiave e CA attraverso canali protetti separati. Conservare la chiave nel vault
|
||||
dell'installazione o in un file regolare accessibile soltanto all'account autorizzato. Non
|
||||
inserirla in Git, file YAML, argomenti, log o schermate condivise.
|
||||
|
||||
Ricevere chiave e CA, se necessaria, attraverso canali protetti separati. Confermare fuori banda il
|
||||
fingerprint TLS prima dell'uso: [guida TLS](dwh-auth-tls.md). Conservare la chiave nel vault o in
|
||||
un file protetto, mai Git, `.env` con il valore, argv, ambiente, log o evidenze. Annotare solo ID
|
||||
pubblico.
|
||||
## Configurazione ACME Limited
|
||||
|
||||
## Percorso GUI: vault dell'installazione
|
||||
|
||||
1. In **Workspace management**, eseguire **Update workspace repository** se necessario e
|
||||
selezionare il workspace.
|
||||
2. Il trasporto `rest_api` è una precondizione amministrativa del binding locale, non una scelta della GUI. Controllare URL/trust locali e usare **Validate workspace source**.
|
||||
3. Inserire la chiave nel campo write-only **Data warehouse API key**, poi **Save entered secrets**.
|
||||
La GUI la conserva nel vault cifrato `workspace-secrets`, non la rileggere né la restituisce.
|
||||
4. Eseguire **Test workspace connections**. Il controllo innocuo è `/rpc/ping`: atteso 2xx e
|
||||
database/schema dichiarati.
|
||||
5. Comunicare al server solo ID pubblico, timestamp e risultato. **Forget stored value** rimuove il valore e va
|
||||
usato soltanto dopo conferma di sostituzione o revoca.
|
||||
|
||||
## Percorso headless: binding reale
|
||||
|
||||
`API_KEY_FILE` significa che il valore è nel file, non nella variabile. Questo è l'esempio Mac/local/remoto nel file PSD non tracciato `workspace-bindings.env`; non è il binding del server PSD Project A, che resta `postgres_direct`. I binding REST sono:
|
||||
Esempio di binding headless per il workspace `acme-ebikes`:
|
||||
|
||||
```dotenv
|
||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api
|
||||
THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://supabase-aritmolab.policlinicosandonato.it/dwh/
|
||||
THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-clinical-dwh-api-key
|
||||
THT_WS_PSD_CLINICAL_DWH_TLS_CA_FILE=/run/secrets/psd-clinical-dwh-ca.pem
|
||||
THT_WS_ACME_EBIKES_DWH_TRANSPORT=rest_api
|
||||
THT_WS_ACME_EBIKES_DWH_BASE_URL=https://dwh.acme.example/dwh/
|
||||
THT_WS_ACME_EBIKES_DWH_API_KEY_FILE=/run/secrets/acme-ebikes-dwh-api-key
|
||||
THT_WS_ACME_EBIKES_DWH_TLS_CA_FILE=/run/secrets/acme-ebikes-dwh-ca.pem
|
||||
```
|
||||
|
||||
Nel file `operator.env` non tracciato, ogni suffisso `_SOURCE` indica solo il percorso assoluto del
|
||||
file protetto di origine. Il comando genera un override non tracciato che monta quei file nel
|
||||
`core`; non installa né avvia `dwh-auth` con Compose:
|
||||
Il suffisso del workspace deriva dall'ID immutabile trasformando i trattini in underscore e
|
||||
usando lettere maiuscole. `API_KEY_FILE` contiene il percorso del file montato, non il valore
|
||||
della chiave.
|
||||
|
||||
```bash
|
||||
bash scripts/generate-connector-secrets-override.sh \
|
||||
--bindings-env /absolute/protected/workspace-bindings.env \
|
||||
--operator-env /absolute/protected/operator.env \
|
||||
--output /absolute/protected/connector-secrets.override.yaml \
|
||||
--service core --role dwh
|
||||
```
|
||||
## Rotazione e revoca
|
||||
|
||||
La chiave sorgente è un file regolare `0600` per il solo account autorizzato. Per altri workspace,
|
||||
sostituire `PSD_CLINICAL` con ID immutabile maiuscolo (trattini in underscore). Vedere anche il
|
||||
[protocollo diagnostico](../workspace-diagnostic-protocol.md).
|
||||
Durante la rotazione, ricevere la nuova generazione, aggiornare il vault o il file montato e
|
||||
confermare la connettività sulla route innocua `/rpc/ping`. Solo dopo questa conferma il
|
||||
responsabile del server revoca la generazione precedente.
|
||||
|
||||
## Ping, rotazione e revoca
|
||||
|
||||
Usare solo **Test workspace connections** su `/rpc/ping`: successo è 2xx con TLS verificato; il server
|
||||
conferma l'ID con `key status`. Durante rotazione, ricevere nuova generazione, aggiornare vault o
|
||||
file `API_KEY_FILE`, ripetere ping, attendere osservazione e far revocare la precedente. Dopo la
|
||||
revoca: nuova positiva, precedente `401`.
|
||||
|
||||
`401` non distingue chiave assente, scaduta o revocata. `503` è un guasto fail-closed di servizio,
|
||||
socket o registro: non usare connessione diretta e non ridurre TLS. Non riattivare una chiave
|
||||
revocata. Il percorso PSD è nel [runbook](../operations/psd-dwh-auth-rollout.md).
|
||||
Un `401` indica una chiave assente, sconosciuta, scaduta o revocata. Un `503` indica che il
|
||||
servizio di autorizzazione o il registro non sono disponibili. In entrambi i casi non aggirare
|
||||
REST e non ridurre la verifica TLS.
|
||||
|
||||
+54
-345
@@ -1,356 +1,65 @@
|
||||
# `dwh-auth`: guida server
|
||||
|
||||
`dwh-auth` autentica la route REST `/dwh/` con una chiave per installazione. È un componente Linux
|
||||
opzionale e server-side: usa `systemd`, non `tht` né Docker Compose, non legge risultati clinici e
|
||||
non si collega a PostgreSQL. La chiave serve solo a `rest_api`; `postgres_direct` e `ssh_tunnel`
|
||||
non la usano.
|
||||
`dwh-auth` protegge la route REST `/dwh/` con una chiave distinta per ogni installazione
|
||||
ThothII. Il componente gira come servizio Linux separato, non legge i dati del DWH e non si
|
||||
collega direttamente a PostgreSQL.
|
||||
|
||||
## Prerequisiti e confini
|
||||
|
||||
- Usare un checkout revisionato, Docker per la build e un operatore autorizzato sul server DWH.
|
||||
- Una chiave identifica un'installazione, non una persona. L'`installation-id` è unico, non
|
||||
personale e senza dati clinici.
|
||||
- Chiavi, digest, file di consegna e backup restano in file protetti: mai Git, argv, variabili
|
||||
d'ambiente, log, JSON pubblico o evidenze.
|
||||
- Preparare backup e rollback prima di Nginx. Installare il servizio non autorizza una modifica
|
||||
della route pubblica.
|
||||
|
||||
## Percorsi, owner e mode
|
||||
|
||||
| Oggetto | Percorso | Owner e mode |
|
||||
| --- | --- | --- |
|
||||
| Binario | `/usr/local/sbin/dwh-auth` | `root:root`, `0755` |
|
||||
| Unit | `/etc/systemd/system/dwh-auth.service` | `root:root`, `0644` |
|
||||
| Tmpfiles | `/usr/lib/tmpfiles.d/dwh-auth.conf` | `root:root`, `0644` |
|
||||
| Registro, `active`, `revoked` | `/var/lib/dwh-auth/` | `root:dwh-auth`, `2750` |
|
||||
| Lock | `/var/lib/dwh-auth/.writer.lock` | `root:dwh-auth`, `0640` |
|
||||
| Record | `/var/lib/dwh-auth/{active,revoked}/<public-key-id>.json` | `root:dwh-auth`, `0640` |
|
||||
| Socket runtime | `/run/dwh-auth/verify.sock` | `dwh-auth:www-data`, `0660` |
|
||||
| Consegne e backup | `/root/dwh-auth-provision/` | directory `root:root` `0700`, file `0600` |
|
||||
|
||||
Il record conserva un digest interno (`secret_sha256`) e metadati, mai la chiave in chiaro. Non
|
||||
leggere, stampare, calcolare o mettere quel digest in una prova operativa.
|
||||
|
||||
## Build, installazione e avvio
|
||||
|
||||
Costruire dal commit congelato e registrare solo checksum del binario e SHA sorgente:
|
||||
|
||||
```bash
|
||||
cd /srv/thothii/app
|
||||
bash scripts/build-dwh-auth.sh --output /tmp/dwh-auth-release
|
||||
sha256sum /tmp/dwh-auth-release/dwh-auth-linux-amd64
|
||||
```mermaid
|
||||
flowchart LR
|
||||
CLIENT["Installazione ThothII"] -->|"X-API-Key"| NGINX["Nginx"]
|
||||
NGINX --> AUTH["dwh-auth\nUnix socket"]
|
||||
AUTH --> REGISTRY["Registro chiavi\nactive e revoked"]
|
||||
AUTH -->|"authorized"| REST["DWH REST"]
|
||||
```
|
||||
|
||||
Scegliere l'architettura corretta. Il template PSD usa il gruppo Nginx `www-data`; confermarlo
|
||||
prima dell'installazione su un host diverso.
|
||||
## Confini di sicurezza
|
||||
|
||||
```bash
|
||||
sudo groupadd --system dwh-auth
|
||||
sudo useradd --system --no-create-home --shell /usr/sbin/nologin --gid dwh-auth dwh-auth
|
||||
sudo install -o root -g root -m 0755 /tmp/dwh-auth-release/dwh-auth-linux-amd64 /usr/local/sbin/dwh-auth
|
||||
sudo install -o root -g root -m 0644 deploy/dwh-auth/dwh-auth.service /etc/systemd/system/dwh-auth.service
|
||||
sudo install -o root -g root -m 0644 deploy/dwh-auth/dwh-auth.tmpfiles.conf /usr/lib/tmpfiles.d/dwh-auth.conf
|
||||
sudo install -d -o root -g root -m 0700 /root/dwh-auth-provision
|
||||
sudo systemd-tmpfiles --create /usr/lib/tmpfiles.d/dwh-auth.conf
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root /var/lib/dwh-auth check
|
||||
sudo systemd-analyze verify /etc/systemd/system/dwh-auth.service
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now dwh-auth
|
||||
sudo systemctl status dwh-auth --no-pager
|
||||
```
|
||||
- Una chiave identifica un'installazione, non una persona.
|
||||
- Chiavi e backup restano in file protetti e non entrano in Git, log, argomenti o JSON pubblico.
|
||||
- Il registro conserva digest e metadati, mai la chiave in chiaro.
|
||||
- La route REST deve essere esposta esclusivamente tramite TLS verificato.
|
||||
|
||||
Controllare i mode con `stat`. Il servizio apre il registro in sola lettura e crea solo il socket.
|
||||
Non creare JSON, lock o socket a mano: oggetti insicuri devono fallire chiusi.
|
||||
## Installazione
|
||||
|
||||
## Check, elenco e stato
|
||||
Il servizio usa questi percorsi:
|
||||
|
||||
Usare sempre un root assoluto. Questi comandi espongono solo ID pubblici, stato, date e scadenza:
|
||||
|
||||
```bash
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root /var/lib/dwh-auth check
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root /var/lib/dwh-auth key list --json
|
||||
key_id=public-key-id
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root /var/lib/dwh-auth key status --key-id "$key_id" --json
|
||||
```
|
||||
|
||||
Un errore di integrità, permessi, symlink o JSON malformato richiede ripristino da backup protetto,
|
||||
non una correzione manuale del record.
|
||||
|
||||
## Creazione, consegna, scadenza e revoca
|
||||
|
||||
Il comando crea la chiave una volta in un nuovo file assoluto `0600`; stdout contiene solo ID
|
||||
pubblico, installazione e percorso. Il file di output non deve esistere.
|
||||
|
||||
```bash
|
||||
installation_id=psd-mac-primary
|
||||
description=operatore-mac-primario
|
||||
key_output=/root/dwh-auth-provision/psd-mac-primary.key
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root /var/lib/dwh-auth key create \
|
||||
--installation-id "$installation_id" \
|
||||
--description "$description" \
|
||||
--output "$key_output"
|
||||
```
|
||||
|
||||
Aggiungere `--expires-at "YYYY-MM-DDTHH:MM:SSZ"` solo se la policy impone una scadenza; il default è nessuna
|
||||
scadenza. Consegnare il file solo con vault aziendale, secret manager, MDM o trasferimento
|
||||
autenticato ristretto. Mai email, chat, ticket, `cat` o copia-incolla. Il client conferma ID
|
||||
pubblico e ping, poi il materiale temporaneo viene rimosso secondo policy.
|
||||
|
||||
L'import legacy è temporaneo PSD: il file sorgente è già `root:root` `0600` e non viene mai letto o
|
||||
stampato dall'operatore.
|
||||
|
||||
```bash
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root /var/lib/dwh-auth key import \
|
||||
--legacy-raw --installation-id legacy-shared \
|
||||
--from-file /root/dwh-auth-provision/legacy-shared.key
|
||||
```
|
||||
|
||||
Per rotare: creare seconda generazione, consegnarla, configurarla e provare `/rpc/ping`; confermare
|
||||
l'ID pubblico; attendere l'osservazione; poi revocare la precedente e provare nuova=successo,
|
||||
precedente=401.
|
||||
|
||||
```bash
|
||||
previous_key_id=public-key-id
|
||||
revocation_reason=shared-credential-rotation
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root /var/lib/dwh-auth key revoke \
|
||||
--key-id "$previous_key_id" --reason "$revocation_reason"
|
||||
```
|
||||
|
||||
La revoca non è annullabile e un ID revocato non si ricrea.
|
||||
|
||||
## Backup, rollback e disinstallazione
|
||||
|
||||
Prima di mutare, creare un archivio root-only `0600` del registro e copie protette delle sole
|
||||
configurazioni coinvolte. L'archivio contiene digest, quindi è materiale riservato: custodirlo su
|
||||
storage cifrato approvato; l'evidenza ammessa riporta solo percorso, owner, mode, timestamp e
|
||||
checksum dell'archivio. Il rollback dual-key ripristina la route e il servizio revisionati, esegue
|
||||
`nginx -t` e fa reload solo autorizzato; non ripristina chiavi revocate, PostgreSQL, sessioni
|
||||
legacy, indici Qdrant o cache Ollama.
|
||||
|
||||
La disinstallazione richiede autorizzazione esplicita, client REST migrati/revocati e rollback non
|
||||
più necessario. Solo allora disabilitare l'unità; conservare registro e backup fino alla retention
|
||||
approvata. Non inserire `dwh-auth` in Compose o in `tht start`/`tht stop`.
|
||||
|
||||
## Procedure riproducibili e secret-safe
|
||||
|
||||
Eseguire soltanto nel gate autorizzato. Le variabili seguenti contengono percorsi, timestamp e
|
||||
codici, mai una chiave. Il manifest e l'archivio del registro sono `0600`; l'archivio resta
|
||||
materiale riservato su storage cifrato approvato.
|
||||
|
||||
```bash
|
||||
run_id=$(date -u +%Y%m%dT%H%M%SZ)
|
||||
backup_root=/root/dwh-auth-provision
|
||||
registry_root=/var/lib/dwh-auth
|
||||
registry_backup="$backup_root/registry-$run_id.tar"
|
||||
manifest="$backup_root/registry-$run_id.manifest"
|
||||
sudo install -o root -g root -m 0600 /dev/null "$registry_backup"
|
||||
sudo install -o root -g root -m 0600 /dev/null "$manifest"
|
||||
sudo tar --acls --xattrs -C /var/lib -cf "$registry_backup" dwh-auth
|
||||
sudo sh -c 'sha256sum "$1" > "$2"' sh "$registry_backup" "$manifest"
|
||||
if sudo sha256sum -c "$manifest" >/dev/null; then printf 'registry_manifest=PASS\n'; else printf 'registry_manifest=FAIL\n' >&2; exit 1; fi
|
||||
```
|
||||
|
||||
Il ripristino non sovrappone mai un tar al registro attivo. Estrarre prima in staging nello stesso
|
||||
filesystem di `/var/lib`, verificare il candidato, rinominare il registro attuale in una copia
|
||||
recuperabile e sostituirlo. Non cancellare il pre-ripristino: serve al rollback se `check` o
|
||||
l'avvio falliscono.
|
||||
|
||||
```bash
|
||||
registry_staging="/var/lib/.dwh-auth-restore-$run_id"
|
||||
registry_candidate="$registry_staging/dwh-auth"
|
||||
registry_previous="/var/lib/dwh-auth.pre-restore-$run_id"
|
||||
if [ -e "$registry_staging" ] || [ -e "$registry_previous" ]; then printf 'registry_restore=FAIL\n' >&2; exit 1; fi
|
||||
if ! sudo sha256sum -c "$manifest" >/dev/null; then printf 'registry_restore=FAIL\n' >&2; exit 1; fi
|
||||
if ! sudo install -d -o root -g root -m 0700 "$registry_staging"; then printf 'registry_restore=FAIL\n' >&2; exit 1; fi
|
||||
if ! sudo tar --acls --xattrs -C "$registry_staging" -xf "$registry_backup"; then printf 'registry_restore=FAIL\n' >&2; exit 1; fi
|
||||
if ! sudo /usr/local/sbin/dwh-auth --registry-root "$registry_candidate" check; then printf 'registry_restore=FAIL\n' >&2; exit 1; fi
|
||||
if ! sudo systemctl stop dwh-auth; then printf 'registry_restore=FAIL\n' >&2; exit 1; fi
|
||||
if ! sudo mv -T -- "$registry_root" "$registry_previous"; then
|
||||
if sudo systemctl start dwh-auth; then printf 'registry_restore_rollback=PASS\n' >&2; else printf 'registry_restore_rollback=FAIL\n' >&2; fi
|
||||
exit 1
|
||||
fi
|
||||
if ! sudo mv -T -- "$registry_candidate" "$registry_root"; then
|
||||
if ! sudo mv -T -- "$registry_previous" "$registry_root"; then printf 'registry_restore_rollback=FAIL\n' >&2; exit 1; fi
|
||||
if sudo /usr/local/sbin/dwh-auth --registry-root "$registry_root" check && sudo systemctl start dwh-auth; then printf 'registry_restore_rollback=PASS\n' >&2; else printf 'registry_restore_rollback=FAIL\n' >&2; fi
|
||||
exit 1
|
||||
fi
|
||||
if sudo /usr/local/sbin/dwh-auth --registry-root "$registry_root" check && sudo systemctl start dwh-auth; then
|
||||
printf 'registry_restore=PASS\n'
|
||||
else
|
||||
sudo systemctl stop dwh-auth || true
|
||||
if ! sudo mv -T -- "$registry_root" "$registry_staging/failed-dwh-auth"; then printf 'registry_restore_rollback=FAIL\n' >&2; exit 1; fi
|
||||
if ! sudo mv -T -- "$registry_previous" "$registry_root"; then printf 'registry_restore_rollback=FAIL\n' >&2; exit 1; fi
|
||||
if sudo /usr/local/sbin/dwh-auth --registry-root "$registry_root" check && sudo systemctl start dwh-auth; then printf 'registry_restore_rollback=PASS\n' >&2; else printf 'registry_restore_rollback=FAIL\n' >&2; fi
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
Per le prove, creare file header `0600` che contengono esattamente `X-API-Key: valore`. Il valore
|
||||
passa dal file chiave al file header senza argv, ambiente o stdout. I file header sono materiale
|
||||
segreto con la stessa custodia e retention delle chiavi.
|
||||
|
||||
```bash
|
||||
v1_key_file="$key_output"
|
||||
legacy_key_file=/root/dwh-auth-provision/legacy-shared.key
|
||||
v1_header_file=/root/dwh-auth-provision/dwh-auth-v1.header
|
||||
legacy_header_file=/root/dwh-auth-provision/dwh-auth-legacy.header
|
||||
random_header_file=/root/dwh-auth-provision/dwh-auth-random.header
|
||||
if ! sudo python3 -c '
|
||||
import pathlib, sys
|
||||
if any(b"\n" in pathlib.Path(path).read_bytes() for path in sys.argv[1:]):
|
||||
raise SystemExit(1)
|
||||
' "$v1_key_file" "$legacy_key_file"; then
|
||||
printf 'key_file_bytes=FAIL\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
printf 'key_file_bytes=PASS\n'
|
||||
for header_file in "$v1_header_file" "$legacy_header_file" "$random_header_file"; do
|
||||
sudo install -o root -g root -m 0600 /dev/null "$header_file"
|
||||
done
|
||||
sudo sh -c '{ printf "%s" "X-API-Key: "; dd if="$1" bs=65536 status=none; printf "\n"; } > "$2"' sh "$v1_key_file" "$v1_header_file"
|
||||
sudo sh -c '{ printf "%s" "X-API-Key: "; dd if="$1" bs=65536 status=none; printf "\n"; } > "$2"' sh "$legacy_key_file" "$legacy_header_file"
|
||||
sudo sh -c 'printf "%s\n" "X-API-Key: invalid-test" > "$1"' sh "$random_header_file"
|
||||
```
|
||||
|
||||
Il socket `/verify` deve restituire 204 per v1 e legacy durante il dual-key, 401 per file casuale
|
||||
e richiesta senza header. Stampare solo PASS/FAIL.
|
||||
|
||||
```bash
|
||||
status=$(sudo curl --header "@$v1_header_file" --unix-socket /run/dwh-auth/verify.sock --output /dev/null --silent --show-error --write-out '%{http_code}' http://localhost/verify)
|
||||
[ "$status" = 204 ] && printf 'socket_v1=PASS\n' || { printf 'socket_v1=FAIL\n' >&2; exit 1; }
|
||||
status=$(sudo curl --header "@$legacy_header_file" --unix-socket /run/dwh-auth/verify.sock --output /dev/null --silent --show-error --write-out '%{http_code}' http://localhost/verify)
|
||||
[ "$status" = 204 ] && printf 'socket_legacy=PASS\n' || { printf 'socket_legacy=FAIL\n' >&2; exit 1; }
|
||||
status=$(sudo curl --header "@$random_header_file" --unix-socket /run/dwh-auth/verify.sock --output /dev/null --silent --show-error --write-out '%{http_code}' http://localhost/verify)
|
||||
[ "$status" = 401 ] && printf 'socket_random=PASS\n' || { printf 'socket_random=FAIL\n' >&2; exit 1; }
|
||||
status=$(sudo curl --unix-socket /run/dwh-auth/verify.sock --output /dev/null --silent --show-error --write-out '%{http_code}' http://localhost/verify)
|
||||
[ "$status" = 401 ] && printf 'socket_missing=PASS\n' || { printf 'socket_missing=FAIL\n' >&2; exit 1; }
|
||||
```
|
||||
|
||||
Per HTTPS reale usare i file header protetti e la CA approvata contro `/dwh/rpc/ping`: PostgREST
|
||||
può restituire qualsiasi 2xx, non si pretende 204. Prima della revoca, v1 e legacy devono dare
|
||||
2xx; il file casuale deve dare 401.
|
||||
|
||||
```bash
|
||||
ping_url=https://supabase-aritmolab.policlinicosandonato.it/dwh/rpc/ping
|
||||
ca_file=/root/dwh-auth-provision/psd-dwh-ca.pem
|
||||
status=$(sudo curl --header "@$v1_header_file" --cacert "$ca_file" --connect-timeout 5 --max-time 15 --output /dev/null --silent --show-error --write-out '%{http_code}' "$ping_url")
|
||||
case "$status" in 2??) printf 'https_v1_pre_revoke=PASS\n' ;; *) printf 'https_v1_pre_revoke=FAIL\n' >&2; exit 1 ;; esac
|
||||
status=$(sudo curl --header "@$legacy_header_file" --cacert "$ca_file" --connect-timeout 5 --max-time 15 --output /dev/null --silent --show-error --write-out '%{http_code}' "$ping_url")
|
||||
case "$status" in 2??) printf 'https_legacy_pre_revoke=PASS\n' ;; *) printf 'https_legacy_pre_revoke=FAIL\n' >&2; exit 1 ;; esac
|
||||
status=$(sudo curl --header "@$random_header_file" --cacert "$ca_file" --connect-timeout 5 --max-time 15 --output /dev/null --silent --show-error --write-out '%{http_code}' "$ping_url")
|
||||
[ "$status" = 401 ] && printf 'https_random=PASS\n' || { printf 'https_random=FAIL\n' >&2; exit 1; }
|
||||
```
|
||||
|
||||
Dopo l'osservazione, revocare solo la legacy usando il suo ID pubblico già registrato. Dopo la
|
||||
revoca v1 resta 2xx e legacy diventa 401 anche via HTTPS.
|
||||
|
||||
```bash
|
||||
legacy_key_id=legacy-shared
|
||||
sudo /usr/local/sbin/dwh-auth --registry-root "$registry_root" key revoke --key-id "$legacy_key_id" --reason shared-credential-rotation
|
||||
status=$(sudo curl --header "@$v1_header_file" --cacert "$ca_file" --connect-timeout 5 --max-time 15 --output /dev/null --silent --show-error --write-out '%{http_code}' "$ping_url")
|
||||
case "$status" in 2??) printf 'https_v1_post_revoke=PASS\n' ;; *) printf 'https_v1_post_revoke=FAIL\n' >&2; exit 1 ;; esac
|
||||
status=$(sudo curl --header "@$legacy_header_file" --cacert "$ca_file" --connect-timeout 5 --max-time 15 --output /dev/null --silent --show-error --write-out '%{http_code}' "$ping_url")
|
||||
[ "$status" = 401 ] && printf 'https_legacy_post_revoke=PASS\n' || { printf 'https_legacy_post_revoke=FAIL\n' >&2; exit 1; }
|
||||
```
|
||||
|
||||
Per provare 503 in una finestra approvata, registrare l'orario, fermare temporaneamente l'unità,
|
||||
eseguire il ping con timeout e trap di ripristino; il comando deve stampare solo PASS/FAIL.
|
||||
|
||||
```bash
|
||||
was_active=$(sudo systemctl is-active dwh-auth || true)
|
||||
[ "$was_active" = active ] || { printf 'https_auth_down=FAIL\n' >&2; exit 1; }
|
||||
restore_auth() { sudo systemctl start dwh-auth; }
|
||||
trap restore_auth EXIT INT TERM
|
||||
sudo systemctl stop dwh-auth
|
||||
status=$(sudo curl --header "@$random_header_file" --cacert "$ca_file" --connect-timeout 5 --max-time 15 --output /dev/null --silent --show-error --write-out '%{http_code}' "$ping_url" || true)
|
||||
[ "$status" = 503 ] && printf 'https_auth_down=PASS\n' || { printf 'https_auth_down=FAIL\n' >&2; exit 1; }
|
||||
sudo systemctl start dwh-auth
|
||||
trap - EXIT INT TERM
|
||||
```
|
||||
|
||||
Lo scan journal non salva righe grezze: controlla davvero le chiavi v1 e legacy leggendo solo i
|
||||
percorsi dei file da argv, e conserva anche la difesa generica per prefisso e digest. Il filtro
|
||||
emette solo PASS/FAIL.
|
||||
|
||||
```bash
|
||||
since=$(date -u -d '15 minutes ago' +%Y-%m-%dT%H:%M:%SZ)
|
||||
if sudo python3 -c '
|
||||
import pathlib, subprocess, sys
|
||||
max_journal_bytes = 1_048_576
|
||||
max_journal_lines = 10_000
|
||||
max_chunk_bytes = 65_536
|
||||
process = None
|
||||
try:
|
||||
actual_keys = {pathlib.Path(path).read_bytes() for path in sys.argv[2:]}
|
||||
needles = (b"thtdwh_v1", b"secret_sha256", *actual_keys)
|
||||
max_needle_length = max(map(len, needles))
|
||||
process = subprocess.Popen(
|
||||
["journalctl", "-u", "dwh-auth", "--since", sys.argv[1], "--no-pager", "--output=cat"],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
except OSError:
|
||||
raise SystemExit(2)
|
||||
def stop_child():
|
||||
if process is not None:
|
||||
if process.poll() is None:
|
||||
process.kill()
|
||||
process.wait()
|
||||
bytes_seen = 0
|
||||
line_count = 0
|
||||
line_open = False
|
||||
carry = b""
|
||||
try:
|
||||
while True:
|
||||
remaining = max_journal_bytes - bytes_seen
|
||||
if remaining == 0:
|
||||
if process.stdout.read1(1):
|
||||
raise SystemExit(2)
|
||||
break
|
||||
chunk = process.stdout.read1(min(max_chunk_bytes, remaining))
|
||||
if not chunk:
|
||||
break
|
||||
bytes_seen += len(chunk)
|
||||
searchable = carry + chunk
|
||||
if any(needle in searchable for needle in needles):
|
||||
raise SystemExit(1)
|
||||
carry = searchable[-(max_needle_length - 1):]
|
||||
for byte in chunk:
|
||||
if byte == 10:
|
||||
line_count += 1
|
||||
line_open = False
|
||||
if line_count > max_journal_lines:
|
||||
raise SystemExit(2)
|
||||
else:
|
||||
line_open = True
|
||||
if line_open:
|
||||
line_count += 1
|
||||
if line_count > max_journal_lines:
|
||||
raise SystemExit(2)
|
||||
finally:
|
||||
stop_child()
|
||||
if process.returncode != 0:
|
||||
raise SystemExit(2)
|
||||
' "$since" "$v1_key_file" "$legacy_key_file"; then
|
||||
printf 'journal_actual_key_scan=PASS\n'
|
||||
else
|
||||
printf 'journal_actual_key_scan=FAIL\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
Dopo rollback verificato e migrazione/revoca di ogni client REST, la disinstallazione resta
|
||||
condizionata all'approvazione: eseguire `sudo systemctl disable --now dwh-auth`, ma mantenere
|
||||
registro, backup, manifest e file header protetti per la retention; non cancellarli durante il
|
||||
rollback.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Sintomo | Interpretazione e azione |
|
||||
| Oggetto | Percorso |
|
||||
| --- | --- |
|
||||
| `401` | Chiave assente, malformata, sconosciuta, scaduta, revocata o errata. Verificare trasporto, ID pubblico e consegna; non cercare dettagli nel messaggio. |
|
||||
| `503` | Servizio, socket o registro non disponibile/sicuro. Controllare `systemctl`, socket, mode e `check`; ripristinare il backup approvato. |
|
||||
| `check` fallisce | Integrità del registro non valida. Fermare le scritture, preservare stato e ripristinare; non editare JSON. |
|
||||
| TLS fallisce | CA o SAN non validi. Seguire [TLS](dwh-auth-tls.md), senza bypass. |
|
||||
| Binario | `/usr/local/sbin/dwh-auth` |
|
||||
| Unit systemd | `/etc/systemd/system/dwh-auth.service` |
|
||||
| Registro | `/var/lib/dwh-auth/` |
|
||||
| Socket | `/run/dwh-auth/verify.sock` |
|
||||
| Consegne protette | `/root/dwh-auth-provision/` |
|
||||
|
||||
Per il rollout PSD con i due gate separati vedere il [runbook PSD](../operations/psd-dwh-auth-rollout.md).
|
||||
Installare binario e unit con owner `root`, creare l'utente di servizio `dwh-auth`, quindi
|
||||
abilitare l'unità con `systemctl enable --now dwh-auth`. Il socket deve essere accessibile al
|
||||
gruppo usato da Nginx.
|
||||
|
||||
## Creazione e revoca delle chiavi
|
||||
|
||||
Esempio per l'installazione ACME Limited:
|
||||
|
||||
```bash
|
||||
sudo dwh-auth --registry-root /var/lib/dwh-auth key create \
|
||||
--installation-id acme-factory-primary \
|
||||
--description acme-factory-primary \
|
||||
--output /root/dwh-auth-provision/acme-factory-primary.key
|
||||
```
|
||||
|
||||
Consegnare il file attraverso un vault aziendale o un canale autenticato. Per la rotazione,
|
||||
creare una nuova chiave, distribuirla, aggiornare il client e revocare la precedente usando il
|
||||
suo ID pubblico:
|
||||
|
||||
```bash
|
||||
sudo dwh-auth --registry-root /var/lib/dwh-auth key revoke \
|
||||
--key-id PUBLIC_KEY_ID \
|
||||
--reason scheduled-rotation
|
||||
```
|
||||
|
||||
La revoca è definitiva. Conservare backup cifrati del registro prima di ogni mutazione.
|
||||
|
||||
## Integrazione Nginx
|
||||
|
||||
Nginx inoltra la chiave al socket di `dwh-auth`. Solo una risposta autorizzata permette il
|
||||
passaggio verso il DWH REST; chiavi assenti, sconosciute, scadute o revocate ricevono `401`,
|
||||
mentre indisponibilità del servizio o del registro producono `503`.
|
||||
|
||||
@@ -1,49 +1,40 @@
|
||||
# TLS per DWH REST
|
||||
|
||||
La chiave DWH è accettabile solo sopra TLS verificato. Un errore `401` o `503` non autorizza mai a
|
||||
ridurre la verifica del certificato.
|
||||
La chiave DWH è accettabile solo sopra TLS verificato. Errori di autorizzazione o disponibilità
|
||||
non autorizzano mai a disabilitare la verifica del certificato.
|
||||
|
||||
## Stato PSD
|
||||
## CA privata
|
||||
|
||||
L'origine REST PSD corrente usa il certificato self-issued/private di Nginx. Il SAN copre
|
||||
`supabase-aritmolab.policlinicosandonato.it`, l'origine `.it` approvata, e non copre un dominio
|
||||
`.com`. Non usare `.com` finché non è incluso esplicitamente nel SAN.
|
||||
Quando il DWH REST usa una CA aziendale, consegnare il certificato separatamente dalla chiave
|
||||
API. La CA non è una credenziale, ma la sua integrità è un confine di sicurezza: deve restare
|
||||
fuori da Git e non essere scrivibile da utenti non autorizzati.
|
||||
|
||||
Chi non dispone già di trust equivalente approvato riceve la CA separatamente e configura
|
||||
`TLS_CA_FILE`. La CA non è una credenziale, ma la sua integrità è un confine di sicurezza: fuori da
|
||||
Git e non scrivibile da utenti non autorizzati.
|
||||
Esempio ACME Limited:
|
||||
|
||||
```dotenv
|
||||
THT_WS_ACME_EBIKES_DWH_TLS_CA_FILE=/run/secrets/acme-ebikes-dwh-ca.pem
|
||||
```
|
||||
|
||||
## Fingerprint fuori banda
|
||||
|
||||
Calcolare localmente il fingerprint del file ricevuto:
|
||||
Calcolare il fingerprint del file ricevuto e confrontarlo attraverso un canale indipendente:
|
||||
|
||||
```bash
|
||||
openssl x509 -noout -fingerprint -sha256 -in /absolute/protected/psd-dwh-ca.pem
|
||||
openssl x509 -noout -fingerprint -sha256 \
|
||||
-in /absolute/protected/acme-ebikes-dwh-ca.pem
|
||||
```
|
||||
|
||||
Confrontarlo con il responsabile autorizzato tramite un canale indipendente dalla consegna (vault
|
||||
aziendale o canale telefonico verificato). Nell'evidenza registrare solo conferma, approvatore e
|
||||
timestamp; mai corpo certificato, fingerprint completo o output grezzo.
|
||||
Il SAN del certificato deve includere il nome esatto usato dal binding, per esempio
|
||||
`dwh.acme.example`.
|
||||
|
||||
## Binding e ping
|
||||
## Rinnovo
|
||||
|
||||
Il binding headless PSD effettivo è:
|
||||
1. Preparare certificato e chain nuovi.
|
||||
2. Confermare SAN e fingerprint fuori banda.
|
||||
3. Distribuire la nuova CA ai client mantenendo temporaneamente la precedente.
|
||||
4. Aggiornare il binding e confermare la connettività con TLS normale.
|
||||
5. Installare il certificato server.
|
||||
6. Ritirare il trust precedente dopo la finestra concordata.
|
||||
|
||||
```dotenv
|
||||
THT_WS_PSD_CLINICAL_DWH_TLS_CA_FILE=/run/secrets/psd-clinical-dwh-ca.pem
|
||||
```
|
||||
|
||||
Il file sorgente locale è collegato da file operatore non tracciato. Usare URL `.it`, poi
|
||||
**Test workspace connections** su `/rpc/ping`. Non disabilitare TLS e non usare `curl -k`.
|
||||
|
||||
## Rinnovo coordinato
|
||||
|
||||
1. Preparare certificato e chain nuovi; verificare prima SAN `.it` e assenza di falsa copertura `.com`.
|
||||
2. Confermare fuori banda il nuovo fingerprint.
|
||||
3. Consegnare la CA/chain nuova ai client con `TLS_CA_FILE`, senza rimuovere ancora la precedente.
|
||||
4. Aggiornare vault/binding e verificare ping con TLS normale.
|
||||
5. Solo con gate Nginx approvato installare il certificato server e ripetere il ping.
|
||||
6. Ritirare il trust precedente dopo la finestra approvata.
|
||||
|
||||
Il rinnovo non modifica chiavi `dwh-auth`, record o ruoli PostgreSQL. TLS e rollback della route
|
||||
restano approvazioni e backup distinti.
|
||||
Non usare `curl -k`, non disabilitare TLS e non incorporare certificati o fingerprint completi
|
||||
nei documenti condivisi.
|
||||
|
||||
@@ -1,176 +0,0 @@
|
||||
# Local workspace repository installation (macOS, Windows, and Linux)
|
||||
|
||||
This manual connects a local ThothII installation to one remote Git repository hosted by a Git
|
||||
server such as GitHub, GitLab, or Gitea. ThothII is a read-only consumer: it fetches,
|
||||
validates, and activates workspace revisions, but never edits, commits, pushes, or publishes them.
|
||||
|
||||
## Architecture ownership contract
|
||||
|
||||
| Component | Ownership | Operator contract |
|
||||
| --- | --- | --- |
|
||||
| DWH | External | Configure the external endpoint and complete its runtime credentials in Workspace management. |
|
||||
| LLM | External | Configure the external endpoint and model policy during installation. |
|
||||
| Qdrant | Internal | Compose runs the internal service and persists `qdrant-data`. |
|
||||
| Ollama embedding | Internal | Compose runs the internal `qwen3-embedding:0.6b` service and model-init job. |
|
||||
|
||||
## Semantic index ownership contract
|
||||
|
||||
| Scope | Ownership rule | Isolation rule |
|
||||
| --- | --- | --- |
|
||||
| Workspace semantic index | Each workspace keeps exactly one Qdrant collection reserved for itself. | Schema, Evidence, and memory records share that one collection and are separated by the `kind` payload. |
|
||||
|
||||
The mandatory semantic stack is CPU-first. Set `THOTH_ENABLE_EMBEDDING_GPU=1` only after the
|
||||
documented GPU prerequisites are satisfied. The embedding contract is fixed at
|
||||
`qwen3-embedding:0.6b`, 1024 dimensions, cosine distance.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A working local installation described by [local.md](local.md).
|
||||
- A remote Git repository and a read-only deploy credential for this ThothII installation.
|
||||
- A separate authoring clone in which a workspace curator can edit and publish source revisions.
|
||||
- `tht` built with `bash scripts/build-tht.sh`.
|
||||
|
||||
## Prepare and publish a workspace source
|
||||
|
||||
Create a local workspace in an ordinary source directory outside ThothII's data directories. The
|
||||
canonical repository layout is:
|
||||
|
||||
```text
|
||||
thoth-workspaces.yaml
|
||||
<workspace-id>/workspace.yaml
|
||||
<workspace-id>/evidence/ # optional, repository-owned Evidence
|
||||
<workspace-id>/schema/annotations.yaml # optional curated annotations
|
||||
```
|
||||
|
||||
The catalog lists `{id, name, description?}` and the descriptor at
|
||||
`<workspace-id>/workspace.yaml` must match that metadata. Use the examples in
|
||||
`deploy/workspaces/` as authoring references. Do not store passwords, tokens, private keys, or
|
||||
signed URLs in Git.
|
||||
|
||||
Publishing is an author-side Git operation: validate the source, commit it, and push it from the
|
||||
separate authoring clone to the configured branch. This is the only meaning of “publish” in the
|
||||
workspace lifecycle. ThothII has no author identity and no Git write credential.
|
||||
|
||||
## Use the workspace from the application
|
||||
|
||||
After the installation is started, use Workspace management from the authenticated application:
|
||||
|
||||
1. Run **Update workspace repository** to fetch and validate the configured Git branch into the
|
||||
application-owned registry. The operation is all-or-nothing and does not modify the authoring
|
||||
clone.
|
||||
2. Confirm that the installation-owned `workspace-secrets` storage remains outside the source
|
||||
repository and contains no credentials in the workspace descriptors.
|
||||
3. Select the workspace and run **Validate workspace source** to verify the active descriptor, catalog,
|
||||
Evidence, annotations, and runtime bindings.
|
||||
4. Run **Test workspace connections** only with the approved read-only DWH/Evidence test configuration.
|
||||
Results are redacted and the workspace source remains unchanged.
|
||||
|
||||
<!-- workspace-descriptor-contract:start -->
|
||||
Schema v3 is the only accepted workspace descriptor.
|
||||
Schema v1 and v2 workspace descriptors are rejected before activation.
|
||||
<!-- workspace-descriptor-contract:end -->
|
||||
|
||||
## Configure the remote Git repository
|
||||
|
||||
Copy `docs/install/examples/thothii-installation.local.yaml` to an operator-controlled absolute
|
||||
path. Its `workspaceRepository` block records the remote, branch, and read-only access method.
|
||||
Choose exactly one transport override:
|
||||
|
||||
- SSH: `deploy/compose.git-ssh.yaml`, with a read-only deploy key and pinned `known_hosts` file.
|
||||
- HTTPS: `deploy/compose.git-https.yaml`, with a read-only token in a Git credentials file and an
|
||||
optional private CA file.
|
||||
|
||||
The remote and branch are installation configuration. Git credentials remain protected
|
||||
installation files and are never accepted by Workspace management or returned by its API.
|
||||
|
||||
Example non-secret/operator paths:
|
||||
|
||||
```dotenv
|
||||
THT_WORKSPACE_GIT_REMOTE=git@git.example.com:organization/workspaces.git
|
||||
THT_WORKSPACE_GIT_BRANCH=main
|
||||
THT_WORKSPACE_INSTALLATION_ID=local
|
||||
PI_AUTH_FILE=/absolute/path/to/operator/pi-auth.json
|
||||
THT_SECRETS_FILE=/absolute/path/to/operator/thothii.secrets
|
||||
THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/to/operator/git-ssh-key
|
||||
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/to/operator/git-known-hosts
|
||||
```
|
||||
|
||||
Keep these files outside both the ThothII checkout and the workspace source repository. Protect
|
||||
them with mode `0600` on macOS/Linux or an equivalent single-user ACL on Windows.
|
||||
|
||||
## Start and update the installation
|
||||
|
||||
Use only the installation-aware lifecycle:
|
||||
|
||||
```bash
|
||||
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
||||
THT_BIN=tht
|
||||
INSTALLATION=/absolute/path/to/operator/thothii-installation.yaml
|
||||
"$THT_BIN" --installation "$INSTALLATION" start
|
||||
"$THT_BIN" --installation "$INSTALLATION" doctor
|
||||
```
|
||||
|
||||
At startup ThothII clones or fetches the configured repository into its application-managed
|
||||
`workspace-registry` volume. Later, **Update workspace repository** performs a server-side fetch
|
||||
and fast-forward candidate checkout. It does not copy anything to the user's computer.
|
||||
|
||||
## Complete runtime secrets in Workspace management
|
||||
|
||||
### Chiavi DWH REST per installazione
|
||||
|
||||
Se il binding selezionato è `rest_api`, la chiave DWH è una credenziale per questa installazione e si salva nel vault tramite **Save entered secrets** oppure in un file locale indicato da `API_KEY_FILE`. `postgres_direct` e `ssh_tunnel` non usano questa chiave. Per emissione, TLS, rotazione e verifica `/rpc/ping`, seguire [enrollment DWH REST](dwh-auth-client-enrollment.md).
|
||||
|
||||
Open Workspace management after the first successful repository update.
|
||||
|
||||
1. At the repository level, review the configured host, repository, branch, and current revision.
|
||||
2. Select a workspace. Repository update does not require a selection; validation and connection
|
||||
tests do.
|
||||
3. Review the runtime fields derived from the selected DWH transport and Evidence authentication
|
||||
mechanism.
|
||||
4. Enter or rotate the required values and choose **Save entered secrets**.
|
||||
5. Run **Validate workspace source** and then **Test workspace connections**.
|
||||
|
||||
Secret fields are write-only. The GUI receives only configured/missing status. Values are
|
||||
encrypted by the backend in the platform-neutral `workspace-secrets` volume. ThothII temporarily
|
||||
materializes a restrictive file only while an existing file-oriented connector needs it, then
|
||||
removes that file when the runtime lease ends. **Forget stored value** deletes the selected encrypted value.
|
||||
|
||||
The workspace YAML stays environment-independent: it declares connector mechanisms, not host
|
||||
paths or credentials. Installation trust material such as a Git CA or `known_hosts` remains an
|
||||
operator concern; DWH and Evidence credentials are completed in the GUI.
|
||||
|
||||
## Validation and activation behavior
|
||||
|
||||
An update follows this sequence:
|
||||
|
||||
1. Fetch the configured branch into a candidate checkout managed by ThothII.
|
||||
2. Validate the catalog, every descriptor, repository-relative Evidence, and cross-workspace
|
||||
invariants at the same Git commit.
|
||||
3. If every workspace is valid, atomically mark that complete commit as active.
|
||||
4. If any validation fails, report sanitized diagnostics and keep the previous active revision.
|
||||
|
||||
The active checkout is read-only application state. Never edit files under
|
||||
`/data/workspace-registry`. A source correction must be committed and pushed from the authoring
|
||||
clone, then fetched again with **Update workspace repository**.
|
||||
|
||||
## Backup, rotation, and recovery
|
||||
|
||||
Back up the `workspace-registry`, `workspace-secrets`, `sessions`, `qdrant-data`,
|
||||
`embedding-models`, `settings`, and `pi-state` volumes together. The encrypted vault is useless
|
||||
without its generated master key, so preserve the entire `workspace-secrets` volume and protect
|
||||
the backup as secret material.
|
||||
|
||||
Rotate a runtime credential by saving its replacement in Workspace management and rerunning its
|
||||
connection test. Rotate Git credentials in the installation files and restart `core`. To recover
|
||||
from a bad remote revision, correct or revert it in the authoring repository and run the update;
|
||||
until validation succeeds, the previous active snapshot remains available.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Meaning and action |
|
||||
| --- | --- |
|
||||
| Repository unavailable | Check remote host, branch, read-only deploy credential, CA, and `known_hosts`. |
|
||||
| Candidate rejected | Fix the reported source error in the authoring clone, commit, push, and update again. |
|
||||
| Runtime configuration required | Select the workspace and complete each required secret field. |
|
||||
| Connection test fails | Rotate the relevant secret or correct the non-secret endpoint in the source/installation as appropriate. |
|
||||
| Active revision did not change | The candidate was invalid or was already active; inspect the repository status. |
|
||||
@@ -1,499 +0,0 @@
|
||||
# Install ThothII on a local PC or Mac
|
||||
|
||||
This guide installs one loopback-only ThothII on the same Windows, macOS, or Linux computer that
|
||||
runs Docker. The supported application is one Docker Compose distribution containing exactly
|
||||
`frontend` and `core`; Pi is pinned inside `core`. DWH, vector database, embedding, and LLM remain
|
||||
external configurable services even when they run on this computer.
|
||||
|
||||
No host Pi, Node.js, Python, Go toolchain, Docker socket in core, or browser shell is required.
|
||||
Commands that contain example paths must be changed to absolute paths on your computer.
|
||||
|
||||
## Choose your platform
|
||||
|
||||
- **macOS:** use Terminal and Docker Desktop. Apple Silicon and Intel are supported by the local
|
||||
image build.
|
||||
- **Windows PowerShell:** use Docker Desktop with its WSL2 engine, Git for Windows, the Windows
|
||||
build launcher, and `tht-windows-amd64.exe`.
|
||||
- **Windows WSL2 (recommended):** enable Docker Desktop integration for your Linux distribution,
|
||||
clone under `/home/<user>` rather than `/mnt/c`, and follow the Linux shell commands.
|
||||
- **Linux PC:** use Docker Engine plus the Compose v2 plugin and the Linux `tht` binary.
|
||||
|
||||
Windows users must also read [Windows and WSL2 line endings](windows-line-endings.md) before the
|
||||
first build.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Install only:
|
||||
|
||||
1. Git 2.39 or newer.
|
||||
2. Docker Desktop on macOS/Windows, or Docker Engine on Linux.
|
||||
3. Docker Compose v2 (`docker compose`, not legacy `docker-compose`).
|
||||
4. About 10 GB of free disk for source, images, build cache, and initial volumes.
|
||||
5. Network access to the workspace Git remote and configured DWH/vector/embedding/LLM endpoints.
|
||||
|
||||
Verify the tools:
|
||||
|
||||
```sh
|
||||
git --version
|
||||
docker version
|
||||
docker compose version
|
||||
docker run --rm hello-world
|
||||
```
|
||||
|
||||
On Linux, add the operator to the Docker group only if local policy permits it; sign out and back
|
||||
in afterward. A local installation needs no inbound firewall rule because ports bind only to
|
||||
`127.0.0.1`.
|
||||
|
||||
## Clone and verify LF
|
||||
|
||||
Use a `git clone` command that disables automatic CRLF conversion for this checkout.
|
||||
|
||||
macOS and Linux:
|
||||
|
||||
```sh
|
||||
git -c core.autocrlf=false clone https://github.example.invalid/your-org/ThothII.git
|
||||
cd ThothII
|
||||
git config --local core.autocrlf false
|
||||
bash scripts/verify-line-endings.sh
|
||||
```
|
||||
|
||||
Windows PowerShell:
|
||||
|
||||
```powershell
|
||||
git -c core.autocrlf=false clone https://github.example.invalid/your-org/ThothII.git
|
||||
Set-Location ThothII
|
||||
git config --local core.autocrlf false
|
||||
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh
|
||||
```
|
||||
|
||||
Windows WSL2:
|
||||
|
||||
```sh
|
||||
mkdir -p "$HOME/src" && cd "$HOME/src"
|
||||
git -c core.autocrlf=false clone https://github.example.invalid/your-org/ThothII.git
|
||||
cd ThothII
|
||||
git config --local core.autocrlf false
|
||||
bash scripts/verify-line-endings.sh
|
||||
```
|
||||
|
||||
Stop if the verifier names any path. Do not build from a CRLF checkout.
|
||||
|
||||
## Create the local operator files
|
||||
|
||||
Copy the non-secret template. This untracked `.env` contains addresses and absolute source paths,
|
||||
never secret values:
|
||||
|
||||
```sh
|
||||
cp deploy/env/local.env.example deploy/env/local.env
|
||||
mkdir -p /absolute/path/to/thothii-operator/secrets
|
||||
chmod 0700 /absolute/path/to/thothii-operator/secrets
|
||||
```
|
||||
|
||||
Native Windows PowerShell performs the same setup without POSIX utilities. The ACL commands remove
|
||||
inherited access from the new operator directory and grant full control only to the current Windows
|
||||
identity. Stop if either `icacls.exe` command returns a nonzero exit code:
|
||||
|
||||
```powershell
|
||||
$OperatorDir = Join-Path $env:USERPROFILE 'thothii-operator'
|
||||
$SecretsDir = Join-Path $OperatorDir 'secrets'
|
||||
$CurrentUser = [System.Security.Principal.WindowsIdentity]::GetCurrent().Name
|
||||
if (Test-Path $OperatorDir) { throw 'Use a new operator directory or review its ACLs manually.' }
|
||||
New-Item -ItemType Directory -Force -Path $OperatorDir, $SecretsDir | Out-Null
|
||||
icacls.exe $OperatorDir /inheritance:r
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Could not remove inherited operator-directory ACLs.' }
|
||||
icacls.exe $OperatorDir /grant:r "${CurrentUser}:(OI)(CI)F"
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Could not grant the current user the operator-directory ACL.' }
|
||||
Copy-Item deploy/env/local.env.example deploy/env/local.env
|
||||
Copy-Item docs/install/examples/thothii-installation.local.yaml `
|
||||
(Join-Path $OperatorDir 'thothii-installation.yaml')
|
||||
```
|
||||
|
||||
Edit `deploy/env/local.env`. At minimum set the workspace Git remote, `PI_AUTH_FILE`,
|
||||
`THT_SECRETS_FILE`, and external service endpoints. Create the Pi/application and Git transport
|
||||
files under the protected operator directory and set mode `0600`. On Windows use a user-only ACL
|
||||
instead. DWH and Evidence credentials are entered later through Workspace management and stored
|
||||
in the backend's encrypted `workspace-secrets` volume.
|
||||
|
||||
Do not paste credentials into this guide's commands, `.env`, workspace YAML, Git, URLs, image build
|
||||
arguments, or the installation descriptor. Secret contents are mounted read-only under
|
||||
`/run/secrets` (Pi's auth store has its own protected read-only mount) and must never be committed,
|
||||
embedded, rendered, or logged.
|
||||
|
||||
Follow [the local workspace repository guide](local-workspace-registry.md) to choose exactly one
|
||||
read-only Git SSH/HTTPS override. A fresh install requires a valid private workspace repository;
|
||||
the remote Git repository remains the source of truth.
|
||||
|
||||
Copy the installation example to an operator-controlled file named exactly
|
||||
`thothii-installation.yaml`, then replace all placeholders with absolute paths:
|
||||
|
||||
```sh
|
||||
cp docs/install/examples/thothii-installation.local.yaml \
|
||||
/absolute/path/to/thothii-operator/thothii-installation.yaml
|
||||
```
|
||||
|
||||
For HTTPS, replace the SSH override in that file with `deploy/compose.git-https.yaml`. Add only
|
||||
reviewed local overrides. Paths may contain spaces when correctly represented as YAML strings.
|
||||
|
||||
Native Windows uses the same four fields. Use single-quoted absolute Windows paths so backslashes
|
||||
remain literal YAML characters:
|
||||
|
||||
```yaml
|
||||
profile: local
|
||||
projectDirectory: 'C:\Users\operator\src\ThothII'
|
||||
envFile: 'C:\Users\operator\src\ThothII\deploy\env\local.env'
|
||||
overrides:
|
||||
- 'C:\Users\operator\src\ThothII\deploy\compose.git-ssh.yaml'
|
||||
```
|
||||
|
||||
## Address external services
|
||||
|
||||
An address is interpreted inside `core`. Therefore container 127.0.0.1 means the container itself,
|
||||
not the Docker host. Keep external DWH and LLM addresses configurable in the installation; Qdrant
|
||||
and embedding are internal services in the standard stack.
|
||||
|
||||
- **Docker Desktop (macOS and Windows):** use `host.docker.internal`, for example
|
||||
`http://host.docker.internal:11434`.
|
||||
- **Linux:** if a service runs on the host, create an untracked override and include its absolute
|
||||
path in `thothii-installation.yaml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
core:
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
```
|
||||
|
||||
Then use `host.docker.internal` in the endpoint. `extra_hosts: host.docker.internal:host-gateway`
|
||||
is a host routing aid, not a bundled service. Prefer a real DNS name for independently operated
|
||||
services; retain TLS and authentication even when co-located.
|
||||
|
||||
## Build ThothII and tht
|
||||
|
||||
The canonical local Compose smoke uses the base file plus the local profile. Keep this exact
|
||||
base+profile command available for install verification:
|
||||
|
||||
~~~sh
|
||||
docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local.yaml up --build -d
|
||||
~~~
|
||||
|
||||
After the stack is ready, configure and check authentication with the single host CLI tht; see
|
||||
the [local authentication guide](authentication-local.md). Authentication configuration is
|
||||
installation-global and is checked before workspace tests.
|
||||
|
||||
From the repository root, macOS/Linux/WSL2 users run:
|
||||
|
||||
```sh
|
||||
bash scripts/build-local.sh
|
||||
bash scripts/build-tht.sh
|
||||
```
|
||||
|
||||
Native PowerShell users run:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts/build-local.ps1
|
||||
& "C:\Program Files\Git\bin\bash.exe" scripts/build-tht.sh
|
||||
```
|
||||
|
||||
The second command uses Docker to create native operator binaries under `dist/tht`; users do
|
||||
not need to know or install Go. Select `tht-darwin-arm64` or `-amd64` on macOS,
|
||||
`tht-linux-amd64` or `-arm64` on Linux/WSL2, and `tht-windows-amd64.exe` on Windows.
|
||||
Copy the selected file to the protected operator directory and, on macOS/Linux, run `chmod 0755`
|
||||
on it.
|
||||
|
||||
## Start and verify
|
||||
|
||||
Set convenient variables (PowerShell users use `$THT_BIN` and `$INSTALLATION` with `& $THT_BIN`):
|
||||
Every operator call has the form `tht --installation <absolute-descriptor> <command>`.
|
||||
|
||||
```sh
|
||||
THT_BIN=/absolute/path/to/thothii-operator/tht
|
||||
INSTALLATION=/absolute/path/to/thothii-operator/thothii-installation.yaml
|
||||
"$THT_BIN" --installation "$INSTALLATION" update --check-only
|
||||
"$THT_BIN" --installation "$INSTALLATION" start
|
||||
"$THT_BIN" --installation "$INSTALLATION" status
|
||||
"$THT_BIN" --installation "$INSTALLATION" doctor
|
||||
```
|
||||
|
||||
Native PowerShell uses the same order:
|
||||
|
||||
```powershell
|
||||
$THT_BIN = 'C:\Users\operator\thothii-operator\tht.exe'
|
||||
$INSTALLATION = 'C:\Users\operator\thothii-operator\thothii-installation.yaml'
|
||||
& $THT_BIN --installation $INSTALLATION update --check-only
|
||||
& $THT_BIN --installation $INSTALLATION start
|
||||
& $THT_BIN --installation $INSTALLATION status
|
||||
& $THT_BIN --installation $INSTALLATION doctor
|
||||
```
|
||||
|
||||
Wait for both services, then check the same-origin frontend and direct loopback core:
|
||||
|
||||
```sh
|
||||
curl --fail http://127.0.0.1:8080/health
|
||||
curl --fail http://127.0.0.1:8787/health
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi doctor
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi test
|
||||
```
|
||||
|
||||
Native PowerShell must call `curl.exe` explicitly; Windows PowerShell may otherwise resolve `curl`
|
||||
to `Invoke-WebRequest`:
|
||||
|
||||
```powershell
|
||||
curl.exe --fail --silent --show-error http://127.0.0.1:8080/health
|
||||
curl.exe --fail --silent --show-error http://127.0.0.1:8787/health
|
||||
& $THT_BIN --installation $INSTALLATION pi doctor
|
||||
& $THT_BIN --installation $INSTALLATION pi test
|
||||
```
|
||||
|
||||
Open <http://127.0.0.1:8080>. If a check fails, run `tht ... logs` or `pi logs`; these are
|
||||
bounded and sanitize declared secrets. Do not publish either loopback port.
|
||||
|
||||
## Update an installation
|
||||
|
||||
Commit or back up local operator changes first and finish active sessions. A promoted Pi image is
|
||||
selected by the durable, installation-specific `current-image.yaml` after every base/profile file.
|
||||
Therefore rebuilding `thothii-core:local` followed by `update --check-only` does not reconcile a
|
||||
previous `pi update`: the old promoted core would remain selected.
|
||||
|
||||
Do not delete or edit the selector. `tht status` is the installation-aware selector test. If
|
||||
the running core image is the base `thothii-core:local` image, no Pi update has promoted a durable
|
||||
lifecycle image and an ordinary same-Pi-version source rebuild/start is supported. If status shows
|
||||
a lifecycle image and the pulled Pi pin is unchanged, `pi update` would be a no-op and the procedure
|
||||
must stop. A changed Pi pin uses transactional `pi update --source build` in either case.
|
||||
|
||||
macOS, Linux, and WSL2:
|
||||
|
||||
```sh
|
||||
set -euo pipefail
|
||||
|
||||
abort_update() { printf 'Source update stopped: %s\n' "$1" >&2; exit 1; }
|
||||
require_clean_source() {
|
||||
local source_state
|
||||
if ! source_state="$(git status --porcelain --untracked-files=all)"; then
|
||||
abort_update "git status failed"
|
||||
fi
|
||||
[[ -z "$source_state" ]] || abort_update "commit, remove, or back up every tracked/untracked source change"
|
||||
}
|
||||
|
||||
require_clean_source
|
||||
if ! git pull --ff-only; then abort_update "git pull --ff-only failed"; fi
|
||||
require_clean_source
|
||||
if ! git config --local core.autocrlf false; then abort_update "could not set repository LF policy"; fi
|
||||
if ! bash scripts/verify-line-endings.sh; then abort_update "the pulled checkout contains CRLF files"; fi
|
||||
if ! SOURCE_REVISION="$(git rev-parse HEAD)"; then abort_update "could not record the pulled revision"; fi
|
||||
if ! NEXT_PI_VERSION="$(sed -n 's/^ARG PI_VERSION=//p' docker/core.Dockerfile)"; then
|
||||
abort_update "could not read the pulled Pi pin"
|
||||
fi
|
||||
[[ -n "$NEXT_PI_VERSION" && "$NEXT_PI_VERSION" != *$'\n'* ]] || abort_update "expected one pinned default PI_VERSION"
|
||||
if ! INSTALLATION_STATUS="$("$THT_BIN" --installation "$INSTALLATION" status)"; then
|
||||
abort_update "tht status failed"
|
||||
fi
|
||||
if ! RUNNING_PI_VERSION="$("$THT_BIN" --installation "$INSTALLATION" pi status)"; then
|
||||
abort_update "tht pi status failed"
|
||||
fi
|
||||
RUNNING_PI_VERSION="${RUNNING_PI_VERSION#Pi version: }"
|
||||
[[ -n "$RUNNING_PI_VERSION" ]] || abort_update "tht pi status returned no version"
|
||||
|
||||
COMPACT_STATUS="${INSTALLATION_STATUS//[[:space:]]/}"
|
||||
USES_BASE_CORE=false
|
||||
if [[ "$COMPACT_STATUS" == *'"Image":"thothii-core:local"'* ]]; then
|
||||
USES_BASE_CORE=true
|
||||
fi
|
||||
TRANSACTIONAL_PI_UPDATE=true
|
||||
if [[ "$NEXT_PI_VERSION" == "$RUNNING_PI_VERSION" ]]; then
|
||||
[[ "$USES_BASE_CORE" == true ]] || abort_update "same Pi version is selected by a durable lifecycle image"
|
||||
TRANSACTIONAL_PI_UPDATE=false
|
||||
fi
|
||||
|
||||
if ! bash scripts/build-local.sh; then abort_update "the local image build failed"; fi
|
||||
if ! bash scripts/build-tht.sh; then abort_update "the tht build failed"; fi
|
||||
if ! "$THT_BIN" --installation "$INSTALLATION" update --check-only; then
|
||||
abort_update "the installation render check failed"
|
||||
fi
|
||||
if [[ "$TRANSACTIONAL_PI_UPDATE" == true ]]; then
|
||||
if ! "$THT_BIN" --installation "$INSTALLATION" pi update \
|
||||
--version "$NEXT_PI_VERSION" --source build --yes --drain; then
|
||||
abort_update "the transactional core update failed"
|
||||
fi
|
||||
fi
|
||||
if ! "$THT_BIN" --installation "$INSTALLATION" start; then abort_update "installation start failed"; fi
|
||||
if ! curl --fail http://127.0.0.1:8080/health; then abort_update "frontend health check failed"; fi
|
||||
if ! curl --fail http://127.0.0.1:8787/health; then abort_update "core health check failed"; fi
|
||||
if ! FINAL_STATUS="$("$THT_BIN" --installation "$INSTALLATION" status)"; then abort_update "final status failed"; fi
|
||||
if ! FINAL_PI_STATUS="$("$THT_BIN" --installation "$INSTALLATION" pi status)"; then abort_update "final pi status failed"; fi
|
||||
[[ "${FINAL_PI_STATUS#Pi version: }" == "$NEXT_PI_VERSION" ]] || abort_update "running Pi version does not match the pulled pin"
|
||||
if ! "$THT_BIN" --installation "$INSTALLATION" doctor; then abort_update "final doctor failed"; fi
|
||||
require_clean_source
|
||||
printf 'Built source revision: %s\n%s\n%s\n' "$SOURCE_REVISION" "$FINAL_STATUS" "$FINAL_PI_STATUS"
|
||||
```
|
||||
|
||||
Native Windows PowerShell uses the same fail-closed version comparison and transactional promotion:
|
||||
|
||||
```powershell
|
||||
$ErrorActionPreference = 'Stop'
|
||||
function Assert-NativeSuccess([string]$Step) {
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Step failed with exit code $LASTEXITCODE." }
|
||||
}
|
||||
function Assert-CleanSource {
|
||||
$SourceState = @(git status --porcelain --untracked-files=all)
|
||||
Assert-NativeSuccess 'git status'
|
||||
if ($SourceState.Count -ne 0) {
|
||||
throw 'Commit, remove, or back up every tracked/untracked source change.'
|
||||
}
|
||||
}
|
||||
|
||||
Assert-CleanSource
|
||||
git pull --ff-only
|
||||
Assert-NativeSuccess 'source pull'
|
||||
Assert-CleanSource
|
||||
git config --local core.autocrlf false
|
||||
Assert-NativeSuccess 'repository LF policy'
|
||||
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh
|
||||
Assert-NativeSuccess 'pulled checkout LF verification'
|
||||
$SourceRevision = git rev-parse HEAD
|
||||
Assert-NativeSuccess 'source revision read'
|
||||
$VersionLine = @(Select-String -Path docker/core.Dockerfile -Pattern '^ARG PI_VERSION=(.+)$')
|
||||
if ($VersionLine.Count -ne 1) { throw 'Expected exactly one pinned default PI_VERSION.' }
|
||||
$NextPiVersion = $VersionLine.Matches[0].Groups[1].Value
|
||||
$InstallationStatus = @(& $THT_BIN --installation $INSTALLATION status)
|
||||
Assert-NativeSuccess 'installation status'
|
||||
$RunningPiStatus = (& $THT_BIN --installation $INSTALLATION pi status)
|
||||
Assert-NativeSuccess 'Pi status'
|
||||
$RunningPiVersion = $RunningPiStatus -replace '^Pi version:\s*', ''
|
||||
if ([string]::IsNullOrWhiteSpace($RunningPiVersion)) { throw 'Pi status returned no version.' }
|
||||
$Services = $InstallationStatus | ConvertFrom-Json
|
||||
$CoreServices = @($Services | Where-Object { $_.Service -eq 'core' })
|
||||
if ($CoreServices.Count -ne 1) { throw 'Installation status did not identify exactly one core service.' }
|
||||
$UsesBaseCore = $CoreServices[0].Image -eq 'thothii-core:local'
|
||||
$TransactionalPiUpdate = $true
|
||||
if ($NextPiVersion -eq $RunningPiVersion) {
|
||||
if (-not $UsesBaseCore) { throw 'Same Pi version is selected by a durable lifecycle image.' }
|
||||
$TransactionalPiUpdate = $false
|
||||
}
|
||||
powershell -ExecutionPolicy Bypass -File scripts/build-local.ps1
|
||||
Assert-NativeSuccess 'local image build'
|
||||
& "C:\Program Files\Git\bin\bash.exe" scripts/build-tht.sh
|
||||
Assert-NativeSuccess 'tht build'
|
||||
& $THT_BIN --installation $INSTALLATION update --check-only
|
||||
Assert-NativeSuccess 'installation render check'
|
||||
if ($TransactionalPiUpdate) {
|
||||
& $THT_BIN --installation $INSTALLATION pi update `
|
||||
--version $NextPiVersion --source build --yes --drain
|
||||
Assert-NativeSuccess 'transactional core update'
|
||||
}
|
||||
& $THT_BIN --installation $INSTALLATION start
|
||||
Assert-NativeSuccess 'installation start'
|
||||
curl.exe --fail --silent --show-error http://127.0.0.1:8080/health
|
||||
Assert-NativeSuccess 'frontend health check'
|
||||
curl.exe --fail --silent --show-error http://127.0.0.1:8787/health
|
||||
Assert-NativeSuccess 'core health check'
|
||||
$FinalStatus = @(& $THT_BIN --installation $INSTALLATION status)
|
||||
Assert-NativeSuccess 'final installation status'
|
||||
$FinalPiStatus = (& $THT_BIN --installation $INSTALLATION pi status)
|
||||
Assert-NativeSuccess 'final Pi status'
|
||||
if (($FinalPiStatus -replace '^Pi version:\s*', '') -ne $NextPiVersion) {
|
||||
throw 'Running Pi version does not match the pulled pin.'
|
||||
}
|
||||
& $THT_BIN --installation $INSTALLATION doctor
|
||||
Assert-NativeSuccess 'final doctor'
|
||||
Assert-CleanSource
|
||||
Write-Output "Built source revision: $SourceRevision"
|
||||
Write-Output $FinalStatus
|
||||
Write-Output $FinalPiStatus
|
||||
```
|
||||
|
||||
The revision is printed only after every source/build/start/health/installation-aware check passes
|
||||
and a final porcelain check still reports no tracked or untracked source changes. For a changed Pi
|
||||
pin, status reports the promoted lifecycle candidate; for a same-version installation with no
|
||||
selector, status reports the rebuilt base core. `update --check-only` alone proves only that Compose
|
||||
renders.
|
||||
Review release notes before updating. See [Pi management](pi-management.md) for rollback; never
|
||||
install a package in the running container.
|
||||
|
||||
## Back up and restore
|
||||
|
||||
Back up before source/Pi updates and test restoration periodically. First stop cleanly:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" stop
|
||||
docker volume ls --format '{{.Name}}' | grep '^thothii-'
|
||||
```
|
||||
|
||||
Identify the four exact volumes belonging to this installation: `settings`, `pi-state`,
|
||||
`workspace-registry`, and `sessions`. Confirm their Compose project label with `docker volume
|
||||
inspect`. For each exact volume, archive it to a protected backup directory:
|
||||
|
||||
```sh
|
||||
BACKUP_DIR=/absolute/path/to/backups/2026-08-05
|
||||
VOLUME=exact-installation-volume-name
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
docker run --rm -v "$VOLUME:/source:ro" -v "$BACKUP_DIR:/backup" \
|
||||
alpine:3.22 tar -C /source -czf "/backup/$VOLUME.tgz" .
|
||||
```
|
||||
|
||||
Native PowerShell can run the same read-only archive container:
|
||||
|
||||
```powershell
|
||||
$BackupDir = 'C:\Users\operator\thothii-backups\2026-08-05'
|
||||
$Volume = 'exact-installation-volume-name'
|
||||
New-Item -ItemType Directory -Force $BackupDir | Out-Null
|
||||
docker run --rm -v "${Volume}:/source:ro" -v "${BackupDir}:/backup" `
|
||||
alpine:3.22 tar -C /source -czf "/backup/${Volume}.tgz" .
|
||||
```
|
||||
|
||||
Also back up the installation descriptor, operator environment, generated overrides, and secret
|
||||
files to separate encrypted/protected storage. Never commit them. Record image digests and the Git
|
||||
revision. Do not back up while containers are running.
|
||||
|
||||
Restore only while stopped and only into a new, verified-empty exact target volume. Test the
|
||||
archive in a disposable installation first:
|
||||
|
||||
```sh
|
||||
TARGET_VOLUME=exact-empty-target-volume-name
|
||||
ARCHIVE=/absolute/path/to/backups/2026-08-05/exact-volume-name.tgz
|
||||
docker run --rm -v "$TARGET_VOLUME:/target" alpine:3.22 \
|
||||
sh -c 'test -z "$(ls -A /target)"'
|
||||
docker run --rm -v "$TARGET_VOLUME:/target" -v "$(dirname "$ARCHIVE"):/backup:ro" \
|
||||
alpine:3.22 tar -C /target -xzf "/backup/$(basename "$ARCHIVE")"
|
||||
```
|
||||
|
||||
Native PowerShell uses `Split-Path` to produce the read-only archive mount and archive name:
|
||||
|
||||
```powershell
|
||||
$TargetVolume = 'exact-empty-target-volume-name'
|
||||
$Archive = 'C:\Users\operator\thothii-backups\2026-08-05\exact-volume-name.tgz'
|
||||
$ArchiveDir = Split-Path -Parent $Archive
|
||||
$ArchiveName = Split-Path -Leaf $Archive
|
||||
docker run --rm -v "${TargetVolume}:/target" alpine:3.22 `
|
||||
sh -ceu 'test -z "$(ls -A /target)"'
|
||||
if ($LASTEXITCODE -ne 0) { throw 'The restore target volume is not empty.' }
|
||||
docker run --rm -v "${TargetVolume}:/target" -v "${ArchiveDir}:/backup:ro" `
|
||||
alpine:3.22 tar -C /target -xzf "/backup/${ArchiveName}"
|
||||
if ($LASTEXITCODE -ne 0) { throw 'The volume restore failed.' }
|
||||
```
|
||||
|
||||
Restore all four volumes from the same backup set, restore protected operator files separately,
|
||||
then run `update --check-only`, `start`, `doctor`, registry status/diagnostics, and a known session
|
||||
before normal use. Never merge an archive into a non-empty volume.
|
||||
|
||||
## Data-preserving uninstall
|
||||
|
||||
Run `tht stop`, retain the installation descriptor at the same absolute path, and make one
|
||||
verified backup set. In Docker Desktop, remove only this installation's stopped `core` and
|
||||
`frontend` containers and optional local images; leave its four named volumes. On Linux, use the
|
||||
containers' exact Compose project labels to remove only those stopped containers. Do not prune
|
||||
global Docker data.
|
||||
|
||||
Do **not** run `docker compose down --volumes`: it deletes the application data this procedure is
|
||||
meant to preserve. Keep the operator directory and protected secrets if you intend to reinstall.
|
||||
Using the same descriptor path preserves the `tht` project identity and reconnects the same
|
||||
named volumes after rebuilding the source checkout.
|
||||
|
||||
## Next: workspaces and Pi
|
||||
|
||||
Complete [local workspace-registry installation](local-workspace-registry.md), including Git trust,
|
||||
bindings, pull, validation, diagnostics, and registry recovery. Then use [Pi management](pi-management.md)
|
||||
for provider/model configuration, smoke testing, transactional update, and rollback.
|
||||
|
||||
The Git-backed workspace registry is always the workspace source of truth. Local DWH, vector,
|
||||
embedding, or LLM processes remain independent services and are never added to the mandatory
|
||||
ThothII core.
|
||||
@@ -1,162 +0,0 @@
|
||||
# Pi management
|
||||
|
||||
ThothII bundles Pi in the `core` image. Operators use the Pi Management page for safe application
|
||||
defaults and the host-side `tht` CLI for lifecycle work. A local Pi installation is not
|
||||
required.
|
||||
|
||||
Run these commands from the root of the current ThothII checkout or worktree. `tht` discovers
|
||||
the valid installation descriptor in that project tree, so it uses the `deploy/` files belonging to
|
||||
the checkout from which you run it. Do not use `~/bin`: `~` is the user home directory, not the
|
||||
project root.
|
||||
|
||||
```sh
|
||||
THT_BIN=tht
|
||||
tht version --json
|
||||
```
|
||||
|
||||
If `tht` is not on `PATH`, install the native host CLI using the installation procedure in
|
||||
`local.md` or `server.md`, then set `THT_BIN` to that installed binary. For an installation
|
||||
stored elsewhere, set `THOTHII_INSTALLATION` or pass
|
||||
`--installation <absolute-path>/thothii-installation.yaml` explicitly.
|
||||
|
||||
## Choose application defaults
|
||||
|
||||
Use the **Pi Management** page to select the supported provider, model, and reasoning default, then
|
||||
choose **Save defaults**. The page shows credentials only as present or missing and can run bounded
|
||||
diagnostics; it never accepts or displays a credential, opens a terminal, or updates an image.
|
||||
|
||||
Alternatively, use the CLI from an administrator terminal:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi configure
|
||||
"$THT_BIN" pi configure --provider zai --model glm-5.2 --thinking medium
|
||||
```
|
||||
|
||||
Use GUI Save defaults or CLI `pi configure`, not both for the same change. The CLI's interactive
|
||||
choices are restricted to supported models; non-interactive use must supply all three values. Both
|
||||
methods store application defaults in backend installation settings, not in the project policy file.
|
||||
|
||||
Useful read-only checks are:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi status
|
||||
"$THT_BIN" pi doctor
|
||||
"$THT_BIN" pi test
|
||||
"$THT_BIN" pi check
|
||||
"$THT_BIN" pi logs
|
||||
```
|
||||
|
||||
`pi check` is an alias for `pi test`; logs are a sanitized, bounded snapshot with no follow mode.
|
||||
|
||||
## Edit the provider catalog and enabled-model policy
|
||||
|
||||
Edit these project-root files in source control, then review and deploy the change through the
|
||||
normal project process:
|
||||
|
||||
- `deploy/pi/models.json` is the provider catalog: provider endpoints and the models each provider
|
||||
offers.
|
||||
- `deploy/pi/settings.json` is the enabled-model policy only; it lists the models available to the
|
||||
application and does not store application defaults.
|
||||
|
||||
These files contain configuration, not credentials. Keep provider configuration declarative: Pi
|
||||
management rejects executable `!command` values. Docker Compose mounts the selected configuration
|
||||
and credential files read-only.
|
||||
|
||||
## Store provider credentials
|
||||
|
||||
`PI_AUTH_FILE` is a setting in the installation environment file (for example,
|
||||
`deploy/env/local.env`). Its value is the absolute path of the protected host credential file that
|
||||
this installation selects. Docker Compose mounts that selected file read-only for Pi.
|
||||
Other declared protected material is likewise mounted read-only under `/run/secrets`.
|
||||
|
||||
Set restrictive permissions on the host file (`0600` on macOS/Linux or a user-only ACL on Windows).
|
||||
Never put its contents in installation YAML, Git, command arguments, the browser, screenshots,
|
||||
tickets, rendered Compose output, or logs. Do not print the file while troubleshooting.
|
||||
|
||||
## Reload changed configuration
|
||||
|
||||
After changing the provider catalog, enabled-model policy, or selected credential file, reload the
|
||||
running application with one confirmed restart:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi restart --yes --drain
|
||||
```
|
||||
|
||||
`--yes` confirms that core will be recreated. Without `--drain`, restart refuses active sessions;
|
||||
with it, ThothII closes admission and waits for active sessions to finish without terminating them.
|
||||
The wait is bounded. Restart retains the exact captured running image: it does not build, pull, or
|
||||
upgrade an image. Before recreating core, it pins that image through transaction-scoped Compose
|
||||
override material so a configured tag moving during the operation cannot change the selected
|
||||
image, and Compose is explicitly told never to build or pull. It will restart only core, then
|
||||
verifies health, Pi version, settings/model smoke, non-secret rendered configuration, and
|
||||
persistence mounts before reopening admission.
|
||||
|
||||
Use `pi restart --yes` when there are already no active sessions. Do not substitute `tht stop`
|
||||
and `tht start` or raw Compose commands for this reload workflow.
|
||||
|
||||
## Update the bundled Pi version
|
||||
|
||||
`pi update` is for a new bundled Pi version; it is not a configuration reload. The simple command
|
||||
uses the single `ARG PI_VERSION=...` pin in `docker/core.Dockerfile`, builds that version, waits for
|
||||
active sessions to finish, and recreates only `core`:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi update
|
||||
```
|
||||
|
||||
To build a specific version, pass `--version`; source, confirmation, and drain are automatic for
|
||||
this normal build path:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi update --version 0.81.0
|
||||
```
|
||||
|
||||
A registry update must use an immutable digest, never a mutable tag:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi update \
|
||||
--version 0.81.0 --source pull \
|
||||
--image registry.example.invalid/thothii-core@sha256:<64-lowercase-hex-digits> \
|
||||
--yes --drain
|
||||
```
|
||||
|
||||
Update keeps new-session admission gated while it builds or pulls a candidate, recreates only
|
||||
`core`, verifies it, and promotes the image only after success. It preserves the frontend and named
|
||||
volumes.
|
||||
|
||||
## Recover a failed lifecycle operation
|
||||
|
||||
If a restart or update fails after core recreation, leave maintenance enabled and preserve the
|
||||
reported recovery state and transaction override. Do not delete `.tht`, state files,
|
||||
containers, or volumes. Inspect status and sanitized logs:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi maintenance status
|
||||
"$THT_BIN" pi status
|
||||
"$THT_BIN" pi logs
|
||||
```
|
||||
|
||||
For a failed update, restore its prior image:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi rollback --yes
|
||||
```
|
||||
|
||||
For a failed restart, use maintenance recovery instead of rollback. After repairing the reported
|
||||
Docker, disk, or configuration problem, use the same command to complete either safe recovery path:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" pi maintenance recover --yes
|
||||
"$THT_BIN" pi doctor
|
||||
"$THT_BIN" pi test
|
||||
```
|
||||
|
||||
`pi rollback --yes` restores the prior update image. `pi maintenance recover --yes` checks both
|
||||
restart and update recovery state before it can reopen admission. If either command fails, keep the
|
||||
installation gated and collect only the sanitized diagnostics.
|
||||
|
||||
## Direct support access
|
||||
|
||||
Raw Compose access is unsupported because it can bypass the installation-specific environment and
|
||||
durable image selector. For support, use the installation-aware `tht pi status`,
|
||||
`tht pi doctor`, `tht pi test`, and `tht pi logs` commands.
|
||||
@@ -1,65 +0,0 @@
|
||||
# Policlinico San Donato — setup workspace (nuova gestione)
|
||||
|
||||
Authentication acceptance is documented in the [manual authentication matrix](../testing/authentication-manual-acceptance.md).
|
||||
Use generic OIDC with Authentik as the certified group catalog, map only the exact TOT Users and
|
||||
TOT Admin groups, then run **Validate workspace source**, `tht auth check`, `tht auth check --interactive`,
|
||||
and **Test workspace connections** in that order. Browser callback E2E, native Windows execution, approved PSD
|
||||
manual identities, external L2, and the two parked restore-lock preconditions remain pending the
|
||||
Task 15/release gates.
|
||||
|
||||
Guida operativa per collegare ThothII al DWH di PSD con il nuovo sistema (registry Git + descriptor
|
||||
v3 + `tht`).
|
||||
|
||||
## Stato storico Mac/local (2026-08-13)
|
||||
|
||||
> Questo stato è storico per Mac/local; il server PSD Project A usa binding separato `postgres_direct` read-only.
|
||||
>
|
||||
> Per la rotazione della credenziale DWH, fare riferimento al [runbook PSD](../operations/psd-dwh-auth-rollout.md): non autorizza modifiche finché i due gate non sono approvati. Il ThothII PSD server resta `postgres_direct`; il Mac e i client remoti usano `rest_api` con una chiave per installazione. `postgres_direct` e `ssh_tunnel` non usano chiavi `dwh-auth`.
|
||||
|
||||
- **Repository PSD pubblicato:** `https://github.com/mptyl/tht-workspace-psd` (privato), branch
|
||||
`main`, commit `d4f9185`. Layout P1.1 già migrato e validato.
|
||||
- **Deploy key SSH** (sola lettura, senza passphrase) in
|
||||
`deploy/psd/secrets/git-ssh-key` e registrata sul repo come deploy key `thothii-psd`; il remote
|
||||
Git usato dall'installazione è `git@github.com:mptyl/tht-workspace-psd.git`.
|
||||
- **Config operatore pronta** (file reali gitignored in `deploy/psd/`): `operator.env`,
|
||||
`thothii-installation.yaml` e i secret d'installazione in `secrets/` (pi-auth, secret bundle,
|
||||
chiave SSH, known_hosts). L'API key DWH va completata nella gestione Workspace ed è conservata
|
||||
nel vault cifrato del backend. Il certificato REST è self-issued/private: ogni Mac/local senza trust equivalente deve usare `TLS_CA_FILE` e verificare il fingerprint fuori banda, come in `docs/install/dwh-auth-tls.md`.
|
||||
- **Stack avviato** (progetto `thothii-70417a3e30ea`, via `tht start`): `qdrant`, `embedding`
|
||||
(con `qwen3-embedding:0.6b`), `core`, `frontend` sani. Il registry ha **clonato e attivato**
|
||||
`psd-clinical` (stato `ready`).
|
||||
- **`tht workspace inspect --workspace psd-clinical` = OK** (identità descrittore/catalogo
|
||||
risolte); la configurazione runtime va completata e testata dalla GUI.
|
||||
- **Bloccante residuo: VPN.** `supabase-aritmolab.policlinicosandonato.it` non risolve
|
||||
(`NXDOMAIN`) → il preprocessing DWH e le sessioni live non possono ancora partire.
|
||||
|
||||
## Avvio/arresto (canonico)
|
||||
|
||||
Usare `tht` (stesso project name, quindi stessi volumi named):
|
||||
|
||||
```bash
|
||||
tht=dist/tht/tht-darwin-arm64
|
||||
"$tht" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" start
|
||||
"$tht" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" workspace inspect --workspace psd-clinical --json
|
||||
"$tht" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" stop
|
||||
```
|
||||
|
||||
> **Nota project name:** `tht` calcola un project name stabile dall'installation descriptor
|
||||
> (`thothii-<hash>`); `docker compose` "a mano" usa invece `name: thothii` dal `compose.yaml`, quindi
|
||||
> i volumi named non coinciderebbero. Perciò per lo stack si usa `tht start` (non
|
||||
> `compose-with-preflight.sh up`).
|
||||
|
||||
## Rimane: smoke live di una domanda (P8 L2)
|
||||
|
||||
Il preprocessing è già completato. Resta solo:
|
||||
|
||||
1. Aprire `http://localhost:8080` e selezionare `psd-clinical`.
|
||||
2. Creare una sessione con una domanda reale in linguaggio naturale.
|
||||
3. Seguire le 8 fasi fino al primo gate di revisione.
|
||||
|
||||
## Cosa è già stato fatto
|
||||
|
||||
- Ristrutturazione del repo PSD nel layout P1.1 + validazione locale.
|
||||
- Pubblicazione GitHub + deploy key read-only + configurazione Git d'installazione.
|
||||
- Avvio stack + attivazione registry + `tht inspect` verde.
|
||||
- **Preprocessing live completato** su PSD: DWH → FK → schema → Evidence, idempotente.
|
||||
@@ -1,98 +0,0 @@
|
||||
# Put ThothII behind Caddy
|
||||
|
||||
Choose exactly one authentication mode. Direct ThothII-managed OIDC and deprecated upstream
|
||||
authentication are mutually exclusive proxy contracts; never combine their directives.
|
||||
|
||||
## Direct ThothII-managed OIDC
|
||||
|
||||
Use this mode when `auth.yaml` has `mode: oidc`. Caddy terminates TLS and proxies every request to
|
||||
`frontend`; ThothII performs login, callback validation, session creation, and authorization.
|
||||
Caddy must not apply `forward_auth` or another external authentication gateway.
|
||||
|
||||
The public `/api/auth/oidc/login` and `/api/auth/oidc/callback` paths pass unchanged through the
|
||||
same proxy as the rest of `/api`. The configured `publicUrl` must match the browser origin.
|
||||
|
||||
```caddyfile
|
||||
thoth.example.invalid {
|
||||
reverse_proxy 127.0.0.1:8080 {
|
||||
# No URI rewrite: OIDC login and callback paths reach frontend unchanged.
|
||||
flush_interval -1
|
||||
header_up Host {host}
|
||||
header_up X-Forwarded-Proto https
|
||||
header_up X-Forwarded-Host {host}
|
||||
}
|
||||
|
||||
log {
|
||||
output file /var/log/caddy/thoth-access.log
|
||||
format json
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After reload, run Workspace Validate for static validation, `tht auth check` for live,
|
||||
non-interactive authentication diagnosis, and then Workspace Test for aggregate live validation.
|
||||
|
||||
## Deprecated upstream migration mode
|
||||
|
||||
Use this section only while the installation explicitly uses deprecated `upstream` mode. Do not
|
||||
use it with `mode: oidc` or `mode: local`. Here an external authentication gateway owns login and
|
||||
Caddy applies `forward_auth` before forwarding normalized private identity headers to `frontend`.
|
||||
|
||||
Forwarding identity headers alone does not authenticate a user. The authentication gateway returns
|
||||
2xx only after validating its own credential or session. Clear browser-supplied public and trusted
|
||||
headers before the subrequest, and map identity only from the successful auth response.
|
||||
|
||||
```caddyfile
|
||||
thoth.example.invalid {
|
||||
route {
|
||||
request_header -X-Authenticated-User
|
||||
request_header -X-Thoth-Principal-Issuer
|
||||
request_header -X-Thoth-Principal-Subject
|
||||
request_header -X-Thoth-Principal-Display-Name
|
||||
request_header -X-Thoth-Is-Admin
|
||||
request_header -X-Thoth-Trusted-Principal-Issuer
|
||||
request_header -X-Thoth-Trusted-Principal-Subject
|
||||
request_header -X-Thoth-Trusted-Principal-Display-Name
|
||||
request_header -X-Thoth-Trusted-Is-Admin
|
||||
|
||||
forward_auth auth-gateway:4180 {
|
||||
uri /verify
|
||||
copy_headers {
|
||||
X-Thoth-Principal-Issuer>X-Thoth-Trusted-Principal-Issuer
|
||||
X-Thoth-Principal-Subject>X-Thoth-Trusted-Principal-Subject
|
||||
X-Thoth-Principal-Display-Name>X-Thoth-Trusted-Principal-Display-Name
|
||||
X-Thoth-Is-Admin>X-Thoth-Trusted-Is-Admin
|
||||
}
|
||||
}
|
||||
|
||||
reverse_proxy 127.0.0.1:8080 {
|
||||
flush_interval -1
|
||||
header_up Host {host}
|
||||
header_up X-Forwarded-Proto https
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Trust boundary
|
||||
|
||||
Caddy is the only public listener and proxies only to loopback `frontend`, never directly to
|
||||
`core`. Configure access logs to omit cookies, authorization data, query strings, and identity
|
||||
headers. Keep Caddy keys and state outside ThothII source and operator directories.
|
||||
|
||||
## Validate and reload
|
||||
|
||||
Keep the public firewall closed while validating:
|
||||
|
||||
```sh
|
||||
curl --fail http://127.0.0.1:8080/health
|
||||
caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
|
||||
sudo systemctl reload caddy
|
||||
```
|
||||
|
||||
## Test authentication and SSE
|
||||
|
||||
For direct OIDC, verify the login path redirects to the configured provider, the callback reaches
|
||||
ThothII unchanged, forged identity headers grant nothing, and SSE is unbuffered. For deprecated
|
||||
upstream mode, additionally verify the external gateway rejects unauthenticated traffic and only
|
||||
its 2xx response can create trusted identity headers.
|
||||
@@ -1,143 +0,0 @@
|
||||
# Put ThothII behind Nginx
|
||||
|
||||
Choose exactly one authentication mode. Direct ThothII-managed OIDC and deprecated upstream
|
||||
authentication are mutually exclusive proxy contracts; never combine their locations or headers.
|
||||
|
||||
## Direct ThothII-managed OIDC
|
||||
|
||||
Use this mode when `auth.yaml` has `mode: oidc`. Nginx terminates TLS and proxies every request
|
||||
to loopback `frontend`. ThothII owns OIDC login, callback validation, browser sessions, and
|
||||
authorization. No external `auth_request` or authentication gateway belongs in this server.
|
||||
|
||||
The `location /` block below has a `proxy_pass` without a replacement URI, so public
|
||||
`/api/auth/oidc/login` and `/api/auth/oidc/callback` are forwarded unchanged. The configured
|
||||
`publicUrl` must match the browser origin.
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name thoth.example.invalid;
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name thoth.example.invalid;
|
||||
|
||||
ssl_certificate /etc/nginx/tls/thoth/fullchain.pem;
|
||||
ssl_certificate_key /etc/nginx/tls/thoth/privkey.pem;
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
|
||||
location / {
|
||||
# No auth_request and no URI rewrite: ThothII receives OIDC paths unchanged.
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
proxy_set_header X-Forwarded-For $remote_addr;
|
||||
proxy_set_header Connection "";
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off;
|
||||
proxy_cache off;
|
||||
proxy_read_timeout 3600s;
|
||||
add_header X-Accel-Buffering no always;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After reload, run Workspace Validate for static validation, `tht auth check` for live,
|
||||
non-interactive authentication diagnosis, and then Workspace Test for aggregate live validation.
|
||||
|
||||
## Deprecated upstream migration mode
|
||||
|
||||
Use this section only while the installation explicitly uses deprecated `upstream` mode. Do not
|
||||
use it with `mode: oidc` or `mode: local`. In this mode an external authentication gateway owns
|
||||
login, and Nginx applies `auth_request` before forwarding normalized private identity headers.
|
||||
|
||||
Forwarding identity headers alone does not authenticate a user. The authentication gateway returns
|
||||
2xx only after validating its own credential or session.
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name thoth.example.invalid;
|
||||
|
||||
ssl_certificate /etc/nginx/tls/thoth/fullchain.pem;
|
||||
ssl_certificate_key /etc/nginx/tls/thoth/privkey.pem;
|
||||
ssl_protocols TLSv1.2 TLSv1.3;
|
||||
|
||||
location = /_authenticate {
|
||||
internal;
|
||||
proxy_pass http://auth-gateway:4180/verify;
|
||||
proxy_pass_request_body off;
|
||||
proxy_set_header Content-Length "";
|
||||
proxy_set_header X-Original-URI $request_uri;
|
||||
proxy_set_header X-Original-Method $request_method;
|
||||
proxy_set_header X-Authenticated-User "";
|
||||
proxy_set_header X-Thoth-Principal-Issuer "";
|
||||
proxy_set_header X-Thoth-Principal-Subject "";
|
||||
proxy_set_header X-Thoth-Principal-Display-Name "";
|
||||
proxy_set_header X-Thoth-Is-Admin "";
|
||||
proxy_set_header X-Thoth-Trusted-Principal-Issuer "";
|
||||
proxy_set_header X-Thoth-Trusted-Principal-Subject "";
|
||||
proxy_set_header X-Thoth-Trusted-Principal-Display-Name "";
|
||||
proxy_set_header X-Thoth-Trusted-Is-Admin "";
|
||||
}
|
||||
|
||||
location / {
|
||||
auth_request /_authenticate;
|
||||
auth_request_set $thoth_principal_issuer
|
||||
$upstream_http_x_thoth_principal_issuer;
|
||||
auth_request_set $thoth_principal_subject
|
||||
$upstream_http_x_thoth_principal_subject;
|
||||
auth_request_set $thoth_principal_display_name
|
||||
$upstream_http_x_thoth_principal_display_name;
|
||||
auth_request_set $thoth_is_admin
|
||||
$upstream_http_x_thoth_is_admin;
|
||||
|
||||
proxy_set_header X-Authenticated-User "";
|
||||
proxy_set_header X-Thoth-Principal-Issuer "";
|
||||
proxy_set_header X-Thoth-Principal-Subject "";
|
||||
proxy_set_header X-Thoth-Principal-Display-Name "";
|
||||
proxy_set_header X-Thoth-Is-Admin "";
|
||||
proxy_set_header X-Thoth-Trusted-Principal-Issuer $thoth_principal_issuer;
|
||||
proxy_set_header X-Thoth-Trusted-Principal-Subject $thoth_principal_subject;
|
||||
proxy_set_header X-Thoth-Trusted-Principal-Display-Name $thoth_principal_display_name;
|
||||
proxy_set_header X-Thoth-Trusted-Is-Admin $thoth_is_admin;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
proxy_set_header X-Forwarded-Host $host;
|
||||
proxy_set_header X-Forwarded-For $remote_addr;
|
||||
proxy_set_header Connection "";
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
proxy_http_version 1.1;
|
||||
proxy_buffering off;
|
||||
proxy_cache off;
|
||||
proxy_read_timeout 3600s;
|
||||
add_header X-Accel-Buffering no always;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Trust boundary
|
||||
|
||||
Nginx is the only public listener and proxies only to loopback `frontend`, never directly to
|
||||
`core`. Keep private keys outside the ThothII tree. Do not log cookies, authorization headers,
|
||||
OIDC callback query values, authentication bodies, or trusted identity headers.
|
||||
|
||||
## Validate and reload
|
||||
|
||||
Keep the public firewall closed while validating:
|
||||
|
||||
```sh
|
||||
curl --fail http://127.0.0.1:8080/health
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
## Test authentication and SSE
|
||||
|
||||
For direct OIDC, verify login redirects to the configured provider, callback traffic reaches
|
||||
ThothII unchanged, forged identity headers grant nothing, and SSE is unbuffered. For deprecated
|
||||
upstream mode, additionally verify the external gateway rejects unauthenticated traffic and only
|
||||
its 2xx response can create trusted identity headers.
|
||||
@@ -1,132 +0,0 @@
|
||||
# Server workspace repository installation
|
||||
|
||||
This manual supplements [server.md](server.md). A server installation reads one remote Git
|
||||
repository hosted by GitHub, GitLab, Gitea, Bitbucket, or another Git server. ThothII fetches and
|
||||
validates complete revisions but never edits, commits, pushes, or publishes workspace source.
|
||||
|
||||
## Architecture ownership contract
|
||||
|
||||
| Component | Ownership | Operator contract |
|
||||
| --- | --- | --- |
|
||||
| DWH | External | Configure the external endpoint and complete runtime credentials through the authenticated GUI. |
|
||||
| LLM | External | Configure the external endpoint and model policy under installation control. |
|
||||
| Qdrant | Internal | Compose runs private Qdrant and persists `qdrant-data`; include it in Qdrant backup/restore. |
|
||||
| Ollama embedding | Internal | Compose runs private Ollama with `qwen3-embedding:0.6b`. |
|
||||
|
||||
## Semantic index ownership contract
|
||||
|
||||
| Scope | Ownership rule | Isolation rule |
|
||||
| --- | --- | --- |
|
||||
| Workspace semantic index | Each workspace keeps exactly one Qdrant collection reserved for itself. | Schema, Evidence, and memory records share that one collection and are separated by the `kind` payload. |
|
||||
|
||||
The fixed semantic contract is 1024 dimensions and cosine distance. DWH and LLM remain external;
|
||||
Qdrant, Ollama, and `embedding-model-init` remain private internal services.
|
||||
|
||||
## Service account, storage, and firewall
|
||||
|
||||
Run the application as the documented unprivileged service account. Keep the source checkout,
|
||||
operator files, application data, and workspace authoring clone separate:
|
||||
|
||||
```text
|
||||
/srv/thothii/app/ # ThothII source release
|
||||
/srv/thothii/operator/ # installation descriptor and protected Git files
|
||||
/srv/thothii/data/ # application data, encrypted workspace vault, sessions
|
||||
/srv/workspace-authoring/ # optional curator clone; never mounted into ThothII
|
||||
```
|
||||
|
||||
Expose only the authenticated same-origin reverse proxy. Keep `core`, Qdrant, and Ollama private.
|
||||
|
||||
## Prepare and publish a workspace source
|
||||
|
||||
Create a local workspace in the external authoring repository, which contains
|
||||
`thoth-workspaces.yaml`, one
|
||||
`<workspace-id>/workspace.yaml` per catalog entry, optional repository-owned Evidence, and optional
|
||||
curated schema annotations. It contains no credentials.
|
||||
|
||||
Publishing belongs to the curator workflow outside ThothII: validate, review, commit, and push the
|
||||
source revision to the configured protected branch. Grant the ThothII service only read access.
|
||||
|
||||
<!-- workspace-descriptor-contract:start -->
|
||||
Schema v3 is the only accepted workspace descriptor.
|
||||
Schema v1 and v2 workspace descriptors are rejected before activation.
|
||||
<!-- workspace-descriptor-contract:end -->
|
||||
|
||||
## Configure the remote Git repository
|
||||
|
||||
Copy `docs/install/examples/thothii-installation.server.yaml` to
|
||||
`/srv/thothii/operator/thothii-installation.yaml`. Set `workspaceRepository.remote`, `.branch`, and
|
||||
`.access`, then select exactly one Git transport override. The remote and credential are normally
|
||||
repository-scoped read-only deploy credentials.
|
||||
|
||||
For SSH, mount a private key and pinned known-hosts file. For HTTPS, mount a Git credentials file
|
||||
and the required CA chain. These installation credentials are not editable in Workspace
|
||||
management and are never exposed by the API.
|
||||
|
||||
## Start and update the installation
|
||||
|
||||
Use the installation-aware controller described by `server.md`:
|
||||
|
||||
```bash
|
||||
THT_BIN=/srv/thothii/operator/tht
|
||||
INSTALLATION=/srv/thothii/operator/thothii-installation.yaml
|
||||
"$THT_BIN" --installation "$INSTALLATION" start
|
||||
"$THT_BIN" --installation "$INSTALLATION" doctor
|
||||
```
|
||||
|
||||
The descriptor composes `compose.yaml`, `deploy/compose.server.yaml`, the server session-storage
|
||||
override, and one read-only Git transport override. **Update workspace repository** fetches a
|
||||
candidate on the server; it does not transfer workspace files to the operator workstation.
|
||||
|
||||
## Complete runtime secrets in Workspace management
|
||||
|
||||
### Chiavi DWH REST per installazione
|
||||
|
||||
Un'installazione server che seleziona `rest_api` usa una chiave DWH nel vault cifrato o nel file `API_KEY_FILE`; `postgres_direct` e `ssh_tunnel` non usano chiavi `dwh-auth`. Il servizio `dwh-auth` del DWH ha lifecycle `systemd` separato e non appartiene al Compose di ThothII. Vedere [enrollment client](dwh-auth-client-enrollment.md) e [guida server DWH](dwh-auth-server.md).
|
||||
|
||||
After repository activation, an authenticated user can:
|
||||
|
||||
1. Review the configured repository identity and update it without selecting a workspace.
|
||||
2. Select a workspace to see the DWH/Evidence credential fields required by its connector modes.
|
||||
3. Blind-save or rotate values with **Save entered secrets**; returned responses contain status only.
|
||||
4. Run **Validate workspace source** and then **Test workspace connections**.
|
||||
5. Use **Forget stored value** for an obsolete value after dependent sessions and jobs have ended.
|
||||
|
||||
The backend encrypts values in `/data/workspace-secrets`, including the installation-specific
|
||||
master key. The server profile persists that directory inside `THT_DATA_ROOT`; no workspace YAML
|
||||
path depends on Linux, macOS, or Windows. Plaintext exists only in a restrictive temporary file
|
||||
for the duration of a diagnostic, session, or maintenance lease.
|
||||
|
||||
Authorization is intentionally the current installation-wide authenticated-user policy. A future
|
||||
role model or external secret manager can replace that policy without changing workspace source.
|
||||
|
||||
## Validation and activation behavior
|
||||
|
||||
Repository update is all-or-nothing: ThothII fetches the configured branch, validates catalog,
|
||||
descriptors, Evidence paths, and cross-workspace invariants at one commit, then atomically activates
|
||||
the complete candidate. A rejected candidate never replaces the previous active snapshot. The
|
||||
application-owned checkout and snapshots are read-only runtime state.
|
||||
|
||||
Validation proves descriptor and repository structure. **Test workspace connections** additionally
|
||||
materializes the current runtime secrets and contacts only the selected workspace's configured
|
||||
DWH/Evidence endpoints. Failure does not modify or publish workspace source.
|
||||
|
||||
## Backup, rotation, and recovery
|
||||
|
||||
Back up application data and Qdrant consistently. Qdrant backup/restore must cover `qdrant-data`;
|
||||
application recovery must cover repository snapshots/state, sessions, settings, Pi state, and the
|
||||
entire encrypted `/data/workspace-secrets` directory. Store backup encryption keys separately and
|
||||
test restore procedures without production traffic.
|
||||
|
||||
Rotate DWH/Evidence credentials through Workspace management. Rotate Git access by atomically
|
||||
replacing its protected installation file and restarting `core`. Recover a bad source revision by
|
||||
reverting or correcting it in the external authoring repository and updating again.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Meaning and action |
|
||||
| --- | --- |
|
||||
| Git authentication failed | Verify repository-scoped read permission, branch, key/token, CA, and host-key pinning. |
|
||||
| Candidate validation failed | Correct the source repository; the prior active commit remains in service. |
|
||||
| Runtime configuration required | Select the workspace and complete all required write-only fields. |
|
||||
| Secret store unavailable | Stop writes, preserve `/data/workspace-secrets`, and restore vault plus master key together. |
|
||||
| Connection test failed | Rotate the indicated runtime credential or correct the relevant non-secret endpoint. |
|
||||
@@ -1,568 +0,0 @@
|
||||
# Install ThothII on a Linux server
|
||||
|
||||
Server authentication uses generic OIDC with the reverse proxy preserving the configured public
|
||||
origin and callback path. Follow the [OIDC guide](authentication-oidc.md), [Authentik guide](authentik.md)
|
||||
when applicable, and the [authentication acceptance matrix](../testing/authentication-manual-acceptance.md).
|
||||
The host authentication CLI is `tht`. Use `tht --installation <descriptor> workspace inspect
|
||||
--workspace <id> --json` for the active workspace snapshot, `tht --installation <descriptor>
|
||||
auth check` for live non-interactive authentication diagnosis, `auth check --interactive` for
|
||||
device-flow identity validation, and `tht ... doctor --json` for the aggregate installation gate.
|
||||
|
||||
This guide is for an installer with basic Linux administration and very basic Docker knowledge.
|
||||
It deploys the same Compose distribution used on a local PC: the mandatory application is exactly
|
||||
`frontend` plus `core`, and pinned Pi is inside `core`. The server does not need host Pi, Node.js,
|
||||
Python, Go, a browser shell, or a Docker socket inside either container.
|
||||
|
||||
Examples use `/srv/thothii` as an **example operator root**, `thoth.example.com` as a replaceable
|
||||
DNS name, and systemd-based command names. Adapt them to local policy. Complete the server session
|
||||
storage overlay and migration procedure before exposing a production installation.
|
||||
|
||||
## Deployment contract
|
||||
|
||||
- The generic Linux host and Docker Compose v2 are the deployment platform. No other
|
||||
application's Compose project, network, path, or runtime is required.
|
||||
- `frontend` is the only published service and defaults to `127.0.0.1:8080`; `core` has no host
|
||||
port. A host Nginx or Caddy listener terminates TLS and sends all application traffic to
|
||||
`frontend`, never directly to `core`.
|
||||
- DWH, vector database, embedding service, and LLM are external configurable endpoints. This
|
||||
remains true when they happen to run on the same physical server.
|
||||
- Application, Git, connector, and session credentials are protected host files mounted read-only
|
||||
under `/run/secrets`. Pi's protected auth JSON uses its dedicated read-only Pi mount. No secret
|
||||
value belongs in Git, images, browser storage, environment values, rendered Compose, or logs.
|
||||
- The Git-backed workspace registry is the source of truth. Installation-local bindings identify
|
||||
endpoints and secret-file paths; they do not replace the reviewed Git workspace descriptors.
|
||||
- `tht` is the operator CLI for start, stop, status, health, logs, Pi lifecycle, drain, and
|
||||
rollback. Raw Compose lifecycle commands bypass installation state and are unsupported.
|
||||
|
||||
Read [server workspace-registry installation](server-workspace-registry.md),
|
||||
[Pi management](pi-management.md), and the session-server comments in
|
||||
`deploy/compose.session-server.yaml.example` before the first public start.
|
||||
|
||||
## Service account and directories
|
||||
|
||||
The container runtime identity is fixed at UID/GID 10001. It does not require or permit creation of
|
||||
a matching host account or group. Keep the number unmapped and use numeric ownership only for the
|
||||
dedicated bind paths that the non-root container must read or write. If either lookup below finds a
|
||||
host identity, stop and design an explicit remapping before installation.
|
||||
|
||||
```sh
|
||||
if getent passwd 10001 >/dev/null || getent group 10001 >/dev/null; then
|
||||
printf '%s\n' 'UID/GID 10001 is already mapped; stop' >&2
|
||||
exit 1
|
||||
fi
|
||||
operator_uid="$(id -u)"
|
||||
operator_gid="$(id -g)"
|
||||
test "$operator_uid" -ne 0
|
||||
id -nG | tr ' ' '\n' | grep -Fx docker >/dev/null
|
||||
```
|
||||
|
||||
The invoking, pre-existing administrator owns source and operator files. It must already have the
|
||||
site-approved Docker access required to run `tht`; this guide never changes group membership.
|
||||
Docker-group membership is effectively host-root access and must remain limited to reviewed
|
||||
administrators. Do not grant the operator direct write access to container runtime trees.
|
||||
|
||||
Create explicit directories. `source` contains the clone; `operator` contains untracked path-only
|
||||
configuration; the three writable trees are bind-mounted into `core`; `secrets` contains regular
|
||||
files only. The parent is owned by the operator with numeric group 10001 so both the operator and
|
||||
container can traverse it. Numeric ownership does not add entries to `/etc/passwd` or `/etc/group`.
|
||||
|
||||
```sh
|
||||
operator_uid="$(id -u)"
|
||||
operator_gid="$(id -g)"
|
||||
sudo install -d -o "$operator_uid" -g 10001 -m 0750 /srv/thothii
|
||||
sudo install -d -o "$operator_uid" -g "$operator_gid" -m 0750 /srv/thothii/source
|
||||
sudo install -d -o "$operator_uid" -g "$operator_gid" -m 0750 /srv/thothii/operator
|
||||
sudo install -d -o 10001 -g "$operator_gid" -m 0750 /srv/thothii/secrets
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/data
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/pi-state
|
||||
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/workspace-registry
|
||||
sudo install -d -o root -g root -m 0700 /srv/thothii-backups
|
||||
stat -c '%u:%g %a %n' \
|
||||
/srv/thothii /srv/thothii/source /srv/thothii/operator /srv/thothii/secrets \
|
||||
/srv/thothii/data /srv/thothii/pi-state /srv/thothii/workspace-registry \
|
||||
/srv/thothii-backups
|
||||
```
|
||||
|
||||
Expected: the parent is `operator_uid:10001 750`; source/operator are
|
||||
`operator_uid:operator_gid 750`; secrets are `10001:operator_gid 750`; the three runtime trees are
|
||||
`10001:10001 750`; backups are `0:0 700`. Re-run the empty `getent` checks after creation. Do not
|
||||
make `/srv/thothii` a shared application directory.
|
||||
|
||||
## Projected server authentication: canonical root and runtime projection
|
||||
|
||||
For a server descriptor that declares `authentication.runtimeProjection`, authentication has two
|
||||
different roots. The **canonical authentication root** (`authentication.configDirectory`) is the
|
||||
root-operated source of truth. It and its regular files are `root:root 0700/0600`. The **runtime
|
||||
projection** is a separate Linux-only tree for the container reader: its root, `generations`, and
|
||||
generation directories are `10001:10001 0700`; `CURRENT`, `manifest.json`, `auth.yaml`, and (for
|
||||
local mode) `users.yaml` are `10001:10001 0600`. The publisher assigns the numeric IDs directly;
|
||||
it does not create a host user or group for 10001.
|
||||
|
||||
The runtime projection has only `CURRENT` and `generations/<64-lowercase-hex>/`. `CURRENT` selects
|
||||
one complete immutable generation. A successful configure, user mutation, restore, or explicit
|
||||
publish first blocks `CURRENT`, then verifies a new immutable generation, then makes it ready.
|
||||
The selected generation and up to two predecessor generations are retained; no operator edits a
|
||||
generation or `CURRENT` directly. A ready projection is usable only when its canonical revision is
|
||||
equal to the current canonical authentication root. If the runtime projection is blocked, missing,
|
||||
tampered, or unequal, `start`, `update --check-only`, `auth check`, and `doctor` fail closed before
|
||||
admission or Compose lifecycle work.
|
||||
|
||||
The runtime directory must be an absolute canonical path, distinct from the canonical root, and
|
||||
must exactly equal `THT_AUTH_RUNTIME_ROOT` in the protected installation environment. The server
|
||||
profile and numeric UID/GID values are validated before any projected mutation or publication.
|
||||
|
||||
The descriptor loader adds `compose.auth-runtime-projection.yaml` automatically when
|
||||
`runtimeProjection` is present; do not list that file under `overrides`. The automatic override
|
||||
mounts the runtime projection **read-only and core-only** at `/run/thothii-auth`; the canonical
|
||||
authentication root is never mounted. No other service receives that mount or
|
||||
`THT_AUTH_RUNTIME_PROJECTION_ROOT`. The example descriptor uses
|
||||
`/srv/example/thothii/auth-runtime` only as a replaceable path and contains no credential value.
|
||||
|
||||
This source change is prepared and tested only: Project A has not been started. It does not
|
||||
authorize a raw Compose lifecycle launch, an Nginx change, legacy-stack change, or mutation of
|
||||
`/srv`. A later manual gate needs separate explicit authorization before applying any descriptor
|
||||
or runtime root to a server.
|
||||
|
||||
### Status, repair, and safe evidence
|
||||
|
||||
Use the root-operated installation command; retain only its small redacted JSON result:
|
||||
|
||||
```sh
|
||||
sudo tht --installation "$INSTALLATION" auth status --json
|
||||
```
|
||||
|
||||
`state: "ready"` and `equal: true` are required before a projected server can start. `state:
|
||||
"blocked"`, `equal: false`, or a command refusal means that the runtime projection is blocked or
|
||||
cannot be validated. Do not start the stack, inspect YAML, print an environment, or edit `CURRENT`
|
||||
or a generation. Confirm the protected canonical root is available, then republish it with:
|
||||
|
||||
```sh
|
||||
sudo tht --installation "$INSTALLATION" auth publish
|
||||
sudo tht --installation "$INSTALLATION" auth status --json
|
||||
```
|
||||
|
||||
`auth publish` reconstructs the selected immutable generation from the canonical root; it never
|
||||
uses an older runtime generation as authority. If publish fails, leave the projection blocked and
|
||||
escalate using the sanitized command result plus descriptor path and timestamp only. Do not attach
|
||||
passwords, hashes, YAML, raw environment output, `nginx -T`, or a secret-bearing diff to evidence.
|
||||
|
||||
### Authentication restore
|
||||
|
||||
An authentication-bearing restore first publishes a blocked selector, restores canonical
|
||||
authentication, and publishes a verified candidate generation before any restart. If candidate or
|
||||
recovery verification fails, the verified recovery checkpoint is republished when possible; an
|
||||
unverified result remains blocked and prevents start. A restore without authentication entries
|
||||
does not touch the runtime projection. This is in addition to the normal restore requirement that
|
||||
browser sessions and pending OIDC state are cleared.
|
||||
|
||||
## Firewall and network boundaries
|
||||
|
||||
Set `THOTH_SERVER_BIND=127.0.0.1`. Permit inbound TCP 80/443 only to the TLS proxy; port 80 should
|
||||
redirect to HTTPS. Do not open 8080 externally, and do not add a core port. If a separate proxy
|
||||
host is used, replace loopback with a private, firewalled address and allow only that proxy source.
|
||||
|
||||
Allow outbound DNS and HTTPS to the source/Git registries, plus only the configured ports for the
|
||||
Git remote, DWH, vector database, embedding service, LLM, session PostgreSQL, and any approved
|
||||
bastion. Docker's private `thothii` network carries only `frontend`↔`core` traffic. Do not attach
|
||||
the mandatory stack to another application's network.
|
||||
|
||||
After start, confirm the host listens as intended:
|
||||
|
||||
```sh
|
||||
sudo ss -lntp
|
||||
```
|
||||
|
||||
Expected public listeners are the proxy on 80/443 and the frontend on loopback 8080. There must be
|
||||
no host listener for core port 8787.
|
||||
|
||||
## Address co-resident external services
|
||||
|
||||
Endpoint values are resolved inside `core`. Therefore container 127.0.0.1 means the container
|
||||
itself, not the Linux host. Prefer real DNS names with TLS, authentication, and firewall policy,
|
||||
even for services on this physical server.
|
||||
|
||||
When DNS is unavailable for a host-published service, create an untracked override such as
|
||||
`/srv/thothii/operator/host-gateway.yaml` and add it to the installation descriptor:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
core:
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
```
|
||||
|
||||
Use `host.docker.internal` in the endpoint binding. The `host-gateway` mapping supplies routing;
|
||||
it does not bundle or trust the target service. A host service listening only on host
|
||||
`127.0.0.1` is **not reachable** through this mapping. Bind that service to the ThothII Docker
|
||||
bridge gateway address or to a dedicated private host interface—never to `0.0.0.0` merely to make
|
||||
the check pass. A stable internal DNS record routed through an authenticated private listener is
|
||||
the preferred alternative.
|
||||
|
||||
After the first bounded start attempt, copy the exact core container name from `tht status`
|
||||
into `CORE_NAME`, then derive—not guess—the network ID, Linux bridge interface, gateway, and
|
||||
subnet. Compose networks normally use `br-<first-12-network-id>`; an explicit
|
||||
`com.docker.network.bridge.name` option takes precedence:
|
||||
|
||||
```sh
|
||||
CORE_NAME=replace-with-exact-core-container-name
|
||||
NETWORK_ID=$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.NetworkID}}{{end}}' "$CORE_NAME")
|
||||
NETWORK_NAME=$(docker network inspect --format '{{.Name}}' "$NETWORK_ID")
|
||||
BRIDGE=$(docker network inspect --format '{{index .Options "com.docker.network.bridge.name"}}' "$NETWORK_ID")
|
||||
test -n "$BRIDGE" || BRIDGE="br-${NETWORK_ID%${NETWORK_ID#????????????}}"
|
||||
GATEWAY=$(docker network inspect --format '{{(index .IPAM.Config 0).Gateway}}' "$NETWORK_ID")
|
||||
SUBNET=$(docker network inspect --format '{{(index .IPAM.Config 0).Subnet}}' "$NETWORK_ID")
|
||||
printf 'network=%s bridge=%s gateway=%s subnet=%s\n' "$NETWORK_NAME" "$BRIDGE" "$GATEWAY" "$SUBNET"
|
||||
ip address show dev "$BRIDGE"
|
||||
```
|
||||
|
||||
Bind the co-resident service to `$GATEWAY`. In the host firewall `INPUT` chain, allow its exact
|
||||
TCP port only when source is `$SUBNET`, input interface is `$BRIDGE`, and destination is
|
||||
`$GATEWAY`; reject other sources to that listener and persist the rules using the distribution's
|
||||
firewall manager. Docker's `DOCKER-USER` chain governs forwarded/published traffic and does not
|
||||
replace this host-input rule. Ask the firewall administrator to implement the equivalent policy
|
||||
with nftables when iptables is not the site's source of truth.
|
||||
|
||||
For an iptables-managed host, replace the port before applying these reviewed rules; the second
|
||||
rule prevents any other interface/source from reaching that gateway listener:
|
||||
|
||||
```sh
|
||||
EXTERNAL_PORT=replace-with-exact-service-port
|
||||
sudo iptables -I INPUT 1 -i "$BRIDGE" -s "$SUBNET" -d "$GATEWAY" -p tcp --dport "$EXTERNAL_PORT" -j ACCEPT
|
||||
sudo iptables -I INPUT 2 -d "$GATEWAY" -p tcp --dport "$EXTERNAL_PORT" -j REJECT
|
||||
```
|
||||
|
||||
Confirm reachability with `tht pi test` for the configured LLM/Pi path and with the
|
||||
authenticated Workspace Diagnostics action for DWH, vector collection/embedding pairing, and
|
||||
embedding endpoints. A timeout paired with `ss -lntp`, `ip address show dev "$BRIDGE"`, and the
|
||||
firewall counters distinguishes a loopback bind from a subnet/interface rule failure. Do not add
|
||||
a shell to the browser or mount the Docker socket into core for this diagnostic.
|
||||
|
||||
Configure each boundary independently:
|
||||
|
||||
- DWH: read-only runtime identity, database/schema, verified TLS, and direct or REST endpoint.
|
||||
- Vector database: endpoint plus exact database/schema, collection, distance metric, and writer
|
||||
policy declared by the reviewed workspace.
|
||||
- Embedding service: endpoint and the collection/embedding pairing—model and dimensions must match
|
||||
the existing collection. Co-residence does not permit silently changing that pairing.
|
||||
- LLM: authenticated endpoint selected through deployment and Pi configuration.
|
||||
|
||||
Never add those services to ThothII's mandatory Compose files. Follow
|
||||
[the diagnostic protocol](../workspace-diagnostic-protocol.md) before enabling a workspace.
|
||||
|
||||
## Prepare operator files and secrets
|
||||
|
||||
Clone with LF line endings, then verify before every build:
|
||||
|
||||
```sh
|
||||
git -c core.autocrlf=false clone \
|
||||
https://github.example.invalid/your-org/ThothII.git /srv/thothii/source/ThothII
|
||||
cd /srv/thothii/source/ThothII
|
||||
git config --local core.autocrlf false
|
||||
bash scripts/verify-line-endings.sh
|
||||
sudo /srv/thothii/source/ThothII/scripts/prepare-server-pi-state.sh \
|
||||
/srv/thothii/pi-state 10001 10001
|
||||
```
|
||||
|
||||
The last command is a mandatory clean-install and restore preflight. The server profile bind-mounts
|
||||
the writable Pi-state root and then overlays protected `auth.json` plus tracked `models.json` and
|
||||
`settings.json` read-only below it. Docker requires those three hidden target files to exist under
|
||||
the host parent bind before startup. The initializer creates them atomically with UID/GID 10001,
|
||||
mode `0600`, rejects symlink roots or targets, and never overwrites existing contents. It is safe to rerun
|
||||
after restoring `pi-state`; run it before any `tht start`, Compose render/start, or Pi update.
|
||||
|
||||
Copy the path-only server environment and installation descriptor:
|
||||
|
||||
```sh
|
||||
cp deploy/env/server.env.example /srv/thothii/operator/server.env
|
||||
cp docs/install/examples/thothii-installation.server.yaml \
|
||||
/srv/thothii/operator/thothii-installation.yaml
|
||||
chmod 0600 /srv/thothii/operator/server.env \
|
||||
/srv/thothii/operator/thothii-installation.yaml
|
||||
```
|
||||
|
||||
The invoking operator owns both placeholder files; use an editor that preserves ownership and mode,
|
||||
or create replacements under `umask 0077` in the operator directory.
|
||||
Replace every placeholder with an absolute path. Use exactly one Git transport override. For
|
||||
HTTPS, replace `deploy/compose.git-ssh.yaml` with `deploy/compose.git-https.yaml`. Keep the required
|
||||
session-server overlay. Optional host-gateway or pinned
|
||||
image overrides go after them.
|
||||
|
||||
Create each installation credential (Pi/application, Git, and session storage) as an independent
|
||||
regular file in `/srv/thothii/secrets`, owned by
|
||||
UID 10001, the invoking operator's numeric primary GID, and mode `0640`. Owner access lets the UID
|
||||
10001 container read a file mounted under `/run/secrets`; group access lets the operator run `tht`. The
|
||||
operator environment records only absolute `*_FILE` or `*_SOURCE` paths for those installation
|
||||
credentials. DWH and Evidence values are entered later through Workspace management and persist
|
||||
as ciphertext under `/data/workspace-secrets`; the frontend receives no secret values. Do not
|
||||
print file contents while testing permissions.
|
||||
|
||||
```sh
|
||||
operator_gid="$(id -g)"
|
||||
sudo find /srv/thothii/secrets -type f -exec chown "10001:$operator_gid" {} +
|
||||
sudo find /srv/thothii/secrets -type f -exec chmod 0640 {} +
|
||||
sudo find /srv/thothii/secrets -type f \( ! -uid 10001 -o ! -gid "$operator_gid" -o ! -perm 0640 \) -print
|
||||
```
|
||||
|
||||
Configure the remote repository and exactly one read-only Git transport as described in
|
||||
[server workspace repository installation](server-workspace-registry.md). After startup, complete
|
||||
the selected workspace's DWH and Evidence credentials through Workspace management. Secret values
|
||||
must never be pasted into `server.env`, the installation YAML, a URL, or a shell argument.
|
||||
|
||||
## Build locally or select pinned images
|
||||
|
||||
Choose one image source. For a source build, the repository's reproducible launcher builds the
|
||||
same `core` and `frontend` images used by the local profile. It requires only Git, Docker, and
|
||||
Compose; copy the reviewed path-only server environment to the launcher's untracked input first:
|
||||
|
||||
```sh
|
||||
cd /srv/thothii/source/ThothII
|
||||
cp /srv/thothii/operator/server.env deploy/env/local.env
|
||||
bash scripts/build-local.sh
|
||||
```
|
||||
|
||||
The printed local-profile start command is not the server start command; use `tht` below.
|
||||
|
||||
Alternatively, create a reviewed untracked override with release images pinned by immutable
|
||||
digest. Mutable tags are not a production pin:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
core:
|
||||
build: !reset null
|
||||
image: registry.example.com/thothii/core@sha256:<64-lowercase-hex-digits>
|
||||
session-migrate:
|
||||
build: !reset null
|
||||
image: registry.example.com/thothii/core@sha256:<64-lowercase-hex-digits>
|
||||
frontend:
|
||||
build: !reset null
|
||||
image: registry.example.com/thothii/frontend@sha256:<64-lowercase-hex-digits>
|
||||
```
|
||||
|
||||
Add that absolute file last in `overrides`. `core` and `session-migrate` must use the exact same
|
||||
core digest; neither may retain a local build or `:local` image. Frontend uses its own exact digest.
|
||||
Both images must come from one compatible release; the core image must retain the declared Pi
|
||||
version labels checked by `tht pi doctor`. Pull access
|
||||
belongs in the host Docker credential store, not in Compose or the installation descriptor.
|
||||
|
||||
## Install tht
|
||||
|
||||
Build the operator binaries with Docker. No Go installation or Go knowledge is required:
|
||||
|
||||
```sh
|
||||
cd /srv/thothii/source/ThothII
|
||||
THT_THT_OUTPUT_DIRECTORY=/srv/thothii/operator/build-output \
|
||||
bash scripts/build-tht.sh
|
||||
operator_gid="$(id -g)"
|
||||
sudo install -o root -g "$operator_gid" -m 0750 \
|
||||
/srv/thothii/operator/build-output/tht-linux-amd64 \
|
||||
/srv/thothii/operator/tht
|
||||
```
|
||||
|
||||
The source checkout remains controlled by the invoking operator. The explicit output directory is the only build
|
||||
write boundary; the build script rejects relative or non-canonical output paths. After installation,
|
||||
remove or retain `build-output` according to the site's reviewed artifact policy.
|
||||
|
||||
Use `tht-linux-arm64` on an ARM64 server. Set these variables in the maintenance shell; do
|
||||
not source `server.env` as shell code:
|
||||
|
||||
```sh
|
||||
THT_BIN=/srv/thothii/operator/tht
|
||||
INSTALLATION=/srv/thothii/operator/thothii-installation.yaml
|
||||
"$THT_BIN" --help
|
||||
"$THT_BIN" --installation "$INSTALLATION" update --check-only
|
||||
```
|
||||
|
||||
Every operator command includes the descriptor explicitly. This preserves the installation's
|
||||
profile, overrides, project identity, and durable current-image selector. The general form is
|
||||
`tht --installation /absolute/path/thothii-installation.yaml <command>`.
|
||||
|
||||
## Start and verify readiness
|
||||
|
||||
Keep the TLS proxy stopped or firewalled during bootstrap. First stop the app, run the
|
||||
installation-aware session migration, and inspect its pristine JSON. The command activates only
|
||||
the `session-migrate` profile/service with `--no-deps --no-TTY`; it derives the migrator image from the
|
||||
selected core image after all installation overrides, so this procedure is identical for source
|
||||
and pinned modes. It exits nonzero unless both arrays are empty:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" stop
|
||||
"$THT_BIN" --installation "$INSTALLATION" sessions migrate --yes
|
||||
```
|
||||
|
||||
Successful output has this shape (the `applied` list may contain versions on first use):
|
||||
|
||||
```json
|
||||
{"applied":[],"drifted":[],"pending":[]}
|
||||
```
|
||||
|
||||
Only after seeing `"pending":[]` and `"drifted":[]`, start and verify:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" start
|
||||
"$THT_BIN" --installation "$INSTALLATION" status
|
||||
"$THT_BIN" --installation "$INSTALLATION" doctor
|
||||
curl --fail http://127.0.0.1:8080/health
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi doctor
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi test
|
||||
```
|
||||
|
||||
`/health` proves process liveness. Readiness additionally requires both healthy services, a valid
|
||||
Pi provider/model smoke, a successful Git registry pull with an active validated snapshot, valid
|
||||
workspace diagnostics, and ready session PostgreSQL. Use the authenticated Workspace Management
|
||||
page to pull and diagnose the reviewed workspace. A liveness response alone is not release
|
||||
approval.
|
||||
|
||||
After configuring the proxy, open <https://thoth.example.com> in a browser. Verify an unauthenticated
|
||||
request is denied or redirected by the real identity provider, an authorized user can load the
|
||||
same-origin UI and `/api`, an unauthorized user is denied, and an administrator alone can open Pi
|
||||
Management. Keep port 8080 inaccessible from other hosts.
|
||||
|
||||
## Configure TLS and upstream authentication
|
||||
|
||||
Choose [Nginx](reverse-proxy-nginx.md) or [Caddy](reverse-proxy-caddy.md). Both examples terminate
|
||||
TLS and proxy only to loopback `frontend`. They preserve SSE and clear client-supplied identity
|
||||
headers before authentication.
|
||||
|
||||
The authentication gateway must validate a real login/session and return normalized issuer,
|
||||
subject, display-name, and admin claims only after success. Merely forwarding those headers does
|
||||
not authenticate anyone. Do not enable `AUTH_MODE=upstream` on a listener reachable around the
|
||||
trusted proxy, and never expose `core`.
|
||||
|
||||
## Operate Pi, drain, and roll back
|
||||
|
||||
Configure only closed provider/model/reasoning choices. Credentials remain protected files:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi status
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi configure
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi doctor
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi logs
|
||||
```
|
||||
|
||||
Before an update, announce maintenance and ask users to finish active work. `--drain` closes new
|
||||
admission and waits until no active sessions remain; it does not discard sessions. Build-source
|
||||
and registry-source examples are:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi update \
|
||||
--version 0.81.0 --source build --yes --drain
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi update \
|
||||
--version 0.81.0 --source pull \
|
||||
--image registry.example.com/thothii/core@sha256:<64-lowercase-hex-digits> \
|
||||
--yes --drain
|
||||
```
|
||||
|
||||
The transaction recreates only `core`, preserves volumes, verifies health/configuration/Pi, and
|
||||
automatically attempts rollback after a post-mutation failure. For interrupted or ambiguous state:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi maintenance status
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi rollback --yes
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi maintenance recover --yes
|
||||
```
|
||||
|
||||
Leave maintenance active if rollback cannot be verified. Preserve `.tht/<installation-id>/`
|
||||
recovery state, repair the reported host/configuration issue, and rerun rollback or maintenance
|
||||
recovery. Never delete or edit `current-image.yaml` or `update-state.json` to force progress.
|
||||
|
||||
## Back up and restore
|
||||
|
||||
Back up before source, workspace, session-schema, or Pi changes. Drain work, stop the installation,
|
||||
record `git rev-parse HEAD`, image digests, and `tht status`, then archive the three bind trees
|
||||
with numeric ownership. Do not include live secrets in this ordinary archive.
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" stop
|
||||
BACKUP=/srv/thothii-backups/2026-08-05
|
||||
sudo install -d -o root -g root -m 0700 "$BACKUP"
|
||||
sudo tar --numeric-owner --xattrs --acls -C /srv/thothii -czf "$BACKUP/runtime-data.tgz" \
|
||||
data pi-state workspace-registry
|
||||
sudo sh -ceu 'cd "$1"; sha256sum runtime-data.tgz > SHA256SUMS; sha256sum --check SHA256SUMS' sh "$BACKUP"
|
||||
```
|
||||
|
||||
Back up the installation descriptor, path-only environment, generated overrides, source revision,
|
||||
and secret files to separate encrypted access-controlled storage. Database-backed production
|
||||
sessions require their own PostgreSQL-native consistent backup; the local bind tree is not a
|
||||
substitute. Test both restore paths periodically.
|
||||
|
||||
Restore only while stopped. Verify the checksum, extract first into a new empty root, inspect
|
||||
ownership and expected registry layout, then retain the old trees by renaming them before placing
|
||||
the restored set. This keeps the previous state recoverable:
|
||||
|
||||
```sh
|
||||
RESTORE=/srv/thothii-restore-2026-08-05
|
||||
sudo install -d -o root -g root -m 0700 "$RESTORE"
|
||||
sudo sh -ceu 'cd "$1"; sha256sum --check SHA256SUMS' sh /srv/thothii-backups/2026-08-05
|
||||
sudo tar --numeric-owner --xattrs --acls -C "$RESTORE" \
|
||||
-xzf /srv/thothii-backups/2026-08-05/runtime-data.tgz
|
||||
sudo test -d "$RESTORE/workspace-registry/repo"
|
||||
sudo test -d "$RESTORE/workspace-registry/snapshots"
|
||||
```
|
||||
|
||||
After placing the restored `pi-state` tree and before the first start, rerun
|
||||
`sudo /srv/thothii/source/ThothII/scripts/prepare-server-pi-state.sh /srv/thothii/pi-state 10001 10001`.
|
||||
It validates or recreates only the hidden regular mount targets; it does not alter restored Pi
|
||||
state or any protected configuration source.
|
||||
|
||||
During the reviewed restore window, move each old tree to a timestamped sibling, move the matching
|
||||
restored tree into `/srv/thothii`, restore the PostgreSQL session backup from the same recovery
|
||||
point, and keep the proxy closed. Run `update --check-only`, `start`, `doctor`, `pi test`, registry
|
||||
status, workspace diagnostics, and a known historical session before reopening traffic. Never
|
||||
merge an archive into a non-empty tree.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
Begin with bounded, sanitized installation-aware commands:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" status
|
||||
"$THT_BIN" --installation "$INSTALLATION" doctor
|
||||
"$THT_BIN" --installation "$INSTALLATION" logs
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi status
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi doctor
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi test
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi logs
|
||||
"$THT_BIN" --installation "$INSTALLATION" pi maintenance status
|
||||
```
|
||||
|
||||
Use the authenticated Workspace Management status and diagnostic actions for Git revision,
|
||||
degraded snapshot, bindings, DWH, vector, and embedding checks. Review proxy logs separately, but
|
||||
configure both proxy and log shipping to exclude cookies, authorization data, identity payloads,
|
||||
query strings, and secret values. Do not render Compose or print an environment as a diagnostic.
|
||||
|
||||
Typical boundaries are: `doctor` for Docker/Compose/LF/volume/service health; `pi doctor` for image
|
||||
and provider/model integrity; registry status for Git/snapshot health; workspace diagnostics for
|
||||
external service identity; and the proxy/identity provider for login failures.
|
||||
|
||||
## Data-preserving uninstall
|
||||
|
||||
Drain and stop through `tht`, take and verify one final backup, and disable the TLS proxy
|
||||
route. Set `THT_BACKUP_ROOT=/srv/thothii-backups` in `server.env`; the removal command verifies the
|
||||
filesystem identity of that backup root, all three bind trees, and every declared secret before
|
||||
and after removing anything.
|
||||
|
||||
First run without confirmation. It displays the exact installation project, service, container
|
||||
name, container ID, and stopped state, then exits without mutation. Check every target:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" stop
|
||||
"$THT_BIN" --installation "$INSTALLATION" remove
|
||||
```
|
||||
|
||||
If and only if both targets are the expected stopped `frontend` and `core` containers, confirm:
|
||||
|
||||
```sh
|
||||
"$THT_BIN" --installation "$INSTALLATION" remove --yes exact-core-id exact-frontend-id
|
||||
```
|
||||
|
||||
Replace both example IDs with the values from the immediately preceding dry-run. The command
|
||||
refuses confirmation if the current target set differs. The confirmed operation passes only those
|
||||
previously displayed immutable container IDs to Docker,
|
||||
uses no force or volume option, rejects running/replaced containers, and proves the preservation
|
||||
paths still identify the same filesystem objects. Keep `/srv/thothii/data`, `pi-state`,
|
||||
`workspace-registry`, `operator`, protected secrets, database backups, and the installation
|
||||
descriptor if reinstallation is possible. Do not prune global Docker data.
|
||||
|
||||
Do **not** run `docker compose down --volumes`; it deletes persistent application data. Reusing the
|
||||
same protected descriptor path preserves the `tht` installation identity and allows a later
|
||||
compatible source checkout to reconnect the retained state.
|
||||
@@ -1,250 +0,0 @@
|
||||
# Windows and WSL2 line endings
|
||||
|
||||
ThothII's containers execute shell scripts from the source checkout. Those files must stay LF,
|
||||
even when the PC normally uses CRLF. The repository's `.gitattributes` is authoritative, but a
|
||||
Windows Git setting or an old checkout can still leave incorrect bytes. Check line endings after
|
||||
every clone and pull, before building an image.
|
||||
|
||||
## Recommended WSL2 clone
|
||||
|
||||
Use Docker Desktop with WSL2 integration. Clone inside the Linux filesystem, for example under
|
||||
`/home/<user>/src`, rather than under `/mnt/c`. This avoids slow cross-filesystem builds,
|
||||
permission surprises, and Windows tools rewriting files behind WSL.
|
||||
|
||||
```sh
|
||||
mkdir -p "$HOME/src"
|
||||
cd "$HOME/src"
|
||||
git -c core.autocrlf=false clone https://github.example.invalid/your-org/ThothII.git
|
||||
cd ThothII
|
||||
git config --local core.autocrlf false
|
||||
bash scripts/verify-line-endings.sh
|
||||
```
|
||||
|
||||
Keep Docker Desktop's integration enabled for that WSL distribution. Run the Linux build scripts
|
||||
and the Linux `tht` binary from the same WSL shell.
|
||||
|
||||
## Repository-local LF policy
|
||||
|
||||
Set the option in this repository only. Do not change a company-wide or personal Git policy just
|
||||
for ThothII.
|
||||
|
||||
```sh
|
||||
git config --local core.autocrlf false
|
||||
git config --local --get core.autocrlf
|
||||
```
|
||||
|
||||
The second command must print `false`. `.gitattributes` keeps shell, YAML, Dockerfile, JSON,
|
||||
TypeScript, Python, and Markdown files at LF; PowerShell files remain CRLF.
|
||||
|
||||
For a native PowerShell clone, disable conversion during the first checkout and then store the
|
||||
repository-local setting:
|
||||
|
||||
```powershell
|
||||
git -c core.autocrlf=false clone https://github.example.invalid/your-org/ThothII.git
|
||||
Set-Location ThothII
|
||||
git config --local core.autocrlf false
|
||||
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh
|
||||
```
|
||||
|
||||
## Verify after clone or pull
|
||||
|
||||
From WSL2, Git Bash, macOS, or Linux run:
|
||||
|
||||
```sh
|
||||
bash scripts/verify-line-endings.sh
|
||||
```
|
||||
|
||||
Success exits with code 0 and prints no offending path. If it lists a file, do not build or start
|
||||
ThothII. Correct the checkout first. Native PowerShell users can invoke the same script through
|
||||
Git for Windows as shown above.
|
||||
|
||||
## Recover an existing CRLF clone
|
||||
|
||||
The safest recovery is to reclone into a new directory. First commit wanted work or copy it to a
|
||||
backup outside both clones. Then clone with conversion disabled, run the verifier, and copy back
|
||||
only reviewed changes.
|
||||
|
||||
If a reviewed working tree must be repaired in place, Git must first normalize the index, export
|
||||
that exact index to a separate repair directory, verify the exported bytes, and only then copy the
|
||||
verified tracked files over the worktree. `git add --renormalize .` alone does not change existing
|
||||
worktree bytes.
|
||||
|
||||
> **WARNING — destructive worktree rewrite.** Make a backup outside the clone or commit every
|
||||
> wanted tracked change before continuing. The copy step below overwrites tracked worktree bytes
|
||||
> from the staged index export. Stop if the staged diff does not contain exactly the wanted content;
|
||||
> untracked files are neither exported nor repaired.
|
||||
|
||||
From WSL2, Git Bash, macOS, or Linux:
|
||||
|
||||
```sh
|
||||
set -euo pipefail
|
||||
|
||||
abort_repair() { printf 'CRLF repair stopped: %s\n' "$1" >&2; exit 1; }
|
||||
validate_index_export() {
|
||||
git ls-files -s -z | while IFS= read -r -d '' entry; do
|
||||
metadata="${entry%%$'\t'*}"
|
||||
path="${entry#*$'\t'}"
|
||||
mode="${metadata%% *}"
|
||||
[[ "$path" != "$entry" ]] || exit 1
|
||||
case "$mode" in
|
||||
100644|100755) [[ -f "$REPAIR_DIR/$path" && ! -L "$REPAIR_DIR/$path" ]] || exit 1 ;;
|
||||
120000) [[ -L "$REPAIR_DIR/$path" ]] && readlink "$REPAIR_DIR/$path" >/dev/null || exit 1 ;;
|
||||
*) printf 'Unsupported Git mode %s: %s\n' "$mode" "$path" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
validate_worktree_modes() {
|
||||
git ls-files -s -z | while IFS= read -r -d '' entry; do
|
||||
metadata="${entry%%$'\t'*}"
|
||||
path="${entry#*$'\t'}"
|
||||
mode="${metadata%% *}"
|
||||
case "$mode" in
|
||||
100644|100755) [[ -f "$path" && ! -L "$path" ]] || exit 1 ;;
|
||||
120000) [[ -L "$path" ]] && readlink "$path" >/dev/null || exit 1 ;;
|
||||
*) exit 1 ;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
rewrite_index_entry() {
|
||||
local mode="$1" path="$2" target temporary_link
|
||||
case "$mode" in
|
||||
100644)
|
||||
cp "$REPAIR_DIR/$path" "$path" && chmod a-x "$path"
|
||||
;;
|
||||
100755)
|
||||
cp "$REPAIR_DIR/$path" "$path" && chmod a+x "$path"
|
||||
;;
|
||||
120000)
|
||||
target="$(readlink "$REPAIR_DIR/$path")" || return 1
|
||||
temporary_link="${path}.thoth-lf-repair-link"
|
||||
[[ ! -e "$temporary_link" && ! -L "$temporary_link" ]] || return 1
|
||||
ln -s "$target" "$temporary_link" || return 1
|
||||
rm -f "$path" || { rm -f "$temporary_link"; return 1; }
|
||||
mv "$temporary_link" "$path"
|
||||
;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
if ! git status --short; then abort_repair "git status failed"; fi
|
||||
if ! git config --local core.autocrlf false; then abort_repair "could not set repository LF policy"; fi
|
||||
if ! git add --renormalize .; then abort_repair "index renormalization failed"; fi
|
||||
if ! git diff --cached --check; then abort_repair "normalized index check failed"; fi
|
||||
if ! git diff --cached; then abort_repair "normalized index review failed"; fi
|
||||
REPAIR_DIR="$(cd .. && pwd -P)/ThothII-lf-repair"
|
||||
if [[ -e "$REPAIR_DIR" ]]; then
|
||||
abort_repair "choose a new empty LF repair directory: $REPAIR_DIR"
|
||||
fi
|
||||
if ! mkdir -p "$REPAIR_DIR"; then abort_repair "could not create LF repair directory"; fi
|
||||
REPAIR_PREFIX="$REPAIR_DIR/"
|
||||
if ! git checkout-index --all --force --prefix="$REPAIR_PREFIX"; then abort_repair "index export failed"; fi
|
||||
if ! validate_index_export; then abort_repair "index export is missing entries or Git modes"; fi
|
||||
if ! bash scripts/verify-line-endings.sh "$REPAIR_DIR"; then abort_repair "exported bytes failed LF verification"; fi
|
||||
# WARNING: destructive copy; make a backup or commit wanted changes before this command.
|
||||
if ! git ls-files -s -z | while IFS= read -r -d '' entry; do
|
||||
metadata="${entry%%$'\t'*}"
|
||||
path="${entry#*$'\t'}"
|
||||
mode="${metadata%% *}"
|
||||
rewrite_index_entry "$mode" "$path" || exit 1
|
||||
done; then
|
||||
abort_repair "tracked-file rewrite failed; do not build from this worktree"
|
||||
fi
|
||||
if ! validate_worktree_modes; then abort_repair "repaired worktree does not match Git index modes"; fi
|
||||
if ! bash scripts/verify-line-endings.sh; then abort_repair "repaired worktree failed LF verification"; fi
|
||||
if ! git diff --cached --check; then abort_repair "repaired index check failed"; fi
|
||||
```
|
||||
|
||||
Native Windows PowerShell runs the same Git operations and invokes the byte verifier through Git
|
||||
for Windows:
|
||||
|
||||
```powershell
|
||||
$ErrorActionPreference = 'Stop'
|
||||
function Assert-NativeSuccess([string]$Step) {
|
||||
if ($LASTEXITCODE -ne 0) { throw "$Step failed with exit code $LASTEXITCODE." }
|
||||
}
|
||||
function ConvertFrom-IndexEntry([string]$Entry) {
|
||||
if ($Entry -notmatch '^([0-9]{6}) [0-9a-f]+ [0-3]\t(.+)$') {
|
||||
throw "Invalid Git index entry: $Entry"
|
||||
}
|
||||
[pscustomobject]@{ Mode = $Matches[1]; Path = $Matches[2] }
|
||||
}
|
||||
|
||||
git status --short
|
||||
Assert-NativeSuccess 'git status'
|
||||
git config --local core.autocrlf false
|
||||
Assert-NativeSuccess 'repository LF policy'
|
||||
git add --renormalize .
|
||||
Assert-NativeSuccess 'index renormalization'
|
||||
git diff --cached --check
|
||||
Assert-NativeSuccess 'normalized index check'
|
||||
git diff --cached
|
||||
Assert-NativeSuccess 'normalized index review'
|
||||
$RepairDir = Join-Path (Split-Path -Parent (Get-Location).Path) 'ThothII-lf-repair'
|
||||
if (Test-Path $RepairDir) { throw 'Choose a new empty LF repair directory.' }
|
||||
New-Item -ItemType Directory -Path $RepairDir | Out-Null
|
||||
$RepairPrefix = $RepairDir.Replace('\', '/') + '/'
|
||||
git -c core.symlinks=true checkout-index --all --force --prefix=$RepairPrefix
|
||||
Assert-NativeSuccess 'index export'
|
||||
$RawIndexEntries = @(git ls-files -s)
|
||||
Assert-NativeSuccess 'index inventory'
|
||||
$IndexEntries = @($RawIndexEntries | ForEach-Object { ConvertFrom-IndexEntry $_ })
|
||||
foreach ($Entry in $IndexEntries) {
|
||||
$ExportPath = Join-Path $RepairDir $Entry.Path
|
||||
$ExportItem = Get-Item -LiteralPath $ExportPath -Force -ErrorAction Stop
|
||||
switch ($Entry.Mode) {
|
||||
{ $_ -in '100644', '100755' } {
|
||||
if ($ExportItem.LinkType -eq 'SymbolicLink') { throw "Regular export became a symlink: $($Entry.Path)" }
|
||||
}
|
||||
'120000' {
|
||||
if ($ExportItem.LinkType -ne 'SymbolicLink') { throw "Symlink export is not mode 120000: $($Entry.Path)" }
|
||||
if ([string]::IsNullOrWhiteSpace([string]$ExportItem.Target)) { throw "Symlink target is empty: $($Entry.Path)" }
|
||||
}
|
||||
default { throw "Unsupported Git mode $($Entry.Mode): $($Entry.Path)" }
|
||||
}
|
||||
}
|
||||
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh $RepairDir
|
||||
Assert-NativeSuccess 'exported byte LF verification'
|
||||
# WARNING: destructive copy; make a backup or commit wanted changes before this command.
|
||||
foreach ($Entry in $IndexEntries) {
|
||||
$ExportPath = Join-Path $RepairDir $Entry.Path
|
||||
switch ($Entry.Mode) {
|
||||
{ $_ -in '100644', '100755' } {
|
||||
Copy-Item -LiteralPath $ExportPath -Destination $Entry.Path -Force -ErrorAction Stop
|
||||
}
|
||||
'120000' {
|
||||
$LinkTarget = [string](Get-Item -LiteralPath $ExportPath -Force -ErrorAction Stop).Target
|
||||
$TemporaryLink = "$($Entry.Path).thoth-lf-repair-link"
|
||||
if (Test-Path -LiteralPath $TemporaryLink) { throw "Temporary symlink path exists: $TemporaryLink" }
|
||||
New-Item -ItemType SymbolicLink -Path $TemporaryLink -Target $LinkTarget -ErrorAction Stop | Out-Null
|
||||
Remove-Item -LiteralPath $Entry.Path -Force -ErrorAction Stop
|
||||
Move-Item -LiteralPath $TemporaryLink -Destination $Entry.Path -ErrorAction Stop
|
||||
}
|
||||
default { throw "Unsupported Git mode $($Entry.Mode): $($Entry.Path)" }
|
||||
}
|
||||
}
|
||||
foreach ($Entry in $IndexEntries) {
|
||||
$WorktreeItem = Get-Item -LiteralPath $Entry.Path -Force -ErrorAction Stop
|
||||
switch ($Entry.Mode) {
|
||||
{ $_ -in '100644', '100755' } {
|
||||
if ($WorktreeItem.LinkType -eq 'SymbolicLink') { throw "Regular worktree entry became a symlink: $($Entry.Path)" }
|
||||
}
|
||||
'120000' {
|
||||
if ($WorktreeItem.LinkType -ne 'SymbolicLink') { throw "Repaired worktree symlink is not mode 120000: $($Entry.Path)" }
|
||||
if ([string]::IsNullOrWhiteSpace([string]$WorktreeItem.Target)) { throw "Repaired symlink target is empty: $($Entry.Path)" }
|
||||
}
|
||||
default { throw "Unsupported Git mode $($Entry.Mode): $($Entry.Path)" }
|
||||
}
|
||||
}
|
||||
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh
|
||||
Assert-NativeSuccess 'repaired worktree LF verification'
|
||||
git diff --cached --check
|
||||
Assert-NativeSuccess 'repaired index check'
|
||||
```
|
||||
|
||||
The export inventory must contain every regular mode (`100644`/`100755`) and recreate every tracked
|
||||
workspace compatibility symlink (`120000`). The first verifier proves the complete
|
||||
export before any overwrite; every copy/link operation is fail-closed; the final verifier examines
|
||||
the repaired worktree bytes. On native Windows, creating symlinks requires Developer Mode or an
|
||||
elevated account; failure stops the rewrite. Review the staged diff again before committing, then
|
||||
remove the separate repair directory only after inspecting it. The procedure intentionally avoids
|
||||
`git reset --hard`; replacing the clone is easier to audit and safer for uncommitted work.
|
||||
Reference in New Issue
Block a user