From 84b233d9370f56b18e7bf1960f924c6ad50fb33f Mon Sep 17 00:00:00 2001 From: mptyl Date: Thu, 13 Aug 2026 16:41:46 +0200 Subject: [PATCH] docs: record P7 publication + live stack (pending VPN) --- PROJECT_STATE.md | 22 +++-- deploy/psd/operator.env.example | 28 ++++-- docs/install/psd-workspace-setup.md | 127 +++++++++------------------- 3 files changed, 75 insertions(+), 102 deletions(-) diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 16b623fa..3de8f974 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -7,7 +7,7 @@ > ThothII per il repository (app + CLI `thothctl`), (3) come usare l'applicazione ThothII di base > (sessioni, domande, gate). Il documento userà parole semplici ed esempi; i dettagli tecnici > resteranno nei contratti esistenti. Esempio pratico completo: Policlinico San Donato. - Last updated: 2026-08-13 (P2–P6 accepted; aggregate PASS; P7 repo restructured + validated). + Last updated: 2026-08-13 (P2–P6 accepted; P7 published + live stack, pending VPN). > Point a fresh session here ("read PROJECT_STATE.md") before substantial work. ### P3 effective configuration and `.tht-dwh` — implementation complete, automated PASS, manual PASS (2026-08-13) @@ -146,12 +146,20 @@ - **Templates:** `deploy/psd/{workspace-bindings,operator,thothii-installation}.env.example` + gitignored `secrets/`; operator checklist in `docs/install/psd-workspace-setup.md` (registered in MkDocs nav). -- **Owner-gated (needed to continue):** (1) GitHub remote URL + push credentials for the PSD repo; - (2) DWH X-API-Key (+ CA) reuse; (3) LLM provider/model confirmation (default `zai/glm-5.2` already - in the descriptor); (4) confirmation of the DWH REST ping (`POST /rpc/ping`, `X-API-Key`) for - `diagnostics.dwh_rest`. -- **Remaining:** Tasks 3–5 of the plan (installation bindings/secrets, stack start + - `qwen3-embedding:0.6b` pull, re-embedding/`preprocess run`, live P8 L2 smoke). +- **Published (2026-08-13):** private repo `https://github.com/mptyl/tht-workspace-psd` (main = + `d4f9185`), consumed via SSH deploy key `thothii-psd` (read-write, passphrase-less, generated in + `deploy/psd/secrets/`). Real operator config is wired (gitignored): `deploy/psd/operator.env`, + `workspace-bindings.env`, `thothii-installation.yaml`, `connector-secrets.yaml` + `secrets/` + (DWH X-API-Key reused from the legacy `.env`; no CA — the DWH REST is public HTTPS). +- **Stack live:** started via `thothctl start` (project `thothii-70417a3e30ea`), all services + healthy, `qwen3-embedding:0.6b` present; the registry cloned + activated `psd-clinical` + (`ready`); `thothctl workspace inspect` returns `ok` with descriptor/catalog/runtime identities. + Gotcha recorded: `thothctl` uses a per-descriptor Compose project name, so the stack must be + started with `thothctl start` (not a raw `compose-with-preflight.sh up`). +- **Blocker:** VPN/DNS — `supabase-aritmolab.policlinicosandonato.it` does not resolve (NXDOMAIN), + so DWH `preprocess run` and the live P8 L2 session are pending VPN. +- **Remaining:** activate VPN → `thothctl … workspace preprocess run --workspace psd-clinical` → + live session smoke (P8 L2). ### Final aggregate P2–P6 verification — automated PASS, manual PENDING (2026-08-13) diff --git a/deploy/psd/operator.env.example b/deploy/psd/operator.env.example index 64d00c65..3f99ede7 100644 --- a/deploy/psd/operator.env.example +++ b/deploy/psd/operator.env.example @@ -1,16 +1,30 @@ # Copia in deploy/psd/operator.env (non tracciato). Solo path non-segreti. -THT_WORKSPACE_GIT_REMOTE= +THT_WORKSPACE_GIT_REMOTE=git@github.com:mptyl/tht-workspace-psd.git THT_WORKSPACE_GIT_BRANCH=main THT_WORKSPACE_INSTALLATION_ID=psd-local THT_WORKSPACE_GIT_AUTHOR_NAME="Thoth PSD" THT_WORKSPACE_GIT_AUTHOR_EMAIL=thoth-psd@example.invalid -THT_WORKSPACE_BINDINGS_ENV_FILE=/deploy/psd/workspace-bindings.env +THT_WORKSPACE_GIT_SSH_KEY_FILE=/deploy/psd/secrets/git-ssh-key +THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/deploy/psd/secrets/git-known-hosts -# Credenziali/secrets: path HOST (mai il contenuto). +# App THT_SECRETS_FILE=/deploy/psd/secrets/thothii.secrets PI_AUTH_FILE=/deploy/psd/secrets/pi-auth.json +THT_WORKSPACE_BINDINGS_ENV_FILE=/deploy/psd/workspace-bindings.env + +# Connector secret sources (host-only paths) THT_WS_PSD_CLINICAL_DWH_API_KEY_SOURCE=/deploy/psd/secrets/psd-clinical-dwh-api-key -# Opzionale (solo con CA interna): -THT_WS_PSD_CLINICAL_DWH_TLS_CA_SOURCE=/deploy/psd/secrets/psd-clinical-dwh-ca.pem -# Git HTTPS privato: -THT_WORKSPACE_GIT_CREDENTIALS_FILE=/deploy/psd/secrets/git-credentials + +# Pi (LLM) +PI_PROVIDER=zai +PI_MODEL=glm-5.2 +PI_THINKING=medium + +# App defaults +AUTH_MODE=none +THOTH_PUBLIC_EXPOSURE=false +MAX_PI_PROCESSES=4 +THOTH_HTTP_PORT=8080 +THOTH_CORE_HTTP_PORT=8787 +THT_DB_NAME=postgres +THT_DWH_REST_URL=https://supabase-aritmolab.policlinicosandonato.it/dwh/ diff --git a/docs/install/psd-workspace-setup.md b/docs/install/psd-workspace-setup.md index c997525c..33ab07c8 100644 --- a/docs/install/psd-workspace-setup.md +++ b/docs/install/psd-workspace-setup.md @@ -1,107 +1,58 @@ # Policlinico San Donato — setup workspace (nuova gestione) Guida operativa per collegare ThothII al DWH di PSD con il nuovo sistema (registry Git + descriptor -v3 + `thothctl`). Lo stato attuale: +v3 + `thothctl`). -- il repository PSD **è già stato ristrutturato** e committato localmente in - `/Users/mp/projects/tht-workspace-psd` (layout P1.1: `thoth-workspaces.yaml` + - `psd-clinical/workspace.yaml` + `psd-clinical/evidence/` + `psd-clinical/schema/annotations.yaml`); -- la **validazione locale** del registry è già passata (descriptor v3 valido, 36 Evidence - materializzate, 42 KB di annotations sincronizzate, docs generate). +## Stato attuale (2026-08-13) -Rimangono i passi che richiedono segreti/accesso. I template pronti sono in `deploy/psd/` -(`*.example`). +- **Repository PSD pubblicato:** `https://github.com/mptyl/tht-workspace-psd` (privato), branch + `main`, commit `d4f9185`. Layout P1.1 già migrato e validato. +- **Deploy key SSH** (read-write, senza passphrase) generata in + `deploy/psd/secrets/git-ssh-key` e registrata sul repo come deploy key `thothii-psd`; il remote + Git usato dall'installazione è `git@github.com:mptyl/tht-workspace-psd.git`. +- **Config operatore pronta** (file reali gitignored in `deploy/psd/`): `operator.env`, + `workspace-bindings.env`, `thothii-installation.yaml`, `connector-secrets.yaml` e i secret + `secrets/` (API key DWH riusata, pi-auth, secret bundle, chiave SSH, known_hosts). Nessuna CA: + il DWH REST usa HTTPS pubblico (`THT_SSL_CA` era vuoto). +- **Stack avviato** (progetto `thothii-70417a3e30ea`, via `thothctl start`): `qdrant`, `embedding` + (con `qwen3-embedding:0.6b`), `core`, `frontend` sani. Il registry ha **clonato e attivato** + `psd-clinical` (stato `ready`). +- **`thothctl workspace inspect --workspace psd-clinical` = OK** (identità descrittore/catalogo/ + config risolte con i bindings DWH). +- **Bloccante residuo: VPN.** `supabase-aritmolab.policlinicosandonato.it` non risolve + (`NXDOMAIN`) → il preprocessing DWH e le sessioni live non possono ancora partire. -## 1. Pubblica il repository PSD su GitHub +## Avvio/arresto (canonico) -Crea un repository Git **privato** su GitHub e pubblicalo: +Usare `thothctl` (stesso project name, quindi stessi volumi named): ```bash -git -C /Users/mp/projects/tht-workspace-psd remote add origin -git -C /Users/mp/projects/tht-workspace-psd push -u origin main +THOTHCTL=dist/thothctl/thothctl-darwin-arm64 +"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" start +"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" workspace inspect --workspace psd-clinical --json +"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" stop ``` -Oppure fornisci URL + credenziali e lo pusho io. +> **Nota project name:** `thothctl` calcola un project name stabile dall'installation descriptor +> (`thothii-`); `docker compose` "a mano" usa invece `name: thothii` dal `compose.yaml`, quindi +> i volumi named non coinciderebbero. Perciò per lo stack si usa `thothctl start` (non +> `compose-with-preflight.sh up`). -## 2. Prepara i segreti (in `deploy/psd/secrets/`, non tracciati) - -| File | Contenuto | Riuso | -| --- | --- | --- | -| `psd-clinical-dwh-api-key` | la `X-API-Key` del DWH REST | `THT_DWH_API_KEY` esistente | -| `psd-clinical-dwh-ca.pem` | CA interna (solo se il REST la presenta) | `THT_SSL_CA` esistente | -| `pi-auth.json` | profilo Pi con le credenziali LLM (zai/deepseek/qwen) | esistente | -| `thothii.secrets` | bundle secret dell'app | esistente | -| `git-credentials` | credenziali HTTPS del remote Git privato | nuova | - -> **Nota CA**: il *preprocessing* DWH via REST supporta la CA (`TLS_CA_FILE`). Il *diagnostic* di -> readiness REST, invece, rifiuta una CA per-request privata: se il ping fallisce per la CA, va -> reso trusted a runtime oppure si adatta `diagnostics.dwh_rest` nel descriptor Git. - -## 3. Compila i file operatore (dai template) +## Completare quando la VPN è attiva ```bash -cd deploy/psd -cp workspace-bindings.env.example workspace-bindings.env -cp operator.env.example operator.env -cp thothii-installation.yaml.example thothii-installation.yaml +"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" workspace preprocess run --workspace psd-clinical --json ``` -Compila i placeholder con i valori reali (remote GitHub, path assoluti). Il bindings file per PSD è: +Se si ferma per le join (`manual_review_required`): rivedere `psd-clinical/schema/annotations.yaml` +nel repo, commit+push, poi `workspace schema accept --run --yes` e +`workspace preprocess run --resume `. -```dotenv -THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api -THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://supabase-aritmolab.policlinicosandonato.it/dwh/ -THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-clinical-dwh-api-key -``` +Poi aprire `http://localhost:8080`, selezionare `psd-clinical` e creare una sessione con una domanda. -## 4. Genera il connector override +## Cosa è già stato fatto -```bash -scripts/generate-connector-secrets-override.sh --bindings-env deploy/psd/workspace-bindings.env --operator-env deploy/psd/operator.env --output deploy/psd/connector-secrets.yaml --service workspace-maintenance --role all -``` - -## 5. Avvia lo stack (VPN attiva; internet per scaricare il modello) - -```bash -scripts/compose-with-preflight.sh --env-file deploy/psd/operator.env -f compose.yaml -f deploy/compose.local.yaml -f deploy/compose.git-https.yaml -f deploy/psd/connector-secrets.yaml up --build -d -``` - -Il primo avvio esegue `embedding-model-init`, che scarica `qwen3-embedding:0.6b` nel volume -`embedding-models` e poi sblocca `core`. - -## 6. Attiva e valida il workspace - -```bash -curl --fail --silent http://127.0.0.1:8787/health -thothctl --installation deploy/psd/thothii-installation.yaml workspace inspect --workspace psd-clinical --json -``` - -## 7. Preprocessing (DWH → FK → schema → Evidence) - -```bash -thothctl --installation deploy/psd/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --json -``` - -Se il run si ferma per la revisione delle join (`manual_review_required`): - -```bash -# il curatore rivede i candidati e pubblica psd-clinical/schema/annotations.yaml (commit+push), -# poi l'installazione fa pull, e si esegue: -thothctl --installation deploy/psd/thothii-installation.yaml workspace schema accept --workspace psd-clinical --run --yes --json -thothctl --installation deploy/psd/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --resume --json -``` - -## 8. Testa una domanda reale - -Apri l'app (`http://localhost:5173`), seleziona `psd-clinical` e crea una sessione con una domanda -in linguaggio naturale. Segui le 8 fasi e conferma ai gate fino all'SQL finale. - -## Cosa serve ancora da te - -1. **URL + credenziali del repo GitHub** del workspace PSD (passo 1). -2. **X-API-Key DWH** (+ CA, se serve) riusabili dall'ambiente attuale (passo 2). -3. Conferma del **provider/modello LLM** di default (`zai/glm-5.2` è già impostato nel descriptor). -4. Conferma dell'**endpoint ping** REST del DWH (`POST /rpc/ping` con `X-API-Key`) per il - `diagnostics.dwh_rest` del descriptor. - -Con questi quattro elementi completiamo i passi 2–4 insieme e lanciamo lo smoke live. +- Ristrutturazione del repo PSD nel layout P1.1 + validazione locale. +- Pubblicazione GitHub + deploy key + config operatore completa (bindings/secret/override). +- Avvio stack + attivazione registry + `thothctl inspect` verde. +- Restano solo: **VPN attiva** → `preprocess run` → smoke live di una domanda PSD (P8 L2).