From 3046ac34c60547ae47a27f603a22b311eca34f7b Mon Sep 17 00:00:00 2001 From: mptyl Date: Thu, 13 Aug 2026 16:14:32 +0200 Subject: [PATCH] feat: restructure PSD repo (P7) and add operator setup templates + checklist --- PROJECT_STATE.md | 21 +++- deploy/psd/.gitignore | 7 ++ deploy/psd/operator.env.example | 16 +++ deploy/psd/secrets/.gitkeep | 0 deploy/psd/thothii-installation.yaml.example | 8 ++ deploy/psd/workspace-bindings.env.example | 8 ++ docs/install/psd-workspace-setup.md | 107 +++++++++++++++++++ mkdocs.yml | 1 + 8 files changed, 167 insertions(+), 1 deletion(-) create mode 100644 deploy/psd/.gitignore create mode 100644 deploy/psd/operator.env.example create mode 100644 deploy/psd/secrets/.gitkeep create mode 100644 deploy/psd/thothii-installation.yaml.example create mode 100644 deploy/psd/workspace-bindings.env.example create mode 100644 docs/install/psd-workspace-setup.md diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 24bd1d8a..16b623fa 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -7,7 +7,7 @@ > ThothII per il repository (app + CLI `thothctl`), (3) come usare l'applicazione ThothII di base > (sessioni, domande, gate). Il documento userà parole semplici ed esempi; i dettagli tecnici > resteranno nei contratti esistenti. Esempio pratico completo: Policlinico San Donato. - Last updated: 2026-08-13 (P2–P6 accepted; aggregate PASS; user guide written). + Last updated: 2026-08-13 (P2–P6 accepted; aggregate PASS; P7 repo restructured + validated). > Point a fresh session here ("read PROJECT_STATE.md") before substantial work. ### P3 effective configuration and `.tht-dwh` — implementation complete, automated PASS, manual PASS (2026-08-13) @@ -134,6 +134,25 @@ - **Manual acceptance:** PASS (owner approval 2026-08-13) — walkthrough section P6 in `docs/testing/p2-p6-manual-verification.md`. +### P7 PSD migration — plan + repository restructured + local validation PASS; owner-gated (2026-08-13) + +- **Plan:** `docs/superpowers/plans/2026-08-13-p7-psd-migration.md`. +- **Done (autonomous):** `/Users/mp/projects/tht-workspace-psd` restructured to the P1.1 layout and + committed (`thoth-workspaces.yaml` + `psd-clinical/workspace.yaml` schema v3 + `psd-clinical/ + evidence/` 36 `.md` + `psd-clinical/schema/annotations.yaml` 42 KB); legacy runtime dirs gitignored + and the old flat `psd.yaml` retired. A local `WorkspaceRegistry.bootstrap()` against a scratch bare + clone **activated `psd-clinical`** (descriptor valid, 36 Evidence materialized + manifest, 42 KB + annotations synced, `workspace-docs` generated) with no DWH/secret access. +- **Templates:** `deploy/psd/{workspace-bindings,operator,thothii-installation}.env.example` + + gitignored `secrets/`; operator checklist in `docs/install/psd-workspace-setup.md` (registered in + MkDocs nav). +- **Owner-gated (needed to continue):** (1) GitHub remote URL + push credentials for the PSD repo; + (2) DWH X-API-Key (+ CA) reuse; (3) LLM provider/model confirmation (default `zai/glm-5.2` already + in the descriptor); (4) confirmation of the DWH REST ping (`POST /rpc/ping`, `X-API-Key`) for + `diagnostics.dwh_rest`. +- **Remaining:** Tasks 3–5 of the plan (installation bindings/secrets, stack start + + `qwen3-embedding:0.6b` pull, re-embedding/`preprocess run`, live P8 L2 smoke). + ### Final aggregate P2–P6 verification — automated PASS, manual PENDING (2026-08-13) - **Aggregate process goal:** one clean-state run exercises the complete DWH → FK → schema → diff --git a/deploy/psd/.gitignore b/deploy/psd/.gitignore new file mode 100644 index 00000000..ba5e56e0 --- /dev/null +++ b/deploy/psd/.gitignore @@ -0,0 +1,7 @@ +# File operatore reali (contengono o referenziano segreti): non tracciare. +operator.env +workspace-bindings.env +thothii-installation.yaml +connector-secrets.yaml +secrets/* +!secrets/.gitkeep diff --git a/deploy/psd/operator.env.example b/deploy/psd/operator.env.example new file mode 100644 index 00000000..64d00c65 --- /dev/null +++ b/deploy/psd/operator.env.example @@ -0,0 +1,16 @@ +# Copia in deploy/psd/operator.env (non tracciato). Solo path non-segreti. +THT_WORKSPACE_GIT_REMOTE= +THT_WORKSPACE_GIT_BRANCH=main +THT_WORKSPACE_INSTALLATION_ID=psd-local +THT_WORKSPACE_GIT_AUTHOR_NAME="Thoth PSD" +THT_WORKSPACE_GIT_AUTHOR_EMAIL=thoth-psd@example.invalid +THT_WORKSPACE_BINDINGS_ENV_FILE=/deploy/psd/workspace-bindings.env + +# Credenziali/secrets: path HOST (mai il contenuto). +THT_SECRETS_FILE=/deploy/psd/secrets/thothii.secrets +PI_AUTH_FILE=/deploy/psd/secrets/pi-auth.json +THT_WS_PSD_CLINICAL_DWH_API_KEY_SOURCE=/deploy/psd/secrets/psd-clinical-dwh-api-key +# Opzionale (solo con CA interna): +THT_WS_PSD_CLINICAL_DWH_TLS_CA_SOURCE=/deploy/psd/secrets/psd-clinical-dwh-ca.pem +# Git HTTPS privato: +THT_WORKSPACE_GIT_CREDENTIALS_FILE=/deploy/psd/secrets/git-credentials diff --git a/deploy/psd/secrets/.gitkeep b/deploy/psd/secrets/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/deploy/psd/thothii-installation.yaml.example b/deploy/psd/thothii-installation.yaml.example new file mode 100644 index 00000000..dc833d5f --- /dev/null +++ b/deploy/psd/thothii-installation.yaml.example @@ -0,0 +1,8 @@ +# Copia in deploy/psd/thothii-installation.yaml. Sostituisci i path assoluti. +# Seleziona UN solo override Git (https o ssh). +profile: local +projectDirectory: "/projects/ThothII" +envFile: "/projects/ThothII/deploy/psd/operator.env" +overrides: + - "/projects/ThothII/deploy/compose.git-https.yaml" + - "/projects/ThothII/deploy/psd/connector-secrets.yaml" diff --git a/deploy/psd/workspace-bindings.env.example b/deploy/psd/workspace-bindings.env.example new file mode 100644 index 00000000..b8754a2d --- /dev/null +++ b/deploy/psd/workspace-bindings.env.example @@ -0,0 +1,8 @@ +# Copia in un file operatore non tracciato (workspace-bindings.env). +# Contiene SOLO bindings THT_WS_* non segreti. I *_FILE sono path DI CONTENITORE +# (/run/secrets/...), popolati dal connector override generato dai *_SOURCE dell'operatore env. +THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api +THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://supabase-aritmolab.policlinicosandonato.it/dwh/ +THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-clinical-dwh-api-key +# Opzionale: solo se il DWH REST presenta una CA interna/privata. +THT_WS_PSD_CLINICAL_DWH_TLS_CA_FILE=/run/secrets/psd-clinical-dwh-ca.pem diff --git a/docs/install/psd-workspace-setup.md b/docs/install/psd-workspace-setup.md new file mode 100644 index 00000000..c997525c --- /dev/null +++ b/docs/install/psd-workspace-setup.md @@ -0,0 +1,107 @@ +# Policlinico San Donato — setup workspace (nuova gestione) + +Guida operativa per collegare ThothII al DWH di PSD con il nuovo sistema (registry Git + descriptor +v3 + `thothctl`). Lo stato attuale: + +- il repository PSD **è già stato ristrutturato** e committato localmente in + `/Users/mp/projects/tht-workspace-psd` (layout P1.1: `thoth-workspaces.yaml` + + `psd-clinical/workspace.yaml` + `psd-clinical/evidence/` + `psd-clinical/schema/annotations.yaml`); +- la **validazione locale** del registry è già passata (descriptor v3 valido, 36 Evidence + materializzate, 42 KB di annotations sincronizzate, docs generate). + +Rimangono i passi che richiedono segreti/accesso. I template pronti sono in `deploy/psd/` +(`*.example`). + +## 1. Pubblica il repository PSD su GitHub + +Crea un repository Git **privato** su GitHub e pubblicalo: + +```bash +git -C /Users/mp/projects/tht-workspace-psd remote add origin +git -C /Users/mp/projects/tht-workspace-psd push -u origin main +``` + +Oppure fornisci URL + credenziali e lo pusho io. + +## 2. Prepara i segreti (in `deploy/psd/secrets/`, non tracciati) + +| File | Contenuto | Riuso | +| --- | --- | --- | +| `psd-clinical-dwh-api-key` | la `X-API-Key` del DWH REST | `THT_DWH_API_KEY` esistente | +| `psd-clinical-dwh-ca.pem` | CA interna (solo se il REST la presenta) | `THT_SSL_CA` esistente | +| `pi-auth.json` | profilo Pi con le credenziali LLM (zai/deepseek/qwen) | esistente | +| `thothii.secrets` | bundle secret dell'app | esistente | +| `git-credentials` | credenziali HTTPS del remote Git privato | nuova | + +> **Nota CA**: il *preprocessing* DWH via REST supporta la CA (`TLS_CA_FILE`). Il *diagnostic* di +> readiness REST, invece, rifiuta una CA per-request privata: se il ping fallisce per la CA, va +> reso trusted a runtime oppure si adatta `diagnostics.dwh_rest` nel descriptor Git. + +## 3. Compila i file operatore (dai template) + +```bash +cd deploy/psd +cp workspace-bindings.env.example workspace-bindings.env +cp operator.env.example operator.env +cp thothii-installation.yaml.example thothii-installation.yaml +``` + +Compila i placeholder con i valori reali (remote GitHub, path assoluti). Il bindings file per PSD è: + +```dotenv +THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api +THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://supabase-aritmolab.policlinicosandonato.it/dwh/ +THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-clinical-dwh-api-key +``` + +## 4. Genera il connector override + +```bash +scripts/generate-connector-secrets-override.sh --bindings-env deploy/psd/workspace-bindings.env --operator-env deploy/psd/operator.env --output deploy/psd/connector-secrets.yaml --service workspace-maintenance --role all +``` + +## 5. Avvia lo stack (VPN attiva; internet per scaricare il modello) + +```bash +scripts/compose-with-preflight.sh --env-file deploy/psd/operator.env -f compose.yaml -f deploy/compose.local.yaml -f deploy/compose.git-https.yaml -f deploy/psd/connector-secrets.yaml up --build -d +``` + +Il primo avvio esegue `embedding-model-init`, che scarica `qwen3-embedding:0.6b` nel volume +`embedding-models` e poi sblocca `core`. + +## 6. Attiva e valida il workspace + +```bash +curl --fail --silent http://127.0.0.1:8787/health +thothctl --installation deploy/psd/thothii-installation.yaml workspace inspect --workspace psd-clinical --json +``` + +## 7. Preprocessing (DWH → FK → schema → Evidence) + +```bash +thothctl --installation deploy/psd/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --json +``` + +Se il run si ferma per la revisione delle join (`manual_review_required`): + +```bash +# il curatore rivede i candidati e pubblica psd-clinical/schema/annotations.yaml (commit+push), +# poi l'installazione fa pull, e si esegue: +thothctl --installation deploy/psd/thothii-installation.yaml workspace schema accept --workspace psd-clinical --run --yes --json +thothctl --installation deploy/psd/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --resume --json +``` + +## 8. Testa una domanda reale + +Apri l'app (`http://localhost:5173`), seleziona `psd-clinical` e crea una sessione con una domanda +in linguaggio naturale. Segui le 8 fasi e conferma ai gate fino all'SQL finale. + +## Cosa serve ancora da te + +1. **URL + credenziali del repo GitHub** del workspace PSD (passo 1). +2. **X-API-Key DWH** (+ CA, se serve) riusabili dall'ambiente attuale (passo 2). +3. Conferma del **provider/modello LLM** di default (`zai/glm-5.2` è già impostato nel descriptor). +4. Conferma dell'**endpoint ping** REST del DWH (`POST /rpc/ping` con `X-API-Key`) per il + `diagnostics.dwh_rest` del descriptor. + +Con questi quattro elementi completiamo i passi 2–4 insieme e lanciamo lo smoke live. diff --git a/mkdocs.yml b/mkdocs.yml index bbc64d70..1e3b3001 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -48,6 +48,7 @@ markdown_extensions: nav: - Home: index.md - Guida utente: guida-utente.md +- Setup Policlinico San Donato: install/psd-workspace-setup.md - ThothII (Documentazione Tecnica): - Panoramica Architettura: architecture/overview.md - Installazione Docker (4 contesti): installazione-docker-4-contesti.md