docs: record P7 live preprocessing PASS on the real PSD DWH

This commit is contained in:
2026-08-13 20:06:23 +02:00
parent 378aa6e652
commit 55567f17f2
2 changed files with 23 additions and 15 deletions
+17 -5
View File
@@ -7,7 +7,7 @@
> ThothII per il repository (app + CLI `thothctl`), (3) come usare l'applicazione ThothII di base
> (sessioni, domande, gate). Il documento userà parole semplici ed esempi; i dettagli tecnici
> resteranno nei contratti esistenti. Esempio pratico completo: Policlinico San Donato.
Last updated: 2026-08-13 (P2–P6 accepted; P7 published + live stack, pending VPN).
Last updated: 2026-08-13 (P2–P6 accepted; P7 live preprocessing PASS on PSD).
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
### P3 effective configuration and `.tht-dwh` — implementation complete, automated PASS, manual PASS (2026-08-13)
@@ -156,10 +156,22 @@
(`ready`); `thothctl workspace inspect` returns `ok` with descriptor/catalog/runtime identities.
Gotcha recorded: `thothctl` uses a per-descriptor Compose project name, so the stack must be
started with `thothctl start` (not a raw `compose-with-preflight.sh up`).
- **Blocker:** VPN/DNS — `supabase-aritmolab.policlinicosandonato.it` does not resolve (NXDOMAIN),
so DWH `preprocess run` and the live P8 L2 session are pending VPN.
- **Remaining:** activate VPN → `thothctl … workspace preprocess run --workspace psd-clinical` →
live session smoke (P8 L2).
- **Preprocessing live (2026-08-13):** with VPN active, `thothctl workspace preprocess run
--workspace psd-clinical` **succeeded** against the real PSD DWH — DWH introspection + LSH
(163 tables / 2275 columns), FK review (no new candidates: the 42 KB curated annotations are
authoritative), schema index (2438 records) and filesystem Evidence index (36 docs / 43 chunks).
Qdrant `psd-clinical` now holds **2482 revision-scoped points** (`schema_table` 164,
`schema_column` 2275, `evidence` 43; all carry `workspace_revision`). Rerun is idempotent
(Evidence `unchanged: 36`).
- **Fixes shipped during the live run** (real-DWH scale revealed them): (1) pruned ~95 GB of orphaned
acceptance-run Docker volumes; (2) raised `workspace-maintenance` tmpfs `/tmp` 64 MiB → 1 GiB
(PSD LSH snapshot is ~105 MB); (3) `vector rebuild` now recreates the 8 keyword payload indexes
(it only created dimensions/distance); (4) Qdrant upserts are chunked (256 points/batch) — a 2438-
record schema batch exceeded Qdrant's 32 MiB JSON limit; (5) frozen Evidence metadata lists now
stay lists (`FrozenList`) instead of tuples, preserving JSON shape; (6) embedding timeout 30 s →
300 s and batch 32 → 16 for large CPU corpora.
- **Remaining:** live session smoke on `psd-clinical` (P8 L2) — create a session with a real
natural-language question and reach the first reviewer gate.
### Final aggregate P2–P6 verification — automated PASS, manual PENDING (2026-08-13)
+6 -10
View File
@@ -38,21 +38,17 @@ THOTHCTL=dist/thothctl/thothctl-darwin-arm64
> i volumi named non coinciderebbero. Perciò per lo stack si usa `thothctl start` (non
> `compose-with-preflight.sh up`).
## Completare quando la VPN è attiva
## Rimane: smoke live di una domanda (P8 L2)
```bash
"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" workspace preprocess run --workspace psd-clinical --json
```
Il preprocessing è già completato. Resta solo:
Se si ferma per le join (`manual_review_required`): rivedere `psd-clinical/schema/annotations.yaml`
nel repo, commit+push, poi `workspace schema accept --run <run-id> --yes` e
`workspace preprocess run --resume <run-id>`.
Poi aprire `http://localhost:8080`, selezionare `psd-clinical` e creare una sessione con una domanda.
1. Aprire `http://localhost:8080` e selezionare `psd-clinical`.
2. Creare una sessione con una domanda reale in linguaggio naturale.
3. Seguire le 8 fasi fino al primo gate di revisione.
## Cosa è già stato fatto
- Ristrutturazione del repo PSD nel layout P1.1 + validazione locale.
- Pubblicazione GitHub + deploy key + config operatore completa (bindings/secret/override).
- Avvio stack + attivazione registry + `thothctl inspect` verde.
- Restano solo: **VPN attiva** → `preprocess run` → smoke live di una domanda PSD (P8 L2).
- **Preprocessing live completato** su PSD: DWH → FK → schema → Evidence, idempotente.