diff --git a/docs/prd/2026-08-09-workspace-preprocessing-prd.md b/docs/prd/2026-08-09-workspace-preprocessing-prd.md index 776516dd..60fc54db 100644 --- a/docs/prd/2026-08-09-workspace-preprocessing-prd.md +++ b/docs/prd/2026-08-09-workspace-preprocessing-prd.md @@ -9,6 +9,16 @@ revisione e conferma del proprietario --- +> **Aggiornamento P1.1 (2026-08-11):** il layout del repository registry descritto nelle sezioni +> attive di questo PRD segue il contratto P1.1 accettato: catalogo di root `thoth-workspaces.yaml`, +> descriptor `/workspace.yaml`, evidence embedded `/evidence/`, annotazioni FK curate +> `/schema/annotations.yaml` (P5), docs generate `workspace-docs//`. I vecchi percorsi +> piatti (`workspaces/.yaml`, `workspace-content//evidence/`) sono superseded; le uniche +> occorrenze rimaste sono storiche (changelog/revisioni). Vedi +> `docs/superpowers/plans/2026-08-11-p2-p6-adaptation-to-p1-1-registry.md`. + +--- + ## 1. Contesto ThothII è passato da un indice semantico **pgvector sul server PSD** (descrizioni di tabelle/colonne, @@ -103,8 +113,8 @@ documentato e verificato da smoke end-to-end. ## 5. Scenario target (end-to-end) 1. **Setup repo**: l'operatore usa un **unico repository Git registry** per tutti i workspace e crea - `workspaces/.yaml` (schema-v3: DWH, collection, LLM policy) insieme al tree curato - `workspace-content//evidence/`; descriptor e contenuti sono pubblicati nello stesso commit. + `/workspace.yaml` (schema-v3: DWH, collection, LLM policy) insieme al tree curato + `/evidence/`; descriptor e contenuti sono pubblicati nello stesso commit. 2. **Installazione**: `.env` + bindings `THT_WS_*` + secrets; `up` dello stack (frontend/core/qdrant/embedding). 3. **Registry**: pull → validazione → snapshot attivo; diagnostic DWH verdi. 4. **Preprocessing DWH**: introspezione (physical.yaml: tabelle/colonne/descrizioni/esempi/eligibility) + @@ -138,7 +148,7 @@ documentato e verificato da smoke end-to-end. direct o tunnel. - RF1.5 Il registry usa **un unico repository Git** per più workspace. Per una sorgente evidence `filesystem`, l'URI è relativa alla root del repository ed è confinata lessicalmente a - `workspace-content//`; path assoluti, traversal (`..`) e riferimenti al namespace di un + `/`; path assoluti, traversal (`..`) e riferimenti al namespace di un altro workspace sono invalidi. Descriptor e sorgente devono essere risolti dalla **stessa revisione Git**. ### RF2 — Preprocessing DWH (tabelle/colonne) @@ -172,8 +182,8 @@ documentato e verificato da smoke end-to-end. ### RF5 — Evidence - RF5.1 Sorgente evidence dichiarabile per-workspace nel descriptor (protocollo/tipo + URI). Per PSD è un **tree di file `.md` versionato nell'unico repository registry**, sotto - `workspace-content/psd/evidence/`; in generale ogni workspace usa - `workspace-content//evidence/`. HTTP manifest e S3 restano opzioni del motore per sorgenti + `psd/evidence/`; in generale ogni workspace usa + `/evidence/`. HTTP manifest e S3 restano opzioni del motore per sorgenti esterne. - RF5.2 `tht preprocess evidence` per-workspace: discover → acquire → normalize/chunk → embed → upsert (kind `evidence`, payload `document_id`/`vector_generation`) → publish ACTIVE nel corpus root del workspace, @@ -339,8 +349,8 @@ Ogni piano tecnico riporta, adattandoli al proprio scope: - Il descriptor v3 guadagna una sezione `evidence` che configura **tutta la lettura della sorgente**: protocollo/tipo, URI/sorgente, eventuali parametri non-secret. - Si usa **un unico repository Git registry** per tutti i workspace. Ogni workspace possiede il proprio tree - versionato sotto `workspace-content//evidence/`; per PSD il path canonico è - `workspace-content/psd/evidence/`. + versionato sotto `/evidence/`; per PSD il path canonico è + `psd/evidence/`. - Per `filesystem`, l'URI del descriptor è repo-relative, confinata al namespace dello stesso workspace e risolta dalla stessa revisione Git del descriptor. Sono vietati path assoluti, traversal e riferimenti al contenuto di un altro workspace; il controllo reale di symlink/containment durante la materializzazione @@ -380,7 +390,7 @@ Ogni piano tecnico riporta, adattandoli al proprio scope: ### D6 — Evidence: **a) nell'unico repository registry, con namespace per-workspace** - Ogni workspace contiene il proprio tree versionato sotto - `workspace-content//evidence/`; le dimensioni non sono un vincolo. + `/evidence/`; le dimensioni non sono un vincolo. - P6 materializza il tree dalla **stessa revisione Git** del descriptor, verifica il containment reale (inclusi i symlink) e lo rende disponibile al preprocessing senza usare un checkout mobile. - HTTP/S3 restano opzioni future per sorgenti esterne (il motore le supporta già). @@ -416,12 +426,12 @@ accettazione applicabili (sez. 9) e adotta lo standard integration-first (sez. 8 | Piano | Punto PRD | Contenuto sintetico | Dipende da | | --- | --- | --- | --- | -| P1 | D1 | Descriptor v3: sezione `evidence` (protocollo/tipo, URI repo-relative sotto `workspace-content//evidence/`) + policy e isolamento namespace; goal automatico Git→registry→HTTP→render→harness, seguito da walkthrough manuale | — | +| P1 | D1 | Descriptor v3: sezione `evidence` (protocollo/tipo, URI repo-relative sotto `/evidence/`) + policy e isolamento namespace; goal automatico Git→registry→HTTP→render→harness, seguito da walkthrough manuale | — | | P2 | D2 | **CLI di preprocessing sul host (release 0)**: comando per-workspace che esegue l'intera catena (DWH, FK, index-schema, evidence) con la config derivata da descriptor+bindings; funziona su PC/Mac utente e server DWH | P1 | | P3 | D3 | Vincolo fingerprint `.tht-dwh` (test: preprocess con config identica alla runtime) + **documentazione di progetto su cos'è `.tht-dwh`** | P2 | | P4 | D4 | Bootstrap collection: **self-heal all'ammissione** (creazione 1024/cosine + keyword-index) + **comandi CLI delete/recreate** con guardie | — | | P5 | D5 | `annotations.yaml` versionata nel repository registry + sync registry → roots runtime | P1 | -| P6 | D6 | Materializzazione del tree `workspace-content//evidence/` dalla revisione Git fissata → preprocess; containment reale e protezione da symlink escape | P1 | +| P6 | D6 | Materializzazione del tree `/evidence/` dalla revisione Git fissata → preprocess; containment reale e protezione da symlink escape | P1 | | P7 | D7 | Migrazione PSD: riuso catalogo/annotations/evidence, **export pgvector (accesso server)**, re-embedding, dry-run | P1–P6 | | P8 | D8 | Verifica end-to-end: smoke CI + gate L2 su PSD (**namespace PSD nel repository registry alimentato prima dell'uso**; remote Git a scelta) | P1–P7 | | P9 | D9 | GC/retention per-workspace: policy configurabili, default invariati | P1 | @@ -512,7 +522,7 @@ traccia separatamente implementazione, automated integration e manual acceptance | v0.1 | 2026-08-09 | Bozza da analisi dello stato attuale (gap preprocessing per-workspace) | | v0.2 | 2026-08-09 | Decisioni D1–D9 chiuse con il proprietario; mappa piani P1–P10; requisiti RF1–RF8 aggiornati (evidence nel descriptor, CLI sul host, self-heal collection, multi-trasporto DWH) | | v0.3 | 2026-08-09 | Revisione di coerenza (numerazioni, riferimenti incrociati, header di stato) — pronto per revisione del proprietario | -| v0.4 | 2026-08-09 | D1/D6: repository registry unico, namespace `workspace-content//evidence/`, pin alla stessa revisione Git e gate manuale P1 con remote locale usa-e-getta sotto `.artifacts/` | +| v0.4 | 2026-08-09 | D1/D6: repository registry unico, namespace `workspace-content//evidence/` *(percorso storico P1, superseded da P1.1)*, pin alla stessa revisione Git e gate manuale P1 con remote locale usa-e-getta sotto `.artifacts/` | | v0.5 | 2026-08-09 | Standard integration-first per P1–P10: process goal automatico completo da ambiente simulato e pulito, gestione esplicita degli interventi umani inevitabili e walkthrough manuale successivo su stato separato | --- diff --git a/docs/superpowers/plans/2026-08-10-p2-host-workspace-preprocessing-cli.md b/docs/superpowers/plans/2026-08-10-p2-host-workspace-preprocessing-cli.md index 89dab8d9..a4f872df 100644 --- a/docs/superpowers/plans/2026-08-10-p2-host-workspace-preprocessing-cli.md +++ b/docs/superpowers/plans/2026-08-10-p2-host-workspace-preprocessing-cli.md @@ -19,7 +19,7 @@ P2 is complete only when all of the following are true: 1. The only public host interface is the installed native `thothctl` binary. Docker/Compose is required, but host Python, Node, Pi, `tht`, and a running Fastify backend are not. -2. Every command consumes an already-active, validated registry snapshot and binds the exact workspace ID, 40-hex commit, descriptor blob/digest, installation bindings, runtime roots, and selected internal semantic contract before mutation. +2. Every command consumes an already-active, validated P1.1 registry snapshot and binds the exact workspace ID, 40-hex commit, catalog blob (`thoth-workspaces.yaml`), descriptor blob/digest, installation bindings, runtime roots, and selected internal semantic contract before mutation. A docs-only or content-only commit is still a distinct revision even when the descriptor blob is unchanged, because the commit is authoritative. 3. Operator and session configuration use the same `resolveRuntimeBindings` and `renderRuntimeConfig` implementation. P2 uses one deterministic same-revision config-source path so current schema-v1 DWH/Evidence resume works; P3 later introduces cross-revision canonical effective identity and explicit migrations. 4. DWH introspection+LSH, FK suggestion/check, schema indexing, and HTTP Evidence preprocessing invoke the existing harness engine through fixed argv and pristine JSON machine interfaces. No second preprocessing engine is added. 5. A full run with new FK candidates stops before schema/Evidence writes. Continuation requires a reviewer-supplied annotations file and an explicit acknowledgement of the exact candidate digest; `schema check` alone is not treated as human approval. @@ -141,7 +141,7 @@ interface WorkspaceOperationResult { - Lock order is always P2 workspace writer lock → existing harness stage lock. Harness code never acquires the P2 lock, preventing inversion/deadlock. - Every directory component is opened/validated without following symlinks. State files are `0600`, written to an exclusive sibling, fsynced, renamed, and parent-fsynced. Hardlink count must be one. - The deterministic config path fixes P2 same-revision `config_source` identity. Its manifest binds workspace, revision, descriptor blob, config SHA-256, file identity, and the current existing harness ownership binding. Same path + different bytes returns `effective_config_mismatch`; P3 introduces semantic cross-revision equivalence. -- Job state binds operation, revision, descriptor blob, config digest, non-secret binding identity, completed stage records, child run IDs, candidate/review digests, and terminal status. Resume revalidates all fields and reconciles a child publication that completed immediately before an outer-state crash. +- Job state binds operation, revision, catalog blob, descriptor blob, config digest, non-secret binding identity, completed stage records, child run IDs, candidate/review digests, and terminal status. Resume revalidates all fields and reconciles a child publication that completed immediately before an outer-state crash. - Before any schema/Evidence mutation, enumerate resumable session manifests for the workspace. A different pinned revision returns `preprocessing_conflict`; no write begins. This is the explicit P2 bridge until P3 revision isolation. ## One-shot service security contract @@ -473,7 +473,7 @@ The public command is: ``` - [ ] **Step 1: Write RED acceptance-runner tests** for ownership-first state, unique run/project/container/image names, exact cleanup, `--keep`, injected failure, signal cleanup, report bounds, and no automatic retry. -- [ ] **Step 2: Build a clean owned topology** under `.artifacts/p2-integration/p2-/`: local bare Git + author clone, active P1 snapshot, installation descriptor/env, fixture-only secrets, controlled REST DWH, controlled HTTP Evidence, real compatible Qdrant, deterministic Ollama-compatible embedding fixture, selected core image, and no backend/Pi/frontend. +- [ ] **Step 2: Build a clean owned topology** under `.artifacts/p2-integration/p2-/`: local bare Git + author clone, active P1.1 snapshot (root catalog + `/workspace.yaml` + `/evidence`), installation descriptor/env, fixture-only secrets, controlled REST DWH, controlled HTTP Evidence, real compatible Qdrant, deterministic Ollama-compatible embedding fixture, selected core image, and no backend/Pi/frontend. - [ ] **Step 3: Pre-provision the exact compatible Qdrant collection** outside the product operation and record that setup as a P4-deferred fixture step. - [ ] **Step 4: Exercise only built `thothctl` product commands** and assert: 1. exact inspect revision/config identity; diff --git a/docs/superpowers/specs/2026-08-10-p2-p6-workspace-preprocessing-design.md b/docs/superpowers/specs/2026-08-10-p2-p6-workspace-preprocessing-design.md index d026dc81..6ecd72b9 100644 --- a/docs/superpowers/specs/2026-08-10-p2-p6-workspace-preprocessing-design.md +++ b/docs/superpowers/specs/2026-08-10-p2-p6-workspace-preprocessing-design.md @@ -44,7 +44,9 @@ Every operation binds these values before doing work: - workspace ID; - exact 40-hex active Git commit; -- exact canonical descriptor snapshot; +- catalog entry (`thoth-workspaces.yaml`) and exact canonical descriptor snapshot (the catalog blob + and descriptor blob at that same commit; a docs-only or content-only commit is still a distinct + revision even when the descriptor blob is unchanged, because the commit is authoritative); - installation-local bindings resolved under configured secret roots; - runtime roots beneath `/data/sessions/`; - internal Qdrant/Ollama contract; @@ -169,7 +171,7 @@ rollback of lost vector data. The canonical path is fixed, not descriptor-configurable: ```text -workspace-content//schema/annotations.yaml +/schema/annotations.yaml ``` The registry validates that the object is a regular Git blob at the same commit as the descriptor. @@ -206,7 +208,7 @@ accepted blob and compatible reusable DWH binding; otherwise it starts a new run ## 8. P6 — commit-addressed Evidence materialization For filesystem Evidence, the registry materializes exactly -`workspace-content//evidence` from the pinned commit into an immutable revision content root. +`/evidence` from the pinned commit into an immutable revision content root. It does not consume the mobile registry checkout and does not resolve against author files. Materialization uses fixed Git plumbing to enumerate object type, mode, path, object ID, and bytes. diff --git a/docs/testing/p2-p6-manual-verification.md b/docs/testing/p2-p6-manual-verification.md index 8b782500..741b0e4a 100644 --- a/docs/testing/p2-p6-manual-verification.md +++ b/docs/testing/p2-p6-manual-verification.md @@ -79,7 +79,7 @@ Decision: **PENDING**. **Status:** instructions to be finalized by P5 implementation; not yet runnable. -Manual goal: curate `workspace-content//schema/annotations.yaml` in an author clone, publish it, +Manual goal: curate `/schema/annotations.yaml` in an author clone, publish it, pull the new revision, explicitly accept the reviewed blob, and prove atomic revision-correct sync without changing `physical.yaml` in Git.