diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 2fd1bfbf..d54ba2d1 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -1,25 +1,31 @@ # ThothII — Project State -## Evidence restructuring owner gate (#46) — automated acceptance observed; PSD migration pending (#47) (2026-08-25) +## Evidence restructuring — PSD migration and real acceptance PASS (#47/#35) (2026-08-25) -- **Observed local automation:** `bash scripts/evidence-restructuring-acceptance.sh` completed its - isolated fake-restructurer probe and the selected Evidence/L0 contract suite. The owner-gate - run at `d4818c8` observed **225 passed, 1 known pytest deprecation warning**. The final-review - fix added five regressions to the selected files and its fresh run observed - **230 passed, 1 known pytest deprecation warning**. Both probes used only a `mktemp` workspace, - confirmed the ThothII worktree status was unchanged, and did not receive or mutate an external - PSD authoring path. The separately authorized PSD preparation was read-only as recorded below. -- **Read-only owner-gate package:** authorized inspection of the clean PSD repository at - `47516f85b4db4a67cfa8a86cea4cb2e7b98c5813` recorded 35 moveable source documents plus the - retained `evidence/README.md` (36 current evidence files total), and the legacy `psd-clinical` - Qdrant baseline (163 schema tables, 2,275 schema columns, 2 memory, 1 solved question) in - `docs/testing/evidence-restructuring-manual.md`. The counts and representative IDs now come - only from exact filtered counts and three-ID, no-payload/no-vector scrolls. The before/after - Git status remained clean; no secret was read and no external repository mutation occurred. -- **Scope boundary:** the real Pi call, PSD migration, Git review, authoritative pre/post vector - inspection, activation, and human walkthrough remain **PENDING in issue #47** until the owner - authorizes the migration window and exact target branch. No PSD migration is represented by - this entry. +- **Automated owner-gate history:** the isolated runs recorded + **225 passed, 1 known pytest deprecation warning** at `d4818c8` and, after five added + regressions, **230 passed, 1 known pytest deprecation warning**. The final authorized run and + full-suite results are linked from the acceptance record below. +- **Approved corpus:** Marco Pancotti recorded Human Git review PASS for all 35 proposed Evidence + and all 60 review items. PSD PRs `#2`, `#3`, and `#4` are merged; published workspace revision + `1c304efa02547a4c10826f557376a38e959b8bbf` contains 35 validated curated units with zero + unresolved findings. +- **Safe publication:** snapshot + `psd-clinical-6759909623621226-2026-08-25-18-43-21.snapshot` was taken before mutation. + Publication run `c564d7fdb36436b3ae76dc0c2ce1e20d` activated + `gen:f968808223bf462fa406c9a6df8f6a55` only after 20/20 retrieval queries passed Hit@10. The + existing unnamed 1,024-d cosine vector and collection remain; only BM25/IDF was added. The + previous generation is retained for rollback. +- **Real walkthrough:** finalized session `20301df7-cb3c-421a-b14f-dec8cf8d9620` exercised every + required phase against the VPN-backed PSD DWH, persisted five generation-bound Evidence + receipts, executed all three CTEs successfully, and returned 78 patients from approved + read-only SQL. Memory/synthesis did not search Evidence; formulas and session Evidence stayed + unpublished. +- **Acceptance record:** + `docs/testing/evidence/evidence-restructuring-psd-acceptance-2026-08-25.md` records the seven + manual gates, commit/run/generation IDs, retrieval ranks, recovery material, durable artifact + paths, and suite results. State: **PASS**; issues `#47` and `#35` may be closed after PR `#48` + is green and merged. ## Modular workflow refactor candidate — live (2026-08-24) diff --git a/docs/testing/evidence/evidence-restructuring-psd-acceptance-2026-08-25.md b/docs/testing/evidence/evidence-restructuring-psd-acceptance-2026-08-25.md new file mode 100644 index 00000000..58cbee95 --- /dev/null +++ b/docs/testing/evidence/evidence-restructuring-psd-acceptance-2026-08-25.md @@ -0,0 +1,119 @@ +# PSD Evidence restructuring — real acceptance + +Date and completion time: `2026-08-25T21:44:26Z` (UTC) +Reviewer: Marco Pancotti +Human Git review: **PASS** +Scope: issues `#47` and parent `#35` + +The reviewer approved all 35 proposed Evidence and authorized resolution of all 60 review +items. The accepted rules were: retain `domain` when fully qualified identifiers are absent; +invent no schema, table, column, or value; mark incomplete lists as non-exhaustive; preserve +caveats and ambiguities; consolidate examples into one unit per source; treat N=5 as a +recommendation rather than a mandatory limit; and interpret "SEF seguito da ablazione" as a +later event in time. + +## Source and recovery record + +- ThothII implementation candidate `66f9fa2821decf30bec4468a33a499d8ea459510` and final + Linux deployment repair `d49c644b611a7cc50f19fd40f30464ec0a12d2db` (PR `#48`). +- PSD authoring review: PR `#2`, head `c93174841da253512ab81b32cf8c68304bc02e31`, + merged as `07ae6930a21299082685d2b3668912ac2d188079`. +- Canonical-root correction: PR `#3`, head `93c4b9a3189eb3113bb5d2d0b313332c064e1a1b`, + merged as `a4b27c6fe1cf41aac8102933a4100da8fee345e6`. +- Atomic-unit retention: PR `#4`, head `384f76a11d1e705b41d9df785337f81bc7515e2d`, + merged as the published PSD revision + `1c304efa02547a4c10826f557376a38e959b8bbf`. +- Pre-migration Qdrant snapshot: + `psd-clinical-6759909623621226-2026-08-25-18-43-21.snapshot`, 28,121,600 bytes, + SHA-256 `da7fdf114fdd1a638eb6f828ac127126258451dc617e8d409b5895bfd15397a6`. + It is retained for collection-level rollback; no collection was deleted or renamed. + +Command output and durable runtime records are in: + +- this report and `docs/testing/evidence-restructuring-manual.md`; +- `/data/sessions/psd-clinical/preprocessing/jobs/44bedc056f256d983ce88b9a565d9fd6.json` + (dry run) and + `/data/sessions/psd-clinical/preprocessing/jobs/c564d7fdb36436b3ae76dc0c2ce1e20d.json` + (publication); +- `/data/sessions/psd-clinical/corpus/ACTIVE` and + `/data/sessions/psd-clinical/corpus/gen-f968808223bf462fa406c9a6df8f6a55/manifest.json`; +- `/data/sessions/psd-clinical/sessions/20301df7-cb3c-421a-b14f-dec8cf8d9620/` + for the real walkthrough artifacts. + +No secret, patient identifier, payload, or vector is copied into this report. + +## Manual acceptance results + +1. **Authoring and Git review — PASS.** Exactly 35 source documents were migrated and + validated into 35 curated units (34 `domain`, 1 `glossary`), with one unit per source, + zero remaining review items, zero validation findings, stable `evidence:` IDs, and + no automatic orphan deletion. `psd-clinical/evidence/README.md` remains present. PR `#2` + records the human-reviewed content; PRs `#3` and `#4` are path/policy corrections without + semantic invention. + +2. **Additive BM25 upgrade — PASS.** The existing `psd-clinical` collection and unnamed + 1,024-dimensional cosine dense vector were preserved. The only vector-schema addition is + sparse vector `bm25` with modifier `idf`. There was no rebuild, dense-vector rename, + fallback engine, or collection replacement. + +3. **Schema and Memory non-regression — PASS.** Immediately before and after Evidence + publication the protected counts remained 163 `schema_table`, 2,275 `schema_column`, + 2 `memory`, and 1 `solved_question`; the saved representative IDs and repeated dense + Schema/Memory neighbors were unchanged. The later real walkthrough intentionally promoted + two approved memories and one solved question, so the final live counts are 4 and 2 while + all baseline IDs remain present. The 35-unit migration itself did not modify those families. + +4. **Inactive candidate, evaluation, activation — PASS.** Dry run + `44bedc056f256d983ce88b9a565d9fd6` completed without activation. Publication run + `c564d7fdb36436b3ae76dc0c2ce1e20d` built child + `f968808223bf462fa406c9a6df8f6a55` as an inactive candidate, evaluated that exact + generation, then activated `gen:f968808223bf462fa406c9a6df8f6a55`. It contains 35 + documents and 35 chunks; the run reported 35 changed and 36 legacy removals from the active + set. All 20 evaluation queries passed Hit@10. Representative diagnostic ranks were: + + | Profile | Query | Dense | BM25 | Fused | + | --- | --- | ---: | ---: | ---: | + | lexical | `lexical-chirone-meta` | 1 | 1 | 1 | + | semantic | `semantic-controllo-device` | 1 | 1 | 1 | + | mixed | `mixed-deduplica-codici` | 7 | 5 | 2 | + + The previous generation `gen:f91ccc1ae1dc4ccab05e7d70a7675a97` is retained. The live + collection has 78 Evidence points (43 retained older points plus the 35 active-generation + points); generation filtering, rather than destructive deletion, determines publication. + +5. **Hybrid and Formula retrieval — PASS.** The contract suite proves dense and BM25 receive + the identical NFC-normalized, newline-preserving, outer-trim-only query. The isolated + acceptance fixture accepts a PostgreSQL expression, rejects a full query, and retrieves an + approved formula through its typed Evidence path. No PSD formula was invented for this + migration. The real session persisted `concept_formulas: []` and `evidence.json: []`, so its + proposals remain unpublished. + +6. **Empty versus unavailable — PASS.** The acceptance probe recorded an available empty + retrieval that may continue and a controlled unavailable-Qdrant retrieval that blocks the + stage. The unavailable path used neither stale generation nor purpose fallback. + +7. **Complete real session — PASS.** Session + `20301df7-cb3c-421a-b14f-dec8cf8d9620`, named + `Accettazione Evidence #47 — SEF seguito da ablazione`, ran with `zai/glm-5.3` through + clarification, rewriting, schema linking, three executed CTEs, final SQL, and finalization. + It used the approved interpretation of two distinct events in 2024 and the temporal predicate + `ablazione > SEF`; all three CTE executions returned `ok`. The approved read-only SQL has + SHA-256 `b26d26c9c1e8579d7e3d5ccabe874e02b570bc15cfbe1b2ce822c28ae8b0e2ac`, + parsed without warnings, and returned **78 patients**. Five independently persisted Evidence + receipts cover clarification/disambiguation, rewriting, schema linking, CTE/SQL generation, + and final SQL, all bound to `gen:f968808223bf462fa406c9a6df8f6a55`. Memory and synthesis + did not invoke Evidence search. The authenticated UI showed the finalized session to Local + Admin. The abandoned provider preflight session was archived without deletion. + +## Automated verification + +- `bash scripts/evidence-restructuring-acceptance.sh`: Evidence acceptance contracts PASS. +- Backend Vitest: 78 files passed, 1 skipped; 1,119 tests passed, 40 skipped; TypeScript PASS. +- Frontend Vitest and TypeScript PASS. +- Harness: 1,103 tests passed, 4 deselected; Ruff PASS. +- Go: 19 packages, zero failures. +- Task 13 runtime fixtures: local PASS; server PASS; shell syntax PASS. The server regression + proves the root-owned `0700` canonical authentication directory is created before OIDC + configuration. + +manual acceptance: PASS