ThothII/docs/plans/2026-08-24-evidence-restructuring.md

Evidence Restructuring Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Build a Git-reviewed, typed Evidence authoring pipeline and publish its approved output to the existing revision-scoped Qdrant lifecycle with dense+BM25 hybrid retrieval.

Architecture: Keep authoring outside NL→SQL sessions: deterministic preparation wraps one read-only Pi restructuring call, writes reviewable Markdown into the workspace repository, and blocks publication on unresolved review items. Reuse the current Evidence module, corpus generations, rollback, active-revision checks, and workspace-owned semantic collection; extend them instead of introducing a parallel store.

Tech Stack: Python 3.12, Pydantic 2, Typer, PyYAML, sqlglot, pytest, Pi CLI, TypeScript, Fastify workspace maintenance, Vitest, Qdrant 1.18.2 Query API, server-side qdrant/bm25, Git.

---

Preconditions

• Work in this dedicated worktree.
• Read PROJECT_STATE.md, CONTEXT.md, docs/plans/2026-08-24-evidence-restructuring-design.md, docs/contracts/workspace-evidence-v3.md, and docs/contracts/workspace-preprocessing-cli.md.
• Preserve the public facade in harness/tht/evidence/__init__.py.
• Do not edit harness/.pi/skills/tht-sessione/SKILL.md directly; regenerate it with python -m tht.pi_skill_projection --write.
• Keep runtime workspace access read-only. Only the authoring CLI may write evidence/curated/ and evidence/manifest.yaml in a curator clone.
• Do not migrate the external PSD repository until all code and contract gates pass.

Task 1: Add the typed Curated Evidence model

Files:

• Create: harness/tht/evidence/canonical.py
• Modify: harness/tht/evidence/__init__.py
• Create: harness/tests/test_evidence_canonical.py

Step 1: Write failing tests for the common envelope

Cover:

• all eight kind values;
• all five purpose values;
• immutable provenance;
• strict unknown-field rejection;
• namespaced stable IDs;
• SHA-256 syntax;
• directory/kind agreement;
• parsing and dumping Markdown with YAML frontmatter.

Start with:

def test_formula_requires_formula_payload():
    with pytest.raises(ValidationError):
        CuratedEvidence.model_validate({
            **COMMON,
            "kind": "formula",
            "payload": {"concept": "fascia pediatrica"},
        })


def test_reference_rejects_formula_payload():
    with pytest.raises(ValidationError):
        CuratedEvidence.model_validate({
            **COMMON,
            "kind": "reference",
            "payload": {
                "concept": "x",
                "columns": ["clinical.patient.birth_date"],
                "sql": "CASE WHEN true THEN 1 END",
            },
        })

Step 2: Run the focused tests and confirm RED

Run:

cd harness
.venv/bin/pytest tests/test_evidence_canonical.py -q

Expected: import failure for tht.evidence.canonical.

Step 3: Implement the discriminated model

Use a strict Pydantic model with these public types:

EvidenceKind = Literal[
    "glossary", "domain", "enum", "example", "mapping",
    "normalization", "formula", "reference",
]
EvidencePurpose = Literal[
    "disambiguation", "rewriting", "schema_linking", "sql_generation", "memory",
]

class EvidenceScope(StrictModel):
    concepts: tuple[str, ...] = ()
    tables: tuple[str, ...] = ()
    columns: tuple[str, ...] = ()

class EvidenceProvenance(StrictModel):
    source_file: str
    source_sha256: str

class FormulaPayload(StrictModel):
    concept: str
    columns: tuple[str, ...]
    sql: str

class ReferencePayload(StrictModel):
    url: AnyHttpUrl
    label: str
    description: str

Define equally strict payloads for the other six kinds and expose a single CuratedEvidence API. The implementation may use an internal Pydantic discriminated union, but callers must not switch between eight unrelated loaders.

Add:

def parse_curated_markdown(text: str, *, path: Path | None = None) -> CuratedEvidence: ...
def dump_curated_markdown(value: CuratedEvidence) -> str: ...
def load_curated_tree(root: Path) -> list[CuratedEvidence]: ...

Validate formula SQL with sqlglot and validate schema.table / schema.table.column identifiers without querying the DWH.

Step 4: Export only the stable API

Add the model and loader functions to harness/tht/evidence/__init__.py. Do not export internal union member helpers unless another module needs them.

Step 5: Run tests and lint

Run:

cd harness
.venv/bin/pytest tests/test_evidence_canonical.py tests/test_evidence_facade_contract.py -q
.venv/bin/ruff check tht/evidence/canonical.py tests/test_evidence_canonical.py

Expected: PASS.

Step 6: Commit

git add harness/tht/evidence/canonical.py harness/tht/evidence/__init__.py \
  harness/tests/test_evidence_canonical.py
git commit -m "feat(evidence): add typed curated evidence model"

Task 2: Add deterministic validation and the Evidence manifest

Files:

• Create: harness/tht/evidence/authoring.py
• Create: harness/tests/test_evidence_authoring.py
• Modify: harness/tht/evidence/__init__.py

Step 1: Write failing tests for publication validation

Test that:

• review_items != [] is valid as a draft but blocks publication;
• a missing or mismatched source hash blocks publication;
• two units cannot share an ID;
• one unit cannot claim two source files;
• curated/formula/x.md must contain kind: formula;
• credentials in URLs, YAML, or body are rejected;
• only Markdown, .txt, and .sql.md sources are accepted;
• UTF-8 and per-file limits are enforced.

Expose errors as bounded structured values:

@dataclass(frozen=True)
class ValidationFinding:
    severity: Literal["error", "warning"]
    code: str
    path: str
    message: str

Step 2: Write failing tests for the versioned manifest

Use this minimum shape:

schema_version: 1
pipeline_version: evidence-authoring-v1
sources:
  source/domain/patient.md:
    sha256: sha256:...
    units:
      - domain:patient
orphans: []

Test deterministic key ordering, stable round-trip, unknown fields, duplicate unit IDs, and orphan preservation.

Step 3: Run and confirm RED

Run:

cd harness
.venv/bin/pytest tests/test_evidence_authoring.py -q

Expected: missing authoring API.

Step 4: Implement EvidenceManifest and validate_workspace_evidence

Provide:

def load_manifest(path: Path) -> EvidenceManifest: ...
def dump_manifest(manifest: EvidenceManifest) -> str: ...
def validate_workspace_evidence(workspace_root: Path) -> ValidationReport: ...

ValidationReport.publishable is true only when there are no errors and no unresolved review items. Warnings alone do not block publication.

Do not use status fields such as draft/reviewed as an approval mechanism. The approved Git revision is the publication boundary.

Step 5: Run tests and lint

cd harness
.venv/bin/pytest tests/test_evidence_authoring.py tests/test_evidence_canonical.py -q
.venv/bin/ruff check tht/evidence/authoring.py tests/test_evidence_authoring.py

Expected: PASS.

Step 6: Commit

git add harness/tht/evidence/authoring.py harness/tht/evidence/__init__.py \
  harness/tests/test_evidence_authoring.py
git commit -m "feat(evidence): validate curated corpus and manifest"

Task 3: Implement incremental preparation with one Pi restructuring call

Files:

• Modify: harness/tht/evidence/authoring.py
• Create: harness/.pi/skills/tht-evidence-authoring/SKILL.md
• Modify: harness/tests/test_evidence_authoring.py
• Create: harness/tests/test_evidence_pi_restructurer.py

Step 1: Define the restructuring port and request/response models

Add:

class EvidenceRestructurer(Protocol):
    def restructure(self, request: RestructureRequest) -> tuple[CuratedEvidence, ...]: ...

class RestructureRequest(StrictModel):
    source_file: str
    source_sha256: str
    normalized_text: str
    previous_units: tuple[CuratedEvidence, ...] = ()

The response is validated through the models from Task 1 before any write.

Step 2: Write RED tests for the incremental rules

Test:

• unchanged source: no model call and no file write;
• changed source: exactly one model call;
• new source: new stable IDs;
• removed source: old units become orphans and remain on disk;
• one source may produce several units;
• no returned unit may cite another source;
• prior curated units are included in the request;
• a dirty evidence/curated/ or evidence/manifest.yaml fails before the model call;
• all output writes are staged and atomically replaced only after complete validation.

Inject the Git status runner and filesystem writer in tests; do not require a real Git repository for every unit test.

Step 3: Implement deterministic source normalization

Normalize UTF-8 text with NFC, LF newlines and terminal newline. Preserve meaningful Markdown, tables, fenced SQL, URLs and list structure. Do not rewrite vocabulary or infer domain facts in this step.

Step 4: Implement PiEvidenceRestructurer

Invoke Pi as an ephemeral, no-tools process using an argument list, never a shell:

argv = [
    pi_executable,
    "--mode", "text",
    "--print",
    "--no-session",
    "--no-tools",
    "--no-extensions",
    "--no-context-files",
    "--skill", str(skill_path),
    f"@{request_path}",
    "Return only the JSON object required by the Evidence authoring skill.",
]

Use a private TemporaryDirectory, a bounded timeout, bounded stdout/stderr, and strict JSON parsing. Do not forward the model's raw output to public JSON errors. The skill must state:

• use only facts present in normalized_text;
• preserve prior reviewed wording where it remains supported;
• never merge sources;
• emit review_items for uncertainty;
• emit exactly the schema-versioned JSON object and no Markdown fence.

Step 5: Implement prepare_workspace_evidence

Return a bounded report with changed, unchanged, created, orphaned, findings, and model_calls. Keep the model implementation behind EvidenceRestructurer.

Step 6: Run tests

cd harness
.venv/bin/pytest tests/test_evidence_authoring.py tests/test_evidence_pi_restructurer.py -q
.venv/bin/ruff check tht/evidence/authoring.py tests/test_evidence_pi_restructurer.py

Expected: PASS with no live model call.

Step 7: Commit

git add harness/tht/evidence/authoring.py \
  harness/.pi/skills/tht-evidence-authoring/SKILL.md \
  harness/tests/test_evidence_authoring.py harness/tests/test_evidence_pi_restructurer.py
git commit -m "feat(evidence): prepare curated evidence incrementally"

Task 4: Add the authoring CLI

Files:

• Create: harness/tht/cli/evidence_cmd.py
• Modify: harness/tht/cli/__init__.py
• Create: harness/tests/test_evidence_cli.py

Step 1: Write CLI grammar tests

Cover:

tht evidence prepare <workspace-root> [--json]
tht evidence validate <workspace-root> [--json]

Require an existing canonical Git worktree root. Reject unknown flags, symlinks, a workspace path outside the Git root, duplicate options and dirty curated state. Ensure --json writes pristine JSON to stdout.

Step 2: Run and confirm RED

cd harness
.venv/bin/pytest tests/test_evidence_cli.py -q

Expected: evidence command group is unknown.

Step 3: Implement the Typer group

Use one top-level authoring group:

evidence_app = typer.Typer(help="Prepare and validate workspace Evidence")

@evidence_app.command("prepare")
def prepare_cmd(workspace_root: Path, json_output: bool = False) -> None: ...

@evidence_app.command("validate")
def validate_cmd(workspace_root: Path, json_output: bool = False) -> None: ...

Register it in harness/tht/cli/__init__.py. Keep this separate from the existing runtime command tht preprocess evidence.

Exit codes:

• 0: prepared/unchanged or valid;
• 2: unsafe path or CLI misuse;
• 3: valid drafts but review required;
• 1: operational/model/structural failure.

Step 4: Run tests and CLI help

cd harness
.venv/bin/pytest tests/test_evidence_cli.py tests/test_preprocess_cli.py -q
.venv/bin/tht evidence --help
.venv/bin/tht preprocess evidence --help

Expected: both command families are present and unambiguous.

Step 5: Commit

git add harness/tht/cli/evidence_cmd.py harness/tht/cli/__init__.py \
  harness/tests/test_evidence_cli.py
git commit -m "feat(evidence): expose prepare and validate commands"

Task 5: Project typed units into semantic Evidence Fragments

Files:

• Modify: harness/tht/evidence/corpus/models.py
• Modify: harness/tht/evidence/corpus/chunk.py
• Modify: harness/tht/evidence/corpus/normalize.py
• Modify: harness/tht/evidence/corpus/pipeline.py
• Modify: harness/tht/evidence/preprocessing.py
• Modify: harness/tests/test_corpus_models.py
• Modify: harness/tests/test_corpus_chunk.py
• Modify: harness/tests/test_corpus_pipeline.py

Step 1: Write failing projection tests

Test that:

• a short formula remains one fragment;
• a domain document splits only at semantic section boundaries;
• enum entries are not split in the middle of a value/meaning pair;
• fixed-size fallback is used only for a single oversized section;
• all fragments carry evidence_id, evidence_kind, purposes, scope, language and provenance;
• fragment IDs and ordinals are deterministic;
• only curated/**/*.md is accepted as canonical filesystem content.

Step 2: Extend the immutable corpus models

Keep CanonicalDocument and CanonicalChunk as transport-neutral storage models. Put typed Evidence metadata in their already-safe metadata field, with exact allowlisted keys. Do not make corpus storage depend on Pydantic subtype classes at read time.

Step 3: Implement kind-aware fragment rendering

Construct embedding text from title, purpose, scope and type-specific data. Example for a formula:

Formula: Fascia pediatrica
Concept: fascia pediatrica
Columns: clinical.patient.birth_date
SQL: CASE WHEN ... END
Limitations: ...

The rendered text is derived; provenance and canonical content remain in the corpus manifest.

Step 4: Run focused pipeline tests

cd harness
.venv/bin/pytest tests/test_corpus_models.py tests/test_corpus_chunk.py \
  tests/test_corpus_normalize.py tests/test_corpus_pipeline.py \
  tests/test_corpus_publish.py -q

Expected: PASS, including existing rollback, compensation, retention and resume tests.

Step 5: Commit

git add harness/tht/evidence/corpus harness/tht/evidence/preprocessing.py \
  harness/tests/test_corpus_models.py harness/tests/test_corpus_chunk.py \
  harness/tests/test_corpus_normalize.py harness/tests/test_corpus_pipeline.py
git commit -m "feat(evidence): build semantic fragments from typed units"

Task 6: Upgrade the Qdrant collection contract to named dense and BM25 vectors

Files:

• Modify: backend/src/workspaces/qdrant-collection.ts
• Modify: backend/test/qdrant-collection.test.ts
• Modify: harness/tht/adapters/vector/qdrant.py
• Modify: harness/tests/test_qdrant_vector_store.py
• Modify: docs/contracts/workspace-preprocessing-cli.md

Step 1: Write RED TypeScript collection-contract tests

The required vector contract is:

{
  "vectors": {
    "dense": {"size": 1024, "distance": "Cosine"}
  },
  "sparse_vectors": {
    "bm25": {"modifier": "idf"}
  }
}

Test that unnamed dense-only, wrong named dimension/distance, missing BM25, or wrong BM25 modifier are incompatible. Self-heal may add missing payload indexes, but must not silently convert an incompatible vector configuration.

Add keyword indexes only for fields used by filters:

content_hash, document_id, kind, record_key, record_kind,
vector_generation, workspace_id, workspace_revision,
evidence_id, evidence_kind, purposes, concepts, tables, columns, language

Step 2: Run the TypeScript test and confirm RED

cd backend
npx vitest run test/qdrant-collection.test.ts

Expected: current unnamed-vector expectations fail.

Step 3: Implement strict named-vector reconciliation

Update createCollection, vectorCompatibility and payload-index reconciliation. Retain require_existing behavior: validation never performs an incompatible migration.

Step 4: Update the Python adapter's collection validation

QdrantVectorStore.health() and _ensure_collection() must recognize exactly the same contract as TypeScript. Add a shared test fixture shape even though the two languages do not share implementation code.

Step 5: Run backend and harness tests

cd backend
npx vitest run test/qdrant-collection.test.ts
npx tsc --noEmit -p .
cd ../harness
.venv/bin/pytest tests/test_qdrant_vector_store.py tests/test_vector_port_contract.py -q

Expected: PASS.

Step 6: Document the required guarded rebuild

Update the CLI contract to say that the vector-shape change is incompatible and must be applied with the existing exact-name guarded command:

tht --installation <absolute>/thothii-installation.yaml workspace vector rebuild
  --workspace <id> --collection <name> --confirm <name> --destroy

No automatic deletion is allowed.

Step 7: Commit

git add backend/src/workspaces/qdrant-collection.ts \
  backend/test/qdrant-collection.test.ts harness/tht/adapters/vector/qdrant.py \
  harness/tests/test_qdrant_vector_store.py docs/contracts/workspace-preprocessing-cli.md
git commit -m "feat(evidence): require dense and bm25 qdrant vectors"

Task 7: Add server-side BM25 ingestion and hybrid Query API retrieval

Files:

• Modify: harness/tht/ports/vector.py
• Modify: harness/tht/adapters/vector/qdrant.py
• Modify: harness/tht/vectorstore/records.py
• Modify: harness/tht/evidence/corpus/pipeline.py
• Modify: harness/tests/test_vector_port_contract.py
• Modify: harness/tests/test_qdrant_vector_store.py
• Modify: harness/tests/test_corpus_pipeline.py

Step 1: Write RED port tests

Extend, do not replace, the current record:

@dataclass(frozen=True)
class VectorWriteRecord:
    record: VectorRecord
    embedding: list[float]
    content_hash: str
    sparse_text: str | None = None
    sparse_language: str | None = None

Extend VectorStore.search with keyword-only query_text and query_language. Existing callers that omit them remain dense-only.

Step 2: Write exact Qdrant request tests

For Evidence upsert, assert:

"vector": {
  "dense": [0.1, 0.2],
  "bm25": {
    "text": "...",
    "model": "qdrant/bm25",
    "options": {"language": "italian"}
  }
}

For hybrid search, assert two filtered prefetches and default RRF:

{
  "prefetch": [
    {"query": [0.1, 0.2], "using": "dense", "limit": 20, "filter": {}},
    {
      "query": {
        "text": "fascia pediatrica",
        "model": "qdrant/bm25",
        "options": {"language": "italian"}
      },
      "using": "bm25",
      "limit": 20,
      "filter": {}
    }
  ],
  "query": {"rrf": {}},
  "limit": 10,
  "with_payload": true
}

Both prefetch filters must include workspace, revision, active generation and record kind. Do not add hand-tuned weights.

Step 3: Implement named dense writes for all semantic records

All existing schema/memory/Evidence records use the dense vector name after the collection rebuild. Only records with sparse_text receive bm25.

Step 4: Implement BM25 Evidence ingestion

Populate sparse_text and map workspace language it to Qdrant's italian. Reject an unsupported language before uploading the generation. Use the same options at ingest and query time.

Step 5: Implement hybrid search with dense fallback only for non-Evidence callers

An Evidence hybrid request must fail as unavailable if the configured Qdrant version or collection contract does not support BM25. It must not silently claim to have run hybrid search. Existing non-Evidence dense requests continue to work.

Step 6: Run focused tests

cd harness
.venv/bin/pytest tests/test_vector_port_contract.py tests/test_qdrant_vector_store.py \
  tests/test_corpus_pipeline.py tests/test_search_pack.py -q
.venv/bin/ruff check tht/ports/vector.py tht/adapters/vector/qdrant.py \
  tht/evidence/corpus/pipeline.py

Expected: PASS.

Step 7: Commit

git add harness/tht/ports/vector.py harness/tht/adapters/vector/qdrant.py \
  harness/tht/vectorstore/records.py harness/tht/evidence/corpus/pipeline.py \
  harness/tests/test_vector_port_contract.py harness/tests/test_qdrant_vector_store.py \
  harness/tests/test_corpus_pipeline.py
git commit -m "feat(evidence): add qdrant bm25 hybrid retrieval"

Task 8: Add the typed Evidence search contract and workflow-owned purposes

Files:

• Modify: harness/tht/evidence/search.py
• Modify: harness/tht/evidence/__init__.py
• Modify: harness/tht/cli/search_cmd.py
• Modify: harness/tests/test_evidence_facade_contract.py
• Modify: harness/tests/test_search_pack.py
• Create: harness/.pi/skills/tht-sessione/modules/evidence/runtime-search.md
• Modify: harness/.pi/skills/tht-sessione/projection.md.tmpl
• Modify: harness/tht/pi_skill_projection.py
• Modify: harness/tests/test_pi_skill_projection.py
• Regenerate: harness/.pi/skills/tht-sessione/SKILL.md

Step 1: Write RED search-facade tests

Introduce:

class EvidenceSearchContext(BaseModel):
    concepts: tuple[str, ...] = ()
    tables: tuple[str, ...] = ()
    columns: tuple[str, ...] = ()
    required_kinds: tuple[EvidenceKind, ...] = ()

def search_evidence(
    query: str,
    purpose: EvidencePurpose,
    context: EvidenceSearchContext,
    *,
    searcher: ActiveEvidenceSearcher,
    embedder: EvidenceQueryEmbedder,
    top_n: int = 10,
) -> list[EvidenceResult]: ...

Test hard filters for workspace/revision/generation and explicit required_kinds. Test that purpose/kind/scope preferences are deterministic tie-breakers when they were not requested as hard filters.

Step 2: Test fragment grouping

Two returned fragments with the same evidence_id must become one EvidenceResult, with the best score, ordered matching excerpts, canonical citation and no duplicate unit.

Step 3: Preserve fail-closed graceful degradation

Test absent ACTIVE corpus, revision mismatch, unavailable Qdrant and malformed payload. All return no Evidence plus a bounded warning through the existing search-pack contract; none uses stale rows.

Step 4: Implement the facade and CLI mapping

Keep Qdrant syntax inside tht.evidence. search_cmd.py translates command inputs to the facade and renders results; it must not duplicate ranking logic.

Step 5: Extract Evidence instructions into a module fragment

Move the common F1/F3/F4 Evidence rules from the projection template into modules/evidence/runtime-search.md. The fragment must state:

• candidates are not truth;
• pass the phase-appropriate purpose;
• show provenance;
• formulas are kind=formula, not a separate search store;
• absence of Evidence is visible but does not stop the whole session.

Register the fragment in the static FRAGMENT_ORDER and regenerate.

Step 6: Run tests

cd harness
python -m tht.pi_skill_projection --write
python -m tht.pi_skill_projection --check
.venv/bin/pytest tests/test_evidence_facade_contract.py tests/test_search_pack.py \
  tests/test_pi_skill_projection.py -q

Expected: PASS and no direct edit drift in generated SKILL.md.

Step 7: Commit

git add harness/tht/evidence/search.py harness/tht/evidence/__init__.py \
  harness/tht/cli/search_cmd.py harness/tests/test_evidence_facade_contract.py \
  harness/tests/test_search_pack.py \
  harness/.pi/skills/tht-sessione/modules/evidence/runtime-search.md \
  harness/.pi/skills/tht-sessione/projection.md.tmpl \
  harness/.pi/skills/tht-sessione/SKILL.md harness/tht/pi_skill_projection.py \
  harness/tests/test_pi_skill_projection.py
git commit -m "refactor(evidence): own typed runtime retrieval"

Task 9: Migrate formulas into Curated Evidence

Files:

• Modify: harness/tht/evidence/formula_store.py
• Modify: harness/tht/evidence/session.py
• Modify: harness/tht/cli/search_cmd.py
• Create: harness/.pi/skills/tht-sessione/modules/evidence/formula-proposals.md
• Modify: harness/.pi/skills/tht-sessione/projection.md.tmpl
• Modify: harness/tht/pi_skill_projection.py
• Modify: harness/tests/test_formula.py
• Modify: harness/tests/test_formula_wiring.py
• Create: harness/tests/test_evidence_formula_migration.py
• Modify: harness/tests/test_pi_skill_projection.py
• Regenerate: harness/.pi/skills/tht-sessione/SKILL.md

Step 1: Write RED migration tests

Map an approved legacy ConceptFormula to a Curated Evidence formula while preserving:

• concept;
• SQL;
• columns;
• sources as provenance notes;
• stable deterministic ID;
• reviewed content wording.

Reject auto and unresolved draft formulas from direct publication; they become Formula proposals.

Step 2: Define the session proposal contract

Add a schema-versioned formula-proposal projection in the session artifact. Keep concept_formula_approved / concept_formula_rejected decisions unchanged because they record a session-local choice, not repository publication.

Step 3: Remove the separate runtime formula lookup

Change tht search find --kind formula to call typed Evidence search with required_kinds=("formula",). Keep the legacy store readable only for the migration command/window, with a deprecation warning in human output and no warning leakage into pristine JSON.

Step 4: Add and project formula instructions

The Evidence fragment must say that a newly synthesized formula is a session proposal and cannot be treated as Published Evidence.

Step 5: Run tests

cd harness
python -m tht.pi_skill_projection --write
.venv/bin/pytest tests/test_formula.py tests/test_formula_wiring.py \
  tests/test_evidence_formula_migration.py tests/test_pi_skill_projection.py \
  tests/test_decision_min_phase.py tests/test_workflow_observable_contract.py -q

Expected: PASS; decision phase ownership remains F4.

Step 6: Commit

git add harness/tht/evidence/formula_store.py harness/tht/evidence/session.py \
  harness/tht/cli/search_cmd.py \
  harness/.pi/skills/tht-sessione/modules/evidence/formula-proposals.md \
  harness/.pi/skills/tht-sessione/projection.md.tmpl \
  harness/.pi/skills/tht-sessione/SKILL.md harness/tht/pi_skill_projection.py \
  harness/tests/test_formula.py harness/tests/test_formula_wiring.py \
  harness/tests/test_evidence_formula_migration.py harness/tests/test_pi_skill_projection.py
git commit -m "refactor(evidence): unify formulas with typed evidence"

Task 10: Add the small retrieval evaluation command

Files:

• Create: harness/tht/evidence/evaluation.py
• Modify: harness/tht/cli/evidence_cmd.py
• Create: harness/tests/test_evidence_evaluation.py
• Modify: harness/tests/test_evidence_cli.py

Step 1: Write RED schema tests

Use a deliberately small format:

schema_version: 1
queries:
  - id: pediatric-formula
    query: Come distinguo i pazienti pediatrici?
    purpose: sql_generation
    expected:
      - formula:fascia-pediatrica

Require unique query IDs, nonempty expected IDs and only public purpose values.

Step 2: Write RED metric tests

Compute hit_at_5, hit_at_10, missing expected IDs, empty-result queries and counts by expected kind. Do not add nDCG, relevance grading or an evaluation database in v1.

Step 3: Implement the evaluator and CLI

Expose:

tht evidence evaluate <workspace-root> -c <runtime-config> [--json]

The report includes workspace revision, active vector generation and the fixed default RRF configuration. It is read-only.

Step 4: Run tests

cd harness
.venv/bin/pytest tests/test_evidence_evaluation.py tests/test_evidence_cli.py -q
.venv/bin/ruff check tht/evidence/evaluation.py tests/test_evidence_evaluation.py

Expected: PASS.

Step 5: Commit

git add harness/tht/evidence/evaluation.py harness/tht/cli/evidence_cmd.py \
  harness/tests/test_evidence_evaluation.py harness/tests/test_evidence_cli.py
git commit -m "feat(evidence): evaluate retrieval with a small fixture"

Task 11: Enforce curated-only runtime ingestion and update contracts

Files:

• Modify: backend/src/workspaces/schema.ts
• Modify: backend/src/workspaces/runtime-renderer.ts
• Modify: backend/test/workspaces/schema.test.ts
• Modify: backend/test/workspaces/runtime-renderer.test.ts
• Modify: docs/contracts/workspace-evidence-v3.md
• Modify: docs/contracts/workspace-preprocessing-cli.md
• Modify: docs/architecture/overview.md
• Modify: PROJECT_STATE.md

Step 1: Write RED descriptor/rendering tests

For filesystem Evidence, make patterns: ["curated/**/*.md"] the documented and generated default for the new authoring layout. Continue accepting an explicitly configured safe pattern for non-Git HTTP/S3 compatibility, but reject a filesystem descriptor that includes both source/** and curated/** once it declares the new layout version.

If a schema-version field is required to preserve compatibility, add it to the Evidence subcontract, not to the whole workspace descriptor.

Step 2: Implement the narrowest compatible descriptor change

The materializer continues to copy the entire evidence/ tree at the pinned commit. Only the rendered runtime acquisition patterns restrict preprocessing to curated/. Do not duplicate or move P6 materialization logic.

Step 3: Update documentation contracts

Document:

• source/curated layout;
• Git publication boundary;
• no runtime writes;
• validation before indexing;
• dense+BM25 collection contract and guarded rebuild;
• exact public operation names and JSON status additions, if any.

Step 4: Run backend and harness contract gates

cd backend
npx vitest run test/workspaces/schema.test.ts test/workspaces/runtime-renderer.test.ts \
  test/workspaces/evidence/materialization.test.ts \
  test/workspaces/evidence/preprocessing.test.ts
npx tsc --noEmit -p .
cd ../harness
.venv/bin/pytest tests/test_registry_evidence_config.py \
  tests/test_filesystem_evidence_source.py tests/test_preprocess_cli.py -q

Expected: PASS; P6 materialization safety remains unchanged.

Step 5: Commit

git add backend/src/workspaces/schema.ts backend/src/workspaces/runtime-renderer.ts \
  backend/test/workspaces/schema.test.ts \
  backend/test/workspaces/runtime-renderer.test.ts \
  docs/contracts/workspace-evidence-v3.md \
  docs/contracts/workspace-preprocessing-cli.md docs/architecture/overview.md PROJECT_STATE.md
git commit -m "docs(evidence): publish curated-only workspace contract"

Task 12: Migrate PSD and perform acceptance

Files in ThothII:

• Create: docs/testing/evidence-restructuring-manual.md
• Create: scripts/evidence-restructuring-acceptance.sh
• Create: harness/tests/fixtures/evidence_authoring/poorly_structured.md
• Modify: PROJECT_STATE.md

Files in the external authoring repository:

• Move: /Users/mp/projects/tht-workspace-psd/psd-clinical/evidence/<current-folders> to /Users/mp/projects/tht-workspace-psd/psd-clinical/evidence/source/
• Create: /Users/mp/projects/tht-workspace-psd/psd-clinical/evidence/curated/<kind>/
• Create: /Users/mp/projects/tht-workspace-psd/psd-clinical/evidence/manifest.yaml
• Create: /Users/mp/projects/tht-workspace-psd/psd-clinical/evidence/evaluation.yaml
• Modify: /Users/mp/projects/tht-workspace-psd/psd-clinical/evidence/README.md
• Modify: /Users/mp/projects/tht-workspace-psd/psd-clinical/workspace.yaml

Do not modify the external repository until the owner confirms the migration window and the exact target branch. Treat that as the only manual authorization gate in this task.

Step 1: Add a hermetic badly-structured fixture

The fixture must contain prose, a rough list, an enum, an URL, an ambiguous statement and a SQL formula candidate. The acceptance runner must prove:

• split into multiple typed units;
• ambiguity becomes review_items;
• no cross-source merge;
• unchanged rerun is a no-op;
• human edit is preserved;
• dirty-tree refusal;
• validation blocks unresolved review;
• validated corpus indexes and searches hybrid;
• Qdrant failure remains fail-closed.

Use a fake restructurer for hermetic CI. The real Pi call is a separate manual check.

Step 2: Run the complete automated gates before external writes

cd harness
.venv/bin/pytest -q
.venv/bin/ruff check .
cd ../backend
npx vitest run
npx tsc --noEmit -p .
cd ../tools/tht
go test ./...
go build ./cmd/tht
cd ../..
bash scripts/evidence-restructuring-acceptance.sh

Expected: all suites and acceptance checks PASS. If pre-existing unrelated failures remain, record exact names and prove they reproduce at the baseline commit before continuing.

Step 3: Stop for the owner migration gate

Provide:

• clean ThothII commit;
• test and acceptance summary;
• proposed PSD branch name;
• exact list of 36 source files to move;
• rollback command based on the pre-migration PSD commit;
• notice that Qdrant rebuild is destructive but scoped by exact collection-name guards.

Do not infer approval from prior design acceptance.

Step 4: Migrate the PSD repository after approval

Use tht evidence prepare, review the Git diff, resolve all review items manually, run tht evidence validate, and create the approximately twenty evaluation queries. Do not auto-merge or auto-push unless separately requested.

Step 5: Activate and rebuild with the existing guarded operator path

After the PSD merge/pull and activation, inspect first:

tht --installation <absolute>/thothii-installation.yaml workspace vector inspect
  --workspace psd-clinical --json

Then use the exact descriptor-owned name in the guarded rebuild command. Run workspace preprocess evidence, tht evidence evaluate, and the manual F1/F3/F4 walkthrough.

Step 6: Record acceptance and commit ThothII documentation

docs/testing/evidence-restructuring-manual.md must record separate outcomes for:

• authoring and Git review;
• collection rebuild;
• preprocessing generation publication;
• hybrid retrieval evaluation;
• formula retrieval;
• graceful degradation;
• complete session behavior.

Update PROJECT_STATE.md only with observed results and immutable commit/run IDs.

git add docs/testing/evidence-restructuring-manual.md \
  scripts/evidence-restructuring-acceptance.sh \
  harness/tests/fixtures/evidence_authoring/poorly_structured.md PROJECT_STATE.md
git commit -m "test(evidence): record restructuring acceptance"

Final verification checklist

Before claiming completion, verify:

• CONTEXT.md and both Evidence plan documents use the same terminology.
• All eight Evidence kinds have type-specific positive and negative tests.
• Pi is called once per changed source, with no tools and no saved session.
• prepare refuses dirty curated state and never deletes orphans.
• validate blocks unresolved review items.
• Runtime reads only curated/**/*.md from the pinned Git revision.
• The TypeScript and Python Qdrant compatibility checks agree.
• Qdrant collection uses named dense plus bm25 with IDF.
• Italian BM25 options are identical during ingest and query.
• Hybrid search uses two prefetches and default RRF.
• Hard filters always include workspace, revision and active generation.
• Formula runtime lookup uses typed Evidence; session proposals remain non-published.
• Fragment hits are grouped into complete Evidence Units.
• Evaluation reports hit@5 and hit@10 against a versioned fixture.
• Existing corpus rollback, compensation, retention and resume tests still pass.
• Backend Vitest and TypeScript gates pass.
• Harness pytest and Ruff gates pass.
• Native tht Go tests/build pass.
• External PSD writes occurred only after explicit migration authorization.