ThothII/docs/testing/p2-p6-manual-verification.md

P2–P6 Manual Verification Walkthrough

Living document. Each section is completed with exact released commands and artifacts during its corresponding plan. Automated integration and manual acceptance use separate clean state.

Global rules

• Use a new temporary operator root and a new private fixture Git remote for each Px.
• Never use production PSD credentials in a retained report or screenshot.
• Keep descriptor/content in Git; keep endpoints, bindings, credentials, and certificates in the installation-local protected directory.
• Do not print secret files, rendered signed URLs, Compose environments, or unbounded logs.
• Record the ThothII commit, workspace commit, installation descriptor path, Compose project name, command exit status, and report path.
• A focused manual PASS does not replace the automated process goal.

P2 — Host preprocessing CLI

Status: P2 implementation complete; automated integration PASS; manual acceptance PENDING.

Manual goal: from a clean local installation, use only thothctl on the host to inspect one registry workspace and execute the controlled REST-DWH/HTTP-Evidence preprocessing path without a host Python or Node runtime. Use a fresh operator root and a fresh fixture Git remote; never reuse the automated .artifacts/p2-integration/** state.

Commands (contract: docs/contracts/workspace-preprocessing-cli.md):

thothctl --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --resume <run-id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
thothctl --installation <abs>/thothii-installation.yaml workspace schema check --workspace <id> --annotations <reviewed>.yaml --reviewed-candidates <sha256:hex> --json
thothctl --installation <abs>/thothii-installation.yaml workspace index-schema --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --json

Checks:

1. installation/render preflight (inspect returns exact revision + catalog/descriptor digests);
2. DWH introspection+LSH succeeds, rerun is unchanged, --resume <run-id> is unchanged/succeeded;
3. schema suggest-fks returns pristine JSON with suggestedFksYaml and a manual_review_required block (exit 3) when candidates exist; the suggested YAML digest equals the reported digest;
4. schema check --annotations <reviewed> --reviewed-candidates <digest> succeeds after review;
5. index-schema counts against a pre-provisioned compatible collection and rerun is unchanged;
6. HTTP Evidence --dry-run returns dry_run, the real run publishes, rerun is unchanged, an input mutation produces a new generation/ACTIVE;
7. filesystem Evidence returns a stable evidence_materialization_required block with no partial corpus/vector publication;
8. negatives: missing workspace (workspace_not_activatable), resume of a nonexistent run (preprocessing_resume_mismatch), invalid annotations digest (annotation_invalid), no-Evidence skip warning, no collection creation, no backend/Pi/frontend listener;
9. secret scan over retained artifacts and exact owned-resource cleanup.

Decision: PENDING (independent manual gate; automation never records PASS).

P3 — Effective configuration and .tht-dwh

Status: P3 implementation complete; automated integration PASS; manual acceptance PASS (owner approval 2026-08-13).

Manual goal: prove that the operator CLI and application sessions derive the same effective configuration, that a content-only revision reuses the prepared DWH generation (fast, unchanged), that a DWH-affecting change fails closed and regenerates, that the workspace memory migration is safe, and that search records are revision-scoped. See docs/contracts/tht-dwh.md.

Checks:

1. run thothctl ... workspace preprocess dwh twice with only an Evidence/content change between them: the second run reports unchanged and does not re-introspect;
2. change a DWH-affecting field (host/port/database/schema/user/collection) in the descriptor, push, pull: the next run refuses the old generation and regenerates, with a clear effective_config_mismatch-style outcome and no mixed artifacts;
3. inspect .tht-dwh generations: immutable directories, OWNER.json with the canonical fingerprints, ACTIVE pointer; old generations still present;
4. memory: after the guarded migration the workspace uses <dataRoot>/sessions/<workspace-id>/memory/; the JSONL registry and Qdrant projection are rebuilt and consistent; a conflicting legacy registry fails closed;
5. search records: schema/Evidence points carry the pinned workspace_revision; memory/solved records remain workspace-wide;
6. documentation: docs/contracts/tht-dwh.md matches the observed behavior.

Decision: PASS (owner approval 2026-08-13).

P4 — Qdrant bootstrap and guarded rebuild

Status: superseded by the "P4 Qdrant collection lifecycle" section below (implemented; manual acceptance PASS).

Manual goal: prove admission creates a missing compatible collection and indexes, refuses an incompatible collection, and permits destructive rebuild only under durable maintenance with no active readers/jobs and exact repeated confirmation.

Checks to fill during P4:

1. missing-collection self-heal;
2. missing-index self-heal;
3. dimensions/distance/index-type refusal;
4. confirmation mismatch refusal;
5. active-reader/job refusal;
6. successful drained rebuild;
7. interrupted rebuild recovery with maintenance retained.

Decision: PASS (owner approval 2026-08-13; see the section below).

P4 Qdrant collection lifecycle

Manual goal: verify admission self-heal and the guarded rebuild through the real product surface.

Checks to complete during P4 manual acceptance (decision: PASS (owner approval 2026-08-13)):

1. On a fresh installation with no Qdrant collection, a session admission creates the descriptor collection with exactly 1024 dimensions, cosine distance, and the 8 required keyword payload indexes (content_hash, document_id, kind, record_key, record_kind, vector_generation, workspace_id, workspace_revision).
2. A pre-existing collection with incompatible dimensions/distance (e.g. 768-dim or dot) is refused with semantic_index_incompatible and is never mutated.
3. thothctl ... workspace vector inspect --workspace <id> --json reports the collection contract without mutation (pristine JSON, exit 0).
4. thothctl ... workspace vector rebuild --workspace <id> --collection <name> --confirm <name> --destroy deletes and recreates the descriptor-owned collection and verifies the recreated contract; a mismatched --confirm or a missing --destroy is refused (exit 2) without touching the collection.
5. Rebuild writes durable state before deletion, deletes only the descriptor collection, and the recreated collection preserves the P3 revision-scoped payload contract.

P5 — Curated FK annotations in Git

Status: P5 implementation complete; automated integration PASS; manual acceptance PASS (owner approval 2026-08-13).

Manual goal: curate <id>/schema/annotations.yaml in an author clone, publish it, pull the new revision, and prove the revision-pinned sync and the explicit schema accept review, without ever pushing curated content from the operator CLI.

Commands (contract: docs/contracts/workspace-preprocessing-cli.md):

thothctl --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
# curate the candidate into <id>/schema/annotations.yaml in the author clone, then commit/push/pull
thothctl --installation <abs>/thothii-installation.yaml workspace schema accept --workspace <id> --run <run-id> --yes --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --resume <run-id> --json

Checks:

1. schema suggest-fks returns pristine JSON with suggestedFksYaml and a manual_review_required block (exit 3) when candidates exist; the suggested YAML digest equals the reported digest;
2. after commit/push/pull, activation reads <id>/schema/annotations.yaml as a regular Git blob at the same commit as the descriptor and synchronizes it to <data>/sessions/<id>/revisions/<commit>/artifacts/mschema/annotations.yaml with a restrictive mode and an adjacent ownership manifest { workspace, commit, blobId, contentDigest, destination };
3. two revisions write two different directories; a session pinned to an older revision reads its own revision's annotations;
4. schema accept --run <id> --yes records the accepted candidate/current-blob digests and the new revision; missing --yes, an unknown run, an empty file, a malformed blob, or a blob not matching the recorded candidate is refused (exit 1, annotation_invalid) without recording a review;
5. preprocess run --resume <id> continues only with the exact accepted blob digest and compatible DWH binding; otherwise it records a new manual_review_required checkpoint;
6. negatives: symlink/tree-at-path, cross-namespace, oversized (>16 MiB), non-UTF-8, and malformed annotation objects are refused at activation without mutating the snapshot or runtime roots;
7. the operator CLI never stages/commits/pushes curated content; secret scan and exact owned-resource cleanup pass.

Decision: PASS (owner approval 2026-08-13).

P6 — Commit-addressed Evidence materialization

Status: P6 implementation complete; automated integration PASS; manual acceptance PENDING.

Manual goal: materialize filesystem Evidence from the pinned Git commit, inspect its bounded manifest, preprocess/index it, retrieve only the pinned revision, and exercise unsafe-tree and aggregate-limit failures without partial publication.

Commands (contract: docs/contracts/workspace-preprocessing-cli.md):

thothctl --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json   # idempotent rerun

Checks:

1. exact commit/tree/object identities: after activation the materialized root is <registry>/snapshots/<commit>/<id>/evidence and its sibling manifest <id>/evidence.manifest.json records workspace, commit, tree, per-file oid/digest, entryCount, totalBytes; snapshot.json chains the manifest digest;
2. successful atomic materialization: every regular blob is present byte-for-byte; the manifest digests match;
3. manifest and file digest verification: re-activation reuses a valid root and fails closed on a tampered manifest;
4. filesystem Evidence dry-run/run/idempotency: --dry-run returns dry_run, the real run publishes, rerun is unchanged;
5. revision-filtered Qdrant retrieval and corpus ACTIVE: Evidence records carry the pinned workspace_revision;
6. nested symlink/gitlink/traversal/special-file refusal: a commit introducing one of these fails activation (workspace_invalid) and the previous valid revision stays active;
7. file-count/total-byte/path/manifest limit refusal: an oversized or over-count tree fails closed without a partial publication;
8. retention while pinned and owned cleanup after release: the materialized root persists for a pinned revision and is removed with its snapshot directory once unreferenced.

Decision: PENDING.

Final aggregate P2–P6 verification

Status: runnable only after P6.

The final manual pass will start with a new registry and two independent installations. It will run the complete DWH → FK → schema → filesystem Evidence chain, prove idempotency and revision isolation, confirm the second installation uses its own secrets/state, and compare its observations to the retained aggregate automated report.

Decision: PENDING.