219 lines
13 KiB
Markdown
219 lines
13 KiB
Markdown
# P2–P6 Manual Verification Walkthrough
|
||
|
||
> Living document. Each section is completed with exact released commands and artifacts during its
|
||
> corresponding plan. Automated integration and manual acceptance use separate clean state.
|
||
|
||
## Global rules
|
||
|
||
- Use a new temporary operator root and a new private fixture Git remote for each Px.
|
||
- Never use production PSD credentials in a retained report or screenshot.
|
||
- Keep descriptor/content in Git; keep endpoints, bindings, credentials, and certificates in the
|
||
installation-local protected directory.
|
||
- Do not print secret files, rendered signed URLs, Compose environments, or unbounded logs.
|
||
- Record the ThothII commit, workspace commit, installation descriptor path, Compose project name,
|
||
command exit status, and report path.
|
||
- A focused manual PASS does not replace the automated process goal.
|
||
|
||
## P2 — Host preprocessing CLI
|
||
|
||
**Status:** P2 implementation complete; automated integration PASS; manual acceptance PENDING.
|
||
|
||
Manual goal: from a clean local installation, use only `tht` on the host to inspect one
|
||
registry workspace and execute the controlled REST-DWH/HTTP-Evidence preprocessing path without a
|
||
host Python or Node runtime. Use a fresh operator root and a fresh fixture Git remote; never reuse
|
||
the automated `.artifacts/p2-integration/**` state.
|
||
|
||
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
|
||
|
||
```bash
|
||
tht --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --resume <run-id> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace schema check --workspace <id> --annotations <reviewed>.yaml --reviewed-candidates <sha256:hex> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace index-schema --workspace <id> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --json
|
||
```
|
||
|
||
Checks:
|
||
|
||
1. installation/render preflight (`inspect` returns exact revision + catalog/descriptor digests);
|
||
2. DWH introspection+LSH succeeds, rerun is `unchanged`, `--resume <run-id>` is `unchanged`/`succeeded`;
|
||
3. `schema suggest-fks` returns pristine JSON with `suggestedFksYaml` and a `manual_review_required`
|
||
block (exit 3) when candidates exist; the suggested YAML digest equals the reported digest;
|
||
4. `schema check --annotations <reviewed> --reviewed-candidates <digest>` succeeds after review;
|
||
5. `index-schema` counts against a pre-provisioned compatible collection and rerun is `unchanged`;
|
||
6. HTTP Evidence `--dry-run` returns `dry_run`, the real run publishes, rerun is `unchanged`, an input
|
||
mutation produces a new generation/ACTIVE;
|
||
7. filesystem Evidence returns a stable `evidence_materialization_required` block with no partial
|
||
corpus/vector publication;
|
||
8. negatives: missing workspace (`workspace_not_activatable`), resume of a nonexistent run
|
||
(`preprocessing_resume_mismatch`), invalid annotations digest (`annotation_invalid`), no-Evidence
|
||
skip warning, no collection creation, no backend/Pi/frontend listener;
|
||
9. secret scan over retained artifacts and exact owned-resource cleanup.
|
||
|
||
Decision: **PENDING** (independent manual gate; automation never records PASS).
|
||
|
||
## P3 — Effective configuration and `.tht-dwh`
|
||
|
||
**Status:** P3 implementation complete; automated integration PASS; manual acceptance PASS (owner approval 2026-08-13).
|
||
|
||
Manual goal: prove that the operator CLI and application sessions derive the same effective
|
||
configuration, that a content-only revision reuses the prepared DWH generation (fast, `unchanged`),
|
||
that a DWH-affecting change fails closed and regenerates, that the workspace memory migration is
|
||
safe, and that search records are revision-scoped. See `docs/contracts/tht-dwh.md`.
|
||
|
||
Checks:
|
||
|
||
1. run `tht ... workspace preprocess dwh` twice with only an Evidence/content change between
|
||
them: the second run reports `unchanged` and does not re-introspect;
|
||
2. change a DWH-affecting field (host/port/database/schema/user/collection) in the descriptor,
|
||
push, pull: the next run refuses the old generation and regenerates, with a clear
|
||
`effective_config_mismatch`-style outcome and no mixed artifacts;
|
||
3. inspect `.tht-dwh` generations: immutable directories, `OWNER.json` with the canonical
|
||
fingerprints, `ACTIVE` pointer; old generations still present;
|
||
4. memory: after the guarded migration the workspace uses
|
||
`<dataRoot>/sessions/<workspace-id>/memory/`; the JSONL registry and Qdrant projection are
|
||
rebuilt and consistent; a conflicting legacy registry fails closed;
|
||
5. search records: schema/Evidence points carry the pinned `workspace_revision`; memory/solved
|
||
records remain workspace-wide;
|
||
6. documentation: `docs/contracts/tht-dwh.md` matches the observed behavior.
|
||
|
||
Decision: **PASS** (owner approval 2026-08-13).
|
||
## P4 — Qdrant bootstrap and guarded rebuild
|
||
|
||
**Status:** superseded by the "P4 Qdrant collection lifecycle" section below (implemented; manual acceptance PASS).
|
||
|
||
Manual goal: prove admission creates a missing compatible collection and indexes, refuses an
|
||
incompatible collection, and permits destructive rebuild only under durable maintenance with no
|
||
active readers/jobs and exact repeated confirmation.
|
||
|
||
Checks to fill during P4:
|
||
|
||
1. missing-collection self-heal;
|
||
2. missing-index self-heal;
|
||
3. dimensions/distance/index-type refusal;
|
||
4. confirmation mismatch refusal;
|
||
5. active-reader/job refusal;
|
||
6. successful drained rebuild;
|
||
7. interrupted rebuild recovery with maintenance retained.
|
||
|
||
Decision: **PASS** (owner approval 2026-08-13; see the section below).
|
||
|
||
|
||
## P4 Qdrant collection lifecycle
|
||
|
||
Manual goal: verify admission self-heal and the guarded rebuild through the real product surface.
|
||
|
||
Checks to complete during P4 manual acceptance (decision: **PASS** (owner approval 2026-08-13)):
|
||
|
||
1. On a fresh installation with no Qdrant collection, a session admission creates the
|
||
descriptor collection with exactly 1024 dimensions, cosine distance, and the 8 required
|
||
keyword payload indexes (`content_hash`, `document_id`, `kind`, `record_key`,
|
||
`record_kind`, `vector_generation`, `workspace_id`, `workspace_revision`).
|
||
2. A pre-existing collection with incompatible dimensions/distance (e.g. 768-dim or dot)
|
||
is refused with `semantic_index_incompatible` and is never mutated.
|
||
3. `tht ... workspace vector inspect --workspace <id> --json` reports the collection
|
||
contract without mutation (pristine JSON, exit 0).
|
||
4. `tht ... workspace vector rebuild --workspace <id> --collection <name>
|
||
--confirm <name> --destroy` deletes and recreates the descriptor-owned collection and
|
||
verifies the recreated contract; a mismatched `--confirm` or a missing `--destroy` is
|
||
refused (exit 2) without touching the collection.
|
||
5. Rebuild writes durable state before deletion, deletes only the descriptor collection,
|
||
and the recreated collection preserves the P3 revision-scoped payload contract.
|
||
|
||
## P5 — Curated FK annotations in Git
|
||
|
||
**Status:** P5 implementation complete; automated integration PASS; manual acceptance PASS (owner approval 2026-08-13).
|
||
|
||
Manual goal: curate `<id>/schema/annotations.yaml` in an author clone, publish it, pull the new
|
||
revision, and prove the revision-pinned sync and the explicit `schema accept` review, without ever
|
||
pushing curated content from the operator CLI.
|
||
|
||
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
|
||
|
||
```bash
|
||
tht --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
|
||
# curate the candidate into <id>/schema/annotations.yaml in the author clone, then commit/push/pull
|
||
tht --installation <abs>/thothii-installation.yaml workspace schema accept --workspace <id> --run <run-id> --yes --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --resume <run-id> --json
|
||
```
|
||
|
||
Checks:
|
||
|
||
1. `schema suggest-fks` returns pristine JSON with `suggestedFksYaml` and a `manual_review_required`
|
||
block (exit 3) when candidates exist; the suggested YAML digest equals the reported digest;
|
||
2. after commit/push/pull, activation reads `<id>/schema/annotations.yaml` as a regular Git blob at
|
||
the same commit as the descriptor and synchronizes it to
|
||
`<data>/sessions/<id>/revisions/<commit>/artifacts/mschema/annotations.yaml` with a restrictive
|
||
mode and an adjacent ownership manifest `{ workspace, commit, blobId, contentDigest, destination }`;
|
||
3. two revisions write two different directories; a session pinned to an older revision reads its own
|
||
revision's annotations;
|
||
4. `schema accept --run <id> --yes` records the accepted candidate/current-blob digests and the new
|
||
revision; missing `--yes`, an unknown run, an empty file, a malformed blob, or a blob not matching
|
||
the recorded candidate is refused (exit 1, `annotation_invalid`) without recording a review;
|
||
5. `preprocess run --resume <id>` continues only with the exact accepted blob digest and compatible
|
||
DWH binding; otherwise it records a new `manual_review_required` checkpoint;
|
||
6. negatives: symlink/tree-at-path, cross-namespace, oversized (>16 MiB), non-UTF-8, and malformed
|
||
annotation objects are refused at activation without mutating the snapshot or runtime roots;
|
||
7. the operator CLI never stages/commits/pushes curated content; secret scan and exact owned-resource
|
||
cleanup pass.
|
||
|
||
Decision: **PASS** (owner approval 2026-08-13).
|
||
## P6 — Commit-addressed Evidence materialization
|
||
|
||
**Status:** P6 implementation complete; automated integration PASS; manual acceptance PASS (owner approval 2026-08-13).
|
||
|
||
Manual goal: materialize filesystem Evidence from the pinned Git commit, inspect its bounded
|
||
manifest, preprocess/index it, retrieve only the pinned revision, and exercise unsafe-tree and
|
||
aggregate-limit failures without partial publication.
|
||
|
||
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
|
||
|
||
```bash
|
||
tht --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
|
||
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json # idempotent rerun
|
||
```
|
||
|
||
Checks:
|
||
|
||
1. exact commit/tree/object identities: after activation the materialized root is
|
||
`<registry>/snapshots/<commit>/<id>/evidence` and its sibling manifest
|
||
`<id>/evidence.manifest.json` records `workspace`, `commit`, `tree`, per-file `oid`/`digest`,
|
||
`entryCount`, `totalBytes`; `snapshot.json` chains the manifest digest;
|
||
2. successful atomic materialization: every regular blob is present byte-for-byte; the manifest
|
||
digests match;
|
||
3. manifest and file digest verification: re-activation reuses a valid root and fails closed on a
|
||
tampered manifest;
|
||
4. filesystem Evidence dry-run/run/idempotency: `--dry-run` returns `dry_run`, the real run
|
||
publishes, rerun is `unchanged`;
|
||
5. revision-filtered Qdrant retrieval and corpus ACTIVE: Evidence records carry the pinned
|
||
`workspace_revision`;
|
||
6. nested symlink/gitlink/traversal/special-file refusal: a commit introducing one of these fails
|
||
activation (`workspace_invalid`) and the previous valid revision stays active;
|
||
7. file-count/total-byte/path/manifest limit refusal: an oversized or over-count tree fails closed
|
||
without a partial publication;
|
||
8. retention while pinned and owned cleanup after release: the materialized root persists for a
|
||
pinned revision and is removed with its snapshot directory once unreferenced.
|
||
|
||
Decision: **PASS** (owner approval 2026-08-13).
|
||
## Final aggregate P2–P6 verification
|
||
|
||
**Status:** runnable; automated integration PASS; manual acceptance PENDING.
|
||
|
||
The automated aggregate (run `p2p6-ee542112c526ef0d4c25ddf6c8bc164b`, report
|
||
`.artifacts/p2p6-integration/...`) already executed the complete DWH → FK → schema → filesystem
|
||
Evidence chain, idempotency, revision isolation, a second installation, unsafe-tree/bound negatives,
|
||
secret scan, and exact cleanup.
|
||
|
||
The final manual pass will start with a new registry and two independent installations. It will
|
||
run the complete DWH → FK → schema → filesystem Evidence chain, prove idempotency and revision
|
||
isolation, confirm the second installation uses its own secrets/state, and compare its observations
|
||
to the retained aggregate automated report.
|
||
|
||
Decision: **PENDING**.
|