# Workspace preprocessing CLI contract

`tht` is the only supported host entrypoint for workspace preprocessing.

## Invocation

```text
tht --installation <absolute>/thothii-installation.yaml workspace inspect
  --workspace <id> [--json]

tht --installation <absolute>/thothii-installation.yaml workspace preprocess dwh
  --workspace <id> [--resume <32hex>] [--json]

tht --installation <absolute>/thothii-installation.yaml workspace schema suggest-fks
  --workspace <id>
  [--from-sql <regular-file>]... [--assume <column=table>]...
  [--output <new-file>] [--json]

tht --installation <absolute>/thothii-installation.yaml workspace schema check
  --workspace <id>
  [--annotations <regular-file> --reviewed-candidates <sha256:hex>]
  [--json]

tht --installation <absolute>/thothii-installation.yaml workspace schema accept
  --workspace <id> --run <32hex> --yes [--json]

tht --installation <absolute>/thothii-installation.yaml workspace index-schema
  --workspace <id> [--json]

tht --installation <absolute>/thothii-installation.yaml workspace preprocess evidence
  --workspace <id> [--dry-run] [--resume <32hex>] [--json]

tht --installation <absolute>/thothii-installation.yaml workspace preprocess run
  --workspace <id> [--resume <32hex>] [--json]

tht --installation <absolute>/thothii-installation.yaml workspace vector inspect
  --workspace <id> [--json]

tht --installation <absolute>/thothii-installation.yaml workspace vector rebuild
  --workspace <id> --collection <name> --confirm <name> --destroy [--json]
```

## Qdrant collection lifecycle (P4)

- `workspace vector inspect` reports the descriptor-owned Qdrant collection contract
  (name, dimensions, distance, keyword indexes) **without mutation**.
- `workspace vector rebuild` deletes and recreates the descriptor-owned collection
  with the exact contract (1024 dimensions, cosine distance, the 8 required keyword
  payload indexes) under guards:
  - `--collection <name>` must equal the descriptor's `semantic_index.vector_store.collection`;
  - `--confirm <name>` must equal `--collection` (exact repetition);
  - `--destroy` is required to confirm the destructive operation;
  - the operator refuses any other combination with exit code 2 (usage).
- Self-heal at session admission: a missing collection is created and missing
  keyword indexes are added by the shared collection manager; incompatible
  dimensions/distance/index types are never mutated (`semantic_index_incompatible`).
- The operator path (`workspace-maintenance.js vector-inspect|vector-rebuild`)
  performs the guarded rebuild; rebuild state is written before deletion and the
  collection is verified after recreation. No prefix matching or global Qdrant
  mutation is performed.

## Additive BM25 for Evidence

- Only `workspace preprocess evidence` (and the Evidence portion of `workspace preprocess run`)
  may add the named sparse vector `bm25` with Qdrant modifier `idf`.
- The upgrade uses Qdrant's additive named-vector operation. It preserves the existing unnamed
  dense vector and never deletes, renames, or rebuilds the shared collection.
- Session readiness remains read-only with respect to BM25. Schema, Memory, and solved-question
  records therefore continue to use their existing dense-only points during and after an Evidence
  upgrade.
- A missing `bm25` is added and reread before Evidence preprocessing starts. An existing definition
  other than `modifier: idf` fails as `semantic_index_incompatible` without any collection mutation.
  If a later Evidence candidate fails, the compatible additive schema remains in place; it does not
  make the dense-only records unavailable.
```

## Curated FK annotations (P5)

- The canonical curated annotations file is `<workspace-id>/schema/annotations.yaml`, a regular
  Git blob at the same commit as the descriptor. Absence is compatible (empty canonical set +
  warning); symlinks, trees/gitlinks, oversized (>16 MiB), non-UTF-8, and malformed objects are
  refused at activation.
- Activation synchronizes the blob to the immutable revision-qualified root
  `/data/sessions/<id>/revisions/<commit>/artifacts/mschema/annotations.yaml` with a restrictive
  mode and an adjacent ownership manifest `{ workspace, commit, blobId, contentDigest,
  destination }`. Pinned runtimes resolve annotations from `paths.annotations_root`.
- `workspace schema accept --run <id> --yes` is the only human FK review primitive: after
  commit/push/pull, it reads the current synced blob, validates it with the harness parser against
  the physical schema and the recorded candidate digest, and records
  `{ reviewedCandidatesDigest, annotationsDigest, workspaceRevision, blobId }`. `--yes` is
  required; an empty file, an unknown run, a malformed blob, or a non-matching candidate fails
  closed without recording a review. `schema check` alone is not evidence of human review.
- `preprocess run` continues only with the exact accepted blob digest and a compatible reusable
  DWH binding; otherwise it records a new review checkpoint.

## Commit-addressed Evidence materialization (P6)

- Filesystem Evidence `<workspace-id>/evidence` is materialized from the exact pinned Git commit
  into the immutable revision content root `<registry>/snapshots/<commit>/<id>/evidence` at
  activation, with a sibling bounded manifest `<id>/evidence.manifest.json` whose digest is chained
  into `snapshot.json`.
- Materialization uses fixed Git plumbing (`ls-tree -r -z` + `cat-file blob`) and refuses symlinks
  and gitlinks at any depth, traversal/absolute/duplicate/cross-namespace paths, and non-regular
  modes. Installation-local limits bound entry count (default 4096), total bytes (64 MiB),
  per-file bytes (8 MiB), path bytes (4096), and manifest bytes (1 MiB); a size-sum preflight runs
  before any bytes are written and no partial root is published.
- `preprocess evidence` and `preprocess run` operate directly on the materialized root; the
  temporary `evidence_materialization_required` stop is retired (the code remains only for
  pre-P6 compatibility). For `evidence.schema_version: 2`, runtime acquisition receives exactly
  `curated/**/*.md`; `source/` and support files remain in the materialized tree for traceability.
  HTTP/S3 Evidence is unchanged.
- The curator validates Evidence before merge. Preprocessing validates the pinned curated corpus
  again before it constructs a candidate generation, so an invalid revision is never indexed.
- The runtime writes only its immutable materialized snapshot and derived index state. It never
  writes, stages, commits, or pushes the workspace authoring repository.
- Materialized roots are retained with their commit-addressed snapshot directory and removed only
  when the revision becomes unreferenced.

## Validation

- `--installation` is mandatory and absolute.
- `--workspace` is mandatory exactly once and must match `[a-z][a-z0-9-]{2,62}`.
- `--resume` values must be 32 lowercase hex characters.
- `--json` may be supplied once.
- `schema suggest-fks`
  - allows at most 32 `--from-sql` files;
  - each SQL file must be a canonical regular file, UTF-8, non-symlink, max 1 MiB;
  - total SQL ingress must not exceed 16 MiB;
  - allows at most 256 `--assume` values, each `column=table`, max 256 bytes;
  - `--output` must name a new canonical path; existing targets are refused.
- `schema check`
  - `--annotations` and `--reviewed-candidates` are all-or-nothing;
  - annotations must be UTF-8, canonical, non-symlink, max 16 MiB;
  - `--reviewed-candidates` must match `sha256:<64 lowercase hex>`.
- `schema accept`
  - `--run` is mandatory and must be 32 lowercase hex characters;
  - `--yes` is mandatory and may be supplied once;
  - `--annotations`/`--reviewed-candidates`/`--from-sql`/`--assume` are not accepted.
- Unknown flags, passthrough separators, and shell fragments are rejected before Docker runs.

## Container boundary

`tht` resolves the selected `core` image from the rendered installation, converts it to an immutable local image ID, writes a one-shot final override that pins both `core` and `workspace-maintenance` to that ID with `pull_policy: never`, and runs only:

```text
docker compose run --rm --no-deps --no-TTY --name <owned-name> workspace-maintenance <fixed-command>
```

The request is streamed as one schema-versioned JSON document over stdin. Public stdout is always one schema-versioned JSON result; human mode is rendered from an allowlisted subset of that same result.

## Public JSON result

```json
{
  "schemaVersion": 1,
  "status": "succeeded|unchanged|dry_run|blocked|failed",
  "code": "ok|workspace_not_found|workspace_not_activatable|binding_missing|preprocessing_conflict|preprocessing_resume_mismatch|manual_review_required|evidence_materialization_required|effective_config_mismatch|semantic_index_incompatible|annotation_invalid|egress_policy_refused",
  "workspaceId": "abc",
  "workspaceRevision": "1234567890abcdef1234567890abcdef12345678",
  "descriptorBlob": "sha256:<64 lowercase hex>",
  "operation": "inspect|preprocess-dwh|schema-suggest-fks|schema-check|index-schema|preprocess-evidence|preprocess-run",
  "runId": "<optional 32hex>",
  "childRuns": {"stage": "<optional 32hex>"},
  "completedStages": ["stage"],
  "counts": {"name": 1},
  "artifactIdentities": [{"kind": "fk_candidates", "digest": "sha256:<64 lowercase hex>"}],
  "warnings": ["safe warning"]
}
```

`tht --json` parses the operator stdout strictly and re-encodes only the public fields above.

`evidence_materialization_required` is retained for pre-P6 compatibility; since P6, filesystem
Evidence is materialized at activation and preprocesses directly.

## Exit codes

- `0`: `succeeded`, `unchanged`, or `dry_run`
- `3`: `blocked`
- `2`: host-side grammar or local file safety failure
- `1`: operational failure or operator-reported `failed`