diff --git a/docs/superpowers/plans/2026-08-09-prd-p1-descriptor-evidence.md b/docs/superpowers/plans/2026-08-09-prd-p1-descriptor-evidence.md new file mode 100644 index 00000000..d377668d --- /dev/null +++ b/docs/superpowers/plans/2026-08-09-prd-p1-descriptor-evidence.md @@ -0,0 +1,1697 @@ +# P1 Descriptor Evidence Configuration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Implement P1/D1 so a schema-v3 workspace can declare a complete, non-secret Evidence source and policy, bind any source credentials from installation-local files, preserve descriptor/source identity at one Git revision, and render a harness configuration that passes `tht config check`. + +**Architecture:** The canonical descriptor stays in `workspaces/.yaml` inside the shared registry repository. Filesystem Evidence is named by a lexical repo-relative URI under `workspace-content//evidence`; the registry verifies that the tree and descriptor exist in the same commit, while P6—not P1—will materialize and realpath-check that tree. The backend derives source-specific installation bindings, immutable revision identity, `evidence.sources`, and vector policy into the same runtime YAML already used by sessions; the harness parses file-backed HTTP/S3 credentials without acquiring content. P1 is proven first by a clean-state, real-Git/real-HTTP automated process and then by an independent manual artifact walkthrough. + +**Tech Stack:** TypeScript 5, Zod 4, Fastify 5, Git CLI, YAML, React 18, Python 3.12, Pydantic 2, pytest, Vitest, Node.js 22, Bash. + +**Source PRD:** `docs/prd/2026-08-09-workspace-preprocessing-prd.md` v0.5 (`208b299`), especially RF1, RNF1–RNF8, D1, D6, D8, D9, P1, and “Standard di verifica obbligatorio del futuro piano P1”. + +## P1 completion contract + +P1 implements D1 and the P1-owned prerequisites of RF1; it does not claim all of RF1 complete. In particular, P2 owns the backend-independent host renderer/preprocessing consumer needed to finish RF1.3, and P10 owns operational `ssh_tunnel` support for RF1.4. + +P1 is complete only when all of the following are true: + +1. Schema v3 accepts an optional strict `evidence` section. Absence remains operational for existing workspaces (RNF3); P2 will warn when preprocessing is requested without Evidence. +2. `filesystem`, `http`, and `s3` are supported descriptor source types. The descriptor contains source identity and non-secret behavior only. +3. Filesystem URI is exactly the workspace Evidence root, `workspace-content//evidence`, using normalized POSIX segments. It is checked lexically in schema validation and checked as a Git tree at the same immutable commit during publication/activation. +4. HTTP query-bearing signed URLs and S3 static credentials enter only through installation-local `*_FILE` bindings. Public descriptor HTTP URIs never contain userinfo, query, or fragment. No secret file content is emitted into Git, registry docs, exports, API errors, reports, or rendered YAML. +5. The existing backend production renderer emits a harness-compatible `evidence.sources` entry plus `vector.max_chunk_chars` and `vector.retain_published_generations`, under the same `runtime_identity.workspace_revision` used by sessions. P1 proves that backend-runtime half of RF1.3; P2 must provide and prove a backend-independent host-CLI render path before claiming preprocessing uses the same effective config. +6. Existing frontend workspace flows parse, preserve, conflict-resolve, and re-publish Evidence. P1 adds only a read-only summary, not a new preprocessing or Evidence-authoring UI. +7. The automated gate passes from clean state through local Git → registry → real HTTP → snapshot/docs/export → production render → real `tht config check`, with deterministic outputs, negative cases, secret scanning, inspectable reports, and ownership-confined cleanup. +8. The manual gate uses entirely new state and remains `PENDING` until a human reviewer records approval. + +## Canonical descriptor contract + +Use this exact filesystem form in the generic examples and acceptance fixture: + +```yaml +evidence: + source: + type: filesystem + uri: workspace-content/example/evidence + patterns: + - "**/*.md" + max_bytes: 10485760 + policy: + max_chunk_chars: 4000 + retain_published_generations: 3 +``` + +HTTP is an explicit, non-secret manifest. `authentication: signed_urls_file` requires one installation file containing a JSON array of signed transport URLs in the same order as `uris`; the harness must verify that stripping each transport URL's query produces the declared URI before it accepts the config. + +```yaml +evidence: + source: + type: http + uris: + - https://evidence.example.test/guide.md + authentication: signed_urls_file # or none + connect_timeout_ms: 5000 + read_timeout_ms: 30000 + max_bytes: 10485760 + max_redirects: 5 + allow_private_hosts: false + max_cache_bytes: 67108864 + policy: + max_chunk_chars: 4000 + retain_published_generations: 3 +``` + +S3 uses one canonical `s3://` URI. `credentials: static_files` requires access-key and secret-key files and permits an optional session-token file; `ambient` delegates to the installation's AWS-compatible provider chain. + +```yaml +evidence: + source: + type: s3 + uri: s3://evidence-bucket/example/ + region: eu-west-1 + credentials: static_files # or ambient + trusted_endpoint: false + allow_private_endpoint: false + allow_insecure_endpoint: false + max_bytes: 10485760 + max_objects: 10000 + max_pages: 100 + page_size: 1000 + policy: + max_chunk_chars: 4000 + retain_published_generations: 3 +``` + +The implementation must preserve these existing engine defaults exactly: + +| Field | Default | +|---|---:| +| filesystem `patterns` | `["**/*.md"]` | +| per-object `max_bytes` | `10 * 1024 * 1024` | +| HTTP connect/read timeout | `5000 ms` / `30000 ms` | +| HTTP redirects/cache | `5` / `64 * 1024 * 1024` bytes | +| S3 objects/pages/page size | `10000` / `100` / `1000` | +| `max_chunk_chars` | `4000` | +| `retain_published_generations` | `3` | + +P1 deliberately covers the **configuration-only** path for all three already-supported engine source types: strict declaration, file-path binding, render, and parse. This is required by RF1.1/D1's “sorgente completa” and the agreed three-source P1 scope. D6's “HTTP/S3 future” boundary still applies to network acquisition, materialization, preprocessing, and operational acceptance: P1 never calls either adapter. The signed-URL loader is the narrow bridge needed to keep authenticated transport values out of Git; it is not a new HTTP auth/header protocol. + +Installation variables are derived only when the selected source mode needs them: + +```text +THT_WS__EVIDENCE_SIGNED_URLS_FILE +THT_WS__EVIDENCE_ACCESS_KEY_FILE +THT_WS__EVIDENCE_SECRET_KEY_FILE +THT_WS__EVIDENCE_SESSION_TOKEN_FILE # optional +``` + +`` uses the existing `contractNamespace` normalization. These values are absolute file paths below configured secret roots, never secret values. + +## Hard scope boundaries + +- Do **not** run `tht preprocess evidence`, `tht evidence extract`, adapter discovery/acquisition, embeddings, Qdrant writes, ACTIVE publication, corpus GC, or retention execution. +- Do **not** create or validate `artifacts/evidence`, `corpus/ACTIVE`, evidence points, or embedding vectors. Those belong to P2/P4/P6/P8/P9. +- Do **not** copy the Evidence tree into current immutable descriptor snapshots. P6 owns commit-addressed materialization, realpath checks, nested symlink escape rejection, and race-safe consumption. +- Do **not** resolve a filesystem source against the mobile registry checkout. P1 renders the reserved future path `/snapshots//workspace-content//evidence`; `tht config check` validates structure without requiring that path to exist. +- Do **not** broaden `writeRegistryFile`/`commitAndPush` to arbitrary `workspace-content` writes. Curators change Evidence content through a normal Git clone; the API publishes descriptors and generated docs only. +- Do **not** add a browser preprocessing endpoint or host render/preprocessing CLI. The backend-independent host CLI and its equivalence proof are P2; full Evidence editor UX is future work. +- Do **not** make `ssh_tunnel` operational. Preserve the renderer's fail-closed behavior until P10. +- Do **not** claim same-revision identity from descriptor blob equality. A content-only Evidence commit has the same descriptor blob but a different authoritative commit. + +## Target file map + +**Backend contract and registry** + +- `backend/src/workspaces/schema.ts`: authoritative Evidence interfaces, Zod schemas, and cross-field invariants. +- `backend/src/workspaces/types.ts`: remove or synchronize the duplicate exported v3 shape so it cannot contradict the authoritative contract. +- `backend/src/workspaces/git-repository.ts`: fixed-argv read-only tree-at-revision assertion. +- `backend/src/workspaces/registry.ts`: contextual filesystem tree checks at publish/activate and content-only revision behavior. +- `backend/src/workspaces/contracts.ts`: Evidence installation variables and generated public README. +- `backend/src/workspaces/bindings.ts`: source-specific Evidence file binding resolution. +- `backend/src/workspaces/diagnostics.ts`: report installation-local `binding_missing` when required Evidence files are missing/unsafe. +- `backend/src/workspaces/runtime-renderer.ts`: runtime `evidence.sources` and vector policy mapping. +- `backend/src/tht/tht-runner.ts`: commit-addressed render context. +- `backend/src/routes/workspaces.ts`: validation/read/publish/import/export round-trip and safe errors. + +**Harness compatibility** + +- `harness/tht/config.py`: strict Evidence runtime config, signed-URL file resolution, provenance validation. +- `harness/tht/adapters/factory.py`: consume validated HTTP transport URLs without exposing them. +- `harness/tests/test_config_resources.py`: source/policy parsing and secret masking. +- `harness/tests/test_registry_evidence_config.py`: renderer-facing runtime contract and config-check behavior. + +**Frontend compatibility** + +- `frontend/src/api/workspaces.ts`: canonical Evidence types and conflict field allowlist. +- `frontend/src/workspaces/drafts.ts`: strict sanitizer/copy functions that preserve Evidence. +- `frontend/src/shell/WorkspaceEditor.tsx`: read-only Evidence summary; existing edits preserve the object. + +**Examples, docs, and gates** + +- `deploy/workspaces/example.yaml`, `deploy/workspaces/psd.yaml.example`: generic descriptor examples. +- `docs/contracts/workspace-evidence-v3.md`: human-readable canonical contract. +- `docs/install/local-workspace-registry.md`, `docs/install/server-workspace-registry.md`: operator registry layout and secret boundary. +- `docs/install/examples/workspace-bindings.env.example`: file-path binding examples only. +- `scripts/verify-workspace-install-docs.sh`, `scripts/test-verify-workspace-install-docs.sh`: executable doc contract. +- `scripts/p1-acceptance.sh`, `backend/scripts/p1-acceptance.mjs`, `scripts/test-p1-acceptance.sh`: automated process goal. +- `scripts/p1-manual-acceptance.sh`, `backend/scripts/p1-manual-acceptance.mjs`, `backend/scripts/p1-render-snapshot.mjs`, `docs/testing/p1-manual-acceptance.md`: manual walkthrough and explicit production-render tooling. +- `PROJECT_STATE.md`: separate automated/manual status and retained artifact path. + +--- + +### Task 1: Define the optional, strict schema-v3 Evidence contract + +**Files:** +- Modify: `backend/src/workspaces/schema.ts` +- Modify: `backend/src/workspaces/types.ts` +- Modify: `backend/test/workspaces-schema.test.ts` +- Modify: `backend/test/workspaces-migrate-v2-qdrant.test.ts` +- Modify: `backend/test/workspaces-migrate-legacy.test.ts` + +**Interfaces:** + +```ts +export interface EvidencePolicy { + max_chunk_chars: number; + retain_published_generations: number; +} + +export type EvidenceSource = + | { + type: "filesystem"; + uri: string; + patterns: string[]; + max_bytes: number; + } + | { + type: "http"; + uris: string[]; + authentication: "none" | "signed_urls_file"; + connect_timeout_ms: number; + read_timeout_ms: number; + max_bytes: number; + max_redirects: number; + allow_private_hosts: boolean; + max_cache_bytes: number; + } + | { + type: "s3"; + uri: string; + endpoint_url?: string; + region?: string; + credentials: "ambient" | "static_files"; + trusted_endpoint: boolean; + allow_private_endpoint: boolean; + allow_insecure_endpoint: boolean; + max_bytes: number; + max_objects: number; + max_pages: number; + page_size: number; + }; + +export interface WorkspaceEvidence { + source: EvidenceSource; + policy: EvidencePolicy; +} +``` + +`WorkspaceV3` gains `evidence?: WorkspaceEvidence`; v1/v2 remain strict and unchanged. Prefer deleting the unused duplicate `WorkspaceV2`/`WorkspaceV3` declarations from `backend/src/workspaces/types.ts` and importing the authoritative schema types wherever needed. If a public compatibility reason prevents deletion, re-export the schema types instead of maintaining a second handwritten structure. + +- [ ] **Step 1: Write failing schema tests** + +Add table-driven positive tests for: + +- filesystem with explicit values; +- filesystem with all defaults applied; +- HTTP `none` and `signed_urls_file` modes; +- S3 `ambient` and `static_files` modes; +- `evidence` absent on a valid v3 workspace; +- parse → canonical object → `serializeWorkspaceYaml` → parse equality. + +Add table-driven negative tests for: + +- absolute filesystem paths, `..`, `.`, empty/doubled segments, trailing traversal, backslashes, NUL/control characters, and another workspace's namespace; +- a filesystem URI above or below the canonical root (patterns select descendants; URI itself is the exact root); +- unsupported source discriminators and unknown keys; +- credential-shaped Git fields such as `password`, `api_key`, `access_key`, `secret_key`, `session_token`, `signed_url`, `headers`, and `ca_contents`; +- HTTP userinfo, query, fragment, non-HTTP schemes, duplicates after canonicalization, empty manifests, and invalid bounds; +- invalid S3 schemes, empty bucket, userinfo/query/fragment, unsafe endpoint syntax, inconsistent endpoint opt-ins, and invalid limits; +- chunk/retention values outside explicit bounds; +- `evidence` on schema v1/v2. + +Use explicit canaries in tests and assert the resulting public error message contains a safe field path but not the canary value. + +- [ ] **Step 2: Run the focused test and verify RED** + +Run: + +```bash +cd backend +npx vitest run test/workspaces-schema.test.ts +``` + +Expected: FAIL because `WorkspaceV3Schema` rejects `evidence` and the types do not exist. + +- [ ] **Step 3: Implement strict source schemas and defaults** + +In `schema.ts`, create `.strict()` Zod objects and one discriminated union. Use safe-integer validation while preserving the engine's existing lower-bound semantics: + +```ts +const EvidencePolicySchema = z.object({ + max_chunk_chars: z.number().int().safe().positive().default(4_000), + retain_published_generations: z.number().int().safe().min(1).default(3), +}).strict(); +``` + +Define source defaults exactly as listed in the completion contract. Default the whole `policy` object to `{ max_chunk_chars: 4000, retain_published_generations: 3 }`, so the canonical parsed object and serialized YAML are explicit even when the author omitted policy fields. Convert descriptor milliseconds to harness seconds only in the renderer; keep integer milliseconds in Git. Validate URLs with `URL`, but return sanitized Zod issues that identify the field/index rather than interpolating the rejected URL. + +Add small, named helpers used by `workspaceInvariants`, for example: + +```ts +function expectedEvidenceRoot(workspaceId: string): string { + return `workspace-content/${workspaceId}/evidence`; +} + +function isNormalizedRepoRelativePath(value: string): boolean { + const parts = value.split("/"); + return value.length > 0 + && !value.startsWith("/") + && !value.includes("\\") + && !/[\u0000-\u001f\u007f]/u.test(value) + && parts.every((part) => part !== "" && part !== "." && part !== ".."); +} +``` + +The cross-field issue must be attached to `evidence.source.uri` and require exact equality with `expectedEvidenceRoot(workspace.workspace.id)`. Do not call `resolve`, `realpath`, or inspect the filesystem here. Require a nonempty, stably deduplicated `patterns` list; each glob must be relative, slash-normalized, free of empty/`.`/`..` segments, backslashes, and control characters, so a glob cannot escape the declared root. + +For HTTP, reject userinfo/query/fragment in descriptor URIs and reject duplicates after a stable canonical form. For S3, parse the `s3://` URI into a nonempty bucket and optional prefix, and keep `endpoint_url`, region, trust flags, and limits non-secret. + +- [ ] **Step 4: Preserve migration behavior** + +Legacy and v2→v3 migration output must omit `evidence`, not invent a filesystem tree. Add assertions to both migration test files. This makes migrated workspaces operational for existing sessions but not yet configured for preprocessing. + +- [ ] **Step 5: Run focused tests and typecheck** + +Run: + +```bash +cd backend +npx vitest run \ + test/workspaces-schema.test.ts \ + test/workspaces-migrate-v2-qdrant.test.ts \ + test/workspaces-migrate-legacy.test.ts +npx tsc --noEmit -p . +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add backend/src/workspaces/schema.ts backend/src/workspaces/types.ts \ + backend/test/workspaces-schema.test.ts \ + backend/test/workspaces-migrate-v2-qdrant.test.ts \ + backend/test/workspaces-migrate-legacy.test.ts +git commit -m "feat: define workspace evidence descriptor contract" +``` + +--- + +### Task 2: Bind filesystem declarations to a Git tree at the same revision + +**Files:** +- Modify: `backend/src/workspaces/git-repository.ts` +- Modify: `backend/src/workspaces/registry.ts` +- Modify: `backend/test/workspaces-git-repository.test.ts` +- Modify: `backend/test/workspace-registry.test.ts` + +**Interfaces:** + +```ts +// Read-only. It never stages, checks out, or follows worktree symlinks. +GitWorkspaceRepository.assertTreeAtRevision( + revision: string, + repoRelativePath: string, +): Promise +``` + +The helper invokes Git with a fixed argv, equivalent to: + +```text +git cat-file -t <40-hex-commit>:workspace-content//evidence +``` + +and accepts only output `tree`. A missing path, blob/symlink at the declared root, malformed revision, or Git failure becomes a sanitized `workspace_invalid`/`git_unavailable` registry error without command stderr or source content. + +- [ ] **Step 1: Write failing Git repository tests** + +Extend the existing `temporaryRemote()`-based suite. Seed one commit with: + +```text +workspaces/research.yaml +workspace-content/research/evidence/guide.md +workspace-content/other/evidence/other.md +``` + +Test that the helper: + +- accepts the `research` Evidence tree at that commit; +- rejects a missing path; +- rejects a blob and a Git symlink at the declared root; +- distinguishes old/new commits after a content-only Evidence change; +- uses an argv array and never passes the path through a shell. + +Do not recursively reject a symlink nested inside the tree. Add an explicit test/comment that nested containment is intentionally deferred to P6. + +- [ ] **Step 2: Run the Git test and verify RED** + +```bash +cd backend +npx vitest run test/workspaces-git-repository.test.ts +``` + +Expected: FAIL because `assertTreeAtRevision` is missing. + +- [ ] **Step 3: Implement the fixed-argv read-only helper** + +Reuse the repository's existing Git runner and revision/path safety helpers. Do not add `workspace-content` to `isRegistryArtifactPath`, `writeRegistryFile`, the publish staging allowlist, or generated API artifacts. + +- [ ] **Step 4: Write failing registry tests for contextual validation** + +Add real-bare-repository cases proving: + +1. publication succeeds only when the descriptor's filesystem tree already exists in the pulled base commit; +2. the resulting publication commit contains both the descriptor and the unchanged Evidence tree; +3. activation validates the tree against the exact `safeHead` used for the descriptor; +4. a remote descriptor with a missing/non-tree source fails pull and retains the previously active snapshot; +5. a content-only remote commit creates a new `WorkspaceRevision.commit` and immutable descriptor snapshot even when the descriptor blob is unchanged; +6. a stale API update based on the pre-content commit receives `workspace_stale`/409 semantics rather than overwriting the curator's content commit; +7. retained and pinned historical revisions remain distinguishable by commit. + +Assertions must compare commit IDs and Git object existence, not mutable checkout paths. + +- [ ] **Step 5: Run the registry test and verify RED** + +```bash +cd backend +npx vitest run test/workspace-registry.test.ts +``` + +Expected: at least the missing-tree and content-only revision cases FAIL. + +- [ ] **Step 6: Add one contextual validation function in the registry** + +Use a private helper such as: + +```ts +private async assertEvidenceContext( + workspace: WorkspaceDescriptor, + revision: string, +): Promise { + if (workspace.workspace.schema_version !== 3) return; + if (workspace.evidence?.source.type !== "filesystem") return; + await this.repository.assertTreeAtRevision(revision, workspace.evidence.source.uri); +} +``` + +Call it: + +- after pull/read of the base and before descriptor publication; +- during activation against `safeHead`, before replacing the active revision; +- during integrity repair/reload wherever an existing snapshot is re-associated with a Git commit. + +Publication still stages only `workspaces/.yaml` and generated `workspace-docs//...`. Activation still snapshots only descriptor/contract/README/manifest. Document the deliberate P6 boundary adjacent to the code. + +- [ ] **Step 7: Run focused tests** + +```bash +cd backend +npx vitest run \ + test/workspaces-git-repository.test.ts \ + test/workspace-registry.test.ts +npx tsc --noEmit -p . +``` + +Expected: PASS, including the content-only revision regression. + +- [ ] **Step 8: Commit** + +```bash +git add backend/src/workspaces/git-repository.ts backend/src/workspaces/registry.ts \ + backend/test/workspaces-git-repository.test.ts backend/test/workspace-registry.test.ts +git commit -m "feat: bind evidence trees to registry revisions" +``` + +--- + +### Task 3: Resolve HTTP/S3 credentials from installation-local files only + +**Files:** +- Modify: `backend/src/workspaces/contracts.ts` +- Modify: `backend/src/workspaces/bindings.ts` +- Modify: `backend/src/workspaces/diagnostics.ts` +- Read/verify wiring: `backend/src/app.ts` +- Modify: `backend/test/workspaces-contracts.test.ts` +- Modify: `backend/test/workspaces-bindings.test.ts` +- Modify: `backend/test/workspaces-diagnostics.test.ts` +- Modify: `backend/test/routes-sessions.test.ts` +- Modify: `harness/tht/config.py` +- Modify: `harness/tht/adapters/factory.py` +- Modify: `harness/tests/test_config_resources.py` +- Create: `harness/tests/test_registry_evidence_config.py` + +**Backend interfaces:** + +```ts +export type InstallationRole = /* existing roles */ | "EVIDENCE"; +export type InstallationSuffix = + | /* existing suffixes */ + | "SIGNED_URLS_FILE" + | "ACCESS_KEY_FILE" + | "SECRET_KEY_FILE" + | "SESSION_TOKEN_FILE"; + +export interface ResolvedEvidenceBinding { + values: Record; // safe file paths only + missing: string[]; +} + +export interface RuntimeBindings { + // existing roles... + evidence: ResolvedEvidenceBinding; +} +``` + +`resolveEvidenceBinding(workspace, env, secretRoots)` returns no variables/missing values for filesystem, HTTP `none`, or S3 `ambient`. It requires the signed-URL file for HTTP `signed_urls_file`; it requires access-key and secret-key files for S3 `static_files`, and accepts the session-token file only when present and safe. + +**Harness runtime shape for signed HTTP:** + +```yaml +evidence: + sources: + - type: http + provenance_urls: + - https://evidence.example.test/guide.md + signed_urls_file: /run/secrets/example-evidence-signed-urls + # limits follow +``` + +The file is UTF-8 JSON with a nonempty array of strings, maximum 1 MiB. `load_config` replaces the file reference in memory with `urls: list[SecretStr]`; it verifies one-to-one order and `canonical_provenance_uri(signed_url) == provenance_urls[index]`. It never stores the file contents in the Pydantic repr, validation message, CLI output, or returned public metadata. + +- [ ] **Step 1: Write failing backend contract/binding tests** + +Cover: + +- stable namespace and exact variable names; +- no Evidence variables for modes without file credentials; +- source-specific variables only (HTTP never gets S3 files and vice versa); +- HTTP signed file required; +- S3 access/secret required together; session token optional; +- absolute readable regular files under a configured realpath secret root accepted; +- relative paths, missing files, directories, unreadable files, and symlink escapes rejected as `missing`; +- binding results contain file paths, never file contents; +- generated contract/README and diagnostic errors do not contain secret canaries; +- `/workspaces/:id/test` reports local `binding_missing` without changing the canonical registry revision; +- real session admission through `buildApp`/`routes-sessions` refuses before Pi spawn when a declared required Evidence file is missing/unsafe, accepts a safe binding far enough to reach the existing next admission boundary, and preserves no-Evidence compatibility; +- schema-v3 DWH/vector/embedding behavior remains unchanged. + +- [ ] **Step 2: Run backend tests and verify RED** + +```bash +cd backend +npx vitest run \ + test/workspaces-contracts.test.ts \ + test/workspaces-bindings.test.ts \ + test/workspaces-diagnostics.test.ts \ + test/routes-sessions.test.ts +``` + +Expected: FAIL because Evidence is not an installation role and `RuntimeBindings` has no Evidence binding. + +- [ ] **Step 3: Implement conditional contract generation and binding resolution** + +Keep the existing principle from `bindings.ts`: validate paths and pass them through, but never read their contents. Add `resolveEvidenceBinding` rather than overloading the DWH/vector transport resolver with a source type that is not one of those transports. + +Update all `RuntimeBindings` construction sites/tests. `supportsSessionRuntime` must return false when a descriptor-selected Evidence credential file is missing/unsafe, while an absent Evidence section yields an empty successful binding and preserves RNF3 session compatibility. Prove the existing `app.ts` admission closure actually applies that result in `routes-sessions.test.ts`; registry publication/activation remains installation-independent and must not read local bindings. In V3 `/workspaces/:id/test` diagnostic preflight, convert missing required Evidence bindings to existing sanitized `binding_missing` diagnostics with the descriptor field (`evidence.source.authentication` or `evidence.source.credentials`) and variable name; never include an environment value. + +- [ ] **Step 4: Run backend tests and typecheck** + +```bash +cd backend +npx vitest run \ + test/workspaces-contracts.test.ts \ + test/workspaces-bindings.test.ts \ + test/workspaces-diagnostics.test.ts \ + test/routes-sessions.test.ts +npx tsc --noEmit -p . +``` + +Expected: PASS. + +- [ ] **Step 5: Write failing harness config tests** + +In `test_registry_evidence_config.py`, build raw runtime YAML for: + +- filesystem with a deliberately nonexistent absolute root (parse/check succeeds; no adapter construction); +- public HTTP URLs; +- signed HTTP with a valid JSON secret file and matching provenance; +- signed HTTP with missing/oversized/malformed/non-list files; +- signed HTTP with reordered, extra, query-free mismatch, userinfo, or duplicate canonical provenance; +- S3 ambient credentials; +- S3 `access_key_file`, `secret_key_file`, and optional `session_token_file` resolved into `SecretStr`; +- all source/policy defaults and non-default values; +- unknown Evidence keys rejected. + +Capture config model repr, `tht config check` stdout/stderr, and exception text; assert no signed query/access key/secret key/session token canary occurs. + +- [ ] **Step 6: Run harness tests and verify RED** + +```bash +cd harness +.venv/bin/pytest -q \ + tests/test_config_resources.py \ + tests/test_registry_evidence_config.py +``` + +Expected: signed URL file cases FAIL because the loader only knows scalar password/access/secret/session `*_file` fields. + +- [ ] **Step 7: Implement bounded signed-URL file loading and strict Evidence models** + +Add a dedicated loader before Pydantic validation; do not teach the generic scalar secret resolver to parse arbitrary JSON. The outline is: + +```py +def _resolve_http_signed_url_files(value: Any) -> Any: + # Recurse only through mappings/lists. + # For {type: "http", signed_urls_file: ...}: + # lstat/stat/read at most 1 MiB as UTF-8 + # parse JSON array[str] + # set urls to the array and remove signed_urls_file + # Never interpolate array values in ConfigError. + ... +``` + +`HttpEvidenceSourceConfig` accepts `provenance_urls` for the file-backed form, holds actual transport `urls` as `SecretStr`, validates the one-to-one canonical mapping, and exposes a method returning secret values only to the adapter factory. Add `model_config = {"extra": "forbid"}` to the three modern source models, `EvidenceSourcesConfig`, and the policy model involved in this contract so renderer typos fail loudly. + +Do not continue formatting raw `ValidationError` with `f"{e}"` after secret files have been resolved: Pydantic may include rejected input. Add a safe formatter based on `e.errors(include_input=False, include_url=False)` that retains only location, stable error type, and a custom message that never interpolates transport URLs/credential values. Apply it to `load_config` and prove existing non-secret diagnostics remain useful. + +The factory may unwrap transport URLs only at the last moment when constructing `HttpManifestEvidenceSource`; `config check` must not construct any Evidence adapter or touch network/source roots. + +- [ ] **Step 8: Run focused harness tests and lint** + +```bash +cd harness +.venv/bin/pytest -q \ + tests/test_config_resources.py \ + tests/test_registry_evidence_config.py +.venv/bin/ruff check \ + tht/config.py \ + tht/adapters/factory.py \ + tests/test_config_resources.py \ + tests/test_registry_evidence_config.py +``` + +Expected: PASS. + +- [ ] **Step 9: Commit** + +```bash +git add \ + backend/src/workspaces/contracts.ts \ + backend/src/workspaces/bindings.ts \ + backend/src/workspaces/diagnostics.ts \ + backend/test/workspaces-contracts.test.ts \ + backend/test/workspaces-bindings.test.ts \ + backend/test/workspaces-diagnostics.test.ts \ + backend/test/routes-sessions.test.ts \ + harness/tht/config.py \ + harness/tht/adapters/factory.py \ + harness/tests/test_config_resources.py \ + harness/tests/test_registry_evidence_config.py +git commit -m "feat: bind evidence credentials through local files" +``` + +--- + +### Task 4: Render Evidence through the production runtime handoff + +**Files:** +- Modify: `backend/src/workspaces/runtime-renderer.ts` +- Modify: `backend/src/tht/tht-runner.ts` +- Modify: `backend/test/workspace-runtime-renderer.test.ts` +- Modify: `backend/test/workspace-runtime-handoff.test.ts` + +**Renderer context:** + +Extend the current explicit context, rather than reading the registry checkout or ambient cwd: + +```ts +export interface RuntimeRenderContext { + // existing identity, roots, semantic resources... + revisionContentRoot: string; // /snapshots/ +} +``` + +For a filesystem URI, render: + +```yaml +runtime_identity: + workspace_id: example + workspace_revision: <40-hex-commit> +evidence: + sources: + - type: filesystem + root: /snapshots//workspace-content/example/evidence + patterns: ["**/*.md"] + max_bytes: 10485760 +vector: + max_chunk_chars: 4000 + retain_published_generations: 3 +``` + +HTTP mapping: + +- descriptor `authentication: none` → harness `urls` containing the public descriptor `uris`; +- descriptor `authentication: signed_urls_file` → `provenance_urls` plus `signed_urls_file` from `RuntimeBindings.evidence`; +- convert `_ms` descriptor timeouts to exact seconds without lossy rounding; +- map all limits/SSRF policy fields. + +S3 mapping: + +- split canonical `s3://bucket/prefix` into `bucket` and `prefix`; +- map endpoint/region/trust/limits; +- ambient mode emits no credential keys; +- static mode emits only `access_key_file`, `secret_key_file`, and an optional `session_token_file` path. + +If `evidence` is absent, omit `evidence` and its policy override. Preserve all existing roots, DWH/Qdrant/Ollama behavior and the intentional `ssh_tunnel` error. + +- [ ] **Step 1: Write failing renderer tests** + +Add exact parsed-YAML assertions for: + +- filesystem root under the commit directory, never under `/repo`; +- public/signed HTTP; +- ambient/static S3; +- default and non-default policy; +- no-Evidence omission; +- missing required Evidence bindings rejected before rendering; +- `runtime_identity.workspace_revision` equal to the directory commit; +- no secret file contents in YAML; +- two renders with identical inputs are byte-identical; +- a content-only commit changes identity/root even when descriptor YAML is unchanged; +- existing `ssh_tunnel` fail-closed test remains unchanged. + +- [ ] **Step 2: Run renderer test and verify RED** + +```bash +cd backend +npx vitest run test/workspace-runtime-renderer.test.ts +``` + +Expected: FAIL because no Evidence/runtime content root is rendered. + +- [ ] **Step 3: Implement pure renderer mapping** + +Keep path derivation lexical and deterministic: + +```ts +const filesystemRoot = join( + context.revisionContentRoot, + workspace.evidence.source.uri, +); +``` + +This is safe only because Task 1 canonicalized the URI and Task 2 checked it as a tree at the same commit. Do not `realpath` it or require existence in P1. + +Do not access `process.env` inside the renderer. Receive already validated binding file paths through `RuntimeBindings` so the backend has one deterministic runtime path. Treat the exact descriptor→rendered-YAML mapping and golden handoff tests as P2's compatibility contract; do not claim that the future host CLI can import this backend TypeScript module or that RF1.3 is complete before P2 supplies its backend-independent caller. + +- [ ] **Step 4: Write failing production handoff tests** + +Extend `workspace-runtime-handoff.test.ts` using its real local bare Git fixture and harness invocation. Test: + +1. a registry revision with descriptor plus Evidence tree is activated; +2. `ThtRunner.acquireWorkspaceRuntime(snapshotPath)` reads canonical descriptor/identity and passes `` as `revisionContentRoot`; +3. its leased runtime YAML has the exact Evidence source/policy mapping; +4. `harness/.venv/bin/tht config check -c ` exits zero; +5. acquire/check/release twice yields identical copied YAML while lease file names may differ; +6. release removes only the owned lease file; +7. signed HTTP/S3 file paths resolve from configured secret roots and no canary reaches captured output. + +Use the exact CLI ordering: + +```bash +harness/.venv/bin/tht config check -c /absolute/path/to/rendered.yaml +``` + +Never place `-c` before `config check`. + +- [ ] **Step 5: Run handoff test and verify RED** + +```bash +cd backend +npx vitest run test/workspace-runtime-handoff.test.ts +``` + +Expected: new Evidence assertions FAIL. + +- [ ] **Step 6: Pass the immutable render context from `ThtRunner`** + +`readCanonicalWorkspaceSnapshot` already proves that the descriptor path is under `//` and matches `runtime_identity`. Derive `revisionContentRoot` from that validated path/commit and pass it to the renderer. Never consult the live checkout after the snapshot is acquired. + +- [ ] **Step 7: Run focused cross-layer tests and checks** + +```bash +cd backend +npx vitest run \ + test/workspace-runtime-renderer.test.ts \ + test/workspace-runtime-handoff.test.ts +npx tsc --noEmit -p . +npm run build +``` + +Expected: PASS. + +- [ ] **Step 8: Commit** + +```bash +git add \ + backend/src/workspaces/runtime-renderer.ts \ + backend/src/tht/tht-runner.ts \ + backend/test/workspace-runtime-renderer.test.ts \ + backend/test/workspace-runtime-handoff.test.ts +git commit -m "feat: render revision-bound evidence configuration" +``` + +--- + +### Task 5: Preserve Evidence across registry docs, HTTP routes, conflicts, and exports + +**Files:** +- Modify: `backend/src/workspaces/contracts.ts` +- Modify: `backend/src/workspaces/registry.ts` +- Modify: `backend/src/routes/workspaces.ts` +- Modify: `backend/test/workspaces-contracts.test.ts` +- Modify: `backend/test/workspace-registry.test.ts` +- Modify: `backend/test/routes-workspaces.test.ts` + +**Public artifact rule:** Generated docs describe the source type, canonical non-secret URI(s), limits/policy, same-revision rule, and required installation file variable names. They never include secret contents. Export remains exactly: + +```text +manifest.json +workspace.yaml +contract.env.example +README.md +``` + +P1 does not include `workspace-content` bytes in the browser/API ZIP. + +- [ ] **Step 1: Write failing contract and registry artifact tests** + +Assert for all three sources: + +- contract ordering and generated README are deterministic; +- only applicable variables are present; +- filesystem docs explain same-revision Git ownership and P6 materialization boundary; +- HTTP/S3 docs explain file/ambient credential modes without sample secrets; +- snapshot descriptor/contract/README/manifest hashes match the active commit; +- the snapshot directory contains no copied Evidence tree; +- a secret canary present only in a fixture file never appears in Git blobs, generated docs, snapshot metadata, or error messages. + +Run and confirm RED where the generated docs omit Evidence: + +```bash +cd backend +npx vitest run \ + test/workspaces-contracts.test.ts \ + test/workspace-registry.test.ts +``` + +- [ ] **Step 2: Extend generated public documentation** + +Render a compact `Evidence source` section from canonical descriptor fields. Never read binding files while generating it. Preserve current stable ordering so re-publication of the same descriptor/base remains idempotent. + +If registry snapshot integrity lists expected files, keep the current allowlist deliberately descriptor-only and add a comment pointing to P6 rather than adding `workspace-content` now. + +- [ ] **Step 3: Write failing real-route tests** + +Using `app.inject()` route tests plus the real registry fixture, cover: + +- `/workspaces/validate` returns the canonical Evidence defaults and conditional contract; +- publish create/update, pull, list, and read preserve the whole descriptor; +- an Evidence-only concurrent edit reports a safe conflict field such as `evidence.source.uri` or `evidence.policy.max_chunk_chars`; +- invalid absolute/traversal/cross-workspace/protocol/credential payloads return safe 400 `workspace_invalid`, do not mutate HEAD, and do not echo canaries; +- contextual missing Git tree fails publish/pull safely; +- export, safe extraction, and import preserve canonical descriptor/docs; +- extracted file bytes/hashes are stable across two exports; +- ZIP contains no Evidence bytes and no secrets. + +Do not require raw ZIP byte equality unless production ZIP metadata is explicitly fixed; the P1 determinism contract is stable extracted files and manifest hashes. + +- [ ] **Step 4: Run route test and verify RED** + +```bash +cd backend +npx vitest run test/routes-workspaces.test.ts +``` + +Expected: Evidence route/export expectations FAIL until all payload/artifact paths use the new canonical schema and docs. + +- [ ] **Step 5: Make the smallest route/artifact changes** + +Prefer existing `validateCanonicalWorkspace`, `serializeWorkspaceYaml`, recursive conflict folding, and `exportBundle` paths. Do not create a parallel Evidence DTO and do not add an Evidence upload/preprocess route. Keep public errors on the current sanitized `WorkspaceRegistryError` path. + +- [ ] **Step 6: Run focused backend tests and typecheck** + +```bash +cd backend +npx vitest run \ + test/workspaces-contracts.test.ts \ + test/workspace-registry.test.ts \ + test/routes-workspaces.test.ts +npx tsc --noEmit -p . +``` + +Expected: PASS. + +- [ ] **Step 7: Commit** + +```bash +git add \ + backend/src/workspaces/contracts.ts \ + backend/src/workspaces/registry.ts \ + backend/src/routes/workspaces.ts \ + backend/test/workspaces-contracts.test.ts \ + backend/test/workspace-registry.test.ts \ + backend/test/routes-workspaces.test.ts +git commit -m "feat: preserve evidence in workspace artifacts" +``` + +--- + +### Task 6: Keep existing browser workspace flows lossless without adding authoring UX + +**Files:** +- Modify: `frontend/src/api/workspaces.ts` +- Modify: `frontend/src/api/workspaces.test.ts` +- Modify: `frontend/src/workspaces/drafts.ts` +- Modify: `frontend/src/workspaces/drafts.test.ts` +- Modify: `frontend/src/shell/WorkspaceEditor.tsx` +- Modify: `frontend/src/shell/WorkspaceEditor.test.tsx` + +**Scope:** The browser must accept and preserve canonical Evidence returned by the backend. P1 does not add source-edit controls or launch preprocessing. A small read-only summary prevents the field from being invisible while the registry descriptor remains its authoring surface. + +- [ ] **Step 1: Write failing frontend contract tests** + +Add one fixture for each source type and assert: + +- `sanitizeCanonicalWorkspace` accepts the new top-level key and returns a deep sanitized copy; +- all unknown keys, secret-shaped keys, unsafe URIs, invalid policy values, and malformed unions are rejected rather than passed into browser state; +- draft save/load preserves Evidence; +- API validate/read/publish/conflict parsing preserves Evidence; +- conflict fields under `evidence.source.*` and `evidence.policy.*` are accepted by the sanitized allowlist; +- changing an existing DWH/LLM editor field and publishing does not drop or mutate Evidence; +- no-Evidence workspaces continue to work. + +- [ ] **Step 2: Run focused tests and verify RED** + +```bash +cd frontend +npx vitest run \ + src/workspaces/drafts.test.ts \ + src/api/workspaces.test.ts \ + src/shell/WorkspaceEditor.test.tsx +``` + +Expected: FAIL because `exactRecord` currently rejects the `evidence` top-level key. + +- [ ] **Step 3: Add explicit frontend source types and strict copy helpers** + +Mirror the backend wire contract in `CanonicalWorkspace` without importing server code into the frontend build. Add focused helpers such as `copyEvidencePolicy`, `copyFilesystemEvidence`, `copyHttpEvidence`, and `copyS3Evidence`; use the same bounds and lexical checks as defense-in-depth. + +Update the top-level sanitizer allowlist: + +```ts +const source = exactRecord(value, [ + "workspace", "dwh", "semantic_index", "llm_policy", "diagnostics", "evidence", +]); +``` + +Return `...(evidence ? { evidence } : {})` in the sanitized object. Add the complete stable Evidence field paths to `conflictFields`; do not accept arbitrary server-provided conflict paths. + +- [ ] **Step 4: Add a read-only editor summary** + +When Evidence exists, show source type, safe canonical URI/count, chunk size, and retention with copy such as “Evidence is managed by the registry descriptor in P1.” Never render a signed URL or secret-file binding—those are not descriptor fields. Existing immutable updates already spread the workspace; add the regression test before relying on that behavior. + +- [ ] **Step 5: Run tests and typecheck** + +```bash +cd frontend +npx vitest run \ + src/workspaces/drafts.test.ts \ + src/api/workspaces.test.ts \ + src/shell/WorkspaceEditor.test.tsx +npx tsc -b +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add \ + frontend/src/api/workspaces.ts \ + frontend/src/api/workspaces.test.ts \ + frontend/src/workspaces/drafts.ts \ + frontend/src/workspaces/drafts.test.ts \ + frontend/src/shell/WorkspaceEditor.tsx \ + frontend/src/shell/WorkspaceEditor.test.tsx +git commit -m "fix: preserve workspace evidence in browser drafts" +``` + +--- + +### Task 7: Document and mechanically verify the shared-registry Evidence contract + +**Files:** +- Modify: `deploy/workspaces/example.yaml` +- Modify: `deploy/workspaces/psd.yaml.example` +- Create: `docs/contracts/workspace-evidence-v3.md` +- Modify: `docs/install/local-workspace-registry.md` +- Modify: `docs/install/server-workspace-registry.md` +- Modify: `docs/install/examples/workspace-bindings.env.example` +- Modify: `scripts/verify-workspace-install-docs.sh` +- Modify: `scripts/test-verify-workspace-install-docs.sh` + +**Required documented repository layout:** + +```text +registry.git/ +├── workspaces/ +│ ├── example.yaml +│ └── another.yaml +├── workspace-content/ +│ ├── example/evidence/... +│ └── another/evidence/... +└── workspace-docs/ + ├── example/{contract.env.example,README.md} + └── another/{contract.env.example,README.md} +``` + +Correct any current prose that claims generated `.env.example`/`.md` files live directly under `workspaces/`; production writes them under `workspace-docs//`. + +- [ ] **Step 1: Add failing verifier self-tests** + +Create mutated fixture copies that must fail when they: + +- omit the `workspace-content//evidence` layout or same-commit rule; +- show an absolute/cross-workspace Evidence path; +- place generated docs in the wrong registry directory; +- omit HTTP/S3 file credential boundaries; +- contain credential literals, signed query examples, or unsafe placeholder values; +- claim P1 materializes/extracts/indexes Evidence; +- omit the exact `tht config check -c ` ordering; +- omit separate automated/manual acceptance states. + +Also retain all existing adversarial doc-verifier cases. + +- [ ] **Step 2: Run self-test and verify RED** + +```bash +bash scripts/test-verify-workspace-install-docs.sh +``` + +Expected: new mutation cases are not detected yet. + +- [ ] **Step 3: Update generic descriptors and the canonical contract document** + +Add the explicit filesystem section to: + +- `deploy/workspaces/example.yaml` with `workspace-content/example/evidence`; +- `deploy/workspaces/psd.yaml.example` using its generic fixture ID and matching namespace. + +Do not add real PSD/client content or secrets to ThothII. + +`docs/contracts/workspace-evidence-v3.md` must include: + +- exact strict shapes/defaults for filesystem, public/signed HTTP, and ambient/static S3; +- safe/unsafe URI examples; +- installation file formats and variable naming; +- one Git repo for all workspace namespaces; +- descriptor/tree commit identity and content-only revision semantics; +- browser/export behavior; +- optional Evidence/no-Evidence compatibility; +- P1 lexical/tree checks versus P6 materialization/symlink checks; +- exact `tht config check -c` command; +- explicit statement that P1 does no acquisition, extraction, embeddings, Qdrant writes, ACTIVE publication, or GC. + +- [ ] **Step 4: Update local/server operator guides and binding example** + +Describe curator flow in the right order: + +1. clone/pull shared registry; +2. place source content under the workspace namespace and commit/push it; +3. validate/publish descriptor against that base commit; +4. inspect generated public docs; +5. provision any `*_FILE` paths outside Git below allowed secret roots; +6. render/check config; +7. stop—preprocessing/materialization is later P2/P6. + +The env example contains only non-secret values and file paths. It may use obvious non-working paths such as `/run/secrets/...`; never include a credential/signed URL. + +- [ ] **Step 5: Strengthen the verifier and run both directions** + +```bash +bash scripts/test-verify-workspace-install-docs.sh +bash scripts/verify-workspace-install-docs.sh --fixtures-only +``` + +Expected: both PASS; every adversarial mutation fails inside the self-test for the intended reason. + +- [ ] **Step 6: Commit** + +```bash +git add \ + deploy/workspaces/example.yaml \ + deploy/workspaces/psd.yaml.example \ + docs/contracts/workspace-evidence-v3.md \ + docs/install/local-workspace-registry.md \ + docs/install/server-workspace-registry.md \ + docs/install/examples/workspace-bindings.env.example \ + scripts/verify-workspace-install-docs.sh \ + scripts/test-verify-workspace-install-docs.sh +git commit -m "docs: define workspace evidence registry contract" +``` + +--- + +### Task 8: Automated integration goal — complete P1 configuration process + +**Files:** +- Create: `scripts/p1-acceptance.sh` +- Create: `backend/scripts/p1-acceptance.mjs` +- Create: `backend/scripts/p1-acceptance.test.mjs` +- Create: `scripts/test-p1-acceptance.sh` +- Modify: `.gitignore` only if `.artifacts/` is not already ignored (it currently is; normally no edit) +- Modify: `PROJECT_STATE.md` + +**Public command:** + +```bash +./scripts/p1-acceptance.sh integration --keep +``` + +It uses Node stdlib plus the built backend's normal dependencies. It must not require Docker, frontend, DWH, Qdrant, Ollama, external network, or a real remote. It starts a real Fastify listener on `127.0.0.1` with OS-assigned port `0` and makes actual HTTP requests with `fetch`; `app.inject()` does not satisfy this gate. + +**Exact run topology:** + +```text +.artifacts/p1-integration// +├── ownership.json +├── remote.git/ +├── author/ +├── installation/ +│ ├── registry/ +│ ├── data/ +│ ├── runtime/ +│ └── bindings.env +├── fixture-secrets/ # sole secret-scan exclusion +├── fixtures/ +│ ├── descriptors/ +│ └── requests/ +├── requests/ +├── responses/ +├── exports/ +│ ├── raw/ +│ └── extracted/ +├── rendered/ +├── logs/ +├── report.json +└── report.md +``` + +`ownership.json` is written before creating child resources and includes schema version, run ID, random nonce, absolute root, repository root, start time, current PID, owned listener identity, and the exact resources the run may delete/stop. Never store tokens, secret contents, or full signed URLs in ownership/report files. + +- [ ] **Step 1: Write failing acceptance-runner unit tests** + +Use `node:test` for library-level guards. Test: + +- run ID/root validation accepts only a direct child of the repository's canonical `.artifacts/p1-integration`; +- cleanup refuses a missing/malformed/mismatched ownership file, wrong nonce, symlink root, parent root, manual-acceptance root, and foreign sibling; +- cleanup removes one correctly owned synthetic run and nothing else; +- report schema requires a single result per check, no duplicate/retry attempt field, safe relative artifact paths, hashes, timestamps, command names, and `overall` derived from checks; +- injected failure records exactly one failed scenario, retains its run for diagnosis, and exits nonzero; +- secret scanner skips only `fixture-secrets` and detects canaries everywhere else, including JSON/Markdown/logs/responses/rendered/export files; +- successful non-`--keep` cleanup and successful `--keep` retention; +- no command helper accepts shell strings; Git/tht/backend commands use argv arrays. + +Provide a test-only `P1_ACCEPTANCE_FAIL_AT=` hook. It is not a retry mechanism; it deterministically proves failure reporting. + +- [ ] **Step 2: Run the runner tests and verify RED** + +```bash +bash scripts/test-p1-acceptance.sh +``` + +Expected: FAIL because the acceptance runner does not exist. + +- [ ] **Step 3: Implement preflight and owned lab lifecycle** + +`scripts/p1-acceptance.sh` must: + +1. resolve repository root from the script location; +2. require `node`, `npm`, `git`, and an executable `harness/.venv/bin/tht` (or an explicit `THT_BIN` override); +3. run `npm --prefix backend run build` once; +4. invoke `node backend/scripts/p1-acceptance.mjs integration [--keep]`; +5. preserve the Node exit code. + +The Node runner must: + +- create a cryptographically random run ID/nonce; +- use `mkdir`-exclusive semantics and refuse reuse; +- write files atomically where they are process evidence; +- use `spawn`/`execFile` with argv and bounded timeouts; +- execute each scenario exactly once; +- always close its owned Fastify instance in `finally`; +- retain a failed run unconditionally; +- delete a successful run only when `--keep` is absent and ownership validation passes. + +Do not poll/restart a failed scenario. Awaiting `app.listen()` or a bounded `/health` readiness probe is startup synchronization, not a scenario retry; record it separately. + +- [ ] **Step 4: Create the local Git registry from zero** + +Inside the run: + +```bash +git init --bare --initial-branch=main /remote.git +git clone /remote.git /author +``` + +Configure fixture-only author identity. In the author clone, create non-secret curated trees for at least the filesystem workspace and push the bootstrap commit: + +```text +workspace-content/p1-filesystem/evidence/guide.md +workspace-content/p1-filesystem/evidence/domain/table.md +``` + +The API, not the fixture writer, publishes `workspaces/*.yaml` and `workspace-docs/*`. Additional HTTP/S3 descriptor fixtures may share the same repository but do not pretend to be acquired. + +Create file bindings under `fixture-secrets/` for: + +- the normal DWH config required by the renderer/config loader; +- one signed-HTTP JSON list containing a unique query canary; +- S3 access/secret/session values containing distinct canaries. + +Set only file-path environment bindings and configure `THT_WORKSPACE_SECRET_ROOTS` to the fixture secret directory. Provision the required DWH transport/host/port/user/password-file variables separately for the `P1_FILESYSTEM`, `P1_HTTP`, and `P1_S3` contract namespaces (they may reference one shared fixture password file); then add the source-specific Evidence variables. Capture a redacted `bindings.env` containing paths and non-secret endpoints, not values. + +- [ ] **Step 5: Start production backend boundaries and use real HTTP** + +Import `loadConfig`, `buildApp`, `WorkspaceRegistry`, and `ThtRunner` from `backend/dist`. Build them with the run's remote/root/data/runtime settings and pass those production instances to `buildApp`. Listen on `127.0.0.1:0`; save only the loopback base URL. + +Perform and persist each request/response once: + +1. `GET /workspace-registry/status`; +2. positive `POST /workspaces/validate` for filesystem, signed HTTP, and static-file S3 descriptors; +3. `POST /workspaces/publish` create for each descriptor, based on the current commit returned by the preceding step/read; +4. `POST /workspace-registry/pull`; +5. `GET /workspaces/:id` and list; +6. `GET /workspaces/:id/export` for each workspace; +7. safe extraction with the production ZIP dependency and manifest/file hash verification. + +Use a sequential current-base workflow; do not blindly replay a stale base after each publication. Every recorded response must be parsed/sanitized before entering `report.json`. + +- [ ] **Step 6: Prove one immutable Git identity** + +For the filesystem workspace, assert and report hashes for: + +```text +API revision.commit +installation registry checkout HEAD +snapshot manifest commit +runtime_identity.workspace_revision +``` + +All four must be identical. Then use fixed-argv Git object checks: + +```text +git cat-file -e :workspaces/p1-filesystem.yaml +git cat-file -e :workspace-content/p1-filesystem/evidence/guide.md +git cat-file -t :workspace-content/p1-filesystem/evidence +``` + +The last output must be `tree`. Also assert the immutable snapshot contains descriptor/docs/manifest but **not** a materialized `workspace-content` tree. + +Fast-forward the curator clone to the API publication HEAD, then create and push a content-only update to `guide.md`; invoke the registry pull once and prove: + +- active revision changes to the new commit; +- descriptor blob stays equal; +- runtime identity/root changes to the new commit; +- old retained snapshot remains immutable. + +This is the regression that prevents blob identity from masquerading as revision identity. + +- [ ] **Step 7: Exercise production rendering and the real harness check** + +For each workspace snapshot: + +1. call the production `ThtRunner.acquireWorkspaceRuntime`; +2. copy the lease YAML into `rendered/-1.yaml`; +3. execute `harness/.venv/bin/tht config check -c ` exactly once; +4. release the lease; +5. repeat the acquire/check as an explicit determinism/idempotence check, not a retry; +6. copy `-2.yaml` and compare bytes; +7. assert identity/source/policy fields and that all leased files were released. + +The HTTP/S3 commands parse file credentials but never construct adapters or touch network. The filesystem root is allowed not to exist because P6 has not materialized it. Any Evidence acquisition call is a gate failure. + +- [ ] **Step 8: Execute negative scenarios with no mutation/no leak** + +Send separate validate requests for: + +- absolute, traversal, backslash, and cross-workspace filesystem URIs; +- unsupported source type/protocol; +- descriptor credential field containing a canary; +- HTTP userinfo/query canary; +- malformed policy/limits. + +For contextual validation, use a separate invalid workspace/branch state whose canonical filesystem path is absent at the referenced commit; pull/publish must fail and preserve the last valid active snapshot. Do not damage and repair the primary scenario as a hidden retry. + +For every negative case assert: + +- expected safe status/code/field; +- Git HEAD/snapshot state unchanged where applicable; +- response/log/report contains no rejected canary or Git stderr. + +- [ ] **Step 9: Verify export/docs/determinism/secrets/cleanup and write reports** + +The final report has stable check IDs including at least: + +```text +preflight +clean_state +ownership +local_git_bootstrap +http_validate_publish_pull_read_export +same_revision_git_objects +content_only_revision +snapshot_and_docs +runtime_render_determinism +tht_config_check +negative_schema_cases +negative_context_case +no_p1_scope_artifacts +secret_scan +cleanup_confinement +``` + +`no_p1_scope_artifacts` asserts that the run contains no `artifacts/evidence`, `corpus/ACTIVE`, embedding output, Qdrant records, or preprocessing invocation. + +Scan every regular file below the run except `fixture-secrets/` for all fixture canaries and known credential values. Scan Git blobs reachable from the remote, extracted ZIPs, requests/responses, logs, YAML, JSON, and Markdown. File paths and safe variable names are allowed; values are not. + +Write `report.json` atomically, then derive `report.md` from it. End with exactly: + +```text +automated integration: PASS +manual acceptance: PENDING +``` + +when all checks pass. With `--keep`, print the absolute retained run path. Without `--keep`, validate ownership and clean only that run after printing/writing the successful result. + +- [ ] **Step 10: Run acceptance-runner tests** + +```bash +bash -n scripts/p1-acceptance.sh scripts/test-p1-acceptance.sh +bash scripts/test-p1-acceptance.sh +``` + +Expected: PASS. + +- [ ] **Step 11: Run the complete automated process once from clean state** + +Before running, verify no previous command is active and do not reuse a run root: + +```bash +./scripts/p1-acceptance.sh integration --keep +``` + +Expected: exit `0`; output names one new retained run; its `report.json` has `overall: "PASS"`; `report.md` says automated PASS/manual PENDING; no scenario has a retry/attempt count greater than one. + +If it fails: stop. Diagnose from the retained run, add/fix a regression test and implementation, then invoke a **new** full run with a new ID. Do not rerun the same failed scenario blindly and do not overwrite the old report. + +- [ ] **Step 12: Inspect retained evidence and update project state** + +Manually inspect the report plus a sample descriptor, Git object proof, snapshot manifest, generated README, extracted export, and the two rendered configs. Record the retained relative run path and: + +```text +automated integration: PASS +manual acceptance: PENDING +``` + +in `PROJECT_STATE.md`. Do not mark manual acceptance complete. + +- [ ] **Step 13: Commit** + +```bash +git add \ + scripts/p1-acceptance.sh \ + backend/scripts/p1-acceptance.mjs \ + backend/scripts/p1-acceptance.test.mjs \ + scripts/test-p1-acceptance.sh \ + PROJECT_STATE.md +git commit -m "test: prove P1 configuration process end to end" +``` + +--- + +### Task 9: Build the independent manual-acceptance tooling + +**Files:** +- Create: `scripts/p1-manual-acceptance.sh` +- Create: `backend/scripts/p1-manual-acceptance.mjs` +- Create: `backend/scripts/p1-manual-acceptance.test.mjs` +- Create: `backend/scripts/p1-render-snapshot.mjs` +- Create: `backend/scripts/p1-render-snapshot.test.mjs` +- Create: `scripts/test-p1-manual-acceptance.sh` +- Create: `docs/testing/p1-manual-acceptance.md` +- Modify after human approval only in Task 11: `PROJECT_STATE.md` + +**Public lifecycle:** + +```bash +./scripts/p1-manual-acceptance.sh prepare +./scripts/p1-manual-acceptance.sh serve +./scripts/p1-manual-acceptance.sh stop +./scripts/p1-manual-acceptance.sh cleanup +``` + +The helper supports only those four lifecycle actions. It uses the fixed, independent root `.artifacts/manual-acceptance/p1/` and backend address `http://127.0.0.1:8791`. It never reads or copies an automated integration run. + +**Manual topology:** + +```text +.artifacts/manual-acceptance/p1/ +├── ownership.json +├── backend.pid # only while served +├── remote.git/ +├── author/ +├── installation/ +├── fixture-secrets/ +├── fixtures/ +├── requests/ +├── responses/ +├── exports/ +├── rendered/ +├── logs/ +├── commands/ # generated concrete reviewer commands +├── GUIDE.md +└── VERDICT.md # created by reviewer, never by automation +``` + +The generated render commands use this tracked, acceptance-only interface (not an HTTP route and not the future P2 host renderer): + +```bash +node backend/scripts/p1-render-snapshot.mjs \ + --ownership .artifacts/manual-acceptance/p1/ownership.json \ + --snapshot \ + --output .artifacts/manual-acceptance/p1/rendered/runtime-1.yaml +``` + +The script imports the built production `ThtRunner`, reconstructs its non-secret settings from the owned manual installation, resolves descriptor bindings from the command environment, acquires one runtime lease, copies it atomically with mode `0600`, and releases the lease in `finally`. It accepts only owned snapshot/output paths under the fixed manual root; it never starts a backend, calls a render HTTP route, or reads secret contents itself. + +- [ ] **Step 1: Write failing lifecycle guard tests** + +Test without approving the gate: + +- `prepare` refuses a pre-existing root, a symlink root, automated-run input, or missing prerequisites; +- `prepare` creates fresh ownership, bare remote, author clone/content commit, installation directories, descriptor/request fixtures, secret files, output directories, commands, and guide; +- `serve` refuses unowned state, an occupied `127.0.0.1:8791`, an existing live PID, a stale/mismatched PID, or any non-loopback bind; +- `stop` signals only the PID whose ownership nonce, executable, cwd/root, and recorded start identity match; +- `cleanup` refuses while the owned server is live and removes only the exact owned fixed root after stop; +- foreign siblings and `.artifacts/p1-integration` are never removed; +- no lifecycle action writes `VERDICT.md` or changes manual status to PASS; +- the generated render commands fail safely before rendering when the saved read response is missing/malformed, its snapshot path escapes the owned installation, or its revision differs from the published Git commit; +- `p1-render-snapshot.mjs` rejects unowned/symlink/out-of-root snapshot or output paths, copies one production lease, always releases it on success/failure, writes mode `0600`, and produces byte-identical outputs for two identical invocations without leaving runtime lease files. + +- [ ] **Step 2: Run lifecycle tests and verify RED** + +```bash +bash scripts/test-p1-manual-acceptance.sh +``` + +Expected: FAIL because the helper does not exist. + +- [ ] **Step 3: Implement guarded preparation and backend-only serving** + +`prepare` must: + +- require that Task 8 has been implemented, but not consume its state; +- build backend once; +- create the fixed root exclusively and write ownership first; +- initialize a new bare remote and curator clone; +- seed a fresh filesystem Evidence tree and secret-file fixtures; +- create concrete positive/negative JSON request files; +- generate `commands/render-1.sh` and `render-2.sh` that, after the reviewer has saved the successful read response, extract `revision.snapshotPath` with a bounded Node JSON parser, verify its commit equals the saved API/Git commit and that it lies below the owned installation snapshot root, then invoke `backend/scripts/p1-render-snapshot.mjs` with concrete owned output paths/environment; also generate safe scripts for Git inspection, `diff`, config checks, ZIP extraction/manifest verification, and secret scanning; +- generate `GUIDE.md` with absolute/concrete paths and expected safe outcomes; +- leave the server stopped and status `PENDING`. + +`serve` must start only `node backend/dist/server.js` with the lab's environment, bind exactly `127.0.0.1:8791`, redirect stdout/stderr to owned logs, atomically persist PID/start identity, and perform one bounded health readiness wait. It must not start Docker or frontend. + +`stop` must validate ownership/process identity before sending TERM, wait a bounded interval, and report if the operator must intervene; it must never fall back to a broad `pkill`. `cleanup` must apply the same root/nonce/symlink checks as Task 8. + +- [ ] **Step 4: Write the permanent manual guide** + +`docs/testing/p1-manual-acceptance.md` explains prerequisites, four lifecycle commands, separation from automated state, expected outputs, how to preserve a failed lab, and the verdict format. It must state that the reviewer—not the helper—performs and judges the walkthrough. + +The generated `GUIDE.md` must contain this ordered checklist: + +1. inspect `ownership.json`, the pre-publication Evidence tree, fixture descriptor, and binding paths; +2. run `serve` and verify only `127.0.0.1:8791` listens; +3. personally execute real `curl` status → validate → publish → pull → read → export calls, saving each response; +4. only after publish, use `git log`, `git ls-tree`, and `git show :workspaces/.yaml` plus `git show :workspace-content//evidence/...` to inspect descriptor/content identity at that one commit; +5. inspect generated `workspace-docs`, immutable descriptor snapshot, and snapshot manifest; +6. safely extract ZIP and verify manifest hashes and absence of Evidence bytes/secrets; +7. run the generated production render command twice and `diff` the YAML; +8. inspect runtime identity, absolute reserved filesystem root, Evidence limits, and policy; +9. personally execute `harness/.venv/bin/tht config check -c ` and the second config; +10. submit invalid absolute/traversal/cross-workspace/protocol/credential requests and verify safe rejection/no mutation/no canary; +11. run the generated secret scan outside `fixture-secrets`; +12. confirm no preprocessing, Evidence materialization, embedding, Qdrant, ACTIVE, or retention artifact exists; +13. run `stop` and confirm the PID/port are gone; +14. record `VERDICT.md` with reviewer, UTC time, every checklist result, observations, and either `manual acceptance: PASS` or `manual acceptance: FAIL`. + +The guide must not tell the reviewer to inspect raw secret file contents. It may verify file ownership/mode and canary absence outside the excluded directory. + +- [ ] **Step 5: Run lifecycle tests and syntax checks** + +```bash +bash -n \ + scripts/p1-manual-acceptance.sh \ + scripts/test-p1-manual-acceptance.sh +bash scripts/test-p1-manual-acceptance.sh +``` + +Expected: PASS. This proves tooling only; it does **not** approve manual acceptance. + +- [ ] **Step 6: Commit the manual gate tooling** + +```bash +git add \ + scripts/p1-manual-acceptance.sh \ + backend/scripts/p1-manual-acceptance.mjs \ + backend/scripts/p1-manual-acceptance.test.mjs \ + backend/scripts/p1-render-snapshot.mjs \ + backend/scripts/p1-render-snapshot.test.mjs \ + scripts/test-p1-manual-acceptance.sh \ + docs/testing/p1-manual-acceptance.md +git commit -m "test: add P1 manual configuration walkthrough" +``` + +--- + +### Task 10: Re-run branch-wide verification and hand off explicit gate status + +**Files:** +- Modify only if status/path changes: `PROJECT_STATE.md` + +This task runs after all implementation/tooling commits and immediately before the human walkthrough. It must leave manual acceptance PENDING. + +- [ ] **Step 1: Run the complete backend gate** + +```bash +cd backend +npx vitest run +npx tsc --noEmit -p . +npm run build +``` + +Expected: all tests, typecheck, and build PASS. + +- [ ] **Step 2: Run the complete harness gate** + +```bash +cd harness +.venv/bin/pytest -q +.venv/bin/ruff check . +``` + +Expected: PASS under the repository's default marker selection. Do not enable L2 or external services for P1. + +- [ ] **Step 3: Run the complete frontend gate** + +```bash +cd frontend +npx vitest run +npx tsc -b +npm run build +``` + +Expected: PASS. + +- [ ] **Step 4: Run every root verifier/tooling test** + +```bash +bash scripts/test-verify-workspace-install-docs.sh +bash scripts/verify-workspace-install-docs.sh --fixtures-only +bash scripts/test-p1-acceptance.sh +bash scripts/test-p1-manual-acceptance.sh +bash -n \ + scripts/p1-acceptance.sh \ + scripts/p1-manual-acceptance.sh \ + scripts/test-p1-acceptance.sh \ + scripts/test-p1-manual-acceptance.sh +``` + +Expected: PASS. + +- [ ] **Step 5: Execute one final clean automated run at branch HEAD** + +```bash +./scripts/p1-acceptance.sh integration --keep +``` + +Expected: a new run ID, exit `0`, all report checks PASS, secret scan PASS, cleanup confinement PASS, and `manual acceptance: PENDING`. No human walkthrough has occurred in this plan sequence yet. + +Do not reuse Task 8's run as the final evidence after later commits. If this command fails, follow the diagnose/fix/new-clean-run rule; never loop it automatically. + +- [ ] **Step 6: Inspect status and diff integrity** + +```bash +git diff --check +git status --short --branch +git log --oneline --decorate -12 +``` + +Expected: + +- `git diff --check` exits zero; +- no `.artifacts` file is tracked; +- no secret/example canary is present in tracked files; +- implementation commits are small and correspond to plan tasks; +- `PROJECT_STATE.md` names the latest retained automated run and reports manual status truthfully. + +If only the retained run path/status changed, commit that state: + +```bash +git add PROJECT_STATE.md +git commit -m "docs: finalize P1 verification status" +``` + +- [ ] **Step 7: Report completion with two independent gates** + +The pre-walkthrough handoff must state, on separate lines: + +```text +automated integration: PASS — +manual acceptance: PENDING — awaiting the independent reviewer walkthrough +``` + +Also state explicitly: + +- P1 proves configuration, same-revision Git identity, rendering, and harness parsing; +- P1 does not prove acquisition/materialization/preprocessing/indexing/ACTIVE/GC; +- P6 is the next required step before filesystem Evidence can be consumed; +- P2/P4/P9/P10 retain their documented responsibilities. + +Do not summarize P1 as fully accepted when manual status is PENDING or FAIL. + +--- + +### Task 11: Manual acceptance gate — descriptor and rendered configuration artifacts + +**Files:** +- Read: `.artifacts/manual-acceptance/p1/GUIDE.md` +- Create by reviewer only: `.artifacts/manual-acceptance/p1/VERDICT.md` +- Modify after explicit human approval only: `PROJECT_STATE.md` + +- [ ] **Step 1: Stop and request the human checkpoint** + +Only after Task 10 reports automated PASS at branch HEAD, ask the reviewer to execute: + +```bash +./scripts/p1-manual-acceptance.sh prepare +./scripts/p1-manual-acceptance.sh serve +# follow .artifacts/manual-acceptance/p1/GUIDE.md in full +./scripts/p1-manual-acceptance.sh stop +``` + +This is a controlled, resumable pause. If access/session is interrupted, leave the owned root intact, rerun only `serve` if the guide says no stateful scenario has begun, or use `cleanup` then `prepare` for a genuinely fresh walkthrough. Never infer approval from automated outputs. + +- [ ] **Step 2: Record the human result without hiding failures** + +If `VERDICT.md` says FAIL or is absent, keep: + +```text +automated integration: PASS +manual acceptance: PENDING (or FAIL with observation) +``` + +in `PROJECT_STATE.md` and preserve the manual root for diagnosis. + +Only if the reviewer explicitly records PASS, update `PROJECT_STATE.md` with reviewer/date and: + +```text +automated integration: PASS +manual acceptance: PASS +``` + +Then optionally clean the lab after the reviewer confirms artifacts are no longer needed: + +```bash +./scripts/p1-manual-acceptance.sh cleanup +``` + +Commit only the project-state update, never `.artifacts`: + +```bash +git add PROJECT_STATE.md +git commit -m "docs: record P1 manual acceptance" +``` + +--- + +## Requirement traceability + +| Requirement/decision | Implemented/proven by | +|---|---| +| RF1.1 / D1 complete source + policy | Tasks 1, 3, 4, 5, 7 | +| RF1.2 / RNF1 no secrets in Git | Tasks 1, 3, 5, 7, 8 | +| RF1.3 same runtime rendering | P1 prerequisite only: Task 4 and Task 8 prove backend session render → harness parse; P2 must prove its backend-independent host render is equivalent before RF1.3 is complete | +| RF1.4 three DWH transports represented | Existing descriptor/contract retained; SSH remains fail-closed for P10; regression tests Tasks 3–4 | +| RF1.5 one registry, namespace isolation, same revision | Tasks 1–2 and Git-object proof Task 8 | +| RNF2 deterministic/idempotent relevant outputs | Tasks 4–5, Task 8 repeated read/render/config-check/content-only revision | +| RNF3 workspace without Evidence still works | Task 1 optional field and Tasks 4/6 regressions | +| RNF4 workspace isolation | Lexical namespace + exact Git tree checks Tasks 1–2; runtime point filtering remains outside P1 | +| RNF5 renderer compatibility | Task 4 production handoff and harness check | +| RNF7 one canonical path | Schema/renderer only; no parallel fixture contract | +| RNF8 integration-first | Automated Task 8 and branch-wide Task 10 before human Task 11 | +| D6/P6 boundary | No materialization/realpath/symlink claim; reserved commit root only | +| D9 policy defaults | Tasks 1 and 4; execution/GC remains P9 | +| Automated acceptance standard | Task 8 clean state, no retry, reports, secret scan, cleanup | +| Manual acceptance standard | Task 9 independent tooling plus Task 11 explicit resumable human checkpoint | + +## Execution notes + +- Every task is red → green → focused verification → commit. Do not batch several red tasks into one implementation change. +- Use existing local bare-Git fixtures and production interfaces rather than mocks where a boundary already exists. +- A test-only fake is acceptable only for an external dependency that P1 explicitly excludes; the core P1 path itself must remain real Git, real Fastify HTTP, production registry/renderer, and real harness CLI. +- Keep failed acceptance artifacts. Fix the cause with a regression test, then start a new clean run. “Try it again” is not a diagnostic step. +- Human approval is a durable decision, not a command exit code. The manual helper must never create a PASS verdict.