1671 lines
137 KiB
Markdown
1671 lines
137 KiB
Markdown
# P5 Curated FK Annotations in Git Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||
|
||
**Goal:** Implement PRD D5 so the only curated FK source is the bounded, validated Git blob at `workspace-content/<workspace-id>/schema/annotations.yaml`, materialized for one exact workspace commit and accepted through an explicit, auditable P2-run revision transition.
|
||
|
||
**Architecture:** Extend the P1 registry plumbing with a fixed-argv Git blob reader built on one shared bounded-child lifecycle and one shared `AnnotationSynchronizer`; the synchronizer asks the harness's strict parser to validate bytes, then publishes exact bytes and a hash-bound manifest through one dirfd/openat-style protected-filesystem primitive under P3's immutable revision root. The synchronizer has an explicit prepare lifecycle for repository pull and pre-READY acceptance, resolved only with P3's `futureWorkspaceLayoutPaths`, and a runtime lifecycle for active/pinned consumption that requires the exact committed READY state before calling the four-argument `workspaceRuntimePaths`. Extend P2's `PreprocessingStateStore` and `WorkspacePreprocessingService.execute` rather than creating another workflow: one legal V1→V2 checkpoint migration preserves audit facts but requires explicit acceptance, a bounded ambiguity-safe `resolveRun` owns workspace discovery, and `workspace schema export-fks --run` exports the paused run's exact candidate. After the curator pushes, P5 registers an exact P2 `CapabilityAwareRegistryPublicationParticipant<AnnotationRegistryPreparationV1>` with P2's `RegistryAddressedRequestV1` and invokes only `WorkspaceRegistry.publishAddressed`—never the writer-first common migration lifecycle. That released `registry_pull` exception alone has RW/Git capability: `repository.lock` pins `RegistryPullAddressedPlanV1`; `WorkspaceRegistry` enters the sole `runUnderOrderedWorkspaceWriterLocks` callback for the complete `changedWorkspaceIds` in strict lexical order and exposes one `OrderedWorkspaceWriterCapabilitySet`; the participant receives each exact `AddressedWorkspacePublicationLeaseV1`. The set remains held through the single active-snapshot-pointer write, file fsync, atomic rename, parent fsync, target-byte verification, and `terminal_durable`; ordered callback settlement invalidates and closes the set in reverse lexical order, and only then is `repository.lock` released. Exact same-ID resume uses `RegistryPullJobStateV1`: at `publication_intent_durable` an all-base state resumes without a fetch, while an all-target state after rename+fsync reconciles lost acknowledgement without refetch or a second publication; third identity or target drift refuses. The curator must then run P3's exact non-activating `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json` operation: its fixed `p3_migrate_dwh_cache`/`p3_prepare_dwh_cache` plus `p3_materialize_dwh_snapshot` stages select the pulled commit and effective binding, publish and strictly reverify that revision's binding-qualified physical/LSH snapshot without reading or publishing READY, and never fall back to the prior revision's snapshot. Only after that preparation does `schema accept` consume the already-active immutable annotation and DWH snapshots under the writer lock; it never pulls, takes the repository lock, or receives Git capability. Admission and preprocessing resume remain `migration_required` after pull, after DWH snapshot preparation, and after acceptance. P3's exact `workspace migrate activate-revision-layout` operation must then publish and verify the accepted commit+binding READY before either can succeed.
|
||
|
||
**Tech Stack:** TypeScript 5, Node.js 22, Git plumbing (`git ls-tree`, `git cat-file` with fixed argv), Python 3.12, Pydantic 2, PyYAML, Typer, Go, Docker Compose, Vitest, pytest, Ruff, Go test, Bash.
|
||
|
||
---
|
||
|
||
## Scope, prerequisites, and non-negotiable contracts
|
||
|
||
**Source:** `docs/prd/2026-08-09-workspace-preprocessing-prd.md` D5/RF3 and `docs/superpowers/specs/2026-08-10-p2-p6-workspace-preprocessing-design.md` §7. The reviewed planning baseline was commit `16ee92c8f980b363a82cb0e2777c354ed25519fa` (tree `ced13dcedff78dfba40313608f97946f716d2e11`); execution must record its own clean source commit and tree in the retained report.
|
||
|
||
Do not begin Task 1 until the P1, P2, P3, and P4 checkpoints are accepted and the worktree is clean. P5 technically builds on P1 plus the P2/P3 operator/state/path work; P4 is an independent delivery checkpoint but precedes P5 in the P2–P6 release sequence. First run:
|
||
|
||
```bash
|
||
git status --short
|
||
git rev-parse HEAD
|
||
git rev-parse HEAD^{tree}
|
||
test -f docs/superpowers/plans/2026-08-10-p2-host-workspace-preprocessing-cli.md
|
||
test -f docs/superpowers/plans/2026-08-10-p3-effective-config-fingerprint.md
|
||
test -f docs/superpowers/plans/2026-08-10-p4-qdrant-bootstrap-rebuild.md
|
||
test -f backend/src/workspaces/registry-pull-job.ts
|
||
test -f backend/test/registry-pull-job-imports.compile.ts
|
||
```
|
||
|
||
Expected: empty `git status --short`; three plan files plus P3's exact job export/compile-import files exist; `PROJECT_STATE.md` records accepted P1–P4 checkpoints. This plan is already rebased on the frozen P2/P3 handoff: `backend/src/workspace-maintenance.ts::main`, `WorkspacePreprocessingService.execute`, `PreprocessingStateStore`, `WorkspaceRuntimeConfigLeaseFactory`, `backend/src/workspaces/revision-layout.ts::{WorkspaceLayout,futureWorkspaceLayoutPaths,workspaceRuntimePaths,readRevisionLayoutState}`, `runUnderWorkspaceWriterLock` / `probeWorkspaceWriterLock`, P2 `RegistryPullAddressedPlanV1`, `AddressedWorkspacePublicationLeaseV1`, `OrderedWorkspaceWriterCapabilitySet`, `CapabilityAwareRegistryPublicationLifecycleOwner`, `CapabilityAwareRegistryPublicationParticipant<T>`, `RegistryAddressedRequestV1`, `RegistryPullAddressedResultV1`, and `WorkspaceRegistry.publishAddressed`, plus `tools/thothctl/internal/workspaceops::{ParseWorkspaceCommand,Run}`. It also consumes P3's released `thothctl --installation <abs> workspace registry pull --workspace <id> [--resume <32-hex-outer-run-id>] [--json]`, closed Go `RegistryPullCommand`, literal operation/result/Compose capability label `registry_pull`, and exact `RegistryPullPublicJobRequestV1`, `RegistryPullPhaseV1`, `RegistryPullAddressedJobRequestV1`, `RegistryPullParticipantStateV1`, `RegistryPullSynchronizerStateV1`, and `RegistryPullJobStateV1`; the existing read-only active immutable-snapshot API `WorkspaceRegistry.read(id)`; and released `thothctl --installation <abs> workspace migrate dwh-cache --workspace <id> [--resume <32-hex-outer-run-id>] [--json]` / `migrate_dwh_cache` operation. `registry_pull` is the explicit exception to P3's writer-first common migration lifecycle. Its exact repository-first order and fields are quoted below: the complete lexical capability set stays owned through active-pointer rename+parent fsync, byte verification, and terminal durability; reverse set release precedes repository release. Resume reads only the recorded `RegistryPullJobStateV1`, never refetches or reselects a target, and accepts only exact all-base/all-target/mixed recorded identities while refusing third identity or target drift. The DWH operation's exact `p3_migrate_dwh_cache` or `p3_prepare_dwh_cache` cache stage is followed by `p3_materialize_dwh_snapshot` for the selected unready commit+binding and must publish and strictly reverify its revision-qualified physical/LSH snapshot without READY. Only `registry_pull` receives writable registry storage plus the selected validated HTTPS/SSH Git transport files; DWH migration and ordinary maintenance mutations remain registry RO with no Git credentials. Do not create duplicate operator, state-store, lock, renderer, `internal/workspace` package, path abstraction, or second pull implementation.
|
||
|
||
### Exact released P2/P3 addressed registry-pull exception
|
||
|
||
P5 must import and consume the released names verbatim. The six P3 job names have one owning module;
|
||
production consumers import them only as follows (tests use the corresponding
|
||
`../src/workspaces/registry-pull-job.js` path):
|
||
|
||
```ts
|
||
import type {
|
||
RegistryPullPublicJobRequestV1,
|
||
RegistryPullPhaseV1,
|
||
RegistryPullAddressedJobRequestV1,
|
||
RegistryPullParticipantStateV1,
|
||
RegistryPullSynchronizerStateV1,
|
||
RegistryPullJobStateV1,
|
||
} from "./registry-pull-job.js";
|
||
```
|
||
|
||
`backend/src/workspaces/registry-pull-job.ts` exports all six names. It defines
|
||
`RegistryPullPhaseV1 = RegistryAddressedPublicationPhaseV1`,
|
||
`RegistryPullAddressedJobRequestV1 = Extract<RegistryAddressedRequestV1,
|
||
{ readonly operation: "registry_pull" }>`, and
|
||
`RegistryPullJobStateV1 = RegistryPullAddressedPublicationStateV1`; these are exact aliases owned by
|
||
P3, not names P5 may redeclare. `RegistryPullPublicJobRequestV1`,
|
||
`RegistryPullParticipantStateV1`, and `RegistryPullSynchronizerStateV1` are the other three exact P3
|
||
exports. Add a compile-only import in P5 and preserve P3's bidirectional parity assertions. No barrel,
|
||
compatibility export, local alias, or second job shape is permitted. Before P5 begins, P3's production
|
||
`registry.publishAddressed(...)` call must compile against this exact surface: the validated boundary
|
||
passes all three identities on create/resume and the expected base on create. If that owning P3 call or
|
||
its mismatch-before-mutation tests are absent, stop and reopen P3; P5 must not patch it or add an overload.
|
||
|
||
P5 consumes these exact P2 exports from `backend/src/workspaces/registry-publication.ts` and
|
||
`backend/src/workspaces/preprocessing-state.ts`. `AddressedWorkspacePublicationLeaseV1` is owned by
|
||
`registry-publication.ts`, and its exact `readers` field type
|
||
`BorrowedWorkspaceSessionReadersExclusiveLockLease` is owned by `preprocessing-state.ts`:
|
||
|
||
```ts
|
||
export type RegistryAddressedPublicationPhaseV1 =
|
||
| "request_claimed" | "target_advertised" | "target_fetched" | "planned"
|
||
| "participants_prepared" | "publication_intent_durable"
|
||
| "target_published" | "terminal_durable";
|
||
interface RegistryAddressedPlanFieldsV1 {
|
||
readonly schemaVersion: 1;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
readonly jobArtifactPath: RegistryAddressedJobArtifactPathV1;
|
||
readonly advertisedTargetCommit: Revision40;
|
||
readonly immutableTargetRef: `refs/thoth/addressed-runs/${RegistryRunId32}/target`;
|
||
readonly fetchedTargetCommit: Revision40;
|
||
readonly targetCommit: Revision40;
|
||
readonly targetManifestSha256: Sha256Hex;
|
||
readonly targetWorkspaces: readonly RegistryWorkspaceManifestIdentityV1[];
|
||
readonly changedWorkspaceIds: readonly CanonicalWorkspaceId[];
|
||
readonly changedSetSha256: Sha256Hex;
|
||
}
|
||
export interface RegistryPullAddressedPlanV1 extends RegistryAddressedPlanFieldsV1 {
|
||
readonly operation: "registry_pull";
|
||
readonly changedSetRule: "symmetric_base_target_workspace_difference";
|
||
readonly baseCommit: Revision40;
|
||
readonly baseManifestSha256: Sha256Hex;
|
||
readonly baseWorkspaces: readonly RegistryWorkspaceManifestIdentityV1[];
|
||
}
|
||
export type RegistryAddressedPlanV1 =
|
||
| RegistryBootstrapAddressedPlanV1 | RegistryPullAddressedPlanV1;
|
||
interface RegistryAddressedPublicationStateFieldsV1 {
|
||
readonly schemaVersion: 1;
|
||
readonly runId: RegistryRunId32;
|
||
readonly requestSha256: Sha256Hex;
|
||
readonly jobArtifactPath: RegistryAddressedJobArtifactPathV1;
|
||
readonly phase: RegistryAddressedPublicationPhaseV1;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
readonly advertisedTargetCommit: Revision40 | null;
|
||
readonly immutableTargetRef: `refs/thoth/addressed-runs/${RegistryRunId32}/target` | null;
|
||
readonly fetchedTargetCommit: Revision40 | null;
|
||
readonly targetCommit: Revision40 | null;
|
||
readonly targetManifestSha256: Sha256Hex | null;
|
||
readonly targetWorkspaces: readonly RegistryWorkspaceManifestIdentityV1[] | null;
|
||
readonly changedWorkspaceIds: readonly CanonicalWorkspaceId[] | null;
|
||
readonly planSha256: Sha256Hex | null;
|
||
readonly changedSetSha256: Sha256Hex | null;
|
||
readonly participantsSha256: Sha256Hex | null;
|
||
readonly synchronizersSha256: Sha256Hex | null;
|
||
readonly publicationIntentSha256: Sha256Hex | null;
|
||
readonly publishedActiveStateSha256: Sha256Hex | null;
|
||
readonly terminalResultSha256: Sha256Hex | null;
|
||
readonly priorStateSha256: Sha256Hex | null;
|
||
}
|
||
export interface RegistryPullAddressedPublicationStateV1
|
||
extends RegistryAddressedPublicationStateFieldsV1 {
|
||
readonly operation: "registry_pull";
|
||
readonly baseCommit: Revision40;
|
||
readonly baseManifestSha256: Sha256Hex;
|
||
readonly baseWorkspaces: readonly RegistryWorkspaceManifestIdentityV1[];
|
||
readonly changedSetRule: "symmetric_base_target_workspace_difference" | null;
|
||
}
|
||
export interface AddressedWorkspacePublicationLeaseV1
|
||
extends BorrowedOrderedWorkspaceWriterLeaseV1 {
|
||
readonly quiescence: BorrowedWorkspaceMaintenanceQuiescenceLease;
|
||
readonly readers: BorrowedWorkspaceSessionReadersExclusiveLockLease;
|
||
}
|
||
export interface CapabilityAwareRegistryPublicationParticipant<T> {
|
||
readonly participantId: string;
|
||
prepare(plan: RegistryAddressedPlanV1,
|
||
workspace: AddressedWorkspacePublicationLeaseV1): Promise<T>;
|
||
reconcile(plan: RegistryAddressedPlanV1,
|
||
workspace: AddressedWorkspacePublicationLeaseV1, prepared: T,
|
||
phase: RegistryAddressedPublicationPhaseV1): Promise<void>;
|
||
}
|
||
export class CapabilityAwareRegistryPublicationLifecycleOwner {
|
||
run<T>(input: {
|
||
readonly plan: RegistryAddressedPlanV1;
|
||
readonly capabilities: OrderedWorkspaceWriterCapabilitySet;
|
||
readonly participants: readonly CapabilityAwareRegistryPublicationParticipant<unknown>[];
|
||
readonly synchronizers: readonly CapabilityAwareRegistryPublicationSynchronizer[];
|
||
readonly action: () => Promise<T>;
|
||
}): Promise<T>;
|
||
}
|
||
export type RegistryAddressedRequestV1 =
|
||
| {
|
||
readonly mode: "create";
|
||
readonly operation: "registry_bootstrap";
|
||
readonly runId: RegistryRunId32;
|
||
readonly requestSha256: Sha256Hex;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly expectedBaseCommit: null;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
}
|
||
| {
|
||
readonly mode: "resume";
|
||
readonly operation: "registry_bootstrap";
|
||
readonly runId: RegistryRunId32;
|
||
readonly requestSha256: Sha256Hex;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
}
|
||
| {
|
||
readonly mode: "create";
|
||
readonly operation: "registry_pull";
|
||
readonly runId: RegistryRunId32;
|
||
readonly requestSha256: Sha256Hex;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly expectedBaseCommit: Revision40;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
}
|
||
| {
|
||
readonly mode: "resume";
|
||
readonly operation: "registry_pull";
|
||
readonly runId: RegistryRunId32;
|
||
readonly requestSha256: Sha256Hex;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
};
|
||
export class WorkspaceRegistry {
|
||
publishAddressed(request: RegistryAddressedRequestV1): Promise<RegistryAddressedResultV1>;
|
||
}
|
||
```
|
||
|
||
This is a field-for-field dependency quotation, not a compatibility sketch. The validated
|
||
installation/registry boundary derives the production `installationIdentitySha256`,
|
||
`repositoryIdentitySha256`, and `remoteRefIdentitySha256` and supplies all three on every create and
|
||
resume member; pull create additionally supplies the exact `expectedBaseCommit`. The same three
|
||
identities are required, non-optional fields in the immutable plan and durable state, and same-ID
|
||
resume revalidates them before network or state mutation. `P5` adds no optional identity, partial
|
||
request, constructor decoration, compatibility overload, or second `publishAddressed` signature.
|
||
|
||
The production registry constructor, not the request, owns the single
|
||
`VerifiedWorkspaceLockRootLeaseFactory`, `CapabilityAwareRegistryPublicationLifecycleOwner`,
|
||
participant list, and synchronizer list. The sole pull order is repository lock → durable
|
||
`request_claimed` → durable target advertisement → exact-OID fetch to the create-only immutable ref →
|
||
`target_fetched` → immutable pull plan → `planned` → acquire/provision every complete changed root →
|
||
one `runUnderOrderedWorkspaceWriterLocks` callback → lexical quiescence/readers → participant prepare
|
||
→ `participants_prepared` → durable publication intent → active-state publication/reconciliation →
|
||
`target_published` → `terminal_durable` → callback settlement invalidates and reverse-closes
|
||
readers/quiescence/writers/roots → repository lock release. The artifact is exactly
|
||
`addressed-publication-jobs/<run-id>.json`.
|
||
|
||
P5 calls only `WorkspaceRegistry.publishAddressed` with the `registry_pull` member. Its participant is
|
||
called by the existing lifecycle owner with the same callback-scoped
|
||
`AddressedWorkspacePublicationLeaseV1`; it consumes that lease's opaque `rootLease` and
|
||
`writerCapability`, never reacquires either, and never calls a repository method. The complete ordered
|
||
capability set stays live through active-pointer sibling write, file fsync, rename, parent fsync,
|
||
byte verification, and terminal durability. Same-ID recovery uses only
|
||
`RegistryPullJobStateV1`: it never refetches/reselects after the durable target pin, accepts only the
|
||
recorded all-base/all-target/mixed identities, reconciles a lost publication acknowledgement without a
|
||
second rename, and refuses a third identity or target drift. A new process may reacquire the recorded
|
||
complete set once; no live attempt permits nested acquisition or participant reentry.
|
||
|
||
### Canonical content and runtime contract
|
||
|
||
```text
|
||
Git path (fixed):
|
||
workspace-content/<workspace-id>/schema/annotations.yaml
|
||
|
||
Runtime bytes (exact Git blob or canonical absent value):
|
||
/data/sessions/<workspace-id>/revisions/<commit>/artifacts/mschema/annotations.yaml
|
||
|
||
Adjacent publication record:
|
||
/data/sessions/<workspace-id>/revisions/<commit>/artifacts/mschema/annotations.manifest.json
|
||
```
|
||
|
||
Use these constants and shapes:
|
||
|
||
```ts
|
||
export const MAX_FK_CANDIDATE_BYTES = 716_800; // exact P2 decoded candidate/export maximum
|
||
export const MAX_CURATED_ANNOTATION_BYTES = 16 * 1024 * 1024; // Git blob only
|
||
export const EMPTY_ANNOTATIONS = Buffer.from("tables: {}\n", "utf8");
|
||
|
||
export interface AnnotationGitObject {
|
||
source: "git" | "absent";
|
||
path: string;
|
||
mode: "100644" | "100755" | null;
|
||
blobId: string | null; // exact 40-hex object ID when present
|
||
bytes: Buffer; // exact blob bytes; absent uses EMPTY_ANNOTATIONS
|
||
sha256: string;
|
||
warning?: "annotation_missing";
|
||
}
|
||
|
||
export interface AnnotationOwnershipManifest {
|
||
schema_version: 1;
|
||
kind: "workspace_fk_annotations";
|
||
workspace_id: string;
|
||
workspace_revision: string; // exact 40-hex commit
|
||
source_path: string;
|
||
source: "git" | "absent";
|
||
blob_id: string | null;
|
||
content_sha256: string;
|
||
byte_length: number;
|
||
destination: string; // exact absolute annotations.yaml destination
|
||
}
|
||
```
|
||
|
||
The manifest JSON is canonical single-line JSON plus `\n`, mode `0400`; the annotation is mode `0400`; created directories are `0700`. A present object may use Git regular-file mode `100644` or `100755`; the synchronized result is never executable. Reject mode `120000` (symlink), `160000` (gitlink), `040000` (tree), other types/modes, a curated Git blob greater than exactly 16 MiB, malformed UTF-8, NUL, duplicate YAML keys, multiple documents, aliases/merge keys, unknown fields, and invalid annotation models. Missing path is compatible: validate and publish `tables: {}\n`, set `source: "absent"`, `blob_id: null`, and return a safe warning. A present zero-byte/empty YAML document is `annotation_invalid`; absence and an empty file are not equivalent.
|
||
|
||
The two byte limits are intentionally different and must never share a constant or fallback. Every P2/P5 state-owned suggested candidate and the decoded `schema-export-fks` payload is at most exactly 716,800 bytes. Its base64 `hostExport` plus the complete machine envelope remains at most exactly 1 MiB. Only the separately curated Git annotation blob may be as large as exactly 16 MiB; it is read from Git after the author edits/commits it and never traverses `hostExport`. Tests must pass at 716,800/16 MiB and fail before buffering/publication at 716,801/16 MiB + 1 respectively.
|
||
|
||
`physical.yaml` remains local DWH-derived content. Never add it to Git, an author export, a registry bundle, or an annotation manifest.
|
||
|
||
### Shared fixed-Git child contract
|
||
|
||
P5 introduces the single reusable lifecycle boundary `backend/src/workspaces/fixed-git-child.ts::runFixedGitChild`; P6 must extend/reuse it rather than adding another subprocess wrapper. Repository methods supply only reviewed, fixed Git argv vectors and one caller-owned monotonic absolute deadline shared across every stage of the logical operation. The helper starts Git shell-free in a new owned process group, writes only the exact declared stdin (empty for P5 plumbing) and closes it, streams rather than `execFile`-buffers, enforces both a per-stage stdout byte bound and parser record bound plus a small fixed stderr bound, and returns only parsed allowlisted values.
|
||
|
||
On deadline, caller cancellation, parser rejection, early/abnormal exit, stdout/record overflow, or stderr overflow, stop parsing and close stdin, send TERM only to the owned process group, wait the fixed grace, send KILL only to that group if necessary, and always await child exit plus stream settlement before returning. No lock-sensitive caller may continue while a child or descendant is alive. Raw stdout, stderr, argv-derived paths, repository endpoints, and child exceptions never enter public errors or logs. Real-process tests cover a hang, stdout and record floods, stderr flood, early exit, parser abort, cancellation, a descendant holding a pipe open, TERM→KILL escalation, and proof that exit/stdio cleanup was awaited. P5's `ls-tree`, `cat-file -s`, and `cat-file blob` stages share one operation deadline; their stdout/record bounds are respectively the exact bounded control record, exact bounded size record, and advertised blob length no greater than `MAX_CURATED_ANNOTATION_BYTES`.
|
||
|
||
### Controlled acceptance contract
|
||
|
||
P2's same-revision manual-review state is explicitly migrated, not silently reinterpreted. P2 stores state in `/data/sessions/<workspace-id>/preprocessing/jobs/<outer-run-id>.json`, candidates under `fk-candidates/<run-id>.yaml`, reviews under `fk-reviews/<run-id>.json`, all guarded by the persistent regular `0600` advisory lock `/data/sessions/<workspace-id>/preprocessing/writer.lock`. P5 keeps those exact `PreprocessingStateStore` paths and freezes one V2 checkpoint shape:
|
||
|
||
```ts
|
||
interface FkReviewCheckpointV2Base {
|
||
kind: "fk_review";
|
||
candidateDigest: `sha256:${string}`;
|
||
candidateCount: number;
|
||
baseAnnotationsDigest: `sha256:${string}`;
|
||
reviewedDigest?: `sha256:${string}`; // copied P2 audit fact only; never canonical/accepted
|
||
baseWorkspaceRevision: string;
|
||
baseGitBlob: string | null;
|
||
}
|
||
interface PendingFkReviewCheckpointV2 extends FkReviewCheckpointV2Base {
|
||
status: "pending";
|
||
}
|
||
interface AcceptedFkReviewCheckpointV2 extends FkReviewCheckpointV2Base {
|
||
status: "accepted";
|
||
acceptedWorkspaceRevision: string;
|
||
acceptedGitBlob: string; // exact present 40-hex Git blob
|
||
acceptedCandidateDigest: `sha256:${string}`;
|
||
acceptedBlobDigest: `sha256:${string}`;
|
||
acceptedEffectiveDwhBinding: string;
|
||
acceptedAt: string;
|
||
}
|
||
type FkReviewCheckpointV2 = PendingFkReviewCheckpointV2 | AcceptedFkReviewCheckpointV2;
|
||
```
|
||
|
||
There is no V2 `required` or `reviewed` status. `manual_review_required` remains the public P2 operation result/error code, not an on-disk checkpoint status. The only normal state edge is `pending → accepted`; an exact replay of the identical complete acceptance object is idempotent, while any other `accepted → *`, rewind, skip, or partial acceptance is invalid.
|
||
|
||
The V1 source in this table means the exact canonical JSON bytes emitted by the completed P2 `PreprocessingStateStore` and `FkReviewRecordV1`, not a structurally plausible handwritten object. At Task 5 start, run the unmodified P2 writer to commit two byte fixtures—paused without a review and paused with a valid same-revision review—under `backend/test/fixtures/preprocessing-v1-fk-review/`, record their SHA-256 and candidate bytes in `README.md`, and prove the P2 reader accepts them before adding a V2 reader. Any P2 V1 field/version drift is a prerequisite failure to repair at P2, not a permissive P5 parser change.
|
||
|
||
Freeze this migration/transition table before code changes:
|
||
|
||
| Exact source bytes/state | Required validation | V2 result / behavior |
|
||
|---|---|---|
|
||
| P2 V1 nonterminal job at `manual_review_required`, state-owned candidate present, no `FkReviewRecordV1` | strict V1 keys/version/identity; exact regular no-follow candidate; rehash equals recorded digest/count; exact base revision is still available; synchronize/verify that revision's Git annotation identity | atomically write V2 `pending`; copy candidate fields and verified base revision/blob/content digest; no `reviewedDigest` |
|
||
| Same paused P2 V1 job plus a valid same-revision `FkReviewRecordV1`, with no later stage recorded | all checks above; review candidate/workspace/revision/digest exactly match the V1 job | atomically write V2 `pending`; copy the V1 reviewed annotation digest to optional `reviewedDigest` for audit only; still require explicit Git acceptance |
|
||
| V2 `pending` plus a complete verified acceptance object | candidate rehash matches; repository→writer protocol below; new active revision, present blob, checked bytes, unchanged effective-DWH binding | atomically write V2 `accepted` with all accepted fields |
|
||
| V2 `accepted` plus byte-identical complete acceptance object | revalidate the stored object, candidate, active identity, synchronized blob, and binding | return the existing `accepted` value without rewriting |
|
||
| P2 V1 run that already recorded any stage after FK review or is terminal | strict V1 validation only | preserve immutable legacy audit state; it is not exportable/acceptable/resumable under P5; return `preprocessing_resume_mismatch` and require a new run |
|
||
| Unknown V1/V2 keys/version/status, missing or mismatched candidate/review, partial accepted fields, identity conflict, or any unlisted source/edge | none may be repaired heuristically | fail closed before publication or later-stage work |
|
||
|
||
The migration service supplies the verified base Git identity; the state store never invents a blob ID from P2 bytes. Migration and transition use `PreprocessingStateStore`'s closed transition union and atomic writer, never a partial merge.
|
||
|
||
`workspace schema accept --run <id> --yes` is the only human-decision transition. Because the public grammar intentionally omits a workspace, `WorkspacePreprocessingService.execute` first calls the bounded `resolveRun(runId)` contract: validate the exact 32-hex run ID; enumerate only IDs from the strict active workspace set (bounded by the existing registry limit, never recursively walk `/data/sessions`); for each ID open only `<dataRoot>/<id>/preprocessing/jobs/<run>.json` as an exact regular no-follow file through P2 `PreprocessingStateStore`'s installation-root-relative no-follow reader; bound every file and the whole operation; verify embedded run/workspace identity; and require exactly one match. Zero matches returns `preprocessing_run_not_found`; two or more returns `preprocessing_run_ambiguous`; malformed IDs, symlink/hardlink/embedded-identity mismatch, unsafe state, or traversal input returns `preprocessing_state_invalid`; a run whose former workspace is not in the active set is deliberately indistinguishable from not found. These three codes are stable P5 public codes with no searched paths or workspace list. The same resolver is used by exact candidate export. After the later writer lock is acquired, the service reopens and revalidates the exact resolved state so a stale locator result is never trusted.
|
||
|
||
The curator must run P3's released `thothctl --installation <absolute>/thothii-installation.yaml workspace registry pull --workspace <resolved-workspace-id> --json` after the curated commit is pushed and before acceptance (`--resume <32-hex-outer-run-id>` is only that exact interrupted pull's continuation). That distinct `RegistryPullCommand` / `registry_pull` operation alone receives writable registry storage plus the selected validated HTTPS/SSH Git transport files and invokes P2 `WorkspaceRegistry.publishAddressed(request: RegistryAddressedRequestV1)` rather than the common writer-first migration action. Inside the validated installation/registry boundary, create and resume both supply the exact production `installationIdentitySha256`, `repositoryIdentitySha256`, and `remoteRefIdentitySha256`; create also supplies the exact active `expectedBaseCommit`. While holding `repository.lock`, create fetches once, pins all three identities in the exact `RegistryPullAddressedPlanV1`, and persists them in `RegistryPullJobStateV1` at `planned`. `WorkspaceRegistry` enters the sole `runUnderOrderedWorkspaceWriterLocks` callback for the complete strict-lexical `changedWorkspaceIds`, and P5's exact `CapabilityAwareRegistryPublicationParticipant<AnnotationRegistryPreparationV1>` receives each matching `AddressedWorkspacePublicationLeaseV1`. It prepares target manifests (or a closed base-only removal result) before `publication_intent_durable`; it never calls `runUnderWorkspaceWriterLock`, acquires roots/readers, invokes `publishAddressed`, or calls repository. The `OrderedWorkspaceWriterCapabilitySet` stays live through the single active-snapshot-pointer write+file fsync+atomic rename+parent fsync and synchronous target-byte verification, then through `target_published` and `terminal_durable`; only then does ordered callback settlement invalidate and reverse-close readers/quiescence/capabilities/root leases, followed by `repository.lock` release. Same-ID resume loads only the persisted addressed state: all-base before publication continues without fetch, all-target after rename+fsync advances lost acknowledgement without refetch or a second publication, and mixed exact base/target converges to target while the reacquired full recorded set is held. Any third active identity, changed pinned target object/inventory/digest, installation/repository/remote-ref identity drift, or moved local remote-tracking OID returns `preprocessing_resume_mismatch` before network or state mutation and preserves nonterminal ownership. This is the sole supported pull path.
|
||
|
||
Immediately after pull and still before acceptance, the curator must run the exact released command `thothctl --installation <absolute>/thothii-installation.yaml workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json`, where `WORKSPACE_ID` is the resolved workspace ID (`--resume <32-hex-outer-run-id>` is only that interrupted `migrate_dwh_cache` run's continuation). This is a non-activating P3 maintenance transition: the fixed `p3_migrate_dwh_cache`/`p3_prepare_dwh_cache` cache stage plus `p3_materialize_dwh_snapshot` must bind to the pulled revision and probed effective binding, reuse or prepare the immutable cache, publish the exact `revisions/<commit>/dwh-snapshots/<binding-key>/{artifacts,indexes}` physical/LSH snapshot, and strictly reverify its identities/digests without requiring or publishing `READY.json`. It must never read the prior commit's revision snapshot as a fallback. After the dedicated job records terminal durability, performs owner-only clear, and releases its exclusive reader gate, session admission and preprocessing resume still return `migration_required`; that remains true through acceptance and changes only after activation publishes READY. Every migration and ordinary mutation—including `schema-accept`—keeps P2's registry RO mount and receives no Git transport.
|
||
|
||
`schema accept` itself never pulls, spawns Git, calls a repository method, or takes the repository lock. After bounded `resolveRun`, it reads the already-active immutable snapshot and prepared annotation identity, acquires the existing root through P2's installation-bound factory, and enters a one-element `runUnderOrderedWorkspaceWriterLocks` callback. Inside `forWorkspace`, it receives the callback-scoped `rootLease` plus `writerCapability`, rereads `active.json`, reopens and revalidates the resolved paused state/candidate, and calls only `AnnotationSynchronizer.verifyPrepared(..., { kind: "prepare", rootLease, writerCapability })`. The verifier never acquires a lock or spawns directly: its exact `annotation_verify` request goes through `writerCapability.spawnChild`, and it reads only the already-materialized revision selected by `futureWorkspaceLayoutPaths(rootLease, workspaceId, workspaceRevision)`. Because P3 deliberately leaves a newly pulled layout-v1 revision without `READY.json`, accept is a maintenance-only check using P3's trusted `layoutIntent: "prepare-revision-layout-v1"` future resolver; it never calls `readRevisionLayoutState`/`workspaceRuntimePaths`, admits a session, or publishes READY. It runs `schema check` against the already-published and strictly reverified P3 physical/LSH snapshot for that exact future revision+binding (never a prior-revision fallback), compares and reports candidate/current digests (they may differ after editing), requires a new revision and present blob, requires the paused `canonical_effective_dwh_binding(cfg)`, and atomically publishes V2 `accepted`. A racing pull may hold repository but must block on writer before changing `active.json`; acceptance never requests repository and therefore sees one stable identity without deadlock. The exact `CapabilityAwareRegistryPublicationParticipant` retains the released repository-first order without acquiring locks itself; P2/P3 holds the complete `OrderedWorkspaceWriterCapabilitySet` through active-pointer rename+parent fsync before reverse release, and tests structurally reject every writer→repository path.
|
||
|
||
After acceptance, admission and the explicit preprocessing run still fail `migration_required`. Run P3's released `thothctl --installation <abs> workspace migrate activate-revision-layout --workspace <id> --yes [--resume <32-hex-outer-run-id>] [--json]` to reverify the prepared snapshot/materializations and publish the accepted revision's exact `READY.json`; only then may the run resume on its human-accepted A→B transition. The P5 acceptance record is the only authority for that preprocessing-run rebind. Ordinary sessions remain pinned to their recorded revision and are never silently retargeted.
|
||
|
||
An explicit `--resume <id>` fails with `preprocessing_resume_mismatch` if the accepted commit/blob/digest or effective DWH binding no longer matches. An unqualified new run may start with the newer revision. `schema check`, an empty file, digest equality, Git publication, or the legacy P2 review record alone never records human acceptance; no silent rebinding is allowed.
|
||
|
||
### Hard boundaries
|
||
|
||
- Do not publish or push curated bytes from the application checkout or maintenance container. Curators use an ordinary Git author clone.
|
||
- Do not broaden P1 HTTP workspace ZIP import/export to carry `workspace-content`; P5's “export/import” means candidate export to an author clone, ordinary Git commit/push, registry pull, pre-READY DWH snapshot preparation, then acceptance/import from the exact Git blob.
|
||
- Do not create a mutable `current` symlink, copy annotations into a workspace-global artifact root, or rewrite an old revision directory.
|
||
- Do not implement Evidence enumeration/materialization, nested Evidence limits, Evidence retention, Qdrant Evidence writes, or P6 cleanup.
|
||
- Do not change DWH fingerprint semantics, memory migration, or collection lifecycle; consume P3/P4 APIs.
|
||
- Do not add a GUI or backend preprocessing HTTP endpoint.
|
||
- Do not broaden maintenance capability to make acceptance work: only P3 `registry_pull` is registry RW/Git; released `migrate_dwh_cache`, `schema-accept`, export, resume, and every other ordinary mutation remain registry RO/no-Git. The DWH migration's existing bounded DWH capability does not grant Git or registry mutation.
|
||
- Do not route `registry_pull` through P3's common writer-first migration lifecycle, enter repository while any writer is already held, or let `AnnotationSynchronizer.ensure` acquire/reacquire a writer. Invoke only `WorkspaceRegistry.publishAddressed(RegistryAddressedRequestV1)` with the exact `CapabilityAwareRegistryPublicationLifecycleOwner`, `OrderedWorkspaceWriterCapabilitySet`, and `CapabilityAwareRegistryPublicationParticipant`; keep the set held through active-pointer rename+parent fsync and terminal durability, reverse-release it, then release repository.
|
||
|
||
## Target file map
|
||
|
||
**Harness parser**
|
||
- Modify: `harness/tht/mschema/models.py`
|
||
- Modify: `harness/tht/cli/schema_cmd.py`
|
||
- Modify: `harness/tests/test_schema_fk_annotations.py`
|
||
- Create: `harness/tests/test_annotation_validation_cli.py`
|
||
- Create: `harness/tht/protected_fs_helper.py` (internal fixed-protocol Linux `openat`/`renameat` helper; not a public `tht` command)
|
||
- Create: `harness/tests/test_protected_fs_helper.py`
|
||
- Modify: `harness/tht/locked_child_stdin.py`
|
||
- Modify: `harness/tests/test_locked_child_stdin.py`
|
||
|
||
**Git and materialization**
|
||
- Create: `backend/src/workspaces/fixed-git-child.ts`
|
||
- Create: `backend/test/fixed-git-child.test.ts`
|
||
- Modify: `backend/src/workspaces/git-repository.ts`
|
||
- Modify: `backend/src/workspaces/types.ts`
|
||
- Create: `backend/src/workspaces/annotations.ts`
|
||
- Create: `backend/src/workspaces/protected-workspace-fs.ts` (capability-only adapter over P2 `rootLease` + `writerCapability.spawnChild`; no path/direct-spawn fallback)
|
||
- Create: `backend/src/workspaces/registry-factory.ts`
|
||
- Modify: `backend/src/workspaces/registry.ts`
|
||
- Modify: `backend/src/app.ts`
|
||
- Modify: `backend/test/workspaces-git-repository.test.ts`
|
||
- Create: `backend/test/workspace-annotations.test.ts`
|
||
- Create: `backend/test/protected-workspace-fs.test.ts`
|
||
- Create: `backend/test/workspace-registry-factory.test.ts`
|
||
- Modify: `backend/test/workspace-registry.test.ts`
|
||
- Modify: `backend/test/workspace-runtime-handoff.test.ts`
|
||
|
||
**P2/P3 operator and runtime integration (these files must already exist)**
|
||
- Modify: `backend/src/workspaces/preprocessing-state.ts` (sole locked-child union/dispatcher extension, strict V1→V2 migration, bounded `resolveRun` support; no second filesystem/spawn path)
|
||
- Modify: `backend/src/workspaces/preprocessing-service.ts`
|
||
- Modify: `backend/src/workspace-maintenance.ts`
|
||
- Modify: `backend/src/workspaces/runtime-config-lease.ts`
|
||
- Read/consume unchanged: `backend/src/workspaces/revision-layout.ts`
|
||
- Modify: `backend/test/workspace-preprocessing-state.test.ts`
|
||
- Modify: `backend/test/workspace-preprocessing-service.test.ts`
|
||
- Modify: `backend/test/workspace-maintenance.test.ts`
|
||
- Modify: `backend/test/workspace-runtime-config-lease.test.ts`
|
||
- Modify: `backend/test/workspace-revision-layout.test.ts`
|
||
- Modify: `backend/test/workspace-runtime-renderer.test.ts`
|
||
- Modify: `backend/test/routes-sessions.test.ts`
|
||
|
||
**Host CLI**
|
||
- Modify: `tools/thothctl/internal/workspaceops/operations.go`
|
||
- Modify: `tools/thothctl/internal/workspaceops/operations_test.go`
|
||
- Read/verify unchanged from repaired P3: `tools/thothctl/internal/config/installation.go` and tests (operation-specific registry/Git bindings)
|
||
- Read/verify unchanged from repaired P3: `compose.yaml`, `deploy/compose.git-{https,ssh}.yaml`, `scripts/generate-connector-secrets-override.sh`, and their Compose/secret-policy tests (`registry_pull` RW/Git; ordinary mutations RO/no-Git)
|
||
- Modify: `tools/thothctl/internal/safeio/files.go`
|
||
- Modify: `tools/thothctl/internal/safeio/files_unix_test.go`
|
||
- Modify: `tools/thothctl/internal/safeio/files_windows_test.go`
|
||
- Modify: `tools/thothctl/cmd/thothctl/main.go`
|
||
- Modify: `tools/thothctl/cmd/thothctl/main_test.go`
|
||
|
||
**Process goal, docs, and checkpoint**
|
||
- Create: `backend/scripts/p5-acceptance.mjs`
|
||
- Create: `backend/scripts/p5-acceptance.test.mjs`
|
||
- Create: `scripts/p5-acceptance.sh`
|
||
- Create: `scripts/test-p5-acceptance.sh`
|
||
- Create: `backend/scripts/p5-manual-verification.mjs`
|
||
- Create: `backend/scripts/p5-manual-verification.test.mjs`
|
||
- Create: `scripts/p5-manual-verification.sh`
|
||
- Modify: `docs/testing/p2-p6-manual-verification.md`
|
||
- Create: `docs/contracts/workspace-annotations.md`
|
||
- Modify: `docs/install/local-workspace-registry.md`
|
||
- Modify: `docs/install/server-workspace-registry.md`
|
||
- Modify: `PROJECT_STATE.md`
|
||
|
||
---
|
||
|
||
### Task 1: Make the harness annotation parser a strict reusable validation boundary
|
||
|
||
**Files:**
|
||
- Modify: `harness/tht/mschema/models.py`
|
||
- Modify: `harness/tht/cli/schema_cmd.py`
|
||
- Modify: `harness/tests/test_schema_fk_annotations.py`
|
||
- Create: `harness/tests/test_annotation_validation_cli.py`
|
||
|
||
- [ ] **Step 1 (RED): write parser tests before production code.**
|
||
|
||
Add `parse_annotations_yaml(source: bytes) -> Annotations` tests for valid Italian UTF-8, `tables: {}`, all existing FK fields, and deterministic counts. Add one parameterized rejection test for invalid UTF-8, NUL, zero bytes, whitespace-only content, duplicate keys, multiple YAML documents, anchors/aliases, merge keys, unknown root/table/column/FK keys, non-mapping root, mismatched/empty FK column lists, and a 16 MiB + 1 input. Preserve `Annotations.from_yaml(path)` missing-path compatibility, but make an existing malformed file fail.
|
||
|
||
Add CLI tests using `typer.testing.CliRunner`:
|
||
|
||
```python
|
||
def test_validate_annotations_reads_stdin_and_emits_pristine_json():
|
||
result = runner.invoke(app, ["schema", "validate-annotations", "--stdin", "--json"],
|
||
input="tables: {}\n")
|
||
assert result.exit_code == 0
|
||
assert json.loads(result.stdout) == {
|
||
"valid": True, "tables": 0, "foreign_keys": 0, "byte_length": 11
|
||
}
|
||
assert result.stderr == ""
|
||
```
|
||
|
||
A bad document must exit nonzero with no stdout and a bounded generic stderr message that does not echo input.
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/pytest tests/test_schema_fk_annotations.py tests/test_annotation_validation_cli.py -q
|
||
```
|
||
|
||
Expected: FAIL because `parse_annotations_yaml` and `validate-annotations` do not exist and current Pydantic/PyYAML parsing accepts extras/duplicates.
|
||
|
||
- [ ] **Step 3 (GREEN): implement the minimal strict parser.**
|
||
|
||
In `models.py`, add `MAX_ANNOTATION_BYTES = 16 * 1024 * 1024`, strict annotation-only Pydantic configs (`extra="forbid"`), annotation-FK validators requiring equal nonempty `columns`/`ref_columns`, and a `yaml.SafeLoader` subclass whose mapping constructor rejects duplicate keys and whose alias/merge handling fails closed. Decode bytes with `utf-8` strict, reject NUL/empty, call `yaml.load_all`, require exactly one mapping document, then `Annotations.model_validate` it. Keep physical-schema compatibility unchanged: do not globally make `_YamlModel` or the shared physical `ForeignKey` stricter. Reject annotation FK extras with an annotation-only model or a pre-validation allowlist, then map to the shared FK value object.
|
||
|
||
Add `schema validate-annotations` with exact `--stdin --json` flags. Use `sys.stdin.buffer.read(MAX_ANNOTATION_BYTES + 1)`, never accept a caller path, and emit only the count envelope on stdout.
|
||
|
||
- [ ] **Step 4 (verify GREEN and lint):**
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/pytest tests/test_schema_fk_annotations.py tests/test_annotation_validation_cli.py \
|
||
tests/test_protected_fs_helper.py -q
|
||
.venv/bin/ruff check tht/mschema/models.py tht/cli/schema_cmd.py tht/protected_fs_helper.py \
|
||
tests/test_schema_fk_annotations.py tests/test_annotation_validation_cli.py \
|
||
tests/test_protected_fs_helper.py
|
||
```
|
||
|
||
Expected: all focused tests PASS and Ruff reports no errors.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add harness/tht/mschema/models.py harness/tht/cli/schema_cmd.py \
|
||
harness/tests/test_schema_fk_annotations.py harness/tests/test_annotation_validation_cli.py
|
||
git commit -m "feat: validate curated annotation documents strictly"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: Read one exact bounded annotation Git blob without checkout semantics
|
||
|
||
**Files:**
|
||
- Create: `backend/src/workspaces/fixed-git-child.ts`
|
||
- Create: `backend/test/fixed-git-child.test.ts`
|
||
- Modify: `backend/src/workspaces/git-repository.ts`
|
||
- Modify: `backend/src/workspaces/types.ts`
|
||
- Modify: `backend/test/workspaces-git-repository.test.ts`
|
||
|
||
- [ ] **Step 1 (RED): add real-Git tests for `readAnnotationBlobAtRevision`.**
|
||
|
||
Extend the local bare-remote fixture with present, missing, executable regular blob, symlink, tree, and gitlink annotation paths. Tests must prove:
|
||
|
||
1. path is derived only as `workspace-content/<validated-id>/schema/annotations.yaml`;
|
||
2. lookup uses the supplied historical 40-hex commit, not `HEAD`;
|
||
3. a present regular blob returns exact mode/blob/bytes/SHA-256;
|
||
4. absence returns the canonical absent object and warning identity;
|
||
5. symlink/tree/gitlink are `annotation_invalid`;
|
||
6. exactly 16 MiB passes and 16 MiB + 1 fails before buffering the body;
|
||
7. malformed revision/ID, caller-controlled namespace/path attempts, and shell-like strings never create a marker file or escape the derived requested path;
|
||
8. a commit containing valid `workspace-content/other-workspace/schema/annotations.yaml` coexists: a request for the target reads only the target blob, never rejects or mutates the unrelated namespace, and a target absence remains the canonical absent value even when the unrelated blob exists;
|
||
9. corrupt Git storage is `git_unavailable`, not falsely “absent”; Git stderr/path/remote is redacted.
|
||
|
||
First test the shared `runFixedGitChild` with real fixture processes: one monotonic absolute deadline spans all stages and is never reset; stdin is written exactly then closed; stdout bytes, parsed record length, and stderr are independently bounded. Cover a hang, infinite stdout, an individual record crossing its bound while aggregate output is still below its limit, infinite stderr, early/nonzero exit, parser throw, `AbortSignal` cancellation, and a descendant that keeps a pipe open or ignores TERM. Each failure must immediately stop parsing, TERM then KILL only the owned group, await the exit/close and every stdio settlement, retain at most the configured bounds, and return only a safe classification. Assert the parent/test process and an unrelated process group survive.
|
||
|
||
Instrument the repository seam and assert only fixed vectors are used through that shared helper: `ls-tree -z <commit> -- <path>`, `cat-file -s <blob>`, `cat-file blob <blob>`. All three share one logical-operation deadline; empty stdin is closed for each stage. Bound `ls-tree` to one complete derived-path NUL record, `cat-file -s` to one complete decimal size record, `cat-file blob` to the already-validated advertised length at most `MAX_CURATED_ANNOTATION_BYTES`, and stderr to the shared small fixed cap. Do not use `git show`, a shell, checkout, archive extraction, a mobile author file, `execFile`/`maxBuffer`, or a private second child runner.
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run test/fixed-git-child.test.ts \
|
||
test/workspaces-git-repository.test.ts -t "fixed Git child|annotation"
|
||
```
|
||
|
||
Expected: FAIL because `runFixedGitChild`, the repository method, and `annotation_invalid` do not exist.
|
||
|
||
- [ ] **Step 3 (GREEN): implement the fixed-argv reader.**
|
||
|
||
Add `annotation_invalid` to `WorkspaceErrorCode`. Parse the single NUL-terminated `ls-tree` record as `<mode> <type> <40hex>\t<exact path>\0`; zero records means absent, more than one or any mismatch is invalid. Derive the path only from the validated requested ID; another valid workspace namespace in the same commit is unrelated content and must not affect this lookup. Accept only `type=blob` and mode `100644|100755`. Query size before bytes; require safe integer `0..MAX_CURATED_ANNOTATION_BYTES`.
|
||
|
||
Implement `runFixedGitChild` in the shared module exactly as frozen above and make the repository reader use it for all three stages with one caller-created monotonic deadline. The helper must stream bounded binary data, enforce record and stderr bounds during reads, close exact stdin, abort on parser/caller/limit/deadline failures, own and terminate only its process group with TERM→KILL, and await exit/stream teardown on every path. There is no private `spawnGit`, `execFile` buffer, or returned raw child output. Require exit 0 and exact advertised blob size, then compute SHA-256 after the exact bounded read.
|
||
|
||
- [ ] **Step 4 (GREEN): add regression coverage to the complete repository test.**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run test/fixed-git-child.test.ts test/workspaces-git-repository.test.ts
|
||
npx tsc --noEmit -p .
|
||
```
|
||
|
||
Expected: PASS; no existing Evidence-tree behavior regresses.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add backend/src/workspaces/fixed-git-child.ts backend/test/fixed-git-child.test.ts \
|
||
backend/src/workspaces/git-repository.ts backend/src/workspaces/types.ts \
|
||
backend/test/workspaces-git-repository.test.ts
|
||
git commit -m "feat: read revision-pinned annotation blobs"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: Atomically synchronize exact bytes with a verified ownership manifest
|
||
|
||
**Files:**
|
||
- Create: `backend/src/workspaces/protected-workspace-fs.ts`
|
||
- Create: `backend/test/protected-workspace-fs.test.ts`
|
||
- Create: `harness/tht/protected_fs_helper.py`
|
||
- Create: `harness/tests/test_protected_fs_helper.py`
|
||
- Create: `backend/src/workspaces/annotations.ts`
|
||
- Create: `backend/test/workspace-annotations.test.ts`
|
||
- Modify: `backend/src/workspaces/preprocessing-state.ts` (extend the sole `WorkspaceLockedChildRequest` union and capability dispatcher; keep P2/P3 members and P2 state I/O unchanged)
|
||
- Create: `backend/test/workspace-locked-child-cumulative.compile.ts` (exact compile-time base-three + P3-fourteen + P5-two union fence)
|
||
- Modify: `backend/test/workspace-preprocessing-state.test.ts` (P3 locked-child focused cumulative runtime dispatcher regression)
|
||
- Modify: `backend/test/fixtures/workspace-lock-root-worker.mjs`
|
||
- Modify: `harness/tht/locked_child_stdin.py`
|
||
- Modify: `harness/tests/test_locked_child_stdin.py`
|
||
|
||
- [ ] **Step 1 (RED): specify `AnnotationSynchronizer.ensure`.**
|
||
|
||
Use real temporary directories plus an injected repository and validator. Add tests for:
|
||
|
||
- present and absent publication at the exact revision-qualified destination;
|
||
- validator is called before any destination becomes visible;
|
||
- byte-for-byte equality and manifest equality to the contract above;
|
||
- `0700` directories and `0400` files on POSIX;
|
||
- idempotent reuse only after no-follow type/mode/size/digest/manifest validation;
|
||
- conflict if an existing manifest differs, a destination/ancestor is a symlink, or either file is modified;
|
||
- validator failure, injected short write, rename failure, and process interruption leave no manifest publication and remove only owned staging files;
|
||
- two concurrent repository-owned transactions serialize through P2's real workspace writer lock and end with one valid immutable publication; `ensure` receives the already-held matching capability and never acquires or re-enters that lock itself;
|
||
- old and new commits publish separate roots and never mutate each other;
|
||
- a deterministic ancestor-replacement race at each boundary (after validating inherited FD 4, after opening `revisions/<commit>/artifacts/mschema` relative to it, after staging fsync, and immediately before each publication rename) either completes only in the originally retained directory or fails closed; it never writes into the replacement tree;
|
||
- unrelated valid workspace roots and Git namespaces remain byte-identical and are never inspected as candidate destinations;
|
||
- a compile-only cumulative-union fence defines the fourteen exact P3 kind literals and the two exact
|
||
P5 annotation kind literals as separate readonly tuples, rejects duplicates, and uses bidirectional
|
||
`Equal`/`Assert` checks to prove `WorkspaceLockedChildRequest["kind"]` is exactly the original three
|
||
P2 kinds plus those fourteen plus those two—neither a subset nor a widened string;
|
||
- a table-driven runtime regression creates one real `WorkspaceWriterLockCapability`, submits the
|
||
original three requests, all fourteen P3 requests, and both annotation requests through that same
|
||
capability, and asserts the sole exhaustive dispatcher selects each existing fixed argv/result path
|
||
exactly once. Its switch has an `assertNever` default, and the fixture records one capability/root
|
||
identity throughout; no parallel P3 or annotation spawner may make the test pass.
|
||
|
||
Freeze the compile-only fence around the discriminants (with local `Equal`, `Assert`, and recursive
|
||
`Unique` type helpers) so the member count cannot be satisfied by duplicate or widened entries:
|
||
|
||
```ts
|
||
const p3LockedKinds = [
|
||
"p3_migrate_dwh_cache", "p3_prepare_dwh_cache", "p3_materialize_dwh_snapshot",
|
||
"p3_migrate_memory_root", "p3_rebuild_memory_projection",
|
||
"p3_inventory_semantic_legacy", "p3_check_semantic_readiness",
|
||
"p3_publish_semantic_replacements", "p3_verify_semantic_replacements",
|
||
"p3_delete_confirmed_semantic_legacy", "p3_prepare_layout_markers",
|
||
"p3_publish_layout_version", "p3_publish_revision_ready",
|
||
"p3_verify_revision_readiness",
|
||
] as const satisfies readonly WorkspaceLockedChildRequest["kind"][];
|
||
const p2LockedKinds = ["dwh_preprocess", "schema_preprocess", "evidence_preprocess"] as const
|
||
satisfies readonly WorkspaceLockedChildRequest["kind"][];
|
||
const p5LockedKinds = ["annotation_publish", "annotation_verify"] as const
|
||
satisfies readonly WorkspaceLockedChildRequest["kind"][];
|
||
const p5CumulativeLockedKinds = [
|
||
...p2LockedKinds, ...p3LockedKinds, ...p5LockedKinds,
|
||
] as const;
|
||
type P5CumulativeLockedKind = typeof p5CumulativeLockedKinds[number];
|
||
type _P5CumulativeKindsAreUnique = Assert<Unique<typeof p5CumulativeLockedKinds>>;
|
||
type _P5CumulativeUnionIsExact = Assert<
|
||
Equal<WorkspaceLockedChildRequest["kind"], P5CumulativeLockedKind>
|
||
>;
|
||
```
|
||
|
||
The shared filesystem boundary is capability-only and cumulatively extends P3's single closed child
|
||
union in `backend/src/workspaces/preprocessing-state.ts`; it never accepts or derives an absolute workspace
|
||
path. `P3LockedChildRequest` remains the exact fourteen-member alias frozen by P3 and is not copied,
|
||
renamed, narrowed, or re-declared:
|
||
|
||
```ts
|
||
export const ANNOTATION_BYTES_MAX = 16_777_216;
|
||
export const ANNOTATION_MANIFEST_BYTES_MAX = 65_536;
|
||
export const ANNOTATION_CHILD_STDIN_MAX = 16_908_288;
|
||
export const PROTECTED_FS_CHILD_STDOUT_MAX = 16_384;
|
||
export const PROTECTED_FS_CHILD_STDERR_MAX = 65_536;
|
||
export const PROTECTED_FS_CHILD_RESULT_MAX = 81_920;
|
||
export interface AnnotationPublishLockedChildRequest {
|
||
readonly kind: "annotation_publish";
|
||
readonly workspaceId: CanonicalWorkspaceId;
|
||
readonly revision: Revision40;
|
||
readonly rootIdentity: WorkspaceLockRootIdentityV1;
|
||
readonly childRunId: string;
|
||
readonly annotationBytes: Uint8Array;
|
||
readonly annotationSha256: Sha256Hex;
|
||
readonly manifestBytes: Uint8Array;
|
||
readonly manifestSha256: Sha256Hex;
|
||
}
|
||
export interface AnnotationVerifyLockedChildRequest {
|
||
readonly kind: "annotation_verify";
|
||
readonly workspaceId: CanonicalWorkspaceId;
|
||
readonly revision: Revision40;
|
||
readonly rootIdentity: WorkspaceLockRootIdentityV1;
|
||
readonly childRunId: string;
|
||
readonly annotationSha256: Sha256Hex;
|
||
readonly manifestSha256: Sha256Hex;
|
||
}
|
||
export type WorkspaceLockedChildRequest =
|
||
| DwhLockedChildRequest | SchemaLockedChildRequest | EvidenceLockedChildRequest
|
||
| P3LockedChildRequest
|
||
| AnnotationPublishLockedChildRequest | AnnotationVerifyLockedChildRequest;
|
||
export interface ProtectedWorkspaceFs {
|
||
publishImmutablePair(input: ImmutablePairPublication): Promise<ImmutablePairIdentityV1>;
|
||
verifyImmutablePair(input: ImmutablePairVerification): Promise<ImmutablePairIdentityV1>;
|
||
}
|
||
export function openProtectedWorkspaceFs(input: {
|
||
readonly rootLease: BorrowedVerifiedWorkspaceLockRootLease;
|
||
readonly writerCapability: WorkspaceWriterLockCapability;
|
||
}): ProtectedWorkspaceFs;
|
||
```
|
||
|
||
`openProtectedWorkspaceFs` checks the opaque lease/capability workspace and root identities match and
|
||
retains neither beyond the caller's borrow. Each method calls only
|
||
`input.writerCapability.spawnChild()` with the matching exact request variant. There is no overload for
|
||
`workspaceRoot`, `dataRoot`, a root FD/number, raw argv, raw stdio, a spawn function, or ambient lookup.
|
||
The capability implementation owns the only spawn and fixes the executable plus argv to the selected
|
||
image's Python and exactly `-I -m tht.protected_fs_helper`. It canonical-frames the semantic request,
|
||
rejects an annotation or manifest over its individual cap and a complete stdin frame over 16,908,288
|
||
bytes before spawn, streams into fixed-capacity collectors, caps stdout at 16,384 bytes and stderr at
|
||
65,536 bytes (81,920 combined), enforces the fixed protected-helper deadline, terminates its owned
|
||
process group on the first overflow/cancellation/deadline, and awaits exit and both stdio pipelines.
|
||
The result parser accepts one allowlisted identity envelope only. The request union has no path, argv,
|
||
stdio, environment, executable, or callback field.
|
||
|
||
The child receives the actual locked writer open file description on FD 3 and the same retained root
|
||
directory open file description on FD 4. `protected_fs_helper.py` calls P2's shared
|
||
`require_workspace_writer_lock` before any workspace read or write: it `fstat`s FD 4 as the expected
|
||
root identity, opens `preprocessing/writer.lock` relative to FD 4 with `O_NOFOLLOW`, proves its
|
||
(device,inode) equals FD 3, validates FD 3 owner/mode/nlink/type and already-held flock, and rejects
|
||
missing, closed, substituted, independently locked, or cross-root descriptors. It derives the fixed
|
||
revision annotation leaves solely from the validated workspace/revision request and FD 4. It retains an
|
||
FD for every accepted component and uses only `openat`/`mkdirat`/`unlinkat` plus
|
||
`renameat2(RENAME_NOREPLACE)`, all relative to retained dirfds with no-follow semantics. It rechecks
|
||
root and ancestor identities before/after every read, write, fsync, and rename; there is no path reopen,
|
||
`lstat`-then-path fallback, or overwrite-capable rename fallback.
|
||
|
||
Publication writes/fsyncs annotation and manifest staging leaves, publishes annotation then the manifest
|
||
marker, fsyncs/revalidates the retained parent and ancestors, and removes only an operation-owned partial
|
||
whose retained inode/digest still matches. Verification performs the same FD-3/FD-4 authorization and
|
||
anchored reads and returns identities only. The helper is not a public `tht` command and direct Python
|
||
invocation without both inherited descriptors fails before workspace access.
|
||
|
||
Freeze real-process tests, not mock locks: pause the helper on an explicit barrier after the parent has
|
||
acquired the writer and installed child FDs but before the child validates/uses FD 4; replace the
|
||
canonical workspace pathname with a different directory/symlink, then release the barrier. Publication
|
||
may complete only in the retained original inode or fail, and the replacement tree must remain empty.
|
||
Also test missing FD 3, missing FD 4, substituted writer FD, substituted root FD, FD 3 from workspace A
|
||
with FD 4 from B, another independently locked writer, a forged marker, direct spawn, wrong requested
|
||
workspace/root identity, capability use after the ordered callback settles, and helper timeout/output
|
||
flood. Every case returns `preprocessing_conflict` or the bounded safe helper error with zero publication.
|
||
|
||
Desired single-synchronizer API (do not add a prepare/runtime synchronizer pair):
|
||
|
||
```ts
|
||
export type AnnotationMaterializationLifecycle =
|
||
| {
|
||
readonly kind: "prepare";
|
||
readonly rootLease: BorrowedVerifiedWorkspaceLockRootLease;
|
||
readonly writerCapability: WorkspaceWriterLockCapability;
|
||
}
|
||
| {
|
||
readonly kind: "runtime";
|
||
readonly rootLease: BorrowedVerifiedWorkspaceLockRootLease;
|
||
readonly descriptorBlob: Revision40;
|
||
readonly effectiveDwhCacheKey: Sha256Hex;
|
||
};
|
||
|
||
export class AnnotationSynchronizer {
|
||
ensure(
|
||
workspaceId: CanonicalWorkspaceId,
|
||
commit: Revision40,
|
||
lifecycle: Extract<AnnotationMaterializationLifecycle, { kind: "prepare" }>,
|
||
): Promise<AnnotationMaterialization>;
|
||
verifyPrepared(
|
||
workspaceId: CanonicalWorkspaceId,
|
||
commit: Revision40,
|
||
lifecycle: Extract<AnnotationMaterializationLifecycle, { kind: "prepare" }>,
|
||
): Promise<AnnotationMaterialization>;
|
||
readRuntime(
|
||
workspaceId: CanonicalWorkspaceId,
|
||
commit: Revision40,
|
||
lifecycle: Extract<AnnotationMaterializationLifecycle, { kind: "runtime" }>,
|
||
): Promise<AnnotationMaterialization>;
|
||
}
|
||
```
|
||
|
||
The exact P2 participant first narrows `RegistryAddressedPlanV1` to `operation === "registry_pull"`.
|
||
For a target it calls `ensure(target.workspaceId, target.revision,
|
||
{ kind: "prepare", rootLease: workspace.rootLease, writerCapability:
|
||
workspace.writerCapability })`; for a base-only removal it returns the closed removal variant without
|
||
publication. `reconcile` uses `verifyPrepared` with those same borrowed capabilities. Thus the
|
||
participant consumes both objects supplied by `AddressedWorkspacePublicationLeaseV1` and cannot invent
|
||
a root path or acquire another lock. The prepare resolver calls P3
|
||
`futureWorkspaceLayoutPaths(rootLease, workspaceId, revision)` only to verify the fixed semantic
|
||
layout; the helper itself receives no path and derives/opens beneath inherited FD 4.
|
||
|
||
Pre-READY acceptance must be executed inside the existing P2 writer-capability lifetime and uses the
|
||
same `annotation_verify` child variant; it never directly spawns the helper. Runtime/session reads do
|
||
not call the mutating helper: while their existing verified root borrow and reader lease are alive they
|
||
use P3's root-relative safe reader after
|
||
`readRevisionLayoutState(rootLease, expected)` and
|
||
`workspaceRuntimePaths(rootLease, workspaceId, revision, state)`. This preserves runtime read-only
|
||
capability while ensuring every `protected_fs_helper` invocation, including verification, is authorized
|
||
by FD 3 and FD 4.
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/pytest -q tests/test_protected_fs_helper.py tests/test_locked_child_stdin.py \
|
||
tests/test_layout_marker_commands.py tests/test_p3_internal_cli.py
|
||
cd ../backend
|
||
npx vitest run test/protected-workspace-fs.test.ts test/workspace-annotations.test.ts \
|
||
test/workspace-preprocessing-state.test.ts
|
||
npx tsc --noEmit --target ES2022 --module ES2022 --moduleResolution Bundler \
|
||
--strict --skipLibCheck test/workspace-locked-child-cumulative.compile.ts
|
||
```
|
||
|
||
Expected: the existing P3 locked-child focused cases remain green, while the cumulative dispatcher,
|
||
compile-only union fence, helper, shared wrapper, and `annotations.ts` cases fail for only the missing
|
||
P5 surface.
|
||
|
||
- [ ] **Step 3 (GREEN): implement safe publication.**
|
||
|
||
Validate the branded workspace ID, revision, opaque borrowed-root identity, and matching writer capability. Construct `openProtectedWorkspaceFs({ rootLease, writerCapability })` only inside that borrow, then call `publishImmutablePair` or `verifyImmutablePair`; both dispatch the exact union variant through `writerCapability.spawnChild`. The child derives the fixed `revisions/<commit>/artifacts/mschema` components from semantic fields and inherited FD 4, creates unique same-directory staging leaves, writes/fsyncs/rechecks both, publishes annotation first and manifest last, then fsyncs and revalidates the retained parent and every ancestor. The manifest is the publication marker. Do not overwrite an existing immutable publication—verify it or fail closed. Map parser/model failures to `annotation_invalid`; descriptor/capability mismatch, ancestor replacement, helper/syscall uncertainty, or filesystem corruption remains a safe workspace error. Tests must use the real pre-FD-validation process barrier, replace the canonical root after writer acquisition, and prove the replacement tree receives no file.
|
||
|
||
Production validation must spawn the installed `tht` directly with fixed argv:
|
||
|
||
```text
|
||
tht schema validate-annotations --stdin --json
|
||
```
|
||
|
||
Use no shell, pass exact bytes on stdin, bound stdout/stderr, enforce a short timeout, parse the exact success envelope, and discard raw errors. Put this adapter in `annotations.ts` as `createHarnessAnnotationValidator(...)` so backend and maintenance use one implementation.
|
||
|
||
- [ ] **Step 4 (verify GREEN):**
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/pytest -q tests/test_protected_fs_helper.py tests/test_locked_child_stdin.py \
|
||
tests/test_layout_marker_commands.py tests/test_p3_internal_cli.py
|
||
.venv/bin/ruff check tht/protected_fs_helper.py tht/locked_child_stdin.py \
|
||
tests/test_protected_fs_helper.py tests/test_locked_child_stdin.py \
|
||
tests/test_layout_marker_commands.py tests/test_p3_internal_cli.py
|
||
cd ../backend
|
||
npx vitest run test/protected-workspace-fs.test.ts test/workspace-annotations.test.ts \
|
||
test/workspace-preprocessing-state.test.ts
|
||
npx tsc --noEmit --target ES2022 --module ES2022 --moduleResolution Bundler \
|
||
--strict --skipLibCheck test/workspace-locked-child-cumulative.compile.ts
|
||
npx tsc --noEmit -p .
|
||
```
|
||
|
||
Expected: PASS. The compile fence proves the exact base-three + fourteen-P3 + two-annotation kind
|
||
union, and the runtime table proves all nineteen requests still traverse one exhaustive
|
||
`WorkspaceWriterLockCapability.spawnChild` dispatcher.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add harness/tht/protected_fs_helper.py harness/tests/test_protected_fs_helper.py \
|
||
backend/src/workspaces/protected-workspace-fs.ts \
|
||
backend/test/protected-workspace-fs.test.ts \
|
||
backend/src/workspaces/annotations.ts backend/test/workspace-annotations.test.ts \
|
||
backend/src/workspaces/preprocessing-state.ts \
|
||
backend/test/workspace-locked-child-cumulative.compile.ts \
|
||
backend/test/workspace-preprocessing-state.test.ts \
|
||
backend/test/fixtures/workspace-lock-root-worker.mjs \
|
||
harness/tht/locked_child_stdin.py harness/tests/test_locked_child_stdin.py
|
||
git commit -m "feat: materialize immutable workspace annotations"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: Gate activation, active sessions, pinned sessions, and runtime rendering on synchronization
|
||
|
||
**Files:**
|
||
- Modify: `backend/src/workspaces/registry.ts`
|
||
- Create: `backend/src/workspaces/registry-factory.ts`
|
||
- Modify: `backend/src/app.ts`
|
||
- Modify: `backend/src/workspace-maintenance.ts`
|
||
- Modify: `backend/src/workspaces/runtime-config-lease.ts`
|
||
- Read/consume unchanged: `backend/src/workspaces/revision-layout.ts`
|
||
- Modify: `backend/test/workspace-registry.test.ts`
|
||
- Create: `backend/test/workspace-registry-factory.test.ts`
|
||
- Create: `backend/test/p5-registry-pull-imports.compile.ts` (imports all six names only from `../src/workspaces/registry-pull-job.js`)
|
||
- Modify: `backend/test/workspace-runtime-handoff.test.ts`
|
||
- Modify: `backend/test/workspace-runtime-config-lease.test.ts`
|
||
- Modify: `backend/test/workspace-revision-layout.test.ts`
|
||
- Modify: `backend/test/workspace-runtime-renderer.test.ts`
|
||
- Modify: `backend/test/routes-sessions.test.ts`
|
||
|
||
Freeze `backend/test/p5-registry-pull-imports.compile.ts` as a real field-parity fence, not merely an
|
||
import smoke test. It imports the six released P3 names only from
|
||
`../src/workspaces/registry-pull-job.js`, imports their owning P2 types directly—including
|
||
`AddressedWorkspacePublicationLeaseV1` from `registry-publication.js` and
|
||
`BorrowedWorkspaceSessionReadersExclusiveLockLease` from `preprocessing-state.js`—and contains these
|
||
exact bidirectional assertions (plus an `AllRegistryPullExports` tuple referencing the other three
|
||
released names):
|
||
|
||
```ts
|
||
import type {
|
||
RegistryPullPublicJobRequestV1,
|
||
RegistryPullPhaseV1,
|
||
RegistryPullAddressedJobRequestV1,
|
||
RegistryPullParticipantStateV1,
|
||
RegistryPullSynchronizerStateV1,
|
||
RegistryPullJobStateV1,
|
||
} from "../src/workspaces/registry-pull-job.js";
|
||
import type {
|
||
AddressedWorkspacePublicationLeaseV1,
|
||
RegistryAddressedPublicationPhaseV1,
|
||
RegistryAddressedRequestV1,
|
||
RegistryPullAddressedPlanV1,
|
||
RegistryPullAddressedPublicationStateV1,
|
||
RegistryRunId32,
|
||
} from "../src/workspaces/registry-publication.js";
|
||
import type {
|
||
BorrowedWorkspaceSessionReadersExclusiveLockLease,
|
||
} from "../src/workspaces/preprocessing-state.js";
|
||
import type { WorkspaceRegistry } from "../src/workspaces/registry.js";
|
||
import type {
|
||
Revision40,
|
||
Sha256Hex,
|
||
} from "../src/workspaces/workspace-lock-root-lease.js";
|
||
|
||
type Equal<A, B> =
|
||
(<T>() => T extends A ? 1 : 2) extends (<T>() => T extends B ? 1 : 2)
|
||
? (<T>() => T extends B ? 1 : 2) extends (<T>() => T extends A ? 1 : 2)
|
||
? true : false
|
||
: false;
|
||
type Assert<T extends true> = T;
|
||
type RequiredKey<T, K extends keyof T> = {} extends Pick<T, K> ? false : true;
|
||
type ExpectedPullAddressedRequestV1 =
|
||
| {
|
||
readonly mode: "create";
|
||
readonly operation: "registry_pull";
|
||
readonly runId: RegistryRunId32;
|
||
readonly requestSha256: Sha256Hex;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly expectedBaseCommit: Revision40;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
}
|
||
| {
|
||
readonly mode: "resume";
|
||
readonly operation: "registry_pull";
|
||
readonly runId: RegistryRunId32;
|
||
readonly requestSha256: Sha256Hex;
|
||
readonly installationIdentitySha256: Sha256Hex;
|
||
readonly repositoryIdentitySha256: Sha256Hex;
|
||
readonly remoteRefIdentitySha256: Sha256Hex;
|
||
};
|
||
type PullCreate = Extract<RegistryPullAddressedJobRequestV1, { readonly mode: "create" }>;
|
||
type PullResume = Extract<RegistryPullAddressedJobRequestV1, { readonly mode: "resume" }>;
|
||
type AllRegistryPullExports = readonly [
|
||
RegistryPullPublicJobRequestV1,
|
||
RegistryPullPhaseV1,
|
||
RegistryPullAddressedJobRequestV1,
|
||
RegistryPullParticipantStateV1,
|
||
RegistryPullSynchronizerStateV1,
|
||
RegistryPullJobStateV1,
|
||
];
|
||
type ExactP2Parity = readonly [
|
||
Assert<Equal<
|
||
AddressedWorkspacePublicationLeaseV1["readers"],
|
||
BorrowedWorkspaceSessionReadersExclusiveLockLease
|
||
>>,
|
||
Assert<Equal<RegistryPullPhaseV1, RegistryAddressedPublicationPhaseV1>>,
|
||
Assert<Equal<
|
||
RegistryPullAddressedJobRequestV1,
|
||
Extract<RegistryAddressedRequestV1, { readonly operation: "registry_pull" }>
|
||
>>,
|
||
Assert<Equal<RegistryPullAddressedJobRequestV1, ExpectedPullAddressedRequestV1>>,
|
||
Assert<Equal<RegistryPullJobStateV1, RegistryPullAddressedPublicationStateV1>>,
|
||
Assert<Equal<Parameters<WorkspaceRegistry["publishAddressed"]>[0], RegistryAddressedRequestV1>>,
|
||
Assert<RequiredKey<PullCreate, "installationIdentitySha256">>,
|
||
Assert<RequiredKey<PullCreate, "repositoryIdentitySha256">>,
|
||
Assert<RequiredKey<PullCreate, "remoteRefIdentitySha256">>,
|
||
Assert<RequiredKey<PullCreate, "expectedBaseCommit">>,
|
||
Assert<RequiredKey<PullResume, "installationIdentitySha256">>,
|
||
Assert<RequiredKey<PullResume, "repositoryIdentitySha256">>,
|
||
Assert<RequiredKey<PullResume, "remoteRefIdentitySha256">>,
|
||
Assert<RequiredKey<RegistryPullAddressedPlanV1, "installationIdentitySha256">>,
|
||
Assert<RequiredKey<RegistryPullAddressedPlanV1, "repositoryIdentitySha256">>,
|
||
Assert<RequiredKey<RegistryPullAddressedPlanV1, "remoteRefIdentitySha256">>,
|
||
Assert<RequiredKey<RegistryPullJobStateV1, "installationIdentitySha256">>,
|
||
Assert<RequiredKey<RegistryPullJobStateV1, "repositoryIdentitySha256">>,
|
||
Assert<RequiredKey<RegistryPullJobStateV1, "remoteRefIdentitySha256">>,
|
||
];
|
||
export type { AllRegistryPullExports, ExactP2Parity };
|
||
```
|
||
|
||
Also add source-boundary assertions that `WorkspaceRegistry` has exactly one `publishAddressed`
|
||
declaration and it takes exactly the required `RegistryAddressedRequestV1`; an omitted identity on
|
||
create or resume is covered by `@ts-expect-error`, and adding an optional/partial compatibility overload
|
||
must make the boundary test fail.
|
||
|
||
- [ ] **Step 1 (RED): add registry/session/runtime tests.**
|
||
|
||
Register exactly one object returned by `createAnnotationRegistryPublicationParticipant(...)` in the `WorkspaceRegistry` constructor-owned participant list; its public type is P2 `CapabilityAwareRegistryPublicationParticipant<AnnotationRegistryPreparationV1>`, not a P5 hook alias. Assert `WorkspaceRegistry.publishAddressed`, not the participant or `AnnotationSynchronizer`, owns the released lifecycle: inside the validated installation/registry boundary it derives the exact production installation, repository, and remote-ref identities, passes all three required fields on both pull create and pull resume, passes the exact expected active base on pull create, takes real `repository.lock`, fetches once, pins every exact `RegistryPullAddressedPlanV1` field/digest including `installationIdentitySha256`, persists all three identities in `RegistryPullJobStateV1`, computes complete strict-lexical `changedWorkspaceIds` including target additions and base removals, and calls the participant with each exact `AddressedWorkspacePublicationLeaseV1`. For target records, `prepare` calls `ensure(id, exact target revision, { kind: "prepare", rootLease: workspace.rootLease, writerCapability: workspace.writerCapability })`; for base-only removals it records the closed removal result without file publication. Every target annotation prepares before `publication_intent_durable`; failure preserves exact base. This path succeeds for a deliberately unready target and derives its immutable annotation destination with exact `futureWorkspaceLayoutPaths(rootLease, workspaceId, workspaceRevision)` without READY. Assert `acquireSessionRevision()` and `readPinned()` instead call the same synchronizer with the `runtime` lifecycle and exact descriptor blob/effective-DWH cache key before returning. A content-only annotation commit has the same descriptor blob but a new workspace commit and distinct runtime annotation path. A resumed historical session still renders the old commit's bytes after a newer pull. Replaced manifest/file refuses both new and resumed sessions.
|
||
|
||
Use real independent `flock`-contending processes, the real `addressed-publication-jobs/<runId>.json`, and exact P2/P3 types—never a mock mutex, anonymous job shape, or timeout-only trace. Present at least two changed workspace identities in reverse input order, including an addition/removal, and assert the persisted `RegistryPullJobStateV1.changedWorkspaceIds` is the unique complete strict-lexical symmetric difference and `changedSetSha256`/`planSha256` match canonical bytes. Barriers must prove `repository.lock acquire → request_claimed → target_advertised → exact-OID immutable-ref fetch → target_fetched → RegistryPullJobStateV1 phase planned → one runUnderOrderedWorkspaceWriterLocks callback → lexical root/writer for the complete set → lexical quiescence/readers for the complete set → CapabilityAwareRegistryPublicationParticipant.prepare with each matching AddressedWorkspacePublicationLeaseV1 → participants_prepared → publication_intent_durable → active-pointer sibling write+file fsync+atomic rename+parent fsync+target-byte verification → target_published → terminal_durable → ordered callback settlement and reverse close → repository.lock release`. The complete `OrderedWorkspaceWriterCapabilitySet` must still contend at every changed ID until after publication rename+parent fsync; a competing repository operation serializes, and a process holding the later lexical writer makes pull wait while repository remains held. AST/capability fakes fail if participant `prepare`/`reconcile` can call repository, root/reader acquisition, `runUnderWorkspaceWriterLock`, or `publishAddressed`; counters prove each live create/resume attempt acquires each ID once, uses the same opaque capability in participant calls, and has no nested/duplicate reentry.
|
||
|
||
Run the exact same-ID SIGKILL dependency matrix with `core` absent and a fetch/remote-resolution/publication counter. Case 1 kills the real one-shot at persisted phase `publication_intent_durable` immediately before the active-pointer rename: the active pointer and projections remain exact base; resuming the same 32-hex `runId` and identical `requestSha256` loads the recorded `RegistryPullJobStateV1`, performs no fetch/ls-remote/remote resolution/target selection, reacquires the recorded complete lexical set once in the new process, and converges to target. Case 2 kills immediately after the active-pointer rename and parent fsync but before the `target_published` state rewrite: exact target is active while durable job phase is still `publication_intent_durable`; same-ID resume accepts all-target as lost acknowledgement, performs no fetch and no second active publication rename, advances through `target_published`/`terminal_durable`, owner-clears, reverse-releases the reacquired set, then releases repository. Assert one fetch total and one publication rename total in each case. Separately inject a third active revision/descriptor/manifest identity, changed/deleted pinned target object, target manifest/inventory/digest drift, installation identity drift, repository identity drift, remote-ref identity drift, moved local remote-tracking OID, different ID, same ID with different digest, and cross-operation resume; each returns `preprocessing_resume_mismatch` or the released closed conflict code before network or state mutation and without refetch, participant entry, or owner clear. A crash releases OS locks but never silently clears nonterminal ownership.
|
||
|
||
Freeze the lifecycle transition test explicitly: pull revision B and prove its prepared annotation manifest verifies at the future revision path while B has no READY; session admission and preprocessing resume both refuse B with `migration_required`. Run the exact released `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json` transition and prove its `p3_migrate_dwh_cache`/`p3_prepare_dwh_cache` plus `p3_materialize_dwh_snapshot` stages publish and strictly reverify B's binding-qualified physical/LSH snapshot under the future B/binding root without reading or publishing READY or falling back to A; admission and resume must still refuse `migration_required`. Only then may `schema accept` with the prepare lifecycle succeed; assert admission/resume remain `migration_required` after accept. Run the exact P3 `workspace migrate activate-revision-layout --workspace <id> --yes --json` operation and strictly re-read the READY for B plus the probed binding; only then do session admission, pinned/runtime annotation consumption, and accepted preprocessing resume succeed. A READY or snapshot for another revision or binding must not satisfy this test.
|
||
|
||
Update renderer tests to consume P3's exact roots:
|
||
|
||
```ts
|
||
const state = readRevisionLayoutState(rootLease, {
|
||
workspaceId,
|
||
workspaceRevision,
|
||
descriptorBlob,
|
||
effectiveDwhCacheKey,
|
||
}); // exact synchronous P3 no-follow state read
|
||
const runtimePaths = workspaceRuntimePaths(
|
||
rootLease,
|
||
workspaceId,
|
||
workspaceRevision,
|
||
state,
|
||
);
|
||
// artifacts/indexes/corpus under .../revisions/<commit>/
|
||
// memory workspace-global; dwh_cache is <ws>/preprocessing/dwh-cache
|
||
// acquireSession and ordinary committed pinned/runtime acquisition use this helper
|
||
// pre-READY acceptance maintenance instead uses futureWorkspaceLayoutPaths and never this state
|
||
// harness/tht/effective_dwh.py::effective_dwh_cache_root(cfg) appends the v2 binding digest
|
||
```
|
||
|
||
Assert `paths.artifacts + /mschema/annotations.yaml` is the verified synchronized destination and that there is no workspace-global annotations path or compatibility symlink.
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run \
|
||
test/workspace-registry.test.ts \
|
||
test/workspace-registry-factory.test.ts \
|
||
test/workspace-runtime-handoff.test.ts \
|
||
test/workspace-runtime-config-lease.test.ts \
|
||
test/workspace-revision-layout.test.ts \
|
||
test/workspace-runtime-renderer.test.ts \
|
||
test/routes-sessions.test.ts -t "annotation|pinned|revision"
|
||
```
|
||
|
||
Expected: FAIL because registry/runtime paths do not yet require annotation synchronization.
|
||
|
||
- [ ] **Step 3 (GREEN): wire one shared synchronizer.**
|
||
|
||
Create `createWorkspaceRegistry(...)` in `registry-factory.ts`; it constructs one production `AnnotationSynchronizer` from the registry repository and the fixed selected-image annotation validator, then creates exactly one `CapabilityAwareRegistryPublicationParticipant<AnnotationRegistryPreparationV1>` with `createAnnotationRegistryPublicationParticipant`. Both `buildApp()` and `backend/src/workspace-maintenance.ts::main` call this factory and pass that exact participant to the `WorkspaceRegistry` constructor-owned participant list; do not create a second synchronizer, participant interface, lease-set alias, lock file, or publication path. The pull path delegates once to `WorkspaceRegistry.publishAddressed`. P2 `WorkspaceRegistry.publishAddressed` alone acquires the complete set and passes it to the `CapabilityAwareRegistryPublicationLifecycleOwner`, which owns it through active-pointer write/file-fsync/rename/parent-fsync/verification and `terminal_durable`, then lets the ordered callback settle and reverse-close before repository release. `AnnotationSynchronizer.verifyPrepared` takes no lock itself and uses only the supplied writer capability; participant `prepare`/`reconcile` use only the supplied `AddressedWorkspacePublicationLeaseV1` and never re-enter repository or a writer acquisition.
|
||
|
||
Implement the synchronizer's lifecycle split exactly:
|
||
|
||
- `prepare`: validate the borrowed root identity, workspace ID, and 40-hex revision, then call only P3's exact `futureWorkspaceLayoutPaths(rootLease, workspaceId, workspaceRevision)`. Repository-owned pull/activation synchronization uses `ensure(..., { kind: "prepare", rootLease })`; pre-READY acceptance uses capability-only `verifyPrepared(..., { kind: "prepare", rootLease, writerCapability })`. Neither path calls `readRevisionLayoutState`, requires READY, accepts a caller path, or falls back to currently active/P2 roots. It publishes or validates the manifest at `futurePaths.artifacts/mschema` before `active.json` changes.
|
||
- `runtime`: session and pinned runtime consumption synchronously call the exact P3 state reader `readRevisionLayoutState(rootLease, { workspaceId, workspaceRevision, descriptorBlob, effectiveDwhCacheKey })`, require the returned strict committed `RevisionLayoutState` to be `revision-layout-v1` with the exact non-null READY for that commit+binding, and only then call the exact four-argument P3 export `workspaceRuntimePaths(rootLease, workspaceId, workspaceRevision, state)`. It must not omit the borrowed root lease, pass a layout string in place of `state`, duplicate joins, use `futureWorkspaceLayoutPaths`, or fall back to `p2-global`/another READY.
|
||
|
||
Make `WorkspaceRuntimeConfigLeaseFactory.acquireSession(snapshotPath)` and normal post-READY pinned/resume runtime paths use the runtime lifecycle before `renderRuntimeConfig`; the prepare-mode acceptance maintenance path uses only the prepare lifecycle and cannot spawn a session. Render only P3's DWH-cache base. `harness/tht/effective_dwh.py::effective_dwh_cache_root(cfg)` appends the v2 binding digest returned by `canonical_effective_dwh_binding(cfg)`; do not compute or reimplement that digest in TypeScript.
|
||
|
||
A missing/relative data root is `workspace_not_activatable`; there is no annotation-unaware production constructor path. Keep `revisionContentRoot` reserved for P6 Evidence; annotation consumption is through revision-qualified `RuntimePaths.artifacts`.
|
||
|
||
- [ ] **Step 4 (GREEN): run focused and complete backend gates.**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run \
|
||
test/workspace-registry.test.ts \
|
||
test/workspace-registry-factory.test.ts \
|
||
test/workspace-runtime-handoff.test.ts \
|
||
test/workspace-runtime-config-lease.test.ts \
|
||
test/workspace-revision-layout.test.ts \
|
||
test/workspace-runtime-renderer.test.ts \
|
||
test/routes-sessions.test.ts
|
||
npx tsc --noEmit -p .
|
||
npx tsc --noEmit --target ES2022 --module ES2022 --moduleResolution Bundler \
|
||
--strict --skipLibCheck test/registry-pull-job-imports.compile.ts \
|
||
test/p5-registry-pull-imports.compile.ts
|
||
npm run build
|
||
```
|
||
|
||
Expected: PASS; `dist/workspaces/annotations.js` exists.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add backend/src/workspaces/registry.ts backend/src/workspaces/registry-factory.ts \
|
||
backend/src/app.ts backend/src/workspace-maintenance.ts \
|
||
backend/src/workspaces/runtime-config-lease.ts \
|
||
backend/test/workspace-registry.test.ts backend/test/workspace-registry-factory.test.ts \
|
||
backend/test/p5-registry-pull-imports.compile.ts \
|
||
backend/test/workspace-runtime-handoff.test.ts \
|
||
backend/test/workspace-runtime-config-lease.test.ts \
|
||
backend/test/workspace-revision-layout.test.ts \
|
||
backend/test/workspace-runtime-renderer.test.ts backend/test/routes-sessions.test.ts
|
||
git commit -m "feat: bind session annotations to workspace revisions"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: Upgrade P2 run state to a digest-bound explicit revision transition
|
||
|
||
**Files:**
|
||
- Create from the unmodified P2 writer: `backend/test/fixtures/preprocessing-v1-fk-review/README.md`
|
||
- Create from the unmodified P2 writer: `backend/test/fixtures/preprocessing-v1-fk-review/{paused-unreviewed,paused-reviewed}/{job.json,candidate.yaml}`
|
||
- Create from the unmodified P2 writer: `backend/test/fixtures/preprocessing-v1-fk-review/paused-reviewed/review.json`
|
||
- Modify: `backend/src/workspaces/preprocessing-state.ts`
|
||
- Modify: `backend/test/workspace-preprocessing-state.test.ts`
|
||
|
||
- [ ] **Step 1 (characterization, before RED): capture and verify exact P2 V1 bytes.**
|
||
|
||
Use the final P2 test writer/API to produce both fixture directories; do not hand-author JSON. Record the generation command and SHA-256 of every file. Run the existing P2 reader tests plus new characterization assertions that the exact fixtures load as their documented paused states. Expected: PASS before any production edit. If not, stop and amend the P2 checkpoint.
|
||
|
||
- [ ] **Step 2 (RED): add state-store migration/transition tests.**
|
||
|
||
Create byte fixtures for every row of the frozen V1→V2 table above and a table-driven transition test. Tests must cover:
|
||
|
||
- a strict P2 V1 `manual_review_required` job records the existing `fk-candidates/<run-id>.yaml`, count/`candidateDigest`, and base revision under the state-store-owned `preprocessing/fk-candidates/` directory;
|
||
- candidate and optional V1 review are restrictive regular non-symlinks, match embedded workspace/run/revision identity, are rehashed on every migration/export/accept, accept exactly 716,800 decoded candidate bytes, and reject 716,801 before state transition or export;
|
||
- V1 paused-without-review maps only to V2 `pending`; V1 paused-with-valid-local-review also maps to `pending` with `reviewedDigest` audit-only; neither is accepted;
|
||
- progressed/terminal V1 remains immutable legacy audit state and cannot be exported, accepted, or resumed; malformed/conflicting/unknown V1 fails closed;
|
||
- `acceptAnnotationRevision(runId, acceptance)` is allowed once only from V2 `pending`, under the workspace writer lock;
|
||
- acceptance requires candidate digest equality and every canonical accepted field shown above; partial accepted fields are invalid;
|
||
- exact `accepted → accepted` replay is idempotent only after revalidation; a different commit/blob/digest/binding fails; every skip, rewind, `required`, or `reviewed` status is rejected;
|
||
- the migration cannot manufacture `baseGitBlob`: it requires a caller-supplied, already synchronized base identity and rejects a revision/content mismatch;
|
||
- a resume validator accepts exactly the recorded accepted revision/blob/content/binding and rejects each single-field mutation with `preprocessing_resume_mismatch`.
|
||
|
||
- [ ] **Step 3 (verify RED):**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run test/workspace-preprocessing-state.test.ts -t "annotation|candidate|accept"
|
||
```
|
||
|
||
Expected: FAIL because P2 state has no acceptance record or controlled cross-revision transition.
|
||
|
||
- [ ] **Step 4 (GREEN): implement atomic state/candidate ownership.**
|
||
|
||
Use the existing `PreprocessingStateStore` closed transitions, the shared `ProtectedWorkspaceFs`, state directory, and writer lock. Extend the existing candidate/review path and add only `migrateFkReviewCheckpoint`, `acceptAnnotationRevision`, and `assertAcceptedAnnotation`; do not invent caller-supplied paths. Implement exactly the frozen table and two-state V2 union—do not retain `required`/`reviewed` aliases or infer acceptance from P2 local review. Never accept a candidate path from CLI JSON. Reuse P2's exact `MAX_FK_CANDIDATE_BYTES = 716_800` decoded-byte limit for state-owned suggestions, V1 fixtures, migration, reread, and export; never substitute the 16 MiB curated-blob limit. Write candidates exclusively/no-follow, store digest before publishing state, and delete an unreferenced staging candidate on failure. Keep timestamps audit-only; no timestamp participates in identity.
|
||
|
||
- [ ] **Step 5 (verify GREEN):**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run test/workspace-preprocessing-state.test.ts
|
||
npx tsc --noEmit -p .
|
||
```
|
||
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 6: commit.**
|
||
|
||
```bash
|
||
git add backend/test/fixtures/preprocessing-v1-fk-review \
|
||
backend/src/workspaces/preprocessing-state.ts \
|
||
backend/test/workspace-preprocessing-state.test.ts \
|
||
backend/test/fixtures/workspace-lock-root-worker.mjs \
|
||
harness/tht/locked_child_stdin.py harness/tests/test_locked_child_stdin.py
|
||
git commit -m "feat: record annotation review acceptance"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 6: Add `schema accept` and enforce the accepted blob on continuation
|
||
|
||
**Files:**
|
||
- Modify: `backend/src/workspaces/preprocessing-state.ts`
|
||
- Modify: `backend/src/workspaces/preprocessing-service.ts`
|
||
- Modify: `backend/src/workspace-maintenance.ts`
|
||
- Modify: `backend/test/workspace-preprocessing-state.test.ts`
|
||
- Modify: `backend/test/workspace-preprocessing-service.test.ts`
|
||
- Modify: `backend/test/workspace-maintenance.test.ts`
|
||
- Modify: `backend/test/workspace-registry.test.ts` (strict active-ID enumeration and lock-order barriers)
|
||
|
||
- [ ] **Step 1 (RED): write run-resolution, operator, and machine-envelope tests.**
|
||
|
||
Add two closed request discriminators:
|
||
|
||
```ts
|
||
{ operation: "schema-export-fks"; runId: string }
|
||
{ operation: "schema-accept"; runId: string; yes: true }
|
||
```
|
||
|
||
`schema-export-fks` uses `resolveRun`, requires the run to be the paused P2 `manual_review_required` / V2 `pending` checkpoint, opens the exact state-owned `fk-candidates/<run>.yaml` through `PreprocessingStateStore`, rejects more than exactly 716,800 decoded bytes, rehashes it against `candidateDigest`, and returns only P2's bounded internal `hostExport` plus run/workspace/digest/count. Base64 plus the complete child envelope must remain within P2's exact 1 MiB machine-output maximum. It never invokes FK suggestion, DWH, Git, or the annotation synchronizer, and it never exports a progressed, terminal, ambiguous, removed-workspace, or accepted run.
|
||
|
||
First freeze and test the bounded locator:
|
||
|
||
```ts
|
||
export interface ResolvedPreprocessingRun {
|
||
workspaceId: string;
|
||
rootIdentity: WorkspaceLockRootIdentityV1;
|
||
runId: string;
|
||
}
|
||
export function resolveRun(input: {
|
||
dataRoot: string;
|
||
activeWorkspaceIds: readonly string[];
|
||
runId: string;
|
||
}): Promise<ResolvedPreprocessingRun>;
|
||
```
|
||
|
||
Tests use the real protected filesystem and cover zero and one match, duplicate IDs in two active workspaces, traversal/shell-like/non-32-hex input, removed/inactive workspace, a job symlink, a job hardlink, swapped ancestor, oversized/unknown JSON, and embedded workspace/run mismatch. Assert there is no recursive directory walk, every candidate path is the one exact job leaf under a validated active ID, total work is capped by the existing registry workspace limit and P2 state-size limit, and all public errors are the three frozen safe codes. After resolution, replace the file before writer acquisition and prove the mandatory locked reread refuses it.
|
||
|
||
Then prove this exact `schema-accept` transaction, assuming the curator already completed the separate P3 pull followed by exact `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json`, and the latter strictly reverified the selected unready revision's binding-qualified physical/LSH snapshot without READY:
|
||
|
||
1. validate `runId`/`yes` and obtain the strict bounded active workspace ID set without taking repository or writer lock;
|
||
2. call `resolveRun`, open/rehash the paused V1/V2 state and exact state-owned candidate (at most 716,800 bytes) only to establish the candidate workspace, then close all handles;
|
||
3. call only P3's existing read-only `WorkspaceRegistry.read(workspaceId)` active immutable-snapshot API, then read the prepared annotation manifest identity and the strict P3-selected physical/LSH snapshot identity at that exact future revision+binding path; refuse a missing/mismatched snapshot, prior-revision fallback, or READY-dependent resolver; do not call pull, migration, a repository-owned method/lock, Git, or `AnnotationSynchronizer.ensure()`;
|
||
4. acquire that workspace's P2 writer lock;
|
||
5. reread `active.json` and require the exact same active revision/snapshot, then reopen/revalidate the resolved paused state and candidate; call only repository-free `AnnotationSynchronizer.verifyPrepared(..., { kind: "prepare", rootLease, writerCapability })` for that exact already-materialized active identity, using `futureWorkspaceLayoutPaths` and no READY/runtime-state requirement;
|
||
6. reject changed active identity, unchanged base revision, absent blob, invalid materialization, wrong workspace, removed workspace, or already advanced run;
|
||
7. compare the P3 harness-owned effective-DWH binding to the paused binding;
|
||
8. run harness `schema check --json` against the current revision paths and exact binding-qualified physical/LSH snapshot that the preceding P3 DWH migration published and strictly reverified without READY;
|
||
9. atomically record candidate/current blob identities only after all checks pass;
|
||
10. return a safe envelope with run/workspace/base/current revisions, candidate/current `sha256:<64hex>` digests, blob ID, `matches_candidate`, and status `accepted`.
|
||
|
||
Add deterministic lock-seam tests for both directions. The P3 `registry_pull` path must record `repository.lock acquire → request_claimed → target_advertised → exact-OID fetch → target_fetched → planned → one runUnderOrderedWorkspaceWriterLocks callback in strict lexical order → participant prepare/reconcile with the matching AddressedWorkspacePublicationLeaseV1 objects → active-pointer sibling write + file fsync + atomic rename + parent fsync + target-byte verification while every capability remains held → target_published → terminal_durable → ordered callback settlement and reverse close → repository.lock release`; fail if any writer/root/quiescence/reader lease is released before the active rename+parent fsync or if repository releases before the reverse set release. A normal `schema-accept` records `active snapshot read → one-element ordered writer callback → rootLease/writerCapability verification → active/state/materialization revalidation → callback settlement` with zero repository/Git events. In the race, acceptance holds writer after its snapshot read, a pull acquires repository and blocks while entering the sole `runUnderOrderedWorkspaceWriterLocks` callback before active publication, acceptance revalidates and completes without requesting repository, releases writer, and the pull then prepares and publishes while its full set remains held. A pull that wins before acceptance acquires writer changes the active pointer, so acceptance's locked identity reread fails closed and the curator retries against the new active snapshot. Fail structurally on any writer→repository attempt, participant lock reentry, or repository/Git dependency reachable from the accept callback. This proves the global order and absence of the P5/P6 deadlock rather than relying on a timeout-only assertion.
|
||
|
||
Add the pre-READY/READY resume tests as one ordered transition: after pull, the prepare manifest verifies and both session admission and `preprocess run --resume <id>` return `migration_required`. Execute exact released `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json`; assert the selected unready commit+binding's physical/LSH snapshot is published and strictly reverified by the fixed cache/materialization stages without READY, and admission/resume still return `migration_required`. Only then does `schema accept` succeed; immediately retest both refusals and prove no later schema/Evidence stage ran. Execute P3's exact `workspace migrate activate-revision-layout --workspace <id> --yes --json` command, strictly re-read the accepted revision's exact commit+effective-binding READY, and only then require the same accepted resume to continue at the stage after FK review. A READY for another commit/binding, a subsequent Git update, byte replacement, manifest corruption, or DWH binding change returns `preprocessing_resume_mismatch` or `migration_required` as owned by the violated boundary and does not index schema/Evidence. A new invocation without `--resume` creates a new run. Assert raw parser/Git/harness errors and endpoints never enter JSON.
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run \
|
||
test/workspace-preprocessing-state.test.ts \
|
||
test/workspace-preprocessing-service.test.ts \
|
||
test/workspace-registry.test.ts \
|
||
test/workspace-maintenance.test.ts -t "resolve run|lock order|schema export|schema accept|accepted annotation|resume"
|
||
```
|
||
|
||
Expected: FAIL because `schema-export-fks`, secure run resolution, and `schema-accept` are unknown.
|
||
|
||
- [ ] **Step 3 (GREEN): implement minimal dispatch and resume checks.**
|
||
|
||
Extend `WorkspacePreprocessingService.execute()`; reuse `AnnotationSynchronizer`, `PreprocessingStateStore`, `resolveRun`, `WorkspaceRuntimeConfigLeaseFactory`, the P3 harness-owned effective-binding result, P3 `WorkspaceRegistry.read(id)` active immutable-snapshot API, and P2's child invocation. Require `yes === true`; do not prompt inside the container. The service, not `backend/src/workspace-maintenance.ts::main`, owns the transaction. Inject only the RO active-snapshot reader, P3 strict pre-READY physical/LSH snapshot resolver/verifier, and capability-only prepared-annotation verifier into the accept path: no repository method, repository lock, pull or migration function, `ensure()`, or Git child is present in its capability/type. The DWH verifier must accept only the selected revision+binding publication produced by `migrate_dwh_cache`, require its manifest/digests to strictly reverify without READY, and reject a prior-revision or sibling-binding snapshot. `verifyPrepared(..., { kind: "prepare", rootLease, writerCapability })` accepts an already prepared immutable identity, resolves only through `futureWorkspaceLayoutPaths`, and performs no lock or READY acquisition. `schema check` success is necessary but not sufficient—the subsequent state-store acceptance call is the human decision. Update `preprocess-run` resume handling to call `assertAcceptedAnnotation` before any later stage.
|
||
|
||
- [ ] **Step 4 (verify GREEN):**
|
||
|
||
```bash
|
||
cd backend
|
||
npx vitest run \
|
||
test/workspace-preprocessing-state.test.ts \
|
||
test/workspace-preprocessing-service.test.ts \
|
||
test/workspace-registry.test.ts \
|
||
test/workspace-maintenance.test.ts
|
||
npx tsc --noEmit -p .
|
||
npm run build
|
||
```
|
||
|
||
Expected: PASS and stdout remains one pristine JSON document.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add backend/src/workspaces/preprocessing-state.ts \
|
||
backend/src/workspaces/preprocessing-service.ts backend/src/workspace-maintenance.ts \
|
||
backend/test/workspace-preprocessing-state.test.ts \
|
||
backend/test/workspace-preprocessing-service.test.ts \
|
||
backend/test/workspace-maintenance.test.ts backend/test/workspace-registry.test.ts
|
||
git commit -m "feat: accept reviewed annotation revisions"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 7: Expose safe author-clone export and explicit acceptance through `thothctl`
|
||
|
||
**Files:**
|
||
- Modify: `tools/thothctl/internal/workspaceops/operations.go`
|
||
- Modify: `tools/thothctl/internal/workspaceops/operations_test.go`
|
||
- Modify: `tools/thothctl/internal/safeio/files.go`
|
||
- Modify: `tools/thothctl/internal/safeio/files_unix_test.go`
|
||
- Modify: `tools/thothctl/internal/safeio/files_windows_test.go`
|
||
- Modify: `tools/thothctl/cmd/thothctl/main.go`
|
||
- Modify: `tools/thothctl/cmd/thothctl/main_test.go`
|
||
|
||
- [ ] **Step 1 (RED): add parse/run/file tests.**
|
||
|
||
The released P5 host syntax is below; the pull, DWH snapshot preparation, and activation commands are required, already-released P3 operations and are not reimplemented by P5:
|
||
|
||
```text
|
||
thothctl --installation <absolute>/thothii-installation.yaml \
|
||
workspace schema export-fks --run <32-hex-id> --output <absolute-file> --json
|
||
# after edit + ordinary git commit/push:
|
||
thothctl --installation <absolute>/thothii-installation.yaml \
|
||
workspace registry pull --workspace <resolved-workspace-id> --json
|
||
WORKSPACE_ID=<resolved-workspace-id>
|
||
thothctl --installation <absolute>/thothii-installation.yaml \
|
||
workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json
|
||
# only after strict pre-READY physical/LSH snapshot verification:
|
||
thothctl --installation <absolute>/thothii-installation.yaml \
|
||
workspace schema accept --run <32-hex-id> --yes --json
|
||
# the accepted revision is still unready; this exact P3 operation must complete next:
|
||
thothctl --installation <absolute>/thothii-installation.yaml \
|
||
workspace migrate activate-revision-layout --workspace "$WORKSPACE_ID" --yes --json
|
||
# only after strict commit+binding READY verification:
|
||
thothctl --installation <absolute>/thothii-installation.yaml \
|
||
workspace preprocess run --resume <32-hex-id> --json
|
||
```
|
||
|
||
P2's `workspace schema suggest-fks --workspace ...` remains the operation that creates a new candidate, but it is **not** the curator export after a full run pauses and is never substituted for `export-fks`. `export-fks` must require exactly one run and output, use the same secure `resolveRun` and state-owned candidate read as acceptance, and report a `sha256:<64hex>` equal to the paused state's recorded `candidateDigest`. It must not accept `--workspace`, SQL, assumptions, input, resume, or suggestion flags.
|
||
|
||
Keep P2's `--output` safety contract and name; do not add a second synonym. Tests require an absolute canonical output below an existing author-owned directory, refuse a symlink/reparse point in every existing component, refuse an existing destination, accept exactly 716,800 decoded candidate bytes and reject 716,801, require base64 plus the complete child envelope to remain within exactly 1 MiB, write a same-directory exclusive temp file, fsync, rename, and leave no partial on child failure. Candidate bytes come only from P2's dedicated framed/bounded `hostExport`, never logs/public JSON and never the separate 16 MiB curated-blob channel. Verify the child envelope run/workspace/digest/count, recompute the host bytes, require both equal the paused state digest, then remove `hostExport` before pristine JSON. On Windows use the existing safeio retained-handle/reparse policy; document its trusted-local-ACL limitation rather than weakening it.
|
||
|
||
`schema accept` must require exactly one `--run`, literal `--yes`, optional `--json`, and no workspace/path/input flags. Both P5 commands reject traversal, zero/multiple run matches, removed workspaces, unsafe job/candidate files, and mismatched embedded identity using the frozen safe codes. Freeze the host sequence test: dispatch pull; observe admission/resume `migration_required`; dispatch the exact already-released `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json`; prove its returned operation is `migrate_dwh_cache`, its fixed cache plus `p3_materialize_dwh_snapshot` stages publish and strictly reverify the active unready revision's binding-qualified physical/LSH snapshot without READY, and admission/resume remain `migration_required`; dispatch accept and prove both remain refused; then dispatch exact `workspace migrate activate-revision-layout --workspace "$WORKSPACE_ID" --yes` (after interruption, only the corresponding command's returned 32-hex outer run ID may be supplied to that same command with `--resume`). Verify commit+binding READY and only then dispatch preprocessing resume successfully. Parser/dispatcher tests reject missing/duplicate `--workspace`, arbitrary paths, `--yes` on DWH preparation, and cross-operation/cross-workspace resume IDs. Assert the Go layer sends only fixed maintenance requests and redacts Compose stderr/secrets. Add usage text.
|
||
|
||
Add capability-regression tests against P3's generated final override. `workspace registry pull` alone must select the registry RW mount and only the exact validated HTTPS/SSH Git transport needed by that installation. `workspace migrate dwh-cache`, `schema accept`, export, resume, and representative ordinary mutations must select the registry RO mount, omit every Git transport file/environment variable, and fail a fixture attempt to create the repository lock, alter the checkout, or contact the remote. Conversely, pull must not inherit unrelated DWH/Evidence/Pi secrets. Validate the rendered Compose structure before invocation and assert no P5 code broadens or duplicates P3's binding-discovery/generator surface.
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
cd tools/thothctl
|
||
go test ./internal/workspaceops ./cmd/thothctl -run 'Annotation|SchemaAccept|Export|RegistryPull|MigrateDwhCache' -count=1
|
||
```
|
||
|
||
Expected: FAIL because P5 acceptance parsing is absent and P2 export does not yet enforce the P5 author-clone contract.
|
||
|
||
- [ ] **Step 3 (GREEN): implement host orchestration.**
|
||
|
||
Extend the frozen `workspaceops.ParseWorkspaceCommand` / `workspaceops.Run`; continue using the exact P2 maintenance invocation for the P5 export/accept processes:
|
||
|
||
```text
|
||
docker compose ... --profile workspace-maintenance run --rm --no-deps -T \
|
||
workspace-maintenance <fixed argv>
|
||
```
|
||
|
||
Reuse P3's existing `registry_pull`, `migrate_dwh_cache`, and `activate_revision_layout` orchestration and generated capability overrides unchanged. Do not mount the author clone. `export-fks` exports only the already-persisted candidate owned by the paused run through the bounded child channel and safe host writer; it never calls `suggest-fks`. Git add/commit/push remains a manual ordinary Git action outside `thothctl`. The curator invokes P3 pull explicitly after push, then explicitly invokes P3 `migrate dwh-cache` and preserves that operation's own durable returned run ID for same-command `--resume` recovery. The DWH preparation must finish and strictly reverify the selected unready revision's physical/LSH snapshot before the later accept process, which has only RO/no-Git capability and validates the registry's already-active exact blob/snapshot; it never pulls or migrates implicitly. Acceptance does not make the revision runnable: dispatch the exact P3 activation after accept, preserve its distinct durable returned run ID for exact same-command `--resume` recovery, verify the accepted commit+binding READY, and dispatch preprocessing resume only afterward.
|
||
|
||
- [ ] **Step 4 (verify GREEN):**
|
||
|
||
```bash
|
||
cd tools/thothctl
|
||
gofmt -w internal/workspaceops/operations.go internal/workspaceops/operations_test.go \
|
||
internal/safeio/files.go \
|
||
cmd/thothctl/main.go cmd/thothctl/main_test.go
|
||
go test ./... -count=1
|
||
cd ../..
|
||
bash scripts/test-preprocess-compose-config.sh
|
||
bash scripts/test-compose-secret-policy.sh
|
||
```
|
||
|
||
Expected: all Go tests and P3/P5 grammar/capability regressions PASS; the exact DWH migration is dispatched between pull and accept, generated `registry_pull` is RW/Git, generated `migrate_dwh_cache`, `schema-accept`, and ordinary mutation overrides are registry RO/no-Git, and no production credential is read.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add tools/thothctl/internal/workspaceops tools/thothctl/internal/safeio \
|
||
tools/thothctl/cmd/thothctl/main.go \
|
||
tools/thothctl/cmd/thothctl/main_test.go
|
||
git commit -m "feat: add curated annotation acceptance commands"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 8: Document the Git author workflow and create an independent manual walkthrough
|
||
|
||
**Files:**
|
||
- Create: `docs/contracts/workspace-annotations.md`
|
||
- Modify: `docs/install/local-workspace-registry.md`
|
||
- Modify: `docs/install/server-workspace-registry.md`
|
||
- Modify: `docs/testing/p2-p6-manual-verification.md`
|
||
- Create: `backend/scripts/p5-manual-verification.mjs`
|
||
- Create: `backend/scripts/p5-manual-verification.test.mjs`
|
||
- Create: `scripts/p5-manual-verification.sh`
|
||
|
||
- [ ] **Step 1 (RED): write manual-helper contract tests.**
|
||
|
||
Follow the P1 helper's ownership discipline but use independent `.artifacts/manual-acceptance/p5`. Test only `prepare`, `status`, and `cleanup`; the helper must not run the reviewer commands or decide PASS. `prepare` creates a new local bare remote, ordinary `author/` clone, isolated installation/env/fixture-secret files, generated immutable command scripts, `ownership.json`, and `GUIDE.md`. Contract tests parse the generated guide/commands and fail unless pull → validated `WORKSPACE_ID` → exact `migrate dwh-cache --workspace "$WORKSPACE_ID" --json` → strict pre-READY physical/LSH verification → accept → activation/READY → resume appears in that order, with `migration_required` checks after pull, migration, and accept and distinct same-command recovery IDs for migration and activation. `cleanup` refuses live/unowned/replaced roots and deletes only the exact owned Compose project/resources and manual root; no prune or broad process matching.
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
node --test backend/scripts/p5-manual-verification.test.mjs
|
||
bash -n scripts/p5-manual-verification.sh
|
||
```
|
||
|
||
Expected: FAIL because helpers do not exist.
|
||
|
||
- [ ] **Step 3 (GREEN): write the exact P5 section and generated guide.**
|
||
|
||
Replace only the P5 placeholder in `docs/testing/p2-p6-manual-verification.md`. Keep P2/P3/P4/P6 decisions untouched. The generated guide must have the reviewer personally:
|
||
|
||
1. inspect clean author and installation roots plus exact ThothII commit/tree;
|
||
2. run `workspace preprocess run --workspace p5-fk --json` and record `manual_review_required`, the 32-hex run ID, and candidate digest;
|
||
3. run `workspace schema export-fks --run <recorded-id> --output <author>/workspace-content/p5-fk/schema/annotations.yaml --json`, hash the exported file, and require both the command digest and file digest to equal the paused run's recorded candidate digest **before** editing;
|
||
4. inspect/edit the exported YAML in the author clone and run the provided harness validation command without printing secrets;
|
||
5. prove `git status` contains annotations but no `physical.yaml`, then `git add`, `git commit`, `git push`;
|
||
6. record `git rev-parse HEAD`, `git rev-parse HEAD:workspace-content/p5-fk/schema/annotations.yaml`, mode/type/size, and SHA-256;
|
||
7. explicitly run the P3 `workspace registry pull --workspace <resolved-workspace-id> --json` command, record the activated-but-unready revision, verify its prepared annotation manifest exists at the exact future revision path, and verify its generated maintenance override alone used registry RW plus exact Git transport;
|
||
8. immediately prove production session admission and `workspace preprocess run --resume <id> --json` both refuse with `migration_required`, with no Pi/later preprocessing stage;
|
||
9. export `WORKSPACE_ID` as that exact resolved ID and run released `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json`; if interrupted, validate and use only its returned 32-hex outer run ID with the same DWH command plus `--resume`; inspect the fixed `p3_migrate_dwh_cache`/`p3_prepare_dwh_cache` and `p3_materialize_dwh_snapshot` evidence, require the selected revision+binding physical/LSH snapshot and manifest/digests under `revisions/<commit>/dwh-snapshots/<binding-key>/{artifacts,indexes}`, prove no prior-revision fallback and no READY publication, and recheck that admission/resume still refuse `migration_required`;
|
||
10. while still pre-READY, run `workspace schema accept --run <id> --yes --json` as a distinct invocation, verify its override was registry RO/no-Git, prove it performed no pull/migration/repository-lock/Git child, inspect acceptance candidate/current digests plus `matches_candidate` (no requirement that it be true after curation), and recheck admission/resume remain `migration_required`;
|
||
11. inspect synchronized exact bytes, restrictive modes, and `annotations.manifest.json` fields;
|
||
12. run the exact P3 command `workspace migrate activate-revision-layout --workspace "$WORKSPACE_ID" --yes --json`; if interrupted, export only its distinct returned 32-hex outer run ID and recover with the same command plus `--resume <activation-run-id>`; then strictly inspect `READY.json` and require the accepted 40-hex commit plus the probed effective-DWH binding/cache key;
|
||
13. only after that READY verification, resume the accepted preprocessing run and confirm it crosses FK review with the accepted blob; admit a session and prove runtime consumption now succeeds;
|
||
14. pin/open the historical revision, publish a second annotation commit, explicitly pull again, repeat refusal → DWH cache/snapshot preparation → refusal → accept → refusal → exact activation → READY verification before use, and prove old/new runtime files remain different and unchanged;
|
||
15. prove an unrelated valid `workspace-content/other-workspace/schema/annotations.yaml` coexists and is never read/modified for `p5-fk`; separately attempt a caller-controlled mismatched ID/path and confirm it cannot escape the derived namespace;
|
||
16. run prepared negatives separately for candidate/export 716,801 decoded bytes and curated Git blob 16 MiB + 1, plus malformed, symlink, tree/gitlink, run traversal/duplicate/removed-workspace/symlink cases, accept without pull, accept without pre-READY DWH preparation, resume before READY at every boundary, wrong-commit/wrong-binding physical snapshot or READY, interrupted DWH/activation recovery with a wrong or cross-operation run ID, post-accept blob change, and DWH-binding mismatch; confirm no partial publication or later-stage work;
|
||
17. run the generated secret scan and exact cleanup;
|
||
18. record `Decision: **PASS**` or `**FAIL**` manually with report paths and notes.
|
||
|
||
The install docs must state: only the canonical path is supported; absence yields warning/empty set; `physical.yaml` is never committed; the API does not push curated bytes; after a pause, `schema export-fks --run` (not a new suggestion) must reproduce the recorded candidate digest before editing; the candidate/export maximum is exactly 716,800 decoded bytes while the separate curated Git blob maximum is exactly 16 MiB. Freeze the supported order as author clone → Git commit/push → explicit P3 `workspace registry pull --workspace <id> --json` → pre-READY admission/resume refusal with `migration_required` → exact released `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json` → strict selected-revision+binding physical/LSH snapshot verification without READY or prior-revision fallback → repeated `migration_required` refusal → distinct explicit `workspace schema accept --run <id> --yes --json` → repeated `migration_required` refusal → exact P3 `workspace migrate activate-revision-layout --workspace "$WORKSPACE_ID" --yes --json` → strict accepted commit+effective-binding READY verification → preprocessing resume. Pull alone is registry RW/Git; DWH migration, accept, and ordinary mutations remain registry RO/no-Git. Both local and server install/recovery sections must show that interrupted DWH preparation and activation are resumed only with the distinct returned 32-hex outer run ID on the same exact originating command; operators must validate each ID, never cross-use it, never hand-create/edit READY or snapshot files, and never reuse the preprocessing run ID as a maintenance run ID. Unrelated workspace namespaces coexist; old pinned revisions keep old annotations.
|
||
|
||
- [ ] **Step 4 (verify GREEN):**
|
||
|
||
```bash
|
||
node --test backend/scripts/p5-manual-verification.test.mjs
|
||
./scripts/p5-manual-verification.sh prepare
|
||
./scripts/p5-manual-verification.sh status
|
||
./scripts/p5-manual-verification.sh cleanup
|
||
rg -n "schema export-fks|migrate dwh-cache|schema accept|annotations.yaml|physical.yaml|Decision: \*\*PENDING\*\*" \
|
||
docs/testing/p2-p6-manual-verification.md \
|
||
docs/install/local-workspace-registry.md docs/install/server-workspace-registry.md
|
||
```
|
||
|
||
Expected: tests PASS; prepare/status/cleanup PASS; P5 remains `PENDING` until a human decision; no other Px decision changed.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add docs/contracts/workspace-annotations.md docs/install/local-workspace-registry.md \
|
||
docs/install/server-workspace-registry.md docs/testing/p2-p6-manual-verification.md \
|
||
backend/scripts/p5-manual-verification.mjs \
|
||
backend/scripts/p5-manual-verification.test.mjs scripts/p5-manual-verification.sh
|
||
git commit -m "docs: add curated annotation review walkthrough"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 9: Build the clean-state P5 automated process goal
|
||
|
||
**Files:**
|
||
- Create: `backend/scripts/p5-acceptance.mjs`
|
||
- Create: `backend/scripts/p5-acceptance.test.mjs`
|
||
- Create: `scripts/p5-acceptance.sh`
|
||
- Create: `scripts/test-p5-acceptance.sh`
|
||
|
||
The one-command interface is:
|
||
|
||
```text
|
||
./scripts/p5-acceptance.sh integration [--keep]
|
||
```
|
||
|
||
Each run owns only:
|
||
|
||
```text
|
||
.artifacts/p5-integration/<run-id>/
|
||
├── ownership.json
|
||
├── remote.git/
|
||
├── author/
|
||
├── installation/
|
||
├── runtime-data/
|
||
├── fixture-secrets/
|
||
├── candidates/
|
||
├── responses/
|
||
├── manifests/
|
||
├── logs/
|
||
├── report.json
|
||
└── report.md
|
||
```
|
||
|
||
and a unique Compose project/container/network/volume/image reference labelled with the run ID. Use a child process group and one hard deadline; TERM then KILL only that group. There are no retries. All remote/DWH credentials are generated fixture canaries. Every command event records bounded sanitized argv category, exit status, start/end, and artifact hashes—not raw secret-bearing environment or unbounded stderr.
|
||
|
||
- [ ] **Step 1 (RED): write harness unit/contract tests.**
|
||
|
||
Test canonical run-root validation, exclusive ownership creation, symlink/root replacement refusal, unique run/Compose identities, fixed check IDs, no-retry execution, report schema/hash validation, secret scanning, external-network guard, cleanup refusal for foreign resources, process-group shutdown, and `--keep` semantics. Freeze the production event order as pull → first refusal → exact DWH cache/snapshot preparation → second refusal → accept → third refusal → activation/READY → resume, and fail if `pre_ready_dwh_snapshot_publication` or the all-three-boundaries `pre_ready_admission_resume_refusal` event is missing, duplicated, or reordered. Include mutation tests showing removal of any required negative case, secret scan, final ownership check, or report hash makes the harness test fail.
|
||
|
||
Required report check IDs:
|
||
|
||
```text
|
||
clean_source_identity
|
||
isolated_real_git_author_flow
|
||
paused_candidate_export_and_digest
|
||
secure_run_resolution
|
||
exact_commit_blob_validation
|
||
fixed_git_child_lifecycle
|
||
harness_parser_validation
|
||
dirfd_atomic_revision_materialization
|
||
operation_specific_registry_capability
|
||
registry_pull_same_id_recovery
|
||
registry_pull_target_drift_refusal
|
||
pre_ready_dwh_snapshot_publication
|
||
explicit_acceptance_transition
|
||
pre_ready_admission_resume_refusal
|
||
accepted_revision_layout_activation
|
||
repository_writer_lock_order
|
||
unrelated_namespace_coexistence
|
||
accepted_resume_and_mismatch_refusal
|
||
pinned_revision_isolation
|
||
negative_object_and_size_cases
|
||
distinct_candidate_blob_limits
|
||
physical_yaml_not_published
|
||
secret_scan
|
||
owned_resource_shutdown
|
||
exact_cleanup
|
||
```
|
||
|
||
- [ ] **Step 2 (verify RED):**
|
||
|
||
```bash
|
||
node --test backend/scripts/p5-acceptance.test.mjs
|
||
bash scripts/test-p5-acceptance.sh
|
||
```
|
||
|
||
Expected: FAIL because the P5 harness does not exist.
|
||
|
||
- [ ] **Step 3 (GREEN): implement the acceptance scenario through production interfaces.**
|
||
|
||
From a clean source commit, the harness must build/use the real `thothctl`, real `workspace-maintenance` service/core image, real local bare Git remote and author clone, real registry pull, real P3 `migrate_dwh_cache` maintenance path, production parser/synchronizer/state store, and controlled REST DWH fixture sufficient to generate/check FK. It must:
|
||
|
||
1. create a descriptor with no annotations and prove the safe absent publication/warning;
|
||
2. start a P2 run and capture run ID/candidate digest at `manual_review_required`;
|
||
3. invoke production `workspace schema export-fks --run <id> --output ...`, require its returned digest and a fresh SHA-256 of the exact output to equal the paused run digest before editing, assert no suggestion/DWH child ran, and prove 716,800 decoded bytes fits the complete 1 MiB child envelope while 716,801 is refused before host publication;
|
||
4. edit that file in the ordinary author clone to a valid curated document, commit/push, and record exact commit/blob/mode/size/SHA-256;
|
||
5. invoke P3 production `workspace registry pull --workspace <resolved-workspace-id> --json` explicitly through the exact `RegistryAddressedRequestV1`/`RegistryPullJobStateV1` path; prove create/resume both carry required production installation/repository/remote-ref identities, create also carries the exact expected base, plan/state retain all three identities, and only this operation receives registry RW/exact Git transport. In isolated real-process subcases with at least two reverse-presented changed workspace identities, SIGKILL at `publication_intent_durable` immediately before active-pointer rename and resume the exact returned 32-hex ID, then SIGKILL a fresh fixture immediately after active-pointer rename+parent fsync but before `target_published` persistence and resume that exact ID. Require complete strict-lexical `changedWorkspaceIds`, matching `AddressedWorkspacePublicationLeaseV1` participant calls, no participant/root/writer/reader reentry within each attempt, no fetch/ls-remote/target reselection on either resume, no second publication rename in the post-publication case, full-set ownership through rename+fsync, reverse release, then repository release. Inject third active identity and changed/deleted target object/inventory/digest and require refusal without mutation/owner clear. Only after those dependency subcases pass, run the scenario's successful C1 pull and prove the annotation participant prepares/verifies through exact `futureWorkspaceLayoutPaths` without READY; record `operation_specific_registry_capability`, `registry_pull_same_id_recovery`, `registry_pull_target_drift_refusal`, and `repository_writer_lock_order`.
|
||
6. before READY, invoke production session admission and `workspace preprocess run --resume <id> --json`; require both to return `migration_required` and prove no Pi or later preprocessing stage starts;
|
||
7. set and validate the exact resolved `WORKSPACE_ID`, then invoke production `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json`; prove the `migrate_dwh_cache` run uses registry RO/no-Git and its fixed `p3_migrate_dwh_cache`/`p3_prepare_dwh_cache` plus `p3_materialize_dwh_snapshot` stages publish and strictly reverify the curated commit+binding physical/LSH snapshot under its own future revision root without reading or publishing READY or falling back to the base revision; exercise interrupted recovery only with this command's returned 32-hex outer run ID and record `pre_ready_dwh_snapshot_publication`;
|
||
8. retest session admission and preprocessing resume after DWH preparation; require `migration_required` and prove no Pi/later stage;
|
||
9. still before READY, invoke production `workspace schema accept --run <id> --yes --json` separately; prove its generated container has registry RO/no-Git, its service calls no repository lock/pull/migration/Git child, its prepare verifiers succeed against the exact annotation and physical/LSH snapshots without READY, and acceptance records both digests and P3 effective binding; immediately prove admission/resume remain `migration_required`, then record `pre_ready_admission_resume_refusal` only after all three post-pull, post-DWH-preparation, and post-accept refusal pairs pass;
|
||
10. invoke the exact P3 production `workspace migrate activate-revision-layout --workspace "$WORKSPACE_ID" --yes --json`; exercise one interrupted activation and recover only with its distinct returned 32-hex outer run ID plus `--resume`; strictly verify the resulting READY names the accepted commit and exact probed effective-DWH binding/cache key before recording `accepted_revision_layout_activation`;
|
||
11. only after that check, resume the accepted preprocessing run and admit a session; require runtime annotation consumption to use strict committed state plus exact four-argument `workspaceRuntimePaths`, then verify exact runtime bytes and manifest, including restrictive modes and destination;
|
||
12. retain an old pin across a second commit and explicit second pull, repeat pre-READY refusal → exact DWH cache/snapshot preparation → refusal → accept → refusal → exact activation → READY verification, then prove byte/path isolation;
|
||
13. exercise `resolveRun` through production commands for traversal, duplicate run ID across two active workspaces, removed workspace, job/candidate symlink, hardlink, and embedded identity mismatch; require stable safe codes and no Git/DWH/later stage;
|
||
14. deterministically replace an already-opened revision/artifacts/mschema ancestor at the pull fixture's dirfd barrier and prove publication remains in the retained inode or fails with no file in the replacement tree;
|
||
15. add a valid annotation blob under another workspace namespace, prove the target lookup/accept ignores and preserves it, then separately attempt a mismatched caller-controlled lookup and prove confinement;
|
||
16. use separate fresh remotes/commits for malformed UTF-8/YAML, curated blob exactly 16 MiB and 16 MiB + 1, candidate/export exactly 716,800 and 716,801, symlink, tree, gitlink, missing-compatible case, accept without prior pull, accept without pre-READY DWH preparation, resume before READY at every boundary, wrong-commit/wrong-binding physical snapshot or READY, wrong/cross-operation DWH or activation resume ID, blob changed after acceptance, and DWH-binding mismatch;
|
||
17. exercise the shared fixed-Git child with real hang, stdout/record/stderr flood, early exit, parser abort, cancellation, and TERM-ignoring descendant fixtures; prove bounded capture, owned-group TERM→KILL, awaited exit, and no output exposure;
|
||
18. run the controlled pull/DWH-preparation/accept barriers and record repository→writer for pull, independent P3 maintenance for snapshot publication, snapshot→writer-only for accept, stable accepted identity, completion without deadlock, and the later pull advancing only after writer release;
|
||
19. prove every failed case preserves the last active state, publishes no incomplete annotation or DWH snapshot, performs no later preprocessing stage, and emits only stable safe codes;
|
||
20. inspect the Git tree to prove no `physical.yaml` exists;
|
||
21. shut down every owned child/listener/container, refuse connections, remove exact owned Compose resources (never global prune), remove fixture secrets, run the final secret scan, and record cleanup evidence.
|
||
|
||
No frontend, browser API, Qdrant rebuild, Evidence materialization, Evidence adapter, or P6 retention is part of this process goal.
|
||
|
||
`--keep` retains the bounded disk evidence/report after secret-file deletion and live-resource cleanup. Without `--keep`, copy the final report to a caller-selected safe location only if such behavior already exists in P2; otherwise remove the owned root. The release evidence run uses `--keep`.
|
||
|
||
- [ ] **Step 4 (verify GREEN):**
|
||
|
||
```bash
|
||
node --test backend/scripts/p5-acceptance.test.mjs
|
||
bash scripts/test-p5-acceptance.sh
|
||
```
|
||
|
||
Expected: all harness/mutation tests PASS without running the expensive integration scenario.
|
||
|
||
- [ ] **Step 5: commit.**
|
||
|
||
```bash
|
||
git add backend/scripts/p5-acceptance.mjs backend/scripts/p5-acceptance.test.mjs \
|
||
scripts/p5-acceptance.sh scripts/test-p5-acceptance.sh
|
||
git commit -m "test: add curated annotation process goal"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 10: Verify P5, run one clean retained goal, and publish the checkpoint report
|
||
|
||
**Files:**
|
||
- Modify: `PROJECT_STATE.md`
|
||
- Modify only if evidence requires it: `docs/testing/p2-p6-manual-verification.md`
|
||
|
||
- [ ] **Step 1: prove a clean committed source before the process goal.**
|
||
|
||
```bash
|
||
git status --short
|
||
git diff --check
|
||
git rev-parse HEAD
|
||
git rev-parse HEAD^{tree}
|
||
```
|
||
|
||
Expected: clean status, no whitespace errors, exact commit/tree captured. Do not run the goal from uncommitted production bytes.
|
||
|
||
- [ ] **Step 2: run focused layer verification.**
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/pytest tests/test_schema_fk_annotations.py tests/test_annotation_validation_cli.py \
|
||
tests/test_protected_fs_helper.py tests/test_locked_child_stdin.py \
|
||
tests/test_layout_marker_commands.py tests/test_p3_internal_cli.py -q
|
||
.venv/bin/ruff check tht/mschema/models.py tht/cli/schema_cmd.py tht/protected_fs_helper.py \
|
||
tht/locked_child_stdin.py tests/test_schema_fk_annotations.py \
|
||
tests/test_annotation_validation_cli.py tests/test_protected_fs_helper.py \
|
||
tests/test_locked_child_stdin.py tests/test_layout_marker_commands.py \
|
||
tests/test_p3_internal_cli.py
|
||
|
||
cd ../backend
|
||
npx vitest run \
|
||
test/workspaces-git-repository.test.ts \
|
||
test/protected-workspace-fs.test.ts \
|
||
test/workspace-annotations.test.ts \
|
||
test/workspace-registry.test.ts \
|
||
test/workspace-registry-factory.test.ts \
|
||
test/workspace-runtime-handoff.test.ts \
|
||
test/workspace-runtime-config-lease.test.ts \
|
||
test/workspace-revision-layout.test.ts \
|
||
test/workspace-runtime-renderer.test.ts \
|
||
test/workspace-preprocessing-state.test.ts \
|
||
test/workspace-preprocessing-service.test.ts \
|
||
test/workspace-maintenance.test.ts \
|
||
test/routes-sessions.test.ts
|
||
npx tsc --noEmit --target ES2022 --module ES2022 --moduleResolution Bundler \
|
||
--strict --skipLibCheck test/workspace-locked-child-cumulative.compile.ts
|
||
npx tsc --noEmit -p .
|
||
npm run build
|
||
node --test scripts/p5-acceptance.test.mjs scripts/p5-manual-verification.test.mjs
|
||
|
||
cd ../tools/thothctl
|
||
go test ./... -count=1
|
||
|
||
cd ../..
|
||
bash scripts/test-p5-acceptance.sh
|
||
bash -n scripts/p5-acceptance.sh scripts/p5-manual-verification.sh
|
||
git diff --check
|
||
```
|
||
|
||
Expected: every command exits 0. Record exact test counts; do not summarize a failure as PASS. P6 owns the aggregate P2–P6 Docker smoke and full repository suites, so do not expand P5 into P6 scope.
|
||
|
||
- [ ] **Step 3: run the process goal once, with no retry.**
|
||
|
||
```bash
|
||
RETAINED_REPORT="$(./scripts/p5-acceptance.sh integration --keep)"
|
||
[[ "$RETAINED_REPORT" =~ ^\.artifacts/p5-integration/([0-9a-f]{32})/report\.md$ ]] || {
|
||
printf 'unexpected retained report path: %q\n' "$RETAINED_REPORT" >&2
|
||
exit 2
|
||
}
|
||
RUN_ID="${BASH_REMATCH[1]}"
|
||
printf 'export RUN_ID=%q\n' "$RUN_ID"
|
||
```
|
||
|
||
Expected: exit 0, exactly one captured retained path `.artifacts/p5-integration/<32-hex-run-id>/report.md`, a validated 32-lowercase-hex `RUN_ID`, and one safe export line for the next independently runnable step. If it fails, use @superpowers:systematic-debugging, add the smallest failing regression test, fix it, commit the scoped fix, and rerun the **entire scenario from a new run ID/root**. Never relabel or overwrite a failed report; the checkpoint names only the final green run.
|
||
|
||
- [ ] **Step 4: independently inspect the retained report and cleanup evidence.**
|
||
|
||
```bash
|
||
RUN_ID="${RUN_ID:?export RUN_ID as the 32-hex ID captured and printed by Step 3}"
|
||
[[ "$RUN_ID" =~ ^[0-9a-f]{32}$ ]] || {
|
||
printf 'invalid RUN_ID\n' >&2
|
||
exit 2
|
||
}
|
||
RUN=".artifacts/p5-integration/$RUN_ID"
|
||
[[ -d "$RUN" && ! -L "$RUN" ]] || {
|
||
printf 'missing or unsafe retained run root: %q\n' "$RUN" >&2
|
||
exit 2
|
||
}
|
||
python3 - "$RUN" <<'PY'
|
||
import hashlib, json, pathlib, sys
|
||
root = pathlib.Path(sys.argv[1])
|
||
report = json.loads((root / "report.json").read_text())
|
||
assert report["overall"] == "PASS"
|
||
assert all(check["status"] == "PASS" for check in report["checks"])
|
||
assert report["cleanup"]["owned_resources_remaining"] == []
|
||
for name in ("report.json", "report.md"):
|
||
print(name, hashlib.sha256((root / name).read_bytes()).hexdigest())
|
||
PY
|
||
```
|
||
|
||
Then run the harness's documented retained-artifact secret scan verification. Expected: all required check IDs present exactly once, all PASS, no fixture secret files/canary values, no live owned resources, report hashes printed. Do not manually grep secret contents into terminal history.
|
||
|
||
- [ ] **Step 5: update the checkpoint without claiming manual acceptance.**
|
||
|
||
Add a P5 section to `PROJECT_STATE.md` containing:
|
||
|
||
- implementation commit and tree;
|
||
- retained run/report path;
|
||
- `report.json` and `report.md` SHA-256;
|
||
- focused command/test counts;
|
||
- `automated integration: PASS`;
|
||
- `manual acceptance: PENDING`;
|
||
- known platform/manual limitations;
|
||
- explicit statement that P6 Evidence materialization and aggregate P2–P6 verification have not run.
|
||
|
||
Keep the P5 manual section decision `PENDING` until the reviewer performs it. Run:
|
||
|
||
```bash
|
||
git diff --check
|
||
git status --short
|
||
git add PROJECT_STATE.md
|
||
git commit -m "docs: record P5 automated verification checkpoint"
|
||
git status --short
|
||
```
|
||
|
||
Expected: final status clean.
|
||
|
||
- [ ] **Step 6: stop for the required user checkpoint.**
|
||
|
||
Provide a checkpoint report with scoped commits, exact commands/counts, retained path/hashes, secret-scan and cleanup result, and the exact manual entry command:
|
||
|
||
```bash
|
||
./scripts/p5-manual-verification.sh prepare
|
||
```
|
||
|
||
State plainly: `automated integration: PASS / manual acceptance: PENDING`. Do not begin P6 or the aggregate test until the user records the P5 decision and explicitly authorizes continuation.
|
||
|
||
## Final verification checklist
|
||
|
||
- [ ] Canonical Git path is fixed and cross-workspace paths are impossible.
|
||
- [ ] Object is resolved at the descriptor's exact commit and only regular blob modes pass.
|
||
- [ ] Suggested candidate/export is exactly bounded at 716,800 decoded bytes and complete `hostExport` at 1 MiB; the separate curated Git blob alone is exactly bounded at 16 MiB; both boundaries and +1 failures are tested.
|
||
- [ ] Absence is compatible and distinguishable from an invalid empty file.
|
||
- [ ] Runtime publication is immutable, revision-qualified, restrictive, atomic, and manifest-owned through retained dirfds/openat-style operations; real ancestor-replacement races cannot redirect it.
|
||
- [ ] Active and pinned sessions/operators revalidate the exact manifest before use.
|
||
- [ ] Unrelated valid workspace namespaces coexist and are neither rejected nor read/modified; caller-controlled mismatch attempts cannot escape the requested namespace.
|
||
- [ ] P3 revision roots and effective DWH binding are reused; no global annotations pointer exists.
|
||
- [ ] `schema export-fks --run` resolves exactly one active workspace and exports/re-hashes the paused run's state-owned candidate; it never reruns suggestion.
|
||
- [ ] Run resolution is bounded/no-follow/ambiguity-safe and rejects traversal, duplicates, removed workspaces, links, and embedded identity drift with stable safe codes.
|
||
- [ ] Candidate export goes to an ordinary author clone; application code never pushes curated bytes.
|
||
- [ ] The only V2 checkpoint states are `pending` and `accepted`; every exact V1 row and legal transition is tested.
|
||
- [ ] Explicit accept records candidate digest, current blob ID/digest, new revision, and DWH binding.
|
||
- [ ] Curator explicitly runs P3 `workspace registry pull --workspace <resolved-workspace-id> --json` after push; only `RegistryPullCommand` / `registry_pull` has writable registry storage and selected exact Git transport, while DWH migration, accept, and ordinary mutations remain registry RO/no-Git.
|
||
- [ ] After every pull and before every accept, exact released `workspace migrate dwh-cache --workspace "$WORKSPACE_ID" --json` completes; its `p3_migrate_dwh_cache`/`p3_prepare_dwh_cache` plus `p3_materialize_dwh_snapshot` stages publish and strictly reverify the selected unready revision's binding-qualified physical/LSH snapshot without READY or prior-revision fallback, and `pre_ready_dwh_snapshot_publication` is present exactly once.
|
||
- [ ] The one `AnnotationSynchronizer` uses exact `futureWorkspaceLayoutPaths(rootLease, workspaceId, workspaceRevision)` for repository pull and pre-READY accept. Admission/resume are explicitly tested as `migration_required` after pull, after DWH snapshot preparation, and after accept; session/pinned runtime alone requires the strict committed READY state and exact four-argument `workspaceRuntimePaths(rootLease, workspaceId, workspaceRevision, state)`.
|
||
- [ ] After every successful accept and before every resume, the exact P3 `workspace migrate activate-revision-layout --workspace <id> --yes [--resume <activation-run-id>] --json` operation completes and the accepted commit+effective-binding READY is strictly verified; success occurs only afterward, and both refusal/activation report check IDs are present.
|
||
- [ ] `registry_pull` uses only `publishAddressed`: complete strict-lexical `OrderedWorkspaceWriterCapabilitySet` acquisition, exact participant leases, active-pointer write/file-fsync/rename/parent-fsync while the full set remains held, `terminal_durable`, reverse set release, then repository release. Accept uses active-snapshot-read→writer/revalidation only; controlled races prove no inversion, reentry, deadlock, or repository/Git call from accept.
|
||
- [ ] Real-process dependency tests use multiple reverse-presented changed IDs and exact `RegistryPullJobStateV1` fields/phases; same-ID SIGKILL before active rename and after rename+parent fsync resumes without refetch/reentry (and without a second post-publication rename), while target drift/third identity refuses without mutation or owner clear.
|
||
- [ ] The final sole `WorkspaceLockedChildRequest` alias is cumulative: the original three P2 members, unchanged fourteen-member `P3LockedChildRequest`, and both P5 annotation members. A bidirectional compile-only exact-kind fence and a table-driven runtime `assertNever` regression send all nineteen variants through the same `WorkspaceWriterLockCapability.spawnChild`; the P3 locked-child focused gates remain in the P5 release set.
|
||
- [ ] Every annotation publish/verify uses that sole cumulative `WorkspaceLockedChildRequest` union and `writerCapability.spawnChild`, validates actual FD 3 plus retained-root FD 4 before reads/writes, enforces fixed stdin/result caps, and has real barrier replacement, missing/substituted/cross-root, direct-spawn, and post-settlement tests.
|
||
- [ ] Every P5 Git stage uses shared `runFixedGitChild` with one monotonic operation deadline, bounded stdout/record/stderr, parser/cancel abort, owned-group TERM→KILL, and awaited exit/stdio cleanup.
|
||
- [ ] Resume refuses any later blob/revision/binding mismatch before schema/Evidence stages.
|
||
- [ ] `physical.yaml` remains local and absent from Git/export.
|
||
- [ ] Clean-state goal passes once without retry; its exact retained report output yields a validated 32-hex `RUN_ID`, every retained path use is quoted, and the report is hash-bound.
|
||
- [ ] Secret scan and exact owned-resource cleanup pass.
|
||
- [ ] Manual P5 guide is runnable and still PENDING until human decision.
|
||
- [ ] No P6 Evidence materialization, retention, or aggregate scope was implemented.
|