mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
e9613767c5e4e30ecb0fe2b135d1ffca483a7c77
ThothII/docs/superpowers/plans/2026-08-10-p4-qdrant-bootstrap-rebuild.md
T

56 KiB
Raw Blame History

P4 Qdrant Bootstrap and Guarded Rebuild Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Implement PRD D4 so session admission safely self-heals a missing workspace Qdrant collection or missing keyword indexes, while thothctl can inspect, destructively rebuild, and recover only the descriptor-owned collection under durable maintenance and complete quiescence.

Architecture: Move Qdrant collection inspection/reconciliation out of ThtRunner into one shared TypeScript manager used by session admission and P2's WorkspacePreprocessingService.execute. The manager has safe readiness reconciliation plus narrow destructive primitives—never a public monolithic rebuild()—so the guarded service can durably record every mutation boundary. Readiness may create only the fixed 1024/cosine collection contract and missing keyword indexes; incompatible vector or index types remain fail-closed. Destructive rebuild is a host-orchestrated transaction: thothctl holds one installation lifecycle lock, activates the backend's durable admission barrier, proves the complete session inventory and live process/admission counts are quiescent, stops core, and runs the dedicated Pi-free backend/src/workspace-maintenance.ts::main entrypoint under P2's writer lock. Success persists deleting before DELETE, deleted after a confirmed 404, recreated after exact collection/index creation, and verified only after a separate final inspection; a post-delete failure leaves maintenance active for explicitly confirmed recovery.

Tech Stack: TypeScript 5, Node.js 22 fetch, Fastify 5, Vitest, Qdrant REST API v1.18, Go 1.26, gofrs/flock, Docker Compose, Bash, real Git fixtures, and JSON/YAML.

Source, prerequisites, and hard boundary

  • Source requirement: docs/prd/2026-08-09-workspace-preprocessing-prd.md D4, RF4.2–RF4.4, RNF1–RNF9, and section 8.
  • Reviewed design: docs/superpowers/specs/2026-08-10-p2-p6-workspace-preprocessing-design.md section 6 and the common error/verification contracts.
  • P1 is already accepted. Execute and checkpoint P2, then P3, before this plan because this implementation deliberately extends, rather than duplicates, P2's exact maintenance surface:
    • backend/src/workspaces/{runtime-config-lease.ts,preprocessing-state.ts,preprocessing-service.ts} and backend/src/workspace-maintenance.ts;
    • tools/thothctl/internal/workspaceops/operations.go;
    • profile-only workspace-maintenance in compose.yaml;
    • /data/sessions/<workspace-id>/preprocessing/writer.lock and runUnderWorkspaceWriterLock(...).
  • This plan is rebased on P2/P3's frozen handoff: backend/src/workspace-maintenance.ts::main, WorkspacePreprocessingService.execute, PreprocessingStateStore, WorkspaceRuntimeConfigLeaseFactory, backend/src/workspaces/revision-layout.ts::{workspaceRuntimePaths,readRevisionLayoutState}, runUnderWorkspaceWriterLock / probeWorkspaceWriterLock, and tools/thothctl/internal/workspaceops::{ParseWorkspaceCommand,Run}. Run the recorded P2/P3 focused gates before Task 1. Do not create a second operator, renderer, state root, lock, or internal/workspace package.
  • P4 does not implement P5 Git annotations, P6 Evidence materialization, P7 migration, P8 aggregate smoke, P9 retention policy, P10 SSH runtime support, a GUI maintenance endpoint, or automatic semantic reindexing after rebuild.
  • Qdrant remains derived data. Rebuild intentionally discards the selected collection's schema/Evidence/Memory projection. Canonical Git descriptors, phase artifacts, corpus state, and the memory registry remain untouched; operators must rerun the already-available indexing commands afterward.
  • No prefix matching, collection enumeration followed by bulk deletion, Qdrant-wide cleanup, forced Pi termination, automatic session closure, or automatic retry is permitted.

Exact public command and result contract

P2's read-only command gains a semantic_index and collection_recovery section:

thothctl --installation /abs/thothii-installation.yaml \
  workspace inspect --workspace research --json

P4 adds only these destructive commands:

thothctl --installation /abs/thothii-installation.yaml \
  workspace collection rebuild \
  --workspace research \
  --confirm-workspace research \
  --confirm-collection research-semantic \
  --destructive --json

thothctl --installation /abs/thothii-installation.yaml \
  workspace collection recover \
  --workspace research \
  --confirm-workspace research \
  --confirm-collection research-semantic \
  --destructive --json

Rules:

  1. --workspace, --confirm-workspace, --confirm-collection, and the literal --destructive are all required once and only once for rebuild/recover. Values are compared byte-for-byte after the normal identifier syntax validation; no case folding, trimming, defaults, interactive prompts, or --yes alias.
  2. The collection is always re-derived from the active, operational schema-v3 descriptor. A caller-supplied collection is never used as a mutation target until it exactly equals that value.
  3. Confirmation mismatch exits 2 before maintenance activation, lifecycle state creation, collection mutation, or stopping core.
  4. inspect is read-only and may run concurrently. It reports missing, compatible, repairable (only keyword indexes are absent), or incompatible, plus safe expected/observed dimensions, distance, required index names/types, active revision, maintenance state, and recovery phase. It never self-heals.
  5. Machine output is one pristine JSON object. Stable P4 codes are semantic_index_incompatible, preprocessing_conflict, collection_confirmation_mismatch, collection_recovery_required, collection_recovery_not_required, session_inventory_active, maintenance_not_quiescent, and workspace_not_activatable. No response includes a Qdrant response body, arbitrary exception, raw child stderr, endpoint, credential, signed URL, or secret-file content.

The fixed collection contract remains:

export const REQUIRED_QDRANT_KEYWORD_INDEXES = [
  "content_hash",
  "document_id",
  "kind",
  "record_key",
  "record_kind",
  "vector_generation",
  "workspace_id",
  "workspace_revision",
] as const;

Vectors are exactly size 1024, distance Cosine; every listed payload index is exactly keyword.

Durable transaction and fail-safe matrix

Store the collection transaction at:

/data/sessions/<workspace-id>/preprocessing/qdrant-rebuild-state.json

Use strict schema version 1:

interface CollectionRebuildStateV1 {
  version: 1;
  transaction_id: string;
  operation: "collection-rebuild";
  workspace_id: string;
  workspace_revision: string;       // exact 40-hex active commit
  collection: string;               // descriptor-derived exact name
  phase: "prepared" | "deleting" | "deleted" | "recreated" | "verified" | "failed_pre_delete";
  mutation_started: boolean;
  created_at: string;
  updated_at: string;
  error_code?: "semantic_index_incompatible" | "workspace_not_activatable";
}

Write prepared durably before DELETE and write deleting with mutation_started: true durably before issuing DELETE. Therefore a crash with mutation_started: true is always treated as potentially destructive even when Qdrant still contains the collection. Atomic replace must fsync the file and owning directory, use regular no-follow 0600 files under the P2-owned preprocessing root, reject unknown keys/versions/identity drift, and never interpolate an exception into error_code.

  • Failure before mutation_started may mark failed_pre_delete, restart core, verify health, and clear maintenance.
  • Failure or ambiguous subprocess output after mutation_started leaves core stopped and the durable maintenance marker active. The CLI prints the exact recover command using only safe workspace/collection identifiers.
  • Recovery accepts only a nonterminal state matching the current descriptor workspace, revision, and collection. If the collection is missing it recreates it; if compatible it verifies it (covering a crash after recreate); if incompatible it may delete/recreate the exact same collection only because recovery repeats all destructive confirmations and the transaction proves mutation already began. It never targets a different or prefix-matched collection.
  • verified is retained for inspection/audit. A later rebuild may atomically supersede it only after proving it is terminal.

Automated P4 process goal

Start the execution goal before Task 1 and complete it only after this single clean-state command succeeds:

./scripts/p4-acceptance.sh integration --keep

The command must refuse a dirty tracked source tree, bind the run to the exact Git commit/tree, use one unique Compose project and ownership manifest below .artifacts/p4-integration/<run-id>/, execute once with no automatic retry, and retain report.json, report.md, bounded command events, safe Qdrant observations, recovery state copies, artifact hashes, and cleanup proof. External LLM/DWH/Ollama behavior is outside D4 and uses fixture-only values; Qdrant, the compiled core image, backend routes, thothctl, Git registry snapshot, dedicated maintenance service, persistent volumes, and Compose lifecycle are real.

The integration must prove: missing collection admission self-heal; missing-index self-heal; concurrent compatible creation/index reconciliation; incompatible dimension/distance/index refusal without PUT/DELETE; exact confirmation refusal before maintenance; complete open-session refusal; live admission/Pi count quiescence; held P2 writer-lock refusal before deletion; exact target-only deletion with a neighbor collection unchanged; verified rebuild/core restart/maintenance clear; a deterministic interruption after deletion with core stopped and marker retained; explicit recovery; secret scan; no retry; and exact owned-resource cleanup. --keep retains filesystem evidence, never live containers/networks/volumes or secrets.

No unavoidable human action exists inside this automated goal. Manual acceptance happens afterward in a new independent environment.

Task 0: Establish the execution baseline and persistent goal

Files:

  • Read: docs/superpowers/plans/2026-08-10-p2-host-workspace-preprocessing-cli.md

  • Read: docs/superpowers/plans/2026-08-10-p3-effective-config-fingerprint.md

  • Read: backend/src/workspaces/{runtime-config-lease.ts,revision-layout.ts,preprocessing-state.ts,preprocessing-service.ts} and backend/src/workspace-maintenance.ts

  • Read: tools/thothctl/internal/workspaceops/operations.go

  • Read: compose.yaml, deploy/compose.local.yaml, deploy/compose.server.yaml

  • Step 1: Confirm the worktree and prerequisites

git status --short
git log -1 --format='%H %T'
test -f backend/src/workspaces/preprocessing-state.ts
test -f tools/thothctl/internal/workspaceops/operations.go

Expected: clean output from git status; one commit/tree line; both files exist. Confirm P2 and P3 checkpoint reports say automated integration PASS. If not, stop—do not fold their scope into P4.

  • Step 2: Start the process goal

If the execution environment supports persistent goals, create: “P4 clean-state Qdrant bootstrap/rebuild/recovery process passes once without retry, retains a secret-clean report, and cleans only owned resources.” Keep it open through Task 11.

  • Step 3: Run prerequisite focused gates

Run the exact P2/P3 focused verification commands recorded in their checkpoint reports.

Expected: PASS. A failure is prerequisite drift; fix it in its owning plan before P4.

No commit.

Task 1: Extract the shared Qdrant collection manager

Files:

  • Create: backend/src/semantic/qdrant-collection-manager.ts
  • Create: backend/test/qdrant-collection-manager.test.ts
  • Modify: backend/src/tht/tht-runner.ts (QdrantEnsureResult, REQUIRED_QDRANT_PAYLOAD_INDEXES, qdrantEnsure)
  • Modify: backend/test/tht-qdrant-readiness.test.ts

Required interface:

export type CollectionState = "missing" | "compatible" | "repairable" | "incompatible";
export interface QdrantCollectionSpec {
  collection: string;
  dimensions: 1024;
  distance: "cosine";
  keywordIndexes: readonly string[];
}
export interface CollectionInspection {
  state: CollectionState;
  expected: { dimensions: 1024; distance: "cosine"; keyword_indexes: readonly string[] };
  observed?: { dimensions?: number; distance?: string; keyword_indexes: Record<string, string> };
  missing_keyword_indexes: string[];
  incompatible_fields: string[];
}
export class QdrantCollectionManager {
  constructor(options: { baseUrl: string; request?: typeof fetch });
  inspect(spec: QdrantCollectionSpec, timeoutMs: number): Promise<CollectionInspection>;
  ensure(spec: QdrantCollectionSpec, timeoutMs: number): Promise<{ ok: boolean; code?: SemanticReadinessCode }>;
  deleteExactAndConfirmMissing(spec: QdrantCollectionSpec, timeoutMs: number): Promise<void>;
  ensureCollection(spec: QdrantCollectionSpec, timeoutMs: number): Promise<void>;
  ensureIndexes(spec: QdrantCollectionSpec, timeoutMs: number): Promise<void>;
}
export function qdrantCollectionSpec(workspace: WorkspaceDescriptor): QdrantCollectionSpec;

There is deliberately no exported rebuild() and no manager method that can cross more than one durable destructive phase. deleteExactAndConfirmMissing issues one exact DELETE and succeeds only after a separate GET confirms 404. ensureCollection creates or converges only the exact 1024/Cosine collection and finishes with a read proving the vector contract (keyword indexes may still be absent). ensureIndexes creates only missing required keyword indexes in canonical order and finishes with a read proving the exact complete contract. The guarded service in Task 3 is the sole production composer of these primitives; readiness calls only ensure.

  • Step 1: Write RED inspection tests

Use a scripted vi.fn fetch boundary. Cover exact URL encoding, GET 404 → missing, exact 1024/Cosine/eight keyword indexes → compatible, missing index → repairable, and wrong size/distance/index type → incompatible. Reject malformed success JSON as workspace_not_activatable; never propagate a body canary.

  • Step 2: Verify RED
cd backend
npx vitest run test/qdrant-collection-manager.test.ts

Expected: FAIL because the module is missing.

  • Step 3: Implement read-only inspection

Use only new URL('/collections/' + encodeURIComponent(name), baseUrl), an operation-wide AbortController, allowlisted parsed fields, and lowercase comparison for Qdrant's distance/type response. Distinguish missing from incompatible; an unavailable/unparseable service is not semantic incompatibility.

  • Step 4: Write RED self-heal/concurrency tests

Cover these exact request sequences:

  1. GET 404 → PUT collection with {"vectors":{"size":1024,"distance":"Cosine"}} → final GET compatible.
  2. GET 404 → PUT 409/already exists → final GET compatible (concurrent compatible creator).
  3. GET repairable → PUT /collections/<name>/index?wait=true with {"field_name":"kind","field_schema":"keyword"} → final GET compatible.
  4. Index PUT conflict → final GET compatible (concurrent compatible index creator).
  5. A barrier-controlled unit race launches two independent manager.ensure calls (not a shared promise): both initial GETs observe 404/repairable, one PUT succeeds, the other receives the allowed conflict, both perform their own final GET, and both converge compatible. Repeat for collection creation and one missing index; assert no retry loop.
  6. Concurrent creator/index ends incompatible → semantic_index_incompatible.
  7. Existing incompatible size/distance/index performs no PUT or DELETE.
  8. A missing-index run never recreates the collection and creates only missing indexes in canonical order.
  9. Timeout/unreachable response returns workspace_not_activatable with no endpoint/body leak.
  10. deleteExactAndConfirmMissing issues DELETE for the one URL-encoded exact collection and then independently reads 404; DELETE failure or a non-404 post-delete state fails.
  11. ensureCollection and ensureIndexes are separately observable: collection creation ends with the exact vector contract present, index creation ends with the exact complete contract, and neither ever deletes. A later independent inspect supplies final verification. No other collection name, list endpoint, prefix, or global mutation is ever requested.
  12. No exported rebuild, callback that hides multiple phases, or test-only destructive shortcut exists.
  • Step 5: Verify RED, then implement minimal reconciliation
cd backend
npx vitest run test/qdrant-collection-manager.test.ts

Expected before implementation: FAIL on PUT sequences. Implement create/index calls, tolerate only conflict/already-exists as a reason to re-read, and always decide success from one final GET. Do not accept the mutating response as proof.

  • Step 6: Delegate ThtRunner.qdrantEnsure

Keep descriptor operational validation in qdrantEnsure, derive qdrantCollectionSpec, and call the shared manager. Remove the duplicate constant/parsing logic. Preserve ThtConfig.qdrantRequest as the injectable fetch boundary.

  • Step 7: Run focused tests and typecheck
cd backend
npx vitest run test/qdrant-collection-manager.test.ts test/tht-qdrant-readiness.test.ts test/readiness-manager.test.ts
npx tsc --noEmit -p .

Expected: PASS. Update the old missing-collection assertion from incompatibility to success after the exact create/final-read sequence; existing incompatible cases remain fail-closed.

  • Step 8: Commit
git add backend/src/semantic/qdrant-collection-manager.ts \
  backend/src/tht/tht-runner.ts \
  backend/test/qdrant-collection-manager.test.ts \
  backend/test/tht-qdrant-readiness.test.ts \
  backend/test/readiness-manager.test.ts
git commit -m "feat: self-heal workspace qdrant collections"

Task 2: Prove session admission uses self-heal before Ollama and persistence

Files:

  • Modify: backend/test/routes-sessions.test.ts (readiness cases around current Qdrant tests)

  • Modify if dependency injection requires it: backend/src/app.ts (ThtRunner construction only)

  • Step 1: Write RED route tests

At the real buildApp boundary, inject a ThtRunner/fetch script and assert:

  • missing collection creates the exact contract, then Ollama runs, then session persistence is allowed;
  • missing index creates only that index before Ollama;
  • incompatible collection returns 503 with semantic_index_incompatible, does not call Ollama, does not call sessionNew, acquire a revision lease permanently, or spawn Pi;
  • two simultaneous local-mode, same-principal, same-workspace admissions hit ReadinessManager's in-flight key and share one promise: exactly one Qdrant create/reconciliation occurs, both callers receive the same compatible result, and no false incompatibility is emitted;
  • one separate ReadinessManager unit test proves the same-principal dedup key and cleanup after resolution/rejection;
  • one upstream-auth, two-distinct-principal route test uses distinct readiness keys, coordinates both initial GET 404 reads, lets one PUT create and the other PUT receive the compatible-creator conflict, and proves both converge after their final reads. Do not describe this as a local same-principal race;
  • raw Qdrant body/endpoint canaries do not appear in HTTP JSON or logs captured by the test.

Keep the compatible-creator and compatible-index conflict sequences in qdrant-collection-manager.test.ts as the direct manager race proof. Route tests prove the real dedup/cross-principal semantics rather than attempting to bypass ReadinessManager.

  • Step 2: Run and verify RED
cd backend
npx vitest run test/routes-sessions.test.ts -t "Qdrant|semantic index|concurrent collection"

Expected: the new call-order, local deduplication, and distinct-principal convergence assertions FAIL until route fixtures use the real manager behavior and upstream identities produce distinct readiness keys.

  • Step 3: Make the minimal wiring change

Do not add a new route. Session admission continues through ReadinessManager.ensure(..., descriptor) and ThtRunner.qdrantEnsure; change only construction/injection necessary to share the manager.

  • Step 4: Verify
cd backend
npx vitest run \
  test/routes-sessions.test.ts \
  test/readiness-manager.test.ts \
  test/tht-qdrant-readiness.test.ts
npx tsc --noEmit -p .

Expected: PASS, including Qdrant-before-Ollama ordering, exactly one create for local same-principal deduplication, two independently convergent manager calls for distinct upstream principals, and no persistence on incompatibility.

  • Step 5: Commit
git add backend/src/app.ts backend/test/routes-sessions.test.ts backend/test/readiness-manager.test.ts
git commit -m "test: prove qdrant admission self-heal"

Omit unchanged paths from git add.

Task 3: Add safe inspection and durable rebuild/recovery to the P2 operator

Files:

  • Create: backend/src/workspaces/qdrant-rebuild-state.ts
  • Create: backend/test/qdrant-rebuild-state.test.ts
  • Modify: backend/src/workspaces/preprocessing-service.ts (MaintenanceCommand/result union)
  • Modify: backend/src/workspaces/preprocessing-service.ts (WorkspacePreprocessingService.execute)
  • Modify: backend/src/workspace-maintenance.ts (main parser/encoder)
  • Modify: backend/src/workspaces/preprocessing-state.ts
  • Modify: backend/test/workspace-preprocessing-service.test.ts
  • Modify: backend/test/workspace-maintenance.test.ts
  • Modify: backend/test/workspace-preprocessing-state.test.ts

Operator additions:

type MaintenanceCommand =
  | /* P2/P3 requests */
  | { operation: "collection-rebuild"; workspace_id: string; confirm_workspace: string;
      confirm_collection: string; destructive: true }
  | { operation: "collection-recover"; workspace_id: string; confirm_workspace: string;
      confirm_collection: string; destructive: true };

inspect gains safe semantic_index and collection_recovery; it never acquires a writer lock or mutates Qdrant. Rebuild/recover must call runUnderWorkspaceWriterLock for the complete state-read → mutation → verification transaction. A conflict maps to preprocessing_conflict before state or Qdrant mutation.

  • Step 1: Write RED state-store tests

Test restrictive creation, atomic durable transitions, same-inode/root safety, unknown schema/keys, workspace/revision/collection mismatch, symlink/path swap, injected file-fsync and directory-fsync ambiguity, and terminal supersession. Table-test the only legal forward edges prepared → deleting → deleted → recreated → verified plus prepared → failed_pre_delete; reject skips, rewinds, cross-transaction updates, and failed_pre_delete after mutation. Add crash-visible assertions at every boundary: deleting/mutation_started is durable before mocked DELETE, deleted only after the manager confirms 404, recreated only after both exact collection and required indexes are confirmed, and verified only after one additional independent inspect returns compatible.

  • Step 2: Verify RED
cd backend
npx vitest run test/qdrant-rebuild-state.test.ts

Expected: FAIL because CollectionRebuildStateStore does not exist.

  • Step 3: Implement the strict store

Reuse P2's trusted preprocessing-root and atomic state primitives rather than another root policy. Expose only read, begin, and transition(expectedTransaction, next); enforce legal monotonic phase transitions. Report durability uncertainty as recovery-required whenever the intended bytes may have reached the canonical path.

  • Step 4: Write RED operator tests

Cover:

  • inspect returns active workspace/revision, exact descriptor collection, manager inspection, and safe recovery state without mutation/lock;

  • confirmation mismatch and missing destructive: true return collection_confirmation_mismatch before marker/lock/Qdrant;

  • rebuild refuses unless /data/settings/maintenance.json is a trusted regular file whose strict JSON says active;

  • held /preprocessing/writer.lock returns preprocessing_conflict before DELETE;

  • rebuild writes/fsyncs prepared, then writes/fsyncs deleting with mutation_started:true before deleteExactAndConfirmMissing; after that primitive's confirmed 404 it writes/fsyncs deleted; after ensureCollection and ensureIndexes have separately confirmed the exact contract it writes/fsyncs recreated; only a subsequent standalone inspect returning compatible permits verified;

  • injected crashes after each durable phase leave exactly that phase visible, never infer a later phase from a mutating response, and resume through the same narrow manager primitives;

  • neighbor collections are never requested;

  • failure before deleting becomes failed_pre_delete with mutation_started: false;

  • failure/timeout at or after deleting retains the last durable nonterminal state with mutation_started: true and returns a safe recovery-required envelope;

  • recover handles every nonterminal phase: prepared may start deletion only after repeated confirmations; deleting first inspects and conservatively establishes missing/compatible/incompatible exact state; deleted recreates; recreated independently verifies; missing, already-compatible-after-crash, and explicitly confirmed incompatible exact targets all converge without skipping a durable boundary;

  • recover rejects absent/verified state, changed active revision, changed descriptor collection, and unsafe/unknown state;

  • error output excludes Qdrant body, marker contents beyond allowlisted booleans, paths, endpoints, and canaries.

  • Step 5: Run and verify RED

cd backend
npx vitest run \
  test/workspace-preprocessing-service.test.ts \
  test/workspace-preprocessing-state.test.ts \
  test/workspace-maintenance.test.ts

Expected: new operation cases FAIL because dispatch/state enforcement is absent.

  • Step 6: Implement guarded dispatch

Derive the descriptor and revision through WorkspacePreprocessingService.execute and its existing P2 resolver, validate confirmations, verify the marker through lstat/open(O_NOFOLLOW)/fstat and strict JSON, then enter runUnderWorkspaceWriterLock. Compose only inspect, deleteExactAndConfirmMissing, ensureCollection, and ensureIndexes, persisting/fsyncing the state between calls exactly as specified above. Neither backend/src/workspace-maintenance.ts::main nor any other caller receives a monolithic destructive API. The CLI accepts fixed argv generated by workspaceops.Run and preserves one pristine JSON envelope; direct manual invocation without marker/confirmations remains harmless.

  • Step 7: Verify focused backend tests
cd backend
npx vitest run \
  test/qdrant-collection-manager.test.ts \
  test/qdrant-rebuild-state.test.ts \
  test/workspace-preprocessing-state.test.ts \
  test/workspace-preprocessing-service.test.ts \
  test/workspace-maintenance.test.ts
npx tsc --noEmit -p .
npm run build

Expected: PASS; build emits /app/backend/dist/workspace-maintenance.js and the new shared manager/state module.

  • Step 8: Commit
git add backend/src/workspaces/qdrant-rebuild-state.ts \
  backend/src/workspaces/preprocessing-state.ts \
  backend/src/workspaces/preprocessing-service.ts \
  backend/src/workspace-maintenance.ts \
  backend/test/qdrant-rebuild-state.test.ts \
  backend/test/workspace-preprocessing-state.test.ts \
  backend/test/workspace-preprocessing-service.test.ts \
  backend/test/workspace-maintenance.test.ts
git commit -m "feat: add guarded qdrant maintenance operations"

Task 4: Expose loopback-only maintenance quiescence

Files:

  • Modify: backend/src/app.ts (maintenance status handlers, isMaintenanceControl)
  • Modify: backend/src/runtime/maintenance-gate.ts only if a named status type is needed
  • Modify: backend/test/routes-sessions.test.ts (current maintenance endpoint tests)
  • Modify: backend/test/maintenance-gate.test.ts only for status typing/durability regressions
  • Modify: backend/test/pi-process-manager.test.ts

Response:

{
  "active": true,
  "admissions": 0,
  "piProcesses": 0,
  "quiescent": true
}

Include recoveryRequired: true only when already produced by MaintenanceBarrier. quiescent is true only when the durable marker is active, admissions === 0, mgr.count() === 0, and no durability recovery is pending.

  • Step 1: Write RED endpoint tests

Test POST /internal/maintenance/activate, GET /internal/maintenance/status, and new GET /internal/maintenance/quiescence with injected manager counts 0/1. Prove activation waits for an in-flight admission lease, later admissions receive 503, Pi count is observational (never killed), spoofed non-loopback callers receive 403 in none and upstream auth, and durability ambiguity makes quiescent: false.

  • Step 2: Verify RED
cd backend
npx vitest run test/routes-sessions.test.ts -t "maintenance|quiescence"

Expected: FAIL because responses omit piProcesses/quiescent and the route is not allowlisted.

  • Step 3: Implement one response composer

In buildApp, define maintenanceStatus() from maintenanceBarrier.status() plus mgr.count(). Use it for activate/deactivate/status/quiescence responses so the fields cannot drift. Do not expose runtime IDs, principals, session questions, or child process details.

  • Step 4: Verify
cd backend
npx vitest run \
  test/maintenance-gate.test.ts \
  test/routes-sessions.test.ts \
  test/pi-process-manager.test.ts
npx tsc --noEmit -p .

Expected: PASS. Existing admission behavior remains unchanged except for additive internal response fields.

  • Step 5: Commit
git add backend/src/app.ts backend/src/runtime/maintenance-gate.ts \
  backend/test/routes-sessions.test.ts backend/test/maintenance-gate.test.ts \
  backend/test/pi-process-manager.test.ts
git commit -m "feat: report complete maintenance quiescence"

Task 5: Share one installation lifecycle lock with Pi operations

Files:

  • Create: tools/thothctl/internal/lifecycle/lock.go
  • Create: tools/thothctl/internal/lifecycle/lock_test.go
  • Modify: tools/thothctl/internal/config/installation.go (LifecycleLockPath, CollectionRebuildStatePath only if host metadata is retained)
  • Modify: tools/thothctl/internal/config/installation_test.go
  • Modify: tools/thothctl/internal/pi/state.go (acquireLock, updateLock, ErrLockHeld removal/delegation)
  • Modify: tools/thothctl/internal/pi/update.go (Update, Rollback, RecoverMaintenance lock acquisition)
  • Modify: tools/thothctl/internal/pi/update_test.go
  • Modify: tools/thothctl/internal/pi/state_test.go

Path: .thothctl/<project-name>/lifecycle.lock, with owner metadata .thothctl/<project-name>/lifecycle.lock.owner.json.

  • Step 1: Write RED lifecycle tests

Test stable 0600 lock inode, 0600 atomic owner metadata, nonblocking cross-process contention, metadata removal only by the owner on release, crash-safe kernel release, symlink/unsafe control-directory rejection, and distinct installation descriptors not contending. Add a test that a Pi update lock blocks a simulated collection rebuild and vice versa.

  • Step 2: Verify RED
cd tools/thothctl
go test ./internal/lifecycle ./internal/config

Expected: FAIL because the lifecycle package/path does not exist.

  • Step 3: Extract the current lock without changing Pi transaction semantics

Move the gofrs/flock implementation from internal/pi/state.go into lifecycle.Acquire(path, operation). Owner JSON contains PID, host, start time, transaction, and an allowlisted operation (pi-update, pi-rollback, pi-maintenance-recover, qdrant-rebuild, qdrant-recover), never argv/environment. Add LifecycleLockPath string to pi.Request; change Rollback and RecoverMaintenance to accept (statePath, lifecycleLockPath, confirm); and make main.go pass installation.LifecycleLockPath() to all three paths. Reject an empty/noncanonical lock path. The update state remains UpdateStatePath(); only mutual exclusion moves from update-state.json.lock to the installation lock.

  • Step 4: Run Pi regressions
cd tools/thothctl
go test ./internal/lifecycle ./internal/config ./internal/pi

Expected: PASS; interrupted update/rollback behavior and recovery metadata remain identical, but P4 and Pi now contend on one installation lock.

  • Step 5: Commit
git add tools/thothctl/internal/lifecycle \
  tools/thothctl/internal/config/installation.go \
  tools/thothctl/internal/config/installation_test.go \
  tools/thothctl/internal/pi/state.go tools/thothctl/internal/pi/state_test.go \
  tools/thothctl/internal/pi/update.go tools/thothctl/internal/pi/update_test.go
git commit -m "refactor: share installation lifecycle lock"

Task 6: Parse exact collection commands and confirmations in thothctl

Files:

  • Modify: tools/thothctl/internal/workspaceops/operations.go

  • Modify: tools/thothctl/internal/workspaceops/operations_test.go

  • Create: tools/thothctl/internal/workspaceops/collection.go

  • Create: tools/thothctl/internal/workspaceops/collection_test.go

  • Modify: tools/thothctl/cmd/thothctl/main.go

  • Modify: tools/thothctl/cmd/thothctl/main_test.go

  • Step 1: Write RED parse tests

Table-test the exact commands above. Reject missing/duplicate flags, swapped confirmations, whitespace/case differences, unknown flags/subcommands, positional targets, --yes, absent --destructive, unsafe workspace/collection identifiers, and rebuild/recover input or output file flags inherited from P2. Confirm usage errors exit 2 and never call Docker.

  • Step 2: Verify RED
cd tools/thothctl
go test ./internal/workspaceops ./cmd/thothctl -run 'Collection|collection|Confirmation'

Expected: FAIL because P2 parsing knows no workspace collection family.

  • Step 3: Extend workspaceops.ParseWorkspaceCommand and main dispatch

Use typed operations CollectionRebuild and CollectionRecover; do not add another top-level parser. Perform syntactic equality checks in Go before Docker, then pass all values to TypeScript for authoritative descriptor equality checks.

  • Step 4: Extend inspect parsing/output

P2 workspace inspect keeps the same command syntax. Validate the returned semantic/recovery JSON structure and print pristine JSON under --json; human output lists exact workspace, revision, collection, state, missing/incompatible fields, maintenance, and recovery phase without endpoint details.

  • Step 5: Verify
cd tools/thothctl
go test ./internal/workspaceops ./cmd/thothctl

Expected: PASS.

  • Step 6: Commit
git add tools/thothctl/internal/workspaceops/operations.go \
  tools/thothctl/internal/workspaceops/operations_test.go \
  tools/thothctl/internal/workspaceops/collection.go \
  tools/thothctl/internal/workspaceops/collection_test.go \
  tools/thothctl/cmd/thothctl/main.go tools/thothctl/cmd/thothctl/main_test.go
git commit -m "feat: add qdrant collection operator commands"

Task 7: Orchestrate maintenance, inventory, stop, rebuild, restart, and recovery

Files:

  • Modify: tools/thothctl/internal/workspaceops/collection.go
  • Modify: tools/thothctl/internal/workspaceops/collection_test.go
  • Modify: tools/thothctl/internal/workspaceops/operations.go
  • Modify: tools/thothctl/internal/workspaceops/operations_test.go
  • Modify: tools/thothctl/cmd/thothctl/main.go
  • Modify: tools/thothctl/cmd/thothctl/main_test.go
  • Modify: tools/thothctl/internal/pi/update.go only to share strict maintenance parsing/helpers where appropriate
  • Modify: tools/thothctl/internal/pi/update_test.go

Required sequence for rebuild:

Acquire installation lifecycle lock
→ workspace inspect and authoritative confirmation recheck
→ POST core /internal/maintenance/activate
→ GET complete /sessions?scope=mine(local)|all(server)
→ reject unless every item is archived OR status closed/finalized
→ poll /internal/maintenance/quiescence until active + admissions=0 + piProcesses=0
→ compose stop core
→ compose ps --status running -q core must be empty
→ compose --profile workspace-maintenance run --rm --no-deps -T
     workspace-maintenance collection-rebuild <fixed argv>
→ parse exact safe JSON and require verified state
→ compose up --detach --no-deps core
→ poll Compose health and loopback /health
→ require maintenance still active and quiescent
→ POST /internal/maintenance/deactivate
→ require active=false
→ release lifecycle lock

Use a bounded monotonic timeout (document 60 seconds for quiescence and 120 seconds for core health), one-second polling, and no retry of a failed operation. Polling observation is not retrying a mutation.

  • Step 1: Write RED orchestration tests with a scripted runner

Assert exact argv/order and failure behavior for:

  • confirmation mismatch before lock/HTTP;

  • lifecycle contention with Pi;

  • activation durability failure;

  • malformed/incomplete/duplicate session inventory;

  • open/failed unarchived session refusal; archived, closed, and finalized acceptance;

  • admissions > 0 and piProcesses > 0 poll, then success;

  • timeout without forced teardown;

  • core stop failure or still-running proof;

  • P2 writer-lock conflict returned by the maintenance service before deletion, followed by safe core restart/health/maintenance clear;

  • successful operator verified response, exact core restart, health, and marker clear;

  • malformed/ambiguous operator result treated as mutation-started/recovery-required unless a trusted response proves mutation_started:false;

  • post-delete error leaves core stopped, marker active, state retained, and prints exact recover command;

  • neighbor services/collections never appear in stop/delete argv;

  • all child stderr is sanitized and bounded.

  • Step 2: Verify RED

cd tools/thothctl
go test ./internal/workspaceops -run 'Rebuild|Quiescence|Inventory|Recovery'

Expected: FAIL because lifecycle orchestration is not implemented.

  • Step 3: Implement strict inventory and quiescence clients

Reuse the existing core-side curl identity headers. Decode exactly one bounded JSON document. A complete local install uses scope=mine; server uses admin scope=all. Never infer quiescence only from manifests: both admissions and piProcesses must reach zero after the durable marker is active.

  • Step 4: Implement core stop/start proof and fail-safe cleanup

Use fixed Compose argv and explicit core. Do not call down, stop Qdrant, stop frontend, use --remove-orphans, or prune. Clear maintenance only after a trusted pre-delete failure or complete verified success and healthy core.

  • Step 5: Write RED recovery tests

Test core-already-stopped recovery, core-running-but-maintained recovery, missing marker refusal, no/nonterminal/verified state, descriptor revision drift, recovery service failure, compatible-after-crash verification, missing recreate, explicitly confirmed incompatible recreate, successful restart/health/deactivate, and repeated recover returning collection_recovery_not_required without mutation.

  • Step 6: Implement recovery

When core is stopped, the durable marker plus matching nonterminal state is the admission barrier; the maintenance service re-verifies both before mutation. When core is running, repeat activation/inventory/quiescence before stopping it. Never clear the marker merely because recovery cannot read state.

  • Step 7: Run Go suites
cd tools/thothctl
go test ./internal/workspaceops ./internal/lifecycle ./internal/pi ./cmd/thothctl
go test ./...

Expected: PASS.

  • Step 8: Commit
git add tools/thothctl/internal/workspaceops/collection.go \
  tools/thothctl/internal/workspaceops/collection_test.go \
  tools/thothctl/internal/workspaceops/operations.go \
  tools/thothctl/internal/workspaceops/operations_test.go \
  tools/thothctl/internal/pi/update.go tools/thothctl/internal/pi/update_test.go \
  tools/thothctl/cmd/thothctl/main.go tools/thothctl/cmd/thothctl/main_test.go
git commit -m "feat: orchestrate recoverable qdrant rebuilds"

Task 8: Harden the dedicated maintenance service and Compose contract

Files:

  • Modify: compose.yaml (workspace-maintenance only)
  • Modify: deploy/compose.server.yaml (workspace-maintenance bind roots)
  • Verify/modify as P2 created them: deploy/compose.git-https.yaml, deploy/compose.git-ssh.yaml
  • Modify: docker/core.Dockerfile (ensure /usr/bin/flock from util-linux is present)
  • Modify: scripts/test-internal-semantic-compose.sh
  • Modify: scripts/test-unified-compose.sh
  • Modify: scripts/test-compose-secret-policy.sh
  • Modify: scripts/test-no-deployment-coupling.sh
  • Modify: scripts/test-deployment-command-contract.sh

The service gains the shared settings mount needed to read the durable maintenance marker. It retains P2's sessions/registry mounts, internal Qdrant network, same selected core image, and direct Node entrypoint. It must still have no /home/thoth/.pi, PI auth/models/settings, writable Pi state, frontend dependency, host Docker socket, published port, or normal core entrypoint/trust initialization.

  • Step 1: Extend RED Compose contract tests

Assert local named volumes and server bind roots make the same /data/settings, /data/sessions, and /data/workspace-registry visible to core and workspace-maintenance; Qdrant is reachable only on the private network; only the exact required Git/connector secret files attach; service is profile-only; flock exists in the built core image; and service cannot start Pi or expose HTTP.

  • Step 2: Verify RED
./scripts/test-internal-semantic-compose.sh
./scripts/test-unified-compose.sh
./scripts/test-compose-secret-policy.sh
./scripts/test-deployment-command-contract.sh

Expected: at least the settings/marker and flock assertions FAIL.

  • Step 3: Make minimal Compose/image changes

Add util-linux explicitly rather than relying on a transitive base package. Extend the P2 service only; do not introduce a second maintenance service or mount Pi state.

  • Step 4: Run deployment gates
./scripts/test-default-compose.sh
./scripts/test-unified-compose.sh
./scripts/test-internal-semantic-compose.sh
./scripts/test-compose-secret-policy.sh
./scripts/test-no-deployment-coupling.sh
./scripts/test-deployment-command-contract.sh

Expected: PASS.

  • Step 5: Commit
git add compose.yaml deploy/compose.server.yaml \
  deploy/compose.git-https.yaml deploy/compose.git-ssh.yaml \
  docker/core.Dockerfile scripts/test-internal-semantic-compose.sh \
  scripts/test-unified-compose.sh scripts/test-compose-secret-policy.sh \
  scripts/test-no-deployment-coupling.sh scripts/test-deployment-command-contract.sh
git commit -m "build: isolate qdrant maintenance service"

Task 9: Build the clean-state P4 process goal and retained report

Files:

  • Create: scripts/p4-acceptance.sh
  • Create: scripts/test-p4-acceptance.sh
  • Create: backend/scripts/p4-acceptance.mjs
  • Create: backend/scripts/p4-acceptance.test.mjs
  • Create: backend/scripts/fixtures/p4-qdrant-proxy.mjs
  • Modify: .gitignore only if the existing .artifacts/ rule is insufficient

Owned layout:

.artifacts/p4-integration/<run-id>/
├── ownership.json
├── source.json
├── installation/thothii-installation.yaml
├── registry/{remote.git,author}/
├── fixture-secrets/
├── compose/acceptance.override.yaml
├── requests/
├── responses/
├── observations/
├── recovery/qdrant-rebuild-state.json
├── logs/
├── artifact-manifest.json
├── cleanup.json
├── report.json
└── report.md
  • Step 1: Write RED wrapper/orchestrator tests

Test argument parsing (integration [--keep] only), dirty tracked-tree refusal, unique run/project naming, fixture-only credentials, fixed toolchain resolution, no retry loop, bounded output, ownership checks, report schema, artifact hashes, secret-canary scan, cleanup behavior on success/failure/signal, and --keep retaining only the owned filesystem root.

  • Step 2: Verify RED
./scripts/test-p4-acceptance.sh

Expected: FAIL because the wrapper/orchestrator are missing.

  • Step 3: Implement the isolated topology

Use a real local bare Git remote and schema-v3 descriptor, compiled core image, real Qdrant storage, and production thothctl. An acceptance-only private proxy named qdrant fronts a real uniquely named Qdrant service; it forwards normal REST calls and provides a deterministic, test-owned barrier after DELETE so the one-shot maintenance container can be killed after durable mutation_started state and before recreate. It must not alter production code paths or accept secrets. Ollama/DWH/LLM use safe fixtures because they are outside D4.

  • Step 4: Implement positive and negative checks

Record named PASS checks for every assertion in the automated goal. For the session self-heal check, traverse the real backend admission route and intentionally stop at a later fixture readiness boundary only after Qdrant creation has been final-read compatible; assert no session manifest/Pi process survives. For concurrency, prove both layers separately: coordinate two direct real manager calls through the proxy so one compatible create/index conflicts, and send two simultaneous local same-principal route admissions to prove ReadinessManager performs exactly one create. Also run the upstream-auth route with two distinct principals if the cross-principal route proof is retained. Do not loop/retry until green and do not claim two local same-principal route calls both reached Qdrant.

Preseed one target point and one neighbor collection/point. Successful rebuild must remove/recreate only the target, leave the neighbor byte/count identity unchanged, retain canonical source files, and report that reindexing is required.

  • Step 5: Implement interruption and recovery check

Block recreate after the exact target DELETE, kill only the uniquely labelled one-shot container, and assert: command failure; core stopped; maintenance marker active; state mutation_started:true; admissions refused if core is deliberately restarted under the marker. Release the proxy barrier and run the exact confirmed collection recover once; assert verified collection, healthy core, inactive marker, terminal state, and neighbor preservation.

  • Step 6: Implement secret scan, artifact binding, and exact cleanup

Scan every retained regular file and bounded report/log field for all fixture secret canaries and credential-shaped URLs. Hash all declared artifacts, then remove only containers/volumes/network/proxy image/reference carrying this run's exact Compose project/run label. Prove no owned resource remains and no pre-existing resource was changed. There is no global docker system prune, volume prefix wildcard, or broad process kill.

  • Step 7: Verify harness syntax/unit contract
bash -n scripts/p4-acceptance.sh scripts/test-p4-acceptance.sh
node --check backend/scripts/p4-acceptance.mjs
node --check backend/scripts/p4-acceptance.test.mjs
node --check backend/scripts/fixtures/p4-qdrant-proxy.mjs
./scripts/test-p4-acceptance.sh

Expected: PASS without starting the full process from the unit-contract test.

  • Step 8: Commit the process goal
git add scripts/p4-acceptance.sh scripts/test-p4-acceptance.sh \
  backend/scripts/p4-acceptance.mjs backend/scripts/p4-acceptance.test.mjs \
  backend/scripts/fixtures/p4-qdrant-proxy.mjs .gitignore
git commit -m "test: add p4 qdrant lifecycle acceptance"
  • Step 9: Confirm the goal is runnable, but defer the one authoritative full run
git status --short
./scripts/test-p4-acceptance.sh

Expected: clean status and PASS. Do not launch the authoritative Docker/process run yet: Task 10 still adds the manual helper/docs included in the final source boundary. The one retained release run occurs in Task 11 from the final clean implementation/manual-documentation commit.

Task 10: Publish the independent manual walkthrough

Files:

  • Modify: docs/testing/p2-p6-manual-verification.md (replace only the P4 placeholder section)

  • Create: scripts/p4-manual-verification.sh

  • Create: scripts/test-p4-manual-verification.sh

  • Modify: docs/install/local-workspace-registry.md

  • Modify: docs/install/server-workspace-registry.md

  • Modify: tools/thothctl/cmd/thothctl/main.go usage text if not already complete

  • Step 1: Write RED manual-helper contract tests

The helper accepts only prepare, interrupt-after-delete, release-recovery, status, and cleanup; owns .artifacts/manual-acceptance/p4; refuses reuse without cleanup; emits GUIDE.md; uses a new Git remote/Compose project/volumes distinct from automated state; never performs the reviewer commands or records PASS on the reviewer's behalf.

./scripts/test-p4-manual-verification.sh

Expected: FAIL until the helper exists.

  • Step 2: Implement prepare/status/fault-control/cleanup only

prepare builds the fixture and prints exact non-secret variables. interrupt-after-delete arms the deterministic fixture barrier but does not invoke rebuild. release-recovery releases only that barrier. cleanup removes only the manual run's labelled resources/root after explicit reviewer confirmation. No helper calls workspace collection rebuild/recover for the reviewer.

  • Step 3: Replace the P4 walkthrough placeholder with exact commands

Document, in order:

  1. prepare new manual state;
  2. workspace inspect --json on missing collection;
  3. trigger one admission and inspect exact self-healed contract;
  4. delete one safe fixture keyword index, trigger admission, and inspect its repair;
  5. seed dimension, distance, and index-type incompatibilities and record nonmutation refusal;
  6. run workspace/collection confirmation mismatches and prove marker/core/collection unchanged;
  7. create a fixture open session and prove session_inventory_active;
  8. hold the P2 writer lock and prove preprocessing_conflict before delete with safe restart/clear;
  9. run the exact confirmed rebuild, inspect terminal state, neighbor preservation, healthy core, and inactive maintenance;
  10. arm interruption, run rebuild, observe core stopped/maintenance retained/recovery state, release barrier, run exact confirmed recovery, and verify final state;
  11. inspect report/state without secrets, decide PASS/FAIL, then cleanup.

For every step explain the component crossed, expected JSON fields/code, state/artifact produced, and invariant. Include Decision: **PENDING** and blank reviewer/date/evidence fields; only the human changes it to PASS/FAIL.

  • Step 4: Update operator manuals

Explain self-heal vs destructive rebuild, exact confirmations, closed/finalized/archived inventory, lifecycle lock contention with Pi update, maintenance marker semantics, writer-lock refusal, post-delete recovery, intentional vector loss/reindex sequence, and backup recommendation. Do not claim P5/P6 materialization or automated reindexing.

  • Step 5: Verify docs/helper
bash -n scripts/p4-manual-verification.sh scripts/test-p4-manual-verification.sh
./scripts/test-p4-manual-verification.sh
./scripts/verify-workspace-install-docs.sh --fixtures-only

Expected: PASS; the living P4 section contains no placeholder text and remains PENDING.

  • Step 6: Commit
git add docs/testing/p2-p6-manual-verification.md \
  docs/install/local-workspace-registry.md docs/install/server-workspace-registry.md \
  scripts/p4-manual-verification.sh scripts/test-p4-manual-verification.sh \
  tools/thothctl/cmd/thothctl/main.go
git commit -m "docs: add p4 qdrant lifecycle walkthrough"

Task 11: Final verification, checkpoint report, and hard stop

Files:

  • Create after green evidence exists: docs/testing/p4-qdrant-bootstrap-rebuild-checkpoint.md

  • Modify: PROJECT_STATE.md

  • Step 1: Run the scoped P4 verification from a clean commit

git status --short
cd backend && npx vitest run \
  test/qdrant-collection-manager.test.ts \
  test/tht-qdrant-readiness.test.ts \
  test/readiness-manager.test.ts \
  test/routes-sessions.test.ts \
  test/maintenance-gate.test.ts \
  test/pi-process-manager.test.ts \
  test/qdrant-rebuild-state.test.ts \
  test/workspace-preprocessing-state.test.ts \
  test/workspace-preprocessing-service.test.ts \
  test/workspace-maintenance.test.ts
cd backend && npx tsc --noEmit -p . && npm run build
cd tools/thothctl && go test ./...
./scripts/test-default-compose.sh
./scripts/test-unified-compose.sh
./scripts/test-internal-semantic-compose.sh
./scripts/test-compose-secret-policy.sh
./scripts/test-no-deployment-coupling.sh
./scripts/test-deployment-command-contract.sh
./scripts/test-p4-acceptance.sh
./scripts/test-p4-manual-verification.sh
./scripts/verify-workspace-install-docs.sh --fixtures-only
git diff --check
git status --short

Expected: every command exits 0; TypeScript/build and all Go tests PASS; final tracked status is clean. Do not run or claim the aggregate P2–P6 smoke, full harness/frontend suites, real PSD, or P5/P6 flows—those remain the post-P6 gate.

  • Step 2: Run the one authoritative clean-state process and retain it
./scripts/p4-acceptance.sh integration --keep

Expected: exit 0; report.json and report.md say every named check PASS, automated integration: PASS, manual acceptance: PENDING, secret_scan: PASS, cleanup: PASS, and retry_count: 0. If it fails, retain the failed run, diagnose the root cause, add a focused regression, fix and commit, rerun Step 1, then launch a new run ID from the beginning—never resume a partial run or retry a mutation inside a run.

  • Step 3: Verify the retained process report independently

Check the retained P4 run is bound to the clean Step 1 source commit/tree, contains one scenario execution and no automatic retry, has all named checks PASS, secret scan PASS, exact cleanup PASS, a valid artifact SHA-256 manifest, no undeclared regular files, and no owned live Docker resource. Record report.json and report.md hashes.

  • Step 4: Write the checkpoint report

docs/testing/p4-qdrant-bootstrap-rebuild-checkpoint.md must state:

  • source commit/tree and plan/design/PRD references;
  • exact commands and outcomes (do not invent counts);
  • retained report path and hashes;
  • self-heal, manager compatible-creator race, local route deduplication, distinct-principal route convergence, and incompatibility evidence;
  • lifecycle lock, inventory, admissions/Pi quiescence, writer lock, stop/restart evidence;
  • successful rebuild and interrupted recovery evidence;
  • secret-scan and exact-cleanup evidence;
  • automated integration: PASS and manual acceptance: PENDING;
  • explicit P5/P6/post-P6 exclusions and any real-platform checks not run.

Update only the evolving P4 section of PROJECT_STATE.md with the same truthful status.

  • Step 5: Commit checkpoint documentation
git add docs/testing/p4-qdrant-bootstrap-rebuild-checkpoint.md PROJECT_STATE.md
git commit -m "docs: record p4 automated checkpoint"

The retained run is intentionally bound to this checkpoint commit's parent: the final clean implementation and manual-documentation source boundary. The docs-only checkpoint records immutable report hashes and is not falsely claimed as input to its own report.

  • Step 6: Complete the persistent process goal and stop

Require a clean worktree, then complete the goal only after the authoritative run and cleanup proof. Send the user the checkpoint report and exact P4 manual section. Stop for explicit authorization/manual decision; do not begin P5, P6, aggregate verification, or unrelated cleanup.

Manual acceptance completion (later, reviewer-owned)

After the reviewer executes the independent P4 walkthrough, record only their actual decision and evidence. If PASS, update the P4 checkpoint and PROJECT_STATE.md from manual acceptance: PENDING to PASS in one docs-only commit. If FAIL, retain the failure evidence, reopen the P4 goal, add a focused regression, and repeat the entire automated process from a new clean run before asking for another manual decision.

Reference in New Issue View Git Blame Copy Permalink