ThothII/.superpowers/sdd/evidence-task-5b-report.md

Evidence Task 5B implementation report

Status

Integrated Evidence preprocessing with the Task 4 JobRunner. The CLI now accepts only a 32-character JobRunner run ID for --resume; generation IDs remain outputs. Runs persist the exact ordered stages discover, acquire_normalize_chunk, embed, vector_upsert, stage_validate, publish, and retention_cleanup.

Successful-stage artifacts are copied into the new resume run before execution, allowing later stages to continue without rediscovery, acquisition, normalization, chunking, or embedding. Job compatibility includes workspace, configuration, discovered-input, pipeline, embedding, and chunk-policy fingerprints. Generation-specific filesystem/vector compensation is retained, and a compensated generation is rotated before retry. ACTIVE is mutated only by publish.

Dry-run executes discovery/planning and makes every side-effecting stage a no-op. JSON output is pristine and includes the JobRunner run_id, resumed_from, generation, plan, and publish status.

TDD evidence

• RED: run-ID rejection and resume-artifact tests failed because generation IDs reached configuration and resume runs had empty artifact directories.
• GREEN: the two regression tests passed after strict CLI validation and durable artifact carryover.
• Added pipeline job-plan and dry-run counting-fake coverage; both passed.

Fresh verification

• Focused integration/search suite: 62 passed, 4 warnings.
• Available harness suite excluding sandbox-blocked Docker, loopback HTTP-server, and networked wheel-build tests: 559 passed, 5 deselected, 18 warnings.
• Scoped Ruff: All checks passed!.
• git diff --check: clean.

Environment limitations and concerns

The literal full harness invocation cannot complete in the managed sandbox: Docker socket access, loopback HTTP test servers, and the uv build dependency resolution path are denied. It reached 575 passed, 5 deselected before those environment errors. The available-suite rerun above is green.

One pre-existing Pydantic serialization warning is exposed by the new end-to-end job test when canonical metadata contains frozen tuple values; it does not contaminate CLI stdout. Retention is an explicit stable no-op until a retention policy is configured.

Review fix wave — crash consistency and artifact integrity

Addressed all five follow-up findings:

• JobRunner now supports a test-only post-call/pre-checkpoint fault hook. Each stage seals a canonical artifact manifest containing required flat filenames, SHA-256, byte size, producer stage, and the full spec compatibility fingerprint. Resume validates the checkpoint and every sealed artifact before allocating/copying a new run, rejecting missing, tampered, extra, nested, or symlinked state. A sealed running stage is promoted after a simulated process crash; a sealed failed stage is deliberately retried.
• Vector intent (exact record IDs and content hashes) is sealed before upsert. Execution reconciles existing_hashes and writes only missing/mismatched rows. Crash-after-effect tests prove no duplicate acquire, embed, or vector upsert.
• Raw upsert, stage, recovery-upsert, recovery-stage, and publish exceptions compensate the exact generation. Compensation markers survive failed checkpoints; resume rotates the generation, refreshes generation-bound artifacts, reconciles vectors, and stages idempotently.
• CorpusStore.publish is idempotent and failure-atomic. If replace succeeds but directory fsync fails, it restores the previous ACTIVE value (or removes a newly created pointer), fsyncs the rollback, and re-raises. Pipeline cleanup refuses to discard a generation referenced by ACTIVE.
• Added crash/resume coverage after all seven ordered stages; corrupt/missing plan, manifest, and embeddings; unsafe extra paths; nonexistent run IDs; raw vector/stage failures; and post-replace ACTIVE rollback.

Fresh fix-wave verification:

• Focused jobs/corpus/CLI/search suite: 82 passed, 17 warnings.
• Available harness suite (same sandbox exclusions described above): 579 passed, 5 deselected, 31 warnings.
• Scoped Ruff and git diff --check: clean.

Final P1 fix — effect state and checkpoint-bound manifest roots

• Stage checkpoints now distinguish intent from completed. Vector intent is atomically sealed and checkpointed before upsert. A process-level BaseException after a partial multi-record write leaves the stage running/intent; resume never promotes it and instead reconciles existing_hashes, writing only the missing records. The completed state is persisted only after reconciliation returns successfully.
• Every stage now persists its completed artifact state while still running, before the post-call fault hook. The checkpoint binds the SHA-256 of canonical artifact-manifest.json, effect state, exact producer stage, and exact required-file mapping. Resume validates this root and all bindings before promotion or copying.
• Added process-interruption coverage proving the already-written vector record is not submitted twice, remaining records are written, and publish completes only after reconciliation. Added coordinated artifact/manifest, spec-binding, and producer-binding tamper rejection tests.

Fresh verification:

• Focused jobs/corpus/CLI/search suite: 86 passed, 18 warnings.
• Available broad harness suite: 583 passed, 5 deselected, 32 warnings.
• Scoped Ruff and git diff --check: clean.