ThothII/.superpowers/sdd/evidence-task-4-report.md

Evidence Task 4 — shared job envelope

Status: complete

Delivered

• Immutable JobSpec, JobRun, JobReport, per-stage state, sanitized error, and UTC timestamp records.
• run_job(spec, stages) with a durable checkpoint at job start, before and after every stage, and at terminal state. Successful stages are skipped when a prior run is resumed.
• Atomic JSON checkpoint/report replacement using a unique same-directory temporary file, file fsync, atomic os.replace, and parent-directory fsync.
• Public reports contain fixed operational fields only. Workspace paths, stage return values, exception messages, source content, credentials, and arbitrary metadata are not serialized.
• WorkspaceJobLock uses non-blocking kernel flock on a stable workspace/job-specific inode. Locks are released by the kernel on process exit; lock files are never removed based on PID, avoiding stale-lock and PID-reuse deletion races. Evidence and DWH use distinct lock files.
• Dry-run intent is immutable in the spec/report and exposed to every stage through JobContext.

TDD evidence

Initial focused collection failed because tht.jobs did not exist. Tests then drove:

• failure, sanitized reporting, resume, and idempotent successful-stage skipping;
• corrupt-checkpoint refusal before stage execution;
• JSON schema and path/secret/PII exclusion;
• dry-run propagation and ordered aware timestamps;
• multiprocessing exclusion, distinct Evidence/DWH jobs, traversal rejection, and recovery after a lock-owning process crashes.

Final focused result:

11 passed in 0.42s

Verification

cd harness && .venv/bin/pytest -q
597 passed, 5 deselected, 17 warnings in 28.45s

cd harness && .venv/bin/ruff check tht/jobs tests/test_job_runner.py tests/test_job_locking.py
All checks passed!

The full Ruff invocation was also run. It reports 34 pre-existing violations in unrelated legacy tests; no Task 4 file is among them. L2 tests remain deselected by the repository configuration.

Operational notes

• fcntl.flock intentionally targets the supported Linux/macOS deployment environments; it is not a Windows locking implementation.
• The envelope does not publish or mutate an active corpus. Later pipeline stages must use JobContext.run_dir for staging and perform their own final atomic publish only after validation.
• A dry run is an execution mode foundation: the runner exposes and records it; individual stages remain responsible for suppressing external mutations.