50 lines
3.3 KiB
Markdown
50 lines
3.3 KiB
Markdown
# Evidence Task 6 — final fd-anchored DWH correction
|
|
|
|
All DWH generation state below `.tht-dwh` is now accessed relative to the directory descriptor
|
|
retained by the shared/exclusive generation lease. ACTIVE reads, atomic temp writes, replacement,
|
|
fsync, and rollback use `openat`/`replaceat` operations. Generation staging, validation,
|
|
reconciliation, resume checks, retention classification, and recursive deletion likewise use owned
|
|
root/generations/candidate descriptors with `O_NOFOLLOW`; locked operations no longer reopen
|
|
generation paths through `workspace_root`.
|
|
|
|
Portable reader snapshots are copied from validated generation file descriptors into private 0700
|
|
process-owned temporary directories while the shared lease is held. This avoids Linux-only
|
|
`/proc/self/fd` paths and prevents a renamed/replaced `.tht-dwh` pathname from redirecting later
|
|
schema or LSH reads. Lease-scoped copies are removed on exit and standalone snapshots are removed
|
|
at process exit.
|
|
|
|
Deterministic adversarial tests rename the DWH root after lease acquisition during ACTIVE reads,
|
|
ACTIVE publication, and retention cleanup. Each test proves the replacement tree is never read,
|
|
written, or deleted; the descriptor-pinned original either completes consistently or fails closed.
|
|
Existing owner binding, legacy rejection, crash reconciliation, resume, atomic rollback, retention,
|
|
and reader/writer exclusion behavior remains covered.
|
|
|
|
## Final review correction
|
|
|
|
Snapshot materialization now reads the manifest and every owned artifact exactly once through the
|
|
already-open generation descriptor, validates each hash against those exact bytes, and writes the
|
|
same byte objects to the private snapshot. A deterministic second-read mutation test proves hostile
|
|
pickle bytes can neither pass validation nor enter the snapshot. Reconciliation closes the ACTIVE
|
|
generation descriptor in a `finally` block on matches, mismatches, and exceptions. Pipeline-owned
|
|
snapshot directories are removed and deregistered after `run_job` on both successful and failed
|
|
runs, preventing repeated pipeline use from accumulating temporary directories or registry entries.
|
|
|
|
The cleanup boundary now begins immediately after snapshot materialization. Resume checkpoint
|
|
validation and `JobSpec` construction are guarded by the same release routine as `run_job`, so
|
|
corrupt/mismatched resume state or constructor failure clears the pipeline holder, removes the
|
|
private directory, and restores the snapshot registry to its prior state before propagating.
|
|
|
|
## Shipped preprocessing startup contract
|
|
|
|
Local-vector preprocessing now uses a dedicated Compose override. Both one-shot jobs depend on a
|
|
successfully completed `vector-migrate`, whose transitive chain waits for database health and role
|
|
reconciliation. The generic preprocessing overlay remains independently renderable and contains no
|
|
local-vector services or password secrets. README commands include the local override and build the
|
|
job image before running.
|
|
|
|
The real clean-project smoke no longer injects dependencies or manually starts, reconciles, or
|
|
migrates PostgreSQL. Its first shipped `compose run preprocess-evidence` demonstrably creates the
|
|
database, waits for health, runs reconciliation and migration, then runs the Evidence job. Unchanged
|
|
rerun, changed-source publish, DWH preprocessing, ACTIVE verification, and injected-failure cleanup
|
|
all pass through the same shipped dependency path.
|