ThothII/.superpowers/sdd/evidence-task-2-report.md

Evidence Task 2 Report

Status

Implemented filesystem and explicit-manifest HTTP Evidence source adapters, typed source configuration with legacy compatibility, and factory construction.

Delivered behavior

• Filesystem discovery is deterministic and rooted at a strict canonical directory.
• Symlink/path escapes are rejected before content is exposed.
• Discovery hashing and acquisition reads enforce a configurable byte limit.
• Filesystem fingerprints are content SHA-256 values; stable IDs derive from relative paths.
• HTTP accepts only explicit http/https manifest entries and keeps transport URLs private.
• HTTP provenance strips query strings/fragments, while config and adapter representations hide signed or secret-bearing transport URLs.
• HTTP acquisition uses separate connect/read timeouts, streaming byte limits, bounded redirects, private redirect rejection, and safe transient/permanent error classification.
• HTTP fingerprints prefer a deterministic ETag digest, then Last-Modified, then content SHA-256.
• build_evidence_sources(cfg) supports both typed evidence.sources entries and the legacy source_root plus evidence_dir filesystem configuration.

TDD and verification

• RED: focused tests initially failed during collection because the adapter package did not exist.
• GREEN: 15 passed for filesystem, HTTP, and resource-config tests.
• Full harness: 548 passed, 5 deselected.
• Changed-file Ruff: clean.
• Repository-wide Ruff remains non-clean due to 34 pre-existing findings in unrelated test files; no unrelated lint files were modified.

Notes

The approved SourceObject namespace grammar does not permit raw quoted ETags such as etag:"abc". The adapter therefore uses etag:<sha256-of-opaque-etag>: it preserves ETag-based change identity without weakening the canonical contract or exposing validator contents.

Review hardening follow-up

Four review findings were closed in a separate follow-up commit:

• Filesystem access now anchors a persistent descriptor at the canonical root and walks each component with openat semantics (dir_fd, O_NOFOLLOW, and O_DIRECTORY). The regular-file check, bounded read, metadata, and hash all use the opened descriptor. Acquisition reopens by the same path-safe mechanism and rejects a changed fingerprint. Deterministic tests swap both a leaf and an ancestor to symlinks at open time.
• HTTP network policy defaults to public hosts only. Initial URLs and every redirect reject userinfo, mixed public/private IPv4/IPv6 answers fail closed, and the connected peer must be a public member of the previously validated DNS answer set before any body bytes are consumed. Explicit allow_private_hosts: true is required for trusted private deployments and local tests.
• Every HTTP response is closed in a finally block, including redirects, status failures, policy failures, oversized bodies, and mid-stream exceptions.
• ETag and Last-Modified values remain adapter-internal. Repeated discovery and acquisition send conditional headers; a 304 reuses only previously verified cached bytes and identity. The LRU content cache has an explicit byte bound (max_cache_bytes). Validators are not forwarded across redirect origins.

Conditional cache binding correction

The conditional cache now binds bytes and validators to both the canonical provenance key and the exact final effective representation URL. Redirect traversal recomputes request headers per hop: validators are sent only when that exact URL matches the cached final URL, never merely because a redirect retains an origin. A same-origin path change therefore downloads and replaces the body. The adapter accepts 304 only when the exact request carried a bound ETag or Last-Modified validator; unsolicited and cross-origin 304 responses are permanent protocol errors.