# Evidence preprocessing Task 7 report

Implemented the S3-compatible Evidence adapter, explicit preprocessing Compose overlay, and
operational gates.

- S3 discovery uses bounded paginator pages, page size, and total objects; acquisition enforces a
  byte ceiling and always closes streaming bodies.
- Provenance is canonical `s3://bucket/key`. Versioned objects use `s3-version:<version>`;
  unversioned objects use a hashed exact ETag, and acquisition refuses validator drift.
- The adapter uses boto3/botocore rather than custom signing. TLS verification is enabled by
  default. Custom HTTP and private endpoints require independent explicit opt-ins; endpoint
  userinfo is rejected and public custom endpoints are DNS-policy checked.
- Access, secret, and session credentials support file-secret resolution into masked `SecretStr`
  config fields. They are never emitted in provenance, reports, errors, or Compose environment.
- `deploy/compose.preprocess.yaml` provides separate one-shot Evidence and DWH jobs and is inert
  unless explicitly included with the `preprocess` profile.
- `scripts/preprocess-smoke.sh` verifies both services render without secret material and pins an
  unchanged rerun plus a modified generation through deterministic pipeline tests.

Verification: focused S3/HTTP/filesystem/config tests 34 passed; operational smoke 2 passed; core
image with locked boto3 extra built; full harness 702 passed, 5 deselected; scoped Ruff and diff
checks passed.

Operational risk: custom S3-compatible endpoints remain part of the deployment trust boundary.
Private endpoint access must be explicitly enabled and should be restricted by container egress
policy in production. S3 list consistency semantics are provider-defined; version IDs are preferred
over ETags wherever bucket versioning is available.

## Review correction

The Compose overlay now uses committed, purpose-built Evidence and DWH workspace files with
job-specific dependencies. Its services create their lock roots and mount only the vector secrets
they consume. The operational smoke is a real isolated Compose project: real pgvector migrations,
a deterministic in-project embeddings endpoint, actual Evidence CLI JSON across initial/unchanged/
mutated runs, exact ACTIVE verification, an actual DWH introspection job, and owned cleanup.

S3 custom endpoints now fail closed unless declared trusted; HTTP and private loopback endpoints
need additional independent opt-ins. Boto uses forced path-style addressing. Custom endpoints reject
userinfo, query, fragment, and non-root paths. Buckets use strict DNS syntax; listed keys must remain
under prefix and within the S3 byte bound; validators must be nonempty/bounded. Because
ListObjectsV2 does not provide version IDs, discovery honestly fingerprints the exact ETag and
acquisition rejects ETag drift.

Final correction verification: S3/config focused 20 passed; full harness 721 passed, 5 deselected;
real Compose smoke and image build passed; scoped Ruff, shell syntax, and diff checks passed.