28 lines
1.8 KiB
Markdown
28 lines
1.8 KiB
Markdown
# Evidence preprocessing Task 7 report
|
|
|
|
Implemented the S3-compatible Evidence adapter, explicit preprocessing Compose overlay, and
|
|
operational gates.
|
|
|
|
- S3 discovery uses bounded paginator pages, page size, and total objects; acquisition enforces a
|
|
byte ceiling and always closes streaming bodies.
|
|
- Provenance is canonical `s3://bucket/key`. Versioned objects use `s3-version:<version>`;
|
|
unversioned objects use a hashed exact ETag, and acquisition refuses validator drift.
|
|
- The adapter uses boto3/botocore rather than custom signing. TLS verification is enabled by
|
|
default. Custom HTTP and private endpoints require independent explicit opt-ins; endpoint
|
|
userinfo is rejected and public custom endpoints are DNS-policy checked.
|
|
- Access, secret, and session credentials support file-secret resolution into masked `SecretStr`
|
|
config fields. They are never emitted in provenance, reports, errors, or Compose environment.
|
|
- `deploy/compose.preprocess.yaml` provides separate one-shot Evidence and DWH jobs and is inert
|
|
unless explicitly included with the `preprocess` profile.
|
|
- `scripts/preprocess-smoke.sh` verifies both services render without secret material and pins an
|
|
unchanged rerun plus a modified generation through deterministic pipeline tests.
|
|
|
|
Verification: focused S3/HTTP/filesystem/config tests 34 passed; operational smoke 2 passed; core
|
|
image with locked boto3 extra built; full harness 702 passed, 5 deselected; scoped Ruff and diff
|
|
checks passed.
|
|
|
|
Operational risk: custom S3-compatible endpoints remain part of the deployment trust boundary.
|
|
Private endpoint access must be explicitly enabled and should be restricted by container egress
|
|
policy in production. S3 list consistency semantics are provider-defined; version IDs are preferred
|
|
over ETags wherever bucket versioning is available.
|