fix(preprocess): harden S3 and real Compose jobs

This commit is contained in:
2026-07-12 06:01:06 +02:00
parent 950f88f23e
commit c6966f3d15
10 changed files with 222 additions and 43 deletions
+11 -5
View File
@@ -55,18 +55,24 @@ must not be passed as URL arguments.
## Preprocessing jobs and S3 Evidence
Run one-shot jobs through the explicit overlay, which is inert for normal external/local runtime:
The included job workspaces target the local-vector profile. Point the four
`THT_VECTOR_*_PASSWORD_SECRET_FILE` variables at owner-only files, set `THT_OLLAMA_URL`, mount
Evidence at `/data/source/evidence`, then run the explicit overlays (which are inert for normal
runtime):
```sh
docker compose -f compose.yaml -f deploy/compose.preprocess.yaml --profile preprocess \
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml --profile local-vector --profile preprocess \
run --rm preprocess-evidence
docker compose -f compose.yaml -f deploy/compose.preprocess.yaml --profile preprocess \
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml --profile local-vector --profile preprocess \
run --rm preprocess-dwh
```
S3 Evidence uses the optional `tht[s3]` dependency and canonical `s3://bucket/key` provenance.
TLS and public endpoints are required by default; private or HTTP S3-compatible endpoints require
separate explicit opt-ins. Store access key, secret key, and session token as secret references in
AWS endpoints are used when no custom URL is supplied. Every custom endpoint is an explicit egress
trust-boundary opt-in and uses path-style addressing; private and HTTP endpoints require additional
independent opt-ins. Store access key, secret key, and session token as secret references in
deployment configuration—never in Compose environment values or source URIs. Discovery and reads
are bounded by configured page, object, and byte limits.