99 lines
5.4 KiB
Markdown
99 lines
5.4 KiB
Markdown
# Evidence / Preprocessing Task 1 Report
|
|
|
|
## Outcome
|
|
|
|
Implemented the additive Evidence source port and canonical corpus records. Existing evidence,
|
|
search, vector, and session runtime code is unchanged.
|
|
|
|
## Contract
|
|
|
|
- `EvidenceSource` is a runtime-checkable protocol with `discover` and `acquire` operations.
|
|
- `SourceObject` and `AcquiredDocument` are frozen, reject extra fields, use independent metadata
|
|
defaults, and restrict metadata to Pydantic `JsonValue` values.
|
|
- `CanonicalDocument`, `CanonicalChunk`, and `CorpusManifest` are frozen and reject extra fields.
|
|
- Provenance includes stable source IDs, canonical URIs, fingerprints, modification time, and
|
|
content hashes.
|
|
- Pipeline versions are recorded on documents, chunks, and manifests. Manifests also carry schema
|
|
version, optional publish ID/vector generation, and paired embedding model/dimension fields.
|
|
- Credential-like metadata keys are rejected recursively. Credentials are not model fields and
|
|
therefore cannot enter serialized canonical artifacts through extras.
|
|
|
|
## TDD evidence
|
|
|
|
The initial focused run failed during collection because `tht.ports.evidence` and `tht.corpus`
|
|
did not exist. After implementation, the focused suite passed.
|
|
|
|
## Verification
|
|
|
|
- Focused models/protocol tests: 13 passed.
|
|
- Harness excluding Docker-backed L0 and the network-dependent wheel packaging test: 444 passed,
|
|
5 deselected.
|
|
- Focused Ruff: passed.
|
|
- Full-repository Ruff remains blocked by 34 pre-existing findings outside the task files.
|
|
- An unrestricted `pytest -q` attempt reached 453 passed and 5 deselected, but reported 47 Docker
|
|
setup errors plus 4 Docker parity failures because the sandbox cannot access the Docker socket;
|
|
the wheel packaging test also failed because its isolated `uv build` needs unavailable network.
|
|
|
|
## Concerns / follow-up
|
|
|
|
- Pydantic's `frozen=True` prevents model field reassignment but does not recursively freeze list
|
|
and dict contents. `default_factory` prevents shared mutable defaults. Later pipeline stages should
|
|
treat these value objects as immutable and construct replacements rather than mutate collections.
|
|
- The adapter and normalization tasks should preserve the credential-free boundary by passing only
|
|
these records beyond acquisition.
|
|
|
|
## Review hardening follow-up
|
|
|
|
All six binding review areas were addressed in a separate TDD pass:
|
|
|
|
- JSON metadata is recursively converted to immutable `FrozenDict`/tuple values while retaining
|
|
stable object/array JSON serialization. Manifest document and chunk collections are tuples.
|
|
- Secret-key matching now normalizes camelCase and punctuation. It rejects credential-specific
|
|
names (passwords, API keys, access/refresh tokens, client/private keys, session cookies and
|
|
authorization) recursively, while deliberate benign labels such as generic `token` and `secret`
|
|
remain valid.
|
|
- Canonical URIs require a scheme and reject userinfo or credential-bearing query parameters.
|
|
- Namespaced IDs, SHA-256 content hashes, timezone-aware UTC timestamps, embedding/vector
|
|
compatibility, unique IDs, chunk referential/provenance integrity, contiguous per-document
|
|
ordinals and pipeline-version consistency are validated. Nested Pydantic instances are always
|
|
revalidated so `model_copy(update=...)` cannot bypass a manifest boundary.
|
|
- Acquired arbitrary bytes have explicit base64 JSON encoding and validation, covered by a JSON
|
|
round-trip test.
|
|
- `EvidenceSourceError` classifies transient/retryable versus permanent failures and exposes only
|
|
recursively immutable, credential-screened JSON details.
|
|
|
|
Follow-up verification:
|
|
|
|
- Focused contract suite: 39 passed.
|
|
- Focused Ruff: passed.
|
|
- Harness excluding Docker-backed L0 and the network-dependent wheel packaging test: 470 passed,
|
|
5 deselected.
|
|
- Fresh unrestricted harness attempt: 479 passed, 5 deselected; the same environmental boundary
|
|
remains (47 Docker socket setup errors, four Docker parity failures, one isolated `uv build`
|
|
network failure).
|
|
|
|
## Final blocker follow-up
|
|
|
|
The remaining four contract blockers were closed in a third TDD cycle:
|
|
|
|
- `EvidenceSourceError` now always exposes the fixed public message/`args` value `evidence source
|
|
operation failed`; caller diagnostics are not retained. Category, details and args cannot be
|
|
reassigned, details remain recursively frozen and credential-screened, and an original exception
|
|
is available only when callers use standard exception chaining.
|
|
- Canonical document/chunk provenance stores only URI scheme, authority and path. Userinfo is
|
|
rejected; query strings and fragments are removed unconditionally, including AWS `X-Amz-*`, SAS
|
|
`sig`, and fragment token material.
|
|
- Binding model bases override Pydantic's unchecked `model_copy(update=...)`: merged values always
|
|
pass full field/model validation, so invalid copied records and top-level manifests fail.
|
|
- A canonical document/chunk `content_hash` must equal SHA-256 of the exact stored text encoded as
|
|
UTF-8. This establishes the normalization boundary explicitly: line-ending/frontmatter/text
|
|
normalization happens before model construction; the canonical models never rewrite content.
|
|
|
|
Final follow-up verification:
|
|
|
|
- Focused contract suite: 45 passed.
|
|
- Focused Ruff: passed.
|
|
- Harness excluding Docker-backed L0 and network-dependent packaging: 476 passed, 5 deselected.
|
|
- Fresh unrestricted harness attempt: 486 passed, 5 deselected, with the unchanged environmental
|
|
failures (47 Docker setup errors, four Docker parity failures, one isolated `uv build` failure).
|