51 lines
2.5 KiB
Markdown
51 lines
2.5 KiB
Markdown
# Evidence Task 5D — Real pgvector lifecycle gate
|
|
|
|
## Status
|
|
|
|
Complete. The Docker-backed L0 gate uses one persistent `pgvector/pgvector:pg16`
|
|
database and the production migrations, direct reader/writer `PgVectorStore`,
|
|
`CorpusStore`, `CorpusPipeline.run_as_job`/JobRunner, ACTIVE Evidence retrieval,
|
|
search-pack fusion, owned session artifact copy, retention, and explicit GC.
|
|
|
|
## Lifecycle covered
|
|
|
|
- Four real corpus publications with retention set to two generations.
|
|
- A higher-similarity stale vector proves ACTIVE metadata filtering happens before LIMIT
|
|
for normal Evidence retrieval and the search-pack fusion path.
|
|
- A removed source is absent from ACTIVE retrieval and cannot be copied to a session.
|
|
- An injected process death occurs after one real committed vector upsert. Resume uses the
|
|
real run ID, preserves that record, fills the missing records, and produces no duplicate keys.
|
|
- Database engines and direct store objects are disposed/recreated before persisted ACTIVE
|
|
retrieval is checked again.
|
|
- An exact canonical vector-only orphan generation is discovered and removed by explicit GC.
|
|
- Filesystem and vector inventories converge exactly to ACTIVE plus one rollback; a second GC
|
|
is a no-op.
|
|
- Owned session artifact bytes and SHA-256 match the ACTIVE canonical document.
|
|
|
|
## Production bug found and fixed
|
|
|
|
Production migration `003_roles.sql` intentionally restricted `vector_writer`, but omitted
|
|
the privileges used by the production generation lifecycle: `SELECT(metadata)` for inventory
|
|
and `DELETE` for cleanup on `vectors.evidence`. Consequently a real job published successfully
|
|
and then failed in `retention_cleanup` on its first run.
|
|
|
|
Added versioned migration `004_evidence_generation_gc.sql` granting only those two Evidence
|
|
generation-management privileges. Runtime application code was not redesigned.
|
|
|
|
## Verification
|
|
|
|
- Target lifecycle: `1 passed` (Docker-backed).
|
|
- Full harness: `681 passed, 5 deselected`.
|
|
- Scoped Ruff: passed.
|
|
- `git diff --check`: passed.
|
|
|
|
The existing Pydantic serialization and legacy-workspace deprecation warnings remain unchanged.
|
|
|
|
## Follow-up assertion correction
|
|
|
|
The removal phase now retains the removed canonical document ID/ref before publication and
|
|
asserts both fields are absent from post-resume ACTIVE Evidence hits. It reruns the real
|
|
search-pack fusion after removal, proves active fourth-generation content is positively
|
|
returned in both paths, and proves the removed content remains absent. The owned session
|
|
artifact lookup for the retained removed ID remains empty.
|