feat(compose): use one secret bundle for local services
This commit is contained in:
@@ -0,0 +1,139 @@
|
||||
# Task 3 report — vector port and wrappers
|
||||
|
||||
## Status
|
||||
|
||||
Complete. Added the transport-neutral vector port, HTTP and legacy-direct wrappers, and
|
||||
routed the existing `RestSearcher` and `DirectSearcher` through them while retaining the
|
||||
canonical `VectorRecord` and `VectorHit` models.
|
||||
|
||||
## TDD evidence
|
||||
|
||||
- RED: `cd harness && .venv/bin/pytest tests/test_vector_port_contract.py -q`
|
||||
- Result: collection failed with `ModuleNotFoundError: No module named
|
||||
'tht.adapters.vector'` (expected missing-port failure).
|
||||
- GREEN: `cd harness && .venv/bin/pytest tests/test_vector_port_contract.py -q`
|
||||
- Result: `4 passed in 0.11s`.
|
||||
|
||||
## Verification
|
||||
|
||||
- Required regression command:
|
||||
`cd harness && .venv/bin/pytest tests/test_vector_port_contract.py
|
||||
tests/test_vector_dual_key.py tests/test_search_similar_kinds.py
|
||||
tests/test_memory_save_one.py tests/test_solved_question.py -q`
|
||||
- Result: `26 passed in 0.12s` (fresh final run; earlier run: 26 passed in 0.16s).
|
||||
- Broader vector-focused regression:
|
||||
`cd harness && .venv/bin/pytest tests -q -k 'vector or search_similar or
|
||||
memory_save_one or solved_question'`
|
||||
- Result: `29 passed, 370 deselected in 0.49s`.
|
||||
- Scoped lint:
|
||||
`cd harness && .venv/bin/ruff check tht/ports/vector.py tht/adapters/vector
|
||||
tht/vectorstore/reader.py tests/test_vector_port_contract.py`
|
||||
- Result: `All checks passed!`.
|
||||
- Whitespace check: `git diff --check`
|
||||
- Result: exit 0, no output.
|
||||
- Staged whitespace check: `git diff --cached --check`
|
||||
- Result: exit 0, no output.
|
||||
|
||||
## Commit
|
||||
|
||||
`ff4d662 refactor(vector): define store contract`
|
||||
|
||||
Only the six Task 3 implementation/test files were included in the commit.
|
||||
|
||||
## Self-review / concerns
|
||||
|
||||
- Reader and writer clients remain separate: search and health use only the reader;
|
||||
hashes and upsert use only the writer.
|
||||
- A missing writer reports `upsert=False` and raises `VectorWriteUnavailable`, including
|
||||
for an empty upsert, so deployments cannot silently bypass the write credential gate.
|
||||
- Existing REST kind forwarding, legacy-404 fallback, post-filtering, global similarity
|
||||
merge, and direct table-per-kind behavior remain delegated to their established code.
|
||||
- The port re-exports existing vector models instead of duplicating result types.
|
||||
- Non-blocking design constraint: embedded values for the new generic `upsert` contract are
|
||||
carried in `VectorRecord.metadata['embedding']`; this preserves the existing canonical
|
||||
record model and REST payload without changing out-of-scope `records.py`. A later direct
|
||||
pgvector implementation may choose to formalize that field across adapters.
|
||||
|
||||
## Authorized contract correction
|
||||
|
||||
Status: complete. Commit: `fe8d70d fix(vector): separate write transport fields`.
|
||||
|
||||
The earlier metadata-envelope concern above is superseded. The contract now exports a
|
||||
dedicated frozen `VectorWriteRecord` containing the canonical `VectorRecord`, precomputed
|
||||
embedding, and content hash. `VectorStore.upsert` accepts only that envelope. HTTP row
|
||||
serialization takes transport fields from the envelope and preserves `record.metadata`
|
||||
unchanged, including legitimate metadata keys named `embedding` and `content_hash`.
|
||||
|
||||
### Corrective TDD evidence
|
||||
|
||||
- RED: `cd harness && .venv/bin/pytest tests/test_vector_port_contract.py -q`
|
||||
- Result: collection failed with `ImportError: cannot import name 'VectorWriteRecord' from
|
||||
'tht.ports.vector'` (expected missing-envelope failure).
|
||||
- GREEN: `cd harness && .venv/bin/pytest tests/test_vector_port_contract.py -q`
|
||||
- Result: `7 passed in 0.11s`.
|
||||
- Required regression suite:
|
||||
`cd harness && .venv/bin/pytest tests/test_vector_port_contract.py
|
||||
tests/test_vector_dual_key.py tests/test_search_similar_kinds.py
|
||||
tests/test_memory_save_one.py tests/test_solved_question.py -q`
|
||||
- Result: `29 passed in 0.15s` (fresh final run; earlier run: 29 passed in 0.14s).
|
||||
- Scoped lint:
|
||||
`cd harness && .venv/bin/ruff check tht/ports/__init__.py tht/ports/vector.py
|
||||
tht/adapters/vector tht/vectorstore/reader.py tests/test_vector_port_contract.py`
|
||||
- Result: `All checks passed!`.
|
||||
- Whitespace check: `git diff --check`
|
||||
- Result: exit 0, no output.
|
||||
|
||||
### Corrective self-review
|
||||
|
||||
- A real `evidence_records(...)` canonical builder record is serialized in the contract tests.
|
||||
- Collision coverage proves semantic `embedding` and `content_hash` metadata survive while
|
||||
distinct envelope values occupy the RPC row's top-level transport fields.
|
||||
- Reader health/search still use only the reader client; hashes/upsert still use only writer.
|
||||
- Existing kind forwarding, legacy 404 fallback, post-filtering, similarity merge, and direct
|
||||
search behavior remain unchanged and covered by the required regression suite.
|
||||
- The tracked plan, regenerated Task 3 scratch brief, port exports, adapter signature, and this
|
||||
report now consistently describe `VectorWriteRecord`.
|
||||
- Concerns: none known.
|
||||
|
||||
---
|
||||
|
||||
# Task simple-config-3 report — one bundle for local-vector and preprocess
|
||||
|
||||
## Status
|
||||
|
||||
Complete. Local pgvector bootstrap, reconciliation, migration, and preprocess services now
|
||||
mount only `/run/secrets/thothii.secrets`. `deploy/vector/secret-policy.sh` validates the
|
||||
whole bundle (allowlist, duplicate/empty/unknown keys, comments/blank lines, mode and symlink
|
||||
policy) and returns only the requested value. The core entrypoint exposes DWH/vector/CA values
|
||||
to the harness and materializes short-lived 0600 password files for workspace resolution.
|
||||
|
||||
## TDD evidence
|
||||
|
||||
- RED: `./scripts/test-preprocess-compose-config.sh` failed on the pre-existing
|
||||
`vector_reader_password` Compose secret declaration.
|
||||
- GREEN: the same command passes after the bundle conversion and verifies local-vector
|
||||
workspace interpolation and shared secret mounts.
|
||||
- Added bundle parser regressions to `./scripts/test-vector-secret-policy.sh`: comments/blank
|
||||
lines are accepted and an unrelated duplicate key is rejected.
|
||||
|
||||
## Verification
|
||||
|
||||
- `./scripts/test-vector-secret-policy.sh` — passed.
|
||||
- `./scripts/test-preprocess-compose-config.sh` — passed.
|
||||
- `./scripts/test-vector-backup-restore-safety.sh` — passed.
|
||||
- `./scripts/test-default-compose.sh` — passed.
|
||||
- `./scripts/test-container-deployment.sh` — passed.
|
||||
- `./scripts/local-vector-smoke.sh` — passed (real Docker; bootstrap rotation, role
|
||||
reconciliation, migration, persistence and restart).
|
||||
- `./scripts/preprocess-smoke.sh` — passed (real Docker; unchanged rerun, mutation, DWH job,
|
||||
ACTIVE publication and cleanup).
|
||||
- `git diff --check` and `sh -n` gates — passed.
|
||||
|
||||
## Commit
|
||||
|
||||
Pending: `feat(compose): use one secret bundle for local services`.
|
||||
|
||||
## Concerns
|
||||
|
||||
The rotation helper still accepts old/new scratch files because that is its explicit CLI
|
||||
contract; the smoke script keeps those files outside Compose and mounts only the bundle.
|
||||
Reference in New Issue
Block a user