145 lines
8.5 KiB
Markdown
145 lines
8.5 KiB
Markdown
# Task 6 Report
|
|
|
|
Date: 2026-08-08
|
|
|
|
Status: implemented and verified
|
|
|
|
Summary:
|
|
|
|
- Added schema-v3 Qdrant runtime support to the harness config/resource layer and vector factory.
|
|
- Made Qdrant payloads carry `workspace_id` and `workspace_revision` on every point.
|
|
- Routed schema and memory bulk indexing through the transport-neutral vector port with canonical hash-based dedup.
|
|
- Kept Evidence canonical on filesystem and Memory canonical in JSONL; Qdrant remains derived/rebuildable.
|
|
- Added focused tests for semantic-kind isolation, shared identity fields, search-pack kind boundaries, and the schema-v3 factory/config path.
|
|
|
|
Files changed:
|
|
|
|
- `harness/tht/config.py`
|
|
- `harness/tht/config_compat.py`
|
|
- `harness/tht/adapters/factory.py`
|
|
- `harness/tht/adapters/vector/qdrant.py`
|
|
- `harness/tht/vectorstore/records.py`
|
|
- `harness/tht/cli/vector_cmd.py`
|
|
- `harness/tht/cli/memory_cmd.py`
|
|
- `harness/tests/test_semantic_kind_isolation.py`
|
|
- `harness/tests/test_memory_save_one.py`
|
|
- `harness/tests/test_search_pack.py`
|
|
- `harness/tests/test_qdrant_vector_store.py`
|
|
- `harness/tests/test_adapter_factory.py`
|
|
- `harness/tests/test_config_resources.py`
|
|
|
|
Verification:
|
|
|
|
- Focused RED/GREEN task suite:
|
|
- `cd harness && .venv/bin/pytest tests/test_semantic_kind_isolation.py tests/test_memory_save_one.py tests/test_search_pack.py -q`
|
|
- Relevant harness suite:
|
|
- `cd harness && .venv/bin/pytest tests/test_semantic_kind_isolation.py tests/test_memory_save_one.py tests/test_search_pack.py tests/test_qdrant_vector_store.py tests/test_adapter_factory.py tests/test_config_resources.py tests/test_vector_port_contract.py tests/test_corpus_pipeline.py -q`
|
|
- Result: `131 passed`
|
|
- Changed-file Ruff:
|
|
- `cd harness && .venv/bin/ruff check tht/vectorstore/records.py tht/adapters/vector/qdrant.py tht/config_compat.py tht/config.py tht/adapters/factory.py tht/cli/vector_cmd.py tht/cli/memory_cmd.py tests/test_memory_save_one.py tests/test_search_pack.py tests/test_semantic_kind_isolation.py tests/test_qdrant_vector_store.py tests/test_adapter_factory.py tests/test_config_resources.py`
|
|
- Result: clean
|
|
|
|
Concerns / follow-up:
|
|
|
|
- `memory clear` still retains its older direct-vector assumptions and was not expanded in this task because the brief focused on canonical builders and schema/evidence/memory routing through the active Qdrant path.
|
|
- The relevant suite still emits pre-existing warnings (legacy config deprecation in older fixtures, plus existing Pydantic serializer warnings in corpus tests), but they are not introduced by this task.
|
|
|
|
## Fix round 1 (2026-08-08)
|
|
|
|
Scope:
|
|
|
|
- Fixed qdrant-only schema-v3 command gating for `vector index-schema`, `memory promote`, and `memory index`.
|
|
- Replaced `memory clear`'s direct-pgvector-only path with vector-port deletion by kind.
|
|
- Added focused qdrant-only CLI regression tests and refreshed older CLI fixtures to the enforced internal embedding contract.
|
|
|
|
RED evidence:
|
|
|
|
- `cd harness && .venv/bin/pytest tests/test_qdrant_cli_commands.py -q`
|
|
- Initial result against commit `5e39cfa`: `4 failed`
|
|
- Failure signatures:
|
|
- `ERRORE: sezioni mancanti nel workspace yaml: vector_db o vector_write_rest.`
|
|
- `ERRORE: sezioni mancanti nel workspace yaml: vector_db.`
|
|
|
|
GREEN evidence:
|
|
|
|
- Focused fix suite:
|
|
- `cd harness && .venv/bin/pytest tests/test_qdrant_cli_commands.py tests/test_qdrant_vector_store.py tests/test_adapter_factory.py tests/test_config_resources.py tests/test_memory_save_one.py tests/test_search_pack.py -q`
|
|
- Result: `51 passed`
|
|
- Relevant broader vector/memory/schema/search suite:
|
|
- `cd harness && .venv/bin/pytest tests/test_qdrant_cli_commands.py tests/test_qdrant_vector_store.py tests/test_adapter_factory.py tests/test_config_resources.py tests/test_memory_save_one.py tests/test_search_pack.py tests/test_vector_port_contract.py tests/test_adapter_command_regressions.py tests/test_solved_search_cli.py tests/test_schema_introspect_guard.py tests/test_semantic_kind_isolation.py tests/test_corpus_pipeline.py -q`
|
|
- Result: `154 passed`
|
|
- Ruff on the fix surface:
|
|
- `cd harness && .venv/bin/ruff check tht/ports/vector.py tht/adapters/vector/qdrant.py tht/adapters/vector/pgvector.py tht/adapters/vector/thoth_http.py tht/vectorstore/rest_client.py tht/cli/vector_cmd.py tht/cli/memory_cmd.py tests/test_qdrant_cli_commands.py tests/test_solved_search_cli.py`
|
|
- Result: clean
|
|
|
|
Notes:
|
|
|
|
- `memory clear` now deletes derived `kind=memory` points through the configured writable vector store, while leaving the JSONL registry as the source of truth until the registry file is removed by the command.
|
|
- The broader suite still carries the same pre-existing warnings noted above; this fix round did not add new warnings or failures.
|
|
|
|
## Fix round 2 (2026-08-08)
|
|
|
|
Scope:
|
|
|
|
- Removed the accidental HTTP writer `delete_kinds` capability expansion from `ThothHttpVectorStore` and `VectorRestClient`.
|
|
- Reworked `memory clear` so schema-v3 Qdrant uses scoped `kind=memory` deletion, while legacy transports keep the pre-task direct-sync path instead of advertising a nonexistent RPC.
|
|
- Tightened the qdrant-only memory-clear regression to assert the exact `("memory", ["memory"])` delete scope.
|
|
|
|
RED evidence:
|
|
|
|
- Re-review found a transport contract mismatch in fix round 1:
|
|
- `ThothHttpVectorStore` exposed `delete_kinds(...)`
|
|
- `VectorRestClient` exposed `delete_kinds(...)`
|
|
- but the legacy HTTP writer migration only allowlists `delete_vector_generation`, not `delete_vector_kinds`
|
|
- The new regressions added in this round capture that mismatch and the missing qdrant delete-scope assertion:
|
|
- `tests/test_vector_port_contract.py::test_http_store_supports_writer_without_reader`
|
|
- `tests/l0/test_vector_adapter_parity.py::test_http_rest_client_does_not_advertise_nonexistent_delete_kinds_rpc`
|
|
- `tests/test_qdrant_cli_commands.py::test_memory_clear_accepts_qdrant_only_runtime_config`
|
|
|
|
GREEN evidence:
|
|
|
|
- Focused regression suite:
|
|
- `cd harness && .venv/bin/pytest tests/test_qdrant_cli_commands.py tests/test_vector_port_contract.py tests/l0/test_vector_adapter_parity.py tests/test_adapter_command_regressions.py -q`
|
|
- Result: `53 passed`
|
|
- Broader relevant vector/memory/search suite:
|
|
- `cd harness && .venv/bin/pytest tests/test_qdrant_cli_commands.py tests/test_adapter_command_regressions.py tests/test_vector_port_contract.py tests/l0/test_vector_adapter_parity.py tests/test_solved_search_cli.py tests/test_qdrant_vector_store.py tests/test_search_similar_kinds.py tests/test_corpus_pipeline.py -q`
|
|
- Result: `135 passed`
|
|
- Ruff on the changed fix surface:
|
|
- `cd harness && .venv/bin/ruff check tht/cli/memory_cmd.py tht/ports/vector.py tht/adapters/vector/thoth_http.py tht/vectorstore/rest_client.py tests/test_qdrant_cli_commands.py tests/test_vector_port_contract.py tests/l0/test_vector_adapter_parity.py`
|
|
- Result: clean
|
|
|
|
Notes:
|
|
|
|
- Legacy HTTP/vector-rest deployments do not gain a new destructive RPC surface from this fix; they keep their previous behavior and continue to fail closed for unsupported cleanup.
|
|
- The broader suite still emits the same pre-existing deprecation and serializer warnings already noted above; this round did not introduce new warnings.
|
|
|
|
## Fix round 3 (2026-08-08)
|
|
|
|
Scope:
|
|
|
|
- Added an adapter-level Qdrant regression for mixed semantic kinds within one workspace plus a second workspace memory point.
|
|
- Verified that `delete_kinds("memory", ["memory"])` emits the real adapter filter with both `workspace_id=demo` and `record_kind=memory`.
|
|
- Verified that non-memory semantic kinds in the same workspace and memory from another workspace survive the delete.
|
|
|
|
RED evidence:
|
|
|
|
- Re-review identified a test gap rather than a confirmed runtime bug:
|
|
- existing coverage asserted only the CLI mock call shape for qdrant memory clear
|
|
- there was no adapter-level regression proving the real Qdrant delete filter and resulting fake-Qdrant state across mixed semantic kinds/workspaces
|
|
- Added regression:
|
|
- `tests/test_qdrant_vector_store.py::test_delete_kinds_is_workspace_scoped_and_preserves_other_semantic_kinds`
|
|
|
|
GREEN evidence:
|
|
|
|
- Requested focused suite:
|
|
- `cd harness && .venv/bin/pytest tests/test_qdrant_vector_store.py tests/test_qdrant_cli_commands.py tests/test_semantic_kind_isolation.py -q`
|
|
- Result: `18 passed`
|
|
- Ruff on changed files:
|
|
- `cd harness && .venv/bin/ruff check tests/test_qdrant_vector_store.py`
|
|
- Result: clean
|
|
|
|
Notes:
|
|
|
|
- This round required no production change; the new adapter regression passed against the existing Qdrant implementation.
|
|
- The focused suite still emits the same pre-existing `testcontainers.postgres` deprecation warning from `tests/conftest.py`; no new warnings were introduced.
|