From 7c09b9842fd893e3adf3ee47630563172d452e4d Mon Sep 17 00:00:00 2001 From: mptyl Date: Sat, 8 Aug 2026 22:15:06 +0200 Subject: [PATCH] docs: record qdrant ollama verification --- .../task-13-implementation.md | 123 ++++++++++++++++++ PROJECT_STATE.md | 33 ++++- 2 files changed, 151 insertions(+), 5 deletions(-) create mode 100644 .superpowers/sdd/2026-08-08-internal-qdrant-ollama/task-13-implementation.md diff --git a/.superpowers/sdd/2026-08-08-internal-qdrant-ollama/task-13-implementation.md b/.superpowers/sdd/2026-08-08-internal-qdrant-ollama/task-13-implementation.md new file mode 100644 index 00000000..a358a488 --- /dev/null +++ b/.superpowers/sdd/2026-08-08-internal-qdrant-ollama/task-13-implementation.md @@ -0,0 +1,123 @@ +# Task 13 Implementation Report + +## Status + +DONE_WITH_CONCERNS + +## Changes + +- Updated stale harness/backend/frontend tests and fixtures to the Task 13 internal Qdrant/Ollama contract. +- Made `deploy/workspaces/psd.yaml.example` generic while preserving schema-v3 Qdrant/Ollama shape. +- Fixed `scripts/workspace-registry-smoke.sh` to pass the required legacy migration `--collection` and prove exact Docker cleanup, including its smoke image. +- Updated `PROJECT_STATE.md` with only evidence observed in this run. + +Changed files: + +- `PROJECT_STATE.md` +- `backend/test/routes-workspaces.test.ts` +- `backend/test/workspace-runtime-handoff.test.ts` +- `backend/test/workspaces-contracts.test.ts` +- `backend/test/workspaces-git-repository.test.ts` +- `deploy/workspaces/psd.yaml.example` +- `frontend/src/shell/NewSessionDialog.test.tsx` +- `harness/tests/test_adapter_command_regressions.py` +- `harness/tests/test_workspace.py` +- `scripts/task13-runtime-fixture-check.ts` +- `scripts/test-verify-workspace-install-docs.sh` +- `scripts/workspace-registry-smoke.sh` + +## Verification + +Deterministic gates: + +- `cd harness && .venv/bin/pytest -q && .venv/bin/ruff check .` + - Initial red: 2 harness pytest failures. + - After fixture fixes: harness pytest passed `819 passed, 4 deselected, 74 warnings in 27.73s`. + - Ruff still failed with `Found 220 errors`; treated as existing unrelated debt. + - Touched harness files verified clean with `cd harness && .venv/bin/ruff check tests/test_adapter_command_regressions.py tests/test_workspace.py && .venv/bin/pytest -q tests/test_adapter_command_regressions.py::test_solved_index_writes_through_writer_only_factory_store tests/test_workspace.py::test_load_workspace_expands_env_vars`: `All checks passed!` and `2 passed, 2 warnings in 0.14s`. +- `cd backend && npx vitest run && npx tsc --noEmit -p . && npm run build` + - Initial red: 4 backend Vitest failures. + - After fixes: `Test Files 39 passed (39)`, `Tests 464 passed (464)`, TypeScript passed, build passed. +- `cd frontend && npx vitest run && npx tsc -b && npm run build` + - Initial red: 1 frontend Vitest failure. + - After fix: frontend Vitest passed `374/374`, TypeScript passed, build passed with Vite `built in 6.55s`. +- `git diff --check` + - Passed with no output. + +Focused reruns: + +- `cd backend && npx vitest run test/workspaces-migrate-legacy.test.ts test/workspaces-contracts.test.ts test/routes-workspaces.test.ts test/workspace-runtime-handoff.test.ts test/workspaces-git-repository.test.ts && cd .. && ./scripts/test-no-deployment-coupling.sh && ./scripts/verify-workspace-install-docs.sh --fixtures-only && git diff --check` + - `Test Files 5 passed (5)`, `Tests 35 passed (35)`. + - Coupling guard passed: `no active retired deployment or external semantic coupling found.` + - Install docs fixtures passed through `relative secret-source fixture rejected passed`. + +Deployment contracts: + +- `./scripts/test-default-compose.sh && ./scripts/test-unified-compose.sh && ./scripts/test-internal-semantic-compose.sh && ./scripts/test-no-deployment-coupling.sh && ./scripts/test-compose-secret-policy.sh && ./scripts/verify-workspace-install-docs.sh --fixtures-only` + - Passed. Output included: + - `default Compose contract passed.` + - `unified Compose contract passed.` + - `internal semantic Compose/script contracts passed.` + - `no active retired deployment or external semantic coupling found.` + - `Compose secret policy passed.` + - install-doc fixture checks through `relative secret-source fixture rejected passed`. + +Docker smokes: + +- `/usr/bin/time -p ./scripts/internal-semantic-smoke.sh` + - Passed: `Task 13 internal semantic smoke passed.` + - Cleanup proof: `no labeled containers, volumes, networks, or images remain for 20260808200245-83368-17823.` + - Duration: `real 217.34`. +- `/usr/bin/time -p ./scripts/workspace-registry-smoke.sh` + - Initial red: `usage: migrate-legacy --input --output --collection [--id ]`. + - After fix: `workspace registry smoke passed`. + - Cleanup proof: `no compose containers, volumes, networks, or image remain for thoth-workspace-registry-smoke-89671.` + - Duration: `real 9.93`. +- `/usr/bin/time -p ./scripts/unified-deployment-smoke.sh` + - Passed: `Task 13 full deployment smoke passed.` + - Cleanup proof: `no labeled containers, volumes, networks, or images remain for 20260808200706-85638-13391.` + - Duration: `real 125.57`. +- `/usr/bin/time -p ./scripts/thothctl-update-smoke.sh` + - Passed: `Task 13 update deployment smoke passed.` + - Cleanup proof: `no labeled containers, volumes, networks, or images remain for 20260808200918-87340-10404.` + - Duration: `real 85.40`. +- `/usr/bin/time -p ./scripts/server-deployment-smoke.sh` + - Passed: `Task 13 Linux server deployment smoke passed.` + - Cleanup proof: `no labeled containers, volumes, networks, or images remain for 20260808201047-88645-20675.` + - Duration: `real 55.99`. + +Final audit: + +- `rg -n "pgvector|local-vector|THT_VECTOR_|EMBEDDING_BASE_URL|openai_compatible|ollama_compatible" . --glob '!docs/plans/**' --glob '!docs/superpowers/**' --glob '!**/node_modules/**' --glob '!**/.venv/**' --glob '!**/.git/**'` + - Returned matches in legacy schema-v1/v2 support, migration tests, negative guards, historical notes, and older harness docs/code. + - This remains a concern: the audit is not clean under the brief's strict expected outcome. +- `git status --short` + - Before report/commit, contained only intentional Task 13 changes. + +## Image and Host Evidence + +- Host CPU: `Apple M4 Pro`. +- Host OS: `Darwin MacProM4-di-Marco.local 25.5.0 Darwin Kernel Version 25.5.0: Tue Jun 9 22:28:34 PDT 2026; root:xnu-12377.121.10~1/RELEASE_ARM64_T6041 arm64`. +- Docker server: `29.6.2 linux/arm64`. +- Verified pinned images: + - `qdrant/qdrant:v1.18.2@sha256:75eab8c4ba42096724fdcfde8b4de0b5713d529dde32f285a1f86fdcb2c9e50c`. + - `ollama/ollama:0.32.0@sha256:57f573b47f1f71ebb445789f279fe3e596a8beab182f7cf486db9205bad87c5a`. +- Workspace registry smoke ephemeral image: + - Manifest list: `sha256:4d056bf2cb38d0e8ede91fbf121df1f9f18caee0d401581618ccef9ed8a55e73`. + - Config: `sha256:613f8fb28c0517adee4085f41bc447f2c3813b0fdbb7b26624bfb4cb192b6fd8`. + - Removed during cleanup. + +## Manual Gates + +- GPU exposure gate (`THOTH_ENABLE_EMBEDDING_GPU=1` on Linux): not executed in this run. +- Windows Docker Desktop startup/manual job: not executed in this run. + +## Commits + +- Pending at report creation. + +## Known Limitations + +- Broad harness Ruff remains existing unrelated debt: `Found 220 errors`. +- Final active-reference audit is not clean; it still finds legacy/negative-guard references outside explicit migration fixture files. +- Ephemeral Task 13 core/frontend image IDs from `internal-semantic-smoke.sh`, `unified-deployment-smoke.sh`, `thothctl-update-smoke.sh`, and `server-deployment-smoke.sh` were removed by exact cleanup and were not emitted in stdout; pinned Qdrant/Ollama digests and the workspace-registry smoke image digest were captured. diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 40c0013f..490fd1d3 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -1,6 +1,6 @@ # ThothII — Project State -> Starting-point snapshot for new sessions. Last updated: 2026-08-08 (Task 11 documentation and state update). +> Starting-point snapshot for new sessions. Last updated: 2026-08-08 (Task 13 verification audit). > Point a fresh session here ("read PROJECT_STATE.md") before substantial work. ## Internal Qdrant + Ollama semantic infrastructure — LIVE 2026-08-08 @@ -27,10 +27,33 @@ and archive safety before stopping `qdrant`, stages rollback content, restores in place, and restarts `qdrant` only if it was previously running. Restore does not migrate legacy workspace descriptors, rename collections, or repair a semantic-index incompatibility. -- **Verification recorded for this docs/state update.** The installation-manual contract tests - now require the four-service-plus-init topology, the fixed internal model/dimensions, explicit - schema-v3 migration messaging, Qdrant collection ownership, Qdrant backup/restore safety, and - the absence of active external vector/embedding operator bindings from current manuals. +- **Verification recorded for Task 13 final audit.** On Apple M4 Pro + (`Darwin 25.5.0`, Docker Server `29.6.2 linux/arm64`), harness pytest passed + **819 passed / 4 deselected**; backend Vitest passed **464/464** plus TypeScript and build; + frontend Vitest passed **374/374** plus TypeScript and build; `git diff --check` passed. + Deployment contracts passed: + `test-default-compose.sh`, `test-unified-compose.sh`, `test-internal-semantic-compose.sh`, + `test-no-deployment-coupling.sh`, `test-compose-secret-policy.sh`, and + `verify-workspace-install-docs.sh --fixtures-only`. +- **Task 13 Docker smoke evidence.** CPU semantic smoke passed in **217.34s** and proved + offline Qdrant/Ollama persistence plus exact cleanup. Workspace registry smoke passed in + **9.93s** and now proves exact cleanup of compose containers, volumes, networks, and its + smoke image. Unified deployment smoke passed in **125.57s**; update-only rollback smoke + passed in **85.40s**; Linux server deployment smoke passed in **55.99s**. The previously + observed `thothctl` rollback failure did not recur. +- **Task 13 image and manual-gate notes.** Verified pinned runtime images: + `qdrant/qdrant:v1.18.2@sha256:75eab8c4ba42096724fdcfde8b4de0b5713d529dde32f285a1f86fdcb2c9e50c` + and + `ollama/ollama:0.32.0@sha256:57f573b47f1f71ebb445789f279fe3e596a8beab182f7cf486db9205bad87c5a`. + The workspace-registry smoke built ephemeral manifest list + `sha256:4d056bf2cb38d0e8ede91fbf121df1f9f18caee0d401581618ccef9ed8a55e73` + and removed it during cleanup. Local GPU exposure (`THOTH_ENABLE_EMBEDDING_GPU=1`) and + Windows Docker Desktop startup were not manually executed in this run. +- **Task 13 known limitations.** Broad harness Ruff remains existing unrelated debt + (**220 errors**); touched harness files were verified Ruff-clean. The final active-reference + audit still reports legacy schema-v1/v2 parsing/migration, negative guards, and historical + notes containing `pgvector`, `THT_VECTOR_*`, `EMBEDDING_BASE_URL`, `openai_compatible`, or + `ollama_compatible`; those hits were not all eliminated by this verification task. ## Historical snapshots and archived reference notes