docs: document internal semantic infrastructure
This commit is contained in:
+30
-1
@@ -1,8 +1,37 @@
|
||||
# ThothII — Project State
|
||||
|
||||
> Starting-point snapshot for new sessions. Last updated: 2026-08-05 (Task 13 fix round 3/5).
|
||||
> Starting-point snapshot for new sessions. Last updated: 2026-08-08 (Task 11 documentation and state update).
|
||||
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
||||
|
||||
## Internal Qdrant + Ollama semantic infrastructure — LIVE 2026-08-08
|
||||
|
||||
- **Compose topology.** The mandatory application stack is `frontend`, `core`, `qdrant`,
|
||||
`embedding`, and the one-shot `embedding-model-init`. Startup is CPU-first by default; Linux
|
||||
hosts may opt into GPU exposure with `THOTH_ENABLE_EMBEDDING_GPU=1`. Qdrant is private on the
|
||||
Compose network and persists `/qdrant/storage` in `qdrant-data`. Ollama persists its local model
|
||||
cache in `embedding-models`, and `embedding-model-init` blocks `core` until
|
||||
`qwen3-embedding:0.6b` is present.
|
||||
- **Semantic contract.** Internal semantic indexing is fixed to `qwen3-embedding:0.6b`,
|
||||
`1024` dimensions, and cosine distance. Schema-v3 descriptors are operational; schema-v1/v2 descriptors remain `migration_required` until an explicit reviewed migration writes schema version 3. One workspace owns one Qdrant collection, and schema, Evidence, and Memory records
|
||||
coexist inside that collection with payload `kind` separation.
|
||||
- **Boundary and persistence.** Only DWH and LLM remain external runtime application endpoints.
|
||||
There are no active external vector or embedding endpoint instructions, bindings, or secrets in
|
||||
the supported operator manuals. Qdrant remains a derived but persistent semantic index: the
|
||||
canonical sources of truth stay the workspace Git descriptors, phase artifacts, and memory
|
||||
registry/ledger. The Ollama model cache is recoverable for offline startup but is not the
|
||||
canonical source of semantic content.
|
||||
- **Backup and recovery.** `./scripts/vector-backup.sh --project-name <name> --output <file>`
|
||||
archives exactly one labeled `<project>_qdrant-data` volume and preserves the prior `qdrant`
|
||||
running state. `./scripts/vector-restore.sh --project-name <name> --input <file>
|
||||
--confirm-project <name>` requires the exact repeated project confirmation, validates manifest
|
||||
and archive safety before stopping `qdrant`, stages rollback content, restores in place, and
|
||||
restarts `qdrant` only if it was previously running. Restore does not migrate legacy workspace
|
||||
descriptors, rename collections, or repair a semantic-index incompatibility.
|
||||
- **Verification recorded for this docs/state update.** The installation-manual contract tests
|
||||
now require the four-service-plus-init topology, the fixed internal model/dimensions, explicit
|
||||
schema-v3 migration messaging, Qdrant collection ownership, Qdrant backup/restore safety, and
|
||||
the absence of active external vector/embedding operator bindings from current manuals.
|
||||
|
||||
## Unified deployment release gate — Task 13 (2026-08-05)
|
||||
|
||||
- **Release coverage.** `scripts/unified-deployment-smoke.sh` gates the two-service render/build,
|
||||
|
||||
Reference in New Issue
Block a user