refactor: retire external vector deployment

This commit is contained in:
2026-08-08 19:05:57 +02:00
parent 8f4ec1e1a3
commit 4e3fecbe8e
44 changed files with 370 additions and 1710 deletions
+11 -24
View File
@@ -2,12 +2,12 @@
ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fastify/Pi/`tht`
core. The portable deployment runs exactly two application services; data services remain
external in this profile.
external in this profile, except for the mandatory internal semantic services bundled in Compose.
## Docker Compose: local startup
Requirements: Docker Engine with Compose v2. The mandatory stack is exactly the `core` and
`frontend` application images. DWH, vector DB, embedding, and LLM services are external,
Requirements: Docker Engine with Compose v2. The mandatory stack is `frontend`, `core`,
`qdrant`, `embedding`, and the one-shot `embedding-model-init`. DWH and LLM remain external,
configurable endpoints—even when they are co-located with ThothII.
From a fresh clone, run these commands from the repository root:
@@ -42,9 +42,7 @@ runtime endpoint and secret bindings remain installation-local. Open
loopback port).
Credentials and certificates are local protected files. Do not put them in environment examples,
workspace YAML, URLs, or Compose interpolation values. The optional `local-vector` and
preprocessing overlays are development presets; they do not change the two-service mandatory
stack or the external-endpoint contract.
workspace YAML, URLs, or Compose interpolation values.
Application state is split across the named `settings`, `pi-state`, `workspace-registry`, and
`sessions` volumes. `docker compose down` keeps them. Only an explicit destructive command such
@@ -172,29 +170,20 @@ Startup mode adds bounded image build/two-service health startup, installation-a
status, stopped-container-aware ownership checks, and exact cleanup. The ordinary hosted Windows
job remains deterministic and does not claim Docker startup.
## Optional local pgvector and recovery
The local-vector overlay reads `THT_VECTOR_BOOTSTRAP_PASSWORD`,
`THT_VECTOR_MIGRATOR_PASSWORD`, `THT_VECTOR_READER_PASSWORD`, and
`THT_VECTOR_WRITER_PASSWORD` from the same bundle. Its `vector_data` volume is independent of
application state; passwords are selected at runtime and are never passed as URL arguments.
## Preprocessing jobs and S3 Evidence
The included job workspaces target the optional local-vector profile. Put the four local-vector
password keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then
run the explicit preprocessing preset:
The included preprocessing services reuse the internal Qdrant/Ollama stack. Mount Evidence at
`/data/source/evidence`, then run the explicit preprocessing preset:
```sh
docker compose --env-file deploy/env/local.env \
-f compose.yaml -f deploy/compose.local.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \
--profile local-vector --profile preprocess run --rm preprocess-evidence
-f compose.yaml -f deploy/compose.local.yaml \
-f deploy/compose.preprocess.yaml --profile preprocess run --rm preprocess-evidence
```
Replace the final service with `preprocess-dwh` when required. The overlay makes each job wait for the vector
database health check, role reconciliation, and a successful migration; no separate database
startup or migration command is required.
service health checks and embedding model initialization; no separate semantic-service startup is
required.
S3 Evidence uses the optional `tht[s3]` dependency and canonical `s3://bucket/key` provenance.
AWS endpoints are used when no custom URL is supplied. Every custom endpoint is an explicit egress
@@ -235,9 +224,7 @@ database in the active cluster cannot bypass the guard. It refuses a non-empty t
```
After restore, run `tht vector migrate --status --json`, adapter health, and a known retrieval
query against the target before changing any deployment endpoint. Never test recovery against the
active `vector_data` volume. `./scripts/local-vector-smoke.sh --backup-restore` performs this drill
with disposable source and target volumes.
query against the target before changing any migration/export endpoint.
## Production trust boundary and secrets