mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: implement metadata catalog database management

Browse Source
This commit is contained in:
Codex
2026-08-27 22:43:54 +02:00
parent 705af3aeb2
commit 79c4c925b5
86 changed files with 12566 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+25 -11
View File
@@ -1,15 +1,16 @@
# ThothII
ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fastify/Pi/`tht`
core. The portable deployment runs exactly two application services; data services remain
external in this profile, except for the mandatory internal semantic services bundled in Compose.
core. The portable deployment runs two application services plus the installation-local metadata
catalog; DWH and LLM services remain external. Semantic services are bundled in Compose.
Authentication is configured through the single host CLI tht: see the [local authentication guide](docs/install/authentication-local.md),
[generic OIDC guide](docs/install/authentication-oidc.md), and [manual acceptance matrix](docs/testing/authentication-manual-acceptance.md).
## Docker Compose: local startup
Requirements: Docker Engine with Compose v2. The mandatory stack is `frontend`, `core`, `qdrant`, `embedding`, and the one-shot `embedding-model-init`. DWH and LLM remain external,
Requirements: Docker Engine with Compose v2. The mandatory stack is `frontend`, `core`,
`catalog-db`, `qdrant`, `embedding`, and the one-shot `embedding-model-init`. DWH and LLM remain external,
configurable endpoints—even when they are co-located with ThothII.
From a fresh clone, run these commands from the repository root:
@@ -17,17 +18,28 @@ From a fresh clone, run these commands from the repository root:
```sh
cp deploy/env/local.env.example deploy/env/local.env
# Edit deploy/env/local.env, including PI_AUTH_FILE, THT_SECRETS_FILE, and external endpoints.
docker compose --env-file deploy/env/local.env \
-f compose.yaml -f deploy/compose.local.yaml up --build -d
./scripts/run-stack.sh
```
`./scripts/run-stack.sh` runs this same base+local command in the foreground. The core image
contains its Pi runtime; no host `pi` executable is used. For a server installation:
The launcher builds the core, starts `catalog-db`, runs the explicit one-shot Kysely migrations,
then runs the base+local stack in the foreground. Migrations never run implicitly in backend
startup. The core image contains its Pi runtime; no host `pi` executable is used. For a server
installation, build the image, start the catalog, and run the same migration service before the
application rollout:
```sh
cp deploy/env/server.env.example deploy/env/server.env
# Edit all absolute storage, Pi/secret/session files, and endpoint paths.
sudo scripts/prepare-server-pi-state.sh /srv/thothii/pi-state 10001 10001
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example build core
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example up -d catalog-db
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example run --rm catalog-migrate
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example up --build -d
@@ -101,10 +113,12 @@ schema, Evidence, and Memory records share that collection and stay separated by
`kind`.
<!-- workspace-descriptor-contract:end -->
Connector `ssh_tunnel` bindings are diagnostic-only in this release: their bounded probe always
cleans up the loopback forward and returns `workspace_not_activatable`; session creation is rejected
before persistence. Git registry access over SSH is unaffected. Use direct or REST connector
transport for runtime sessions.
For NL→SQL runtime sessions, connector `ssh_tunnel` bindings remain diagnostic-only: their bounded
probe cleans up the loopback forward and returns `workspace_not_activatable`; session creation is
rejected before persistence. Database management is a separate boundary and supports a strict
OpenSSH tunnel for **Test connection** and **Sync tables**, using a private key, optional passphrase,
mandatory `known_hosts`, and optional PostgreSQL TLS CA/server name. Git registry access over SSH is
unaffected. Use direct or REST connector transport for runtime sessions.
`docker-compose.dev.yml` is deliberately local: both published ports bind to `127.0.0.1`,
`THT_SESSION_STORAGE=local`, and `THT_HOME=/data/local-home`. Do not set