mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: implement metadata catalog database management

Browse Source
This commit is contained in:
Codex
2026-08-27 22:43:54 +02:00
parent 705af3aeb2
commit 79c4c925b5
86 changed files with 12566 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/overview.md
+13 -4
View File
@@ -11,6 +11,7 @@ For sessions, roles, groups, diagnostics, and recovery, see the [authentication
flowchart LR
USER["Reviewer"] --> FE["Frontend\nReact and SSE"]
FE --> BE["Backend\nFastify"]
BE --> CATALOG["Metadata catalog\nPostgreSQL"]
BE --> PI["Pi\nRPC per sessione"]
PI --> THT["tht and harness\nworkflow and persistence"]
THT --> DWH["DWH\nread only"]
@@ -27,8 +28,8 @@ frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness →
| Layer | Stack | Ruolo |
|---|---|---|
| **harness/** | Python (`tht` CLI) + Pi gate extension (JS) | Owns the workflow and **all** persistence |
| **backend/** | Fastify + TypeScript | Thin bridge with no database of its own |
| **harness/** | Python (`tht` CLI) + Pi gate extension (JS) | Owns the workflow and all session persistence |
| **backend/** | Fastify + TypeScript + Kysely | Session bridge plus the isolated administrative metadata catalog |
| **frontend/** | React 18 + Vite | UI that renders gate widgets and rebuilds the live transcript from the SSE stream |
## The harness owns the workflow
@@ -39,14 +40,22 @@ frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness →
A session is a directory under `sessions/` (the workspace defines the path): `session_manifest.yaml`, phase artifacts (`question.md`, `schema_linking.json`, `sql_final.sql`, and others), and `review_decisions.jsonl`. The contract says: *"persisted state is the truth; what is not recorded did not happen"*. There is no verbatim transcript store. A resumed Pi process rebuilds context from `tht session show <id>` and the artifacts on disk.
## The backend is a thin bridge with no database
## The backend bridges sessions and owns the metadata catalog
- `ThtRunner` runs `tht` subcommands in a shell.
- `PiProcessManager` runs one Pi child process per session and bridges its RPC stream.
- `SessionBridge` maps Pi RPC events to client events (`ui_request` / `text_delta` / `info`).
- `SseHub` distributes these events to the browser over SSE.
- `CatalogService` joins authoritative YAML workspace identities with installation-local database
configurations stored in PostgreSQL through Kysely.
- `CatalogTableService` reconciles persisted Catalog Tables with a successful external schema scan;
`ConcreteCatalogTableIntrospector` isolates direct PostgreSQL, typed REST, and SSH-tunnel access.
Application settings live in a JSON file (`backend/data/settings.json`), not in a database.
Application settings remain in `backend/data/settings.json`; session state remains in harness phase
documents. PostgreSQL stores only the administrative database catalog, bindings, observed tables,
and curated descriptions. Connector secrets remain write-only in the encrypted workspace secret
store. Catalog SSH support is limited to connection tests and table synchronization; it does not
change the session runtime binding contract.
## Human-in-the-loop gate contract