180 lines
7.3 KiB
Markdown
180 lines
7.3 KiB
Markdown
# Components, modules, and flows
|
|
|
|
This page complements the [architecture overview](overview.md) with the module structure and flows through ThothII. The diagrams describe the current code, not a future architecture.
|
|
|
|
## Modules and dependencies
|
|
|
|
The frontend communicates with the backend through REST and SSE. The backend does not own session
|
|
persistence: it starts Pi, invokes the `tht` CLI, and forwards events. It does own the separate
|
|
installation-local database catalog. The harness contains the workflow, the Python CLI, and
|
|
adapters for the DWH and vector store.
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
FE["frontend/\nReact + Vite"] -->|REST + SSE| BE["backend/\nFastify + TypeScript"]
|
|
BE -->|RPC stdin/stdout| PI["Pi\n--mode rpc"]
|
|
BE -->|subprocess\nJSON stdout| THT["harness/tht\nCLI Python"]
|
|
PI --> EXT["harness/.pi/extensions/\ntht-gate.js"]
|
|
EXT --> SKILL["harness/.pi/skills/\ntht-sessione"]
|
|
EXT --> THT
|
|
THT --> FS["Sessions and artifacts\nworkspace repository"]
|
|
THT --> DWH["DWH\nread-only"]
|
|
THT --> VDB["Qdrant / vector store"]
|
|
BE --> CFG["settings.json\nworkspace registry"]
|
|
BE --> CAT["catalog-db\nPostgreSQL + Kysely"]
|
|
BE -->|catalog Test + Table Sync| DWH
|
|
FE -.->|renders widgets| EXT
|
|
```
|
|
|
|
Dipendenze principali:
|
|
|
|
| Module | Depends on | Responsibility |
|
|
| --- | --- | --- |
|
|
| `frontend/` | Backend REST and SSE APIs | UI, gate widgets, and in-memory transcript |
|
|
| `backend/src/` | Pi, `tht`, configuration, workspace registry, catalog PostgreSQL, and read-only DWH connectors | Transport, session lifecycle, catalog CRUD, connection tests, table introspection, and APIs |
|
|
| `harness/.pi/` | Pi and `tht phase` | Workflow orchestration and human-in-the-loop gates |
|
|
| `harness/tht/` | Filesystem, DWH, and vector store | Persistence, CLI, Evidence, schema, and preprocessing |
|
|
| workspace repository | `source/`, `curated/`, manifest, and artifacts | Versioned Evidence source and session output |
|
|
|
|
## Session sequence
|
|
|
|
The main path starts with a user question and ends with an SSE event. Reviewer decisions use the same channel and are persisted by the harness.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
actor U as User or reviewer
|
|
participant FE as Frontend
|
|
participant BE as Backend
|
|
participant PI as Pi RPC
|
|
participant THT as CLI tht
|
|
participant WS as Workspace
|
|
participant DWH as DWH
|
|
|
|
U->>FE: Send question or gate decision
|
|
FE->>BE: POST session / risposta widget
|
|
BE->>PI: RPC input o prompt di resume
|
|
PI->>THT: phase/session/evidence commands
|
|
THT->>WS: Read and write phase artifacts
|
|
THT->>DWH: Introspection or read-only query
|
|
DWH-->>THT: Schema, results, or diagnostics
|
|
THT-->>PI: JSON and phase state
|
|
PI-->>BE: RPC events and widget descriptor
|
|
BE-->>FE: SSE text_delta, info, ui_request
|
|
FE-->>U: Text, artifact, or review request
|
|
```
|
|
|
|
The backend uses `ThtRunner` for CLI subprocesses, `PiProcessManager` for one Pi process per session, `SessionBridge` to adapt RPC events, and `SseHub` to distribute them to clients.
|
|
|
|
## Main backend classes
|
|
|
|
The diagram shows the classes that form the bridge between the browser, Pi, and `tht`. Fastify routes receive requests and delegate to these services.
|
|
|
|
```mermaid
|
|
classDiagram
|
|
class ThtRunner {
|
|
+buildArgv(command, args) string[]
|
|
+run(args) Promise~ThtResult~
|
|
+sessionShow(id) Promise~unknown~
|
|
}
|
|
class PiProcessManager {
|
|
-runtimes Map
|
|
+spawnFor(sessionId, mode) SessionRuntime
|
|
+resume(sessionId, tht) Promise~SessionRuntime~
|
|
+stop(sessionId) Promise~void~
|
|
}
|
|
class SessionBridge {
|
|
+handleRpcEvent(event) ClientEvent
|
|
+handleUiResponse(response) Promise~void~
|
|
}
|
|
class SseHub {
|
|
+subscribe(sessionId) AsyncIterable
|
|
+publish(sessionId, event) void
|
|
+close(sessionId) void
|
|
}
|
|
class SessionRoutes {
|
|
+createSession(request) Response
|
|
+resumeSession(id) Response
|
|
+postInput(id, input) Response
|
|
}
|
|
class WorkspaceRegistry {
|
|
+list() Workspace[]
|
|
+resolve(id) Workspace
|
|
}
|
|
class SettingsStore {
|
|
+get() Settings
|
|
+update(patch) Settings
|
|
}
|
|
class App {
|
|
+buildApp() FastifyInstance
|
|
}
|
|
|
|
App --> SessionRoutes
|
|
App --> WorkspaceRegistry
|
|
App --> SettingsStore
|
|
SessionRoutes --> PiProcessManager
|
|
SessionRoutes --> ThtRunner
|
|
SessionRoutes --> SseHub
|
|
PiProcessManager --> SessionBridge
|
|
PiProcessManager --> ThtRunner
|
|
SessionBridge --> SseHub
|
|
```
|
|
|
|
## Python modules in the `tht` CLI
|
|
|
|
The CLI consists of Typer commands and domain modules. `cli/` turns arguments into operations; `evidence/`, `session/`, `db/`, `adapters/`, and the other packages contain the application logic.
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
MAIN["tht/cli/__init__.py"] --> CMD["tht/cli/*_cmd.py"]
|
|
CMD --> CONFIG["config.py\nworkspace.py\npaths.py"]
|
|
CMD --> SESSION["session_cmd.py\nsession/"]
|
|
CMD --> EVIDENCE["evidence_cmd.py\nevidence/"]
|
|
CMD --> PRE["preprocess_cmd.py\nevidence/corpus/"]
|
|
CMD --> PHASE["phase_cmd.py\nphase.py\nworkflow.py"]
|
|
CMD --> SQL["sql_cmd.py\ndb/\nrest/"]
|
|
EVIDENCE --> ACQ["evidence/acquisition.py\nadapters/ filesystem/http/s3"]
|
|
EVIDENCE --> CANON["evidence/canonical.py\ncontracts.py\nmodel.py"]
|
|
EVIDENCE --> AUTHOR["evidence/authoring.py"]
|
|
PRE --> PIPE["evidence/corpus/pipeline.py\nchunk.py normalize.py store.py"]
|
|
PRE --> VECTOR["adapters/vector/qdrant.py"]
|
|
PRE --> DWH["jobs/dwh_pipeline.py\nadapters/dwh/"]
|
|
SESSION --> REPO["session/filesystem_repository.py\npostgres_repository.py"]
|
|
PHASE --> LEDGER["decisions.py\nreview_decisions"]
|
|
```
|
|
|
|
The operator command `tht` in `tools/tht/` is separate from the harness Python CLI. The former handles installation, lifecycle, authentication, and workspaces; the latter runs the workflow and data operations.
|
|
|
|
## Eight-phase workflow and gates
|
|
|
|
The source of truth is `harness/workflow.yaml`. The current phase is computed from the decision ledger, not from a manually updated field.
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
F1["F1\nChiarimento"] --> F2["F2\nMemoria"]
|
|
F2 --> F3["F3\nRiscrittura"]
|
|
F3 --> F4["F4\nSchema linking\nreviewer_decide"]
|
|
F4 --> F5["F5\nSintesi"]
|
|
F5 --> F6["F6\nCTE\nauto o skip"]
|
|
F6 --> F7["F7\nSQL finale\nreviewer_confirm"]
|
|
F7 --> F8["F8\nDatamart\nreviewer_decide"]
|
|
F1 -.->|reviewer_confirm| F1
|
|
F3 -.->|reviewer_confirm| F3
|
|
F4 -.->|decisioni su tabelle, colonne, evidence| F4
|
|
F6 -.->|cte_approved o cte_rejected| F6
|
|
F7 -.->|sql_approved o sql_rejected| F7
|
|
F8 -.->|datamart_requested o declined| F8
|
|
```
|
|
|
|
| Fase | Nome | Avanzamento | Artefatti principali |
|
|
| --- | --- | --- | --- |
|
|
| F1 | chiarimento | `kind:phase` | decisioni di chiarimento |
|
|
| F2 | memoria | automatico se vuota | decisioni memoria |
|
|
| F3 | riscrittura | `kind:phase` | `question.md` |
|
|
| F4 | schema linking | `reviewer_decide` | `schema_linking.json` |
|
|
| F5 | sintesi | `kind:phase` | verifica dello schema linking |
|
|
| F6 | CTE | automatico, oppure skip | `cte_plan.json`, `ctes/`, `cte_tests.json` |
|
|
| F7 | SQL finale | `kind:phase` dopo `sql_approved` | `sql_final.sql` |
|
|
| F8 | datamart | `reviewer_decide` | decisione su richiesta o rifiuto |
|
|
|
|
Un `reviewer_select` con decisione incorporata può confermare direttamente. Un `reviewer_decide` registra le scelte multiple. Un `reviewer_confirm` conferma un artefatto o la chiusura della fase. Il modello propone; il revisore decide e il ledger registrato è la fonte dello stato.
|