# Architecture overview

> For details about modules and flows, see [Components, modules, and flows](components.md).

ThothII is a **human-in-the-loop datamart builder**. It turns a natural-language question into validated SQL, and optionally a dbt datamart, through a **deterministic eight-phase NL-to-SQL workflow** in which the model *proposes* and a human reviewer *decides* at gates.

Production authentication uses local authentication or generic OIDC. `tht` is the only operator CLI.
For sessions, roles, groups, diagnostics, and recovery, see the [authentication documentation](authentication.md).

```mermaid
flowchart LR
    USER["Reviewer"] --> FE["Frontend\nReact and SSE"]
    FE --> BE["Backend\nFastify"]
    BE --> PI["Pi\nRPC per sessione"]
    PI --> THT["tht and harness\nworkflow and persistence"]
    THT --> DWH["DWH\nread only"]
    THT --> EVIDENCE["Evidence\ncurated corpus"]
    EVIDENCE --> THT
    THT --> FE
```

## The three independent projects

```
frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness → DWH (read-only)
```

| Layer | Stack | Ruolo |
|---|---|---|
| **harness/** | Python (`tht` CLI) + Pi gate extension (JS) | Owns the workflow and **all** persistence |
| **backend/** | Fastify + TypeScript | Thin bridge with no database of its own |
| **frontend/** | React 18 + Vite | UI that renders gate widgets and rebuilds the live transcript from the SSE stream |

## The harness owns the workflow

`tht` (Python) is a deterministic CLI. `harness/.pi/extensions/tht-gate.js` is a Pi extension that guides the eight-phase workflow. `harness/workflow.yaml` is the single source of workflow truth, and `harness/.pi/skills/tht-sessione/SKILL.md` contains the orchestration rules the model must follow. The "current phase" is **not stored**. It is computed by folding the decision ledger (`harness/tht/phase.py`), which must be read before reasoning about phase logic.

## Persistence means phase documents, not chat

A session is a directory under `sessions/` (the workspace defines the path): `session_manifest.yaml`, phase artifacts (`question.md`, `schema_linking.json`, `sql_final.sql`, and others), and `review_decisions.jsonl`. The contract says: *"persisted state is the truth; what is not recorded did not happen"*. There is no verbatim transcript store. A resumed Pi process rebuilds context from `tht session show <id>` and the artifacts on disk.

## The backend is a thin bridge with no database

- `ThtRunner` runs `tht` subcommands in a shell.
- `PiProcessManager` runs one Pi child process per session and bridges its RPC stream.
- `SessionBridge` maps Pi RPC events to client events (`ui_request` / `text_delta` / `info`).
- `SseHub` distributes these events to the browser over SSE.

Application settings live in a JSON file (`backend/data/settings.json`), not in a database.

## Human-in-the-loop gate contract

The model proposes; a human reviewer decides at gates through widgets:

- **`reviewer_select`**: single choice. An option with a `decision` payload confirms and persists directly; an option without a payload only asks.
- **`reviewer_decide`**: multiselect. Each choice is a decision.
- **`reviewer_confirm`**: artifact or phase gate.

The frontend renders these widget descriptors (registry in `src/widgets/`). It rebuilds the live transcript in memory from the SSE stream (`src/store/sessionStore.ts`); it is **not persisted**.

## Curated and immutable Evidence

The workspace repository is the publication boundary. The curator prepares `evidence/source/`,
reviews units in `evidence/curated/`, validates them, and merges them. With `evidence.schema_version: 2`,
the runtime materializes the full `evidence/` tree from the exact Git commit, but the renderer passes
only `curated/**/*.md` from the immutable revision root to preprocessing. Sources, manifests, and
evaluation data remain available for traceability. The runtime never modifies, stages, commits, or
publishes the authoring repository.

Before indexing, the curated corpus from the pinned revision is validated. The shared Qdrant
collection keeps the unnamed dense vector used by Schema and Memory. Evidence preprocessing may
add only the sparse `bm25` vector with `idf`, without deleting, renaming, or recreating the collection.
`workspace preprocess evidence` and the Evidence part of `workspace preprocess run` are the only public
operations that perform this upgrade.

## Recurring points of attention

- `tht -c`/`--config` is a **per-command** option. It must follow the subcommand, never precede it (`ThtRunner.buildArgv` enforces this).
- `--json` output must be plain JSON on stdout. It is a machine-readable contract.
- UI strings are in English. Document *content* stays in the workspace language because it is the actual data; only chrome and labels are in English.
- Each workspace defines its DWH target and working directories. Secrets remain in protected installation files, not in the workspace repository.
- Settings are global (`backend/data/settings.json`: workspace/provider/model/thinking); the new-session form asks only for the question.
- **Resume**: a resumable session returns to its last incomplete phase. The backend rejects resume with 409 when `finalized` or `archived`; `PiProcessManager.spawnFor` must send `/riprendi-sessione <id>` for resume and `/nuova-domanda` for a new session. The wrong prompt silently turns a resume into a new question.

## Starting the stack

Start the local stack with `./scripts/run-stack.sh` after creating
`deploy/env/local.env` from `deploy/env/local.env.example`. The Compose core includes Pi; DWH,
the vector database, embeddings, and the LLM are external endpoints configured in the local file.