docs: add PROJECT_STATE.md orientation snapshot
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -0,0 +1,114 @@
|
||||
# ThothII — Project State
|
||||
|
||||
> Starting-point snapshot for new sessions. Last updated: 2026-06-29.
|
||||
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
||||
|
||||
## What ThothII is
|
||||
|
||||
A **human-in-the-loop datamart builder**: it turns a natural-language question into
|
||||
validated SQL (and optionally a dbt datamart) through a deterministic **8-phase
|
||||
NL→SQL workflow**, where the model *proposes* and a human *reviewer decides* at gates.
|
||||
The UI is meant to embed inside the Omics Portal (GSD design system) and is **English**.
|
||||
|
||||
## Architecture — three layers
|
||||
|
||||
```
|
||||
frontend (React, :5173) → backend (Fastify, :8787) → pi --mode rpc → tht / harness → DWH (read-only)
|
||||
```
|
||||
|
||||
- **harness/** — the Pi layer. A deterministic Python CLI **`tht`** + a Pi gate extension
|
||||
(`.pi/extensions/tht-gate.js`) that runs the 8-phase workflow and emits/consumes
|
||||
widget-descriptor JSON. **Owns all persistence.** Workflow truth is `harness/workflow.yaml`;
|
||||
orchestration rules are `harness/.pi/skills/tht-sessione/SKILL.md`.
|
||||
- **backend/** — Fastify + TypeScript. A **thin bridge**: proxies REST routes to the `tht`
|
||||
CLI (`ThtRunner`), manages Pi processes (`PiProcessManager`, one child per session),
|
||||
bridges Pi RPC events to SSE (`SessionBridge` + `SseHub`). No application database.
|
||||
- **frontend/** — React 18 + base-ui + Tailwind + TanStack Query + Zustand. Chat-style
|
||||
shell (`src/shell/AppShell.tsx`); the live transcript is rebuilt in-memory from the SSE
|
||||
stream (`src/store/sessionStore.ts`), **not persisted**.
|
||||
|
||||
### Persistence model (the load-bearing premise)
|
||||
There is **no verbatim chat store**. Each workflow phase persists its own document into the
|
||||
session directory, and that **IS** the persistence. A session = a directory under the
|
||||
workspace's `sessions/` path containing `session_manifest.yaml` + phase artifacts
|
||||
(`question.md`, `schema_linking.json`, `cte_plan.json`, `sql_final.sql`,
|
||||
`validation_report.md`, `review_decisions.jsonl`, …). A fresh Pi process resumes by reading
|
||||
`tht session show <id>` + the on-disk artifacts — never by replaying chat.
|
||||
|
||||
## The 8 phases (harness/workflow.yaml)
|
||||
F1 chiarimento · F2 memoria · F3 riscrittura (`question.md`) · F4 schema_linking
|
||||
(`schema_linking.json`) · F5 sintesi · F6 cte (`cte_plan.json`, `cte_tests.json`) ·
|
||||
F7 sql_finale (`sql_final.sql`) · F8 datamart. Current phase is a fold over the decision
|
||||
ledger (`harness/tht/phase.py`); statuses: `open` / `closed` / `finalized`.
|
||||
|
||||
## How to run
|
||||
|
||||
**Full stack (real Pi + DWH):** `./scripts/run-stack.sh`
|
||||
Prereqs: VPN on; `pi` on PATH (configured model); `harness/.env` populated;
|
||||
`harness/config/tht.yaml` → a workspace; deps installed in all three projects.
|
||||
Opens frontend at http://localhost:5173 → backend :8787.
|
||||
|
||||
**Individual dev:**
|
||||
- backend: `cd backend && npm run dev` (tsx watch; env: `PORT`, `THT_HARNESS_DIR`, `THT_BIN`, `PI_BIN`, `AUTH_MODE`)
|
||||
- frontend: `cd frontend && npm run dev` (Vite; `VITE_BACKEND_URL` → backend)
|
||||
- harness install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` → `tht` on PATH
|
||||
|
||||
## How to test (all green as of 2026-06-29: harness 248 / backend 59 / frontend 73)
|
||||
- harness: `cd harness && .venv/bin/pytest -q` (5 L2/real-DB tests are deselected by default)
|
||||
- backend: `cd backend && npx vitest run` · typecheck `npx tsc --noEmit -p .`
|
||||
- frontend: `cd frontend && npx vitest run` · typecheck `npx tsc -b` · e2e `npm run e2e` (Playwright)
|
||||
|
||||
## Config & workspaces
|
||||
- Workspaces: `harness/workspaces/*.yaml` (`psd`, `tht-test`, `tht.example`). A workspace sets
|
||||
the DB target and the **absolute** `paths.sessions/artifacts/indexes` (psd → a *separate*
|
||||
repo `tht-workspace-psd/`, NOT committed here).
|
||||
- Secrets live ONLY in `harness/.env` (gitignored; `THT_*` — DB, DWH REST, vector, SSL CA…).
|
||||
See `harness/.env.example` for the variable list.
|
||||
- App settings (global): `backend/data/settings.json` (gitignored) — `{ workspace, provider,
|
||||
model, thinking }`. The "New session" form is question-only; these settings supply the rest.
|
||||
|
||||
## Conventions & contracts (don't relearn the hard way)
|
||||
- **`-c`/`--config` is a PER-COMMAND option in `tht`** — append it AFTER the subcommand,
|
||||
never globally (`ThtRunner.buildArgv` handles this).
|
||||
- **`--json` output must be pristine** (only valid JSON on stdout).
|
||||
- **UI strings are English.** Document *content* stays in the workspace language (Italian
|
||||
for psd) because it's the real data; only chrome/labels are English.
|
||||
- **Settings are global**, not per-question.
|
||||
- TDD throughout; tests assert real behavior, not mocks. Frequent, scoped commits.
|
||||
- Global user rules (`~/.claude/CLAUDE.md`): think before coding, simplicity first, surgical
|
||||
changes, goal-driven verification.
|
||||
|
||||
## Most recent feature — Session management (MERGED to main @ 2c21e46)
|
||||
Full session management modeled on Claude's UI, all three layers:
|
||||
- **Read-only "split view" panel** (left drawer, `SessionDocumentsPanel`) showing a session's
|
||||
phase documents read-only (reuses `SqlViewer`/`SchemaLinkingViewer`/`MarkdownView`).
|
||||
- **Rename / Move to group / Archive / Delete** via a kebab menu (`SessionMenu`) → REST →
|
||||
`tht session set-name/set-group/archive/unarchive/delete`. Archive = a manifest `archived`
|
||||
flag (not a dir move); groups = a manifest `group` field; delete = hard `rmtree` + confirm.
|
||||
- Rail: collapsible group headers + "No group" + a separate **Archive** view.
|
||||
- **Resume correctness:** read-only **guard** (HTTP 409 when `finalized` or `archived`);
|
||||
`PiProcessManager.spawnFor` now has a `new`/`resume` mode (resume sends
|
||||
`/riprendi-sessione <id>`); a "Phase 0 — Resume" cold-start section in `SKILL.md`.
|
||||
- Design docs: `docs/superpowers/specs/2026-06-29-session-management-design.md` +
|
||||
`docs/superpowers/plans/2026-06-29-session-management.md`.
|
||||
|
||||
### ⚠️ Open items / pending gates
|
||||
1. **Resume cold-start e2e (MANUAL, not yet run).** The resume chain is structurally
|
||||
complete but unproven live: drive a session to F4, stop it, resume in a fresh process,
|
||||
confirm it re-enters at F4 (not a new-question kickoff). **Do not advertise the "Resume"
|
||||
button as working until this passes.**
|
||||
2. **Full Playwright live-stack verification (MANUAL, not yet run).**
|
||||
3. **Minor backlog (non-blocking):** explicit id-traversal guard in `delete_session`
|
||||
(today gated by `load_session`); `close_session` could reuse `_save_touched` (DRY);
|
||||
delete-via-kebab integration test skipped (base-ui Menu portal not drivable in jsdom —
|
||||
the dialog itself is unit-tested); a couple of test-file lint nits.
|
||||
|
||||
## Where design history lives
|
||||
- Specs: `docs/superpowers/specs/` · Plans: `docs/superpowers/plans/`
|
||||
- SDD execution ledger (gitignored scratch): `.superpowers/sdd/progress.md`
|
||||
- Auto-memory index: `~/.claude/projects/-Users-mp-projects-ThothII/memory/MEMORY.md`
|
||||
(notes on the Pi RPC event vocabulary and the Omics Portal/GSD design system).
|
||||
|
||||
## Git
|
||||
`main` @ `2c21e46`, pushed to `origin` (github.com/mptyl/ThothII), in sync. Other local
|
||||
branches (`feat/settings-menu`, `feat/thothII-debugging`) are pre-existing, untouched.
|
||||
Reference in New Issue
Block a user