diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 3584b0a1..67b0e16c 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -1,6 +1,6 @@ # ThothII — Project State -> Starting-point snapshot for new sessions. Last updated: 2026-07-07. +> Starting-point snapshot for new sessions. Last updated: 2026-07-08 (three efficiency levers deployed). > Point a fresh session here ("read PROJECT_STATE.md") before substantial work. ## What ThothII is @@ -43,10 +43,14 @@ ledger (`harness/tht/phase.py`); statuses: `open` / `closed` / `finalized`. ## How to run -**Full stack (real Pi + DWH):** `./scripts/run-stack.sh` -Prereqs: VPN on; `pi` on PATH (configured model); `harness/.env` populated; -`harness/config/tht.yaml` → a workspace; deps installed in all three projects. -Opens frontend at http://localhost:5173 → backend :8787. +**Full stack (real Pi + DWH):** from project root, +```bash +./scripts/run-stack.sh +# Opens frontend: http://localhost:5173 (proxies backend :8787) +``` +**Prereqs:** VPN on; `pi` on PATH (with a configured model, e.g., `pi model set claude-fable-5`); +`harness/.env` populated (see `.env.example`); `harness/config/tht.yaml` → workspace (psd recommended for testing). +All three layers' deps installed (`npm install` in each, `python -m venv + pip install -e ".[dev]"` in harness). **Individual dev:** - backend: `cd backend && npm run dev` (tsx watch; env: `PORT`, `THT_HARNESS_DIR`, `THT_BIN`, `PI_BIN`, `AUTH_MODE`) @@ -67,6 +71,30 @@ Opens frontend at http://localhost:5173 → backend :8787. - App settings (global): `backend/data/settings.json` (gitignored) — `{ workspace, provider, model, thinking }`. The "New session" form is question-only; these settings supply the rest. +## Efficiency levers (NL→SQL workflow optimization, 2026-07-08) + +Three deployed optimizations target model thinking time (the dominant cost, ~220s in F1 alone): + +1. **Join-graph via FK annotations + `tht schema suggest-fks`** + - DWH has no FK constraints declared. Annotations file (`tht-workspace-*/artifacts/mschema/annotations.yaml`) now stores curated logical FKs. + - Three ranking rules: mine from approved SQL (highest confidence), heuristics (`*_time_key → dim_time.day_key`), same-name PK discovery with `--assume` flag for disambiguity. + - **Psd workspace:** 228 FK suggestions already generated (139 tables); `tht schema suggest-fks --from-sql --assume cod_paz=dim_patient` for updates. + - **Activation:** automatic. The mschema renderer populates the `【Foreign keys】` section. F4 in SKILL.md now reads FK joins from there instead of the model re-deriving them. + +2. **Context-pack consolidation at F1 kickoff (`tht search pack`)** + - Single embedding of the question, reused for schema + evidence + solved-question searches. + - Command: `tht search pack "" --session ` → `sessions//retrieval_pack.md` (tabelle candidate, relevant evidence, solved exemplars). + - Graceful degradation: if Ollama or vector store unreachable (no VPN), sections are empty but exit 0 — session continues with live searches. + - **Activation:** automatic at next session. SKILL.md F1 now prescribes as first call; reduces exploratory turns. + +3. **Phase-summary recap auto-construction from session ledger** + - `tht session show --json` includes the full decisions ledger; `tht phase meta --json` exports decision types per phase. + - Gate appends deterministic `【Decisioni registrate in questa fase】` section to v2 phase-summary artifacts. + - Model authors only `summary` + `checks`; the gate fills the recap table from persisted state → exact by construction. + - **Activation:** automatic at next session and Pi restart. SKILL.md Disciplina 6 updated: model keeps output brief, gate enriches from catalog + ledger. + +**Tests:** 358 Python + 111 JS gate, all pass. L2 (live DWH) verification on psd workspace recommended when time permits. + ## Conventions & contracts (don't relearn the hard way) - **`-c`/`--config` is a PER-COMMAND option in `tht`** — append it AFTER the subcommand, never globally (`ThtRunner.buildArgv` handles this). diff --git a/harness/README.md b/harness/README.md index 299b8e60..1ba0652b 100644 --- a/harness/README.md +++ b/harness/README.md @@ -65,6 +65,27 @@ tht lsh build -c /path/to/tht-workspace-/.yaml tht session new -c /path/to/tht-workspace-/.yaml ``` +## Efficiency levers setup (one-time per workspace) + +To enable the three deployed optimization levers on a **new workspace**: + +```bash +# Lever 1: Foreign key suggestions from approved SQL + heuristics. +# Run once after schema introspection to populate annotations.yaml. +tht schema suggest-fks \ + --from-sql /path/to/customer/sessions \ + --assume cod_paz=dim_patient \ + -c /path/to/customer/workspace.yaml \ + --write + +# (psd workspace: 228 FK suggestions already generated via this command 2026-07-08) + +# Levers 2 + 3: automatic at next session (context-pack + recap from ledger). +# No setup needed — SKILL.md prescribes them automatically. +``` + +See `PROJECT_STATE.md` for what each lever does. + ## The workflow `workflow.yaml` is the **single source of workflow truth** (spec F2). Eight phases