# ThothII — Project State

> Starting-point snapshot for new sessions. Last updated: 2026-07-14 (Qwen connectivity and Resume recovery live).
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.

## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12

ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale).

- **2 container** su `compose.yaml`: `thothii-core` (Fastify + harness tht + Pi) + `thothii-frontend` (Vite + nginx-unprivileged). Rete `omics_portal_omics_network` (external) con alias `thothii-core`/`thothii-frontend`.
- **DB**: Postgres diretto `:5438` (stessa istanza: schema `datawarehouse` 163 tabelle + `vectors` pgvector). Ruoli dedicati `thoth_dwh_reader` (read-only) + `thoth_vector_rw` (read+write). Embeddings: Ollama `:11434`.
- **Secrets**: `deploy/thothii.env` (env_file, gitignored) + `THT_MODEL_API_KEY_FILE` (key modello, file 0600 — meccanismo provider-credentials di Codex) + bind-mount `~/.pi` (pi-config).
- **Backend**: merge di `codex/portable-deployment` (secret-bundle, provider-credentials, auth `upstream`, security hardening, CI multiarch). Setup Docker MIO tenuto (il modello Docker-secrets di Codex è in `deploy/` come alternativa inerte).
- **Portale** (repo `omics_portal`, branch `agent/patient-capabilities-datamart-ui`): `nginx.conf` rotte `/datamart-builder/api`+`/assets` + `auth_request`, template `datamart_builder.html` (mount `<div id="root">` + tag `{% vite_assets %}`), vista `datamart_builder_api_auth`. Auth: `authentik Admins` bypass; utenti normali necessitano gruppo `omics-datamart-builder`.
- **Fix load-bearing**: `configPath` da `THT_CONFIG` (route senza workspace), `vite_assets` `mark_safe` (SPA bianca), `COPY harness/`+`cp workflow.yaml`+`pip install .` (pip 26 / tht module-relative), entrypoint `server` case.
- **Standalone/dev**: `docker-compose.dev.yml` (rete propria, porte host 8787/8090) + `scripts/docker-smoke.sh`.
- Piano dettagliato: `docs/superpowers/plans/2026-07-12-local-docker-deploy-implementation.md`.

### Runtime incident fixes — LIVE 2026-07-13

- The bind-mounted Pi profile came from host paths and did not trust `/app/harness`.
  Pi 0.80 consequently loaded **zero** project extensions, prompts and skills, silently
  sending `/nuova-domanda`/`/riprendi-sessione` to the model as plain text. The core
  entrypoint now idempotently adds only `/app/harness` to the persistent
  `/home/thoth/.pi/agent/trust.json`, preserving all existing decisions.
- The gate embeds the canonical `tht-sessione/SKILL.md` in the one-shot kickoff system
  prompt and explicitly prohibits repository discovery. A live RPC `get_commands` must
  show `torna`, `nuova-domanda`, `riprendi-sessione`, and `skill:tht-sessione` after deploy.
- Workspace identity is derived from the resolved config path, so
  `config/tht.yaml -> workspaces/local.yaml` matches DWH artifact ownership (`local`).
- Direct pgvector now discovers the actual namespaces of the `vector` type and cosine
  operator from PostgreSQL catalogs. This supports server layout `vectors.*` tables with
  the extension installed in `public`.
- Live verification: session `2026-07-13-074712-dammi-la-lista-dei-pazienti-che-haoo-fat`
  resumed directly at F1, ran `tht session show`, and completed `tht search pack`
  (12 tables, 0 evidence, 2 solved) without repository exploration or adapter errors.

### Workflow/UI regression fixes — LIVE 2026-07-14

- **F1 Model Activity restored.** Session create/resume now preserves configured/persisted
  thinking instead of forcing `off`. Pi's nested `thinking_delta` is bridged to a dedicated
  named SSE `activity_delta`; EventSource subscribes to that name and the panel keeps it separate
  from final assistant text. Reasoning remains in-memory and is not persisted to session artifacts.
- **F3 rewrite confirmation remains bypassed.** `rewrite_question` records approval and advances
  automatically without a reviewer widget. The repeated prompt came from old running containers:
  images had been rebuilt but services had not been recreated.
- **Join review is read-only and complete-set safe.** Join-only proposals render informational
  cards with only `Continue` and `Other — specify`. Continue requires the exact complete id set;
  all joins are persisted together by `decision add-join-set`, using an atomic ledger replacement
  under a per-session cross-process writer lock. Other persists none of the rejected proposal.
- **CTE presentation fixed.** F6 CTE cards now structure purpose, rationale, tables, filters, keys,
  and output columns with responsive wrapping/alignment. The Horizontal/Vertical switch is hidden
  for a single SQL block (the per-CTE view), because it only affects multi-block layouts.
- **Latest render failure diagnosed and hardened.** Session
  `2026-07-14-115847-estrai-i-pazienti-che-hanno-fatto-un-abl` sent an object in
  `open_questions`, which React cannot render as a child. The v2 gate now enforces
  `open_questions?: string[]`; the frontend also safely normalizes legacy malformed payloads.
- **Verification/deploy:** Python harness 798 passed / 5 L2 deselected; gate JS 126; backend 143;
  frontend 250; TypeScript/build gates green. Compose rebuilt and force-recreated both services.
  Running image ids: core `sha256:55acef2f12151ea97144c2f5e9164d63f2ca734bc2746fef553df94849e3fb3f`;
  frontend `sha256:1043f79392420149655cc63d70461e2ca2005b2290a1e3e21dcf845ec3bd1c81`.

### Pi-enabled model selector — LIVE 2026-07-14

- **Pi is the allowlist authority.** `/models` reads the mounted Pi `enabledModels`, intersects
  it with models currently available from Pi, and preserves the configured order. Enumeration
  does not require `PI_PROVIDER`, does not inject generic/provider credentials, and fails closed
  for missing or malformed scope.
- **Live scope:** exactly `deepseek/deepseek-v4-flash`, `zai/glm-5.2`, and
  `local-qwen/qwen3.6-35b-a3b`. The live endpoint returned those three composite IDs once each and
  in that order; `zai/glm-5v-turbo` and all other authenticated Pi models are hidden.
- **Validation/process smoke:** live settings updates returned 200 for DeepSeek Flash and local
  Qwen, while hidden GLM-5V returned 400; a post-restore equality check confirmed the original app
  settings were restored. The real `PiProcessManager` configure path succeeded for DeepSeek and
  local Qwen without sending a prompt or starting a DWH operation; local Qwen required no hosted
  provider key. Unknown and compound providers remain fail-closed in the verified backend suite.
- **Verification/deploy (`2026-07-14T19:24:22+02:00`):** backend **154/154** and frontend
  **251/251** passed; both TypeScript gates and `git diff --check` were green. Compose built and
  force-recreated only `core`; container start was `2026-07-14T17:24:01.992971976Z`, health was
  `healthy`, and sanitized post-recreate logs contained only the backend listen line. Rebuilt image
  ID and running container image ID both equal
  `sha256:577f99754fd0731251c8ddd8608b1b8baee09d02fad66c759b23f8221083e676`.

### Qwen connectivity + state-aware Resume recovery — LIVE 2026-07-14

- **Pi turns have an explicit lifecycle.** The bridge tracks `idle`, `running`, `waiting`,
  and `failed`; a reviewer gate is `waiting`, responses/steering return to `running`, and an
  assistant provider error or unexpected Pi child exit becomes `failed`. Provider error details
  are never forwarded to the client; the UI receives a fixed sanitized recovery message.
- **Resume preserves only active work.** `running`/`waiting` runtimes return as already active.
  Every validated cold path—including recovery after a child has already exited—clears stale SSE
  state before reopening and restarts from persisted provider/model/thinking with
  `/riprendi-sessione`; `idle`/`failed` runtimes are torn down at that point. Failed validation does
  not detach the existing stream. A successful Resume of the currently selected session also
  closes and recreates its EventSource, so the replacement runtime cannot be left behind an old
  same-ID stream.
- **Private Qwen routing is live.** Core is attached to both `omics_portal_omics_network` and
  external `localllm_default`; frontend remains only on the portal network. The mounted Pi profile
  resolves `local-qwen/qwen3.6-35b-a3b` at the sanitized base URL
  `http://localllm-vllm:8000/v1`. A direct probe from core verified the model catalog and received
  a non-empty real chat completion.
- **Verification/deploy (`2026-07-14T21:41:52+02:00`):** backend **168/168** and frontend
  **256/256** passed; both TypeScript gates, both production builds, the Qwen Compose network
  contract, and `git diff --check` exited 0. The initial deployment built and force-recreated
  `core` and `frontend`; after the final crash-recovery review, only the affected `core` image was
  rebuilt and force-recreated with no active Pi session. Core is healthy and its post-recreate
  Qwen catalog/completion probe succeeded. Built and running image IDs match: core
  `sha256:9867c2fa002b6f117da9a1d02c73b47bfe0372b8c1c137a5174c3e7ecb1e1db1`, frontend
  `sha256:5a47f81bc887423e05cef8fd3feb075aa600cb33217247186124f60ca5a3005b`.
  The frontend entry hash changed, so only `omics_portal-web-1` was restarted to invalidate its
  indefinite Vite-manifest cache. The application-level Qwen smoke reached its first
  `ui_request`, persisted the expected provider/model, deleted only its uniquely named smoke
  session, restored the exact saved settings object, and left no smoke session or Pi runtime. The
  supplied probe's success path left its keep-alive SSE reader open, so only that probe process was
  terminated (exit 143), without touching backend, Pi, or unrelated runtimes. The same smoke then
  exited 0 with `controller.abort()` in cleanup, preserving the gate, session cleanup, and exact
  settings-restoration evidence.

## What ThothII is

A **human-in-the-loop datamart builder**: it turns a natural-language question into
validated SQL (and optionally a dbt datamart) through a deterministic **8-phase
NL→SQL workflow**, where the model *proposes* and a human *reviewer decides* at gates.
The UI is meant to embed inside the Omics Portal (GSD design system) and is **English**.

## Architecture — three layers

```
frontend (React, :5173) → backend (Fastify, :8787) → pi --mode rpc → tht / harness → DWH (read-only)
```

- **harness/** — the Pi layer. A deterministic Python CLI **`tht`** + a Pi gate extension
  (`.pi/extensions/tht-gate.js`) that runs the 8-phase workflow and emits/consumes
  widget-descriptor JSON. **Owns all persistence.** Workflow truth is `harness/workflow.yaml`;
  orchestration rules are `harness/.pi/skills/tht-sessione/SKILL.md`.
- **backend/** — Fastify + TypeScript. A **thin bridge**: proxies REST routes to the `tht`
  CLI (`ThtRunner`), manages Pi processes (`PiProcessManager`, one child per session),
  bridges Pi RPC events to SSE (`SessionBridge` + `SseHub`). No application database.
- **frontend/** — React 18 + base-ui + Tailwind + TanStack Query + Zustand. Chat-style
  shell (`src/shell/AppShell.tsx`); the live transcript is rebuilt in-memory from the SSE
  stream (`src/store/sessionStore.ts`), **not persisted**.

### Persistence model (the load-bearing premise)
There is **no verbatim chat store**. Each workflow phase persists its own document into the
session directory, and that **IS** the persistence. A session = a directory under the
workspace's `sessions/` path containing `session_manifest.yaml` + phase artifacts
(`question.md`, `schema_linking.json`, `cte_plan.json`, `sql_final.sql`,
`validation_report.md`, `review_decisions.jsonl`, …). A fresh Pi process resumes by reading
`tht session show <id>` + the on-disk artifacts — never by replaying chat.

## The 8 phases (harness/workflow.yaml)
F1 chiarimento · F2 memoria · F3 riscrittura (`question.md`) · F4 schema_linking
(`schema_linking.json`) · F5 sintesi · F6 cte (`cte_plan.json`, `cte_tests.json`) ·
F7 sql_finale (`sql_final.sql`) · F8 datamart. Current phase is a fold over the decision
ledger (`harness/tht/phase.py`); statuses: `open` / `closed` / `finalized`.

## How to run

**Full stack (real Pi + DWH):** from project root,
```bash
./scripts/run-stack.sh
# Opens frontend: http://localhost:5173 (proxies backend :8787)
```
**Prereqs:** VPN on; `pi` on PATH (with a configured model, e.g., `pi model set claude-fable-5`); 
`harness/.env` populated (see `.env.example`); `harness/config/tht.yaml` → workspace (psd recommended for testing).
All three layers' deps installed (`npm install` in each, `python -m venv + pip install -e ".[dev]"` in harness).

**Individual dev:**
- backend: `cd backend && npm run dev` (tsx watch; env: `PORT`, `THT_HARNESS_DIR`, `THT_BIN`, `PI_BIN`, `AUTH_MODE`)
- frontend: `cd frontend && npm run dev` (Vite; `VITE_BACKEND_URL` → backend)
- harness install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` → `tht` on PATH

## How to test (green 2026-07-14: harness pytest 798 / gate JS 126 / backend 168 / frontend 256)
- harness: `cd harness && .venv/bin/pytest -q` (5 L2/real-DB tests are deselected by default)
- backend: `cd backend && npx vitest run` · typecheck `npx tsc --noEmit -p .`
- frontend: `cd frontend && npx vitest run` · typecheck `npx tsc -b` · e2e `npm run e2e` (Playwright)

## Config & workspaces
- Workspaces: `harness/workspaces/*.yaml` (`psd`, `tht-test`, `tht.example`). A workspace sets
  the DB target and the **absolute** `paths.sessions/artifacts/indexes` (psd → a *separate*
  repo `tht-workspace-psd/`, NOT committed here).
- Secrets live ONLY in `harness/.env` (gitignored; `THT_*` — DB, DWH REST, vector, SSL CA…).
  See `harness/.env.example` for the variable list.
- App settings (global): `backend/data/settings.json` (gitignored) — `{ workspace, provider,
  model, thinking }`. The "New session" form is question-only; these settings supply the rest.

## Efficiency levers (NL→SQL workflow optimization, 2026-07-08)

Three deployed optimizations target model thinking time (the dominant cost, ~220s in F1 alone):

1. **Join-graph via FK annotations + `tht schema suggest-fks`**
   - DWH has no FK constraints declared. Annotations file (`tht-workspace-*/artifacts/mschema/annotations.yaml`) now stores curated logical FKs.
   - Three ranking rules: mine from approved SQL (highest confidence), heuristics (`*_time_key → dim_time.day_key`), same-name PK discovery with `--assume` flag for disambiguity.
   - **Psd workspace:** 228 FK suggestions already generated (139 tables); `tht schema suggest-fks --from-sql <session-dir> --assume cod_paz=dim_patient` for updates.
   - **Activation:** automatic. The mschema renderer populates the `【Foreign keys】` section. F4 in SKILL.md now reads FK joins from there instead of the model re-deriving them.

2. **Context-pack consolidation at F1 kickoff (`tht search pack`)**
   - Single embedding of the question, reused for schema + evidence + solved-question searches.
   - Command: `tht search pack "<question>" --session <id>` → `sessions/<id>/retrieval_pack.md` (tabelle candidate, relevant evidence, solved exemplars).
   - Graceful degradation: if Ollama or vector store unreachable (no VPN), sections are empty but exit 0 — session continues with live searches.
   - **Activation:** automatic at next session. SKILL.md F1 now prescribes as first call; reduces exploratory turns.

3. **Phase-summary recap auto-construction from session ledger**
   - `tht session show --json` includes the full decisions ledger; `tht phase meta --json` exports decision types per phase.
   - Gate appends deterministic `【Decisioni registrate in questa fase】` section to v2 phase-summary artifacts.
   - Model authors only `summary` + `checks`; the gate fills the recap table from persisted state → exact by construction.
   - **Activation:** automatic at next session and Pi restart. SKILL.md Disciplina 6 updated: model keeps output brief, gate enriches from catalog + ledger.

**Tests:** 358 Python + 111 JS gate, all pass. L2 (live DWH) verification on psd workspace recommended when time permits.

## Conventions & contracts (don't relearn the hard way)
- **`-c`/`--config` is a PER-COMMAND option in `tht`** — append it AFTER the subcommand,
  never globally (`ThtRunner.buildArgv` handles this).
- **`--json` output must be pristine** (only valid JSON on stdout).
- **UI strings are English.** Document *content* stays in the workspace language (Italian
  for psd) because it's the real data; only chrome/labels are English.
- **Settings are global**, not per-question.
- TDD throughout; tests assert real behavior, not mocks. Frequent, scoped commits.
- Global user rules (`~/.claude/CLAUDE.md`): think before coding, simplicity first, surgical
  changes, goal-driven verification.

## Active memory — F8 promotion gate + solved-question recall — SHIPPED, L2 pending (2026-07-07)

Two additions to close the loop on reusable memory, on top of the existing `tht memory
search` (Phase 2) reuse:

- **F8 promotion gate.** `reviewer_memory_promote` (Phase 8, called with only the session
  id): the gate computes candidates deterministically via `tht memory promote --preview
  --json` (the 3 reusable decision types, already excluding previously promoted/declined
  ones) and shows a pre-selected checklist. Selected → `tht memory save-one` persists to
  the vectordb + records `memory_promoted`; deselected → `memory_promotion_declined`
  (ledger detail `seq:<n>`) so it is never re-proposed. `tht memory promote`/`save-one` were
  added to the gate's anti-bypass FORBIDDEN list (model must go through the gate tool).
- **Solved-question exemplars.** New vector kind `solved_question` reusing the existing
  `memory` pgvector table (no server-side DDL); `harness/tht/solved.py` does a one-row
  upsert keyed by a hash of question+SQL. CLI: `tht memory solved-index` / `solved-search`.
  `tht session finalize` auto-indexes the pair (best-effort: green line on upsert, cyan
  "già aggiornata" on dedup no-op, yellow warning + the recovery command
  `tht memory solved-index <id>` on failure). `SKILL.md` now prescribes calling
  `solved-search` as reference-only context in F4 (schema linking), F6 (CTE plan) and F7
  (final SQL), and documents the finalize auto-index in "Session end".

**Pending L2 gate (not yet run — needs VPN + writer key):** one live end-to-end session on
workspace `psd` via `./scripts/run-stack.sh` to verify (a) the promotion checklist renders
pre-selected and persists selected/declined correctly, (b) finalize indexes the pair,
(c) `tht memory solved-search` returns it with sql + tables.

**Fast-follow:**
- RestSearcher top-k dilution — **client-side DONE** (2026-07-07): `search_similar` manda
  `kinds` alla RPC (filtro server-side esatto) con fallback automatico su server legacy
  (404 → retry senza filtro, post-filter client). **Resta la migrazione server** della
  funzione SQL `search_similar` (+`kinds text[] DEFAULT NULL`): istruzioni pronte in
  `harness/docs/vector-rest-kinds-migration.md`; l'ordine di deploy è libero, ma fino
  alla migrazione il filtro resta client-side e la diluizione persiste.
- ~~`tht memory solved-search` muore con traceback grezzo se il vectordb è irraggiungibile~~
  **DONE** (2026-07-07): degrada a warning di una riga su stderr, stdout puro (`[]` in
  --json), exit 0 — copre VectorRestError/EmbeddingsError/OperationalError.

## Review gates v2 — payload strutturati + viewer dedicati — COMPLETE (2026-07-07)

Plan: `~/.claude/plans/prima-di-passare-ai-inherited-marshmallow.md`. Merged to `main` @ `2410f01`
(ff, pushed). Executed via subagent-driven-development (4 workstreams, task reviews, final
whole-branch review + fix wave).

- **Contracts:** `artifact.data.schema_version: 2` for `cte_plan` / `cte_result` / `phase`,
  built **deterministically by the gate** (catalog descriptions via `tht schema columns`; SQL
  from `ctes/<name>.sql`; preview rows persisted by `tht cte test`); the model contributes only
  purpose/rationale/note. Non-v2 payloads fall through to the legacy renderers — old sessions
  and `tools/replay/replay.json` keep working.
- **Harness (Python):** `CteTestRecord.preview_rows` (+ `_jsonable` coercer, ≤10 rows, cells
  ≤200 chars); new read-only `tht cte info <name> --session <id> --json` (index/total from
  `cte_plan.json`, same source as `next_cte`); `tht cte plan --doc -` writes
  `cte_plan_doc.json` (chain documentation; `cte_plan.json` stays a load-bearing `list[str]`).
- **Gate (JS):** `gate/artifact-contracts.js` (soft validators → self-corrective `textResult`,
  TypeBox untouched) + `gate/enrich.js` (pure, catalog lookups injected);
  `prepareReviewerArguments` now coerces `artifact.data` too (GLM stringified-param
  mitigation); `SKILL.md` Phase 5/6 + disciplines rewritten (plan via `reviewer_confirm
  kind:"cte_plan"` with payload A; `cte_result` gates send THIN data only — never SQL/preview
  as text).
- **Frontend:** `artifactV2.ts` types; `CtePlanViewer` (per-CTE cards + chain strip),
  `CteResultViewer` (shiki SQL + AG Grid preview), `PhaseSummaryViewer` (checks + criteria
  with the VALUES driving choices), `PreviewGrid` extracted from `ResultsPanel`,
  `statusBadge.ts` shared success/warn/error tokens.
- **Replay:** v2 fixtures + `tools/replay/augment-review-gates.mjs`; `replay.json` regenerated;
  offline visual pass ok (screenshots in the SDD scratch dir).
- **LIVE E2E (session `2026-07-07-011858`, GLM 5.2):** all 8 phases completed with the v2
  gates; session **finalized** (DWH validation battery green, needs VPN).
- **Bug found live + FIXED (`2410f01`):** infinite spinner at workflow end — the bridge dropped
  Pi's `agent_end` (the ONLY end-of-turn signal) and `working` was released only by the next
  gate, which the final turn doesn't have. Now: bridge maps `agent_end` → SSE
  `system_event`; FE tracks `agentActive`; an unexpected Pi child exit notifies the client
  (info error + synthetic `agent_end`). Memory: `pi-rpc-event-vocabulary`.
- **Open (non-blocking):** `tht.sqlcheck` maps table aliases by first occurrence (found and
  worked around by the model in F6 — spawned as a separate task); one more live confirmation
  that the spinner stops at F8 (the chain is unit-tested end to end).

## F4 schema-linking column curation + look&feel v2 — COMPLETE (2026-07-06)

Branches `feat/f4-schema-linking-column-curation` (PR #1) + `feat/frontend-lookfeel-v2`, landed
on `main` (`d942635` … `7491e8c`). The F4 gate (`reviewer_schema_linking`) presents
catalog-enriched tables/columns (descriptions from `tht schema columns`, hardened enrichment),
per-table columns modal (suggested pre-checked, suggested-first ordering + filter box);
decisions `column_promoted`/`column_excluded` + deterministic `tht session
sync-schema-linking` projection into `schema_linking.json`. Live-verified including the
clobber test (the model's joins write preserves curated columns). Look&feel v2: shadows/radii/
mono labels, 70% gate modal, structured cards (colors untouched). Memory:
`thothii-visual-language-v2`.

## Workflow contract hardening — COMPLETE (2026-07-01)

Spec: `docs/superpowers/specs/2026-07-01-workflow-contract-hardening-design.md` · Plan:
`docs/superpowers/plans/2026-07-01-workflow-contract-hardening.md`. Merged to `main` @ `3dadc6f`
(pushed). Driven by analysis of Pi session `2026-06-30-165708` (GLM 5.2), where the model spent
~80% of its tool calls reverse-engineering the harness because `SKILL.md` mis-stated the
phase-advance contract — and Phase 6 was a hard dead-end. Three coordinated harness fixes (TDD):

- **F6 CTE-approval dead-end FIXED.** The gate's `reviewer_confirm kind:"cte_result"` used to
  register `cte_approved --subject phase:6`, which `decision_cmd` rejects (exit 5 — it needs a
  real CTE name from the plan) → F6 could never close. New `tht cte next --session <id>` returns
  the first unapproved plan CTE; the gate now approves **by name**. (`tht/cli/cte_cmd.py`,
  `.pi/extensions/tht-gate.js`.)
- **`schema_linking.json` writer/validator.** `store.set_schema_linking` (validates against the
  `SchemaLinking` model, THEN writes — no partial file) → CLI `tht session set-schema-linking
  <id> --file <path|->` (exit 5 on bad JSON / ValidationError) → gate tool `write_schema_linking`
  (stdin). Replaces the model hand-writing the F4 artifact + ad-hoc python validation.
- **`SKILL.md` corrected to match the code.** Only F2-empty / F6-skipped auto-advance
  (`_AUTO_ADVANCE_PHASES={2,6}`); every substantive phase closes with `reviewer_confirm
  kind:"phase"` (F7 is **two-step**: `kind:"sql"` records `sql_approved`, then `kind:"phase"`
  advances). Fixed Discipline 2 + Phase 1/3/4, added a per-phase **cheat-sheet**, documented the
  `SchemaLinking` shape. The old false "the reviewer_decide already advances" (F3) claim — the
  exact cause of the observed thrash — is gone.

Verified: harness pytest **281 passed** / 5 deselected, gate JS **34/34**, changed-files ruff
clean (the 36 `ruff check .` errors are pre-existing on `main`). Final whole-branch review
(opus): READY TO MERGE, no Critical/Important. Executed via subagent-driven-development
(implementer + task-review per task, final opus review). **DEFERRED (needs VPN): live F4/F6
end-to-end** — resuming session `2026-06-30-165708` (stuck at F6) is the ideal live probe.

## UI/UX redesign + Resume — COMPLETE (2026-06-30)

Plan: **`~/.claude/plans/foamy-forging-dahl.md`**. Memory: `thothii-ui-redesign-inprogress.md`.
**All workstreams done and pushed to origin/main:** D + E @ `0eeb3f7`, B + C @ `b056ff3`,
F @ `cef9ae4`, A @ `0a13f71`, G @ `e8cdd00`(scope) + `cbb8e18`(results). Nothing pending from this
plan. Per-workstream detail below for reference.

- **D — DONE** (`c12bdcd`): session display `name` = 3-5 Italian keywords via **YAKE** (no LLM),
  derived in `tht session new` (CLI layer); `create_session` core unchanged (`name=None` default).
  `yake` added to `harness/pyproject.toml`. TDD `tests/test_session_name.py`; harness 269 passed.
- **E — DONE** (`0eeb3f7`): rotating activity icon replaces the red dot in `CentralStatus`
  (inline, clickable → opens the panel); `ModelActivityPanel` is a **5-line expandable
  model-stream tail**; `WorkingSpinner` extracted to its own module; the separate spinner button
  + orphaned `Transcript.tsx` removed. Frontend 87/87, tsc clean. **Live visual check DONE
  (2026-06-30):** inline spinner opens the panel; 5-line collapsed tail; expand → full transcript.
- **B — DONE** (`b056ff3`): `WorkflowBar` is now colored **dots** F1..F8, no phase-name text
  (amber-translucent=running, green=done, red=error, gray=pending; green connectors lead the active
  dot). Each dot carries `data-state`. Error is lightweight: store `phaseError` set when an `info`
  `level=error` arrives during the phase, cleared on the next `ui_request` (`sessionStore.ts`).
  **All four states live-verified** via Playwright.
- **C — DONE** (`b056ff3`): right sidebar — single-line denser rows (inline status dot + name,
  `py-1`), a 3-level type hierarchy via **`/impeccable`** (L1 `SESSIONS` red/bold/wide-tracking ·
  L2 section + group headers muted uppercase · L3 names normal-case), and the **"No group" label
  removed** (ungrouped sessions render after the last group; guarded so the empty-state still
  teaches when there are no groups). **Live-verified.** (Resume in `SessionMenu` stays with A1.)
- **Tests:** frontend **93/93** (was 87; +3 store `phaseError`, +2 `WorkflowBar` dot-state, +1
  AppShell no-"No group"), `tsc -b` clean.
- **F — DONE** (uncommitted; live check deferred to G): single-select answers **auto-confirm**.
  `reviewer_select` options may carry a `decision` payload (`{type, subject, detail?, rationale?}`)
  and an optional `advance`; picking such an option persists the decision directly via
  `tht decision add` (shared `decisionAddArgs` helper, also used by `reviewer_decide`) — no redundant
  `reviewer_decide`/`reviewer_confirm` gate. Options without a payload stay ask-only; back/exit/Other
  never persist. Pure logic extracted to `resolveSelectOutcome`/`decisionAddArgs` (exported, unit-
  tested). Contract docs updated: `reviewer_select` tool desc + `SKILL.md` (widget summary,
  disciplines 2-3, Phase-1 single-pick) + the `CLAUDE.md` gate note. Gate JS **33/33**, harness 269.
  **Live verification (model actually uses `reviewer_select`+decision, no follow-up gate, decision in
  `review_decisions.jsonl`) deferred to G** — it is model-behavior-dependent.
- **A — DONE** (uncommitted): **A1** — `SessionMenu` gains a **Resume** item (gated to
  `status!=="finalized" && !archived`), wired in `AppShell` to the existing `doResume` → `POST
  /sessions/:id/resume`. 3 tests (`SessionMenu.test.tsx`); frontend **96/96**, tsc clean. **A2** —
  diagnosis-first clean-room repro shows the **resume cold-start stall NO LONGER reproduces on pi
  0.79.4** (8/8 chained into the tool calls, fresh + partway; GLM 5.2 now narrates AND emits
  `tht session show`+`read SKILL.md` in-turn). The earlier narrate-and-stop predates the pi upgrade.
  Defense-in-depth applied: `RIPRENDI_KICKOFF` hardened to force the in-turn tool call (gate test +
  live regression 2/2). The cross-model angle (weaker/older models) lives in **G**.
- **G — DONE** (`e8cdd00`+`cbb8e18`): cross-model behavior matrix via a committed clean-room harness
  (`harness/scripts/model-matrix.mjs`). **Tier 1** — kickoff + resume first-turn: all *available*
  models chain in-turn (`zai/glm-5.2`, `deepseek/deepseek-v4-{pro,flash}`, `aritmolab/qwen3.6-35b-a3b`,
  `zai/glm-4.5-air`); the resume stall recurs on none (closes A's cross-model robustness).
  `aritmolab/gemma4-26b-a4b` = **404 unavailable** at the endpoint (listed but not served) — infra
  gap, not a workflow issue. **Tier 2** — F single-select auto-confirm verified live on `glm-5.2`:
  answering the first `reviewer_select` persisted a `concept_clarified` decision **0→1** with **no
  follow-up gate** (closes F's deferred live check). Full results: the G plan doc + memory
  `thothii-cross-model-matrix`. No prompt hardening needed.

**Status:** **All UI-redesign + resume workstreams done and pushed — D, E, B, C, F, A, G.**
Nothing pending from the plan. Optional nice-to-haves (not required): Tier-2 F/multiselect live for
the non-baseline models (cheap re-run with `harness/scripts/model-matrix.mjs` + the Tier-2 method),
and a one-off manual Playwright kebab→resume pass in the live UI.

## Live verification + reviewer_select fix (2026-06-30, afternoon)

Drove the real stack (Playwright → backend → real Pi → GLM 5.2 → DWH) end-to-end.

- **F1 hang fix (`418187a`) VERIFIED LIVE.** Answered an F1 reviewer widget; Pi resumed (model
  socket reopened) and the gate produced new output — vs the old silent hang. The transition
  "silent hang → gate re-presents/advances" proves `ctx.ui.input` now resolves.
- **New bug found + fixed: reviewer_select `choices` vs `choice`.** The gate's `reviewer_select`
  (and `reviewer_confirm` reject) read `resp.choice` (singular) but the frontend uniformly sends
  `choices: [id]` (array) — so every single-select gate answered "Nessuna scelta ricevuta" and
  re-proposed forever (multiselect was fine; it already read `choices`). Fix: a shared
  `selectedChoice(resp)` helper (`harness/.pi/extensions/tht-gate.js`) reading the array; both
  handlers use it. TDD: `gate/__tests__/gate_choice.test.js` RED→GREEN, full gate suite **28/28**.
  VERIFIED LIVE: a single-select answer is now accepted and the workflow advances (2/4 → 3/4).
- **Resume cold-start STALL confirmed (open item #1).** On `/riprendi-sessione`, GLM 5.2 narrates
  the bootstrap step then ends the turn without the tool call → Pi idle, unrecoverable from the UI.
  Memory: `thothii-resume-cold-start-stall.md`.
- **GLM 5.2 F1 is slow (~3-4 min, ~50+ reads) but works** — looks stuck but isn't; don't hit
  "Stop and save" (it `POST /close`s → kills Pi). Memory: `thothii-glm52-f1-slow-not-stuck.md`.

## Earlier work — F1 reviewer-widget hang fix + multiselect guidance (committed 2026-06-30; authored 2026-06-29)

Two fixes, **committed to `main`** (7 files):

1. **Bug: every reviewer widget hung "stuck with no output" after the human answered** — F1
   disambiguation (and any gate) dead-ended. Root cause, confirmed from Pi's own source
   (`@mariozechner/pi-coding-agent` `dist/modes/rpc/rpc-mode.js`, `createDialogPromise`):
   `ctx.ui.input` assigns its OWN RPC id (`crypto.randomUUID`) and correlates
   `extension_ui_response` on THAT id, silently dropping unknown ids. The gate puts a
   different id (`u${Date.now()}`) inside the descriptor carried in `title`. `SessionBridge`
   was replying with the **descriptor** id, so real Pi never resolved `ctx.ui.input` → the
   model never continued. **Fix:** `SessionBridge` now stores Pi's top-level `m.id`
   (`pendingPiId`) on the incoming request and replies `extension_ui_response{ id: pendingPiId,
   value: <uiResponse JSON> }` (value still carries the descriptor id, so the gate's internal
   `resp.id === descriptor.id` check holds). File: `backend/src/bridge/session-bridge.ts`.
   Full write-up: memory `pi-ui-input-id-correlation.md`.
   - **The test double was masking it:** `harness/tests/fake_pi/fake_pi_rpc.mjs` had forced
     `m.id == descriptor.id`. Corrected to mirror real Pi (distinct `randomUUID` top-level id,
     correlate on it, drop unknown ids); `test_fake_pi_contract.mjs` gained a negative
     regression test ("respond with descriptor id → no follow-up").
   - TDD: `backend/test/session-bridge.test.ts` (unit) + `backend/test/e2e-f1.test.ts`
     (integration — now asserts the model's follow-up arrives after the answer) went
     RED→GREEN.

2. **UX: multi-answer disambiguation** — `harness/.pi/skills/tht-sessione/SKILL.md` Phase 1
   now tells the model to use `reviewer_decide` (the existing multiselect/checkbox widget)
   when an ambiguity admits several simultaneously-true answers, instead of single-pick
   `reviewer_select`. Guidance-only — no new widget (`frontend MultiselectWidget` already
   exists).

Verified at commit time: backend `npx vitest run` **67/67 green**; `tsc --noEmit -p .` **OK**;
fake-pi contract `node --test test_fake_pi_contract.mjs` **2/2 green**. **Verified LIVE
2026-06-30** (see the top "Live verification" section).

## Most recent feature — Session management (MERGED to main @ 2c21e46)
Full session management modeled on Claude's UI, all three layers:
- **Read-only "split view" panel** (left drawer, `SessionDocumentsPanel`) showing a session's
  phase documents read-only (reuses `SqlViewer`/`SchemaLinkingViewer`/`MarkdownView`).
- **Rename / Move to group / Archive / Delete** via a kebab menu (`SessionMenu`) → REST →
  `tht session set-name/set-group/archive/unarchive/delete`. Archive = a manifest `archived`
  flag (not a dir move); groups = a manifest `group` field; delete = hard `rmtree` + confirm.
- Rail: collapsible group headers + "No group" + a separate **Archive** view.
- **Resume correctness:** read-only **guard** (HTTP 409 when `finalized` or `archived`);
  `PiProcessManager.spawnFor` now has a `new`/`resume` mode (resume sends
  `/riprendi-sessione <id>`); a "Phase 0 — Resume" cold-start section in `SKILL.md`.
- Design docs: `docs/superpowers/specs/2026-06-29-session-management-design.md` +
  `docs/superpowers/plans/2026-06-29-session-management.md`.

### ⚠️ Open items / pending gates
1. **Resume cold-start stall — RESOLVED on pi 0.79.4 (workstream A, 2026-06-30).** The earlier
   narrate-and-stop (GLM 5.2 narrating the bootstrap step then ending the turn without the tool
   call) **no longer reproduces**: a clean-room repro of the backend's exact resume handshake
   chained into `tht session show`+`read SKILL.md` in-turn **8/8** (fresh + partway sessions). The
   pi upgrade is the likely fix. Defense-in-depth: `RIPRENDI_KICKOFF` hardened to force the in-turn
   tool call (gate test + live 2/2). Memory: `thothii-resume-cold-start-stall.md`. **Remaining:**
   the cross-model angle (older/weaker models) is folded into **G**; a full Playwright kebab→resume
   pass through the live UI is still worth one manual run (item 2).
2. **Full Playwright live-stack verification (MANUAL, not yet run).**
3. **Minor backlog (non-blocking):** explicit id-traversal guard in `delete_session`
   (today gated by `load_session`); `close_session` could reuse `_save_touched` (DRY);
   delete-via-kebab integration test skipped (base-ui Menu portal not drivable in jsdom —
   the dialog itself is unit-tested); a couple of test-file lint nits.
4. **DONE — F1 hang fix live-verified 2026-06-30** (see top section). The live verification
   also surfaced + fixed the reviewer_select `choices` mismatch.
5. **Resolved: settings use `zai/glm-5.2`/medium** (not deepseek-flash); GLM 5.2 drives F1
   fine, just slowly (~3-4 min, ~50+ reads). To tell a truly stalled Pi from a merely-slow one,
   check its sockets/children. Memory: `thothii-glm52-f1-slow-not-stuck.md`.
6. **DONE — Pi migrated to `@earendil-works/pi-coding-agent@0.80.3` (2026-07-04).** The old
   scope `@mariozechner/pi-coding-agent` is frozen at 0.73.1; every release ≥0.74 lives under
   the new scope `@earendil-works` (latest 0.80.3). The live `pi` (`~/.local/bin/pi`) was
   repointed to 0.80.3. Validated by an API/RPC-surface diff (0.73.1→0.80.3: `ExtensionUIContext`
   byte-identical, `rpc-types` additive-only, `createDialogPromise` + provider-registration API
   unchanged) **plus** a live `model-matrix` smoke (GLM 5.2, `new`+`resume` both `CHAINED`,
   full RPC event vocabulary incl. `extension_ui_request` intact). Notable: 0.80.3 adds
   `ctx.mode: "tui"|"rpc"|"json"|"print"` (a proper mode discriminator; the earlier `0.79.4`
   references above are superseded). Rollback: the old package is still on disk — repoint the
   symlink to `@mariozechner/.../dist/cli.js`. Memory: `thothii-pi-earendil-migration.md`.

## Where design history lives
- Specs: `docs/superpowers/specs/` · Plans: `docs/superpowers/plans/`
- SDD execution ledger (gitignored scratch): `.superpowers/sdd/progress.md`
- Auto-memory index: `~/.claude/projects/-Users-mp-projects-ThothII/memory/MEMORY.md`
  (Pi RPC event vocabulary, ui.input id correlation, Omics Portal/GSD design system,
  resume cold-start stall, GLM 5.2 F1 slow≠stuck).

## Git
`main` @ `2410f01`, **pushed to `origin`** (github.com/mptyl/ThothII); working tree clean, no stashes.
Latest arc (2026-07-07): review-gates-v2 ff-merged — `9b4f6b9` (WS1 harness) · `3ad93cd` (WS2
gate+SKILL) · `1c97289` (WS3 viewers) · `4042d0b`+`b089482` (WS4 replay) · `b59b57c` (review fix
wave) · `2410f01` (agent_end spinner fix). Branch `feat/review-gates-v2` still exists (local +
origin), fully merged. Before that (2026-07-05/06): F4 column curation (PR #1) + look&feel v2 —
`0d9e035` · `0a63ba9` · `d942635` · `9fe1c93` · `e9b2934` · `9403147` · `7491e8c`.
Older history (workflow hardening 2026-07-01, UI redesign 2026-06-30): see the sections above;
stale branches were pruned on 2026-06-30 (SHAs recoverable via reflog).