942 lines
72 KiB
Markdown
942 lines
72 KiB
Markdown
# ThothII — Project State
|
||
|
||
> Starting-point snapshot for new sessions. Last updated: 2026-08-08 (final verification).
|
||
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
||
|
||
## P1 configuration-process automated integration — PASS 2026-08-09
|
||
|
||
- Retained evidence: `.artifacts/p1-integration/p1-2f5d861d1151121ad7cc3edb69cf3abb/report.md`
|
||
- automated integration: PASS
|
||
- manual acceptance: PENDING
|
||
- The retained run is bound to clean source commit
|
||
`0cfe5c7c9a4e6dcd395ec32135b5e4999311432b` and tree
|
||
`ad6fcfedb2ca1e8d3c16ec632618aaf2e1bbd78f`. Its hash-bound provenance contains exact
|
||
43-file backend source and 39-file compiled `dist` manifests (manifest SHA-256
|
||
`2188379320f93f4e24205d418ead9dd25b48ed93da663fcf7e3e6e835931c2bc` and
|
||
`9f9e8899f8aca882ff08d49ec6cd00caeea75691c8280095a9b88bc74939a30a`).
|
||
- The retained audit has exactly 15 PASS checks and 134 unique declared artifacts whose final
|
||
bytes match every SHA-256 declaration. It records 749 PASS command events and 1,664 production
|
||
child/network events (831 STARTED, 831 PASS, and the two expected FAIL terminals), with no
|
||
rejected surface event. Git runs with fixed safe configuration, an owned empty hooks directory,
|
||
exact local config/attribute/hook validation, and a fail-closed invocation grammar.
|
||
- The contextual negatives use a separate `invalid-context` branch, remote, checkout, data,
|
||
runtime, registry, and second production Fastify listener, all beneath the owned run root.
|
||
Persisted before/after semantic-state proofs show primary `main` remains valid, a missing-tree
|
||
publish is state-neutral, and an invalid pull advances only its disposable checkout while the
|
||
remote and last-valid active/snapshot/data/runtime state remain unchanged.
|
||
- The hash-bound final ownership artifact records both production listeners closed with refused
|
||
connection checks. Independent final probes found both ports refused and PID 80568 dead. The
|
||
reachable-object scan covered all four expected repositories (156 objects, 48 blobs) with zero
|
||
fixture-secret findings; the frozen final filesystem scan covered 600 non-fixture regular files,
|
||
found no symlinks, and found no fixture-secret values.
|
||
- Final report hashes: `report.json`
|
||
`268099f89dc203589eae35172b217e88d5f7603281157d78ae9e67ec5f792eda`;
|
||
`report.md` `f771aaed585c3b628768484338efedb133fae856840d72aa5ba41a675e81816f`.
|
||
|
||
## Internal Qdrant + Ollama semantic infrastructure — LIVE 2026-08-08
|
||
|
||
- **Compose topology.** The mandatory application stack is `frontend`, `core`, `qdrant`,
|
||
`embedding`, and the one-shot `embedding-model-init`. Startup is CPU-first by default; Linux
|
||
hosts may opt into GPU exposure with `THOTH_ENABLE_EMBEDDING_GPU=1`. Qdrant is private on the
|
||
Compose network and persists `/qdrant/storage` in `qdrant-data`. Ollama persists its local model
|
||
cache in `embedding-models`, and `embedding-model-init` blocks `core` until
|
||
`qwen3-embedding:0.6b` is present.
|
||
- **Semantic contract.** Internal semantic indexing is fixed to `qwen3-embedding:0.6b`,
|
||
`1024` dimensions, and cosine distance. Schema-v3 descriptors are operational; schema-v1/v2 descriptors remain `migration_required` until an explicit reviewed migration writes schema version 3. One workspace owns one Qdrant collection, and schema, Evidence, and Memory records
|
||
coexist inside that collection with payload `kind` separation.
|
||
- **Final review runtime barriers.** Operational routes, retained session pins, and runtime
|
||
rendering now require schema version 3 before resolving bindings, readiness, diagnostics, or
|
||
Pi. Session admission verifies the exact internal Qdrant collection (dimensions, cosine
|
||
distance, and required keyword payload indexes) before Ollama and before manifest persistence.
|
||
The Qdrant adapter binds every search/list/delete filter to its constructed workspace identity
|
||
and rejects conflicting caller namespaces.
|
||
- **Boundary and persistence.** Only DWH and LLM remain external runtime application endpoints.
|
||
There are no active external vector or embedding endpoint instructions, bindings, or secrets in
|
||
the supported operator manuals. Qdrant remains a derived but persistent semantic index: the
|
||
canonical sources of truth stay the workspace Git descriptors, phase artifacts, and memory
|
||
registry/ledger. The Ollama model cache is recoverable for offline startup but is not the
|
||
canonical source of semantic content.
|
||
- **Backup and recovery.** `./scripts/vector-backup.sh --project-name <name> --output <file>`
|
||
archives exactly one labeled `<project>_qdrant-data` volume and preserves the prior `qdrant`
|
||
running state. `./scripts/vector-restore.sh --project-name <name> --input <file>
|
||
--confirm-project <name>` requires the exact repeated project confirmation, validates manifest
|
||
and archive safety before stopping `qdrant`, stages rollback content, restores in place, and
|
||
restarts `qdrant` only if it was previously running. Restore does not migrate legacy workspace
|
||
descriptors, rename collections, or repair a semantic-index incompatibility. Backup and restore
|
||
share one atomic Docker-daemon lock per Compose project/Qdrant volume; contenders fail before
|
||
volume resolution, and cleanup removes the lock only when its ownership labels still match.
|
||
- **Verification recorded for Task 13 final audit.** On Apple M4 Pro
|
||
(`Darwin 25.5.0`, Docker Server `29.6.2 linux/arm64`), harness pytest passed
|
||
**827 passed / 4 deselected**; backend Vitest passed **477/477** plus TypeScript and build;
|
||
frontend Vitest passed **374/374** plus TypeScript and build; `git diff --check` passed.
|
||
Deployment contracts passed:
|
||
`test-default-compose.sh`, `test-unified-compose.sh`, `test-internal-semantic-compose.sh`,
|
||
`test-no-deployment-coupling.sh`, `test-compose-secret-policy.sh`, and
|
||
`verify-workspace-install-docs.sh --fixtures-only`.
|
||
- **Task 13 Docker smoke evidence.** CPU semantic smoke passed in **217.34s** and proved
|
||
offline Qdrant/Ollama persistence plus exact cleanup. Workspace registry smoke passed in
|
||
**42.06s** in fix round 1 with a per-run image tag derived from the unique Compose project,
|
||
and proves exact cleanup of compose containers, volumes, networks, and only that smoke image.
|
||
Unified deployment smoke passed in **125.57s**; update-only rollback smoke
|
||
passed in **85.40s**; Linux server deployment smoke passed in **55.99s**. The previously
|
||
observed `thothctl` rollback failure did not recur.
|
||
- **Task 13 image and manual-gate notes.** Verified pinned runtime images:
|
||
`qdrant/qdrant:v1.18.2@sha256:75eab8c4ba42096724fdcfde8b4de0b5713d529dde32f285a1f86fdcb2c9e50c`
|
||
and
|
||
`ollama/ollama:0.32.0@sha256:57f573b47f1f71ebb445789f279fe3e596a8beab182f7cf486db9205bad87c5a`.
|
||
The workspace-registry smoke fix-round image used tag
|
||
`thothii-workspace-registry-smoke:thoth-workspace-registry-smoke-thoth-workspace-registry-smoke-10vi3a-19157`,
|
||
built manifest list `sha256:715b943057929418cad4aa71806d9edbaf823555d19bda6b875297617463fd4a`
|
||
with config `sha256:a566521981e08958aae9a12bfc7803bb5f3f835536b4bb8c39df8fcf26063161`,
|
||
and removed that exact reference during cleanup. Local GPU exposure (`THOTH_ENABLE_EMBEDDING_GPU=1`) and
|
||
Windows Docker Desktop startup were not manually executed in this run.
|
||
- **Task 13 known limitations.** Broad harness Ruff remains existing unrelated debt
|
||
(**220 errors**); touched harness files were verified Ruff-clean. The final active-reference
|
||
audit remains non-empty only in categorized legacy parser/migration compatibility, legacy
|
||
descriptor/config fixtures, deterministic negative guards, retained off-repository migration
|
||
SQL, L2 legacy fixtures, gitignored task notes, and historical reference notes. No active
|
||
schema-v3 operator manual or supported runtime deployment path retains external vector or
|
||
embedding endpoint coupling.
|
||
- **Final review fix verification.** Backend Vitest passed **477/477** plus TypeScript and build;
|
||
harness pytest passed **827 passed / 4 deselected** with the existing 74 warnings; touched Python
|
||
files are Ruff-clean. The complete `thothctl` Go suite, deterministic backup/restore safety test,
|
||
internal semantic Compose contract, no-deployment-coupling gate, CPU/offline semantic smoke, and
|
||
unified deployment smoke all pass after the final fix. The intermittent `thothctl` rollback failure was
|
||
traced to Docker Desktop alternating equivalent bind sources between `/private/...` and
|
||
`/host_mnt/private/...`. Exact state-v4 source hashes remain unchanged; only fresh bind
|
||
observations made by a Darwin `thothctl` carry non-serialized aliases for the rollback
|
||
comparison, so pre-fix recovery state remains readable and Linux `/host_mnt` paths remain
|
||
distinct. The rollback-only smoke passed twice consecutively after each fix revision, and the
|
||
subsequent full unified smoke passed with exact cleanup.
|
||
|
||
## Historical snapshots and archived reference notes
|
||
|
||
### Historical snapshot — Unified deployment release gate, Task 13 (2026-08-05)
|
||
|
||
- **Release coverage.** `scripts/unified-deployment-smoke.sh` gates the two-service render/build,
|
||
frontend-to-core routing, embedded pinned Pi, Git registry bootstrap, offline recreation, valid
|
||
update, invalid-update retention, and the four persistent stores. `scripts/thothctl-update-smoke.sh`
|
||
independently exercises the bad-Pi update and automatic rollback path.
|
||
`scripts/server-deployment-smoke.sh` starts the server plus required session overlays with the
|
||
same smoke-built core/frontend images, disposable bind roots/secrets/session configuration,
|
||
upstream-auth checks, and fail-closed unavailable-session behavior.
|
||
- **Isolation and disclosure boundary.** Every run generates a unique temporary root, Compose
|
||
project, container/image names, transaction image tags, and run label. The rollback fixture uses
|
||
an immutable `hello-world` digest whose preflight exits successfully, guaranteeing the stopped
|
||
core state required by `thothctl` compensation. Cleanup includes stopped project containers in
|
||
its final ownership check immediately before teardown and removes only exact containers,
|
||
Compose resources, image references, control state, and temporary files. There is no global
|
||
prune. Failure diagnostics are bounded and sanitized, and all credentials/endpoints used by the
|
||
smokes are disposable fixtures rather than operator or repository secrets. Every public smoke
|
||
also has an internal 30-minute process-group supervisor with TERM/KILL of the complete group.
|
||
- **Cross-platform CI contract.** `.github/workflows/deployment.yml` uses immutable action commits,
|
||
pinned supported Node and Go versions, runs LF/Compose/secret/coupling/docs/TypeScript gates on
|
||
Linux, runs each Linux Docker smoke once under its own outer timeout, and copies the Windows
|
||
source into a path containing spaces before building/invoking native `thothctl` and rendering
|
||
Compose. The optional `windows_docker_startup` dispatch targets a labelled self-hosted Windows
|
||
Docker Desktop/WSL2 runner and performs bounded two-service startup and exact cleanup. No local
|
||
Windows or Windows Docker execution is claimed until that manual job is recorded.
|
||
- **Validation status.** Deterministic Phase A gates, backend **434/434** plus TypeScript,
|
||
frontend **386/386** plus TypeScript, and harness **862 passed / 5 L2 deselected** are green.
|
||
Review round 1 ran each Docker smoke exactly once without retry. Unified (`103.86s`) and
|
||
update-only (`46.45s`) passed build/start, core/Pi/registry/persistence setup and the stopped
|
||
candidate preflight, but `thothctl` stopped before mutation at its active-session inventory gate.
|
||
Round 2 replaces presence-only fixture checks with generated Compose renders plus the production
|
||
workspace resolver; this found and fixed missing explicit direct transport selections. The
|
||
clean-server preflight now atomically initializes the three hidden Pi-agent targets under the
|
||
writable parent bind while protected/tracked sources remain separate read-only mounts. Clean
|
||
empty-root render/setup and wrong-service/value/mount mutations are green. The corrected server
|
||
one-shot built and started both healthy services from an empty Pi-state root, then stopped at an
|
||
incorrectly addressed authenticated frontend hop. Fix round 3 adds the exact fourth private
|
||
non-admin claim and proves its nginx/backend transformation in a focused auth test. It also
|
||
centralizes schema-v2 registry descriptor resolution and secret-safe runtime rendering in
|
||
`ThtRunner`, preserving canonical revision identity and durable session roots for inventory,
|
||
create/resume/show, SQL, and Pi calls. The fresh update-only one-shot now passes mutation,
|
||
automatic `rolled_back` compensation, exact prior-image restoration, unchanged registry head
|
||
and mount identities, all four persistence sentinels, post-rollback doctor/workspace checks,
|
||
and exact labeled-resource cleanup. The one authorized server invocation was blocked at its
|
||
first Docker readiness call by the execution sandbox's socket permission before any Compose
|
||
resource could be created, so authenticated workspace/fail-closed session behavior remains an
|
||
explicit release gate. Native Windows PowerShell/Docker execution also remains pending.
|
||
|
||
### Historical snapshot — Portable deployment decoupling (superseded 2026-08-08)
|
||
|
||
- **Mandatory stack.** The supported Compose stack is exactly `frontend` plus `core`; use the
|
||
base file with `deploy/compose.local.yaml`, or with `deploy/compose.server.yaml` plus the
|
||
required public-server session overlay. `run-stack.sh`
|
||
invokes the base+local Compose command and the core image provides Pi, so no host Pi binary is
|
||
part of the launch contract.
|
||
- **External boundaries.** DWH, vector DB, embedding, LLM, and reverse-proxy services are
|
||
external configurable endpoints even when deployed on the same infrastructure. The two
|
||
superseded PSD/portal deployment overlays were removed. Workspace descriptors and migration
|
||
utilities remain separate from deployment runtime configuration.
|
||
- **Legacy PSD deployment ruling.** The PSD bootstrap was deleted because it generated the
|
||
retired overlay and was therefore deployment machinery, not a data migration utility. Its
|
||
remaining live contract checks were renamed for the generic local Compose profile. The coupling
|
||
gate rejects stale active deployment filenames and content while deliberately excluding
|
||
historical plans/specs, canonical workspace descriptors, and non-runtime migration helpers.
|
||
- **Fresh provider and secret contract.** Local, server, and standalone development mount the
|
||
protected Pi auth JSON plus tracked declarative model/settings files read-only under
|
||
`/home/thoth/.pi/agent`. The existing strict application bundle is a core-only Docker secret at
|
||
`/run/secrets/thothii.secrets`; operator env files contain only its absolute source path.
|
||
Provider readiness is exercised from a fresh Compose volume through model listing, configuration,
|
||
and sanitized credential status.
|
||
- **Install and scan closure.** Superseded copied one-service installation examples and the
|
||
provider-owned-network test are retired. Active manuals use the canonical base plus local/server
|
||
and optional overrides, while the category-based coupling scan covers runtime, Docker smoke,
|
||
install, operator, and positive deployment-test contracts and propagates scanner errors.
|
||
|
||
### Historical snapshot — Portable Git workspace registry, pre-schema-v3 (superseded 2026-08-08)
|
||
|
||
- **Source of truth and scope.** The canonical workspace repository is a generic Git remote,
|
||
configured only by `THT_WORKSPACE_GIT_REMOTE` and `THT_WORKSPACE_GIT_BRANCH` (there is no
|
||
committed PSD/Chirone remote or branch default). Both a local Docker installation and a server
|
||
persist its checkout, validated snapshots, state, and locks at `/data/workspace-registry`.
|
||
Connector endpoints, transport choices, and secret-file paths remain local bindings; secret
|
||
contents are never stored in Git, API responses, browser storage, diagnostics, or bundles.
|
||
- **Migration and session safety.** Schema-v2 descriptors are operational; legacy descriptors are
|
||
visible as `migration_required` until migrated by the documented operator workflow. New sessions
|
||
acquire a persistent revision lease before readiness and persist workspace ID plus immutable Git
|
||
revision. Retention hands that lease off only after an authoritative scan observes the manifest,
|
||
so a stale concurrent scan cannot prune the pinned snapshot. Resume resolves that historical
|
||
snapshot, while retention preserves every revision referenced by an open, closed, or failed
|
||
unarchived manifest.
|
||
Reconciliation runs only with a complete local installation list or an administrator's complete
|
||
server list, never from a remote user's partial view.
|
||
- **SSH connector boundary.** The current OpenSSH forward is owned by one bounded diagnostic and is
|
||
always cleaned up afterward. DWH/vector `ssh_tunnel` bindings therefore return
|
||
`workspace_not_activatable`, and new-session creation rejects them before persistence. Direct and
|
||
REST runtime connectors remain supported; Git remote access over SSH is unaffected.
|
||
- **Operator manuals.** Follow [the local manual](docs/install/local-workspace-registry.md) for
|
||
macOS/Windows/Linux Docker Desktop deployment and [the server manual](docs/install/server-workspace-registry.md)
|
||
for Gitea-compatible remotes, reverse proxy, migration, backup, and recovery. The release
|
||
workflow is Git review/push → installation pull → validate → local diagnostic test → browser-local
|
||
workspace/model/reasoning selection → revision-pinned session.
|
||
- **Verification recorded for this source branch.** `git diff --check` passed; backend Vitest
|
||
**371/371** and TypeScript passed; frontend Vitest **398/398** and TypeScript passed; the
|
||
harness document regression passed **10/10**. `./scripts/workspace-registry-smoke.sh` and the
|
||
executable installation-manual fixture verifier passed with Docker. A final unrestricted full
|
||
harness run remains a release command for the deployment environment; the earlier local
|
||
long-running harness run was intentionally cancelled before it produced a final result.
|
||
|
||
### Historical snapshot — Session summary redesign (2026-07-23)
|
||
|
||
- Session documents are projected at read time in outcome-first order: original question,
|
||
final SQL, persisted data preview, revised question, assumptions, one memory list, then
|
||
remaining technical documents. This applies to existing filesystem and repository-backed
|
||
sessions without rewriting their artifacts.
|
||
- Final SQL has an always-visible clipboard action. All prose, including original/revised
|
||
questions, assumptions, memory content, and remaining decisions, renders as Markdown.
|
||
- Memories are shown once, approved before declined. The generic decision list suppresses
|
||
memory ledger records plus `phase_approved`, `phase_auto_approved`, `table_approved`,
|
||
`table_promoted`, and `column_promoted`.
|
||
- The session summary has its own accessible pointer/keyboard resize separator, persists its
|
||
width independently from Model activity, reaches 50% when space permits, and preserves a
|
||
512 px right-side minimum on narrower desktop layouts.
|
||
- Verification: harness **861 passed / 5 L2 deselected**, Pi gate **163/163**, full frontend
|
||
suite, TypeScript check, production build, Ruff, and `git diff --check` all passed. A live
|
||
pre-existing session returned the new canonical order and none of the suppressed decision
|
||
labels. Compose rebuilt and force-recreated both services; core image
|
||
`sha256:3566d1258b956f8ca96d3b5ff8f625503247a7fd3b0dffe77020ae03956403d8`
|
||
is healthy and frontend image
|
||
`sha256:8311ca1308b459ece7236bf143da7b1a226ff4082fed924e1b5a207c24b6ca29`
|
||
is running. Frontend and `/api/health` both returned HTTP 200.
|
||
|
||
### Historical snapshot — Local Pi user auth + startup failure handling (2026-07-21)
|
||
|
||
- The PSD Docker profile now bind-mounts the configurable host `PI_AUTH_FILE` read-only at
|
||
`/home/thoth/.pi/agent/auth.json`; on this Mac it resolves to the real user profile
|
||
`/Users/mp/.pi/agent/auth.json`. The container keeps its correct Linux identity
|
||
`HOME=/home/thoth` while Pi sees the user's independent `deepseek` and `zai` credentials.
|
||
- `deploy/pi/settings.json` is the non-secret model policy and exposes, in order,
|
||
`zai/glm-5.2`, `deepseek/deepseek-v4-flash`, `deepseek/deepseek-v4-pro`, and
|
||
`aritmolab/qwen3.6-35b-a3b`. The core image is aligned to Pi 0.80.3.
|
||
- New-session creation now validates the saved provider/model against Pi before persistence;
|
||
unavailable selections return sanitized `503 model_unavailable` without creating a manifest.
|
||
A synchronous runtime-construction failure after persistence marks that session `failed` and
|
||
returns the fixed startup-recovery message instead of leaving an ambiguous `open` session.
|
||
- Verification: backend 235/235, TypeScript clean, dedicated Compose auth/model contract green
|
||
with a demonstrated RED→GREEN cycle. Rebuilt core image
|
||
`sha256:a8b4dd9f016c2335e4da897073bc6d5bdf1e8ce9b60dcfcca171b6677f563228`
|
||
is healthy; live `/models` returned all four models; a real `deepseek-v4-pro` smoke reached its
|
||
first reviewer gate, deleted only its own session, and restored the exact prior settings.
|
||
- Deleted the three explicitly approved incomplete DeepSeek attempts:
|
||
`a390c8b8-0a91-4a37-967b-ce7ff9be9797`, `a2f974b2-4c48-4967-b4b6-afdbc2b2d541`, and
|
||
`f66e1959-3c71-4b10-8aa1-606992046b7e` (API delete 204, subsequent lookup 404 for each).
|
||
|
||
### Historical snapshot — User-owned sessions cutover (2026-07-16)
|
||
|
||
- **Target contract:** the public server runs `AUTH_MODE=upstream` with Task 4 portal identity
|
||
forwarding and Task 5 principal enforcement deployed together. Its session source of truth is
|
||
direct TLS-verified PostgreSQL `thoth_sessions`; local development remains loopback-only with
|
||
filesystem sessions under `THT_HOME`. The core never receives the migrator credential.
|
||
- **Deployment material:** copy `deploy/compose.session-server.yaml.example` and
|
||
`deploy/workspaces/server-sessions.yaml.example` into reviewed, untracked operator files. The
|
||
runtime password, migrator password, and CA are three separate Docker secret mounts; server
|
||
startup rejects public/local storage and incomplete server DB/TLS configuration.
|
||
- **Readiness behavior:** `/health` remains the unauthenticated process liveness endpoint. Any
|
||
route requiring unavailable session/preferences storage returns fixed HTTP 503 before starting
|
||
Pi; this is intentional and must not be hidden by changing liveness to a database check.
|
||
- **Manual cutover only:** schedule maintenance, drain Pi work, run the one-shot migrator and
|
||
require `pending=[]` and `drifted=[]`, then replace core and perform an authenticated storage
|
||
smoke. Archive/checksum the three reviewed legacy filesystem session directories before deleting
|
||
exactly those three with `docker/cutover-legacy-sessions.sh --delete`; no deletion has been run
|
||
from this repository task. Do not import their untrusted ownership.
|
||
- **Rollback:** PostgreSQL remains the single source of truth. Revert only to a compatible fixed
|
||
release; never re-enable filesystem persistence, restore the archive into production, or
|
||
dual-write during rollback.
|
||
|
||
### Historical snapshot — Docker locale deployment, Profile A (superseded 2026-08-05)
|
||
|
||
ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale).
|
||
|
||
- **2 container** su `compose.yaml`: `thothii-core` (Fastify + harness tht + Pi) + `thothii-frontend` (Vite + nginx-unprivileged). Rete `omics_portal_omics_network` (external) con alias `thothii-core`/`thothii-frontend`.
|
||
- **DB**: Postgres diretto `:5438` (stessa istanza: schema `datawarehouse` 163 tabelle + `vectors` pgvector). Ruoli dedicati `thoth_dwh_reader` (read-only) + `thoth_vector_rw` (read+write). Embeddings: Ollama `:11434`.
|
||
- **Secrets**: `deploy/thothii.env` (env_file, gitignored) + `THT_MODEL_API_KEY_FILE` (key modello, file 0600 — meccanismo provider-credentials di Codex) + bind-mount `~/.pi` (pi-config).
|
||
- **Backend**: merge di `codex/portable-deployment` (secret-bundle, provider-credentials, auth `upstream`, security hardening, CI multiarch). Setup Docker MIO tenuto (il modello Docker-secrets di Codex è in `deploy/` come alternativa inerte).
|
||
- **Portale** (repo `omics_portal`, branch `agent/patient-capabilities-datamart-ui`): `nginx.conf` rotte `/datamart-builder/api`+`/assets` + `auth_request`, template `datamart_builder.html` (mount `<div id="root">` + tag `{% vite_assets %}`), vista `datamart_builder_api_auth`. Auth: `authentik Admins` bypass; utenti normali necessitano gruppo `omics-datamart-builder`.
|
||
- **Fix load-bearing**: `configPath` da `THT_CONFIG` (route senza workspace), `vite_assets` `mark_safe` (SPA bianca), `COPY harness/`+`cp workflow.yaml`+`pip install .` (pip 26 / tht module-relative), entrypoint `server` case.
|
||
- **Standalone/dev**: `docker-compose.dev.yml` (rete propria, porte host 8787/8090) + `scripts/docker-smoke.sh`.
|
||
- Piano dettagliato: `docs/superpowers/plans/2026-07-12-local-docker-deploy-implementation.md`.
|
||
|
||
### Archived snapshot — Runtime incident fixes (2026-07-13)
|
||
|
||
- The bind-mounted Pi profile came from host paths and did not trust `/app/harness`.
|
||
Pi 0.80 consequently loaded **zero** project extensions, prompts and skills, silently
|
||
sending `/nuova-domanda`/`/riprendi-sessione` to the model as plain text. The core
|
||
entrypoint now idempotently adds only `/app/harness` to the persistent
|
||
`/home/thoth/.pi/agent/trust.json`, preserving all existing decisions.
|
||
- The gate embeds the canonical `tht-sessione/SKILL.md` in the one-shot kickoff system
|
||
prompt and explicitly prohibits repository discovery. A live RPC `get_commands` must
|
||
show `torna`, `nuova-domanda`, `riprendi-sessione`, and `skill:tht-sessione` after deploy.
|
||
- Workspace identity is derived from the resolved config path, so
|
||
`config/tht.yaml -> workspaces/local.yaml` matches DWH artifact ownership (`local`).
|
||
- Direct pgvector now discovers the actual namespaces of the `vector` type and cosine
|
||
operator from PostgreSQL catalogs. This supports server layout `vectors.*` tables with
|
||
the extension installed in `public`.
|
||
- Live verification: session `2026-07-13-074712-dammi-la-lista-dei-pazienti-che-haoo-fat`
|
||
resumed directly at F1, ran `tht session show`, and completed `tht search pack`
|
||
(12 tables, 0 evidence, 2 solved) without repository exploration or adapter errors.
|
||
|
||
### Archived snapshot — Workflow/UI regression fixes (2026-07-14)
|
||
|
||
- **F1 Model Activity restored.** Session create/resume now preserves configured/persisted
|
||
thinking instead of forcing `off`. Pi's nested `thinking_delta` is bridged to a dedicated
|
||
named SSE `activity_delta`; EventSource subscribes to that name and the panel keeps it separate
|
||
from final assistant text. Reasoning remains in-memory and is not persisted to session artifacts.
|
||
- **F3 rewrite confirmation remains bypassed.** `rewrite_question` records approval and advances
|
||
automatically without a reviewer widget. The repeated prompt came from old running containers:
|
||
images had been rebuilt but services had not been recreated.
|
||
- **Join review is read-only and complete-set safe.** Join-only proposals render informational
|
||
cards with only `Continue` and `Other — specify`. Continue requires the exact complete id set;
|
||
all joins are persisted together by `decision add-join-set`, using an atomic ledger replacement
|
||
under a per-session cross-process writer lock. Other persists none of the rejected proposal.
|
||
- **CTE presentation fixed.** F6 CTE cards now structure purpose, rationale, tables, filters, keys,
|
||
and output columns with responsive wrapping/alignment. The Horizontal/Vertical switch is hidden
|
||
for a single SQL block (the per-CTE view), because it only affects multi-block layouts.
|
||
- **Latest render failure diagnosed and hardened.** Session
|
||
`2026-07-14-115847-estrai-i-pazienti-che-hanno-fatto-un-abl` sent an object in
|
||
`open_questions`, which React cannot render as a child. The v2 gate now enforces
|
||
`open_questions?: string[]`; the frontend also safely normalizes legacy malformed payloads.
|
||
- **Verification/deploy:** Python harness 798 passed / 5 L2 deselected; gate JS 126; backend 143;
|
||
frontend 250; TypeScript/build gates green. Compose rebuilt and force-recreated both services.
|
||
Running image ids: core `sha256:55acef2f12151ea97144c2f5e9164d63f2ca734bc2746fef553df94849e3fb3f`;
|
||
frontend `sha256:1043f79392420149655cc63d70461e2ca2005b2290a1e3e21dcf845ec3bd1c81`.
|
||
|
||
### Archived snapshot — Pi-enabled model selector (2026-07-14)
|
||
|
||
- **Pi is the allowlist authority.** `/models` reads the mounted Pi `enabledModels`, intersects
|
||
it with models currently available from Pi, and preserves the configured order. Enumeration
|
||
does not require `PI_PROVIDER`, does not inject generic/provider credentials, and fails closed
|
||
for missing or malformed scope.
|
||
- **Live scope:** exactly `deepseek/deepseek-v4-flash`, `zai/glm-5.2`, and
|
||
`local-qwen/qwen3.6-35b-a3b`. The live endpoint returned those three composite IDs once each and
|
||
in that order; `zai/glm-5v-turbo` and all other authenticated Pi models are hidden.
|
||
- **Validation/process smoke:** live settings updates returned 200 for DeepSeek Flash and local
|
||
Qwen, while hidden GLM-5V returned 400; a post-restore equality check confirmed the original app
|
||
settings were restored. The real `PiProcessManager` configure path succeeded for DeepSeek and
|
||
local Qwen without sending a prompt or starting a DWH operation; local Qwen required no hosted
|
||
provider key. Unknown and compound providers remain fail-closed in the verified backend suite.
|
||
- **Verification/deploy (`2026-07-14T19:24:22+02:00`):** backend **154/154** and frontend
|
||
**251/251** passed; both TypeScript gates and `git diff --check` were green. Compose built and
|
||
force-recreated only `core`; container start was `2026-07-14T17:24:01.992971976Z`, health was
|
||
`healthy`, and sanitized post-recreate logs contained only the backend listen line. Rebuilt image
|
||
ID and running container image ID both equal
|
||
`sha256:577f99754fd0731251c8ddd8608b1b8baee09d02fad66c759b23f8221083e676`.
|
||
|
||
### Archived snapshot — Qwen connectivity + state-aware Resume recovery (2026-07-14)
|
||
|
||
- **Pi turns have an explicit lifecycle.** The bridge tracks `idle`, `running`, `waiting`,
|
||
and `failed`; a reviewer gate is `waiting`, responses/steering return to `running`, and an
|
||
assistant provider error or unexpected Pi child exit becomes `failed`. Provider error details
|
||
are never forwarded to the client; the UI receives a fixed sanitized recovery message.
|
||
- **Resume preserves only active work.** `running`/`waiting` runtimes return as already active.
|
||
Every validated cold path—including recovery after a child has already exited—clears stale SSE
|
||
state before reopening and restarts from persisted provider/model/thinking with
|
||
`/riprendi-sessione`; `idle`/`failed` runtimes are torn down at that point. Failed validation does
|
||
not detach the existing stream. A successful Resume of the currently selected session also
|
||
closes and recreates its EventSource, so the replacement runtime cannot be left behind an old
|
||
same-ID stream.
|
||
- **Private Qwen routing is live.** Core is attached to both `omics_portal_omics_network` and
|
||
external `localllm_default`; frontend remains only on the portal network. The mounted Pi profile
|
||
resolves `local-qwen/qwen3.6-35b-a3b` at the sanitized base URL
|
||
`http://localllm-vllm:8000/v1`. A direct probe from core verified the model catalog and received
|
||
a non-empty real chat completion.
|
||
- **Verification/deploy (`2026-07-14T21:41:52+02:00`):** backend **168/168** and frontend
|
||
**256/256** passed; both TypeScript gates, both production builds, the Qwen Compose network
|
||
contract, and `git diff --check` exited 0. The initial deployment built and force-recreated
|
||
`core` and `frontend`; after the final crash-recovery review, only the affected `core` image was
|
||
rebuilt and force-recreated with no active Pi session. Core is healthy and its post-recreate
|
||
Qwen catalog/completion probe succeeded. Built and running image IDs match: core
|
||
`sha256:9867c2fa002b6f117da9a1d02c73b47bfe0372b8c1c137a5174c3e7ecb1e1db1`, frontend
|
||
`sha256:5a47f81bc887423e05cef8fd3feb075aa600cb33217247186124f60ca5a3005b`.
|
||
The frontend entry hash changed, so only `omics_portal-web-1` was restarted to invalidate its
|
||
indefinite Vite-manifest cache. The application-level Qwen smoke reached its first
|
||
`ui_request`, persisted the expected provider/model, deleted only its uniquely named smoke
|
||
session, restored the exact saved settings object, and left no smoke session or Pi runtime. The
|
||
supplied probe's success path left its keep-alive SSE reader open, so only that probe process was
|
||
terminated (exit 143), without touching backend, Pi, or unrelated runtimes. The same smoke then
|
||
exited 0 with `controller.abort()` in cleanup, preserving the gate, session cleanup, and exact
|
||
settings-restoration evidence.
|
||
|
||
### Archived snapshot — Complete activity timeline + CTE spacing (2026-07-15)
|
||
|
||
- **Model activity is complete from F1.** The left panel now records the submitted prompt before
|
||
session creation completes, then projects thinking, assistant output, sanitized tool lifecycle,
|
||
reviewer gates, status, and turn lifecycle in chronological order. It remains in-memory only;
|
||
closing and reopening the panel does not discard it, while Resume intentionally starts a fresh
|
||
live timeline.
|
||
- **Tool activity is a narrow public contract.** Only call id, tool name, and
|
||
`running`/`completed`/`failed` status cross Pi → backend → SSE. Updates, arguments, results,
|
||
commands, output, credentials, and raw errors stay server-side. Resume and SSE replay were also
|
||
hardened for process restarts, concurrent lifecycle requests, stale callbacks, cursor reset, and
|
||
multi-client reconnects.
|
||
- **CTE plan layout uses real Tailwind 3 spacing.** Shared cards use concrete 16 px default / 12 px
|
||
compact padding utilities; F6 CTE headers and content use responsive 16/20 px edge padding.
|
||
Semantic ordered steps, single-boundary divided filter/table lists, long-value wrapping, and a
|
||
plain top-divider rationale preserve the artifact content while improving scanability.
|
||
- **Final verification/deploy (`2026-07-15T02:18:54+02:00`, HEAD `56d73d2`):** backend
|
||
**204/204** and frontend **285/285** passed; both TypeScript gates and production builds exited
|
||
0, and `git diff --check` was clean. The final lifecycle fixes make restarted-hub cursor replay
|
||
generation-aware and mutate frontend delete/resume state only for sessions actually deleted.
|
||
Compose rebuilt and force-recreated `core` and `frontend`; built and running image ids match:
|
||
core `sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147`
|
||
(`healthy`), frontend
|
||
`sha256:dd7ef721b661b57b8c5422c47088e1a9a2e9821af021541cab372c9892fbd638`
|
||
(`running`). The entry changed from `index-DcZviApa.js` to `index-CuIt1NQg.js`, so only
|
||
`omics_portal-web-1` was restarted to refresh its indefinite manifest cache.
|
||
- **Final live local-Qwen smoke:** session
|
||
`2026-07-15-001922-final-no-thinking-activity-smoke-2026-07` observed **0**
|
||
`activity_delta` events while receiving **5** strictly allowlisted tool lifecycle events and the
|
||
first reviewer gate (`bash` running/completed and `reviewer_select` running). No forbidden tool
|
||
field crossed SSE. Cleanup closed with 200, deleted only that session with 204, restored the
|
||
exact settings object, and left no Pi runtime or smoke session.
|
||
|
||
### Archived snapshot — Filtered Model activity projection (2026-07-15)
|
||
|
||
- **Resolved contract.** `activityLog` still folds the complete in-memory prompt, thinking,
|
||
assistant, sanitized tool, reviewer-gate, status, and turn-lifecycle history. The left panel now
|
||
applies a default-deny rendering boundary and shows only prompt, thinking, status, and gate;
|
||
assistant text remains available to the central transcript, while tool, lifecycle, and unknown
|
||
future activity kinds do not render or move the panel scroll.
|
||
- **Frontend-only verification/deploy (`2026-07-15T03:44:02+02:00`, source HEAD `7208599`):**
|
||
frontend **287/287** passed; `npx tsc -b`, `npm run build`, and `git diff --check` exited 0.
|
||
Compose rebuilt and force-recreated only `frontend`; its image changed from
|
||
`sha256:dd7ef721b661b57b8c5422c47088e1a9a2e9821af021541cab372c9892fbd638` to
|
||
`sha256:75fc0b7786c75ded1b488e9fad3232aa061c82d89140c67ad62b5285954d1d36`, with container start
|
||
`2026-07-15T01:42:45.582760612Z`. The active Vite entry changed from
|
||
`index-CuIt1NQg.js` to `index-BdZpFO7j.js`, so only `omics_portal-web-1` was restarted at
|
||
`2026-07-15T01:42:53.715330158Z`; core retained image
|
||
`sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147` and start
|
||
`2026-07-15T00:18:37.040145636Z`.
|
||
- **Real local-Qwen no-COT smoke:** session
|
||
`2026-07-15-014328-activity-filter-smoke-2026-07-15t01-43-2` reached its first reviewer gate
|
||
with **0** `activity_delta` events and **3** tool lifecycle events, each containing exactly the
|
||
four public fields. The probe accepted the close response, deleted only that session with 204,
|
||
restored the exact saved settings object, confirmed the session absent, and left no Pi runtime.
|
||
|
||
### Archived snapshot — Central activity log + compact CTE density (2026-07-15)
|
||
|
||
- **Resolved UI contract.** The central working body now renders every chronological non-blank
|
||
assistant transcript line in one bounded accessible log, without user-entry echoes,
|
||
timer/spinner labels, or step messages. The left Model activity panel is a default-deny
|
||
projection of only thinking and status, while the complete raw activity fold and the existing
|
||
reviewer widgets, artifacts, and workflow state remain unchanged. F6 CTE headers, content,
|
||
table rows, and filter rows use 8 px vertical padding with 12 px lateral padding below `sm` and
|
||
16 px from `sm` upward; divider top padding is 8 px. The final review amendment keeps historical
|
||
log rows at the full muted-foreground token so their normal-size text retains AA contrast.
|
||
- **Source verification (source HEAD
|
||
`09f9bdffe582ff3c66845ca219f60c11472a550a`).** Frontend tests passed **292/292** across
|
||
**43/43** files. `npx tsc -b`, `npm run build`, the Impeccable layout detector, and
|
||
`git diff --check` all exited 0; the detector returned `[]`.
|
||
- **Frontend-only deployment.** The pre-deploy frontend was image
|
||
`sha256:8c9aaee6453b26c69e4057d3f9e53aa791d449b88ba4a0669a25cd442667d582`, started
|
||
`2026-07-15T10:17:16.919166255Z`, serving `index-CveSyban.js`. Compose built and
|
||
force-recreated only `frontend` with `--no-deps`; the final running frontend is image
|
||
`sha256:39dc47d81abcb13466218500476fbb78d1be66a424f4293470437700c848c26d`, started
|
||
`2026-07-15T10:30:30.869532521Z`, serving `index-CihtpQJV.js`. Because the entry changed,
|
||
exactly `omics_portal-web-1` was restarted: it retained image
|
||
`sha256:4580cf2bc85f3656ee515996c24ed02d9096d24ed0d8afe384a2f8bd8cf4bac9` and moved from
|
||
start `2026-07-15T10:17:36.151611451Z` to `2026-07-15T10:30:46.369205604Z`.
|
||
- **Isolation and final state.** Core remained `running`/`healthy` with exactly its original image
|
||
`sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147` and start
|
||
`2026-07-15T00:18:37.040145636Z`; frontend and portal were `running` with no container
|
||
healthcheck. Pre/post `docker top` showed only core's supervisor and backend server, so no
|
||
unrelated Pi runtime existed to disturb. The count-only frontend sensitive/error pattern scan
|
||
was **0**. No live model smoke was run, and settings and sessions were intentionally untouched.
|
||
|
||
### Archived snapshot — Resizable activity split + compact CTE rows (2026-07-15)
|
||
|
||
- **Resolved UI contract.** `activityLog` remains the complete in-memory chronological fold. The
|
||
left Model activity panel default-denies every kind except prompt, thinking, and assistant,
|
||
labels those entries Question, Reasoning, and Response in source order, and hides status, tool,
|
||
gate, lifecycle, and unknown kinds. The desktop panel is pointer/keyboard resizable from 288–576
|
||
px while preserving 512 px centrally, persists its global width in localStorage, and becomes an
|
||
overlay drawer below `lg` or whenever the measured app shell is narrower than 800 px. F6 CTE
|
||
cards retain their semantic structure and responsive grids;
|
||
lateral padding is 12/16 px, header/content edge padding is 8 px, internal section gaps are 12
|
||
px, heading/divider spacing is 4 px, and table/filter rows use 4 px vertical padding with compact
|
||
line heights.
|
||
- **Source verification (`2026-07-15T15:22:17+02:00`, source HEAD
|
||
`f1af1f909b387ae12a10f1b534bf6a529ea42505`).** Frontend tests passed **295/295** across
|
||
**44/44** files. `npx tsc -b`, `npm run build`, and `git diff --check` exited 0; the Impeccable
|
||
layout detector returned `[]`. The local source build emitted Vite entry
|
||
`assets/index-PlvQqhNG.js`.
|
||
- **Frontend-only deployment.** The pre-deploy frontend was image
|
||
`sha256:39dc47d81abcb13466218500476fbb78d1be66a424f4293470437700c848c26d`, started
|
||
`2026-07-15T10:30:30.869532521Z`, serving `index-CihtpQJV.js`. Compose built and force-recreated
|
||
only `frontend` with `--no-deps`; the final running frontend is image
|
||
`sha256:0947211373784a860c7507d03c0fcf1f901ac3d1547cd0aa62b914d158f15bd5`, started
|
||
`2026-07-15T13:21:08.29613952Z`, serving `index-Dn7T524a.js`. Because the entry changed, exactly
|
||
`omics_portal-web-1` was restarted: it retained image
|
||
`sha256:4580cf2bc85f3656ee515996c24ed02d9096d24ed0d8afe384a2f8bd8cf4bac9` and moved from
|
||
start `2026-07-15T10:30:46.369205604Z` to `2026-07-15T13:21:21.968808414Z`.
|
||
- **Isolation and final state.** Core remained `running`/`healthy` with exactly its original image
|
||
`sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147` and start
|
||
`2026-07-15T00:18:37.040145636Z`; frontend and portal were `running` with no container
|
||
healthcheck. Pre/post `docker top` showed only core's supervisor and backend server, so no
|
||
unrelated Pi process existed and no Pi process was stopped or steered. The count-only frontend
|
||
sensitive/error pattern scan was **0**. No live model smoke was run; settings and sessions were
|
||
intentionally untouched.
|
||
|
||
### Archived snapshot — Final activity-split fix (2026-07-15)
|
||
|
||
- **Source and verification (`2026-07-15T15:56:57+02:00`).** Deployed source commit
|
||
`1f540fcb78ac9e552e56a21e47edf66e9872b323` (`1f540fc`). Frontend Vitest passed **298/298**
|
||
tests across **44/44** files; `npx tsc -b` and `npm run build` exited 0. The Impeccable detector
|
||
scoped to AppShell, ModelActivityPanel, index.css, and CtePlanViewer returned `[]`; `git diff
|
||
--check` exited 0.
|
||
- **Frontend-only deployment.** Compose built and force-recreated only `frontend` with `--no-deps`.
|
||
The frontend image changed from
|
||
`sha256:0947211373784a860c7507d03c0fcf1f901ac3d1547cd0aa62b914d158f15bd5` to
|
||
`sha256:6e14f55092b7e3aca9a396220394ae484147674d81b051771e394e59b73b1c88`; its active Vite entry
|
||
changed from `index-Dn7T524a.js` to `index-BIznZeLH.js`. Therefore exactly
|
||
`omics_portal-web-1` was restarted to refresh its manifest cache; it retained image
|
||
`sha256:4580cf2bc85f3656ee515996c24ed02d9096d24ed0d8afe384a2f8bd8cf4bac9` and started at
|
||
`2026-07-15T13:56:32.786693915Z`.
|
||
- **Isolation and final state.** Core retained image
|
||
`sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147` and exact original
|
||
start `2026-07-15T00:18:37.040145636Z`, remaining `running`/`healthy`. Final frontend and portal
|
||
states are `running` (no healthcheck). Pre/post core process tables contained only the supervisor
|
||
and backend server, so Pi was preserved and no Pi process was stopped or steered. The count-only
|
||
frontend sensitive/error-pattern scan was **0**. No model smoke was run; settings and sessions
|
||
were intentionally untouched.
|
||
|
||
## What ThothII is
|
||
|
||
A **human-in-the-loop datamart builder**: it turns a natural-language question into
|
||
validated SQL (and optionally a dbt datamart) through a deterministic **8-phase
|
||
NL→SQL workflow**, where the model *proposes* and a human *reviewer decides* at gates.
|
||
The UI is meant to embed inside the Omics Portal (GSD design system) and is **English**.
|
||
|
||
## Architecture — three layers
|
||
|
||
```
|
||
frontend (React, :5173) → backend (Fastify, :8787) → pi --mode rpc → tht / harness → DWH (read-only)
|
||
```
|
||
|
||
- **harness/** — the Pi layer. A deterministic Python CLI **`tht`** + a Pi gate extension
|
||
(`.pi/extensions/tht-gate.js`) that runs the 8-phase workflow and emits/consumes
|
||
widget-descriptor JSON. **Owns all persistence.** Workflow truth is `harness/workflow.yaml`;
|
||
orchestration rules are `harness/.pi/skills/tht-sessione/SKILL.md`.
|
||
- **backend/** — Fastify + TypeScript. A **thin bridge**: proxies REST routes to the `tht`
|
||
CLI (`ThtRunner`), manages Pi processes (`PiProcessManager`, one child per session),
|
||
bridges Pi RPC events to SSE (`SessionBridge` + `SseHub`). No application database.
|
||
- **frontend/** — React 18 + base-ui + Tailwind + TanStack Query + Zustand. Chat-style
|
||
shell (`src/shell/AppShell.tsx`); the live transcript is rebuilt in-memory from the SSE
|
||
stream (`src/store/sessionStore.ts`), **not persisted**.
|
||
|
||
### Persistence model (the load-bearing premise)
|
||
There is **no verbatim chat store**. Each workflow phase persists its own document into the
|
||
session directory, and that **IS** the persistence. A session = a directory under the
|
||
workspace's `sessions/` path containing `session_manifest.yaml` + phase artifacts
|
||
(`question.md`, `schema_linking.json`, `cte_plan.json`, `sql_final.sql`,
|
||
`validation_report.md`, `review_decisions.jsonl`, …). A fresh Pi process resumes by reading
|
||
`tht session show <id>` + the on-disk artifacts — never by replaying chat.
|
||
|
||
## The 8 phases (harness/workflow.yaml)
|
||
F1 chiarimento · F2 memoria · F3 riscrittura (`question.md`) · F4 schema_linking
|
||
(`schema_linking.json`) · F5 sintesi · F6 cte (`cte_plan.json`, `cte_tests.json`) ·
|
||
F7 sql_finale (`sql_final.sql`) · F8 datamart. Current phase is a fold over the decision
|
||
ledger (`harness/tht/phase.py`); statuses: `open` / `closed` / `finalized`.
|
||
|
||
## How to run
|
||
|
||
**Full stack (real Pi + DWH):** from project root,
|
||
```bash
|
||
./scripts/run-stack.sh
|
||
# Opens frontend: http://localhost:5173 (proxies backend :8787)
|
||
```
|
||
**Prereqs:** VPN on; `pi` on PATH (with a configured model, e.g., `pi model set claude-fable-5`);
|
||
`harness/.env` populated (see `.env.example`); `harness/config/tht.yaml` → workspace (psd recommended for testing).
|
||
All three layers' deps installed (`npm install` in each, `python -m venv + pip install -e ".[dev]"` in harness).
|
||
|
||
**Individual dev:**
|
||
- backend: `cd backend && npm run dev` (tsx watch; env: `PORT`, `THT_HARNESS_DIR`, `THT_BIN`, `PI_BIN`, `AUTH_MODE`)
|
||
- frontend: `cd frontend && npm run dev` (Vite; `VITE_BACKEND_URL` → backend)
|
||
- harness install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` → `tht` on PATH
|
||
|
||
## How to test (latest TS gates green 2026-07-15: backend 204 / frontend 292 (43 files); harness 798 / gate JS 126 last recorded 2026-07-14)
|
||
- harness: `cd harness && .venv/bin/pytest -q` (5 L2/real-DB tests are deselected by default)
|
||
- backend: `cd backend && npx vitest run` · typecheck `npx tsc --noEmit -p .`
|
||
- frontend: `cd frontend && npx vitest run` · typecheck `npx tsc -b` · e2e `npm run e2e` (Playwright)
|
||
|
||
## Config & workspaces
|
||
- Workspaces: `harness/workspaces/*.yaml` (`psd`, `tht-test`, `tht.example`). A workspace sets
|
||
the DB target and the **absolute** `paths.sessions/artifacts/indexes` (psd → a *separate*
|
||
repo `tht-workspace-psd/`, NOT committed here).
|
||
- Secrets live ONLY in `harness/.env` (gitignored; `THT_*` — DB, DWH REST, vector, SSL CA…).
|
||
See `harness/.env.example` for the variable list.
|
||
- App settings (global): `{ workspace, provider, model, thinking }`, persisted via harness
|
||
preferences (`tht session preferences get/set` → the configured session repository —
|
||
filesystem or Postgres in server mode). `backend/data/settings.json` (gitignored) remains
|
||
only the file fallback for injected runners/tests. The "New session" form is question-only;
|
||
these settings supply the rest.
|
||
|
||
## Efficiency levers (NL→SQL workflow optimization, 2026-07-08)
|
||
|
||
Three deployed optimizations target model thinking time (the dominant cost, ~220s in F1 alone):
|
||
|
||
1. **Join-graph via FK annotations + `tht schema suggest-fks`**
|
||
- DWH has no FK constraints declared. Annotations file (`tht-workspace-*/artifacts/mschema/annotations.yaml`) now stores curated logical FKs.
|
||
- Three ranking rules: mine from approved SQL (highest confidence), heuristics (`*_time_key → dim_time.day_key`), same-name PK discovery with `--assume` flag for disambiguity.
|
||
- **Psd workspace:** 228 FK suggestions already generated (139 tables); `tht schema suggest-fks --from-sql <session-dir> --assume cod_paz=dim_patient` for updates.
|
||
- **Activation:** automatic. The mschema renderer populates the `【Foreign keys】` section. F4 in SKILL.md now reads FK joins from there instead of the model re-deriving them.
|
||
|
||
2. **Context-pack consolidation at F1 kickoff (`tht search pack`)**
|
||
- Single embedding of the question, reused for schema + evidence + solved-question searches.
|
||
- Command: `tht search pack "<question>" --session <id>` → `sessions/<id>/retrieval_pack.md` (tabelle candidate, relevant evidence, solved exemplars).
|
||
- Graceful degradation: if Ollama or vector store unreachable (no VPN), sections are empty but exit 0 — session continues with live searches.
|
||
- **Activation:** automatic at next session. SKILL.md F1 now prescribes as first call; reduces exploratory turns.
|
||
|
||
3. **Phase-summary recap auto-construction from session ledger**
|
||
- `tht session show --json` includes the full decisions ledger; `tht phase meta --json` exports decision types per phase.
|
||
- Gate appends deterministic `【Decisioni registrate in questa fase】` section to v2 phase-summary artifacts.
|
||
- Model authors only `summary` + `checks`; the gate fills the recap table from persisted state → exact by construction.
|
||
- **Activation:** automatic at next session and Pi restart. SKILL.md Disciplina 6 updated: model keeps output brief, gate enriches from catalog + ledger.
|
||
|
||
**Tests:** 358 Python + 111 JS gate, all pass. L2 (live DWH) verification on psd workspace recommended when time permits.
|
||
|
||
## Conventions & contracts (don't relearn the hard way)
|
||
- **`-c`/`--config` is a PER-COMMAND option in `tht`** — append it AFTER the subcommand,
|
||
never globally (`ThtRunner.buildArgv` handles this).
|
||
- **`--json` output must be pristine** (only valid JSON on stdout).
|
||
- **UI strings are English.** Document *content* stays in the workspace language (Italian
|
||
for psd) because it's the real data; only chrome/labels are English.
|
||
- **Settings are global**, not per-question.
|
||
- TDD throughout; tests assert real behavior, not mocks. Frequent, scoped commits.
|
||
- Global user rules (`~/.claude/CLAUDE.md`): think before coding, simplicity first, surgical
|
||
changes, goal-driven verification.
|
||
|
||
## Active memory — F8 promotion gate + solved-question recall — SHIPPED, L2 pending (2026-07-07)
|
||
|
||
Two additions to close the loop on reusable memory, on top of the existing `tht memory
|
||
search` (Phase 2) reuse:
|
||
|
||
- **F8 promotion gate.** `reviewer_memory_promote` (Phase 8, called with only the session
|
||
id): the gate computes candidates deterministically via `tht memory promote --preview
|
||
--json` (the 3 reusable decision types, already excluding previously promoted/declined
|
||
ones) and shows a pre-selected checklist. Selected → `tht memory save-one` persists to
|
||
the vectordb + records `memory_promoted`; deselected → `memory_promotion_declined`
|
||
(ledger detail `seq:<n>`) so it is never re-proposed. `tht memory promote`/`save-one` were
|
||
added to the gate's anti-bypass FORBIDDEN list (model must go through the gate tool).
|
||
- **Solved-question exemplars.** New vector kind `solved_question` reusing the existing
|
||
`memory` pgvector table (no server-side DDL); `harness/tht/solved.py` does a one-row
|
||
upsert keyed by a hash of question+SQL. CLI: `tht memory solved-index` / `solved-search`.
|
||
`tht session finalize` auto-indexes the pair (best-effort: green line on upsert, cyan
|
||
"già aggiornata" on dedup no-op, yellow warning + the recovery command
|
||
`tht memory solved-index <id>` on failure). `SKILL.md` now prescribes calling
|
||
`solved-search` as reference-only context in F4 (schema linking), F6 (CTE plan) and F7
|
||
(final SQL), and documents the finalize auto-index in "Session end".
|
||
|
||
**Pending L2 gate (not yet run — needs VPN + writer key):** one live end-to-end session on
|
||
workspace `psd` via `./scripts/run-stack.sh` to verify (a) the promotion checklist renders
|
||
pre-selected and persists selected/declined correctly, (b) finalize indexes the pair,
|
||
(c) `tht memory solved-search` returns it with sql + tables.
|
||
|
||
**Fast-follow:**
|
||
- RestSearcher top-k dilution — **client-side DONE** (2026-07-07): `search_similar` manda
|
||
`kinds` alla RPC (filtro server-side esatto) con fallback automatico su server legacy
|
||
(404 → retry senza filtro, post-filter client). **Resta la migrazione server** della
|
||
funzione SQL `search_similar` (+`kinds text[] DEFAULT NULL`): istruzioni pronte in
|
||
`harness/docs/vector-rest-kinds-migration.md`; l'ordine di deploy è libero, ma fino
|
||
alla migrazione il filtro resta client-side e la diluizione persiste.
|
||
- ~~`tht memory solved-search` muore con traceback grezzo se il vectordb è irraggiungibile~~
|
||
**DONE** (2026-07-07): degrada a warning di una riga su stderr, stdout puro (`[]` in
|
||
--json), exit 0 — copre VectorRestError/EmbeddingsError/OperationalError.
|
||
|
||
## Review gates v2 — payload strutturati + viewer dedicati — COMPLETE (2026-07-07)
|
||
|
||
Plan: `~/.claude/plans/prima-di-passare-ai-inherited-marshmallow.md`. Merged to `main` @ `2410f01`
|
||
(ff, pushed). Executed via subagent-driven-development (4 workstreams, task reviews, final
|
||
whole-branch review + fix wave).
|
||
|
||
- **Contracts:** `artifact.data.schema_version: 2` for `cte_plan` / `cte_result` / `phase`,
|
||
built **deterministically by the gate** (catalog descriptions via `tht schema columns`; SQL
|
||
from `ctes/<name>.sql`; preview rows persisted by `tht cte test`); the model contributes only
|
||
purpose/rationale/note. Non-v2 payloads fall through to the legacy renderers — old sessions
|
||
and `tools/replay/replay.json` keep working.
|
||
- **Harness (Python):** `CteTestRecord.preview_rows` (+ `_jsonable` coercer, ≤10 rows, cells
|
||
≤200 chars); new read-only `tht cte info <name> --session <id> --json` (index/total from
|
||
`cte_plan.json`, same source as `next_cte`); `tht cte plan --doc -` writes
|
||
`cte_plan_doc.json` (chain documentation; `cte_plan.json` stays a load-bearing `list[str]`).
|
||
- **Gate (JS):** `gate/artifact-contracts.js` (soft validators → self-corrective `textResult`,
|
||
TypeBox untouched) + `gate/enrich.js` (pure, catalog lookups injected);
|
||
`prepareReviewerArguments` now coerces `artifact.data` too (GLM stringified-param
|
||
mitigation); `SKILL.md` Phase 5/6 + disciplines rewritten (plan via `reviewer_confirm
|
||
kind:"cte_plan"` with payload A; `cte_result` gates send THIN data only — never SQL/preview
|
||
as text).
|
||
- **Frontend:** `artifactV2.ts` types; `CtePlanViewer` (per-CTE cards + chain strip),
|
||
`CteResultViewer` (shiki SQL + AG Grid preview), `PhaseSummaryViewer` (checks + criteria
|
||
with the VALUES driving choices), `PreviewGrid` extracted from `ResultsPanel`,
|
||
`statusBadge.ts` shared success/warn/error tokens.
|
||
- **Replay:** v2 fixtures + `tools/replay/augment-review-gates.mjs`; `replay.json` regenerated;
|
||
offline visual pass ok (screenshots in the SDD scratch dir).
|
||
- **LIVE E2E (session `2026-07-07-011858`, GLM 5.2):** all 8 phases completed with the v2
|
||
gates; session **finalized** (DWH validation battery green, needs VPN).
|
||
- **Bug found live + FIXED (`2410f01`):** infinite spinner at workflow end — the bridge dropped
|
||
Pi's `agent_end` (the ONLY end-of-turn signal) and `working` was released only by the next
|
||
gate, which the final turn doesn't have. Now: bridge maps `agent_end` → SSE
|
||
`system_event`; FE tracks `agentActive`; an unexpected Pi child exit notifies the client
|
||
(info error + synthetic `agent_end`). Memory: `pi-rpc-event-vocabulary`.
|
||
- **Open (non-blocking):** `tht.sqlcheck` maps table aliases by first occurrence (found and
|
||
worked around by the model in F6 — spawned as a separate task); one more live confirmation
|
||
that the spinner stops at F8 (the chain is unit-tested end to end).
|
||
|
||
## F4 schema-linking column curation + look&feel v2 — COMPLETE (2026-07-06)
|
||
|
||
Branches `feat/f4-schema-linking-column-curation` (PR #1) + `feat/frontend-lookfeel-v2`, landed
|
||
on `main` (`d942635` … `7491e8c`). The F4 gate (`reviewer_schema_linking`) presents
|
||
catalog-enriched tables/columns (descriptions from `tht schema columns`, hardened enrichment),
|
||
per-table columns modal (suggested pre-checked, suggested-first ordering + filter box);
|
||
decisions `column_promoted`/`column_excluded` + deterministic `tht session
|
||
sync-schema-linking` projection into `schema_linking.json`. Live-verified including the
|
||
clobber test (the model's joins write preserves curated columns). Look&feel v2: shadows/radii/
|
||
mono labels, 70% gate modal, structured cards (colors untouched). Memory:
|
||
`thothii-visual-language-v2`.
|
||
|
||
## Workflow contract hardening — COMPLETE (2026-07-01)
|
||
|
||
Spec: `docs/superpowers/specs/2026-07-01-workflow-contract-hardening-design.md` · Plan:
|
||
`docs/superpowers/plans/2026-07-01-workflow-contract-hardening.md`. Merged to `main` @ `3dadc6f`
|
||
(pushed). Driven by analysis of Pi session `2026-06-30-165708` (GLM 5.2), where the model spent
|
||
~80% of its tool calls reverse-engineering the harness because `SKILL.md` mis-stated the
|
||
phase-advance contract — and Phase 6 was a hard dead-end. Three coordinated harness fixes (TDD):
|
||
|
||
- **F6 CTE-approval dead-end FIXED.** The gate's `reviewer_confirm kind:"cte_result"` used to
|
||
register `cte_approved --subject phase:6`, which `decision_cmd` rejects (exit 5 — it needs a
|
||
real CTE name from the plan) → F6 could never close. New `tht cte next --session <id>` returns
|
||
the first unapproved plan CTE; the gate now approves **by name**. (`tht/cli/cte_cmd.py`,
|
||
`.pi/extensions/tht-gate.js`.)
|
||
- **`schema_linking.json` writer/validator.** `store.set_schema_linking` (validates against the
|
||
`SchemaLinking` model, THEN writes — no partial file) → CLI `tht session set-schema-linking
|
||
<id> --file <path|->` (exit 5 on bad JSON / ValidationError) → gate tool `write_schema_linking`
|
||
(stdin). Replaces the model hand-writing the F4 artifact + ad-hoc python validation.
|
||
- **`SKILL.md` corrected to match the code.** Only F2-empty / F6-skipped auto-advance
|
||
(`_AUTO_ADVANCE_PHASES={2,6}`); every substantive phase closes with `reviewer_confirm
|
||
kind:"phase"` (F7 is **two-step**: `kind:"sql"` records `sql_approved`, then `kind:"phase"`
|
||
advances). Fixed Discipline 2 + Phase 1/3/4, added a per-phase **cheat-sheet**, documented the
|
||
`SchemaLinking` shape. The old false "the reviewer_decide already advances" (F3) claim — the
|
||
exact cause of the observed thrash — is gone.
|
||
|
||
Verified: harness pytest **281 passed** / 5 deselected, gate JS **34/34**, changed-files ruff
|
||
clean (the 36 `ruff check .` errors are pre-existing on `main`). Final whole-branch review
|
||
(opus): READY TO MERGE, no Critical/Important. Executed via subagent-driven-development
|
||
(implementer + task-review per task, final opus review). **DEFERRED (needs VPN): live F4/F6
|
||
end-to-end** — resuming session `2026-06-30-165708` (stuck at F6) is the ideal live probe.
|
||
|
||
## UI/UX redesign + Resume — COMPLETE (2026-06-30)
|
||
|
||
Plan: **`~/.claude/plans/foamy-forging-dahl.md`**. Memory: `thothii-ui-redesign-inprogress.md`.
|
||
**All workstreams done and pushed to origin/main:** D + E @ `0eeb3f7`, B + C @ `b056ff3`,
|
||
F @ `cef9ae4`, A @ `0a13f71`, G @ `e8cdd00`(scope) + `cbb8e18`(results). Nothing pending from this
|
||
plan. Per-workstream detail below for reference.
|
||
|
||
- **D — DONE** (`c12bdcd`): session display `name` = 3-5 Italian keywords via **YAKE** (no LLM),
|
||
derived in `tht session new` (CLI layer); `create_session` core unchanged (`name=None` default).
|
||
`yake` added to `harness/pyproject.toml`. TDD `tests/test_session_name.py`; harness 269 passed.
|
||
- **E — DONE** (`0eeb3f7`): rotating activity icon replaces the red dot in `CentralStatus`
|
||
(inline, clickable → opens the panel); `ModelActivityPanel` is a **5-line expandable
|
||
model-stream tail**; `WorkingSpinner` extracted to its own module; the separate spinner button
|
||
+ orphaned `Transcript.tsx` removed. Frontend 87/87, tsc clean. **Live visual check DONE
|
||
(2026-06-30):** inline spinner opens the panel; 5-line collapsed tail; expand → full transcript.
|
||
- **B — DONE** (`b056ff3`): `WorkflowBar` is now colored **dots** F1..F8, no phase-name text
|
||
(amber-translucent=running, green=done, red=error, gray=pending; green connectors lead the active
|
||
dot). Each dot carries `data-state`. Error is lightweight: store `phaseError` set when an `info`
|
||
`level=error` arrives during the phase, cleared on the next `ui_request` (`sessionStore.ts`).
|
||
**All four states live-verified** via Playwright.
|
||
- **C — DONE** (`b056ff3`): right sidebar — single-line denser rows (inline status dot + name,
|
||
`py-1`), a 3-level type hierarchy via **`/impeccable`** (L1 `SESSIONS` red/bold/wide-tracking ·
|
||
L2 section + group headers muted uppercase · L3 names normal-case), and the **"No group" label
|
||
removed** (ungrouped sessions render after the last group; guarded so the empty-state still
|
||
teaches when there are no groups). **Live-verified.** (Resume in `SessionMenu` stays with A1.)
|
||
- **Tests:** frontend **93/93** (was 87; +3 store `phaseError`, +2 `WorkflowBar` dot-state, +1
|
||
AppShell no-"No group"), `tsc -b` clean.
|
||
- **F — DONE** (uncommitted; live check deferred to G): single-select answers **auto-confirm**.
|
||
`reviewer_select` options may carry a `decision` payload (`{type, subject, detail?, rationale?}`)
|
||
and an optional `advance`; picking such an option persists the decision directly via
|
||
`tht decision add` (shared `decisionAddArgs` helper, also used by `reviewer_decide`) — no redundant
|
||
`reviewer_decide`/`reviewer_confirm` gate. Options without a payload stay ask-only; back/exit/Other
|
||
never persist. Pure logic extracted to `resolveSelectOutcome`/`decisionAddArgs` (exported, unit-
|
||
tested). Contract docs updated: `reviewer_select` tool desc + `SKILL.md` (widget summary,
|
||
disciplines 2-3, Phase-1 single-pick) + the `CLAUDE.md` gate note. Gate JS **33/33**, harness 269.
|
||
**Live verification (model actually uses `reviewer_select`+decision, no follow-up gate, decision in
|
||
`review_decisions.jsonl`) deferred to G** — it is model-behavior-dependent.
|
||
- **A — DONE** (uncommitted): **A1** — `SessionMenu` gains a **Resume** item (gated to
|
||
`status!=="finalized" && !archived`), wired in `AppShell` to the existing `doResume` → `POST
|
||
/sessions/:id/resume`. 3 tests (`SessionMenu.test.tsx`); frontend **96/96**, tsc clean. **A2** —
|
||
diagnosis-first clean-room repro shows the **resume cold-start stall NO LONGER reproduces on pi
|
||
0.79.4** (8/8 chained into the tool calls, fresh + partway; GLM 5.2 now narrates AND emits
|
||
`tht session show`+`read SKILL.md` in-turn). The earlier narrate-and-stop predates the pi upgrade.
|
||
Defense-in-depth applied: `RIPRENDI_KICKOFF` hardened to force the in-turn tool call (gate test +
|
||
live regression 2/2). The cross-model angle (weaker/older models) lives in **G**.
|
||
- **G — DONE** (`e8cdd00`+`cbb8e18`): cross-model behavior matrix via a committed clean-room harness
|
||
(`harness/scripts/model-matrix.mjs`). **Tier 1** — kickoff + resume first-turn: all *available*
|
||
models chain in-turn (`zai/glm-5.2`, `deepseek/deepseek-v4-{pro,flash}`, `aritmolab/qwen3.6-35b-a3b`,
|
||
`zai/glm-4.5-air`); the resume stall recurs on none (closes A's cross-model robustness).
|
||
`aritmolab/gemma4-26b-a4b` = **404 unavailable** at the endpoint (listed but not served) — infra
|
||
gap, not a workflow issue. **Tier 2** — F single-select auto-confirm verified live on `glm-5.2`:
|
||
answering the first `reviewer_select` persisted a `concept_clarified` decision **0→1** with **no
|
||
follow-up gate** (closes F's deferred live check). Full results: the G plan doc + memory
|
||
`thothii-cross-model-matrix`. No prompt hardening needed.
|
||
|
||
**Status:** **All UI-redesign + resume workstreams done and pushed — D, E, B, C, F, A, G.**
|
||
Nothing pending from the plan. Optional nice-to-haves (not required): Tier-2 F/multiselect live for
|
||
the non-baseline models (cheap re-run with `harness/scripts/model-matrix.mjs` + the Tier-2 method),
|
||
and a one-off manual Playwright kebab→resume pass in the live UI.
|
||
|
||
## Live verification + reviewer_select fix (2026-06-30, afternoon)
|
||
|
||
Drove the real stack (Playwright → backend → real Pi → GLM 5.2 → DWH) end-to-end.
|
||
|
||
- **F1 hang fix (`418187a`) VERIFIED LIVE.** Answered an F1 reviewer widget; Pi resumed (model
|
||
socket reopened) and the gate produced new output — vs the old silent hang. The transition
|
||
"silent hang → gate re-presents/advances" proves `ctx.ui.input` now resolves.
|
||
- **New bug found + fixed: reviewer_select `choices` vs `choice`.** The gate's `reviewer_select`
|
||
(and `reviewer_confirm` reject) read `resp.choice` (singular) but the frontend uniformly sends
|
||
`choices: [id]` (array) — so every single-select gate answered "Nessuna scelta ricevuta" and
|
||
re-proposed forever (multiselect was fine; it already read `choices`). Fix: a shared
|
||
`selectedChoice(resp)` helper (`harness/.pi/extensions/tht-gate.js`) reading the array; both
|
||
handlers use it. TDD: `gate/__tests__/gate_choice.test.js` RED→GREEN, full gate suite **28/28**.
|
||
VERIFIED LIVE: a single-select answer is now accepted and the workflow advances (2/4 → 3/4).
|
||
- **Resume cold-start STALL confirmed (open item #1).** On `/riprendi-sessione`, GLM 5.2 narrates
|
||
the bootstrap step then ends the turn without the tool call → Pi idle, unrecoverable from the UI.
|
||
Memory: `thothii-resume-cold-start-stall.md`.
|
||
- **GLM 5.2 F1 is slow (~3-4 min, ~50+ reads) but works** — looks stuck but isn't; don't hit
|
||
"Stop and save" (it `POST /close`s → kills Pi). Memory: `thothii-glm52-f1-slow-not-stuck.md`.
|
||
|
||
## Earlier work — F1 reviewer-widget hang fix + multiselect guidance (committed 2026-06-30; authored 2026-06-29)
|
||
|
||
Two fixes, **committed to `main`** (7 files):
|
||
|
||
1. **Bug: every reviewer widget hung "stuck with no output" after the human answered** — F1
|
||
disambiguation (and any gate) dead-ended. Root cause, confirmed from Pi's own source
|
||
(`@mariozechner/pi-coding-agent` `dist/modes/rpc/rpc-mode.js`, `createDialogPromise`):
|
||
`ctx.ui.input` assigns its OWN RPC id (`crypto.randomUUID`) and correlates
|
||
`extension_ui_response` on THAT id, silently dropping unknown ids. The gate puts a
|
||
different id (`u${Date.now()}`) inside the descriptor carried in `title`. `SessionBridge`
|
||
was replying with the **descriptor** id, so real Pi never resolved `ctx.ui.input` → the
|
||
model never continued. **Fix:** `SessionBridge` now stores Pi's top-level `m.id`
|
||
(`pendingPiId`) on the incoming request and replies `extension_ui_response{ id: pendingPiId,
|
||
value: <uiResponse JSON> }` (value still carries the descriptor id, so the gate's internal
|
||
`resp.id === descriptor.id` check holds). File: `backend/src/bridge/session-bridge.ts`.
|
||
Full write-up: memory `pi-ui-input-id-correlation.md`.
|
||
- **The test double was masking it:** `harness/tests/fake_pi/fake_pi_rpc.mjs` had forced
|
||
`m.id == descriptor.id`. Corrected to mirror real Pi (distinct `randomUUID` top-level id,
|
||
correlate on it, drop unknown ids); `test_fake_pi_contract.mjs` gained a negative
|
||
regression test ("respond with descriptor id → no follow-up").
|
||
- TDD: `backend/test/session-bridge.test.ts` (unit) + `backend/test/e2e-f1.test.ts`
|
||
(integration — now asserts the model's follow-up arrives after the answer) went
|
||
RED→GREEN.
|
||
|
||
2. **UX: multi-answer disambiguation** — `harness/.pi/skills/tht-sessione/SKILL.md` Phase 1
|
||
now tells the model to use `reviewer_decide` (the existing multiselect/checkbox widget)
|
||
when an ambiguity admits several simultaneously-true answers, instead of single-pick
|
||
`reviewer_select`. Guidance-only — no new widget (`frontend MultiselectWidget` already
|
||
exists).
|
||
|
||
Verified at commit time: backend `npx vitest run` **67/67 green**; `tsc --noEmit -p .` **OK**;
|
||
fake-pi contract `node --test test_fake_pi_contract.mjs` **2/2 green**. **Verified LIVE
|
||
2026-06-30** (see the top "Live verification" section).
|
||
|
||
## Most recent feature — Session management (MERGED to main @ 2c21e46)
|
||
Full session management modeled on Claude's UI, all three layers:
|
||
- **Read-only "split view" panel** (left drawer, `SessionDocumentsPanel`) showing a session's
|
||
phase documents read-only (reuses `SqlViewer`/`SchemaLinkingViewer`/`MarkdownView`).
|
||
- **Rename / Move to group / Archive / Delete** via a kebab menu (`SessionMenu`) → REST →
|
||
`tht session set-name/set-group/archive/unarchive/delete`. Archive = a manifest `archived`
|
||
flag (not a dir move); groups = a manifest `group` field; delete = hard `rmtree` + confirm.
|
||
- Rail: collapsible group headers + "No group" + a separate **Archive** view.
|
||
- **Resume correctness:** read-only **guard** (HTTP 409 when `finalized` or `archived`);
|
||
`PiProcessManager.spawnFor` now has a `new`/`resume` mode (resume sends
|
||
`/riprendi-sessione <id>`); a "Phase 0 — Resume" cold-start section in `SKILL.md`.
|
||
- Design docs: `docs/superpowers/specs/2026-06-29-session-management-design.md` +
|
||
`docs/superpowers/plans/2026-06-29-session-management.md`.
|
||
|
||
### ⚠️ Open items / pending gates
|
||
1. **Resume cold-start stall — RESOLVED on pi 0.79.4 (workstream A, 2026-06-30).** The earlier
|
||
narrate-and-stop (GLM 5.2 narrating the bootstrap step then ending the turn without the tool
|
||
call) **no longer reproduces**: a clean-room repro of the backend's exact resume handshake
|
||
chained into `tht session show`+`read SKILL.md` in-turn **8/8** (fresh + partway sessions). The
|
||
pi upgrade is the likely fix. Defense-in-depth: `RIPRENDI_KICKOFF` hardened to force the in-turn
|
||
tool call (gate test + live 2/2). Memory: `thothii-resume-cold-start-stall.md`. **Remaining:**
|
||
the cross-model angle (older/weaker models) is folded into **G**; a full Playwright kebab→resume
|
||
pass through the live UI is still worth one manual run (item 2).
|
||
2. **Full Playwright live-stack verification (MANUAL, not yet run).**
|
||
3. **Minor backlog (non-blocking):** explicit id-traversal guard in `delete_session`
|
||
(today gated by `load_session`); `close_session` could reuse `_save_touched` (DRY);
|
||
delete-via-kebab integration test skipped (base-ui Menu portal not drivable in jsdom —
|
||
the dialog itself is unit-tested); a couple of test-file lint nits.
|
||
4. **DONE — F1 hang fix live-verified 2026-06-30** (see top section). The live verification
|
||
also surfaced + fixed the reviewer_select `choices` mismatch.
|
||
5. **Resolved: settings use `zai/glm-5.2`/medium** (not deepseek-flash); GLM 5.2 drives F1
|
||
fine, just slowly (~3-4 min, ~50+ reads). To tell a truly stalled Pi from a merely-slow one,
|
||
check its sockets/children. Memory: `thothii-glm52-f1-slow-not-stuck.md`.
|
||
6. **DONE — Pi migrated to `@earendil-works/pi-coding-agent@0.80.3` (2026-07-04).** The old
|
||
scope `@mariozechner/pi-coding-agent` is frozen at 0.73.1; every release ≥0.74 lives under
|
||
the new scope `@earendil-works` (latest 0.80.3). The live `pi` (`~/.local/bin/pi`) was
|
||
repointed to 0.80.3. Validated by an API/RPC-surface diff (0.73.1→0.80.3: `ExtensionUIContext`
|
||
byte-identical, `rpc-types` additive-only, `createDialogPromise` + provider-registration API
|
||
unchanged) **plus** a live `model-matrix` smoke (GLM 5.2, `new`+`resume` both `CHAINED`,
|
||
full RPC event vocabulary incl. `extension_ui_request` intact). Notable: 0.80.3 adds
|
||
`ctx.mode: "tui"|"rpc"|"json"|"print"` (a proper mode discriminator; the earlier `0.79.4`
|
||
references above are superseded). Rollback: the old package is still on disk — repoint the
|
||
symlink to `@mariozechner/.../dist/cli.js`. Memory: `thothii-pi-earendil-migration.md`.
|
||
|
||
## Where design history lives
|
||
- Specs: `docs/superpowers/specs/` · Plans: `docs/superpowers/plans/`
|
||
- SDD execution ledger (gitignored scratch): `.superpowers/sdd/progress.md`
|
||
- Auto-memory index: `~/.claude/projects/-Users-mp-projects-ThothII/memory/MEMORY.md`
|
||
(Pi RPC event vocabulary, ui.input id correlation, Omics Portal/GSD design system,
|
||
resume cold-start stall, GLM 5.2 F1 slow≠stuck).
|
||
|
||
## Git
|
||
`main` @ `2410f01`, **pushed to `origin`** (github.com/mptyl/ThothII); working tree clean, no stashes.
|
||
Latest arc (2026-07-07): review-gates-v2 ff-merged — `9b4f6b9` (WS1 harness) · `3ad93cd` (WS2
|
||
gate+SKILL) · `1c97289` (WS3 viewers) · `4042d0b`+`b089482` (WS4 replay) · `b59b57c` (review fix
|
||
wave) · `2410f01` (agent_end spinner fix). Branch `feat/review-gates-v2` still exists (local +
|
||
origin), fully merged. Before that (2026-07-05/06): F4 column curation (PR #1) + look&feel v2 —
|
||
`0d9e035` · `0a63ba9` · `d942635` · `9fe1c93` · `e9b2934` · `9403147` · `7491e8c`.
|
||
Older history (workflow hardening 2026-07-01, UI redesign 2026-06-30): see the sections above;
|
||
stale branches were pruned on 2026-06-30 (SHAs recoverable via reflog).
|