ThothII/PROJECT_STATE.md

ThothII — Project State

Starting-point snapshot for new sessions. Requisito finale del progetto (owner, 2026-08-11): al termine dell'ultima fase tecnica deve essere prodotto un documento unico che guidi l'utente passo-passo su (1) come preparare il repository dei workspace su Git secondo le regole del progetto, (2) come usare gli strumenti di ThothII per il repository (app + CLI thothctl), (3) come usare l'applicazione ThothII di base (sessioni, domande, gate). Il documento userà parole semplici ed esempi; i dettagli tecnici resteranno nei contratti esistenti. Esempio pratico completo: Policlinico San Donato. Last updated: 2026-08-18 (Task 4 recertification recorded; native Windows authority failed and the authentication feature is not implementation- or release-complete). Point a fresh session here ("read PROJECT_STATE.md") before substantial work.

Task 4 authentication recertification — FAIL, native Windows CHANGES REQUIRED (2026-08-18)

• Frozen source under test is b31b27e5845ffd3adf311429367319beaba263c7 on feat/thoth-auth. No tracked source changed during certification; only .playwright-cli/ and .thothctl/ remain untracked.
• Local PASS: Go focused security tests for safeio, backup, and authstorage; Go race across 18 packages; go vet; host build; Windows amd64 cross-compile; Node v24.16.0 backend 76/1092 and frontend 61/444 with typecheck/build; authentication/F1 smoke and sentinel scan; authentication docs smoke; shell syntax.
• Local FAIL: harness 951 passed / 1 failed / 4 skipped (the F4 column-decision test cannot find workflow.yaml from its test cwd); Ruff 192 errors; default Compose missing its required workspace-remote variable; unified Compose references absent compose.unified.yaml.
• Authorized workflow run 32122302381 (URL in the Task 4 report) has the exact frozen SHA and conclusion failure. Its Windows clone and Compose contract job 95665197885 really ran the native safeio/backup command and failed, including a 10-minute backup lifecycle-lock timeout. The frozen workflow does not request internal/authstorage, so that native evidence is absent rather than inferred from cross-compilation.
• The same run's LF/Compose/docs/TS job failed on an unset TMPDIR after its unified Compose contract passed (baseline/CI contract). Its Linux Docker job failed before deployment because rg was unavailable; cleanup proof passed and no new image manifest was generated (runner prerequisite). The historical five-image manifest remains bound to source 74b062f1a737103524cbe706346cfd65f87cdfd1 and was not rewritten for this candidate. L2, real PSD/manual acceptance, and provider readiness remain PENDING where prerequisites are unavailable.
• Durable current evidence is tracked in .artifacts/task-15/automated-gates.json and .superpowers/sdd/2026-08-18-thothii-authentication-remediation/task-4-report.md; the historical Task 15 report contains a separate Task 4 addendum. Current automated-gates SHA-256 is e0cb84185354b740ce97c8d21d365160b321c88722d08cc31b668ec4cab0353c; the unchanged historical Docker manifest SHA-256 is 9c8dec4546909fd93799dbcf374bcb3a89bc46cfe0fd482472c0cbe757ddf5b6.
• Implementation/release state: NOT COMPLETE. The three Important findings remain CHANGES_REQUIRED; overall release readiness is FAIL with additional PENDING gates. No source fix was attempted in Task 4.

P3 effective configuration and .tht-dwh — implementation complete, automated PASS, manual PASS (2026-08-13)

• Scope: P3 (PRD D3): a versioned shared canonicalizer produces the non-secret effective DWH/preprocessing configuration and a stable logical identity (workspace://<id>@v1:<sha256>), used identically by the application sessions and the operator CLI. OWNER.json writes are versioned; legacy roots remain readable; content-only/Evidence-only changes keep the identity (no forced reconfiguration), while DWH-affecting changes fail closed (never silently reusing the old generation).
• Memory: explicit workspace-global paths.memory root with a guarded migration command (tht memory migrate) that copies and verifies exactly one legacy JSONL under the workspace lock and fails closed on conflicts.
• Revision-scoped records: schema and Evidence Qdrant point IDs, payloads and queries include workspace_revision; memory/solved stay workspace-wide.
• Operator contract: thothctl now carries effectiveConfigIdentity/configFingerprint/ inputFingerprint in results; the operator config lease path is deterministic for the same revision+identity.
• Retained evidence: .artifacts/p3-integration/p3-da9428d84f152fe059d41a89436496b7/ (15/15 checks PASS), bound to clean source commit 3b0726472e15c157….
• Manual gate: P3 walkthrough in docs/testing/p2-p6-manual-verification.md; decision PASS (owner approval 2026-08-13).

P4 Qdrant collection lifecycle — implementation complete, automated PASS, manual PASS (2026-08-13)

• Scope: P4 (PRD D4): one shared TypeScript collection manager owns the Qdrant collection and payload-index contract; session admission self-heals a missing collection (1024/cosine + the 8 required keyword payload indexes) and adds missing indexes, but never mutates an incompatible collection (semantic_index_incompatible); the operator path keeps require_existing semantics.
• Host CLI: thothctl workspace vector inspect (read-only contract report) and thothctl workspace vector rebuild --workspace <id> --collection <name> --confirm <name> --destroy (guarded delete/recreate of only the descriptor-owned collection, with durable state before deletion and verification after recreation; mismatched confirmation or missing --destroy → exit 2).
• Key files: backend/src/workspaces/qdrant-collection.ts (+test), backend/src/tht/tht-runner.ts (qdrantEnsure self-heal for admission; default require_existing elsewhere), backend/src/workspaces/runtime-config-lease.ts (lease exposes semanticQdrantUrl), backend/src/workspace-maintenance.ts + preprocessing-service.ts (vector-inspect/vector-rebuild operator commands), tools/thothctl/internal/workspaceops/operations.go (+tests).
• Automated acceptance: PASS 11/11 (run p4-466bbfdea9ef3111f36baa99fc2d64aa, report .artifacts/p4-integration/p4-466bbfdea9ef3111f36baa99fc2d64aa/ retained via --keep, bound to clean source commit e056c19e6214254a9e3b2390e24c389920b84e95): preflight, clean_state, ownership, qdrant_up, self_heal_create_missing, self_heal_repairs_missing_index, incompatible_refused, require_existing_refused, rebuild_recreates_contract, secret_scan, cleanup_confinement.
• Gates: backend 666/666 + tsc clean; Go build+test 9/9; p4 runner unit tests 3/3; harness 841 passed (only the two pre-existing debt failures unchanged).
• Manual acceptance: PASS (owner approval 2026-08-13) — walkthrough section P4 in docs/testing/p2-p6-manual-verification.md.

P5 curated FK annotations in Git — implementation complete, automated PASS, manual PASS (2026-08-13)

• Scope: P5 (PRD D5): the canonical curated FK file is <workspace-id>/schema/annotations.yaml, a regular Git blob at the same commit as the descriptor. Absence is compatible (empty canonical set
  • warning); symlinks, trees/gitlinks, cross-namespace paths, oversized (>16 MiB), non-UTF-8, and malformed objects are refused at activation. Activation synchronizes the blob to the immutable revision root /data/sessions/<id>/revisions/<commit>/artifacts/mschema/annotations.yaml with a restrictive mode and an adjacent ownership manifest (workspace, commit, blobId, contentDigest, destination); re-sync is idempotent and re-verifies, and tampered destinations fail closed.
• Runtime root: the backend renders paths.annotations_root for the pinned revision while paths.artifacts/indexes/memory/sessions stay workspace-global (the binding-keyed DWH cache at artifacts.parent is untouched); the harness resolves annotations from annotations_root with a legacy fallback.
• Review primitive: thothctl ... workspace schema accept --run <id> --yes is the only human FK review path. It validates the current synced Git blob with the harness parser and records { reviewedCandidatesDigest, annotationsDigest, workspaceRevision, blobId }. Missing --yes, an unknown run, an empty/malformed blob, or a non-matching candidate fails closed (annotation_invalid) without recording a review. The P2 host-file schema check --annotations --reviewed-candidates review write is superseded (read-only validation only).
• Continuation gate: preprocess run continues only when the accepted review's blob digest equals the current revision's synced annotations digest and the DWH binding is compatible; otherwise it records a new manual_review_required checkpoint.
• Key files: backend/src/workspaces/annotations-sync.ts (+test), backend/src/workspaces/ annotations.ts, backend/src/workspaces/git-repository.ts (annotationsObject), backend/src/workspaces/registry.ts (activation validation + sync), backend/src/workspaces/ preprocessing-service.ts (acceptSchema + continuation gate), backend/src/workspace-maintenance.ts (schema-accept), tools/thothctl/internal/workspaceops/operations.go (+tests), harness/tht/ config.py + cli/schema_cmd.py (paths.annotations_root), docs/contracts/ workspace-preprocessing-cli.md.
• Gates: backend 689/689 + tsc clean; Go build+test 9/9; harness focused schema/annotations 52 passed. Full-suite re-run and the clean-state process goal are recorded at the acceptance gate.
• Automated acceptance: PASS 10/10 (run p5-66b1f1e74f147a23c0a4bff04e6d2a4c, report .artifacts/p5-integration/p5-66b1f1e74f147a23c0a4bff04e6d2a4c/ retained via --keep, bound to clean source commit 9db0299063d5068198c05dc467d7f86dc34de85b): preflight, clean_state, ownership, activation_sync, accept_happy_path, revision_isolation, accept_negatives, continuation_gate, secret_scan, cleanup_confinement. Runner: scripts/p5-acceptance.sh / backend/scripts/p5-acceptance.mjs (+unit test scripts/test-p5-acceptance.sh).
• Manual acceptance: PASS (owner approval 2026-08-13) — walkthrough section P5 in docs/testing/p2-p6-manual-verification.md.

P6 commit-addressed Evidence materialization — implementation complete, automated PASS, manual PASS (2026-08-13)

• Scope: P6 (PRD D6): filesystem Evidence <id>/evidence is materialized from the exact pinned Git commit into the immutable revision content root <registry>/snapshots/<commit>/<id>/evidence at activation, with a sibling bounded manifest <id>/evidence.manifest.json whose digest is chained into snapshot.json.
• Safety: fixed Git plumbing (ls-tree -r -z + cat-file blob), no shell, no mobile checkout; symlinks/gitlinks at any depth, traversal/absolute/duplicate/cross-namespace paths, and non-regular modes are refused. Installation-local bounds (defaults): 4096 entries, 64 MiB total, 8 MiB per file, 4096 path bytes, 1 MiB manifest; a size-sum preflight runs before writing and no partial root is published. Re-activation reuses a valid root and fails closed on a tampered manifest.
• Engine: evidencePolicy no longer stops filesystem sources (evidence_materialization_required retired); preprocess evidence/preprocess run operate on the materialized root. Evidence Qdrant records remain revision-scoped; corpus ACTIVE is revision-qualified. HTTP/S3 Evidence is unchanged.
• Retention: materialized roots live inside the commit-addressed snapshot directory, so they are retained while pinned and removed by the existing snapshot retention scan when unreferenced.
• Key files: backend/src/workspaces/evidence-materialization.ts (+test), backend/src/workspaces/git-repository.ts (evidenceTreeObjects/evidenceTreeId/ evidenceBlobBytes/gitObjectSize), backend/src/workspaces/registry.ts (activation staging + integrity chain), backend/src/workspaces/preprocessing-service.ts (stop removal), backend/src/workspaces/types.ts + config.ts (limits), docs/contracts/ workspace-preprocessing-cli.md.
• Gates: backend 698/698 + tsc clean; Go build+test 9/9 (unchanged); harness focused suites pass. Full-suite re-run and the clean-state process goal recorded at the acceptance gate.
• Automated acceptance: PASS 10/10 (run p6-7a4c4d0ebb63399cfa9f674738b9e8fc, report .artifacts/p6-integration/p6-7a4c4d0ebb63399cfa9f674738b9e8fc/ retained via --keep, bound to clean source commit 124891bbfe8dc8270e8b58c4206150eb8bebeaa7): preflight, clean_state, ownership, activation_materialization, evidence_preprocess, revision_isolation, unsafe_tree_refused, bound_refused, secret_scan, cleanup_confinement. Runner: scripts/p6-acceptance.sh / backend/scripts/p6-acceptance.mjs (+unit test scripts/test-p6-acceptance.sh).
• Manual acceptance: PASS (owner approval 2026-08-13) — walkthrough section P6 in docs/testing/p2-p6-manual-verification.md.

P7 PSD migration — plan + repository restructured + local validation PASS; owner-gated (2026-08-13)

• Plan: docs/superpowers/plans/2026-08-13-p7-psd-migration.md.
• Done (autonomous): /Users/mp/projects/tht-workspace-psd restructured to the P1.1 layout and committed (thoth-workspaces.yaml + psd-clinical/workspace.yaml schema v3 + psd-clinical/ evidence/ 36 .md + psd-clinical/schema/annotations.yaml 42 KB); legacy runtime dirs gitignored and the old flat psd.yaml retired. A local WorkspaceRegistry.bootstrap() against a scratch bare clone activated psd-clinical (descriptor valid, 36 Evidence materialized + manifest, 42 KB annotations synced, workspace-docs generated) with no DWH/secret access.
• Templates: deploy/psd/{workspace-bindings,operator,thothii-installation}.env.example + gitignored secrets/; operator checklist in docs/install/psd-workspace-setup.md (registered in MkDocs nav).
• Published (2026-08-13): private repo https://github.com/mptyl/tht-workspace-psd (main = d4f9185), consumed via SSH deploy key thothii-psd (read-write, passphrase-less, generated in deploy/psd/secrets/). Real operator config is wired (gitignored): deploy/psd/operator.env, workspace-bindings.env, thothii-installation.yaml, connector-secrets.yaml + secrets/ (DWH X-API-Key reused from the legacy .env; no CA — the DWH REST is public HTTPS).
• Stack live: started via thothctl start (project thothii-70417a3e30ea), all services healthy, qwen3-embedding:0.6b present; the registry cloned + activated psd-clinical (ready); thothctl workspace inspect returns ok with descriptor/catalog/runtime identities. Gotcha recorded: thothctl uses a per-descriptor Compose project name, so the stack must be started with thothctl start (not a raw compose-with-preflight.sh up).
• Preprocessing live (2026-08-13): with VPN active, thothctl workspace preprocess run --workspace psd-clinical succeeded against the real PSD DWH — DWH introspection + LSH (163 tables / 2275 columns), FK review (no new candidates: the 42 KB curated annotations are authoritative), schema index (2438 records) and filesystem Evidence index (36 docs / 43 chunks). Qdrant psd-clinical now holds 2482 revision-scoped points (schema_table 164, schema_column 2275, evidence 43; all carry workspace_revision). Rerun is idempotent (Evidence unchanged: 36).
• Fixes shipped during the live run (real-DWH scale revealed them): (1) pruned ~95 GB of orphaned acceptance-run Docker volumes; (2) raised workspace-maintenance tmpfs /tmp 64 MiB → 1 GiB (PSD LSH snapshot is ~105 MB); (3) vector rebuild now recreates the 8 keyword payload indexes (it only created dimensions/distance); (4) Qdrant upserts are chunked (256 points/batch) — a 2438- record schema batch exceeded Qdrant's 32 MiB JSON limit; (5) frozen Evidence metadata lists now stay lists (FrozenList) instead of tuples, preserving JSON shape; (6) embedding timeout 30 s → 300 s and batch 32 → 16 for large CPU corpora.
• Remaining: live session smoke on psd-clinical (P8 L2) — create a session with a real natural-language question and reach the first reviewer gate.

Final aggregate P2–P6 verification — automated PASS, manual PENDING (2026-08-13)

• Aggregate process goal: one clean-state run exercises the complete DWH → FK → schema → filesystem Evidence chain through thothctl/the operator surface, proves idempotency and revision isolation, proves a second installation consumes the same Git workspace with its own state, exercises unsafe-tree and bound negatives, and cleans only owned resources.
• Automated acceptance: PASS 12/12 (run p2p6-ee542112c526ef0d4c25ddf6c8bc164b, report .artifacts/p2p6-integration/p2p6-ee542112c526ef0d4c25ddf6c8bc164b/ retained via --keep, bound to clean source commit 1dcf4051b0d9db8ae163e4d7c53871564ca3c564): preflight, clean_state, ownership, activation_materialization, dwh_chain, fk_schema_evidence_chain, revision_isolation, second_installation, unsafe_tree_refused, bound_refused, secret_scan, cleanup_confinement. Runner: scripts/p2p6-acceptance.sh / backend/scripts/p2p6-acceptance.mjs (+unit test).
• Full suites + builds (design §10): harness 873 passed / 4 deselected (with color disabled; the forced-color environment splits --help flags and trips the gate-CLI consistency test only); backend 698/698 + tsc + build; frontend 364/364 + tsc -b + build; thothctl Go build+test 9/9; git diff --check clean.
• Manual acceptance: PENDING — "Final aggregate P2–P6 verification" in docs/testing/p2-p6-manual-verification.md.

User-guide deliverable (owner requirement) — written, review PENDING (2026-08-13)

• docs/guida-utente.md (Italian, simple words + examples) covers: (1) preparing the workspace Git repository (catalog + schema-v3 descriptor + Evidence + curated annotations), (2) using the ThothII tools for the repository (thothctl commands + read-only workspace management), and (3) using the base ThothII application (sessions, questions, gates). It ends with a complete Policlinico San Donato walkthrough and links to the technical contracts.
• Registered in the MkDocs nav (mkdocs.yml). Owner review PENDING.

P2 host preprocessing CLI — implementation complete, automated PASS, manual PENDING (2026-08-11)

• Scope: P2 (PRD D2, based on the P1.1 registry contract): the installed native thothctl binary is the only host interface for workspace preprocessing. Commands: workspace inspect, preprocess dwh, schema suggest-fks, schema check, index-schema, preprocess evidence, preprocess run, with the exact grammar, file-ingress bounds, result contract and exit codes in docs/contracts/workspace-preprocessing-cli.md.
• Operator: workspace-maintenance is a profile-gated Compose service sharing the core image, with no Pi auth/state, no backend/Pi/frontend listener, no Git credentials, and a compiled Node entrypoint (backend/src/workspace-maintenance.ts) driving the existing harness engine through pristine JSON machine interfaces (schema_cmd.py, vector_cmd.py, preprocess_cmd.py).
• Boundaries honored: FK review is digest-bound (candidate digest == persisted artifact; a review accepted for the same candidate content counts); Qdrant collections are never created by the product path (require_existing + pre-provisioned fixture, P4 owns lifecycle); filesystem Evidence stops with evidence_materialization_required (P6); HTTP Evidence enforces an installation private-host allowlist; ssh_tunnel stays fail-closed (P10); cross-revision DWH reuse is explicitly P3.
• Retained evidence: .artifacts/p2-integration/p2-b109757b26388a5ed6b1d173dee86584/ (11/11 checks PASS), bound to clean source commit de5de36f9a4edfd4fbebf277822090871ccdd61f.
• Manual gate: P2 walkthrough in docs/testing/p2-p6-manual-verification.md; the owner approved P2 on 2026-08-11 (manual acceptance PASS). P3 and later start only after an explicit new authorization.

P1.1 workspace-directory registry — automated integration PASS, manual PENDING (2026-08-11)

• Scope: P1 correction (not preprocessing). Root curator-owned catalog thoth-workspaces.yaml; one self-contained directory per workspace (<id>/workspace.yaml, optional <id>/evidence/**); generated docs stay API-owned under workspace-docs/<id>; internal immutable snapshots remain flat (<snapshots>/<commit>/<id>.yaml) to preserve session pins and runtime trust.
• Ownership: the API may create a descriptor once when its catalog slot exists and the descriptor Git object is absent at the exact base commit. Existing descriptors and curated content are curator-owned and change only through Git commit/push then installation pull. Update/delete publish payloads are refused as HTTP 409 workspace_curator_owned. Catalog and Evidence are never written/staged/cleaned by the API. Explicit pull may produce one deterministic docs-only follow-up commit that never touches curator bytes.
• Schema/UI: schema v3 remains the only descriptor schema; filesystem Evidence URI is exactly <id>/evidence. Browser workspace management is read-only for ready workspaces (Pull/Sync, Validate, installation Test, Export, Evidence summary, curator Git guidance) and offers an editable bootstrap form only for configuration_required catalog slots.
• Retained evidence: .artifacts/p11-integration/p11-ac0b047024fb09eeca218512526a6b23/ (report.json sha256 44250145fede36de5de941262beb833c920e8c73366d987cdd738856aac6f6d6), 19/19 checks PASS, bound to clean source commit eac472011e465c24572d9a6bae14de0fb3e246c0 / tree 4fd15ec28d3b7967b7b8757158013307fb20f3b9.
• Verification: backend Vitest 634 passed / 41 files + tsc + build; frontend Vitest 364 passed / 54 files + tsc + build; harness focused Evidence/config pytest 39 passed; install-docs and schema-v3-only gates PASS; workspace-registry-smoke.sh and unified-deployment-smoke.sh full Docker runs PASS with exact cleanup.
• Known limitations: P2–P6 plans/designs are unchanged and their old source paths are inventoried for a later owner-approved adaptation plan. Windows Docker startup and native PowerShell contract were not executed on a Windows host. P1's accepted historical evidence and process artifacts remain untouched; the old P1 process commands are not rerunnable against the superseding P1.1 repository contract.
• Manual gate: .artifacts/manual-acceptance/p11/ prepared for the reviewer; follow docs/testing/p11-manual-acceptance.md. The owner reviewed the walkthrough and approved the implementation on 2026-08-11.

P1.1 automated integration: PASS
P1.1 manual acceptance: PASS   (owner approval 2026-08-11)

P1 configuration process — ACCEPTED 2026-08-10

• Retained evidence: .artifacts/p1-integration/p1-038bf31360180dc831220b33fbadcfe6/report.md
• Final report hashes: report.json f07d49097966de6f0307490089fdb2ae61379c04b7fc7177d3c44cf001e1b46a; report.md 09b6a9e9ad9eed2b049e286af452e12fa1f3174ea8d633253542604470890c6c.
• automated integration: PASS
• manual acceptance: PASS — explicitly approved by the project reviewer on 2026-08-10.
• The retained run is bound to clean source commit c7338969d7c7c1c396d9099b7ab2d309b70ab6cf and tree 5f7013904806054b9f587230be89f34cbc80f5fc. Its hash-bound provenance contains exact 43-file backend source and 39-file compiled dist manifests (manifest SHA-256 eb6d6c77c78d14c798b50d0be430ad124b8fd8965afbc4bb07d358089d23f49 and 9f9e8899f8aca882ff08d49ec6cd00caeea75691c8280095a9b88bc74939a30a).
• The retained audit has exactly 15 PASS checks and 134 unique declared artifacts whose final bytes match every SHA-256 declaration. It records 749 PASS command events and 1,664 production child/network events, with listener shutdown and refusal checks recorded in the final ownership artifact. Raw Git rejects configured executable diff drivers and other helper-bearing state.
• Manual production acceptance binds every regular compiled distribution file through an immutable manifest and cached verified module bytes; imported dependency replacement is refused before RUNNING. Snapshot rendering validates the bounded snapshot.json, expected digest, and Git blob identity, refusing regular source replacement without publishing output.
• Final Task 8/9 focused suites pass (48/48 Task 8; 59/59 manual acceptance and renderer checks), backend TypeScript/build pass, frontend tests/build pass. Historical harness pytest/Ruff debt remains unrelated to this P1 work.

Internal Qdrant + Ollama semantic infrastructure — LIVE 2026-08-08

• Compose topology. The mandatory application stack is frontend, core, qdrant, embedding, and the one-shot embedding-model-init. Startup is CPU-first by default; Linux hosts may opt into GPU exposure with THOTH_ENABLE_EMBEDDING_GPU=1. Qdrant is private on the Compose network and persists /qdrant/storage in qdrant-data. Ollama persists its local model cache in embedding-models, and embedding-model-init blocks core until qwen3-embedding:0.6b is present.

• Semantic contract. Internal semantic indexing is fixed to qwen3-embedding:0.6b, 1024 dimensions, and cosine distance. Schema v3 is the only accepted workspace descriptor. Schema v1 and v2 workspace descriptors are rejected before activation. Candidate snapshot validation makes activation or a pull fail atomically and leaves the prior valid snapshot active; there is no in-product migrator or automatic conversion. One workspace owns one Qdrant collection, and schema, Evidence, and Memory records coexist inside that collection with payload kind separation.

• Final review runtime barriers. Operational routes, retained session pins, and runtime rendering now require schema version 3 before resolving bindings, readiness, diagnostics, or Pi. Session admission verifies the exact internal Qdrant collection (dimensions, cosine distance, and required keyword payload indexes) before Ollama and before manifest persistence. The Qdrant adapter binds every search/list/delete filter to its constructed workspace identity and rejects conflicting caller namespaces.
• Boundary and persistence. Only DWH and LLM remain external runtime application endpoints. There are no active external vector or embedding endpoint instructions, bindings, or secrets in the supported operator manuals. Qdrant remains a derived but persistent semantic index: the canonical sources of truth stay the workspace Git descriptors, phase artifacts, and memory registry/ledger. The Ollama model cache is recoverable for offline startup but is not the canonical source of semantic content.
• Backup and recovery. ./scripts/vector-backup.sh --project-name <name> --output <file> archives exactly one labeled <project>_qdrant-data volume and preserves the prior qdrant running state. ./scripts/vector-restore.sh --project-name <name> --input <file> --confirm-project <name> requires the exact repeated project confirmation, validates manifest and archive safety before stopping qdrant, stages rollback content, restores semantic storage in place, and restarts qdrant only if it was previously running. Recovery requires the registry to already hold a reviewed v3 descriptor revision compatible with the restored collection; the helper does not restore descriptors, rename collections, or repair a semantic-index incompatibility. Backup and restore share one atomic Docker-daemon lock per Compose project/Qdrant volume; contenders fail before volume resolution, and cleanup removes the lock only when its ownership labels still match.
• Verification recorded for Task 13 final audit. On Apple M4 Pro (Darwin 25.5.0, Docker Server 29.6.2 linux/arm64), harness pytest passed 827 passed / 4 deselected; backend Vitest passed 477/477 plus TypeScript and build; frontend Vitest passed 374/374 plus TypeScript and build; git diff --check passed. Deployment contracts passed: test-default-compose.sh, test-unified-compose.sh, test-internal-semantic-compose.sh, test-no-deployment-coupling.sh, test-compose-secret-policy.sh, and verify-workspace-install-docs.sh --fixtures-only.
• Task 13 Docker smoke evidence. CPU semantic smoke passed in 217.34s and proved offline Qdrant/Ollama persistence plus exact cleanup. Workspace registry smoke passed in 42.06s in fix round 1 with a per-run image tag derived from the unique Compose project, and proves exact cleanup of compose containers, volumes, networks, and only that smoke image. Unified deployment smoke passed in 125.57s; update-only rollback smoke passed in 85.40s; Linux server deployment smoke passed in 55.99s. The previously observed thothctl rollback failure did not recur.
• Task 13 image and manual-gate notes. Verified pinned runtime images: qdrant/qdrant:v1.18.2@sha256:75eab8c4ba42096724fdcfde8b4de0b5713d529dde32f285a1f86fdcb2c9e50c and ollama/ollama:0.32.0@sha256:57f573b47f1f71ebb445789f279fe3e596a8beab182f7cf486db9205bad87c5a. The workspace-registry smoke fix-round image used tag thothii-workspace-registry-smoke:thoth-workspace-registry-smoke-thoth-workspace-registry-smoke-10vi3a-19157, built manifest list sha256:715b943057929418cad4aa71806d9edbaf823555d19bda6b875297617463fd4a with config sha256:a566521981e08958aae9a12bfc7803bb5f3f835536b4bb8c39df8fcf26063161, and removed that exact reference during cleanup. Local GPU exposure (THOTH_ENABLE_EMBEDDING_GPU=1) and Windows Docker Desktop startup were not manually executed in this run.
• Task 13 known limitations. Broad harness Ruff remains existing unrelated debt (220 errors); touched harness files were verified Ruff-clean. The final active-reference audit remains non-empty only in deterministic negative guards, retained off-repository migration SQL, L2 compatibility fixtures, gitignored task notes, and historical reference notes. No active schema-v3 operator manual or supported runtime deployment path retains external vector or embedding endpoint coupling.
• Final review fix verification. Backend Vitest passed 477/477 plus TypeScript and build; harness pytest passed 827 passed / 4 deselected with the existing 74 warnings; touched Python files are Ruff-clean. The complete thothctl Go suite, deterministic backup/restore safety test, internal semantic Compose contract, no-deployment-coupling gate, CPU/offline semantic smoke, and unified deployment smoke all pass after the final fix. The intermittent thothctl rollback failure was traced to Docker Desktop alternating equivalent bind sources between /private/... and /host_mnt/private/.... Exact state-v4 source hashes remain unchanged; only fresh bind observations made by a Darwin thothctl carry non-serialized aliases for the rollback comparison, so pre-fix recovery state remains readable and Linux /host_mnt paths remain distinct. The rollback-only smoke passed twice consecutively after each fix revision, and the subsequent full unified smoke passed with exact cleanup.

Historical archive

Historical snapshots and archived reference notes

Historical snapshot — Unified deployment release gate, Task 13 (2026-08-05)

• Release coverage. scripts/unified-deployment-smoke.sh gates the two-service render/build, frontend-to-core routing, embedded pinned Pi, Git registry bootstrap, offline recreation, valid update, invalid-update retention, and the four persistent stores. scripts/thothctl-update-smoke.sh independently exercises the bad-Pi update and automatic rollback path. scripts/server-deployment-smoke.sh starts the server plus required session overlays with the same smoke-built core/frontend images, disposable bind roots/secrets/session configuration, upstream-auth checks, and fail-closed unavailable-session behavior.
• Isolation and disclosure boundary. Every run generates a unique temporary root, Compose project, container/image names, transaction image tags, and run label. The rollback fixture uses an immutable hello-world digest whose preflight exits successfully, guaranteeing the stopped core state required by thothctl compensation. Cleanup includes stopped project containers in its final ownership check immediately before teardown and removes only exact containers, Compose resources, image references, control state, and temporary files. There is no global prune. Failure diagnostics are bounded and sanitized, and all credentials/endpoints used by the smokes are disposable fixtures rather than operator or repository secrets. Every public smoke also has an internal 30-minute process-group supervisor with TERM/KILL of the complete group.
• Cross-platform CI contract. .github/workflows/deployment.yml uses immutable action commits, pinned supported Node and Go versions, runs LF/Compose/secret/coupling/docs/TypeScript gates on Linux, runs each Linux Docker smoke once under its own outer timeout, and copies the Windows source into a path containing spaces before building/invoking native thothctl and rendering Compose. The optional windows_docker_startup dispatch targets a labelled self-hosted Windows Docker Desktop/WSL2 runner and performs bounded two-service startup and exact cleanup. No local Windows or Windows Docker execution is claimed until that manual job is recorded.
• Validation status. Deterministic Phase A gates, backend 434/434 plus TypeScript, frontend 386/386 plus TypeScript, and harness 862 passed / 5 L2 deselected are green. Review round 1 ran each Docker smoke exactly once without retry. Unified (103.86s) and update-only (46.45s) passed build/start, core/Pi/registry/persistence setup and the stopped candidate preflight, but thothctl stopped before mutation at its active-session inventory gate. Round 2 replaces presence-only fixture checks with generated Compose renders plus the production workspace resolver; this found and fixed missing explicit direct transport selections. The clean-server preflight now atomically initializes the three hidden Pi-agent targets under the writable parent bind while protected/tracked sources remain separate read-only mounts. Clean empty-root render/setup and wrong-service/value/mount mutations are green. The corrected server one-shot built and started both healthy services from an empty Pi-state root, then stopped at an incorrectly addressed authenticated frontend hop. Fix round 3 adds the exact fourth private non-admin claim and proves its nginx/backend transformation in a focused auth test. It also centralizes schema-v2 registry descriptor resolution and secret-safe runtime rendering in ThtRunner, preserving canonical revision identity and durable session roots for inventory, create/resume/show, SQL, and Pi calls. The fresh update-only one-shot now passes mutation, automatic rolled_back compensation, exact prior-image restoration, unchanged registry head and mount identities, all four persistence sentinels, post-rollback doctor/workspace checks, and exact labeled-resource cleanup. The one authorized server invocation was blocked at its first Docker readiness call by the execution sandbox's socket permission before any Compose resource could be created, so authenticated workspace/fail-closed session behavior remains an explicit release gate. Native Windows PowerShell/Docker execution also remains pending.

Historical snapshot — Portable deployment decoupling (superseded 2026-08-08)

• Mandatory stack. The supported Compose stack is exactly frontend plus core; use the base file with deploy/compose.local.yaml, or with deploy/compose.server.yaml plus the required public-server session overlay. run-stack.sh invokes the base+local Compose command and the core image provides Pi, so no host Pi binary is part of the launch contract.
• External boundaries. DWH, vector DB, embedding, LLM, and reverse-proxy services are external configurable endpoints even when deployed on the same infrastructure. The two superseded PSD/portal deployment overlays were removed. Workspace descriptors and migration utilities remain separate from deployment runtime configuration.
• Legacy PSD deployment ruling. The PSD bootstrap was deleted because it generated the retired overlay and was therefore deployment machinery, not a data migration utility. Its remaining live contract checks were renamed for the generic local Compose profile. The coupling gate rejects stale active deployment filenames and content while deliberately excluding historical plans/specs, canonical workspace descriptors, and non-runtime migration helpers.
• Fresh provider and secret contract. Local, server, and standalone development mount the protected Pi auth JSON plus tracked declarative model/settings files read-only under /home/thoth/.pi/agent. The existing strict application bundle is a core-only Docker secret at /run/secrets/thothii.secrets; operator env files contain only its absolute source path. Provider readiness is exercised from a fresh Compose volume through model listing, configuration, and sanitized credential status.
• Install and scan closure. Superseded copied one-service installation examples and the provider-owned-network test are retired. Active manuals use the canonical base plus local/server and optional overrides, while the category-based coupling scan covers runtime, Docker smoke, install, operator, and positive deployment-test contracts and propagates scanner errors.

Historical snapshot — Portable Git workspace registry, pre-schema-v3 (superseded 2026-08-08)

• Source of truth and scope. The canonical workspace repository is a generic Git remote, configured only by THT_WORKSPACE_GIT_REMOTE and THT_WORKSPACE_GIT_BRANCH (there is no committed PSD/Chirone remote or branch default). Both a local Docker installation and a server persist its checkout, validated snapshots, state, and locks at /data/workspace-registry. Connector endpoints, transport choices, and secret-file paths remain local bindings; secret contents are never stored in Git, API responses, browser storage, diagnostics, or bundles.
• Migration and session safety. Schema-v2 descriptors are operational; legacy descriptors are visible as migration_required until migrated by the documented operator workflow. New sessions acquire a persistent revision lease before readiness and persist workspace ID plus immutable Git revision. Retention hands that lease off only after an authoritative scan observes the manifest, so a stale concurrent scan cannot prune the pinned snapshot. Resume resolves that historical snapshot, while retention preserves every revision referenced by an open, closed, or failed unarchived manifest. Reconciliation runs only with a complete local installation list or an administrator's complete server list, never from a remote user's partial view.
• SSH connector boundary. The current OpenSSH forward is owned by one bounded diagnostic and is always cleaned up afterward. DWH/vector ssh_tunnel bindings therefore return workspace_not_activatable, and new-session creation rejects them before persistence. Direct and REST runtime connectors remain supported; Git remote access over SSH is unaffected.
• Operator manuals. Follow the local manual for macOS/Windows/Linux Docker Desktop deployment and the server manual for Gitea-compatible remotes, reverse proxy, migration, backup, and recovery. The release workflow is Git review/push → installation pull → validate → local diagnostic test → browser-local workspace/model/reasoning selection → revision-pinned session.
• Verification recorded for this source branch. git diff --check passed; backend Vitest 371/371 and TypeScript passed; frontend Vitest 398/398 and TypeScript passed; the harness document regression passed 10/10. ./scripts/workspace-registry-smoke.sh and the executable installation-manual fixture verifier passed with Docker. A final unrestricted full harness run remains a release command for the deployment environment; the earlier local long-running harness run was intentionally cancelled before it produced a final result.

Historical snapshot — Session summary redesign (2026-07-23)

• Session documents are projected at read time in outcome-first order: original question, final SQL, persisted data preview, revised question, assumptions, one memory list, then remaining technical documents. This applies to existing filesystem and repository-backed sessions without rewriting their artifacts.
• Final SQL has an always-visible clipboard action. All prose, including original/revised questions, assumptions, memory content, and remaining decisions, renders as Markdown.
• Memories are shown once, approved before declined. The generic decision list suppresses memory ledger records plus phase_approved, phase_auto_approved, table_approved, table_promoted, and column_promoted.
• The session summary has its own accessible pointer/keyboard resize separator, persists its width independently from Model activity, reaches 50% when space permits, and preserves a 512 px right-side minimum on narrower desktop layouts.
• Verification: harness 861 passed / 5 L2 deselected, Pi gate 163/163, full frontend suite, TypeScript check, production build, Ruff, and git diff --check all passed. A live pre-existing session returned the new canonical order and none of the suppressed decision labels. Compose rebuilt and force-recreated both services; core image sha256:3566d1258b956f8ca96d3b5ff8f625503247a7fd3b0dffe77020ae03956403d8 is healthy and frontend image sha256:8311ca1308b459ece7236bf143da7b1a226ff4082fed924e1b5a207c24b6ca29 is running. Frontend and /api/health both returned HTTP 200.

Historical snapshot — Local Pi user auth + startup failure handling (2026-07-21)

• The PSD Docker profile now bind-mounts the configurable host PI_AUTH_FILE read-only at /home/thoth/.pi/agent/auth.json; on this Mac it resolves to the real user profile /Users/mp/.pi/agent/auth.json. The container keeps its correct Linux identity HOME=/home/thoth while Pi sees the user's independent deepseek and zai credentials.
• deploy/pi/settings.json is the non-secret model policy and exposes, in order, zai/glm-5.2, deepseek/deepseek-v4-flash, deepseek/deepseek-v4-pro, and aritmolab/qwen3.6-35b-a3b. The core image is aligned to Pi 0.80.3.
• New-session creation now validates the saved provider/model against Pi before persistence; unavailable selections return sanitized 503 model_unavailable without creating a manifest. A synchronous runtime-construction failure after persistence marks that session failed and returns the fixed startup-recovery message instead of leaving an ambiguous open session.
• Verification: backend 235/235, TypeScript clean, dedicated Compose auth/model contract green with a demonstrated RED→GREEN cycle. Rebuilt core image sha256:a8b4dd9f016c2335e4da897073bc6d5bdf1e8ce9b60dcfcca171b6677f563228 is healthy; live /models returned all four models; a real deepseek-v4-pro smoke reached its first reviewer gate, deleted only its own session, and restored the exact prior settings.
• Deleted the three explicitly approved incomplete DeepSeek attempts: a390c8b8-0a91-4a37-967b-ce7ff9be9797, a2f974b2-4c48-4967-b4b6-afdbc2b2d541, and f66e1959-3c71-4b10-8aa1-606992046b7e (API delete 204, subsequent lookup 404 for each).

Historical snapshot — User-owned sessions cutover (2026-07-16)

• Target contract: the public server runs AUTH_MODE=upstream with Task 4 portal identity forwarding and Task 5 principal enforcement deployed together. Its session source of truth is direct TLS-verified PostgreSQL thoth_sessions; local development remains loopback-only with filesystem sessions under THT_HOME. The core never receives the migrator credential.
• Deployment material: copy deploy/compose.session-server.yaml.example and deploy/workspaces/server-sessions.yaml.example into reviewed, untracked operator files. The runtime password, migrator password, and CA are three separate Docker secret mounts; server startup rejects public/local storage and incomplete server DB/TLS configuration.
• Readiness behavior: /health remains the unauthenticated process liveness endpoint. Any route requiring unavailable session/preferences storage returns fixed HTTP 503 before starting Pi; this is intentional and must not be hidden by changing liveness to a database check.
• Manual cutover only: schedule maintenance, drain Pi work, run the one-shot migrator and require pending=[] and drifted=[], then replace core and perform an authenticated storage smoke. Archive/checksum the three reviewed legacy filesystem session directories before deleting exactly those three with docker/cutover-legacy-sessions.sh --delete; no deletion has been run from this repository task. Do not import their untrusted ownership.
• Rollback: PostgreSQL remains the single source of truth. Revert only to a compatible fixed release; never re-enable filesystem persistence, restore the archive into production, or dual-write during rollback.

Historical snapshot — Docker locale deployment, Profile A (superseded 2026-08-05)

ThothII gira in Docker sul server co-locato, embedded nel portale omics_portal a https://aritmolab.policlinicosandonato.it/datamart-builder (backend invisibile, tutto same-origin via nginx del portale).

• 2 container su compose.yaml: thothii-core (Fastify + harness tht + Pi) + thothii-frontend (Vite + nginx-unprivileged). Rete omics_portal_omics_network (external) con alias thothii-core/thothii-frontend.
• DB: Postgres diretto :5438 (stessa istanza: schema datawarehouse 163 tabelle + vectors pgvector). Ruoli dedicati thoth_dwh_reader (read-only) + thoth_vector_rw (read+write). Embeddings: Ollama :11434.
• Secrets: deploy/thothii.env (env_file, gitignored) + THT_MODEL_API_KEY_FILE (key modello, file 0600 — meccanismo provider-credentials di Codex) + bind-mount ~/.pi (pi-config).
• Backend: merge di codex/portable-deployment (secret-bundle, provider-credentials, auth upstream, security hardening, CI multiarch). Setup Docker MIO tenuto (il modello Docker-secrets di Codex è in deploy/ come alternativa inerte).
• Portale (repo omics_portal, branch agent/patient-capabilities-datamart-ui): nginx.conf rotte /datamart-builder/api+/assets + auth_request, template datamart_builder.html (mount <div id="root"> + tag {% vite_assets %}), vista datamart_builder_api_auth. Auth: authentik Admins bypass; utenti normali necessitano gruppo omics-datamart-builder.
• Fix load-bearing: configPath da THT_CONFIG (route senza workspace), vite_assets mark_safe (SPA bianca), COPY harness/+cp workflow.yaml+pip install . (pip 26 / tht module-relative), entrypoint server case.
• Standalone/dev: docker-compose.dev.yml (rete propria, porte host 8787/8090) + scripts/docker-smoke.sh.
• Piano dettagliato: docs/superpowers/plans/2026-07-12-local-docker-deploy-implementation.md.

Archived snapshot — Runtime incident fixes (2026-07-13)

• The bind-mounted Pi profile came from host paths and did not trust /app/harness. Pi 0.80 consequently loaded zero project extensions, prompts and skills, silently sending /nuova-domanda//riprendi-sessione to the model as plain text. The core entrypoint now idempotently adds only /app/harness to the persistent /home/thoth/.pi/agent/trust.json, preserving all existing decisions.
• The gate embeds the canonical tht-sessione/SKILL.md in the one-shot kickoff system prompt and explicitly prohibits repository discovery. A live RPC get_commands must show torna, nuova-domanda, riprendi-sessione, and skill:tht-sessione after deploy.
• Workspace identity is derived from the resolved config path, so config/tht.yaml -> workspaces/local.yaml matches DWH artifact ownership (local).
• Direct pgvector now discovers the actual namespaces of the vector type and cosine operator from PostgreSQL catalogs. This supports server layout vectors.* tables with the extension installed in public.
• Live verification: session 2026-07-13-074712-dammi-la-lista-dei-pazienti-che-haoo-fat resumed directly at F1, ran tht session show, and completed tht search pack (12 tables, 0 evidence, 2 solved) without repository exploration or adapter errors.

Archived snapshot — Workflow/UI regression fixes (2026-07-14)

• F1 Model Activity restored. Session create/resume now preserves configured/persisted thinking instead of forcing off. Pi's nested thinking_delta is bridged to a dedicated named SSE activity_delta; EventSource subscribes to that name and the panel keeps it separate from final assistant text. Reasoning remains in-memory and is not persisted to session artifacts.
• F3 rewrite confirmation remains bypassed. rewrite_question records approval and advances automatically without a reviewer widget. The repeated prompt came from old running containers: images had been rebuilt but services had not been recreated.
• Join review is read-only and complete-set safe. Join-only proposals render informational cards with only Continue and Other — specify. Continue requires the exact complete id set; all joins are persisted together by decision add-join-set, using an atomic ledger replacement under a per-session cross-process writer lock. Other persists none of the rejected proposal.
• CTE presentation fixed. F6 CTE cards now structure purpose, rationale, tables, filters, keys, and output columns with responsive wrapping/alignment. The Horizontal/Vertical switch is hidden for a single SQL block (the per-CTE view), because it only affects multi-block layouts.
• Latest render failure diagnosed and hardened. Session 2026-07-14-115847-estrai-i-pazienti-che-hanno-fatto-un-abl sent an object in open_questions, which React cannot render as a child. The v2 gate now enforces open_questions?: string[]; the frontend also safely normalizes legacy malformed payloads.
• Verification/deploy: Python harness 798 passed / 5 L2 deselected; gate JS 126; backend 143; frontend 250; TypeScript/build gates green. Compose rebuilt and force-recreated both services. Running image ids: core sha256:55acef2f12151ea97144c2f5e9164d63f2ca734bc2746fef553df94849e3fb3f; frontend sha256:1043f79392420149655cc63d70461e2ca2005b2290a1e3e21dcf845ec3bd1c81.

Archived snapshot — Pi-enabled model selector (2026-07-14)

• Pi is the allowlist authority. /models reads the mounted Pi enabledModels, intersects it with models currently available from Pi, and preserves the configured order. Enumeration does not require PI_PROVIDER, does not inject generic/provider credentials, and fails closed for missing or malformed scope.
• Live scope: exactly deepseek/deepseek-v4-flash, zai/glm-5.2, and local-qwen/qwen3.6-35b-a3b. The live endpoint returned those three composite IDs once each and in that order; zai/glm-5v-turbo and all other authenticated Pi models are hidden.
• Validation/process smoke: live settings updates returned 200 for DeepSeek Flash and local Qwen, while hidden GLM-5V returned 400; a post-restore equality check confirmed the original app settings were restored. The real PiProcessManager configure path succeeded for DeepSeek and local Qwen without sending a prompt or starting a DWH operation; local Qwen required no hosted provider key. Unknown and compound providers remain fail-closed in the verified backend suite.
• Verification/deploy (2026-07-14T19:24:22+02:00): backend 154/154 and frontend 251/251 passed; both TypeScript gates and git diff --check were green. Compose built and force-recreated only core; container start was 2026-07-14T17:24:01.992971976Z, health was healthy, and sanitized post-recreate logs contained only the backend listen line. Rebuilt image ID and running container image ID both equal sha256:577f99754fd0731251c8ddd8608b1b8baee09d02fad66c759b23f8221083e676.

Archived snapshot — Qwen connectivity + state-aware Resume recovery (2026-07-14)

• Pi turns have an explicit lifecycle. The bridge tracks idle, running, waiting, and failed; a reviewer gate is waiting, responses/steering return to running, and an assistant provider error or unexpected Pi child exit becomes failed. Provider error details are never forwarded to the client; the UI receives a fixed sanitized recovery message.
• Resume preserves only active work. running/waiting runtimes return as already active. Every validated cold path—including recovery after a child has already exited—clears stale SSE state before reopening and restarts from persisted provider/model/thinking with /riprendi-sessione; idle/failed runtimes are torn down at that point. Failed validation does not detach the existing stream. A successful Resume of the currently selected session also closes and recreates its EventSource, so the replacement runtime cannot be left behind an old same-ID stream.
• Private Qwen routing is live. Core is attached to both omics_portal_omics_network and external localllm_default; frontend remains only on the portal network. The mounted Pi profile resolves local-qwen/qwen3.6-35b-a3b at the sanitized base URL http://localllm-vllm:8000/v1. A direct probe from core verified the model catalog and received a non-empty real chat completion.
• Verification/deploy (2026-07-14T21:41:52+02:00): backend 168/168 and frontend 256/256 passed; both TypeScript gates, both production builds, the Qwen Compose network contract, and git diff --check exited 0. The initial deployment built and force-recreated core and frontend; after the final crash-recovery review, only the affected core image was rebuilt and force-recreated with no active Pi session. Core is healthy and its post-recreate Qwen catalog/completion probe succeeded. Built and running image IDs match: core sha256:9867c2fa002b6f117da9a1d02c73b47bfe0372b8c1c137a5174c3e7ecb1e1db1, frontend sha256:5a47f81bc887423e05cef8fd3feb075aa600cb33217247186124f60ca5a3005b. The frontend entry hash changed, so only omics_portal-web-1 was restarted to invalidate its indefinite Vite-manifest cache. The application-level Qwen smoke reached its first ui_request, persisted the expected provider/model, deleted only its uniquely named smoke session, restored the exact saved settings object, and left no smoke session or Pi runtime. The supplied probe's success path left its keep-alive SSE reader open, so only that probe process was terminated (exit 143), without touching backend, Pi, or unrelated runtimes. The same smoke then exited 0 with controller.abort() in cleanup, preserving the gate, session cleanup, and exact settings-restoration evidence.

Archived snapshot — Complete activity timeline + CTE spacing (2026-07-15)

• Model activity is complete from F1. The left panel now records the submitted prompt before session creation completes, then projects thinking, assistant output, sanitized tool lifecycle, reviewer gates, status, and turn lifecycle in chronological order. It remains in-memory only; closing and reopening the panel does not discard it, while Resume intentionally starts a fresh live timeline.
• Tool activity is a narrow public contract. Only call id, tool name, and running/completed/failed status cross Pi → backend → SSE. Updates, arguments, results, commands, output, credentials, and raw errors stay server-side. Resume and SSE replay were also hardened for process restarts, concurrent lifecycle requests, stale callbacks, cursor reset, and multi-client reconnects.
• CTE plan layout uses real Tailwind 3 spacing. Shared cards use concrete 16 px default / 12 px compact padding utilities; F6 CTE headers and content use responsive 16/20 px edge padding. Semantic ordered steps, single-boundary divided filter/table lists, long-value wrapping, and a plain top-divider rationale preserve the artifact content while improving scanability.
• Final verification/deploy (2026-07-15T02:18:54+02:00, HEAD 56d73d2): backend 204/204 and frontend 285/285 passed; both TypeScript gates and production builds exited 0, and git diff --check was clean. The final lifecycle fixes make restarted-hub cursor replay generation-aware and mutate frontend delete/resume state only for sessions actually deleted. Compose rebuilt and force-recreated core and frontend; built and running image ids match: core sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147 (healthy), frontend sha256:dd7ef721b661b57b8c5422c47088e1a9a2e9821af021541cab372c9892fbd638 (running). The entry changed from index-DcZviApa.js to index-CuIt1NQg.js, so only omics_portal-web-1 was restarted to refresh its indefinite manifest cache.
• Final live local-Qwen smoke: session 2026-07-15-001922-final-no-thinking-activity-smoke-2026-07 observed 0 activity_delta events while receiving 5 strictly allowlisted tool lifecycle events and the first reviewer gate (bash running/completed and reviewer_select running). No forbidden tool field crossed SSE. Cleanup closed with 200, deleted only that session with 204, restored the exact settings object, and left no Pi runtime or smoke session.

Archived snapshot — Filtered Model activity projection (2026-07-15)

• Resolved contract. activityLog still folds the complete in-memory prompt, thinking, assistant, sanitized tool, reviewer-gate, status, and turn-lifecycle history. The left panel now applies a default-deny rendering boundary and shows only prompt, thinking, status, and gate; assistant text remains available to the central transcript, while tool, lifecycle, and unknown future activity kinds do not render or move the panel scroll.
• Frontend-only verification/deploy (2026-07-15T03:44:02+02:00, source HEAD 7208599): frontend 287/287 passed; npx tsc -b, npm run build, and git diff --check exited 0. Compose rebuilt and force-recreated only frontend; its image changed from sha256:dd7ef721b661b57b8c5422c47088e1a9a2e9821af021541cab372c9892fbd638 to sha256:75fc0b7786c75ded1b488e9fad3232aa061c82d89140c67ad62b5285954d1d36, with container start 2026-07-15T01:42:45.582760612Z. The active Vite entry changed from index-CuIt1NQg.js to index-BdZpFO7j.js, so only omics_portal-web-1 was restarted at 2026-07-15T01:42:53.715330158Z; core retained image sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147 and start 2026-07-15T00:18:37.040145636Z.
• Real local-Qwen no-COT smoke: session 2026-07-15-014328-activity-filter-smoke-2026-07-15t01-43-2 reached its first reviewer gate with 0 activity_delta events and 3 tool lifecycle events, each containing exactly the four public fields. The probe accepted the close response, deleted only that session with 204, restored the exact saved settings object, confirmed the session absent, and left no Pi runtime.

Archived snapshot — Central activity log + compact CTE density (2026-07-15)

• Resolved UI contract. The central working body now renders every chronological non-blank assistant transcript line in one bounded accessible log, without user-entry echoes, timer/spinner labels, or step messages. The left Model activity panel is a default-deny projection of only thinking and status, while the complete raw activity fold and the existing reviewer widgets, artifacts, and workflow state remain unchanged. F6 CTE headers, content, table rows, and filter rows use 8 px vertical padding with 12 px lateral padding below sm and 16 px from sm upward; divider top padding is 8 px. The final review amendment keeps historical log rows at the full muted-foreground token so their normal-size text retains AA contrast.
• Source verification (source HEAD 09f9bdffe582ff3c66845ca219f60c11472a550a). Frontend tests passed 292/292 across 43/43 files. npx tsc -b, npm run build, the Impeccable layout detector, and git diff --check all exited 0; the detector returned [].
• Frontend-only deployment. The pre-deploy frontend was image sha256:8c9aaee6453b26c69e4057d3f9e53aa791d449b88ba4a0669a25cd442667d582, started 2026-07-15T10:17:16.919166255Z, serving index-CveSyban.js. Compose built and force-recreated only frontend with --no-deps; the final running frontend is image sha256:39dc47d81abcb13466218500476fbb78d1be66a424f4293470437700c848c26d, started 2026-07-15T10:30:30.869532521Z, serving index-CihtpQJV.js. Because the entry changed, exactly omics_portal-web-1 was restarted: it retained image sha256:4580cf2bc85f3656ee515996c24ed02d9096d24ed0d8afe384a2f8bd8cf4bac9 and moved from start 2026-07-15T10:17:36.151611451Z to 2026-07-15T10:30:46.369205604Z.
• Isolation and final state. Core remained running/healthy with exactly its original image sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147 and start 2026-07-15T00:18:37.040145636Z; frontend and portal were running with no container healthcheck. Pre/post docker top showed only core's supervisor and backend server, so no unrelated Pi runtime existed to disturb. The count-only frontend sensitive/error pattern scan was 0. No live model smoke was run, and settings and sessions were intentionally untouched.

Archived snapshot — Resizable activity split + compact CTE rows (2026-07-15)

• Resolved UI contract. activityLog remains the complete in-memory chronological fold. The left Model activity panel default-denies every kind except prompt, thinking, and assistant, labels those entries Question, Reasoning, and Response in source order, and hides status, tool, gate, lifecycle, and unknown kinds. The desktop panel is pointer/keyboard resizable from 288–576 px while preserving 512 px centrally, persists its global width in localStorage, and becomes an overlay drawer below lg or whenever the measured app shell is narrower than 800 px. F6 CTE cards retain their semantic structure and responsive grids; lateral padding is 12/16 px, header/content edge padding is 8 px, internal section gaps are 12 px, heading/divider spacing is 4 px, and table/filter rows use 4 px vertical padding with compact line heights.
• Source verification (2026-07-15T15:22:17+02:00, source HEAD f1af1f909b387ae12a10f1b534bf6a529ea42505). Frontend tests passed 295/295 across 44/44 files. npx tsc -b, npm run build, and git diff --check exited 0; the Impeccable layout detector returned []. The local source build emitted Vite entry assets/index-PlvQqhNG.js.
• Frontend-only deployment. The pre-deploy frontend was image sha256:39dc47d81abcb13466218500476fbb78d1be66a424f4293470437700c848c26d, started 2026-07-15T10:30:30.869532521Z, serving index-CihtpQJV.js. Compose built and force-recreated only frontend with --no-deps; the final running frontend is image sha256:0947211373784a860c7507d03c0fcf1f901ac3d1547cd0aa62b914d158f15bd5, started 2026-07-15T13:21:08.29613952Z, serving index-Dn7T524a.js. Because the entry changed, exactly omics_portal-web-1 was restarted: it retained image sha256:4580cf2bc85f3656ee515996c24ed02d9096d24ed0d8afe384a2f8bd8cf4bac9 and moved from start 2026-07-15T10:30:46.369205604Z to 2026-07-15T13:21:21.968808414Z.
• Isolation and final state. Core remained running/healthy with exactly its original image sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147 and start 2026-07-15T00:18:37.040145636Z; frontend and portal were running with no container healthcheck. Pre/post docker top showed only core's supervisor and backend server, so no unrelated Pi process existed and no Pi process was stopped or steered. The count-only frontend sensitive/error pattern scan was 0. No live model smoke was run; settings and sessions were intentionally untouched.

Archived snapshot — Final activity-split fix (2026-07-15)

• Source and verification (2026-07-15T15:56:57+02:00). Deployed source commit 1f540fcb78ac9e552e56a21e47edf66e9872b323 (1f540fc). Frontend Vitest passed 298/298 tests across 44/44 files; npx tsc -b and npm run build exited 0. The Impeccable detector scoped to AppShell, ModelActivityPanel, index.css, and CtePlanViewer returned []; git diff --check exited 0.
• Frontend-only deployment. Compose built and force-recreated only frontend with --no-deps. The frontend image changed from sha256:0947211373784a860c7507d03c0fcf1f901ac3d1547cd0aa62b914d158f15bd5 to sha256:6e14f55092b7e3aca9a396220394ae484147674d81b051771e394e59b73b1c88; its active Vite entry changed from index-Dn7T524a.js to index-BIznZeLH.js. Therefore exactly omics_portal-web-1 was restarted to refresh its manifest cache; it retained image sha256:4580cf2bc85f3656ee515996c24ed02d9096d24ed0d8afe384a2f8bd8cf4bac9 and started at 2026-07-15T13:56:32.786693915Z.
• Isolation and final state. Core retained image sha256:edd8f19ef269ee6f45ecaf464ba4053460d94378f9cdc967d2dbcb386f599147 and exact original start 2026-07-15T00:18:37.040145636Z, remaining running/healthy. Final frontend and portal states are running (no healthcheck). Pre/post core process tables contained only the supervisor and backend server, so Pi was preserved and no Pi process was stopped or steered. The count-only frontend sensitive/error-pattern scan was 0. No model smoke was run; settings and sessions were intentionally untouched.

What ThothII is

A human-in-the-loop datamart builder: it turns a natural-language question into validated SQL (and optionally a dbt datamart) through a deterministic 8-phase NL→SQL workflow, where the model proposes and a human reviewer decides at gates. The UI is meant to embed inside the Omics Portal (GSD design system) and is English.

Architecture — three layers

frontend (React, :5173) → backend (Fastify, :8787) → pi --mode rpc → tht / harness → DWH (read-only)

• harness/ — the Pi layer. A deterministic Python CLI tht + a Pi gate extension (.pi/extensions/tht-gate.js) that runs the 8-phase workflow and emits/consumes widget-descriptor JSON. Owns all persistence. Workflow truth is harness/workflow.yaml; orchestration rules are harness/.pi/skills/tht-sessione/SKILL.md.
• backend/ — Fastify + TypeScript. A thin bridge: proxies REST routes to the tht CLI (ThtRunner), manages Pi processes (PiProcessManager, one child per session), bridges Pi RPC events to SSE (SessionBridge + SseHub). No application database.
• frontend/ — React 18 + base-ui + Tailwind + TanStack Query + Zustand. Chat-style shell (src/shell/AppShell.tsx); the live transcript is rebuilt in-memory from the SSE stream (src/store/sessionStore.ts), not persisted.

Persistence model (the load-bearing premise)

There is no verbatim chat store. Each workflow phase persists its own document into the session directory, and that IS the persistence. A session = a directory under the workspace's sessions/ path containing session_manifest.yaml + phase artifacts (question.md, schema_linking.json, cte_plan.json, sql_final.sql, validation_report.md, review_decisions.jsonl, …). A fresh Pi process resumes by reading tht session show <id> + the on-disk artifacts — never by replaying chat.

The 8 phases (harness/workflow.yaml)

F1 chiarimento · F2 memoria · F3 riscrittura (question.md) · F4 schema_linking (schema_linking.json) · F5 sintesi · F6 cte (cte_plan.json, cte_tests.json) · F7 sql_finale (sql_final.sql) · F8 datamart. Current phase is a fold over the decision ledger (harness/tht/phase.py); statuses: open / closed / finalized.

How to run

Full stack (real Pi + DWH): from project root,

./scripts/run-stack.sh
# Opens frontend: http://localhost:5173 (proxies backend :8787)

Prereqs: VPN on; pi on PATH (with a configured model, e.g., pi model set claude-fable-5); harness/.env populated (see .env.example); harness/config/tht.yaml → workspace (psd recommended for testing). All three layers' deps installed (npm install in each, python -m venv + pip install -e ".[dev]" in harness).

Individual dev:

• backend: cd backend && npm run dev (tsx watch; env: PORT, THT_HARNESS_DIR, THT_BIN, PI_BIN, AUTH_MODE)
• frontend: cd frontend && npm run dev (Vite; VITE_BACKEND_URL → backend)
• harness install: cd harness && python -m venv .venv && pip install -e ".[dev]" → tht on PATH

How to test (latest TS gates green 2026-07-15: backend 204 / frontend 292 (43 files); harness 798 / gate JS 126 last recorded 2026-07-14)

• harness: cd harness && .venv/bin/pytest -q (5 L2/real-DB tests are deselected by default)
• backend: cd backend && npx vitest run · typecheck npx tsc --noEmit -p .
• frontend: cd frontend && npx vitest run · typecheck npx tsc -b · e2e npm run e2e (Playwright)

Config & workspaces

• Workspaces: harness/workspaces/*.yaml (psd, tht-test, tht.example). A workspace sets the DB target and the absolute paths.sessions/artifacts/indexes (psd → a separate repo tht-workspace-psd/, NOT committed here).
• Secrets live ONLY in harness/.env (gitignored; THT_* — DB, DWH REST, vector, SSL CA…). See harness/.env.example for the variable list.
• App settings (global): { workspace, provider, model, thinking }, persisted via harness preferences (tht session preferences get/set → the configured session repository — filesystem or Postgres in server mode). backend/data/settings.json (gitignored) remains only the file fallback for injected runners/tests. The "New session" form is question-only; these settings supply the rest.

Efficiency levers (NL→SQL workflow optimization, 2026-07-08)

Three deployed optimizations target model thinking time (the dominant cost, ~220s in F1 alone):

1. Join-graph via FK annotations + tht schema suggest-fks

   • DWH has no FK constraints declared. Annotations file (tht-workspace-*/artifacts/mschema/annotations.yaml) now stores curated logical FKs.
   • Three ranking rules: mine from approved SQL (highest confidence), heuristics (*_time_key → dim_time.day_key), same-name PK discovery with --assume flag for disambiguity.
   • Psd workspace: 228 FK suggestions already generated (139 tables); tht schema suggest-fks --from-sql <session-dir> --assume cod_paz=dim_patient for updates.
   • Activation: automatic. The mschema renderer populates the 【Foreign keys】 section. F4 in SKILL.md now reads FK joins from there instead of the model re-deriving them.

2. Context-pack consolidation at F1 kickoff (tht search pack)

   • Single embedding of the question, reused for schema + evidence + solved-question searches.
   • Command: tht search pack "<question>" --session <id> → sessions/<id>/retrieval_pack.md (tabelle candidate, relevant evidence, solved exemplars).
   • Graceful degradation: if Ollama or vector store unreachable (no VPN), sections are empty but exit 0 — session continues with live searches.
   • Activation: automatic at next session. SKILL.md F1 now prescribes as first call; reduces exploratory turns.

3. Phase-summary recap auto-construction from session ledger

   • tht session show --json includes the full decisions ledger; tht phase meta --json exports decision types per phase.
   • Gate appends deterministic 【Decisioni registrate in questa fase】 section to v2 phase-summary artifacts.
   • Model authors only summary + checks; the gate fills the recap table from persisted state → exact by construction.
   • Activation: automatic at next session and Pi restart. SKILL.md Disciplina 6 updated: model keeps output brief, gate enriches from catalog + ledger.

Tests: 358 Python + 111 JS gate, all pass. L2 (live DWH) verification on psd workspace recommended when time permits.

Conventions & contracts (don't relearn the hard way)

• -c/--config is a PER-COMMAND option in tht — append it AFTER the subcommand, never globally (ThtRunner.buildArgv handles this).
• --json output must be pristine (only valid JSON on stdout).
• UI strings are English. Document content stays in the workspace language (Italian for psd) because it's the real data; only chrome/labels are English.
• Settings are global, not per-question.
• TDD throughout; tests assert real behavior, not mocks. Frequent, scoped commits.
• Global user rules (~/.claude/CLAUDE.md): think before coding, simplicity first, surgical changes, goal-driven verification.

Active memory — F8 promotion gate + solved-question recall — SHIPPED, L2 pending (2026-07-07)

Two additions to close the loop on reusable memory, on top of the existing tht memory search (Phase 2) reuse:

• F8 promotion gate. reviewer_memory_promote (Phase 8, called with only the session id): the gate computes candidates deterministically via tht memory promote --preview --json (the 3 reusable decision types, already excluding previously promoted/declined ones) and shows a pre-selected checklist. Selected → tht memory save-one persists to the vectordb + records memory_promoted; deselected → memory_promotion_declined (ledger detail seq:<n>) so it is never re-proposed. tht memory promote/save-one were added to the gate's anti-bypass FORBIDDEN list (model must go through the gate tool).
• Solved-question exemplars. New vector kind solved_question reusing the existing memory pgvector table (no server-side DDL); harness/tht/solved.py does a one-row upsert keyed by a hash of question+SQL. CLI: tht memory solved-index / solved-search. tht session finalize auto-indexes the pair (best-effort: green line on upsert, cyan "già aggiornata" on dedup no-op, yellow warning + the recovery command tht memory solved-index <id> on failure). SKILL.md now prescribes calling solved-search as reference-only context in F4 (schema linking), F6 (CTE plan) and F7 (final SQL), and documents the finalize auto-index in "Session end".

Pending L2 gate (not yet run — needs VPN + writer key): one live end-to-end session on workspace psd via ./scripts/run-stack.sh to verify (a) the promotion checklist renders pre-selected and persists selected/declined correctly, (b) finalize indexes the pair, (c) tht memory solved-search returns it with sql + tables.

Fast-follow:

• RestSearcher top-k dilution — client-side DONE (2026-07-07): search_similar manda kinds alla RPC (filtro server-side esatto) con fallback automatico su server legacy (404 → retry senza filtro, post-filter client). Resta la migrazione server della funzione SQL search_similar (+kinds text[] DEFAULT NULL): istruzioni pronte in harness/docs/vector-rest-kinds-migration.md; l'ordine di deploy è libero, ma fino alla migrazione il filtro resta client-side e la diluizione persiste.
• tht memory solved-search muore con traceback grezzo se il vectordb è irraggiungibile DONE (2026-07-07): degrada a warning di una riga su stderr, stdout puro ([] in --json), exit 0 — copre VectorRestError/EmbeddingsError/OperationalError.

Review gates v2 — payload strutturati + viewer dedicati — COMPLETE (2026-07-07)

Plan: ~/.claude/plans/prima-di-passare-ai-inherited-marshmallow.md. Merged to main @ 2410f01 (ff, pushed). Executed via subagent-driven-development (4 workstreams, task reviews, final whole-branch review + fix wave).

• Contracts: artifact.data.schema_version: 2 for cte_plan / cte_result / phase, built deterministically by the gate (catalog descriptions via tht schema columns; SQL from ctes/<name>.sql; preview rows persisted by tht cte test); the model contributes only purpose/rationale/note. Non-v2 payloads fall through to the legacy renderers — old sessions and tools/replay/replay.json keep working.
• Harness (Python): CteTestRecord.preview_rows (+ _jsonable coercer, ≤10 rows, cells ≤200 chars); new read-only tht cte info <name> --session <id> --json (index/total from cte_plan.json, same source as next_cte); tht cte plan --doc - writes cte_plan_doc.json (chain documentation; cte_plan.json stays a load-bearing list[str]).
• Gate (JS): gate/artifact-contracts.js (soft validators → self-corrective textResult, TypeBox untouched) + gate/enrich.js (pure, catalog lookups injected); prepareReviewerArguments now coerces artifact.data too (GLM stringified-param mitigation); SKILL.md Phase 5/6 + disciplines rewritten (plan via reviewer_confirm kind:"cte_plan" with payload A; cte_result gates send THIN data only — never SQL/preview as text).
• Frontend: artifactV2.ts types; CtePlanViewer (per-CTE cards + chain strip), CteResultViewer (shiki SQL + AG Grid preview), PhaseSummaryViewer (checks + criteria with the VALUES driving choices), PreviewGrid extracted from ResultsPanel, statusBadge.ts shared success/warn/error tokens.
• Replay: v2 fixtures + tools/replay/augment-review-gates.mjs; replay.json regenerated; offline visual pass ok (screenshots in the SDD scratch dir).
• LIVE E2E (session 2026-07-07-011858, GLM 5.2): all 8 phases completed with the v2 gates; session finalized (DWH validation battery green, needs VPN).
• Bug found live + FIXED (2410f01): infinite spinner at workflow end — the bridge dropped Pi's agent_end (the ONLY end-of-turn signal) and working was released only by the next gate, which the final turn doesn't have. Now: bridge maps agent_end → SSE system_event; FE tracks agentActive; an unexpected Pi child exit notifies the client (info error + synthetic agent_end). Memory: pi-rpc-event-vocabulary.
• Open (non-blocking): tht.sqlcheck maps table aliases by first occurrence (found and worked around by the model in F6 — spawned as a separate task); one more live confirmation that the spinner stops at F8 (the chain is unit-tested end to end).

F4 schema-linking column curation + look&feel v2 — COMPLETE (2026-07-06)

Branches feat/f4-schema-linking-column-curation (PR #1) + feat/frontend-lookfeel-v2, landed on main (d942635 … 7491e8c). The F4 gate (reviewer_schema_linking) presents catalog-enriched tables/columns (descriptions from tht schema columns, hardened enrichment), per-table columns modal (suggested pre-checked, suggested-first ordering + filter box); decisions column_promoted/column_excluded + deterministic tht session sync-schema-linking projection into schema_linking.json. Live-verified including the clobber test (the model's joins write preserves curated columns). Look&feel v2: shadows/radii/ mono labels, 70% gate modal, structured cards (colors untouched). Memory: thothii-visual-language-v2.

Workflow contract hardening — COMPLETE (2026-07-01)

Spec: docs/superpowers/specs/2026-07-01-workflow-contract-hardening-design.md · Plan: docs/superpowers/plans/2026-07-01-workflow-contract-hardening.md. Merged to main @ 3dadc6f (pushed). Driven by analysis of Pi session 2026-06-30-165708 (GLM 5.2), where the model spent ~80% of its tool calls reverse-engineering the harness because SKILL.md mis-stated the phase-advance contract — and Phase 6 was a hard dead-end. Three coordinated harness fixes (TDD):

• F6 CTE-approval dead-end FIXED. The gate's reviewer_confirm kind:"cte_result" used to register cte_approved --subject phase:6, which decision_cmd rejects (exit 5 — it needs a real CTE name from the plan) → F6 could never close. New tht cte next --session <id> returns the first unapproved plan CTE; the gate now approves by name. (tht/cli/cte_cmd.py, .pi/extensions/tht-gate.js.)
• schema_linking.json writer/validator. store.set_schema_linking (validates against the SchemaLinking model, THEN writes — no partial file) → CLI tht session set-schema-linking <id> --file <path|-> (exit 5 on bad JSON / ValidationError) → gate tool write_schema_linking (stdin). Replaces the model hand-writing the F4 artifact + ad-hoc python validation.
• SKILL.md corrected to match the code. Only F2-empty / F6-skipped auto-advance (_AUTO_ADVANCE_PHASES={2,6}); every substantive phase closes with reviewer_confirm kind:"phase" (F7 is two-step: kind:"sql" records sql_approved, then kind:"phase" advances). Fixed Discipline 2 + Phase 1/3/4, added a per-phase cheat-sheet, documented the SchemaLinking shape. The old false "the reviewer_decide already advances" (F3) claim — the exact cause of the observed thrash — is gone.

Verified: harness pytest 281 passed / 5 deselected, gate JS 34/34, changed-files ruff clean (the 36 ruff check . errors are pre-existing on main). Final whole-branch review (opus): READY TO MERGE, no Critical/Important. Executed via subagent-driven-development (implementer + task-review per task, final opus review). DEFERRED (needs VPN): live F4/F6 end-to-end — resuming session 2026-06-30-165708 (stuck at F6) is the ideal live probe.

UI/UX redesign + Resume — COMPLETE (2026-06-30)

Plan: ~/.claude/plans/foamy-forging-dahl.md. Memory: thothii-ui-redesign-inprogress.md. All workstreams done and pushed to origin/main: D + E @ 0eeb3f7, B + C @ b056ff3, F @ cef9ae4, A @ 0a13f71, G @ e8cdd00(scope) + cbb8e18(results). Nothing pending from this plan. Per-workstream detail below for reference.

• D — DONE (c12bdcd): session display name = 3-5 Italian keywords via YAKE (no LLM), derived in tht session new (CLI layer); create_session core unchanged (name=None default). yake added to harness/pyproject.toml. TDD tests/test_session_name.py; harness 269 passed.
• E — DONE (0eeb3f7): rotating activity icon replaces the red dot in CentralStatus (inline, clickable → opens the panel); ModelActivityPanel is a 5-line expandable model-stream tail; WorkingSpinner extracted to its own module; the separate spinner button
  • orphaned Transcript.tsx removed. Frontend 87/87, tsc clean. Live visual check DONE (2026-06-30): inline spinner opens the panel; 5-line collapsed tail; expand → full transcript.
• B — DONE (b056ff3): WorkflowBar is now colored dots F1..F8, no phase-name text (amber-translucent=running, green=done, red=error, gray=pending; green connectors lead the active dot). Each dot carries data-state. Error is lightweight: store phaseError set when an info level=error arrives during the phase, cleared on the next ui_request (sessionStore.ts). All four states live-verified via Playwright.
• C — DONE (b056ff3): right sidebar — single-line denser rows (inline status dot + name, py-1), a 3-level type hierarchy via /impeccable (L1 SESSIONS red/bold/wide-tracking · L2 section + group headers muted uppercase · L3 names normal-case), and the "No group" label removed (ungrouped sessions render after the last group; guarded so the empty-state still teaches when there are no groups). Live-verified. (Resume in SessionMenu stays with A1.)
• Tests: frontend 93/93 (was 87; +3 store phaseError, +2 WorkflowBar dot-state, +1 AppShell no-"No group"), tsc -b clean.
• F — DONE (uncommitted; live check deferred to G): single-select answers auto-confirm. reviewer_select options may carry a decision payload ({type, subject, detail?, rationale?}) and an optional advance; picking such an option persists the decision directly via tht decision add (shared decisionAddArgs helper, also used by reviewer_decide) — no redundant reviewer_decide/reviewer_confirm gate. Options without a payload stay ask-only; back/exit/Other never persist. Pure logic extracted to resolveSelectOutcome/decisionAddArgs (exported, unit- tested). Contract docs updated: reviewer_select tool desc + SKILL.md (widget summary, disciplines 2-3, Phase-1 single-pick) + the CLAUDE.md gate note. Gate JS 33/33, harness 269. Live verification (model actually uses reviewer_select+decision, no follow-up gate, decision in review_decisions.jsonl) deferred to G — it is model-behavior-dependent.
• A — DONE (uncommitted): A1 — SessionMenu gains a Resume item (gated to status!=="finalized" && !archived), wired in AppShell to the existing doResume → POST /sessions/:id/resume. 3 tests (SessionMenu.test.tsx); frontend 96/96, tsc clean. A2 — diagnosis-first clean-room repro shows the resume cold-start stall NO LONGER reproduces on pi 0.79.4 (8/8 chained into the tool calls, fresh + partway; GLM 5.2 now narrates AND emits tht session show+read SKILL.md in-turn). The earlier narrate-and-stop predates the pi upgrade. Defense-in-depth applied: RIPRENDI_KICKOFF hardened to force the in-turn tool call (gate test + live regression 2/2). The cross-model angle (weaker/older models) lives in G.
• G — DONE (e8cdd00+cbb8e18): cross-model behavior matrix via a committed clean-room harness (harness/scripts/model-matrix.mjs). Tier 1 — kickoff + resume first-turn: all available models chain in-turn (zai/glm-5.2, deepseek/deepseek-v4-{pro,flash}, aritmolab/qwen3.6-35b-a3b, zai/glm-4.5-air); the resume stall recurs on none (closes A's cross-model robustness). aritmolab/gemma4-26b-a4b = 404 unavailable at the endpoint (listed but not served) — infra gap, not a workflow issue. Tier 2 — F single-select auto-confirm verified live on glm-5.2: answering the first reviewer_select persisted a concept_clarified decision 0→1 with no follow-up gate (closes F's deferred live check). Full results: the G plan doc + memory thothii-cross-model-matrix. No prompt hardening needed.

Status: All UI-redesign + resume workstreams done and pushed — D, E, B, C, F, A, G. Nothing pending from the plan. Optional nice-to-haves (not required): Tier-2 F/multiselect live for the non-baseline models (cheap re-run with harness/scripts/model-matrix.mjs + the Tier-2 method), and a one-off manual Playwright kebab→resume pass in the live UI.

Live verification + reviewer_select fix (2026-06-30, afternoon)

Drove the real stack (Playwright → backend → real Pi → GLM 5.2 → DWH) end-to-end.

• F1 hang fix (418187a) VERIFIED LIVE. Answered an F1 reviewer widget; Pi resumed (model socket reopened) and the gate produced new output — vs the old silent hang. The transition "silent hang → gate re-presents/advances" proves ctx.ui.input now resolves.
• New bug found + fixed: reviewer_select choices vs choice. The gate's reviewer_select (and reviewer_confirm reject) read resp.choice (singular) but the frontend uniformly sends choices: [id] (array) — so every single-select gate answered "Nessuna scelta ricevuta" and re-proposed forever (multiselect was fine; it already read choices). Fix: a shared selectedChoice(resp) helper (harness/.pi/extensions/tht-gate.js) reading the array; both handlers use it. TDD: gate/__tests__/gate_choice.test.js RED→GREEN, full gate suite 28/28. VERIFIED LIVE: a single-select answer is now accepted and the workflow advances (2/4 → 3/4).
• Resume cold-start STALL confirmed (open item #1). On /riprendi-sessione, GLM 5.2 narrates the bootstrap step then ends the turn without the tool call → Pi idle, unrecoverable from the UI. Memory: thothii-resume-cold-start-stall.md.
• GLM 5.2 F1 is slow (~3-4 min, ~50+ reads) but works — looks stuck but isn't; don't hit "Stop and save" (it POST /closes → kills Pi). Memory: thothii-glm52-f1-slow-not-stuck.md.

Earlier work — F1 reviewer-widget hang fix + multiselect guidance (committed 2026-06-30; authored 2026-06-29)

Two fixes, committed to main (7 files):

1. Bug: every reviewer widget hung "stuck with no output" after the human answered — F1 disambiguation (and any gate) dead-ended. Root cause, confirmed from Pi's own source (@mariozechner/pi-coding-agent dist/modes/rpc/rpc-mode.js, createDialogPromise): ctx.ui.input assigns its OWN RPC id (crypto.randomUUID) and correlates extension_ui_response on THAT id, silently dropping unknown ids. The gate puts a different id (u${Date.now()}) inside the descriptor carried in title. SessionBridge was replying with the descriptor id, so real Pi never resolved ctx.ui.input → the model never continued. Fix: SessionBridge now stores Pi's top-level m.id (pendingPiId) on the incoming request and replies extension_ui_response{ id: pendingPiId, value: <uiResponse JSON> } (value still carries the descriptor id, so the gate's internal resp.id === descriptor.id check holds). File: backend/src/bridge/session-bridge.ts. Full write-up: memory pi-ui-input-id-correlation.md.

   • The test double was masking it: harness/tests/fake_pi/fake_pi_rpc.mjs had forced m.id == descriptor.id. Corrected to mirror real Pi (distinct randomUUID top-level id, correlate on it, drop unknown ids); test_fake_pi_contract.mjs gained a negative regression test ("respond with descriptor id → no follow-up").
   • TDD: backend/test/session-bridge.test.ts (unit) + backend/test/e2e-f1.test.ts (integration — now asserts the model's follow-up arrives after the answer) went RED→GREEN.

2. UX: multi-answer disambiguation — harness/.pi/skills/tht-sessione/SKILL.md Phase 1 now tells the model to use reviewer_decide (the existing multiselect/checkbox widget) when an ambiguity admits several simultaneously-true answers, instead of single-pick reviewer_select. Guidance-only — no new widget (frontend MultiselectWidget already exists).

Verified at commit time: backend npx vitest run 67/67 green; tsc --noEmit -p . OK; fake-pi contract node --test test_fake_pi_contract.mjs 2/2 green. Verified LIVE 2026-06-30 (see the top "Live verification" section).

Most recent feature — Session management (MERGED to main @ 2c21e46)

Full session management modeled on Claude's UI, all three layers:

• Read-only "split view" panel (left drawer, SessionDocumentsPanel) showing a session's phase documents read-only (reuses SqlViewer/SchemaLinkingViewer/MarkdownView).
• Rename / Move to group / Archive / Delete via a kebab menu (SessionMenu) → REST → tht session set-name/set-group/archive/unarchive/delete. Archive = a manifest archived flag (not a dir move); groups = a manifest group field; delete = hard rmtree + confirm.
• Rail: collapsible group headers + "No group" + a separate Archive view.
• Resume correctness: read-only guard (HTTP 409 when finalized or archived); PiProcessManager.spawnFor now has a new/resume mode (resume sends /riprendi-sessione <id>); a "Phase 0 — Resume" cold-start section in SKILL.md.
• Design docs: docs/superpowers/specs/2026-06-29-session-management-design.md + docs/superpowers/plans/2026-06-29-session-management.md.

⚠️ Open items / pending gates

1. Resume cold-start stall — RESOLVED on pi 0.79.4 (workstream A, 2026-06-30). The earlier narrate-and-stop (GLM 5.2 narrating the bootstrap step then ending the turn without the tool call) no longer reproduces: a clean-room repro of the backend's exact resume handshake chained into tht session show+read SKILL.md in-turn 8/8 (fresh + partway sessions). The pi upgrade is the likely fix. Defense-in-depth: RIPRENDI_KICKOFF hardened to force the in-turn tool call (gate test + live 2/2). Memory: thothii-resume-cold-start-stall.md. Remaining: the cross-model angle (older/weaker models) is folded into G; a full Playwright kebab→resume pass through the live UI is still worth one manual run (item 2).
2. Full Playwright live-stack verification (MANUAL, not yet run).
3. Minor backlog (non-blocking): explicit id-traversal guard in delete_session (today gated by load_session); close_session could reuse _save_touched (DRY); delete-via-kebab integration test skipped (base-ui Menu portal not drivable in jsdom — the dialog itself is unit-tested); a couple of test-file lint nits.
4. DONE — F1 hang fix live-verified 2026-06-30 (see top section). The live verification also surfaced + fixed the reviewer_select choices mismatch.
5. Resolved: settings use zai/glm-5.2/medium (not deepseek-flash); GLM 5.2 drives F1 fine, just slowly (~3-4 min, ~50+ reads). To tell a truly stalled Pi from a merely-slow one, check its sockets/children. Memory: thothii-glm52-f1-slow-not-stuck.md.
6. DONE — Pi migrated to @earendil-works/pi-coding-agent@0.80.3 (2026-07-04). The old scope @mariozechner/pi-coding-agent is frozen at 0.73.1; every release ≥0.74 lives under the new scope @earendil-works (latest 0.80.3). The live pi (~/.local/bin/pi) was repointed to 0.80.3. Validated by an API/RPC-surface diff (0.73.1→0.80.3: ExtensionUIContext byte-identical, rpc-types additive-only, createDialogPromise + provider-registration API unchanged) plus a live model-matrix smoke (GLM 5.2, new+resume both CHAINED, full RPC event vocabulary incl. extension_ui_request intact). Notable: 0.80.3 adds ctx.mode: "tui"|"rpc"|"json"|"print" (a proper mode discriminator; the earlier 0.79.4 references above are superseded). Rollback: the old package is still on disk — repoint the symlink to @mariozechner/.../dist/cli.js. Memory: thothii-pi-earendil-migration.md.

Where design history lives

• Specs: docs/superpowers/specs/ · Plans: docs/superpowers/plans/
• SDD execution ledger (gitignored scratch): .superpowers/sdd/progress.md
• Auto-memory index: ~/.claude/projects/-Users-mp-projects-ThothII/memory/MEMORY.md (Pi RPC event vocabulary, ui.input id correlation, Omics Portal/GSD design system, resume cold-start stall, GLM 5.2 F1 slow≠stuck).

Git

main @ 2410f01, pushed to origin (github.com/mptyl/ThothII); working tree clean, no stashes. Latest arc (2026-07-07): review-gates-v2 ff-merged — 9b4f6b9 (WS1 harness) · 3ad93cd (WS2 gate+SKILL) · 1c97289 (WS3 viewers) · 4042d0b+b089482 (WS4 replay) · b59b57c (review fix wave) · 2410f01 (agent_end spinner fix). Branch feat/review-gates-v2 still exists (local + origin), fully merged. Before that (2026-07-05/06): F4 column curation (PR #1) + look&feel v2 — 0d9e035 · 0a63ba9 · d942635 · 9fe1c93 · e9b2934 · 9403147 · 7491e8c. Older history (workflow hardening 2026-07-01, UI redesign 2026-06-30): see the sections above; stale branches were pruned on 2026-06-30 (SHAs recoverable via reflog).