Commit Graph
539 Commits
Author SHA1 Message Date
marcopan 801f847ec4 fix: use Pi user auth and handle startup failures 2026-07-21 14:01:47 +02:00
marcopan 2ff63d371f feat: harden workflow gates and expose token usage 2026-07-21 12:14:26 +02:00
marcopan 8aa1676811 Expose PSD frontend locally 2026-07-20 20:27:37 +02:00
marcopan ad6057f0dd Fix PSD frontend network wiring 2026-07-20 20:26:13 +02:00
marcopan 32efd647c4 Merge session resume and container configuration fix 2026-07-20 20:23:13 +02:00
marcopan 0cf09777f2 Fix session resume and PSD container configuration 2026-07-20 20:22:47 +02:00
marcopan ec9b12dff4 fix(harness): recover schema table name typos 2026-07-20 17:53:24 +02:00
marcopanandClaude Opus 4.6 3b52c8c08c feat: DWH connectivity probe at startup — modal alert within 5s if unreachable
Backend: GET /health/dwh (unauthenticated) calls tht db ping with a 5s
timeout. Frontend: checkDwhHealth() races a 5s timer against the fetch;
on failure a non-dismissable Dialog with Retry appears immediately.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-20 13:59:11 +02:00
marcopanandClaude Fable 5 83942c0b5c feat(harness): F6/F7 close on their last approval — no echo phase gate
Where completeness is machine-detectable, the reviewer's last substantive
approval now closes the phase itself (same pattern as F3/F4/F8):

- F6: approving the LAST CTE of the plan (kind:"cte_result" with next_cte
  now empty) advances the phase; a non-final CTE keeps the phase open and
  names the next one.
- F7: kind:"sql" records sql_approved — which IS F7's only advance
  prerequisite — and advances immediately.

Two reviewer interactions per session removed, both pure echoes. The
summary phase gate remains only where completeness is a human judgment
(F1, F2 with recorded memories, F5). SKILL.md states the rule and the
five self-closing gates; L1 tests cover last/non-last CTE and sql close.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 02:23:45 +02:00
marcopanandClaude Fable 5 c4951e2aa5 chore: hygiene pass — ruff clean, docs storage-model truth, replay /me, failSession log
Audit findings 6.1-6.4 + the audit's remediation plan itself
(docs/superpowers/plans/2026-07-20-full-audit-remediation-plan.md).

- ruff: 34 → 0 (unused imports/f-strings auto-fixed; E702 semicolon lines
  split in test files; one unused local dropped). Suite still 819 green.
- CLAUDE.md + PROJECT_STATE.md no longer claim "no database / settings in
  settings.json": the harness selects filesystem OR PostgreSQL session
  storage (repository.py, server mode), and settings flow through harness
  preferences with the JSON file as fallback only.
- tools/replay: stub /me (SPA boot was parsing the SPA's own HTML as JSON)
  and /runtime/prewarm.
- failSession best-effort persistence now logs its failure server-side
  instead of vanishing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:55:41 +02:00
marcopanandClaude Fable 5 1ab0015460 fix(harness): state-integrity pass — reopen order, atomic decision batch, bash anti-bypass
Audit findings 5.1-5.3.

5.1 `phase reopen` now appends `phase_reopened` BEFORE the artifact
teardown: a crash between the two used to leave later-phase artifacts
deleted with the ledger still at the old phase (resume entered a phase
missing its artifacts). The inverse half-state — reopened with stale later
artifacts — is benign. Order locked by tests/test_phase_reopen_order.py.

5.2 New `tht decision add-batch --doc -`: N substantive decisions in ONE
atomic ledger write (meta types and cte_approved stay on `decision add`;
strictest min-phase enforced). reviewer_schema_linking now builds the
complete curation set and persists it with a single add-batch call — a
mid-loop failure can no longer leave the audit ledger half-written, and a
retry cannot duplicate the first K decisions.

5.3 The anti-bypass hook now also blocks BASH mutations of protected
state (`echo >> review_decisions.jsonl`, `sed -i` on the manifest,
`cat > tht-gate.js`, python open('w'), mv/rm/tee/…): FORBIDDEN only
covered tht subcommands and the write/edit hook only covered pi's own
tools. Read-only access (cat/grep/tail/ls) stays allowed.

Also: knownDecisionTypes is defensive — a workflow meta declaring NO
emits at all (older tht, minimal stubs) skips pre-validation instead of
rejecting every substantive type; with emits present, unknown types are
still rejected before the widget (new L1 test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:51:38 +02:00
marcopanandClaude Fable 5 f772ef9dca fix(backend): robustness pass — spawn leak, timeouts, workspace fail-loud, 409 order, respond guard
Audit findings 4.1-4.6.

- spawnFor: a rejected configure/start no longer leaks a registered runtime
  with a live Pi child (identity-checked teardown + rethrow); every later
  start used to hit "session runtime already active".
- ThtRunner.run: default 60s timeout on every tht child (SIGKILL backstop),
  120s for DWH-touching calls (sql preview/export, search pack); a dropped
  VPN mid-call no longer wedges the HTTP request forever.
- configArg: a NAMED workspace whose yaml is missing now throws instead of
  silently falling back to the default config (operations were silently
  targeting the wrong workspace).
- resume: the finalized/archived 409 is evaluated BEFORE the alreadyActive
  fast-path — the manifest is the truth even with a lingering runtime.
- ollamaEnsure: exit-0 with non-JSON stdout is a failed check, not ok:true.
- SessionBridge.respond: only the response matching the pending descriptor
  is forwarded to Pi; stale/duplicate submissions return 409 instead of
  being sent with the current gate's RPC id.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:37:20 +02:00
marcopanandClaude Fable 5 2958b32fd5 fix(backend): generation-aware SSE event ids — stale cursors can no longer eat events
Audit finding 3.1 (high, 3/3 reviewer consensus). Event ids restart at 1
when the backend restarts; a browser auto-reconnect carrying the old
numeric Last-Event-ID was honored whenever the new process had already
emitted that many events, silently suppressing fresh events (same ids,
different content). The previous guard only caught cursor > lastId.

Wire ids are now "<generation>:<seq>" (generation = per-hub instance
token; seq = the existing per-session monotonic counter). The hub parses
raw header/query candidates itself: other-generation and legacy bare-
number cursors are stale → replay from the beginning; same-generation
cursors keep the newest-valid-wins behavior. EventSource treats ids as
opaque, so no frontend change.

Finding 3.2 (eviction) resolved by NOT evicting: close keeps the seq
counter on purpose (sessions reopen; monotonicity is what makes old
cursors detectable) — documented at the call site; buffers are emptied by
clear() and ring-bounded at 200.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:32:47 +02:00
marcopanandClaude Fable 5 3e072fe652 fix: realign workflow advance semantics and phase labels with workflow.yaml
Audit findings 2.1 + 2.2 (high).

2.1 WorkflowBar's static phase list was fiction from F2 on (F2 "Schema
linking" vs memoria, F4 "SQL plan" vs schema_linking, …): every live
session showed the wrong phase name. Both maps now mirror
harness/workflow.yaml (F1 chiarimento … F8 datamart).

2.2 forceAdvance (6ee5bda) let reviewer_decide advance:true bypass the
phase gate on ANY phase, contradicting SKILL.md's "auto-advance only
empty F2 / skipped F6". reviewer_decide is back on advanceIfReady (exit-6
no-op) and tells the model to close via reviewer_confirm; forceAdvance
stays only where selection IS the approval by design: reviewer_schema_linking
(F4) and the F8 promotion close path. SKILL.md now names the three
self-closing gates (F3 rewrite_question, F4 schema-linking advance:true,
F8 memory_promote) so gate and skill state one contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:28:18 +02:00
marcopanandClaude Fable 5 075883370e fix(harness): close the Pi-crash class — top-level try/catch on ALL gate tools
Audit finding 1.1 (high): reviewer_memory_promote, rewrite_question,
write_schema_linking, write_cte_sql and write_final_sql still ran execute
without a top-level catch — the same unhandled-rejection class that killed
Pi in reviewer_schema_linking (fixed in 87cb806 for the four reviewer_*
tools). All 9 registered tools now share the pattern: any uncaught throw
becomes a textResult the model can react to.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:26:27 +02:00
marcopanandClaude Fable 5 803b98de36 fix(frontend): center gate dialogs on the app area, not the browser viewport
Embedded in the Omics Portal the SPA sits to the right of the portal sidebar,
but dialogs used `fixed left-1/2` — centered on the whole browser window, so
gates rendered visibly off-center (overlapping the sidebar).

- AppShell publishes the app area's horizontal geometry (--app-area-left/width)
  as CSS vars on <html> via ResizeObserver + window resize.
- DialogContent centers on those vars (falls back to the viewport when the app
  is standalone, where the two coincide).
- ArtifactGateWidget: narrower — cap at 52rem instead of 70vw.
- tools/replay: resume now returns {id, alreadyActive} JSON — the post-f979ada
  frontend reads result.alreadyActive and a bare 204 broke replay resume.

Verified on the replay harness: with a simulated 300px portal sidebar the gate
centers on the app area (center 790 = 300 + 980/2); standalone unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 17:08:20 +02:00
marcopanandClaude Opus 4.6 87cb806dfc fix(harness): prevent Pi crash on unhandled throw in reviewer tool execute
The last live session crashed during reviewer_schema_linking: an uncaught
exception (likely from execFileSync in currentPhase/phaseMeta or from
cat.columns being undefined) rejected the async execute() Promise. Pi does
not catch rejected tool Promises — Node.js treats them as unhandled
rejections and kills the process.

Fix:
- Wrap all four reviewer tool execute bodies (select/decide/confirm/
  schema_linking) in a top-level try/catch → returns a textResult on any
  unexpected error instead of crashing Pi.
- reviewer_schema_linking: defensively re-parse `tables` if still a string
  (belt-and-suspenders over prepareArguments), guard cat.columns before .map().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-19 14:33:42 +02:00
marcopanandClaude Opus 4.6 6ee5bda7f0 feat: gate decision-type validation, force-advance, and frontend fixes
Gate (tht-gate.js):
- Pre-validate decision types against workflow.yaml before showing reviewer widget
- Reject decisions emitted by later phases (min-phase check)
- Copy top-level `kind` into artifact when model forgets it (prevents loop)
- Force-advance on reviewer_decide/schema_linking when advance:true — skip
  redundant reviewer_confirm gate

Backend:
- Emit agent_end on clean Pi exit (code 0 + bridge idle) instead of marking failed

Frontend:
- Strip <think> tags from transcript and activity panel
- Fix mermaid render with offscreen container + cleanup
- Graceful mermaid error: show source code instead of red error, fall back to table

Workflow:
- F2 now emits table_promoted and table_excluded (early schema linking decisions)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-19 14:14:44 +02:00
marcopanandClaude Opus 4.8 9fbca06f7c feat(harness): make tht schema columns glob/pattern-aware
At F4 (schema_linking) a model asking for a whole table family, e.g.
`fact_sost_impianto_*`, used to hit a bare "tabella non nel catalogo" and stall
without recovering. Now a pattern (containing * ? [) resolves to every matching
catalog table and returns their columns (JSON becomes an array of per-table
objects); exact names keep the original single-object contract. A non-glob miss
also suggests sibling tables sharing the leading segment, to aid recovery.

Verified live: `fact_sost_impianto_*` resolves the 20-table family on the psd
catalog. Harness suite green (811 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 13:24:20 +02:00
marcopanandClaude Opus 4.8 ccfa3a18bf feat(frontend): pulse the stop dot while the harness is working
Give live feedback that something is happening the moment a session starts
processing: the red stop dot in the composer beats with a soft heartbeat + halo
while the harness is actively working (running — not waiting at a gate or
finalized), and settles the instant the model hands control back to the reviewer
or the session ends. Ring/scale only (no layout properties animated); honors
prefers-reduced-motion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 12:33:25 +02:00
marcopanandClaude Opus 4.8 3453f3ae23 feat: pre-check DWH reachability before creating a session (local dev only)
New session now refuses to spawn a Pi runtime that would only die in bootstrap
retrieval when the DWH/vector host is unreachable (e.g. a dropped VPN). Before
`session new`, POST /sessions probes the DWH via `tht db ping`; if it is down it
returns 503 {code:"dwh_unreachable"} with a clear message and creates nothing.

- Gated behind the THT_DWH_PRECHECK flag (default off), enabled only by the local
  dev launcher (run-stack.sh) — containers/CI never pay the probe, and existing
  tests that don't set it are unaffected.
- ThtRunner.dbPing() runs `tht db ping` with a 10s timeout (run() gains an optional
  timeout that SIGKILLs a hung child).
- Frontend: apiFetch throws a typed ApiError (status + parsed payload); the new-
  session composer shows the specific alert on `dwh_unreachable` instead of the
  generic retry hint, keeping the question for retry.

Verified live on an isolated backend (precheck on + broken DWH host → 503
dwh_unreachable, no session created) and via unit tests (backend 228, frontend 308).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 12:08:47 +02:00
marcopanandClaude Opus 4.8 84da3b149b fix(backend): log the real bootstrap failure cause server-side
Session bootstrap swallowed configure/retrieval errors and surfaced only the
generic BOOTSTRAP_FAILURE_MESSAGE, so an operator could not tell why a session
"didn't start" — e.g. `tht search pack` failing because the DWH/vector host is
unresolvable behind a dropped VPN. Log the underlying error to the backend
console; the client-facing message stays generic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-18 11:48:05 +02:00
marcopanandClaude Opus 4.8 f979ada5e7 feat(frontend): open a live session straight to its pending gate
Opening an in-progress session that has a live Pi runtime now reconnects to
its pending gate instead of the empty landing screen that read as "stopped".
Cold/completed sessions keep the read-only documents panel with its explicit
Resume, so a mere click never spawns a runtime. Backend GET /sessions now
reports a per-session `active` flag (live runtime bound) to drive this.

Also:
- "New session" now closes any open session detail panel (left box).
- The model-activity separator can be dragged to a full 50/50 split
  (was capped at 576px); central-min still guards narrow viewports.

Test fixes uncovered along the way:
- Node 25 ships an experimental global localStorage that shadows jsdom's and
  lacks clear(), failing every jsdom test at setup; install a spec-compliant
  in-memory Storage (feature-detected, inert on CI/LTS).
- Fix 4 pre-existing session-mgmt tests that used an ambiguous getByText for a
  session shown in both nav and header; target the nav item by test id.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 22:56:12 +02:00
marcopan 6274bf2da0 Merge branch 'feat/per-provider-model-credentials' 2026-07-17 19:31:06 +02:00
marcopanandClaude Opus 4.8 c5fd03ed84 feat(backend): let pi self-authenticate providers from its own auth store
The backend injects a single managed model key (THT_MODEL_API_KEY[_FILE]) as
the selected provider's env var, but that key belongs to one provider — so
selecting a second cloud provider (e.g. DeepSeek while the managed key is zai's)
forced the wrong key onto it and failed auth. This is why the model could not be
switched to DeepSeek.

When the selected provider is present in pi's own auth store
(~/.pi/agent/auth.json), skip injection and let pi resolve that provider's key
itself. Deployments without an auth store (containers) yield an empty set, so the
managed-key injection stays authoritative and fail-fast there. authProviders is
injectable into PiProcessManager for deterministic tests.

Verified live: GLM 5.2, DeepSeek V4 Flash, and aritmolab Qwen3.6 all operate through the ThothII model selector.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 18:26:18 +02:00
marcopanandClaude Opus 4.8 c252c95c5f fix(frontend): pin the question as a stable first line during processing
The central column was top-anchored with no fixed header, so the user's
question was never shown while the model worked and the activity/gate
content drifted upward. Render the active session's question as a sticky,
always-present first row of the central area so it stays evident and
anchored at the top of the form.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 16:30:18 +02:00
User bd7fa6d2c1 Merge branch feat/user-owned-sessions 2026-07-16 20:40:51 +02:00
User 3abe69badc docs: record session storage deployment gotchas 2026-07-16 20:40:51 +02:00
User 5cacf70a0d fix: bootstrap user preferences without invalidating DWH cache 2026-07-16 20:32:20 +02:00
User ccb3cf4aa9 test: cover user-owned session security boundaries 2026-07-16 19:16:58 +02:00
User 7a65fc80a2 fix(deploy): validate session migrator TLS mode 2026-07-16 19:07:53 +02:00
User cadc4c6947 docs(deploy): document user-owned session cutover 2026-07-16 19:02:58 +02:00
User c6a74c9ccb fix(frontend): harden owner action confirmation 2026-07-16 18:51:02 +02:00
User 58e0884df1 feat(frontend): expose owned session scopes 2026-07-16 18:46:18 +02:00
User b454fb478b fix(backend): harden principal child isolation 2026-07-16 18:38:40 +02:00
User 458eb13c89 feat(backend): enforce user-owned sessions 2026-07-16 18:32:52 +02:00
User 72db6b823d fix(harness): record postgres decision phases 2026-07-16 18:08:17 +02:00
User c1cddaa667 refactor(harness): route workflow persistence through repositories 2026-07-16 18:01:29 +02:00
User 259f021313 fix(harness): preserve session migration JSON errors 2026-07-16 17:35:54 +02:00
User 5dbb91390f feat(harness): persist server sessions in postgres 2026-07-16 17:31:04 +02:00
User 8f0154eb9e fix(harness): persist evidence artifact 2026-07-16 17:14:47 +02:00
User 8f364ec564 feat(harness): add local session repository 2026-07-16 17:11:22 +02:00
User 8296937bc0 docs: add user-owned session storage plan 2026-07-16 17:01:39 +02:00
User 671bd20329 chore: ignore Vitest cache 2026-07-16 13:03:38 +02:00
User c587f9dc4a merge: resizable activity timeline and CTE density 2026-07-15 16:06:19 +02:00
User 7fc58287a7 docs: clarify activity split fallback 2026-07-15 15:59:21 +02:00
User 6a5ca7e8cb docs: record final activity split deployment 2026-07-15 15:57:33 +02:00
User 1f540fcb78 fix(frontend): harden activity split responsiveness 2026-07-15 15:52:22 +02:00
User f0ef2654df docs: record activity split deployment 2026-07-15 15:32:29 +02:00
User f1af1f909b style(frontend): tighten CTE plan vertical rhythm 2026-07-15 15:15:56 +02:00