86 Commits
Author SHA1 Message Date
Codex bd416f7327 Fix new-question landing and question-language HITL
Publish documentation / publish (push) Successful in 34s
Reset the activity panel when starting a new question so the landing navigation is restored. Detect and persist the original question language, pass it through runtime and widget descriptors, and scope HITL controls to that language.

Validated with gate, session, backend and frontend tests, TypeScript checks, Ruff and strict docs build. Rebuilt and restarted local core/frontend; both healthy and serving HTTP successfully.
2026-09-21 19:47:22 +02:00
Codex 84084bba37 Fix Qwen session tool calls and expose thinking compatibility
Publish documentation / publish (push) Successful in 30s
2026-09-21 16:23:51 +02:00
Codex d8a29bfbdd Add full shell, replaceable Omics adapter and bilingual interaction
Implement approved specification #32 and tickets #33-#37. Keep host authentication server-verified and pin session interaction language. Compile scoped base selectors for browser compatibility and retain full gutters during CSS pruning.
2026-09-13 14:26:39 +02:00
Codex 82e2c91f42 feat: implement memory and evidence administration with guided repairs
Publish documentation / publish (push) Successful in 1m27s
Add PostgreSQL-backed memory, editable evidence with source review and activation, and human-approved archive repairs across the harness, API, and UI. Include migrations, deployment support, regression coverage, and validation documentation.

Refresh permissions from validated session roles so existing administrator logins can access newly deployed archive management features.
2026-09-10 10:31:34 +02:00
Codex 38f02cfd08 feat: complete evidence restructuring worktree 2026-08-26 11:39:02 +02:00
marcopan 8ba87b68dc fix(evidence): stabilize real Pi authoring 2026-08-25 15:21:59 +02:00
marcopan cc30148b69 refactor(evidence): unify formulas with typed evidence 2026-08-25 01:23:04 +02:00
marcopan f1a9b567ba feat(evidence): contribute to semantic stages (#42) 2026-08-24 22:05:30 +02:00
marcopan f5c7cc6198 feat(evidence): prepare curated evidence incrementally 2026-08-24 20:08:27 +02:00
marcopan fa2298653b fix(ui): stream phase progress without gates 2026-08-24 12:10:56 +02:00
marcopan 36a7a0ab33 refactor(workflow): contract shared core (#33) 2026-08-24 03:12:00 +02:00
marcopan 1e459b073e refactor(disambiguation): own F1 clarification policy (#27) 2026-08-24 01:55:42 +02:00
marcopan 2ed55ef131 refactor(disambiguation): extract F3 rewrite path (#26) 2026-08-24 01:44:40 +02:00
marcopan eccf6212f1 refactor(pi): generate modular session instructions (#25) 2026-08-24 01:36:08 +02:00
marcopan 93fe0d733b refactor(memory): extract F2 recall path (#23) 2026-08-24 01:08:15 +02:00
marcopan beac2e80f4 refactor(memory): extract F8 promotion gate (#22) 2026-08-24 00:51:53 +02:00
marcopan d15bb59c3d test(workflow): complete observable baseline (#21) 2026-08-24 00:40:11 +02:00
marcopan dc9726cb35 test(workflow): freeze observable contracts (#21) 2026-08-24 00:25:17 +02:00
marcopan 694b7dd21f fix: harden reviewer workflow and memory handling 2026-07-23 12:34:23 +02:00
marcopan 2ff63d371f feat: harden workflow gates and expose token usage 2026-07-21 12:14:26 +02:00
marcopan 0cf09777f2 Fix session resume and PSD container configuration 2026-07-20 20:22:47 +02:00
marcopan ec9b12dff4 fix(harness): recover schema table name typos 2026-07-20 17:53:24 +02:00
marcopanandClaude Fable 5 83942c0b5c feat(harness): F6/F7 close on their last approval — no echo phase gate
Where completeness is machine-detectable, the reviewer's last substantive
approval now closes the phase itself (same pattern as F3/F4/F8):

- F6: approving the LAST CTE of the plan (kind:"cte_result" with next_cte
  now empty) advances the phase; a non-final CTE keeps the phase open and
  names the next one.
- F7: kind:"sql" records sql_approved — which IS F7's only advance
  prerequisite — and advances immediately.

Two reviewer interactions per session removed, both pure echoes. The
summary phase gate remains only where completeness is a human judgment
(F1, F2 with recorded memories, F5). SKILL.md states the rule and the
five self-closing gates; L1 tests cover last/non-last CTE and sql close.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 02:23:45 +02:00
marcopanandClaude Fable 5 1ab0015460 fix(harness): state-integrity pass — reopen order, atomic decision batch, bash anti-bypass
Audit findings 5.1-5.3.

5.1 `phase reopen` now appends `phase_reopened` BEFORE the artifact
teardown: a crash between the two used to leave later-phase artifacts
deleted with the ledger still at the old phase (resume entered a phase
missing its artifacts). The inverse half-state — reopened with stale later
artifacts — is benign. Order locked by tests/test_phase_reopen_order.py.

5.2 New `tht decision add-batch --doc -`: N substantive decisions in ONE
atomic ledger write (meta types and cte_approved stay on `decision add`;
strictest min-phase enforced). reviewer_schema_linking now builds the
complete curation set and persists it with a single add-batch call — a
mid-loop failure can no longer leave the audit ledger half-written, and a
retry cannot duplicate the first K decisions.

5.3 The anti-bypass hook now also blocks BASH mutations of protected
state (`echo >> review_decisions.jsonl`, `sed -i` on the manifest,
`cat > tht-gate.js`, python open('w'), mv/rm/tee/…): FORBIDDEN only
covered tht subcommands and the write/edit hook only covered pi's own
tools. Read-only access (cat/grep/tail/ls) stays allowed.

Also: knownDecisionTypes is defensive — a workflow meta declaring NO
emits at all (older tht, minimal stubs) skips pre-validation instead of
rejecting every substantive type; with emits present, unknown types are
still rejected before the widget (new L1 test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:51:38 +02:00
marcopanandClaude Fable 5 3e072fe652 fix: realign workflow advance semantics and phase labels with workflow.yaml
Audit findings 2.1 + 2.2 (high).

2.1 WorkflowBar's static phase list was fiction from F2 on (F2 "Schema
linking" vs memoria, F4 "SQL plan" vs schema_linking, …): every live
session showed the wrong phase name. Both maps now mirror
harness/workflow.yaml (F1 chiarimento … F8 datamart).

2.2 forceAdvance (6ee5bda) let reviewer_decide advance:true bypass the
phase gate on ANY phase, contradicting SKILL.md's "auto-advance only
empty F2 / skipped F6". reviewer_decide is back on advanceIfReady (exit-6
no-op) and tells the model to close via reviewer_confirm; forceAdvance
stays only where selection IS the approval by design: reviewer_schema_linking
(F4) and the F8 promotion close path. SKILL.md now names the three
self-closing gates (F3 rewrite_question, F4 schema-linking advance:true,
F8 memory_promote) so gate and skill state one contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:28:18 +02:00
marcopanandClaude Fable 5 075883370e fix(harness): close the Pi-crash class — top-level try/catch on ALL gate tools
Audit finding 1.1 (high): reviewer_memory_promote, rewrite_question,
write_schema_linking, write_cte_sql and write_final_sql still ran execute
without a top-level catch — the same unhandled-rejection class that killed
Pi in reviewer_schema_linking (fixed in 87cb806 for the four reviewer_*
tools). All 9 registered tools now share the pattern: any uncaught throw
becomes a textResult the model can react to.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 01:26:27 +02:00
marcopanandClaude Opus 4.6 87cb806dfc fix(harness): prevent Pi crash on unhandled throw in reviewer tool execute
The last live session crashed during reviewer_schema_linking: an uncaught
exception (likely from execFileSync in currentPhase/phaseMeta or from
cat.columns being undefined) rejected the async execute() Promise. Pi does
not catch rejected tool Promises — Node.js treats them as unhandled
rejections and kills the process.

Fix:
- Wrap all four reviewer tool execute bodies (select/decide/confirm/
  schema_linking) in a top-level try/catch → returns a textResult on any
  unexpected error instead of crashing Pi.
- reviewer_schema_linking: defensively re-parse `tables` if still a string
  (belt-and-suspenders over prepareArguments), guard cat.columns before .map().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-19 14:33:42 +02:00
marcopanandClaude Opus 4.6 6ee5bda7f0 feat: gate decision-type validation, force-advance, and frontend fixes
Gate (tht-gate.js):
- Pre-validate decision types against workflow.yaml before showing reviewer widget
- Reject decisions emitted by later phases (min-phase check)
- Copy top-level `kind` into artifact when model forgets it (prevents loop)
- Force-advance on reviewer_decide/schema_linking when advance:true — skip
  redundant reviewer_confirm gate

Backend:
- Emit agent_end on clean Pi exit (code 0 + bridge idle) instead of marking failed

Frontend:
- Strip <think> tags from transcript and activity panel
- Fix mermaid render with offscreen container + cleanup
- Graceful mermaid error: show source code instead of red error, fall back to table

Workflow:
- F2 now emits table_promoted and table_excluded (early schema linking decisions)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-19 14:14:44 +02:00
User c1cddaa667 refactor(harness): route workflow persistence through repositories 2026-07-16 18:01:29 +02:00
User 9d4e057426 fix: serialize decision ledger writes 2026-07-14 15:30:06 +02:00
User 333874a755 fix: make join approval atomic 2026-07-14 15:24:06 +02:00
User e228c6f2fd fix: harden phase summary open questions 2026-07-14 15:09:50 +02:00
User 27af6733c8 fix: make join review read only 2026-07-14 15:06:16 +02:00
User 1b1554b353 fix: auto-approve phase 3 rewrite 2026-07-14 13:51:39 +02:00
User c7d586aaf7 fix: clarify memory selection semantics 2026-07-14 13:13:59 +02:00
User 6dbf93fff9 feat: harden runtime readiness and session workflow 2026-07-14 10:27:25 +02:00
marcopan 08f0029793 fix: finalize session after memory promotion 2026-07-12 14:26:48 +02:00
marcopanandClaude Opus 4.6 893ad99594 fix(gate): auto-finalize session after last phase approval
The model sometimes stops after receiving 'Fase approvata' without calling
`tht session finalize`, leaving the session open. Now the gate itself calls
finalize after advancing the max phase (F8), making session closure
deterministic regardless of model behavior.

- reviewer_confirm kind:phase: after phase advance at max_phase, gate calls
  `tht session finalize <session>` (best-effort with recovery message)
- SKILL.md updated: model no longer needs to call finalize itself
- Tests: 2 new JS tests (auto-finalize at max phase; no-finalize at non-max)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-07 18:47:19 +02:00
marcopanandClaude Fable 5 e24b41b156 feat(opt): three efficiency levers for NL→SQL workflow
Lever 1: Join-graph via FK logics in annotations + suggest-fks command
  - TableAnnotation.foreign_keys field stores curated logical FKs (DWH has no FK constraints)
  - tht schema suggest-fks: mine from approved SQL, heuristics (time_key → dim_time),
    same-name discovery + explicit --assume flag for multi-owner PKs
  - mschema renders 【Foreign keys】 section populated; validation in merge.py
  - SKILL.md F4 now reads FKs from mschema-text, no custom data_time_key logic

Lever 2: Context-pack consolidation at kickoff (tht search pack)
  - Single embedding of question, reused for schema + evidence + solved searches
  - One command: tht search pack <question> --session <id> → retrieval_pack.md
  - Graceful degradation when Ollama/vector store unreachable (exit 0, empty sections)
  - SKILL.md F1 prescribes as first call; reduces model thinking turns via pre-retrieval

Lever 3: Phase-summary recap v2 auto-construction from session ledger
  - tht session show --json includes full decisions ledger
  - tht phase meta --json exports 'emits' (substantive decision types per phase)
  - Gate appends deterministic 【Decisioni registrate in questa fase】 section (appendLedgerSection)
  - Model authors only summary + checks; recap table comes from persisted state (exact by construction)
  - SKILL.md Disciplina 6: brief model output, gate fills the rest

Tests: 358 Python (including 10 FK + 3 pack + 1 session-ledger tests) + 111 JS gate tests, all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 17:43:08 +02:00
marcopanandClaude Fable 5 34aeda006e perf(f1): cache-guard schema introspect + F1 toolbox in skill (~-5 min per session)
Transcript analysis (session 2026-07-06-175012, GLM 5.2) showed F1 at 567s:
182s wasted on a useless `tht schema introspect` (re-introspecting the remote
DWH although physical.yaml was already materialized) plus ~220s of model
thinking inflated by ~7 exploratory turns (--help/find/cat). The actual
searches cost ~15s; reviewer gates (~145s, untouched) are the quality contract.

- schema_cmd.py: introspect now exits 0 with "OK (cache)" in ~1s when
  physical.yaml exists; --refresh forces the real re-introspection.
  Deterministic cross-model guarantee, verified live on psd (163 tables, 1.2s).
- SKILL.md: F1 toolbox (only `tht search find` + `tht schema render`; no
  introspect/--help/filesystem browsing; batch all searches in one turn);
  F4 step 1 is render-only with a one-shot introspect fallback.
- tht-gate.js: `tht schema introspect ... --refresh` added to FORBIDDEN
  (maintenance stays shell-only, never in-session).
- tests: 4 new pytest cases (cache hit placement proven with fake credentials,
  refresh bypass, corrupt-catalog fall-through, render fallback message) and
  2 gate anti-bypass JS cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:02:45 +02:00
marcopanandClaude Fable 5 8d427a2ccb fix(gate): actionable recovery for mid-promotion failures; truthful solved-index copy; fast-follow notes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:07:00 +02:00
marcopanandClaude Fable 5 8af0552dcc docs(skill): prescribe solved-question recall in F4/F6/F7; refresh PROJECT_STATE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:54:42 +02:00
marcopanandClaude Fable 5 ae89957175 docs(skill): prescribe the F8 memory-promotion gate; drop optional D11 notes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:34:04 +02:00
marcopanandClaude Fable 5 e25ba5126b feat(gate): reviewer_memory_promote — deterministic F8 memory-promotion gate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:30:35 +02:00
marcopanandClaude Fable 5 b59b57c4e6 fix(review-gates): green success badges, null-safe preview rows, optional section items
- Extract shared statusBadgeClass() helper (src/viewers/statusBadge.ts) so
  SqlViewer, CteResultViewer and PhaseSummaryViewer can't drift: ok/success/
  passed/promoted render green (--success), warn renders amber (--warning),
  error/failed stay destructive red, everything else stays neutral outline.
  Previously CteResultViewer/PhaseSummaryViewer mapped "ok"/"promoted" to the
  default badge variant, which is bg-primary (GSD red) — success states
  rendered red.
- enrich.js: buildCteResultV2 now falls back preview.rows to [] instead of
  null when last_test.preview_rows is missing (pre-upgrade ok records), and
  CteResultViewer reads result.preview?.rows?.length with a null-safe
  fallback so it degrades to the "No preview rows" empty state instead of
  crashing.
- PhaseSummaryViewer: section.items is optional (model-authored sections can
  be prose-only); render (section.items ?? []) instead of crashing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 01:37:35 +02:00
marcopanandClaude Fable 5 3ad93cd02f feat(gate): deterministic v2 review-gate payloads (cte_plan/cte_result/phase)
The tht-gate.js reviewer_confirm now builds structured v2 artifacts before the
widget so the reviewer approves gate-derived data, not raw model text:

- new pure modules gate/artifact-contracts.js (soft validators, {ok,errors},
  legacy-passthrough) and gate/enrich.js (index/description enrichment,
  buildCteResultV2 fusing thin model data with `tht cte info`, phase enrichment)
- cte_plan v2: validate + enrich + persist via `tht cte plan --name … --doc -`
  (names derived from data.ctes[]); legacy `names` param kept as fallback
- cte_result v2: rebuild from `tht cte next`/`tht cte info` (sql + preview from
  the persisted test record); null/error last_test -> actionable textResult
- phase v2: soft-validate + fill phase from meta + catalog descriptions
- prepareReviewerArguments coerces artifact.data too (GLM double-stringify);
  legacy markdown strings pass through unchanged
- SKILL.md: Phase 6 cte_plan payload A + thin cte_result guidance; Discipline 6
  payload C example; Discipline 7 reworded for gate-rebuilt cte_result

Legacy (non-v2) paths unchanged. TypeBox stays Type.Any() for artifact.data;
validation is soft (textResult) so models self-correct instead of looping.
All 102 gate JS tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:42:48 +02:00
marcopanandMarco Pancotti d942635085 fix(f4): harden schema-columns enrichment + ValidationError guard + staged-helper note 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti bd63e3194f docs(skill): Phase 4 uses reviewer_schema_linking; honor curated output columns 2026-07-06 23:25:00 +02:00
ad788278dd feat(gate): reviewer_schema_linking tool with per-column curation + deterministic sync
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 559f52e26e feat(gate): buildSchemaLinkingRequest descriptor builder 2026-07-06 23:25:00 +02:00