Commit Graph
293 Commits
Author SHA1 Message Date
User 4d80ed0b83 fix(compose): use portal network omics_portal_omics_network + DNS aliases
Compose prefissa le reti col project: la rete reale del portale è
omics_portal_omics_network (non omics_network). Aggiunti alias
thothii-core/thothii-frontend per il DNS usato dagli upstream nginx.
Verificato: portal nginx -> thothii-core:8787 = {ok}.
2026-07-12 17:20:51 +02:00
User 5f6647e77a fix(entrypoint): restore server case (lost during doctor->check refactor)
Il caso 'server)' era stato eliminato inavvertitamente: CMD [server]
cadeva nel *) exec $@ -> 'server: not found' (exit 127).
Smoke standalone ora verde: health + config check + db ping (read-only).
2026-07-12 17:17:24 +02:00
User 3f816044c3 fix(sql): move ROLE PASSWORD outside DO block (psql skips :var in dollar-quoting) 2026-07-12 17:12:26 +02:00
User 67d030b24d feat(deploy): docker images, compose, roles SQL, local workspace
- core.Dockerfile: python:3.12-slim + node 22 copied (same bookworm glibc), non-root, tht+pi
- frontend.Dockerfile: vite build (env-driven base/assetsDir) + nginx-unprivileged
- compose.yaml (embedded, omics_network ext, zero host ports) + docker-compose.dev.yml (standalone)
- deploy/sql: thoth_dwh_reader (ro) + thoth_vector_rw (rw) roles
- deploy/thothii.env.example + harness/workspaces/local.yaml (direct DWH+vector, 5438)
- scripts/docker-smoke.sh; .dockerignore; gitignore deploy secrets
- verified: both images build, core health {ok}, config check validates local.yaml
2026-07-12 16:49:03 +02:00
User 3fd4b0db86 feat(deploy): configurable backend HOST + env-driven vite base/assetsDir
- backend: HOST env (default 127.0.0.1, dev-safe; 0.0.0.0 in container)
- frontend: VITE_BASE drives base/assetsDir/manifest for portal embedding
- backward compatible: no env => identical to previous behavior
2026-07-12 16:32:00 +02:00
User 0c9b44e61f docs(deploy): local docker + portal embed implementation plan 2026-07-12 16:27:44 +02:00
marcopan 08f0029793 fix: finalize session after memory promotion 2026-07-12 14:26:48 +02:00
marcopan e02e61ea35 chore: ignore local worktrees 2026-07-11 19:54:05 +02:00
marcopan ef46a4ca09 docs: add portable deployment implementation plans 2026-07-11 19:49:26 +02:00
marcopan 586180eaa1 docs: define portable deployment architecture 2026-07-11 19:39:43 +02:00
marcopanandClaude Opus 4.6 1e4bc11418 fix(embed): fast-fail + auto-restart Ollama on solved-search hang
Embeddings timeout was 120s, causing multi-minute hangs when Ollama was
down during F4/F6/F7 solved-search. Now: connect_timeout=5s across all
HTTP clients (REST + Ollama), read_timeout reduced to 30s for embeddings,
and OllamaEmbeddings auto-restarts the server on ConnectionError before
degrading gracefully.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-07 19:49:55 +02:00
marcopanandClaude Opus 4.6 893ad99594 fix(gate): auto-finalize session after last phase approval
The model sometimes stops after receiving 'Fase approvata' without calling
`tht session finalize`, leaving the session open. Now the gate itself calls
finalize after advancing the max phase (F8), making session closure
deterministic regardless of model behavior.

- reviewer_confirm kind:phase: after phase advance at max_phase, gate calls
  `tht session finalize <session>` (best-effort with recovery message)
- SKILL.md updated: model no longer needs to call finalize itself
- Tests: 2 new JS tests (auto-finalize at max phase; no-finalize at non-max)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-07 18:47:19 +02:00
marcopanandClaude Fable 5 ade6838182 docs: add efficiency-levers setup and current-run instructions
- PROJECT_STATE.md: section on three deployed levers (FK annotations, context-pack, recap v2)
  with setup instructions for new workspaces; psd pre-configured
- README.md: one-time setup for workspace (tht schema suggest-fks); note that levers 2+3 auto-activate
- Updated 'How to run' with explicit command and prereq checklist
- Last-updated timestamp: 2026-07-08

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 17:43:51 +02:00
marcopanandClaude Fable 5 e24b41b156 feat(opt): three efficiency levers for NL→SQL workflow
Lever 1: Join-graph via FK logics in annotations + suggest-fks command
  - TableAnnotation.foreign_keys field stores curated logical FKs (DWH has no FK constraints)
  - tht schema suggest-fks: mine from approved SQL, heuristics (time_key → dim_time),
    same-name discovery + explicit --assume flag for multi-owner PKs
  - mschema renders 【Foreign keys】 section populated; validation in merge.py
  - SKILL.md F4 now reads FKs from mschema-text, no custom data_time_key logic

Lever 2: Context-pack consolidation at kickoff (tht search pack)
  - Single embedding of question, reused for schema + evidence + solved searches
  - One command: tht search pack <question> --session <id> → retrieval_pack.md
  - Graceful degradation when Ollama/vector store unreachable (exit 0, empty sections)
  - SKILL.md F1 prescribes as first call; reduces model thinking turns via pre-retrieval

Lever 3: Phase-summary recap v2 auto-construction from session ledger
  - tht session show --json includes full decisions ledger
  - tht phase meta --json exports 'emits' (substantive decision types per phase)
  - Gate appends deterministic 【Decisioni registrate in questa fase】 section (appendLedgerSection)
  - Model authors only summary + checks; recap table comes from persisted state (exact by construction)
  - SKILL.md Disciplina 6: brief model output, gate fills the rest

Tests: 358 Python (including 10 FK + 3 pack + 1 session-ledger tests) + 111 JS gate tests, all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 17:43:08 +02:00
marcopanandClaude Fable 5 87e875bc81 feat(replay): extract reviewer_schema_linking gates + de-slug question fallback
extract.mjs now emits a schema-linking ui_request descriptor (columns rendered
from the model's suggested_columns, since the live catalog enrichment needs the
DWH) and parses the reviewer's "N tabelle" outcome; when neither the manifest
nor question.md is reachable, the NL question is approximated by de-slugging
the session id. replay.json regenerated from the 2026-07-06-175012 session
(18 answered gates, includes the F4 schema-linking gate).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:04:00 +02:00
marcopanandClaude Fable 5 34aeda006e perf(f1): cache-guard schema introspect + F1 toolbox in skill (~-5 min per session)
Transcript analysis (session 2026-07-06-175012, GLM 5.2) showed F1 at 567s:
182s wasted on a useless `tht schema introspect` (re-introspecting the remote
DWH although physical.yaml was already materialized) plus ~220s of model
thinking inflated by ~7 exploratory turns (--help/find/cat). The actual
searches cost ~15s; reviewer gates (~145s, untouched) are the quality contract.

- schema_cmd.py: introspect now exits 0 with "OK (cache)" in ~1s when
  physical.yaml exists; --refresh forces the real re-introspection.
  Deterministic cross-model guarantee, verified live on psd (163 tables, 1.2s).
- SKILL.md: F1 toolbox (only `tht search find` + `tht schema render`; no
  introspect/--help/filesystem browsing; batch all searches in one turn);
  F4 step 1 is render-only with a one-shot introspect fallback.
- tht-gate.js: `tht schema introspect ... --refresh` added to FORBIDDEN
  (maintenance stays shell-only, never in-session).
- tests: 4 new pytest cases (cache hit placement proven with fake credentials,
  refresh bypass, corrupt-catalog fall-through, render fallback message) and
  2 gate anti-bypass JS cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:02:45 +02:00
marcopanandClaude Opus 4.6 0cad04a8d8 docs(vector): add write-RPC migration instructions (existing_vector_hashes + upsert_vector_records)
The original doc only covered search_similar; the write functions also need
solved_question in their kind whitelist for the memory table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-07 15:15:55 +02:00
marcopanandClaude Fable 5 3d206bf3ef docs(plans): active-memory implementation plan (promotion gate + solved questions)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:38:25 +02:00
marcopanandClaude Fable 5 a26f16ad79 feat(vector): server-side kinds filter for search_similar (legacy fallback) + graceful solved-search degrade
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:23:28 +02:00
marcopanandClaude Fable 5 8d427a2ccb fix(gate): actionable recovery for mid-promotion failures; truthful solved-index copy; fast-follow notes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:07:00 +02:00
marcopanandClaude Fable 5 8af0552dcc docs(skill): prescribe solved-question recall in F4/F6/F7; refresh PROJECT_STATE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:54:42 +02:00
marcopanandClaude Fable 5 4c4099df09 fix(finalize): report the unchanged (no-upsert) solved-question case instead of staying silent
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:50:53 +02:00
marcopanandClaude Fable 5 2742a1fc42 feat(finalize): auto-index the question->SQL pair (best-effort, never blocks)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:48:02 +02:00
marcopanandClaude Fable 5 1f429b6b35 feat(cli): tht memory solved-index / solved-search (question->SQL exemplars)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:43:52 +02:00
marcopanandClaude Fable 5 c0324c127a feat(solved): solved_question vector kind + one-row upsert (D11 pattern)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:38:10 +02:00
marcopanandClaude Fable 5 ae89957175 docs(skill): prescribe the F8 memory-promotion gate; drop optional D11 notes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:34:04 +02:00
marcopanandClaude Fable 5 e25ba5126b feat(gate): reviewer_memory_promote — deterministic F8 memory-promotion gate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:30:35 +02:00
marcopanandClaude Fable 5 bf7e850e6b feat(memory): filter gate-declined candidates from promotion preview
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:26:59 +02:00
marcopanandClaude Fable 5 cadbdf177a feat(memory): memory_promoted/memory_promotion_declined decision types (F8 emits)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:22:07 +02:00
marcopanandClaude Fable 5 ccc4f4c9dd docs: refresh PROJECT_STATE.md (review gates v2, F4 curation, live E2E + spinner fix)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 11:39:54 +02:00
marcopanandClaude Fable 5 2410f01b34 fix(bridge): forward Pi agent_end so the spinner stops at workflow completion
The FE derived 'working' purely as activeSession && !pendingWidget, so the
final workflow turn — the only one that ends without a follow-up gate —
left the spinner on forever (observed live: 21592s after F8 approve).

- SessionBridge maps Pi's agent_end -> SSE system_event {event: agent_end}
- PiProcessManager notifies the client (info error + synthetic agent_end)
  when the child dies unexpectedly; expected teardowns stay silent
- sessionStore tracks agentActive (on: user entry/text_delta/ui_request,
  off: agent_end); AppShell working now requires it; resume sets it
  optimistically

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 10:14:43 +02:00
marcopanandClaude Fable 5 b59b57c4e6 fix(review-gates): green success badges, null-safe preview rows, optional section items
- Extract shared statusBadgeClass() helper (src/viewers/statusBadge.ts) so
  SqlViewer, CteResultViewer and PhaseSummaryViewer can't drift: ok/success/
  passed/promoted render green (--success), warn renders amber (--warning),
  error/failed stay destructive red, everything else stays neutral outline.
  Previously CteResultViewer/PhaseSummaryViewer mapped "ok"/"promoted" to the
  default badge variant, which is bg-primary (GSD red) — success states
  rendered red.
- enrich.js: buildCteResultV2 now falls back preview.rows to [] instead of
  null when last_test.preview_rows is missing (pre-upgrade ok records), and
  CteResultViewer reads result.preview?.rows?.length with a null-safe
  fallback so it degrades to the "No preview rows" empty state instead of
  crashing.
- PhaseSummaryViewer: section.items is optional (model-authored sections can
  be prose-only); render (section.items ?? []) instead of crashing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 01:37:35 +02:00
marcopan b089482b8d chore(replay): regenerated replay.json fixture (v2 artifact-gate payloads)
Regenerated demo fixture output with augment-schema-linking.mjs +
augment-review-gates.mjs applied, so the F6 cte_plan gate, CTE 1 cte_result
gate, and Fase 5 phase-summary gate carry v2 schema_version payloads and
render via CtePlanViewer/CteResultViewer/PhaseSummaryViewer in the replay
server. Regenerable; committed so the replay server serves it without a
build step.
2026-07-07 01:19:49 +02:00
marcopan 4042d0b44d feat(replay): fixtures + augment script for v2 artifact-gate viewers
Add cte-plan-fixture.json, cte-result-fixture.json, and
phase-summary-fixture.json — realistic Italian-language payloads for the
cardiology DWH replay session, conforming exactly to contracts.md (A/B/C).

Add augment-review-gates.mjs (pattern of augment-schema-linking.mjs): finds
the F6 cte_plan gate, first CTE 1 cte_result gate, and Fase 5 phase-summary
gate by title regex and swaps in the v2 artifact.data, so CtePlanViewer,
CteResultViewer, and PhaseSummaryViewer render in the offline replay.
Idempotent; exits 1 if an expected gate is missing.

Document the augment scripts and re-extract note in tools/replay/README.md.
2026-07-07 01:19:44 +02:00
marcopanandClaude Fable 5 1c97289ed8 feat(frontend): v2 artifact viewers for cte_plan/cte_result/phase
Add CtePlanViewer, CteResultViewer, and PhaseSummaryViewer to render the
structured v2 payloads (schema_version: 2) the harness now emits for
artifact-gate widgets, per contracts.md. Extract PreviewGrid from
ResultsPanel as a reusable AG Grid component shared by CteResultViewer.
ArtifactView dispatches to the new viewers on a schema_version/shape
guard, falling back to the existing legacy renderers unchanged for
older sessions and replay fixtures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:57:20 +02:00
marcopanandClaude Fable 5 3ad93cd02f feat(gate): deterministic v2 review-gate payloads (cte_plan/cte_result/phase)
The tht-gate.js reviewer_confirm now builds structured v2 artifacts before the
widget so the reviewer approves gate-derived data, not raw model text:

- new pure modules gate/artifact-contracts.js (soft validators, {ok,errors},
  legacy-passthrough) and gate/enrich.js (index/description enrichment,
  buildCteResultV2 fusing thin model data with `tht cte info`, phase enrichment)
- cte_plan v2: validate + enrich + persist via `tht cte plan --name … --doc -`
  (names derived from data.ctes[]); legacy `names` param kept as fallback
- cte_result v2: rebuild from `tht cte next`/`tht cte info` (sql + preview from
  the persisted test record); null/error last_test -> actionable textResult
- phase v2: soft-validate + fill phase from meta + catalog descriptions
- prepareReviewerArguments coerces artifact.data too (GLM double-stringify);
  legacy markdown strings pass through unchanged
- SKILL.md: Phase 6 cte_plan payload A + thin cte_result guidance; Discipline 6
  payload C example; Discipline 7 reworded for gate-rebuilt cte_result

Legacy (non-v2) paths unchanged. TypeBox stays Type.Any() for artifact.data;
validation is soft (textResult) so models self-correct instead of looping.
All 102 gate JS tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 00:42:48 +02:00
marcopan 9b4f6b9804 feat(harness): persist CTE preview rows, add tht cte info, --doc for cte plan
WS1 of review-gates-v2: gives the JS gate (WS2) deterministic data to build the
cte_result v2 payload.

- CteTestRecord gains optional preview_rows (JSON-coerced, truncated cells);
  test_cmd populates it from the bounded result rows.
- New read-only `tht cte info <name> --session <id> [--json]`: plan
  index/total, persisted .sql, cte_plan_doc.json entry (if any), last
  CteTestRecord. Exits 1 with a clean stderr message on missing
  session/plan/name/sql.
- `tht cte plan --doc -` validates a chain-doc JSON (ctes[].name must match
  --name, same order) and writes it to cte_plan_doc.json; cte_plan.json stays
  a plain list[str] (load-bearing for tht.phase.next_cte). --doc is optional.
2026-07-07 00:23:50 +02:00
marcopanandMarco Pancotti 7491e8c3ac chore(replay): regenerated replay.json fixture (phase tags + F4 schema-linking gate)
Regenerated demo fixture output of extract.mjs (WorkflowBar phase tags) with the
F4 gate-7 augment applied (tools/replay/augment-schema-linking.mjs, on the F4
branch). Regenerable; committed so the replay server serves it without a build step.
2026-07-06 23:29:49 +02:00
marcopanandMarco Pancotti 9403147c98 feat(replay): infer workflow phase from gate title so WorkflowBar lights the right dot
extract.mjs: inferPhase() maps gate titles ("F1 — …", "Fase 3 completata …") to F1..F8;
server.mjs: carries the inferred phase through the replayed ui_request stream.
2026-07-06 23:29:49 +02:00
e9b2934ef7 style(frontend): look&feel v2 refinements — 70% gate modal, structured cards, mono labels
- ArtifactGateWidget: gate modal 90%→70% of viewport, larger heading, more padding.
- ArtifactView StructuredValue: depth-aware hierarchy (top-level hairline dividers,
  warm bg-muted card surfaces + shadow for object-array items).
- index.css: .thot-label micro-labels switched to the mono register (SF Mono) with
  tightened tracking, distinguishing labels/meta from sans body prose.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:29:49 +02:00
9fe1c9321b feat(frontend): suggested-first ordering + filter box in the schema-linking columns modal
Wide tables (e.g. 116 columns, 3 suggested) made curation a scroll hunt. Now the
modal lists suggested columns first (stable sort on the descriptor flag, so
toggling never reorders rows) and adds a filter box matching name + description,
with a "shown/total" count. Selection stays keyed by column name, so the response
column order is unchanged (catalog order). Applies to read-only (excluded) tables too.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:29:21 +02:00
marcopanandMarco Pancotti d942635085 fix(f4): harden schema-columns enrichment + ValidationError guard + staged-helper note 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 3ed570ee60 feat(tht): promoted_columns_for helper (curated column set) 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti ad5a4d938c test(backend): pin schema-linking structured response passthrough 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti bd63e3194f docs(skill): Phase 4 uses reviewer_schema_linking; honor curated output columns 2026-07-06 23:25:00 +02:00
ad788278dd feat(gate): reviewer_schema_linking tool with per-column curation + deterministic sync
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 559f52e26e feat(gate): buildSchemaLinkingRequest descriptor builder 2026-07-06 23:25:00 +02:00
00365cc6e6 feat(tht): sync-schema-linking projects F4 ledger into schema_linking.json
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 1df5d40d05 feat(tht): column_promoted/column_excluded decision types (F4) 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 9339a8272b feat(tht): schema columns reader (name/description/type/pk) from catalog 2026-07-06 23:25:00 +02:00