Commit Graph
39 Commits
Author SHA1 Message Date
marcopan 7f3f6f7ce7 fix(storage): harden portable path diagnostics 2026-07-11 21:25:03 +02:00
marcopan eacb139e22 feat(storage): resolve portable workspace roots 2026-07-11 21:21:06 +02:00
marcopan 45d57756aa fix(core): restore adapter command behavior 2026-07-11 21:04:53 +02:00
marcopan a35efa16de fix(core): preserve adapter command contracts 2026-07-11 21:00:50 +02:00
marcopan dbbab6d005 refactor(core): route integrations through adapter factory 2026-07-11 20:52:41 +02:00
marcopanandClaude Fable 5 e24b41b156 feat(opt): three efficiency levers for NL→SQL workflow
Lever 1: Join-graph via FK logics in annotations + suggest-fks command
  - TableAnnotation.foreign_keys field stores curated logical FKs (DWH has no FK constraints)
  - tht schema suggest-fks: mine from approved SQL, heuristics (time_key → dim_time),
    same-name discovery + explicit --assume flag for multi-owner PKs
  - mschema renders 【Foreign keys】 section populated; validation in merge.py
  - SKILL.md F4 now reads FKs from mschema-text, no custom data_time_key logic

Lever 2: Context-pack consolidation at kickoff (tht search pack)
  - Single embedding of question, reused for schema + evidence + solved searches
  - One command: tht search pack <question> --session <id> → retrieval_pack.md
  - Graceful degradation when Ollama/vector store unreachable (exit 0, empty sections)
  - SKILL.md F1 prescribes as first call; reduces model thinking turns via pre-retrieval

Lever 3: Phase-summary recap v2 auto-construction from session ledger
  - tht session show --json includes full decisions ledger
  - tht phase meta --json exports 'emits' (substantive decision types per phase)
  - Gate appends deterministic 【Decisioni registrate in questa fase】 section (appendLedgerSection)
  - Model authors only summary + checks; recap table comes from persisted state (exact by construction)
  - SKILL.md Disciplina 6: brief model output, gate fills the rest

Tests: 358 Python (including 10 FK + 3 pack + 1 session-ledger tests) + 111 JS gate tests, all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 17:43:08 +02:00
marcopanandClaude Fable 5 34aeda006e perf(f1): cache-guard schema introspect + F1 toolbox in skill (~-5 min per session)
Transcript analysis (session 2026-07-06-175012, GLM 5.2) showed F1 at 567s:
182s wasted on a useless `tht schema introspect` (re-introspecting the remote
DWH although physical.yaml was already materialized) plus ~220s of model
thinking inflated by ~7 exploratory turns (--help/find/cat). The actual
searches cost ~15s; reviewer gates (~145s, untouched) are the quality contract.

- schema_cmd.py: introspect now exits 0 with "OK (cache)" in ~1s when
  physical.yaml exists; --refresh forces the real re-introspection.
  Deterministic cross-model guarantee, verified live on psd (163 tables, 1.2s).
- SKILL.md: F1 toolbox (only `tht search find` + `tht schema render`; no
  introspect/--help/filesystem browsing; batch all searches in one turn);
  F4 step 1 is render-only with a one-shot introspect fallback.
- tht-gate.js: `tht schema introspect ... --refresh` added to FORBIDDEN
  (maintenance stays shell-only, never in-session).
- tests: 4 new pytest cases (cache hit placement proven with fake credentials,
  refresh bypass, corrupt-catalog fall-through, render fallback message) and
  2 gate anti-bypass JS cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:02:45 +02:00
marcopanandClaude Fable 5 a26f16ad79 feat(vector): server-side kinds filter for search_similar (legacy fallback) + graceful solved-search degrade
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:23:28 +02:00
marcopanandClaude Fable 5 8d427a2ccb fix(gate): actionable recovery for mid-promotion failures; truthful solved-index copy; fast-follow notes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:07:00 +02:00
marcopanandClaude Fable 5 4c4099df09 fix(finalize): report the unchanged (no-upsert) solved-question case instead of staying silent
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:50:53 +02:00
marcopanandClaude Fable 5 2742a1fc42 feat(finalize): auto-index the question->SQL pair (best-effort, never blocks)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:48:02 +02:00
marcopanandClaude Fable 5 1f429b6b35 feat(cli): tht memory solved-index / solved-search (question->SQL exemplars)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:43:52 +02:00
marcopan 9b4f6b9804 feat(harness): persist CTE preview rows, add tht cte info, --doc for cte plan
WS1 of review-gates-v2: gives the JS gate (WS2) deterministic data to build the
cte_result v2 payload.

- CteTestRecord gains optional preview_rows (JSON-coerced, truncated cells);
  test_cmd populates it from the bounded result rows.
- New read-only `tht cte info <name> --session <id> [--json]`: plan
  index/total, persisted .sql, cte_plan_doc.json entry (if any), last
  CteTestRecord. Exits 1 with a clean stderr message on missing
  session/plan/name/sql.
- `tht cte plan --doc -` validates a chain-doc JSON (ctes[].name must match
  --name, same order) and writes it to cte_plan_doc.json; cte_plan.json stays
  a plain list[str] (load-bearing for tht.phase.next_cte). --doc is optional.
2026-07-07 00:23:50 +02:00
marcopanandMarco Pancotti d942635085 fix(f4): harden schema-columns enrichment + ValidationError guard + staged-helper note 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 3ed570ee60 feat(tht): promoted_columns_for helper (curated column set) 2026-07-06 23:25:00 +02:00
00365cc6e6 feat(tht): sync-schema-linking projects F4 ledger into schema_linking.json
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 9339a8272b feat(tht): schema columns reader (name/description/type/pk) from catalog 2026-07-06 23:25:00 +02:00
marcopanandClaude Opus 4.8 00777922d8 feat(cli): 'tht session set-schema-linking' (file/stdin, validated)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 16:44:36 +02:00
marcopanandClaude Opus 4.8 47a52170ce feat(cte): add 'tht cte next' — first unapproved plan CTE
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 16:29:52 +02:00
marcopanandClaude Opus 4.8 c12bdcd257 feat(harness): derive a 3-5 keyword session name (YAKE, no LLM)
New sessions get a concise Italian-keyword `name` instead of the truncated
question. `tht session new` (when no --name is given) derives it via a new
`_extract_name` helper using YAKE (pure-Python, unsupervised, Italian, no LLM),
dropping generic query verbs and keeping the top keywords in reading order;
falls back to `_summarize` if YAKE is unavailable. `create_session` core keeps
its `name=None` default — the policy lives at the CLI layer.

TDD: tests/test_session_name.py (unit + CliRunner integration). Full harness
suite 269 passed; ruff clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 15:54:54 +02:00
marcopan 7e19f81173 feat(harness): tht ollama ensure CLI command (hard-fail preflight) 2026-06-29 19:25:23 +02:00
marcopan 9c39ea129e fix(harness): ensure_ollama never raises on start/probe failure (stage server) 2026-06-29 19:19:26 +02:00
marcopan 5f5cac05d5 feat(harness): EmbeddingsConfig bin/start_cmd + ensure_ollama preflight orchestration 2026-06-29 19:15:28 +02:00
marcopan 03053b0716 feat(harness): tht session documents --json read command 2026-06-29 12:25:40 +02:00
marcopanandClaude Opus 4.8 26337d0e32 feat(harness): tht session set-name/set-group/archive/unarchive/delete commands
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 12:20:43 +02:00
marcopan 60fa665d73 feat(harness): manifest archived/group fields + session mutation helpers 2026-06-29 12:16:52 +02:00
marcopanandClaude Sonnet 4.6 d630cac3ab feat(task-10): SQL routes + /workspaces + /models + fix sql preview path bug
Harness:
- preview_cmd FILE positional arg made optional; when omitted with --session,
  path is derived via _session_sql_file (mirrors export_cmd) — fixes the
  deferred Task-5 bug where the backend passed sessions/<id>/sql_final.sql
  relative to harnessDir, which broke for workspace-dependent paths.
- New pytest: test_preview_session_no_file_resolves_sql_final

Backend:
- ThtRunner.sqlPreview: drop positional file arg; use --session only
- New routes/sql.ts: POST /sessions/:id/sql/preview + /export
- New routes/meta.ts: GET /workspaces (yaml scan) + GET /models (injectable
  seam + graceful fallback to {models:[]})
- app.ts: register sqlRoutes + metaRoutes; add listModels to BuildAppDeps
- tht-runner.test.ts: add sqlPreview argv assertion (no file path)
- test/routes-sql-meta.test.ts: 9 tests (sql preview/export + meta routes)

Tests: harness 233 passed; backend 29 passed; build clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 21:31:35 +02:00
marcopanandClaude Sonnet 4.6 5d9a0bb548 feat(harness): manifest provider/model/thinking/name + session new options (BE-6/7)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 20:29:40 +02:00
marcopanandClaude Sonnet 4.6 a10a872a99 feat(cli): tht session list --json + session show --json (Task 7)
- Add _list_sessions(sessions_root) pure helper: scans sessions dir,
  returns list[dict] with id/status/question/summary/created_at/
  updated_at/author, sorted by created_at desc.
- Add `tht session list` command: --json emits pristine JSON array,
  human mode prints one line per session.
- Add --json flag to `tht session show`: emits manifest
  (model_dump by_alias) + phase (current_phase) + has_schema_linking.
- Tests: 8 tests in test_session_list_json.py (TDD red→green).
- Full suite: 230 passed (--ignore=tests/l2).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 20:23:46 +02:00
marcopanandClaude Opus 4.8 85e2fdc00a fix(harness): correct truncated signalling for sql preview offset>0
When offset>0 the wrapper added an outer LIMIT N, so run_controlled's
_inject_limit bailed (a LIMIT IS present) and truncated was always False —
AGGrid could never detect more rows. Fix: for offset>0 probe with LIMIT (N+1)
OFFSET M, then compute truncated = len(rows) > N in do_run and slice back to N.
offset==0 path unchanged (delegates to extracted _run_transport helper). JSON
still reports the user's requested limit N and correct truncated. Adds 3 tests
exercising the real do_run offset>0 path (N+1 -> truncated True, N -> False,
offset==0 verbatim). 222/222 passing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 20:18:58 +02:00
marcopanandClaude Sonnet 4.6 7d9769cfff feat(harness): tht sql preview --json + --offset for AGGrid paging (BE-2)
Add inject_limit_offset (tht/execute/limit.py) — pure subquery wrapper that
applies LIMIT/OFFSET non-destructively without clobbering user-supplied LIMITs.
Wire offset param into do_run (pre-processing when offset>0) and add --offset /
--json flags to preview_cmd; JSON mode emits pristine stdout with columns, rows,
execution_ms, truncated, limit, offset. 5 new tests (4 unit + 1 JSON-purity),
219/219 total passing (no regressions).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 20:13:20 +02:00
marcopanandClaude Opus 4.8 c4d130828f fix(harness): remediation difetti review — gate↔CLI, D15, D7/D6, D14, robustezza
Implementazione del piano di remediation progressiva sui difetti emersi
dall'analisi dell'harness. Tutto verificato: 214 test Python (incl. L0 su
Postgres reale), 14 test JS del gate, ruff pulito.

Blocco 1 (CRITICA, integrazione gate↔CLI):
- phase advance: gate usa --auto + exit 6; reviewer_confirm kind:phase fa
  advance esplicito che applica i prerequisiti (prima non avanzava per le
  fasi a conferma umana).
- cte plan riceve i --name dal gate (param names); set-question con id
  posizionale; skill `tht search find`; nuovo comando `tht memory save-one`
  con dedup hash client-side in save_one_memory.

Blocco 2 (D15, stato post-rollback):
- campo `phase` su DecisionRecord + effective_decisions phase-aware per i
  subject "a nome" (cte_approved ecc.); _compute_promotions e finalize sulla
  vista effective; finalize confronta col piano CTE effettivo, non glob;
  `decision add --retracts` + comando `decision retract`.

Blocco 3 (D7 read-only + D6 manifest):
- assert_read_only su tutti e quattro i codepath (direct + REST);
- manifest author/summary/updated_at/updated_by/schema_version popolati +
  helper touch_manifest sulle mutazioni.

Blocco 4-5 (D14a/D14b):
- decision_min_phase data-driven via `emits:` in workflow.yaml;
- formula evidence: status auto, search_formulas, gruppo CLI `tht formula`,
  `search find --kind formula`, load_evidence_dir salta i .sql.md.

Blocco 6 (robustezza):
- taskdoc slice promoted_tables + bound enforced; report escaping/bound +
  rsplit note; filtro kind reader REST/direct; conteggio upserted robusto;
  guard REST run_query non-list; LSH disallineato -> LshIndexError.

Blocco 7 (pulizia):
- dead code gate e KIND_TO_TABLE morto rimossi; doc Postgres-only
  (README + connection.py).

Blocco 0 (parziale): test di compatibilità firma gate↔CLI
(tests/integration). Rinviati: fake-Pi runtime completo, artifact-gate da
disco (#23), parità eligibility REST/direct (#28), unificazione
reserved-labels (#30), memory_rejected da deselezione (#33).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 17:16:51 +02:00
marcopan daf33f77fe test(harness): report L2 cardioversione+ablazione + fix phase show (#2)
Prima sessione L2 end-to-end dopo il porting. Il loop skill->LLM->gate funziona nel
dominio (ricerche, evidence, quadro corretto su fact_cardioversione/fact_see_ablazione)
ma si blocca a F1 sul bug fatale #4 (ctx.sendRaw non esiste nel runtime Pi).

Bug emersi (4):
  #1 tht non nel PATH di Pi (basso, workaround wrapper)
  #2 phase show non passava config (medio, FIXATO: _cfg() risolve env+default)
  #3 session check signature inconsistente (basso, da verificare)
  #4 ctx.sendRaw is not a function (FATALE, mismatch architetturale: il porting ha
     sostituito i dialog nativi ctx.ui.* con ctx.sendRaw, API non esposta in questo Pi)

Analisi #4 (verificata sul runtime installato):
  - ctx.sendRaw non esiste; extension_ui_request e' emesso solo dal runtime
    (modes/rpc) come traduzione di ctx.ui.select/confirm/input, non come API extension
  - canali RPC per decisioni: enum chiuso select/confirm/input/editor (no custom)
  - ctx.ui.custom (multiselect TUI di ChironeWp3) e' no-op in RPC mode
  - conseguenza: multiselect F4 non ha canale in RPC -> decisione di design del gate
    aperta (opzioni A/B/C nel report, C=scartato), merita brainstorming dedicato

Fix #2 (dal modello in sessione, validato e ripulito): phase_cmd._cfg() ora risolve
THT_WORKSPACE/THT_CONFIG env poi fallback config/tht.yaml (stessa convenzione CONFIG_OPT).
Testato phase show OK, suite 165 passed.

Report: docs/l2-run-report-2026-06-27.md.
2026-06-27 16:14:46 +02:00
marcopan 91a374492c feat(harness): port sql/cte/datamart/lsh cmd (Onda 4) — CLI completa F1→F8
Ultima onda CLI. 4 cmd portati con rename + grep-per-file (3 residui nsp nei messaggi
fixati). Nessun drift costanti phase in questi cmd.

La CLI tht e' ora COMPLETA: 14 gruppi di comandi (phase config schema session vector
memory search evidence db decision sql cte datamart lsh). tht --help li list tutti.
Suite: 165 passed.

Il loop skill->LLM->gate ora ha tutti i comandi che la skill chiamera'. Resta:
skill riscritta (S), setup pre-sessione (0b), sessione L2 manuale.
2026-06-27 14:16:35 +02:00
marcopan 99b01b0407 feat(harness): port memory/search/evidence/db/decision cmd (Onda 3.2)
5 cmd foglia portati con rename + drift fix:
- memory_cmd: portato col modello registry INTATTO (TODO marker per il drop registry
  decisione spec 5 — task separato, richiede L2 per validare il rewrite su vectordb)
- search_cmd: creata search_app sub-app (era funzione standalone in ChironeWp3),
  registrata come 'tht search find'
- evidence_cmd, db_cmd, decision_cmd: port verbatim

Drift fix decision_cmd: DECISION_MIN_PHASE.get(type,1) -> load_workflow().decision_min_phase(type).
Check grep-per-file: ~15 residui nsp/PSD_SSL_CA nei messaggi utente fixati (nsp <cmd>
-> tht <cmd>, nsp.yaml -> workspace yaml, PSD_SSL_CA -> THT_SSL_CA).

Suite: 165 passed. tht --help ora mostra 10 sottocomandi.
2026-06-27 14:14:48 +02:00
marcopan 40b3bac6c6 feat(harness): port vector_cmd (Onda 2) — helper vector condivisi
Esporta make_embedder/open_store/open_searcher/require_vector_cfg usati da
memory/search/evidence cmd (Onda 3). 3 residui nsp nei messaggi fixati (grep-per-file).
2026-06-27 14:08:54 +02:00
marcopan 3120bdc192 feat(harness): port config_cmd + schema_cmd + session_cmd (Onda 1.3-1.4, radici CLI)
Le 3 radici intra-CLI, portate con rename psdwp3->tht + fix path import (session.phase
-> phase). Espongono gli helper condivisi: CONFIG_OPT (config_cmd), _load_config_or_exit/
physical_path/annotations_path (schema_cmd), session_dir/load_session_or_exit (session_cmd).

Drift phase fix in session_cmd (7 siti): MAX_PHASE -> wf.max_phase, PHASE_NAMES ->
wf.phase_name(), SCHEMA_LINKING_PHASE -> wf.schema_linking_phase(). Le vecchie costanti
non esistono piu' in tht.phase (sono metodi su Workflow dalla Onda -1/F2).

Check grep-per-file applicato: trovati e fixati 4 residui (config/nsp.yaml, 2x
'nsp schema introspect', 'Sessioni PsdWp3') che il sed psdwp3->tht non tocca. Lesson
del fix precedente applicata.

session_cmd importa sql_cmd lazy (dentro finalize_cmd) -> non si rompe finche' sql_cmd
(Onda 4) non sara' portato.

Suite: 161 passed. tht --help ora mostra phase, config, schema, session.
2026-06-27 14:08:08 +02:00
marcopan a312746fc8 feat(harness): require_phase_or_exit + phase advance/reopen/show (Onda 1.2)
require_phase_or_exit: guard riscritto vs Workflow (load_workflow().phase_name invece
della costante PHASE_NAMES drift). Exit 1 se la sessione e' sotto soglia. Usato da
cte/decision/datamart cmd.

Comandi phase (portati + adattati al modello ThothII, non copia cieca):
- advance: persiste phase_approved; --auto exit 6 se la fase non e' completa
  (contratto col gate)
- reopen: persiste phase_reopened + teardown_to_phase degli artefatti oltre il target
- show: stato sessione (fase corrente, ultime decisioni)

session_dir helper tenuto qui (mirror di session_cmd) per evitare circular import.
_cfg() fa fallback a THT_WORKSPACE env finche' _load_config_or_exit (Onda 1.4) non
sara' portato.

L1: 4 test require_phase_or_exit (allow at/above, exit below, message con nome fase
dal workflow). Suite: 161 passed.
2026-06-27 13:15:12 +02:00
marcopan fc5fbe6b65 refactor(harness): renaming prodotto tht (Onda -1)
Thoth (tht) è il prodotto, PSD è il cliente. Nessun riferimento al contesto
clinico nel codice.

Rinomine:
- comando+package nsp→tht (dir nsp/→tht/, 46 import, pyproject entry point)
- gate nsp-gate.js→tht-gate.js (+ rewrite token, relayIfNspFails→relayIfThtFails)
- workspace chirone.{example,test}.yaml→tht.{example,test}.yaml (generici)
- env THOTH_→THT_ (19 var) + NSP_ stragglers (NSP_HARNESS_ROOT, NSP_SESSION)
- commenti/docstring chirone/psdwp3/policlinico neutralizzati ('the reference
  implementation', 'the DWH')

Aggiunto [tool.setuptools.packages.find] include=['tht*'] (necessario: l'auto-
discovery rompeva con tht/ + workspaces/ come top-level multipli).

.env operatore aggiornato in-place (prefissi THT_, valori preservati, gitignored).

Verifica: pytest 109 passed, npm test 14 pass, tht phase meta --json OK, zero
residui nsp/THOTH_/NSP_/chirone nel package.
2026-06-27 10:33:16 +02:00