Commit Graph
80 Commits
Author SHA1 Message Date
marcopan a4acee4c70 feat(jobs): add resumable preprocessing envelope 2026-07-12 03:58:37 +02:00
marcopan 11e7ee9ea6 fix(corpus): harden canonical chunk and frontmatter invariants 2026-07-12 03:50:32 +02:00
marcopan 015715d092 feat(corpus): add deterministic normalization and chunking 2026-07-12 03:44:54 +02:00
marcopan d1fdf7d9f5 fix(evidence): bind HTTP validators to final URL 2026-07-12 03:38:11 +02:00
marcopan 81ff1810d1 fix(evidence): harden source acquisition 2026-07-12 03:32:18 +02:00
marcopan ffd683c587 feat(evidence): add filesystem and HTTP sources 2026-07-12 03:23:42 +02:00
marcopan 293d96e1a6 fix(evidence): close canonical contract gaps 2026-07-12 03:15:17 +02:00
marcopan 4424fd3d90 fix(evidence): harden canonical corpus contracts 2026-07-12 03:10:30 +02:00
marcopan d702aad93d feat(evidence): define source and corpus contracts 2026-07-12 03:02:10 +02:00
marcopan 6c67235caf fix(vector): close local pgvector final review 2026-07-12 02:48:10 +02:00
marcopan 015c496bda fix(vector): harden backup restore parity gates 2026-07-12 02:29:53 +02:00
marcopan e4db2ea5e1 docs(vector): add local backup restore and parity gate 2026-07-12 02:18:29 +02:00
marcopan 3588a7749b fix(vector): harden packaged migrations 2026-07-12 01:32:33 +02:00
marcopan c0d50e9b08 feat(vector): version pgvector schema 2026-07-12 01:21:29 +02:00
marcopan 409f806aae fix(vector): verify writer sequence privileges 2026-07-12 01:13:56 +02:00
marcopan e7e948c77c fix(vector): harden direct pgvector parity 2026-07-12 01:09:24 +02:00
marcopan b09341f07e feat(vector): add direct pgvector adapter 2026-07-12 01:01:15 +02:00
marcopan a0daa319ca fix(storage): contain logical corpus root 2026-07-11 21:26:47 +02:00
marcopan 7f3f6f7ce7 fix(storage): harden portable path diagnostics 2026-07-11 21:25:03 +02:00
marcopan eacb139e22 feat(storage): resolve portable workspace roots 2026-07-11 21:21:06 +02:00
marcopan 3e2ad7955c fix(adapter): close final foundation review 2026-07-11 21:14:32 +02:00
marcopan 45d57756aa fix(core): restore adapter command behavior 2026-07-11 21:04:53 +02:00
marcopan a35efa16de fix(core): preserve adapter command contracts 2026-07-11 21:00:50 +02:00
marcopan dbbab6d005 refactor(core): route integrations through adapter factory 2026-07-11 20:52:41 +02:00
marcopan 1e0911bb6a feat(config): add typed resource schema 2026-07-11 20:43:08 +02:00
marcopan fe8d70da47 fix(vector): separate write transport fields 2026-07-11 20:35:57 +02:00
marcopan ff4d662aba refactor(vector): define store contract 2026-07-11 20:24:14 +02:00
marcopan f6302b31dd fix(dwh): validate adapter limits strictly 2026-07-11 20:19:05 +02:00
marcopan 216984aac8 fix(dwh): align adapter sampling contract 2026-07-11 20:14:23 +02:00
marcopan 717e5ecced refactor(dwh): adapt direct and REST transports 2026-07-11 20:03:31 +02:00
marcopan a4eb6cc9e5 refactor(dwh): define adapter contract 2026-07-11 19:57:39 +02:00
marcopanandClaude Opus 4.6 1e4bc11418 fix(embed): fast-fail + auto-restart Ollama on solved-search hang
Embeddings timeout was 120s, causing multi-minute hangs when Ollama was
down during F4/F6/F7 solved-search. Now: connect_timeout=5s across all
HTTP clients (REST + Ollama), read_timeout reduced to 30s for embeddings,
and OllamaEmbeddings auto-restarts the server on ConnectionError before
degrading gracefully.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-07 19:49:55 +02:00
marcopanandClaude Fable 5 e24b41b156 feat(opt): three efficiency levers for NL→SQL workflow
Lever 1: Join-graph via FK logics in annotations + suggest-fks command
  - TableAnnotation.foreign_keys field stores curated logical FKs (DWH has no FK constraints)
  - tht schema suggest-fks: mine from approved SQL, heuristics (time_key → dim_time),
    same-name discovery + explicit --assume flag for multi-owner PKs
  - mschema renders 【Foreign keys】 section populated; validation in merge.py
  - SKILL.md F4 now reads FKs from mschema-text, no custom data_time_key logic

Lever 2: Context-pack consolidation at kickoff (tht search pack)
  - Single embedding of question, reused for schema + evidence + solved searches
  - One command: tht search pack <question> --session <id> → retrieval_pack.md
  - Graceful degradation when Ollama/vector store unreachable (exit 0, empty sections)
  - SKILL.md F1 prescribes as first call; reduces model thinking turns via pre-retrieval

Lever 3: Phase-summary recap v2 auto-construction from session ledger
  - tht session show --json includes full decisions ledger
  - tht phase meta --json exports 'emits' (substantive decision types per phase)
  - Gate appends deterministic 【Decisioni registrate in questa fase】 section (appendLedgerSection)
  - Model authors only summary + checks; recap table comes from persisted state (exact by construction)
  - SKILL.md Disciplina 6: brief model output, gate fills the rest

Tests: 358 Python (including 10 FK + 3 pack + 1 session-ledger tests) + 111 JS gate tests, all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 17:43:08 +02:00
marcopanandClaude Fable 5 34aeda006e perf(f1): cache-guard schema introspect + F1 toolbox in skill (~-5 min per session)
Transcript analysis (session 2026-07-06-175012, GLM 5.2) showed F1 at 567s:
182s wasted on a useless `tht schema introspect` (re-introspecting the remote
DWH although physical.yaml was already materialized) plus ~220s of model
thinking inflated by ~7 exploratory turns (--help/find/cat). The actual
searches cost ~15s; reviewer gates (~145s, untouched) are the quality contract.

- schema_cmd.py: introspect now exits 0 with "OK (cache)" in ~1s when
  physical.yaml exists; --refresh forces the real re-introspection.
  Deterministic cross-model guarantee, verified live on psd (163 tables, 1.2s).
- SKILL.md: F1 toolbox (only `tht search find` + `tht schema render`; no
  introspect/--help/filesystem browsing; batch all searches in one turn);
  F4 step 1 is render-only with a one-shot introspect fallback.
- tht-gate.js: `tht schema introspect ... --refresh` added to FORBIDDEN
  (maintenance stays shell-only, never in-session).
- tests: 4 new pytest cases (cache hit placement proven with fake credentials,
  refresh bypass, corrupt-catalog fall-through, render fallback message) and
  2 gate anti-bypass JS cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 16:02:45 +02:00
marcopanandClaude Fable 5 a26f16ad79 feat(vector): server-side kinds filter for search_similar (legacy fallback) + graceful solved-search degrade
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:23:28 +02:00
marcopanandClaude Fable 5 8d427a2ccb fix(gate): actionable recovery for mid-promotion failures; truthful solved-index copy; fast-follow notes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:07:00 +02:00
marcopanandClaude Fable 5 4c4099df09 fix(finalize): report the unchanged (no-upsert) solved-question case instead of staying silent
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:50:53 +02:00
marcopanandClaude Fable 5 2742a1fc42 feat(finalize): auto-index the question->SQL pair (best-effort, never blocks)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:48:02 +02:00
marcopanandClaude Fable 5 1f429b6b35 feat(cli): tht memory solved-index / solved-search (question->SQL exemplars)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:43:52 +02:00
marcopanandClaude Fable 5 c0324c127a feat(solved): solved_question vector kind + one-row upsert (D11 pattern)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:38:10 +02:00
marcopanandClaude Fable 5 bf7e850e6b feat(memory): filter gate-declined candidates from promotion preview
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:26:59 +02:00
marcopanandClaude Fable 5 cadbdf177a feat(memory): memory_promoted/memory_promotion_declined decision types (F8 emits)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:22:07 +02:00
marcopan 9b4f6b9804 feat(harness): persist CTE preview rows, add tht cte info, --doc for cte plan
WS1 of review-gates-v2: gives the JS gate (WS2) deterministic data to build the
cte_result v2 payload.

- CteTestRecord gains optional preview_rows (JSON-coerced, truncated cells);
  test_cmd populates it from the bounded result rows.
- New read-only `tht cte info <name> --session <id> [--json]`: plan
  index/total, persisted .sql, cte_plan_doc.json entry (if any), last
  CteTestRecord. Exits 1 with a clean stderr message on missing
  session/plan/name/sql.
- `tht cte plan --doc -` validates a chain-doc JSON (ctes[].name must match
  --name, same order) and writes it to cte_plan_doc.json; cte_plan.json stays
  a plain list[str] (load-bearing for tht.phase.next_cte). --doc is optional.
2026-07-07 00:23:50 +02:00
marcopanandMarco Pancotti d942635085 fix(f4): harden schema-columns enrichment + ValidationError guard + staged-helper note 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 3ed570ee60 feat(tht): promoted_columns_for helper (curated column set) 2026-07-06 23:25:00 +02:00
00365cc6e6 feat(tht): sync-schema-linking projects F4 ledger into schema_linking.json
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 1df5d40d05 feat(tht): column_promoted/column_excluded decision types (F4) 2026-07-06 23:25:00 +02:00
marcopanandMarco Pancotti 9339a8272b feat(tht): schema columns reader (name/description/type/pk) from catalog 2026-07-06 23:25:00 +02:00
marcopanandClaude Opus 4.8 00777922d8 feat(cli): 'tht session set-schema-linking' (file/stdin, validated)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 16:44:36 +02:00
marcopanandClaude Opus 4.8 92caaac18d feat(store): set_schema_linking validates then writes the F4 artifact
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 16:40:04 +02:00