Commit Graph
417 Commits
Author SHA1 Message Date
marcopan 7971d73628 feat(harness): pure widget-builder functions + JS golden/fuzzy tests (D2, L1)
The gate's widget-descriptor CONSTRUCTION, extracted into pure testable functions.
Each builder turns plain params into a ui_request descriptor (spec §4.1 taxonomy):
buildSelectRequest, buildMultiselectRequest, buildArtifactGate, buildInfoRequest,
buildFreetextRequest, withChildLinkage. No Pi context, no I/O -- the part of the
gate fully testable in L1 (in JS, in-language, no Python mirror).

Validation in the builders (not just happy-path): select requires title + array
options; multiselect allow_empty:false with zero options throws (a broken widget);
artifact-gate requires an artifact with a kind + a valid action.kind
(confirm/approve_reject/view_only); info level must be info/warning/error. The
Altro escape hatch with freetext linkage is always injected on blocking pick
widgets (no-limbo invariant).

L1: 14 node:test cases -- 3 golden files (select_F1, multiselect_F4,
artifact_gate_F5) pin the exact descriptor shape; fuzzy tests assert bad params
throw clearly rather than silently producing a broken widget.

package.json wires 'npm test' -> node --test (runs alongside pytest). The gate
GLUE (emission, anti-bypass, no-limbo loop) is C2, verified at L2.
2026-06-26 23:07:56 +02:00
marcopan e6eeb2ae3e test(harness): free-text rationale-capture contract (D13, §4.6)
D13 instructs the model to evaluate Altro/Rifiuta/steering free text in context,
act on it, re-ask if ambiguous, and record the user's words in the decision
rationale. The actual interpretation is model behavior enforced by the skill prose
+ gate, validated at L2; this test pins the RECORDING contract the gate relies on:
free text from 'Altro' round-trips into the decision rationale and survives
persistence, never silently discarded.

L1: test_freetext_interpretation (4 tests) -- Altro text preserved, persistence
roundtrip (exact), multiline steering intact, empty rationale allowed.

The skill prose ('Interpretazione del testo libero') ports with the .pi/ skill
in Phase D.
2026-06-26 23:04:22 +02:00
marcopan f104c015a1 feat(harness): SQL formula evidence -- concept->formula units + retrieval (D14b)
New evidence/formula_store.py: ConceptFormula (concept, columns, sql, status,
sources) as a frontmatter-YAML + SQL-body unit, stored one-file-per-formula under
<formulas>/<slug>-<n>.sql.md. save_formula is append-only (competing drafts and
reviewed versions coexist); retrieve_formula(concept) returns all of them so the
gate can surface candidates and let the reviewer choose.

concept_formula_approved / concept_formula_rejected added to DecisionType
(records the reviewer's choice; approved formulas travel with schema-linking).

L1: test_formula (7 tests) -- retrieval by concept, save/reload roundtrip (SQL
body preserved, frontmatter well-formed), multiple formulas per concept, empty
on no-match / missing dir, decision-type existence, default draft status.

Deferred: --kind formula on nsp search (needs search_cmd porting) wires
retrieve_formula into the CLI; lands with the search command.
2026-06-26 23:03:29 +02:00
marcopan c0285e55a0 feat(harness): value grounding -- multi-column LSH + value_grounded (D14a)
Ports search/__init__.py (combined_search/RRF/aggregate) renamed psdwp3->nsp.

New aggregate_lsh_multi (the D14a deviation): groups LSH hits by table keeping
EVERY column where a value appears -- NOT collapsed to a single best column.
The old _aggregate_lsh hid alternative groundings (e.g. 'ablazione' matching both
a boolean flag and a free-text patologia field). aggregate_lsh_multi exposes all
columns so the value-grounding widget lets the reviewer choose the anchor(s).
Within one (table, column) the best-scored value is kept; columns ordered by score.

value_grounded added to DecisionType (records the reviewer's anchor choice).

L1: test_value_grounding (6 tests) -- multi-column exposure, grouping, within-column
best-value, ordering, empty, and the value_grounded decision-type existence.

Deferred: lshindex/ (needs vendor/thoth_lsh) and the L0 test_rrf.py land with the
nsp lsh build command + index-building path; not needed for the pure L1 core here.
2026-06-26 23:02:11 +02:00
marcopan 9fa1e3e498 feat(harness): nsp memory save-one core — targeted upsert via writer key (D11)
The D11 deviation is a single-row pgvector upsert, not a full vectorstore resync.
Ports memory.py + session/{store,artifacts} + textutil (deps of memory), renamed
psdwp3->nsp. Decision import paths rewired from nsp.session.decisions to nsp.decisions
(our A4 port lives at the top level). session/models.py left UNCHANGED to preserve
the Phase-A ThothII additions (D12/D15 author/summary, D14a grounded_values,
D14b concept_formulas).

New in memory.py:
- memory_vector_record_for_decision(records, decision_seq): the single VectorRecord
  for a chosen decision (reuses memory_vector_records, filtered to one).
- save_one_memory(records, decision_seq, writer, embedder): embeds one record and
  calls writer.upsert_records('memory', [row]) -- NEVER writer.sync (that's the
  full-resync, server-side-only path). Returns the upsert count.

L1: test_memory_save_one (5 tests) pins the contract -- single row, one upsert
call, sync never called, None/0 for unknown seq.

Deferred: the full nsp memory save-one CLI command (config/session loading + the
workstation write-guard) lands when memory_cmd.py is ported alongside the other
CLI commands. The pure D11 core is what L1 can honestly cover here.
2026-06-26 23:00:06 +02:00
marcopan 796a39d893 feat(harness): port vectorstore dual-key + reader RPC (D11, §5.4)
Ports vectorstore/{rest_client,rest_writer,store,reader,embeddings,records},
evidence/model (leaf dep of records), and cli/_guards (require_vector_write_allowed
workstation write-guard). Renamed psdwp3->nsp, verbatim.

VectorRestClient gains an api_key property so reader/writer clients carry their
distinct keys visibly (spec D11: vector_reader / vector_writer on the same endpoint).

scripts/create_vector_reader_rpc.sql is NEW: the reader RPCs (search_similar,
list_tables) lived server-side in Supabase and were never versioned. Authored now
mirroring the writer allowlist pattern (table allowlist, security definer, revoke
from anon/authenticated, grant to vector_reader only). Writer RPC ported verbatim.

L1: test_vector_dual_key (7 tests) pins the dual-key construction + the workstation
write-guard (exit 4 without writer key).
2026-06-26 22:55:40 +02:00
marcopan eb3bde90e2 test(harness): L0 testcontainers + L1 contract tests for ported db/mschema/rest (A9, spec §1)
Ports the leaf data-layer modules and validates them:
- mschema/ (models, eligibility, merge, render), db/ (connection, sampling,
  introspect, fetch_ca), rest/client.py -- renamed psdwp3->nsp, verbatim.
- L0 (testcontainers, real Postgres): db connection read-only enforcement
  (psd_ro cannot CREATE/INSERT), introspect against a known schema (tables,
  columns, types, comments, FKs, enum, composite PK), sampling most-frequent
  values + truncation reporting. 15 tests, ~4s.
- L1 (fake data): rest/client RPC contract (mocked transport -- X-API-Key
  header, payloads, base_url slash handling, HTTP/network error surfacing),
  mschema/render 3 formats (markdown, mschema-text, schema-dict) +
  eligibility rules (wide_text excluded, short_text/numeric/enum/temporal/
  boolean eligible, annotation override wins). 25 tests.

pyproject registers l0/l2 markers + addopts '-m not l2' (L2 opt-in).

Deferred to their dependency-porting tasks: test_rrf.py (search needs
vectorstore, B3) and the 11 CLI contract tests (need _guards/session, wired
when each command lands). 'Not assumed reliable' now has real teeth for the
data layer; CLI/search contracts follow.
2026-06-26 22:53:08 +02:00
marcopan 5f24bd1adc feat(harness): port CLI skeleton + nsp phase meta --json (F2, kills JS/Python drift)
nsp.cli app registers phase_app (the gate's workflow-fact source). phase meta
--json emits {schema_version, max_phase, phases:[{num,id,name,advance,artifacts_out}]}
from load_workflow() -- the single source of truth. F8/datamart is present (the
exact JS-drift bug in ChironeWp3, PHASE_NAMES truncated to 7, is structurally gone).

Scope: the 12 other command groups land in their porting tasks (A9 ports
db/mschema/_guards; B1 vectorstore; B3 search/lshindex). Eager-importing them now
would break the app on unported deps -- deferred to keep the suite green at each commit.
2026-06-26 22:47:20 +02:00
marcopan a579cb88e5 feat(harness): taskdoc per-step generator with byte budget (D16)
Genera un documento compatto per fase, derivato da artefatti + effective_decisions,
con byte budget enforced (target <20k token per un 35B/<200k). Mai incorpora
physical.yaml (~190k token, fatale). D15+D16 complementari: il brief delle decisioni
e' effective-aware, quindi post-rollback riflette lo stato corretto (le stale di
fasi > current_phase sono escluse).
6 tests (question+schema, no physical.yaml, budget ok/violato, header fase,
stale-excluded post-rollback). 32 total passing.
2026-06-26 22:38:24 +02:00
marcopan 4b584934c8 feat(harness): teardown_to_phase -- artifact teardown on rollback (D15)
Cancella ogni artefatto la cui fase produttrice > target, usando artifacts_out di
workflow.yaml. Risolve il bug latente di ChironeWp3: ctes/*.sql orfani (non piu'
nel piano dopo un re-derive) restavano su disco e bloccavano finalize.
Da chiamare insieme all'append di phase_reopened per mantenere stato coerente.
6 tests (target 4/1/7, empty, missing, orfani CTE). 26 total passing.
2026-06-26 22:37:18 +02:00
marcopan 7d0562b82d feat(harness): phase.py rewrite + effective_decisions (D15 core fix, F2)
The single most important architectural fix vs ChironeWp3: ALL helpers consult
effective_decisions() instead of raw list_decisions(), so the reopen-aware view is
consistent everywhere (fixes the bug where approved_ctes/advance_problems conflated
stale pre-reopen decisions with new ones).

Model (corrected during TDD):
- current_phase folds the audit (excluding retracted) with guard 'n == cur' --
  already reopen-aware (old phase_approved:N after reopen to M<N don't advance).
- effective_decisions = decisions whose phase <= current_phase. A sql_approved at
  phase 7 is stale when current_phase=4 after a rollback to F4, even if in the ledger.
  Rollback to F4 does NOT invalidate decisions of phases 1-3 (they stay effective).
- decision_retracted markers excluded (audit-only).

Also: session/models.py ported (SchemaLinking + Candidate with grounded_values D14a
+ concept_formulas D14b). MAX_PHASE/PHASE_NAMES read from workflow.yaml via
load_workflow() (no duplication). Strada 2: ladder if-phase-N kept for now,
generic prerequisites evaluator (F2 full) deferred.

7 phase tests + 20 total passing.
2026-06-26 22:35:43 +02:00
marcopan fbcd694ace feat(harness): decisions.py ported + decision_retracted for step rollback (D15)
- ported from ChironeWp3 (22 DecisionType, append-only jsonl, monotonic seq)
- added decision_retracted type + retracts field for step-level rollback (D15):
  the retracted decision stays in the audit log, effective_decisions() (Task A5)
  will exclude it from the active view
- 5 tests: retract marker + monotonic seq + retracts default + empty session +
  literal includes retracted. All 13 harness tests pass.
2026-06-26 22:30:16 +02:00
marcopan 4a1272fa0a feat(harness): workflow.yaml as single source of truth + workflow.py loader (F2)
- workflow.yaml: 8-phase definition, data-driven prerequisites, no hardcoded ladder
- workflow.py: load_workflow() reader; max_phase=len(phases), decision_min_phase
  derived from prerequisites scan (no duplication)
- 6 tests: phase count, decision_min_phase derivation, name lookup (incl. F8
  presence -- the JS drift bug structurally impossible now), artifacts_out,
  advance strategy, schema_version. All 8 harness tests pass.
2026-06-26 22:29:17 +02:00
marcopan 2e34ba33e0 docs(spec): allinea §5.1 workspace YAML al Config reale (decisione B, post Task A2)
La prima stesura usava una struttura 'ideale' (relational/vector_db.collection/
embeddings.provider) che non combaciava col modello Config portato da ChironeWp3.
Allineato alla struttura reale (database/rest/vector_rest/vector_write_rest/
vector_db top-level). Aggiunta nota di allineamento + modello delle key D11.
2026-06-26 22:27:05 +02:00
marcopan 6ba3336775 feat(harness): config.py + workspace.py (D3 confine) + chirone.example.yaml
- config.py ported from ChironeWp3, PSD_PROFILE -> THOTH_PROFILE; dual vector key
  (vector_rest/vector_write_rest top-level) preserved verbatim
- workspace.py: thin boundary wrapper (D3) around load_config
- workspaces/chirone.example.yaml: aligned to the real Config shape
- tests/test_workspace.py: 2 tests (env expand + dual key; missing env raises)
- .env.example: added DWH + DOCS_ROOT vars
All tests pass (2/2).
2026-06-26 22:25:00 +02:00
marcopan 72a2c060d4 feat(harness): scaffold project (pyproject, env, gitignore) 2026-06-26 22:17:10 +02:00
marcopan 561e7aef08 chore: init ThothII repo — gitignore references, baseline docs (PRD, spec, harness plan) 2026-06-26 22:00:43 +02:00