feat(harness): Onda 0b — setup workspace per-cliente + build LSH (D14a validato)
Repo workspace per-cliente tht-workspace-psd/ (git separato): 35 evidence Thoth frontmatturate (da etl/docs/evidence/, non tutto etl/docs), psd.yaml con path assoluti ancorati al repo, THT_DOCS_ROOT -> radice repo cliente. README documeta il deployment shape (2 checkout + .env). LSH build sul DWH reale (REST, VPN): 75737 valori / 491 colonne eligible -> indice 175M nel repo cliente. tht lsh query 'ablazione' -> 8 colonne (D14a non-collapsing validato end-to-end: procedure_type, descrizione_procedura, intervento, ...). Correzioni al piano eseguite durante l'implementazione: - aggiunto 'tht schema introspect' (prerequisito di lsh build, omesso nel piano) - uso -c (il cmd ha --config, non --workspace residuo) - path assoluti nel psd.yaml (paths sono relativi alla CWD, non al file YAML) - evidence corretta: solo etl/docs/evidence/ (35), non tutto etl/docs (895) Test L2 riallineato: WORKSPACE -> psd.yaml nel repo cliente (non tht-test.yaml), index_dir -> paths.indexes/'lsh', nome -> db_schema (non letterale 'datawarehouse'). Bug latente del test (path mismatch) mai emerso prima: ora PASS invece di SKIP. Verifica: pytest 165 passed, 5 deselected; pytest -m l2 test_value_grounding_real PASS.
This commit is contained in:
@@ -33,6 +33,33 @@ Keys are never logged; URLs are fine. Rotate any key that appeared in chat.
|
||||
A workspace wires the relational DWH + the pgvector (dual-key) + embeddings + evidence.
|
||||
See `workspaces/tht.example.yaml`. `${THT_*}}` tokens expand from `.env`.
|
||||
|
||||
### Per-customer workspace repo (deployment shape)
|
||||
|
||||
ThothII is generic; **evidence content and LSH indexes are per-customer** and live in a
|
||||
**separate repo**, not copied into `harness/`. The deployment is two checkouts + one `.env`:
|
||||
|
||||
```
|
||||
ThothII/ generic harness (this repo)
|
||||
└── harness/ tht CLI + .env (THT_* secrets, points at the customer repo)
|
||||
|
||||
tht-workspace-<customer>/ per-customer workspace repo (separate git repo)
|
||||
├── <customer>.yaml workspace YAML (paths ABSOLUTE, anchored here)
|
||||
├── evidence/ curated evidence markdown (frontmatter-structured, by domain)
|
||||
├── artifacts/ derived artifacts (gitignored; regenerated by tht)
|
||||
├── indexes/ LSH index (gitignored; regenerated by `tht lsh build`)
|
||||
└── sessions/ per-session artifacts (gitignored)
|
||||
```
|
||||
|
||||
`THT_DOCS_ROOT` in `.env` points at the **root** of the customer repo
|
||||
(`evidence.source_root` + `evidence.evidence_dir` resolve the curated folder).
|
||||
All `tht` commands take the customer workspace via `-c`:
|
||||
|
||||
```bash
|
||||
tht schema introspect -c /path/to/tht-workspace-<customer>/<customer>.yaml
|
||||
tht lsh build -c /path/to/tht-workspace-<customer>/<customer>.yaml
|
||||
tht session new -c /path/to/tht-workspace-<customer>/<customer>.yaml
|
||||
```
|
||||
|
||||
## The workflow
|
||||
|
||||
`workflow.yaml` is the **single source of workflow truth** (spec F2). Eight phases
|
||||
|
||||
@@ -13,13 +13,17 @@ import pytest
|
||||
from tht.workspace import load_workspace
|
||||
|
||||
pytestmark = [pytest.mark.l2]
|
||||
WORKSPACE = Path(__file__).resolve().parents[2] / "workspaces" / "tht-test.yaml"
|
||||
# Per-customer workspace repo (spec D7): the LSH index lives in the customer repo,
|
||||
# not in harness/. PSD is the reference customer for L2. Skip cleanly if absent.
|
||||
WORKSPACE = Path("/Users/mp/projects/tht-workspace-psd/psd.yaml")
|
||||
|
||||
|
||||
def test_ablazione_returns_multiple_columns(l2_env):
|
||||
"""On the real schema, 'ablazione' should ground to more than one column (e.g.
|
||||
a flag and a free-text patologia field) -- the whole point of D14a's
|
||||
non-collapsing aggregation. Requires a built LSH index (tht lsh build)."""
|
||||
if not WORKSPACE.exists():
|
||||
pytest.skip(f"per-customer workspace repo not found: {WORKSPACE} (clone tht-workspace-psd)")
|
||||
from tht.config import LshConfig
|
||||
try:
|
||||
from tht.lshindex import load_index, query_index # ported with the lsh build path
|
||||
@@ -27,14 +31,15 @@ def test_ablazione_returns_multiple_columns(l2_env):
|
||||
pytest.skip("tht.lshindex not yet ported (deferred from B3; lands with tht lsh build)")
|
||||
from tht.search import aggregate_lsh_multi
|
||||
|
||||
# NOTE: this test assumes the LSH index was built (tht lsh build --workspace
|
||||
# tht-test). If absent, build it first. The index path comes from the config.
|
||||
# The index lives at <indexes>/lsh/<schema>_*.pkl (same convention as tht lsh build/query),
|
||||
# and is keyed by the database schema name (cfg.database.db_schema), not a hard-coded literal.
|
||||
ws = load_workspace(WORKSPACE)
|
||||
index_dir = ws.paths.indexes
|
||||
index_dir = ws.paths.indexes / "lsh"
|
||||
schema_name = ws.database.db_schema
|
||||
try:
|
||||
lsh, minhashes, meta = load_index(index_dir, "datawarehouse")
|
||||
lsh, minhashes, meta = load_index(index_dir, schema_name)
|
||||
except Exception as e:
|
||||
pytest.skip(f"LSH index not built yet (run tht lsh build): {e}")
|
||||
pytest.skip(f"LSH index not built yet (run tht lsh build -c {WORKSPACE}): {e}")
|
||||
|
||||
hits = query_index(lsh, minhashes, "ablazione", meta, top_n=20)
|
||||
grouped = aggregate_lsh_multi(
|
||||
|
||||
Reference in New Issue
Block a user