Author SHA1 Message Date
User 2f53512e4d docs: record server release and hand off remaining acceptance checks
Publish documentation / publish (push) Successful in 32s
2026-09-27 00:41:37 +02:00
Codex 497ab84031 docs: pin server handoff to released main revision
Publish documentation / publish (push) Successful in 33s
2026-09-26 16:42:36 +02:00
Codex 0d2e573e0d fix(ui): reset session view on stop and exit 2026-09-26 16:40:37 +02:00
Codex bd416f7327 Fix new-question landing and question-language HITL
Publish documentation / publish (push) Successful in 34s
Reset the activity panel when starting a new question so the landing navigation is restored. Detect and persist the original question language, pass it through runtime and widget descriptors, and scope HITL controls to that language.

Validated with gate, session, backend and frontend tests, TypeScript checks, Ruff and strict docs build. Rebuilt and restarted local core/frontend; both healthy and serving HTTP successfully.
2026-09-21 19:47:22 +02:00
Codex 23e52c80de Record verified Qwen documentation publication
Publish documentation / publish (push) Successful in 23s
2026-09-21 16:27:10 +02:00
Codex 84084bba37 Fix Qwen session tool calls and expose thinking compatibility
Publish documentation / publish (push) Successful in 30s
2026-09-21 16:23:51 +02:00
pinoricci1956 efd7d788d9 correzione scroller verticale pagina di configurazione catalogo 2026-09-16 11:59:51 +02:00
Codex b1c510a097 fix(docs): preserve theme assets in deny-by-default publication
Publish documentation / publish (push) Successful in 36s
2026-09-16 09:36:30 +02:00
Codex 5f3680a0fb docs: record verified live manual publication
Publish documentation / publish (push) Successful in 28s
2026-09-15 14:39:46 +02:00
Codex 4ff91e8d6e docs: consolidate historical records and verify public manual publication
Publish documentation / publish (push) Successful in 29s
2026-09-15 14:37:29 +02:00
Codex 5f3a7f5975 docs: record stale public site publication blocker
Publish documentation / publish (push) Successful in 23s
2026-09-15 10:28:49 +02:00
Codex 043ffdfad6 docs: separate public manual from internal project documentation
Publish documentation / publish (push) Successful in 27s
2026-09-15 10:26:35 +02:00
Codex 6a4634dcf1 Merge manual standalone installation documentation
Publish documentation / publish (push) Successful in 35s
2026-09-15 10:06:33 +02:00
Codex 84804be9f8 docs: publish bilingual manual standalone installation guides 2026-09-15 10:06:28 +02:00
User c3caba94dd fix(ui): expand session dialogs and repeat confirmation actions
Publish documentation / publish (push) Successful in 24s
2026-09-14 18:13:53 +02:00
User b1723c34c4 docs: record server rollout of session and memory fixes
Publish documentation / publish (push) Successful in 32s
2026-09-14 17:25:23 +02:00
User d6cdffea62 fix: keep embedded session controls visible and handle empty memory
Publish documentation / publish (push) Successful in 34s
Cap the embedded shell at its portal container height so steering and stop controls remain accessible. Skip vector retrieval for an empty authoritative Memory archive and compute SQL-rule embeddings lazily.

Validated with 54 Memory tests, 90 frontend tests, five browser scenarios, frontend and Docker builds, and a read-only comparison against the real empty Memory archive.
2026-09-14 17:15:00 +02:00
Codex 49333a2d35 Merge full and embedded shell, administration UI and server handoff
Publish documentation / publish (push) Successful in 1m21s
2026-09-14 15:08:47 +02:00
Codex b006b94479 docs: prepare server Codex deployment handoff for ThothII and Omics 2026-09-14 15:08:46 +02:00
Codex bdcd8fcd28 fix(ui): open one session accordion panel at a time 2026-09-14 01:09:45 +02:00
Codex cf90c1bd51 fix(ui): collapse session lists and scope selection controls to panels 2026-09-13 18:12:27 +02:00
Codex 571a4bcaa2 fix(ui): show workspace readiness dot and bounded session accordions 2026-09-13 17:39:33 +02:00
Codex 9051463654 docs: document full and embedded rendering with server authentication 2026-09-13 17:28:10 +02:00
Codex 26c5605ff7 fix(ui): unify Memory and Evidence reading typography 2026-09-13 17:07:37 +02:00
Codex 3535fda958 fix(ui): improve knowledge reading and add isolated formatting examples 2026-09-13 16:52:53 +02:00
Codex 2953f6b608 fix(ui): unify Session navigation and restore uniform tab borders 2026-09-13 16:22:20 +02:00
Codex 45db3a239b fix(ui): simplify login and suppress pointer focus ring on locale select 2026-09-13 15:57:57 +02:00
Codex 7d826e46c0 fix(ui): match Omics header and compact workspace layout 2026-09-13 15:36:08 +02:00
Codex 648434a32e docs: record approved Omics GitHub to PSD relay handoff 2026-09-13 15:22:30 +02:00
Codex 023b822f83 Merge visual review into full shell and preserve bilingual layout 2026-09-13 14:55:58 +02:00
Codex d8a29bfbdd Add full shell, replaceable Omics adapter and bilingual interaction
Implement approved specification #32 and tickets #33-#37. Keep host authentication server-verified and pin session interaction language. Compile scoped base selectors for browser compatibility and retain full gutters during CSS pruning.
2026-09-13 14:26:39 +02:00
Codex a59624a68f style: align global context panel and simplify selection copy 2026-09-13 10:43:42 +02:00
Codex 5af4408194 style: unify database block gutters and content alignment 2026-09-13 10:32:55 +02:00
Codex e088abd60a style: align catalog status with summary grid 2026-09-13 01:44:48 +02:00
Codex 803e9e9201 fix: remeasure composer after hidden Core becomes visible 2026-09-13 01:39:19 +02:00
Codex 8c81996896 style: place catalog status indicators after their labels 2026-09-13 01:23:46 +02:00
Codex eed398e569 style: restore prominent ThothII application wordmarks 2026-09-13 01:18:06 +02:00
Codex c8d276ddc6 style: unify workbench typography and prepare isolated visual review 2026-09-12 22:51:58 +02:00
Codex 2d1b714ebe fix: resolve admin issue review findings and record verification 2026-09-12 18:24:23 +02:00
Codex c7e5f295e6 fix: address administration layout and navigation issues #28 #29 #30 #31 2026-09-12 18:16:52 +02:00
Codex f52bf22e05 feat: establish unified administration and model context baseline 2026-09-12 18:03:15 +02:00
Codex 840344706f prototype: restore original Core within context shelf alternatives 2026-09-12 14:00:52 +02:00
Codex 41b9fed4d5 prototype: refine context shelf with remembered defaults and session tabs 2026-09-12 12:29:02 +02:00
Codex 36bf659ea9 prototype: explore global context with one operation at a time 2026-09-12 11:53:52 +02:00
Codex 4a67d60233 prototype: revisit five administration workflows after design review 2026-09-10 19:40:21 +02:00
Codex debb63d87b prototype unified administration page layouts 2026-09-10 16:48:25 +02:00
Codex 3943022a97 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	mkdocs.yml
2026-09-10 12:57:43 +02:00
Codex f5ec2d9313 docs: track security evidence and research notes 2026-09-10 12:53:13 +02:00
Codex 82e2c91f42 feat: implement memory and evidence administration with guided repairs
Publish documentation / publish (push) Successful in 1m27s
Add PostgreSQL-backed memory, editable evidence with source review and activation, and human-approved archive repairs across the harness, API, and UI. Include migrations, deployment support, regression coverage, and validation documentation.

Refresh permissions from validated session roles so existing administrator logins can access newly deployed archive management features.
2026-09-10 10:31:34 +02:00
User 8fe526dd6e fix(frontend): show session catalog in pi management 2026-09-08 14:58:18 +02:00
User e68e80a33d fix(frontend): prevent catalog header overlap 2026-09-08 14:35:36 +02:00
Codex 818563c408 fix(core): keep workspace runtime available 2026-09-08 13:37:10 +02:00
Codex 50c546e42d fix(frontend): accept catalog-owned workspace descriptors 2026-09-07 15:01:31 +02:00
Codex 651a5c7902 fix(frontend): isolate administration rail from portal CSS 2026-09-07 11:05:38 +02:00
535 changed files with 33696 additions and 9594 deletions
+5
View File
@@ -17,6 +17,7 @@ frontend/vite.database-management-prototype.config.ts
!deploy/env/*.env.example
deploy/thothii.env
deploy/secrets/
deploy/psd/
harness/workspaces/*.yaml
!harness/workspaces/local.yaml
!harness/workspaces/tht.example.yaml
@@ -29,3 +30,7 @@ coverage/
data/
sessions/
workspace-registry/
.tht/
deploy/local/
+20 -4
View File
@@ -7,6 +7,11 @@ on:
paths:
- "docs/**"
- "mkdocs.yml"
- "scripts/build-docs.sh"
- "scripts/verify-public-docs.py"
- "scripts/test-verify-public-docs.py"
- "scripts/verify-auth-docs.py"
- "scripts/test-verify-auth-docs.py"
- "docs/requirements.txt"
- ".gitea/workflows/publish-docs.yml"
workflow_dispatch:
@@ -34,14 +39,25 @@ jobs:
with:
python-version: "3.x"
cache: pip
cache-dependency-path: docs/requirements.txt
cache-dependency-path: docs/requirements.lock
- name: Install MkDocs dependencies
run: python -m pip install -r docs/requirements.txt
run: python -m pip install -r docs/requirements.lock
- name: Test public documentation boundary
run: python scripts/test-verify-public-docs.py
- name: Test current authentication documentation
run: |
python scripts/verify-auth-docs.py auth
python scripts/verify-auth-docs.py dwh
python scripts/test-verify-auth-docs.py auth
python scripts/test-verify-auth-docs.py dwh
- name: Build documentation
# Some documented source files intentionally live outside docs/.
run: mkdocs build
run: |
mkdocs build --strict
python scripts/verify-public-docs.py
- name: Publish generated site to the pages branch
working-directory: site
+1
View File
@@ -46,6 +46,7 @@ deploy/secrets/*
# Per-installation configuration generated by `tht setup` (examples stay tracked).
deploy/*/thothii-installation.yaml
deploy/*/operator.env
deploy/*/auth/
deploy/*/generated/
deploy/*/secrets/*
!deploy/*/secrets/.gitkeep
+12 -2
View File
@@ -98,8 +98,18 @@ frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness →
- **`tht`'s `-c`/`--config` is a PER-COMMAND option** — it must follow the subcommand, never
precede it (`ThtRunner.buildArgv` enforces this; prepending caused live 500s).
- **`--json` output must be pristine** (only valid JSON on stdout) — used as a machine contract.
- **UI strings are English; document *content* stays the workspace language** (Italian for
`psd`) because it's the real data. Only chrome/labels are English.
- **Localization:** deterministic UI uses the EN/IT catalogs with English fallback;
model interaction uses the session manifest's immutable `interaction_language`.
Workspace content, SQL, and identifiers remain unchanged. For shell modes, portal
integration, or translations, read `docs/operations/shell-and-localization.md`.
- **Server deployment:** for the coordinated ThothII/Omics upgrade, follow
`docs/operations/server-codex-handoff.md`; it supersedes earlier Omics delivery
instructions. Omics source integration uses GitHub with no repository relay prerequisite.
- **Server identity:** for portal login/logout, proxy headers, or Omics deploy, read
`docs/install/authentication-upstream.md` before changing authentication. Omics
uses embedded/upstream, not a second ThothII OIDC login. Full/embedded rendering
is documented in `docs/architecture/application-shell.md`; release acceptance
is in `docs/testing/authentication-manual-acceptance.md`.
- **Workspace schema v4** defines workspace identity and optional Evidence only. PostgreSQL Metadata
Catalog owns database identity, binding, schema, descriptions, sensitivity, and relationships;
embedding/model facts come from the installation catalog. The legacy `harness/workspaces/*.yaml` runtime snapshots still use
+147 -12
View File
@@ -91,21 +91,79 @@ correzione successiva crea una nuova sessione derivata, collegata a quella prece
dopo la finalizzazione. Non può modificare il ledger, gli artifact canonici o lo stato
terminale della sessione.
## Memory
**Memory Module** — Il modulo che possiede le conoscenze ed esperienze curate per
migliorare schema linking e generazione SQL di domande future. Le Memory appartengono
a un workspace e rimangono distinte dalle Evidence.
**Memory Card** — L'unità di contenuto gestibile del Memory Module, con identità,
ambito di applicazione e provenienza. Il formato è allineato per analogia alle
Evidence, senza implicare la stessa origine o lo stesso percorso di pubblicazione.
**Reusable Memory** — Una Memory Card che esprime un chiarimento di dominio, una
regola di costruzione SQL o un errore da evitare con motivo compreso e approvato.
La sua validità è circoscritta a un ambito esplicito e non deriva dalla sola
approvazione di una scelta occasionale in una domanda.
**Solved Question** — Una Memory Card che conserva una domanda risolta con la
relativa soluzione SQL e il contesto necessario a interpretarla. È un exemplar
consultativo: i parametri e le scelte del caso non diventano regole generali.
**Memory Graph** — L'insieme dei collegamenti espliciti fra card che contribuisce
al recupero di conoscenze pertinenti oltre alla somiglianza del contenuto. Il
ritrovamento di una card tramite un collegamento non ne implica l'approvazione.
**Memory Link** — Un collegamento curato fra card, con destinazione e significato
espliciti, che contribuisce alla consultazione di contenuti pertinenti. La sua
rimozione non comporta la cancellazione delle card collegate.
## Evidence
**Context specialist** — La persona competente sul dominio che redige e cura il
contenuto delle Evidence. Può essere distinta da chi amministra l'installazione;
il suo lavoro di redazione non richiede accesso al database applicativo.
**Evidence draft** — Il documento iniziale scritto dallo specialista di contesto,
che il sistema acquisisce e raffina in Evidence Unit. Può essere redatto e
consegnato indipendentemente dall'installazione che userà le Evidence risultanti.
**Evidence Module** — Il modulo autonomo che possiede la preparazione delle Evidence e
la loro consultazione durante il workflow. La preparazione avviene fuori dalle singole
sessioni; il workflow usa soltanto contenuti già pubblicati. A runtime contribuisce agli
stage semantici esistenti, senza diventare uno stage visibile e senza modificare ledger,
artifact o stato del workflow.
**Source Evidence** — Un documento originale del workspace, conservato senza modifiche
come riferimento umano e origine della successiva ristrutturazione.
**Source Evidence** — Il documento o la dichiarazione che sostiene il contenuto
corrente di una Evidence Unit. Un documento acquisito viene conservato come
riferimento umano; una dichiarazione manuale attribuisce il contenuto alla persona
che lo ha scritto e approvato.
**Manual Evidence declaration** — Una dichiarazione esplicita dell'amministratore
che sostiene una Evidence creata direttamente o una correzione del suo significato.
Non implica una verifica indipendente da parte di una fonte documentale esterna.
**Evidence origin** — Il documento da cui una Evidence Unit è stata inizialmente
derivata. Può restare collegato per provenienza e confronto con gli aggiornamenti
anche quando una dichiarazione manuale sostiene il testo corrente. La sola origine
non dimostra il supporto semantico di una successiva correzione.
**Local Evidence archive** — L'insieme delle Evidence curate custodite
dall'installazione, distinto dalle draft originali e dai contenuti derivati per
la ricerca. Comprende le correzioni manuali e i ritiri deliberati.
**Consolidated Evidence** — Una versione delle Evidence locali controllata come
insieme coerente e pronta per l'attivazione. I file ancora in modifica non ne
cambiano il contenuto.
**Active Evidence** — La versione consolidata disponibile alla consultazione del
core. Un tentativo di aggiornamento fallito conserva la versione attiva precedente.
**Evidence Unit** — La più piccola unità semantica coerente, revisionabile e ricercabile
derivata da una sola Source Evidence. Possiede un identificatore stabile indipendente
dal kind, assegnato una volta nella forma `evidence:<slug>`; fonti diverse non vengono
fuse automaticamente.
fondata su una Source Evidence corrente, anche manuale, e con eventuale origine
documentale distinta. Possiede un identificatore stabile indipendente dal kind,
assegnato una volta nella forma `evidence:<slug>`; fonti diverse non vengono fuse
automaticamente.
**Evidence kind** — La categoria semantica di una Evidence Unit, che ne determina i
campi specifici e ne orienta l'uso. Ogni unità ha un solo kind primario; i tipi iniziali
@@ -151,10 +209,9 @@ avanzare fino a un retry riuscito.
nella sessione: stage semantico, purpose, generazione interrogata e identificatori delle
Evidence restituite. Non duplica il contenuto delle Evidence.
**Curated Evidence** — Una o più Evidence Unit ristrutturate a partire da una Source
Evidence e conservate nel repository del workspace come proposte per la revisione
umana. Git conserva la versione precedente e rende visibile ogni modifica; una Curated
Evidence non è ancora contenuto autorevole del runtime.
**Curated Evidence** — Una o più Evidence Unit preparate da documenti o curate
manualmente. La presenza nell'archivio curato non implica da sola che il contenuto
sia già attivo per il workflow.
**Published Evidence** — Le Curated Evidence valide appartenenti alla revisione attiva
del workspace e alla generazione Evidence pubblicata. L'approvazione umana precede
@@ -176,9 +233,17 @@ una Evidence Unit. Il sistema ne verifica deterministicamente la presenza dopo l
normalizzazione meccanica; il curatore resta responsabile di verificarne la sufficienza
semantica.
**Evidence resolution** — L'operazione esplicita con cui un curatore ritira una
Evidence Unit oppure la ricollega a un Source Evidence esistente. Aggiorna documento e
manifest insieme, lascia un diff Git revisionabile e non pubblica né crea commit.
**Evidence resolution** — La decisione esplicita con cui un curatore risolve un
problema di una Evidence Unit, correggendola, ritirandola oppure ricollegandola a
una fonte adeguata.
**Source update conflict** — Un contrasto fra una fonte aggiornata e una correzione
manuale già approvata. La correzione resta in uso fino alla risoluzione esplicita
del confronto da parte dell'amministratore.
**Evidence source refresh** — La riacquisizione delle fonti esterne richiesta
dall'amministratore per rilevarne le modifiche. Fra due aggiornamenti il contenuto
già acquisito resta il riferimento per preparazione e consultazione.
**Review item** — Un blocco di revisione descritto da codice stabile, messaggio umano e
campo opzionale. Finché viene mantenuto nell'Evidence Unit, ne impedisce la
@@ -506,3 +571,73 @@ _Avoid_: Sensitive Data Suggestion Event
**Introspection Capability** — Una categoria di struttura fisica che una Database Binding
può osservare, come tabelle, colonne, relazioni, indici o enum. Una capability non disponibile
è distinta da una capability osservata che non ha restituito elementi.
## Amministrazione e integrazione
**Workspace Readiness** — La preparazione di uno specifico Workspace per l'uso nel
workflow, comprensiva della disponibilità degli artefatti derivati dai suoi metadati
Database e dalle sue Evidence. Il preprocessing appartiene a questa preparazione;
la configurazione e la sincronizzazione del catalogo restano responsabilità Database.
**Administration Surface** — Una superficie amministrativa autonoma per configurare o curare una
parte dell'installazione. Workspace, Evidence, Memory, Database e Pi sono superfici peer e non
dipendono dall'esistenza di una sessione attiva.
**Administration Page** — La rappresentazione a pagina intera di una Administration Surface, con
una gerarchia condivisa per identità, stato, azioni e contenuto. Un form amministrativo appartiene
alla pagina e non a una popup come contenitore principale.
_Avoid_: management popup, settings modal
**Administration Route** — L'identità navigabile di una Administration Surface nel browser. Deve
essere ripristinabile con refresh e cronologia e non contiene valori transitori o segreti dei form.
**Embedded Thoth Shell** — L'esperienza Thoth ospitata dentro il documento e il contesto visuale di
un portale host. Conserva la propria gerarchia funzionale, ma deve rispettare la geometria,
l'autenticazione e le regole responsive del portale host.
**Full Thoth Shell** — L'esperienza Thoth autonoma che possiede il proprio header e il proprio
layout di pagina. Non replica la navigazione amministrativa del portale host e non dipende dal suo
template visuale.
**Shell mode** — La scelta di installazione fra `embedded` e `full`. Determina chi possiede il
chrome globale, i comandi di identità e le integrazioni visuali, ma non cambia il workflow o la
persistenza delle sessioni.
**Fullscreen state** — Lo stato temporaneo in cui il documento applicativo occupa il fullscreen
del browser. È distinto da `Shell mode`: una Full Thoth Shell può essere aperta senza fullscreen;
il passaggio è attivato da un comando esplicito e può essere annullato con la stessa azione o con
il comando nativo del browser.
**Portal Shell Adapter** — Il confine sostituibile che traduce lo stato e i comandi del chrome di
un portale host nel modello semantico usato da Thoth. L'adapter non possiede autorizzazione,
sessioni di workflow o contenuti del modello.
**Host Shell State** — Il minimo stato visuale fornito dal portale host: locale UI, tema e stato
fullscreen. In una Embedded Thoth Shell è la fonte autorevole per queste preferenze;
non include identità, token o stato di autenticazione, che restano responsabilità dell'accesso.
**UI locale** — La lingua delle label, dei messaggi, dei tooltip, degli stati e delle istruzioni
non generate dal modello nell'interfaccia Thoth. È distinta dalla lingua dei contenuti di un
workspace.
**Interaction language** — La lingua in cui il modello presenta domande, spiegazioni e proposte
al revisore durante una sessione. Viene fissata alla creazione della sessione e rimane invariata
durante una ripresa, anche se la UI locale corrente cambia.
**Administrative Page Family** — L'insieme delle cinque Administration Page che condividono shell,
navigazione, tipografia e regole responsive, pur mantenendo contenuti e operazioni specifici:
Workspace, Evidence, Memory, Database e Pi.
## Installazione
**Manual standalone installation** — Una copia di ThothII predisposta per l'uso autonomo da una
persona che possiede il computer, con una Full Thoth Shell e servizi applicativi locali. La
procedura non implica che DWH o provider LLM siano locali o disponibili offline.
**Installation bootstrap** — L'insieme delle attività iniziali che rende disponibile una
installazione manuale: verifica dell'host, generazione della configurazione, predisposizione
delle credenziali protette e avvio dei servizi. Non è un installer dell'applicazione.
**Platform acceptance** — La verifica che una Manual standalone installation possa essere
predisposta e avviata su una specifica combinazione di sistema operativo, architettura e runtime,
distinta dalla verifica funzionale del collegamento a DWH e provider LLM.
+146 -44
View File
@@ -11,50 +11,50 @@ colors:
warm-graphite: "oklch(26.78% 0.0097 355.6)"
muted-graphite: "oklch(51.33% 0.0088 345.6)"
quiet-border: "oklch(90.93% 0.0035 354.7)"
success-mint: "oklch(75.77% 0.1581 165)"
success-mint: "oklch(46% 0.095 160)"
navigation-active: "oklch(92.5% 0.052 23.2)"
navigation-active-hover: "oklch(89.5% 0.071 23.2)"
navigation-active-foreground: "oklch(36.5% 0.11 23.2)"
navigation-active-border: "oklch(60% 0.135 23.2)"
warning-amber: "oklch(85.23% 0.1386 78.9)"
information-blue: "oklch(70.35% 0.1128 221.3)"
warning-amber: "oklch(48% 0.09 70)"
information-neutral: "oklch(51.33% 0.0088 345.6)"
typography:
display:
fontFamily: "Fraunces, Source Serif Pro, Georgia, Times New Roman, serif"
fontSize: "3rem"
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
fontSize: "1.5rem"
fontWeight: 600
lineHeight: 1.03
letterSpacing: "-0.025em"
headline:
fontFamily: "Fraunces, Source Serif Pro, Georgia, Times New Roman, serif"
fontSize: "1.875rem"
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
fontSize: "1.5rem"
fontWeight: 600
lineHeight: 1.15
letterSpacing: "-0.015em"
title:
fontFamily: "Fraunces, Source Serif Pro, Georgia, Times New Roman, serif"
fontSize: "1.2rem"
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
fontSize: "1.25rem"
fontWeight: 600
lineHeight: 1.25
letterSpacing: "-0.01em"
body:
fontFamily: "Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
fontSize: "0.9375rem"
fontFamily: "Manrope Variable, Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
fontSize: "1rem"
fontWeight: 400
lineHeight: 1.65
letterSpacing: "normal"
control:
fontFamily: "Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
fontFamily: "Manrope Variable, Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
fontSize: "0.875rem"
fontWeight: 600
lineHeight: 1.25
letterSpacing: "0.005em"
label:
fontFamily: "ui-monospace, SF Mono, Cascadia Code, Menlo, Consolas, monospace"
fontSize: "0.6875rem"
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
fontSize: "0.75rem"
fontWeight: 600
lineHeight: 1.25
letterSpacing: "0.06em"
letterSpacing: "normal"
rounded:
xs: "4px"
sm: "6px"
@@ -113,6 +113,16 @@ components:
# Design System: ThothII
## Visual review branch, September 2026
The revision on `codex/ui-visual-review` is approved for implementation and Docker visual review,
not yet for adoption on `main`. The previous look remains recoverable from the base commit and
the preserved Docker image. Historical prototypes must remain untouched.
This revision follows Impeccable's product register: one locally bundled Manrope family for the
whole UI, five fixed size roles, red as the sole brand accent and additional color only for meaningful
state. The primary scene remains an analyst reading data and SQL in a well-lit office.
## Overview
**Creative North Star: "The Clinical Workbench"**
@@ -134,7 +144,7 @@ disciplined and tactile, never playful, sluggish, or visually unstable.
**Key Characteristics:**
- Warm, restrained surfaces with one scarce red accent.
- Editorial headings paired with highly legible operational body text.
- One sans-serif family, with hierarchy expressed through size, weight and spacing.
- Dense information organized through hierarchy, rhythm, and progressive disclosure.
- Persisted artifacts and reviewer decisions presented as the visual source of truth.
- Fast state feedback with reduced-motion parity.
@@ -150,6 +160,15 @@ users can scan structure without adding nested containers.
## Colors
The full-mode application header matches Omics Portal's `--gsd-red-primary`
(`#CB333B`) in both themes. Its complete wordmark, including `II`, and controls
use a near-white foreground. This header is absent in embedded mode. The sidebar
and welcome wordmarks retain their red suffix. Context editing places workspace,
model and Done in one desktop row, stacking on narrow containers. Session-scope
tabs retain their selected fill and accessible keyboard state with a uniform one-pixel
border on every side, gray when inactive and red when active. Their padding is 11px
horizontal and 3px vertical, with a 38px minimum height and wrapping labels.
The palette combines warm porcelain surfaces, warm graphite text, and an instrument red used only
for action, focus, and important state. OKLCH values in the frontmatter are normative because the
frontend uses OKLCH tokens directly.
@@ -178,7 +197,8 @@ frontend uses OKLCH tokens directly.
foreground. It shares Instrument Red's hue but uses a lighter, lower-chroma fill, so location is
visible without carrying the full weight of a primary action.
- **Warning Amber** (`warning-amber`): waiting, attention, and in-progress states.
- **Information Blue** (`information-blue`): informational state when red would imply action.
- **Information**: neutral text and indicators for dates, protocols and ordinary status. The legacy
`--info` token resolves to muted foreground, not an additional blue accent.
The dark theme keeps the same semantic mapping with neutral near-black surfaces and a slightly
lighter red accent. Do not introduce a second visual identity for dark mode.
@@ -191,30 +211,33 @@ for their named states. Color is never the only state indicator.
## Typography
**Display Font:** Fraunces, with Source Serif Pro, Georgia, and Times New Roman fallbacks
**Body Font:** Manrope, with native system sans-serif fallbacks
**Label/Mono Font:** SF Mono or Cascadia Code, with Menlo and Consolas fallbacks
**UI Font:** locally bundled Manrope Variable, with Manrope and native sans-serif fallbacks.
**Technical Font:** SF Mono or Cascadia Code, with Menlo and Consolas fallbacks.
**Character:** Fraunces gives persisted artifacts and key headings editorial authority. Manrope
keeps dense controls and prose calm and readable. The mono register separates machine identity,
metadata, SQL, identifiers, and micro-labels from natural-language content.
Manrope covers headings, labels, controls, navigation and document reading. Monospace is reserved
for SQL, code, paths and machine identifiers, never for ordinary UI labels or status headings.
### Hierarchy
- **Display** (600, `3rem`, `1.03`): authentication and exceptional page-level statements only.
- **Headline** (600, `1.875rem`, `1.15`): major page or artifact titles.
- **Title** (600, `1.2rem`, `1.25`): panel and document section hierarchy.
- **Body** (400, `0.9375rem`, `1.65`): operational prose, with a target line length of 65 to 75
- **Headline** (600, `1.5rem`, `1.3`): page or artifact titles, `--text-page`.
- **Title** (600, `1.25rem`, `1.4`): section hierarchy, `--text-section`.
- **Body** (400, `1rem`, `1.6`): operational prose, `--text-body`, with a target line length of 65 to 75
characters where the surface controls width.
- **Control** (600, `0.875rem`, `1.25`): buttons, inputs, tabs, and compact actions.
- **Label** (600, `0.6875rem`, `0.06em` tracking): uppercase micro-labels, state metadata, and panel
headers. Labels use the mono family.
- **Control** (400–600, `0.875rem`, `1.5`): buttons, inputs, tables, tabs and compact subheadings,
`--text-control`.
- **Metadata** (400–600, `0.75rem`, `1.5`): secondary status, counts and timestamps, `--text-meta`.
Labels use sentence case and normal tracking. Ordinary operational text never falls below 12px.
Typography uses fixed sizes. Responsive changes happen at structural breakpoints, not through fluid
type scaling. Numeric data and identifiers use tabular numerals where comparison matters.
**The Three Registers Rule.** Serif means authority, sans means interaction and reading, mono means
machine identity. Do not exchange these roles for novelty.
**Application wordmark:** ThothII is a brand mark, not a page title: use Manrope semibold at
48px (`3rem`) in the Core welcome area and 32px (`2rem`) in the session sidebar, with the
`II` suffix in brand red. Preserve these sizes across responsive layouts.
**The One Family Rule.** The UI and document readers use sans-serif throughout. The legacy
`--font-heading` alias resolves to `--font-sans`. Preserve technical monospace without turning it
into a second decorative hierarchy. Do not shrink text to solve layout constraints.
**The Read Once Rule.** A heading, label, and body must be distinguishable on first glance through
size and weight. Do not repeat headings in explanatory copy.
@@ -279,27 +302,70 @@ default, hover, focus, active, disabled, loading, and error behavior where those
- **Focus:** three-pixel Instrument Red ring with a clear border shift.
- **Error / Disabled:** errors combine destructive color with explanatory text; disabled controls
retain readable contrast and use 50 percent opacity.
- **Metadata catalog model:** Database Management keeps one compact, installation-level
metadata-generation LLM selector in the application header. The selection persists across
database, table, column, and relationship views; when no usable profile is configured, the
disabled control explains: “No metadata-generation LLM model is configured for this installation.”
- **Global context:** the collapsible top shelf is the sole workspace/model selector for Core and
Admin. Preserve independent remembered choices, installation defaults, operation locks and unsaved
edit guards. Never introduce a separate metadata-generation default or selector.
### Navigation
- **Workspace readiness:** the Workspace navigation button carries an 8px dot to
the right of its label. Green means a selected workspace with confirmed ready
preprocessing and no query error; all other states are red. The button's
tooltip and accessible description retain the translated exact state. Do not
add a separate readiness text row or change the backend readiness gate.
- **Session groups:** one accessible single-open accordion contains Active sessions
and Archive, both initially closed. Below the scope tabs, show only their
adjacent section headers, without a redundant Sessions heading. Selection and
bulk-delete controls belong inside each panel and only appear for nonempty
lists. Select all affects that list only, preserves the other list's selection,
and exposes a mixed state for partial selection. Preserve the existing archived
flag as the grouping rule, independent of whether a Pi process is running.
Opening a section closes the other; either can be collapsed, including both.
Empty lists show only the translated "No sessions yet." message.
The open section uses the rail's remaining height; its list scrolls internally
with a cap of `min(18rem, 35dvh)`, while its trigger remains outside that scroll
area. The mobile navigation dialog supplies a bounded viewport-height container.
Keyboard users can focus and scroll each labelled panel.
- **Session entry:** one Session button returns to the current unfinished session,
including provisional creation, without resetting or reconnecting it. Otherwise
it prepares a new question using the normal readiness and unsaved-work guards.
- **Style:** compact session rows use `8px` corners and restrained vertical padding.
- **Default / Hover / Active:** porcelain at rest, Sunken Surface on hover, and a muted Navigation
Active red with a defined border when current. Exactly one top-level navigation control is current.
- **Administrative controls:** the admin-only Administration accordion groups Database management,
a structural divider, Workspace management, and Pi management in that order. Its trigger exposes
- **Administrative controls:** the admin-only Administration accordion groups Database,
Memory, Evidence, a structural divider, Workspace, and Pi configuration in that order. Its trigger exposes
expanded state and starts collapsed by default, while non-admin users do not receive the accordion
or its navigation actions.
- **Responsive:** collapse navigation structurally at the application breakpoint. Do not shrink
labels into illegibility.
labels into illegibility. Below 768px, Memory and Evidence management use the full content
width; a Navigation button opens the shared accessible dialog. Selecting another archive
page or pressing Escape closes it. Desktop retains the right session sidebar and its My sessions /
All sessions tabs. Core retains question/answer, eight phases, reviewer gates and the left log.
In embedded mode the portal owns the red header and left sidebar; ThothII must not duplicate them. Size to the
actual application container. Narrow session document panels may use the available width.
### Session review and confirmations
Session dialogs use the visible application area, including the portal's header
and side rail. Artifact and schema-column review can grow to 80rem wide and the
available height; short confirmations use up to 40rem and at least 18rem when
space permits. Keep a 24px outer margin on desktop and 8px on small or short
screens. Long review content scrolls internally; on very short screens the
whole dialog can also scroll so every action remains reachable.
Session forms and review gates repeat their existing primary confirmation above
and below the content, sharing selection, validation, pending state and response
handlers. Alternate-response inputs follow the same rule. Reserved navigation
controls remain below the review. Stop/delete initially focus Cancel; rename
initially focuses the name field. Administration dialogs and forms retain their
existing layout and actions.
### Tabs
- **Shape:** compact label tabs sit on a shared baseline with rounded top corners and a two-pixel
lower edge. Inactive labels retain a complete Quiet Border and Porcelain Card surface, so every
lower edge, except session-scope tabs which use a uniform one-pixel border, rounded
corners and a 4px gap without a shared border or negative bottom margin.
Inactive labels retain a Quiet Border and Porcelain Card surface, so every
label reads as a tab before interaction; hover feedback reinforces clickability.
- **Current:** the selected tab uses the muted Navigation Active red for its fill, text, and defined border.
It must expose `aria-selected`, participate in a labelled `tablist`/`tabpanel`, and be the only
@@ -319,9 +385,42 @@ default, hover, focus, active, disabled, loading, and error behavior where those
### Curated Evidence Documents
Memory and Evidence share the `thot-knowledge-reader` reading contract. Use locally
bundled Manrope with normal tracking for prose and labels, and these fixed roles:
- Card title: 24px, weight 600, line-height 1.3 (`thot-knowledge-title`).
- Field/section heading, including Scope and Provenance: 20px, weight 600,
line-height 1.4, 8px clearance below (`thot-knowledge-heading`).
- All narrative text, including scope, lists and provenance: 16px, weight 400,
line-height 1.65. Do not apply compact UI text sizes to these fields.
- Authored Markdown subheadings inside a field: 16px, weight 600, line-height 1.5,
24px above/8px below. They remain subordinate to the enclosing field heading;
their semantic heading levels and original content are preserved.
- Technical metadata labels/values: 14px/1.5, with weight 600 for labels.
Only code, paths and machine identifiers use the technical monospace family at
14px/1.65, identical for inline and fenced code (never compound `em` shrinkage).
Separate reading sections by 24px; keep the first Markdown block flush with its
field heading's 8px bottom gap. The same typography applies in light/dark and at
all responsive widths. Controls and archive indexes retain their compact UI roles.
Memory and Evidence detail readers use the entire available content width, without
the ordinary 72–75ch prose cap. This is the owner's explicit reading-layout choice.
Long unstructured paragraphs are split for display at existing sentence/semicolon
boundaries outside inline code and links; authored Markdown structure and stored
content are unchanged. Paragraph spacing is 1.25em. Scope and provenance share the
available width; provenance excerpts render Markdown rather than literal markers.
Copy actions use the two-overlapping-sheets icon, an accessible name/tooltip and
live success/failure feedback instead of a visible Copy label.
Memory has four explicitly FAKE formatting examples, one per family, in a separate
expandable section. They reuse the real detail reader but never enter persistence,
indexing, link search or model recall, and expose no edit/delete/save actions.
Curated evidence follows a fixed reading order: title, compact type and purpose summary, scope,
typed content, supporting excerpts, review items, then collapsed technical provenance. Machine
metadata stays in invisible comments so GitHub Preview shows only the reviewable document.
typed content, supporting excerpts, review items, then technical provenance. Curated v4 files
use short, visible YAML frontmatter for identity and classification. The Markdown title and
body are authoritative; hidden payload comments are a legacy format converted on consolidation.
`applies_to` is rendered as “Ambito di applicazione” with separate bullet lists for concepts,
tables, and columns. Enum values also use lists. Tables are forbidden for metadata, scope, or any
@@ -329,7 +428,9 @@ one-dimensional collection; reserve tables for genuinely two-dimensional dataset
identifiers use inline code. SQL uses fenced code. Supporting excerpts use blockquotes.
**The Review Surface Rule.** The visible Markdown must be readable without understanding the
machine contract. Technical metadata belongs in progressive disclosure, not above the title.
machine contract. In Administration, explain current and original provenance separately and
keep file-editing templates and Git instructions in progressive disclosure. Show actual host
paths with copy controls, never browser file links to container-only locations.
## Do's and Don'ts
@@ -341,7 +442,8 @@ machine contract. Technical metadata belongs in progressive disclosure, not abov
- **Do** preserve information density with headings, rhythm, and progressive disclosure.
- **Do** keep keyboard focus explicit and pair color with text, shape, icon, or position.
- **Do** respect `prefers-reduced-motion` while preserving immediate non-kinetic feedback.
- **Do** use English for interface chrome and the workspace language for persisted document content.
- **Do** use the selected interface language (English by default) for chrome and preserve the
workspace language for persisted domain content. Session interaction language remains pinned.
- **Do** render curated metadata and scope as Markdown prose or lists, never as a frontmatter table.
- **Do** break long curated rules into paragraphs, labelled subsections, and lists at existing
punctuation boundaries while preserving the exact canonical text for machines.
+84 -292
View File
@@ -1,309 +1,101 @@
# ThothII — Project State
# Project state
Last updated: 2026-09-06.
Updated: 2026-09-27. This is a current snapshot, not a release diary. Stable commands
and invariants are in [AGENTS.md](AGENTS.md); prior snapshots remain in Git.
This file is the short operational snapshot. Stable commands and the architecture mental model
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
`docs/contracts/`, `docs/adr/`, and `docs/evidence.md`. Superseded plans and reports are
available from Git history rather than duplicated in the working tree.
## Current contracts
The guarded server migration from a legacy checkout to the schema-v2 installation, Gitea source,
Authentik, internal catalog/embedding services, and the PSD workspace repository is documented in
`docs/operations/server-upgrade-gitea-workspace-v2.md`. Treat its operator gates and rollback
requirements as mandatory; do not replace the running server stack in place.
- React supports full/embedded rendering independently of local/OIDC/upstream auth,
with EN/IT UI and immutable session interaction language. See
[application shell](docs/architecture/application-shell.md) and
[localization](docs/operations/shell-and-localization.md).
- PostgreSQL Metadata Catalog owns database identity, binding, schema, descriptions,
sensitivity and relationships for all core consumers. Workspace schema v4 contains
identity and optional Evidence only. Installation schema v2 is the authored model
catalog source. See [overview](docs/architecture/overview.md) and
[model configuration](docs/general/pi-configuration.md).
- The harness owns workflow persistence; chat is not the durable session record.
Memory uses PostgreSQL authority and derived Qdrant dense/BM25 search. Editable
Evidence has local archive authority and manual consolidation. File save, search
activation and Git publication have distinct outcomes. See
[Memory](docs/gestione-memory.md), [Evidence](docs/contracts/curated-evidence-v4.md)
and [consolidated release evidence](docs/reports/knowledge-archives-release.md).
- Reference preprocessing must not clear Memory. Use the installation-scoped
`tht --installation /absolute/path/thothii-installation.yaml workspace preprocess run`
and its [contract](docs/contracts/workspace-preprocessing-cli.md).
- DWH sessions are read-only. SSH tunnels support database-management diagnostics
and metadata synchronization, not NL→SQL session creation; use direct or REST
transport for sessions.
## Current product shape
## Installation and workspace boundaries
ThothII is a human-in-the-loop datamart builder with three independently built layers:
Fresh standalone installations follow the manual terminal procedures in
[Italian](docs/install/standalone-manual-it.md) or
[English](docs/install/standalone-manual-en.md), without an installer or launcher.
The [Compose reference](docs/operations/compose-reference.md) is for maintainers,
not another quick start. Catalog and Memory migrations are explicit.
```text
frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness → DWH (read-only)
```
Descriptors, authentication, provider credentials, certificates and runtime bindings
stay in protected installation-local paths. Do not copy secrets into examples or
workspace Git. Legacy runtime snapshots may use absolute paths and protected
`harness/.env`; do not silently relocate them.
The harness owns the deterministic eight-phase NL→SQL workflow and all session persistence.
The backend remains a process/RPC/SSE bridge for sessions and now also owns an isolated PostgreSQL
metadata catalog for administrative database configuration. The frontend renders the review gates
and keeps the live transcript in memory. See
`docs/architecture/components.md` for the detailed component and data-flow map.
PSD authoring is separate at `/Users/mp/projects/tht-workspace-psd`. Its GitHub
repository was copied to private Gitea
[workspace_psd](https://git.tylconsulting.it/mptyl/workspace_psd), preserving both
branches. It is a copy, not automatic synchronization. Running installations were
not repointed to a different workspace remote.
## Evidence restructuring — accepted
## Recorded deployments and server authority
The evidence restructuring and PSD migration completed real acceptance on 2026-08-25.
Use the ordered [server handoff](docs/operations/server-codex-handoff.md) for
coordinated ThothII/Omics upgrades. Omics integration uses GitHub
`Dallavilla-Tiziano/omics_portal`, with no Gitea relay prerequisite. Omics uses
embedded/upstream identity, not another ThothII OIDC login. Read
[upstream authentication](docs/install/authentication-upstream.md) before changes.
- The curated PSD revision contains 35 approved Evidence units and 60 review items.
- The PSD authoring repository publishes all 35 units using Curated unit schema v3. Its table-free
presentation uses hidden canonical metadata, wrapping Markdown scope lists, list-based enum
values, and collapsed technical provenance. Long domain rules now have a deterministic
human-readable presentation while retaining their exact canonical text for vector ingestion.
`tht evidence migrate <workspace-root>` performs the deterministic v1/v2 upgrade and older-v3
presentation rewrite without model calls. The structured-rule PSD rewrite is currently local and
pending commit/publication.
- The accepted snapshot is
`psd-clinical-675990d90eae51da6f2bd51b1ae2609f245772ef-snapshot`.
- The active generation is `gen:f968b3bd7a553dbfef3cf47093698f2bc7f95f11`.
- Retrieval acceptance reached 20/20 Hit@10.
- A real session, `20301df7-cad7-403d-a4c1-9f35c9d07b66`, completed F1–F8 with five
receipts, three CTEs, and a final result of 78 patients.
- The durable acceptance record is
`docs/testing/evidence/evidence-restructuring-psd-acceptance-2026-08-25.md`.
Last recorded application deliveries (not a fresh runtime attestation):
The canonical authoring, validation, publication, materialization, and preprocessing flow is
documented in `docs/evidence.md`. The governing contracts are
`docs/contracts/workspace-evidence-v3.md` and
`docs/contracts/workspace-preprocessing-cli.md`.
The incremental server procedure for the `260906-preprocessing-complete` release is
`docs/operations/server-handoff-260906-preprocessing-complete.md`.
- [Coordinated ThothII/Omics release](docs/reports/2026-09-26-server-release-execution.md):
core/frontend `497ab840-preflight`, Omics proxy fix `928f7e9f` with existing web
image retained. Automated acceptance passed; on September 27 the operator confirmed
browser access, UI controls and session start/stop/resume. Functional browser
acceptance passed; remaining extended checks are handed off in the
[server acceptance follow-up](docs/reports/2026-09-27-server-acceptance-handoff.md).
- [Session dialogs](docs/reports/2026-09-14-session-dialogs-release.md):
`b1723c34-session-dialogs-20260914`, frontend-only.
- [Session layout/Memory fix](docs/reports/2026-09-14-session-layout-memory-fix.md):
`49333a2d-session-memory-fix`, core/frontend.
## Workspace preprocessing and configuration
Keep those reports and rollback instructions while operator gates remain open.
A later deployment does not prove every earlier acceptance item passed.
The native host CLI `tht` is the operator surface. Workspace preprocessing runs through:
## Remaining acceptance and design gates
```sh
tht --installation /absolute/path/thothii-installation.yaml \
workspace preprocess run --workspace <workspace-id>
```
- Fresh-machine Mac, Windows/WSL2 and Linux installation acceptance, including real
DWH/model endpoints, remains a separate operator exercise.
- Real IdP/portal login, logout, embedded interaction and PSD semantic acceptance
follow the [manual matrix](docs/testing/authentication-manual-acceptance.md) and
delivery reports; synthetic tests do not close them.
- [Security hardening](docs/plans/2026-09-08-security-hardening-prd.md) is a draft.
Revalidate SEC01–12 and obtain design approval before implementation or real
server/IdP/DWH mutation.
- Optional NER remains opt-in; labeled Italian quality, benchmark and licensing
acceptance are not implied by document cleanup.
- Semantic aliases, value descriptions, synonyms/concepts, dialect and multi-schema
extensions remain explicit design work. Current sensitivity delivery follows the
Catalog contract; additional policies require their own acceptance.
- Legacy database UI fallback (`?db-ui=legacy`, dev/staging) and prototype removal
remain subject to owner acceptance.
This complete one-shot command uses the profile-gated `workspace-maintenance` service. Partial DWH,
schema, Evidence, and vector mutation commands are retired.
## Documentation maintenance
The right Administration sidebar invokes that same operation for the selected workspace. It shows
only current readiness or the latest bounded failure diagnostic; there is no preprocessing history.
Known non-ready state disables **New session**, while backend admission remains authoritative.
The same control exposes an inline-confirmed **Clear** action to remove replaceable reference
vectors, LSH, corpus, and checkpoints while preserving the separate Memory collection. The host CLI
equivalent is `workspace preprocess clear`.
MkDocs publishes only 20 product/operator pages and five approved assets.
Architecture, contracts, ADRs, plans, research, tests and release evidence are
excluded from HTML and search. The repository itself is public: editorial exclusion
is not confidentiality.
Each workspace now uses `<workspace>-reference` for Schema, relationships, and Evidence and
`<workspace>-memory` for `memory` and `solved_question`. Clear and preprocessing own only the former.
LSH ownership additionally binds the Catalog database ID and Metadata Content Revision, so derived
values cannot be reused across database identities or Catalog revisions.
Workspace descriptors use schema v4 and contain only workspace identity and optional Evidence.
PostgreSQL Metadata Catalog owns database identity, binding, schema, descriptions, sensitivity, and
relationships; model, provider, embedding, and vector-store configuration is installation-owned. For
PSD, workspace content and runtime roots point to the
separate uncommitted repository `/Users/mp/projects/tht-workspace-psd`. Secrets remain outside
Git and are supplied only through installation-local protected files.
## Installation Model Catalog
`thothii-installation.yaml` schema version 2 is the only operator-authored source for session,
metadata-generation, and embedding models. The host `tht` lifecycle validates `modelCatalog` and
regenerates the backend catalog, Pi `models.json`/`settings.json`, and Compose override under the
installation-local `generated/` directory. Those projections are replaceable runtime adapters:
they are not edited, backed up, or treated as configuration.
Session and metadata defaults use canonical `provider/model` IDs. Provider authentication declares
one explicit mode (`secret_env`, `pi_auth`, or `none`); `secret_env` names a protected bundle key.
The backend settings store now owns only the selected workspace and thinking level. Existing v1
installations use the explicit catalog migration command; schema-v3 workspace descriptors are
converted deterministically in their curator-owned repository before commit. Strict runtime loading
does not silently infer or merge legacy sources. ADR 0013 and
`docs/plans/2026-09-02-installation-model-catalog.md` record the decision and implementation.
## Database management
The database, table, and authoritative physical-schema catalog slices are implemented. Database
Management now opens the Fleet Ledger presentation by default inside `AppShell`, lists every YAML
workspace, creates at most one PostgreSQL database configuration per workspace, edits direct
PostgreSQL, REST API, or SSH-tunnel installation bindings, replaces write-only encrypted secrets,
and tests supported connector bindings. The surface keeps one responsive AG Grid visible at a time:
databases lead to tables, tables lead to columns, and relationships are a sibling database view.
Parent navigation remains explicit through the breadcrumb and emphasized back control.
Selection-scoped operations use one action selector plus an explicit **Run** control; ineligible
actions remain visible with their disabled reason, while row-scoped actions stay in the pinned final
column. The KPI strip reads installation-wide or selected-database aggregates from
`GET /catalog/metrics`. Database configuration, metadata editors, synchronization history,
description history, and sensitive-field review/history use the production APIs in right-side
drawers rather than prototype fixtures; closing a history drawer does not stop its background run.
Sensitive-field review is now driven by the versioned local `sensitivity-v4` policy, not by a
catalog model. The backend reads selected source tables through read-only, database-specific
adapters and makes every `sensitive | non_sensitive` draft decision in the TypeScript
`SensitivityClassifier`. A single validated match protects the column. Tables up to 1,000 rows are
fully scanned; larger tables use breadth-first 300, 1,000, and text-only 3,000-value targets, with a
five-second limit per source query and no global request deadline. Source failures fail the run
instead of yielding `unknown`; coverage remains visible separately from the proposal. Draft
assessments remain transient until an administrator explicitly saves them. Optional GLiNER2
evidence is CPU-only, offline, opt-in, and never replaces the deterministic decision point; see
`docs/operations/sensitivity-analysis.md`. The earlier v1 PSD shadow comparison kept NER disabled by
default; see `docs/reports/2026-09-02-psd-sensitivity-shadow.md`. The v2 comparison completed all
2,275 columns: CPU NER added 18 sensitive proposals and increased warm runtime from 50.1 to 61.3
seconds; see `docs/reports/2026-09-03-psd-progressive-sensitivity-shadow.md`.
Version 4 excludes declared `bigint` primary-key columns and conventionally named `pk bigint`
columns before source inspection, reporting both as non-informative structural identifiers while
distinguishing declared constraints from inferred roles.
Physical membership, source
comments, column types/default/nullability/PK positions, and constraint-level ordered FK pairs are
projections of the external schema. They cannot be created, renamed, or structurally edited by
hand, but administrators can explicitly clear catalog tables, columns, or relationships without
touching the source database, binding, configuration, or secrets. Table deletion cascades through
columns and relationships; table-scoped relationship cleanup includes incoming and outgoing
relationships. Curated and generated descriptions are editable; generated descriptions start null
and Database Management can generate or consolidate them for selected tables, selected columns,
all targets, or only targets whose Generated Description is missing.
Relationship Management is now reachable directly from each configured Fleet database. One
Relationship Map shows read-only Physical Relationships together with Generated and Manual Logical
Relationships, with Active, Excluded, and All filters. Administrators can add a single-column
relationship, run deterministic name/PK/type inference, exclude or restore a logical relationship,
or delete it permanently. Exclusion retains a tombstone that a rebuild cannot reactivate; permanent
deletion allows a later rebuild to infer the same endpoints again. Inference uses no LLM, embedding,
or source values. It supports normalized table-qualified names, unique non-generic PK names,
composite-PK source columns, and the `*time_key -> dim_time.<single PK>` warehouse convention while
ignoring bare generic names. Explicit table/column metadata cleanup remains a destructive boundary: it removes
the attached logical relationships and exclusions and requires a full schema synchronization before
inference or runtime publication can continue.
The previous Database Management renderer remains a temporary comparison fallback for development
and staging only: `?db-ui=legacy` is honored in Vite development or when
`VITE_DB_MANAGEMENT_LEGACY=true`; it is not a production presentation. The standalone Fleet Ledger
prototype on port `5173` also remains temporary until owner acceptance of the integrated surface,
after which both migration aids can be removed.
Schema refresh is one durable asynchronous engine with database-table, selected-table-column,
relationship, and full-database actions. Database-level menus expose only the table, relationship,
and full scopes; selecting tables exposes column synchronization plus manual column and relationship cleanup for that subset. Database selections
also expose manual table and relationship cleanup. Cleanup selections are atomic and share the
one-active-operation-per-database exclusion with synchronization. Runs have leases and
restart recovery, atomic apply, destructive-diff confirmation with re-scan, cancellation before
apply, retained history, and a live SSE log with polling fallback. Null metadata renders blank
rather than as a placeholder.
Direct PostgreSQL and strict known-host-verified OpenSSH use `pg_catalog`. REST bindings use the
typed full-snapshot `POST /rpc/schema_snapshot` contract when available. Servers such as the
current PSD endpoint that exposes only `POST /rpc/run_query` use one catalog-owned read-only query
to return the exact same strict v1 snapshot in a single round trip. Both paths remain fail-closed:
an absent capability, query error, partial result, or invalid snapshot applies no catalog changes.
SSH is not yet enabled for NL→SQL session runtime.
The catalog runs in the internal `catalog-db` PostgreSQL service. Kysely migrations are an explicit
one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before local startup. Runtime
sessions consume an immutable Catalog JSON snapshot tied to the runtime-config lease. It contains
the tables, columns, effective descriptions, sensitivity flags, and active relationships used by
the harness; PostgreSQL is the exclusive runtime authority for database metadata. Authored
workspace YAML remains limited to workspace identity and optional Evidence configuration. The
accepted design is recorded in ADR 0016 and the contracts under `docs/contracts/`.
Semantic aliases, value descriptions, synonyms, and concepts remain deferred to their dedicated
slices.
AI Description Generation uses the catalog's human-owned Sensitive Data Flag. The flag defaults to
`false`, including for newly synchronized columns. An administrator may request a local sensitivity
analysis for one selected database, selected tables, or selected columns. One deterministic
TypeScript classifier combines metadata, bounded source-content rules, and optional CPU-only NER;
no generative model decides the result. Its `sensitive` or `non_sensitive` assessments remain an
unsaved draft until the human reviews and saves any chosen flag changes, including a downgrade to
non-sensitive. Coverage is reported separately; interrupted history may count unprocessed columns.
Each started analysis records a separate Sensitivity Analysis Run with aggregate counters and safe
ordered events. The progress drawer opens before the synchronous request completes, polls the run,
and displays sanitized source-scan and local-NER phase/batch activity while classification is in
progress. This operational history never stores per-column assessments, source values,
matched spans, prompts, or free-form diagnostics. Saving a sensitive decision persists a sanitized
Sensitivity Reason as column Catalog Metadata alongside the human-owned flag; clearing the flag
clears that reason. Reloading still discards an unsaved review draft.
For unprotected columns, up to five source rows and five representative non-null values may be sent
transiently to the configured model provider. Protected columns are omitted from source reads and
replaced in the prompt by deterministic plausible values derived only from their metadata. Existing
descriptions are not regenerated when a flag changes.
The accepted AI-description design is recorded in
`docs/plans/2026-08-28-ai-catalog-description-generation.md`, with the formal specification in the
adjacent `-spec.md` document and Gitea issue #4. Gitea issues #5–#11 deliver the implementation.
The runtime deliberately keeps ThothAI's simple operating model: one installation-wide sequential
run owned by the backend, one short-lived Python/LiteLLM completion helper per request, and
persistence limited to the run, its safe ordered text events, and each Generated Description as
soon as it succeeds. The helper performs at most one provider retry and never falls back to another
model. Stop terminates the current helper and retains prior results; three consecutive exhausted
technical batches fail the run. Startup marks stale queued/running work interrupted, and Unlock is
available only when no local start, worker, or helper is live. Runs remain inspectable through a
live SSE log with ordered polling fallback; there is no automatic resume or user-facing generation
CLI. ADRs 0009–0010 record the runtime and source-sampling decisions.
The Installation Model Catalog accepts the protected `DEEPSEEK_API_KEY` and `ZAI_API_KEY`
references for metadata-generation providers.
It also accepts a model with no secret reference only when its OpenAI-compatible endpoint is
explicit; this covers the VPN-only AritmoLab Qwen 3.6 server without creating a fake operator
credential. The Python client supplies only its fixed non-secret compatibility placeholder.
The AritmoLab entry also sets `disableThinking: true`, mapped to the endpoint's chat-template flag,
because its default reasoning prose would violate the worker's exact JSON response contract.
Logical relationship integration with core schema-linking is complete: session creation and resume
materialize the active physical/generated/manual map, retrieval-pack generation and Pi receive the
same runtime config, and snapshot validation fails closed on a declared missing, invalid, or orphaned
endpoint. Broader publication of other Catalog metadata to schema-linking remains a separate future
slice.
**Deferred follow-up — Sensitive Data Policy in schema-linking.** The policy is first delivered
and tested in catalog description generation. Its enforcement for core schema-linking remains
out of scope until the current tickets are closed and the owner has completed the acceptance test.
At that gate, resume the design: `tht` must receive a read-only projection of the current Sensitive
Data Flags and exclude values from columns marked sensitive from every LSH result before it is
given to Pi. Do not start this integration before the owner gives final approval after that test.
## Active deployment work and manual gates
### PSD server deployment program
The approved design and executable entry point are:
- `docs/plans/2026-08-20-psd-server-deployment-program-design.md`
- `docs/plans/2026-08-20-psd-server-deployment-program.md`
- `docs/plans/2026-08-20-psd-server-survey.md`
- `docs/plans/2026-08-20-psd-server-project-a-standalone.md`
- `docs/plans/2026-08-20-psd-server-project-b-authentik.md`
Last recorded state:
- survey: `SURVEY_NO_GO`;
- Project A: `BLOCKED_BY_SURVEY_AND_MUTATION_GATE`;
- Project B: `BLOCKED_BY_PROJECT_A_AND_PRE_B_GATE`.
The deployment is a clean replacement: legacy sessions, indexes, and application configuration
are not migration inputs. The existing stack remains intact until its documented mutation and
rollback gates are explicitly approved. Shared Omics/LocalLLM networks, ETL Evidence, DWH,
`dwh-auth`, Supabase, Authentik, Superset, and Aritmolab are outside cleanup scope.
Human acceptance guides and sanitized report templates live under `docs/testing/` and
`docs/testing/evidence/`. The remediation checklist is
`docs/operations/psd-server-survey-remediation-checklist.md`.
### Authentication
The local/OIDC authentication remediation passed its automated review on 2026-08-18. Release and
PSD mutation gates remain governed by:
- `docs/architecture/authentication.md`;
- `docs/plans/2026-08-18-thothii-authentication-acceptance-and-psd-deployment.md`;
- `docs/operations/psd-dwh-auth-rollout.md`;
- `docs/testing/authentication-manual-acceptance.md`.
Do not infer authorization for server, Nginx, Authentik, database, credential, or cutover changes
from an automated PASS.
## Verification status
- The P1.1 workspace-directory registry and P2–P6 preprocessing workstreams are implemented and
have automated coverage.
- Evidence restructuring has a real PSD acceptance PASS as recorded above.
- AI Description Generation has automated coverage across installation setup, model selection,
generation/consolidation scopes, bounded sampling, cancellation/recovery, history, SSE/polling,
and the LiteLLM helper boundary.
- L2 tests requiring real providers or remote databases remain opt-in.
- Server deployment, release, and owner-operated acceptance steps remain pending wherever the
referenced runbooks require explicit approval.
Run the layer-specific checks documented in `AGENTS.md`. For release-sensitive changes, also run
the repository contract scripts in `scripts/` and build the MkDocs site.
## Operational invariants
- `tht`'s `-c`/`--config` option follows the subcommand; it is not a global option.
- `--json` commands write pristine JSON to stdout.
- Persisted phase documents and the decision ledger are the source of session truth; chat is not.
- UI chrome is English; workspace document content retains the workspace language.
- The backend refuses resume for finalized or archived sessions.
- A resume must send `/riprendi-sessione <id>`; a new session must send `/nuova-domanda`.
- DWH access is read-only.
The [cleanup record](docs/maintenance/2026-09-15-documentation-cleanup.md) records
retired sources and retained gates. Main contains source; Actions generates the
`pages` branch. The live site requires the separate explicit deployment described
in [public manual publication](docs/operations/public-docs-publication.md).
+37 -58
View File
@@ -4,72 +4,51 @@ ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fasti
core. The portable deployment runs two application services plus the installation-local metadata
catalog; DWH and LLM services remain external. Semantic services are bundled in Compose.
Authentication is configured through the single host CLI tht: see the [local authentication guide](docs/install/authentication-local.md),
[generic OIDC guide](docs/install/authentication-oidc.md), and [manual acceptance matrix](docs/testing/authentication-manual-acceptance.md).
The same frontend supports **full** (its own header) and **embedded** (inside a
portal). This choice is independent of authentication: the Mac uses full/local,
Omics uses embedded/upstream with its existing login, and a standalone server
can use full/OIDC. See [rendering architecture](docs/architecture/application-shell.md)
and [configuration, Omics delivery and deploy](docs/operations/shell-and-localization.md).
## Docker Compose: local startup
For the current server upgrade with Omics Portal, follow the ordered
[Codex server handoff](docs/operations/server-codex-handoff.md), including source
integration, embedded/upstream configuration, coordinated rollout and rollback.
Requirements: Docker Engine with Compose v2. The mandatory stack is `frontend`, `core`,
`catalog-db`, `qdrant`, `embedding`, and the one-shot `embedding-model-init`. DWH and LLM remain external,
configurable endpoints—even when they are co-located with ThothII.
Local/OIDC authentication is configured through the host CLI `tht`; portal
authentication is established by the trusted server proxy. See the
[local guide](docs/install/authentication-local.md),
[OIDC guide](docs/install/authentication-oidc.md),
[upstream integration](docs/install/authentication-upstream.md), and
[manual acceptance matrix](docs/testing/authentication-manual-acceptance.md).
From a fresh clone, run these commands from the repository root:
For the clone-based manual standalone installation test on macOS, Windows, and Linux, use the
[Italian procedure](docs/install/standalone-manual-it.md) or the
[English procedure](docs/install/standalone-manual-en.md).
```sh
cp deploy/env/local.env.example deploy/env/local.env
# Copy docs/install/examples/thothii-installation.local.yaml to a protected operator path,
# replace its placeholders, chmod it 600, and set that exact THT_INSTALLATION_CONFIG_SOURCE.
# Edit deploy/env/local.env, including PI_AUTH_FILE, THT_SECRETS_FILE, and external endpoints.
./scripts/run-stack.sh
```
The [public manual](https://git.tylconsulting.it/thothii-docs/) covers the product,
installation, use and administration. Developer architecture, contracts, ADRs, tests,
plans and release records remain in this repository but are excluded from MkDocs
pages and search. This is an editorial boundary, not an access restriction on the
public repository. See the [documentation cleanup review](docs/maintenance/2026-09-15-documentation-cleanup.md)
for the executed consolidation and the inventory of historical sources retained in Git.
The launcher builds the core, starts `catalog-db`, runs the explicit one-shot Kysely migrations,
then runs the base+local stack in the foreground. Migrations never run implicitly in backend
startup. The core image contains its Pi runtime; no host `pi` executable is used. For a server
installation, build the image, start the catalog, and run the same migration service before the
application rollout:
## Docker Compose and installation
```sh
cp deploy/env/server.env.example deploy/env/server.env
# Prepare a mode-600 thothii-installation.yaml from the server example and set its exact
# path as THT_INSTALLATION_CONFIG_SOURCE. Edit all remaining storage/secret/endpoint paths.
sudo scripts/prepare-server-pi-state.sh /srv/thothii/pi-state 10001 10001
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example build core
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example up -d catalog-db
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example run --rm catalog-migrate
docker compose --env-file deploy/env/server.env \
-f compose.yaml -f deploy/compose.server.yaml \
-f deploy/compose.session-server.yaml.example up --build -d
```
For a fresh installation, follow the complete manual procedure in
[Italian](docs/install/standalone-manual-it.md) or
[English](docs/install/standalone-manual-en.md). Configure protected files first;
then run the documented build, explicit migrations and startup commands with the
same installation descriptor and Compose project. There is no installer or launcher.
The initializer is required for an empty or restored server Pi-state bind. It atomically creates
the three regular targets hidden below the writable parent bind; protected Pi auth and tracked
model/settings sources remain separate read-only mounts. See the server manual before substituting
a root other than `/srv/thothii/pi-state`.
The mandatory stack includes frontend, core, PostgreSQL catalog, Qdrant, Ollama
and the embedding initializer. DWH and LLM endpoints remain external dependencies.
Pi is included in the core image. Credentials and certificates belong in protected
installation-local files, never in the workspace repository.
Workspace descriptors come from the Git remote configured by `THT_WORKSPACE_GIT_REMOTE`; their
runtime endpoint and secret bindings remain installation-local. Open
<http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in `deploy/env/local.env` to choose another
loopback port).
Credentials and certificates are local protected files. Do not put them in environment examples,
workspace YAML, URLs, or Compose interpolation values.
Application state is split across the named `settings`, `pi-state`, `workspace-registry`,
`sessions`, `qdrant-data`, and `embedding-models` volumes. `docker compose down` keeps them.
`qdrant-data` is a derived but persistent index store; `embedding-models` is an Ollama model
cache for `qwen3-embedding:0.6b` with fixed `1024`-dimension embeddings. Only an explicit destructive command such as `docker compose
down --volumes` removes them.
The frontend depends on the core health check and proxies `/health` and `/api/*` to it. The
application health endpoint intentionally checks process readiness only; external dependency
diagnostics are exposed by `tht doctor` and do not prevent the UI from starting.
For developer topology, overlays and lifecycle details, see the internal
[Compose reference](docs/operations/compose-reference.md). Ordinary stop/down keeps
persistent data; removing volumes is destructive and is not an upgrade step.
Process health is distinct from external dependency checks performed by doctor.
## Git-backed workspace repository
+18 -1
View File
@@ -7,6 +7,7 @@ import { fileURLToPath } from "node:url";
import { tmpdir } from "node:os";
import type { AppConfig } from "./config.js";
import { ThtRunner } from "./tht/tht-runner.js";
import { createMemoryCleanup } from "./catalog/memory-cleanup.js";
import { PiProcessManager } from "./pi/pi-process-manager.js";
import { SseHub } from "./sse/sse-hub.js";
import { authenticateSession, captureAuthConfigSnapshot, configuredOrigin } from "./auth/auth.js";
@@ -80,6 +81,8 @@ import { createProductionWorkspacePreprocessingService } from "./workspace-maint
import type { WorkspacePreprocessingService } from "./workspaces/preprocessing-service.js";
import { PreprocessingStateStore } from "./workspaces/preprocessing-state.js";
import { workspacePreprocessingRoutes } from "./routes/workspace-preprocessing.js";
import { memoryRoutes } from "./routes/memory.js";
import { evidenceRoutes } from "./routes/evidence.js";
export interface BuildAppDeps {
thtRunner?: ThtRunner;
@@ -93,7 +96,7 @@ export interface BuildAppDeps {
workspaceDiagnoser?: WorkspaceDiagnoser;
workspaceDatabaseTester?: WorkspaceDatabaseTester;
workspaceSecretStore?: WorkspaceSecretStore;
workspacePreprocessingService?: Pick<WorkspacePreprocessingService, "run" | "clear">;
workspacePreprocessingService?: Pick<WorkspacePreprocessingService, "run" | "clear"> & Partial<Pick<WorkspacePreprocessingService, "consolidateEvidence" | "evidenceSources">>;
catalogRepository?: CatalogRepository;
catalogService?: CatalogService;
catalogPostgresAccess?: CatalogPostgresAccess;
@@ -265,6 +268,11 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
catalogSchemaIntrospector,
catalogOperationCoordinator,
config.catalogSyncTimeoutMs,
createMemoryCleanup(tht as ThtRunner, {
internalQdrantUrl: config.internalQdrantUrl, internalEmbeddingUrl: config.internalEmbeddingUrl,
internalEmbeddingId: config.internalEmbeddingId, internalEmbeddingModel: config.internalEmbeddingModel,
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
}),
);
app.addHook("onReady", async () => { await catalogSyncWorker.initialize(); });
app.addHook("onReady", async () => { await descriptionGenerationWorker.initialize(); });
@@ -493,7 +501,16 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
return maintenanceBarrier.status();
});
sqlRoutes(app, { tht: tht as ThtRunner, getSettings, workspaceRegistry });
memoryRoutes(app, { runner: tht as ThtRunner, registry: workspaceRegistry, runtime: {
internalQdrantUrl: config.internalQdrantUrl,
internalEmbeddingUrl: config.internalEmbeddingUrl,
internalEmbeddingModel: config.internalEmbeddingModel,
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
} });
metaRoutes(app, { harnessDir: config.harnessDir, modelCatalog: runtimeModelCatalog });
evidenceRoutes(app, { runner: tht as ThtRunner, registry: workspaceRegistry,
registryRoot: config.workspaceRegistry.root, hostRegistryRoot: config.evidenceHostRegistryRoot,
service: workspacePreprocessingService });
workspaceRoutes(app, {
registry: workspaceRegistry,
config: config.workspaceRegistry,
+3 -1
View File
@@ -119,7 +119,9 @@ export function authenticateSession(deps: AuthDependencies): preHandlerHookHandl
subject: session.subject,
...(session.displayName === undefined ? {} : { displayName: session.displayName }),
roles: session.roles,
permissions: session.permissions,
// Sessions can outlive a deployment that changes the role permission catalog.
// resolve() has already checked validity, including current local user roles.
permissions: rolesToPermissions(session.roles),
isAdmin: session.roles.includes("admin"),
};
if (STATE_CHANGING_METHODS.has(request.method)) {
+1 -1
View File
@@ -38,7 +38,7 @@ const MAX_MAPPED_GROUPS = 128;
const ROLES = ["user", "admin"] as const;
export const PERMISSION_CATALOG: readonly Permission[] = [
"session.use", "session.read_all", "session.manage_all", "settings.manage",
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
];
const invalid = (): Error => new Error("authentication configuration is invalid");
+1 -1
View File
@@ -33,7 +33,7 @@ const EMPTY_HKDF_SALT = Buffer.alloc(0);
const ROLES = ["user", "admin"] as const;
const PERMISSIONS = [
"session.use", "session.read_all", "session.manage_all", "settings.manage",
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
] as const satisfies readonly Permission[];
const invalid = (): Error => new Error("auth_session_store_invalid");
+1 -1
View File
@@ -5,7 +5,7 @@ export type Role = "user" | "admin";
export type Permission =
| "session.use" | "session.read_all" | "session.manage_all"
| "settings.manage" | "workspace.manage" | "workspace.secrets.manage"
| "database.manage" | "pi.manage" | "auth.diagnostics.read";
| "database.manage" | "memory.manage" | "evidence.manage" | "pi.manage" | "auth.diagnostics.read";
export interface AuthenticationSessionConfig {
regularTtlSeconds: number;
+23
View File
@@ -0,0 +1,23 @@
import type { ThtRunner } from "../tht/tht-runner.js";
import type { SemanticRuntimeConfig } from "../workspaces/runtime-renderer.js";
import type { CatalogSyncRun, WorkspaceDatabase } from "./types.js";
/** Internal continuation of an applied physical sync, using the harness Memory boundary. */
export function createMemoryCleanup(runner: Pick<ThtRunner, "withPrincipal">, runtime: SemanticRuntimeConfig) {
return async (database: WorkspaceDatabase, run: CatalogSyncRun): Promise<number> => {
if (run.phase !== "memory_cleanup" || !run.plannedDiff) throw new Error("Physical cleanup is not committed");
const result = await runner.withPrincipal({ issuer: "installation", subject: "catalog-sync",
roles: ["admin"], permissions: ["memory.manage"], isAdmin: true,
}).runWithRuntimeSnapshot(["memory", "admin", "--workspace", database.workspaceId], JSON.stringify({
action: "cleanup", runtime, request: { sync_id: run.id, database: database.databaseName,
schema_name: database.schema, removed_tables: run.plannedDiff.deletedTables,
removed_columns: run.plannedDiff.deletedColumns.map(column => ({ table: column.tableName, column: column.columnName })),
},
}));
const payload = JSON.parse(result.stdout);
if (result.code !== 0 || payload.indexed !== true || !Number.isInteger(payload.deleted) || payload.deleted < 0) {
throw new Error("Memory cleanup is incomplete");
}
return payload.deleted;
};
}
+3 -1
View File
@@ -926,6 +926,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
scope: CatalogSyncScope,
tableIds: readonly string[],
snapshot: ObservedSchemaSnapshot,
syncRunId?: string,
): Promise<CatalogSyncCounts | undefined> {
const database = this.records.get(databaseId);
if (!database || database.version !== expectedDatabaseVersion) return undefined;
@@ -1141,6 +1142,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
if (scope === "all") {
this.records.set(databaseId, { ...database, schemaSyncedVersion: expectedDatabaseVersion, schemaSyncedAt: now });
}
if (syncRunId) await this.updateSyncRun(syncRunId, { phase: "memory_cleanup" });
return {
tables: (await this.listTables(databaseId)).length,
columns: [...this.columns.values()].filter((column) => this.tables.get(column.tableId)?.databaseId === databaseId).length,
@@ -1254,7 +1256,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
for (const run of this.syncRuns.values()) {
if (["queued", "running", "awaiting_confirmation", "applying"].includes(run.state)) {
await this.updateSyncRun(run.id, {
state: "interrupted", phase: "completed", finishedAt: new Date().toISOString(),
state: "interrupted", phase: run.phase === "memory_cleanup" ? "memory_cleanup" : "completed", finishedAt: new Date().toISOString(),
errorCode: "SYNC_INTERRUPTED", errorMessage: "Synchronization was interrupted by a service restart",
});
}
@@ -102,7 +102,7 @@ export function loadMetadataGenerationModels(options: {
...(apiKeyEnv ? { apiKeyEnv, apiKey } : {}),
}));
}
const result = new RestartLoadedMetadataGenerationModels(models, catalog.defaultMetadataGeneration);
const result = new RestartLoadedMetadataGenerationModels(models, models.size ? catalog.defaultInteraction : null);
const safe = result.catalog();
return {
catalog: () => ({
+6 -1
View File
@@ -1604,6 +1604,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
scope: CatalogSyncScope,
tableIds: readonly string[],
snapshot: ObservedSchemaSnapshot,
syncRunId?: string,
): Promise<CatalogSyncCounts | undefined> {
return await this.db.transaction().execute(async (trx) => {
const database = await trx.selectFrom("workspaceDatabases").select("version")
@@ -1779,6 +1780,10 @@ export class KyselyCatalogRepository implements CatalogRepository {
schemaSyncedAt: now,
}).where("id", "=", databaseId).execute();
}
if (syncRunId) {
await trx.updateTable("catalogSyncRuns").set({ phase: "memory_cleanup" })
.where("id", "=", syncRunId).where("databaseId", "=", databaseId).execute();
}
return {
tables: snapshot.tables.length,
columns: scope === "tables" ? undefined : snapshot.columns.length,
@@ -1882,7 +1887,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
async interruptActiveSyncRuns(): Promise<void> {
await this.db.updateTable("catalogSyncRuns").set({
state: "interrupted", phase: "completed", errorCode: "worker_restarted",
state: "interrupted", phase: sql`case when phase='memory_cleanup' then phase else 'completed' end`, errorCode: "worker_restarted",
errorMessage: "Synchronization was interrupted by a backend restart.",
finishedAt: sql`now()`, updatedAt: sql`now()`,
leaseOwner: null, leaseExpiresAt: null,
+46 -19
View File
@@ -58,6 +58,8 @@ export class CatalogSyncWorker {
private readonly introspector: CatalogSchemaIntrospector,
private readonly operations: CatalogOperationCoordinator,
private readonly timeoutMs: number,
private readonly cleanupMemory: (database: WorkspaceDatabase, run: CatalogSyncRun) => Promise<number>
= async () => 0,
) {}
async initialize(): Promise<void> {
@@ -67,6 +69,9 @@ export class CatalogSyncWorker {
}
async start(database: WorkspaceDatabase, scope: CatalogSyncScope, tableIds: readonly string[]): Promise<CatalogSyncRun> {
if ((await this.repository.listSyncRuns(database.id, 100)).some(run => run.phase === "memory_cleanup")) {
throw new CatalogConflictError("Retry the pending Memory cleanup before starting another synchronization");
}
const uniqueTableIds = [...new Set(tableIds)];
if (scope === "columns") {
const tables = await Promise.all(uniqueTableIds.map((tableId) => this.repository.getTable(database.id, tableId)));
@@ -109,7 +114,7 @@ export class CatalogSyncWorker {
async cancel(runId: string): Promise<CatalogSyncRun | undefined> {
const run = await this.repository.getSyncRun(runId);
if (!run) return undefined;
if (run.state === "applying" || TERMINAL_STATES.has(run.state)) return run;
if (run.phase === "memory_cleanup" || run.state === "applying" || TERMINAL_STATES.has(run.state)) return run;
await this.repository.requestSyncRunCancellation(runId);
this.controllers.get(runId)?.abort();
if (run.state === "queued" || run.state === "awaiting_confirmation") {
@@ -138,6 +143,16 @@ export class CatalogSyncWorker {
}
const database = await this.repository.get(previous.databaseId);
if (!database) return undefined;
if (previous.phase === "memory_cleanup") {
const release = this.operations.reserve(database.id);
try {
const queued = await this.repository.updateSyncRun(runId, { state: "queued", cancelRequested: false,
errorCode: null, errorMessage: null, finishedAt: null, leaseOwner: null, leaseExpiresAt: null });
this.reservations.set(runId, release);
this.launch(runId);
return queued;
} catch (error) { release(); throw error; }
}
return await this.start(database, previous.scope, previous.tableIds);
}
@@ -179,6 +194,10 @@ export class CatalogSyncWorker {
if (!database || database.version !== claimed.requestedDatabaseVersion) {
throw new CatalogConflictError("Database binding changed before synchronization started");
}
if (claimed.phase === "memory_cleanup") {
await this.finishMemoryCleanup(database, claimed);
return;
}
const progress: CatalogSchemaScanProgress = async (phase, counts) => {
await this.checkCancelled(runId);
await this.repository.updateSyncRun(runId, {
@@ -230,36 +249,30 @@ export class CatalogSyncWorker {
claimed.scope,
claimed.tableIds,
snapshot,
runId,
);
if (!applied) throw new CatalogConflictError("Database binding changed before schema changes were applied");
await this.repository.updateSyncRun(runId, {
state: "succeeded",
phase: "completed",
counts: applied,
finishedAt: new Date().toISOString(),
observedSnapshot: null,
plannedDiff: null,
confirmationToken: null,
heartbeatAt: new Date().toISOString(),
leaseOwner: null,
leaseExpiresAt: null,
});
await this.repository.appendSyncEvent(runId, "info", "succeeded", "Synchronization completed.", { ...applied });
this.release(runId);
const cleanupRun = await this.repository.updateSyncRun(runId, { counts: applied });
if (!cleanupRun) throw new Error("Synchronization run disappeared");
await this.finishMemoryCleanup(database, cleanupRun);
/* Completion is recorded only after the durable Memory cleanup succeeds. */
return;
} catch (error) {
const current = await this.repository.getSyncRun(runId);
const cancelled = !timedOut && (error instanceof SyncCancelledError || controller.signal.aborted || current?.cancelRequested);
const failure = timedOut
const failure = current?.phase === "memory_cleanup"
? { code: "memory_cleanup_pending", message: "Catalog synchronized. Memory cleanup is pending; retry this synchronization to complete it." }
: timedOut
? { code: "schema_sync_timed_out", message: "Schema synchronization timed out." }
: safeFailure(error);
await this.repository.updateSyncRun(runId, {
state: cancelled ? "cancelled" : "failed",
phase: "completed",
phase: current?.phase === "memory_cleanup" ? "memory_cleanup" : "completed",
errorCode: cancelled ? null : failure.code,
errorMessage: cancelled ? null : failure.message,
finishedAt: new Date().toISOString(),
observedSnapshot: null,
plannedDiff: null,
observedSnapshot: current?.phase === "memory_cleanup" ? current.observedSnapshot : null,
plannedDiff: current?.phase === "memory_cleanup" ? current.plannedDiff : null,
confirmationToken: null,
leaseOwner: null,
leaseExpiresAt: null,
@@ -278,6 +291,20 @@ export class CatalogSyncWorker {
}
}
private async finishMemoryCleanup(database: WorkspaceDatabase, run: CatalogSyncRun): Promise<void> {
if (!run.plannedDiff || run.phase !== "memory_cleanup") throw new Error("Missing committed cleanup context");
await this.repository.appendSyncEvent(run.id, "info", "memory_cleanup", "Removing Memory cards with deleted physical dependencies.");
const memoryDeleted = await this.cleanupMemory(database, run);
const counts = { ...run.counts, memoryDeleted };
await this.repository.updateSyncRun(run.id, {
state: "succeeded", phase: "completed", counts, finishedAt: new Date().toISOString(),
observedSnapshot: null, plannedDiff: null, confirmationToken: null,
heartbeatAt: new Date().toISOString(), leaseOwner: null, leaseExpiresAt: null,
});
await this.repository.appendSyncEvent(run.id, "info", "succeeded", "Synchronization completed.", counts);
this.release(run.id);
}
private assertCapability(scope: CatalogSyncScope, snapshot: ObservedSchemaSnapshot): void {
const required = scope === "all" ? ["tables", "columns", "relationships"] as const : [scope] as const;
for (const name of required) {
+3 -1
View File
@@ -366,7 +366,7 @@ export type CatalogSyncState =
export type CatalogSyncPhase =
| "queued" | "connecting" | "scanning_tables" | "scanning_columns"
| "scanning_relationships" | "planning" | "awaiting_confirmation"
| "applying" | "completed";
| "applying" | "memory_cleanup" | "completed";
export interface CatalogSchemaDiff {
deletedTables: string[];
@@ -375,6 +375,7 @@ export interface CatalogSchemaDiff {
}
export interface CatalogSyncCounts {
memoryDeleted?: number;
tables?: number;
columns?: number;
relationships?: number;
@@ -587,6 +588,7 @@ export interface CatalogRepository {
scope: CatalogSyncScope,
tableIds: readonly string[],
snapshot: ObservedSchemaSnapshot,
syncRunId?: string,
): Promise<CatalogSyncCounts | undefined>;
createSyncRun(
databaseId: string,
+5
View File
@@ -30,8 +30,11 @@ export interface AppConfig {
dataRoot?: string;
ollamaEnsureTimeoutMs: number;
piManagementTimeoutMs: number;
/** Host CLI platform projected into Docker; not the browser or container OS. */
hostPlatform?: string;
secretsFile?: string;
installationConfigFile?: string;
evidenceHostRegistryRoot?: string;
modelCatalogFile?: string;
sensitivityNer?: {
pythonExecutable: string;
@@ -474,8 +477,10 @@ export function loadConfig(
dataRoot: env.THT_DATA_ROOT,
ollamaEnsureTimeoutMs: Number(env.OLLAMA_ENSURE_TIMEOUT_MS ?? 60000),
piManagementTimeoutMs: piManagementTimeout(env.PI_MANAGEMENT_TIMEOUT_MS),
hostPlatform: env.THT_HOST_PLATFORM,
secretsFile,
installationConfigFile,
evidenceHostRegistryRoot: env.THT_EVIDENCE_HOST_REGISTRY_ROOT || undefined,
modelCatalogFile,
sensitivityNer,
piAuthFile,
+18 -17
View File
@@ -41,15 +41,17 @@ const runtimeModelSchema = z.object({
supportsReasoningEffort: z.boolean(),
supportsStore: z.boolean(),
maxTokensField: z.string().optional(),
thinkingFormat: z.enum(["qwen", "qwen-chat-template"]).optional(),
}).strict().optional(),
}).strict().optional(),
}).strict().refine((session) => !session.compatibility?.thinkingFormat || session.reasoning, {
message: "thinkingFormat requires reasoning: true",
}).optional(),
metadataGeneration: z.object({ disableThinking: z.boolean() }).strict().optional(),
}).strict();
const catalogSchema = z.object({
schemaVersion: z.literal(1),
defaultSession: canonicalId,
defaultMetadataGeneration: canonicalId.optional(),
schemaVersion: z.literal(2),
defaultInteraction: canonicalId,
embedding: z.object({ id: canonicalId, dimensions: z.number().int().positive() }).strict(),
models: z.array(runtimeModelSchema).max(64),
}).strict();
@@ -57,8 +59,7 @@ const catalogSchema = z.object({
export type RuntimeModel = z.infer<typeof runtimeModelSchema>;
export interface RuntimeModelCatalog {
readonly defaultSession: string | null;
readonly defaultMetadataGeneration: string | null;
readonly defaultInteraction: string | null;
readonly embedding: Readonly<{ id: string; dimensions: number }> | null;
sessionModels(): readonly RuntimeModel[];
metadataModels(): readonly RuntimeModel[];
@@ -66,19 +67,21 @@ export interface RuntimeModelCatalog {
}
class RestartLoadedRuntimeModelCatalog implements RuntimeModelCatalog {
readonly defaultSession: string | null;
readonly defaultMetadataGeneration: string | null;
readonly defaultInteraction: string | null;
readonly embedding: Readonly<{ id: string; dimensions: number }> | null;
readonly #sessions: readonly RuntimeModel[];
readonly #metadata: readonly RuntimeModel[];
readonly #sessionIds: ReadonlySet<string>;
constructor(catalog?: z.infer<typeof catalogSchema>) {
this.defaultSession = catalog?.defaultSession ?? null;
this.defaultMetadataGeneration = catalog?.defaultMetadataGeneration ?? null;
this.defaultInteraction = catalog?.defaultInteraction ?? null;
this.embedding = catalog ? Object.freeze({ ...catalog.embedding }) : null;
this.#sessions = Object.freeze((catalog?.models ?? []).filter((model) => model.session !== undefined));
this.#metadata = Object.freeze((catalog?.models ?? []).filter((model) => model.metadataGeneration !== undefined));
// One operational list. Session-only installations can still run Core, but once Admin
// LLM models are configured every selectable model must support both adapters.
const hasMetadata = catalog?.models.some((model) => model.metadataGeneration !== undefined);
this.#sessions = Object.freeze((catalog?.models ?? []).filter((model) =>
model.session !== undefined && (!hasMetadata || model.metadataGeneration !== undefined)));
this.#metadata = Object.freeze(this.#sessions.filter((model) => model.metadataGeneration !== undefined));
this.#sessionIds = new Set(this.#sessions.map((model) => model.id));
}
@@ -129,11 +132,9 @@ export function loadRuntimeModelCatalog(file?: string): RuntimeModelCatalog {
if (ids.size !== parsed.data.models.length) throw new Error("runtime model catalog contains duplicate models");
const sessions = parsed.data.models.filter((model) => model.session !== undefined).map((model) => model.id);
const metadata = parsed.data.models.filter((model) => model.metadataGeneration !== undefined).map((model) => model.id);
if (!sessions.includes(parsed.data.defaultSession)) throw new Error("runtime model catalog session default is invalid");
if ((metadata.length > 0) !== (parsed.data.defaultMetadataGeneration !== undefined)
|| (parsed.data.defaultMetadataGeneration !== undefined
&& !metadata.includes(parsed.data.defaultMetadataGeneration))) {
throw new Error("runtime model catalog metadata default is invalid");
if (!sessions.includes(parsed.data.defaultInteraction)
|| (metadata.length > 0 && !metadata.includes(parsed.data.defaultInteraction))) {
throw new Error("runtime model catalog interaction default is invalid");
}
return new RestartLoadedRuntimeModelCatalog(parsed.data);
}
+14 -9
View File
@@ -88,17 +88,22 @@ async function workflowDiagnostics(config: AppConfig): Promise<{ ready: true; wo
if (revisions.length === 0) throw new Error("workflow diagnostics unavailable");
return await withOperatorRunner(config, async (runner) => {
for (const revision of revisions) {
const result = await runner.run(["doctor", "--json"], revision.snapshotPath);
let payload: unknown;
const runtime = await runner.acquireWorkspaceRuntime(revision.snapshotPath);
try {
payload = JSON.parse(result.stdout);
} catch {
throw new Error("workflow diagnostics failed");
const result = await runner.run(["doctor", "--json"], runtime.path);
let payload: unknown;
try {
payload = JSON.parse(result.stdout);
} catch {
throw new Error("workflow diagnostics failed");
}
if (
result.code !== 0 || !payload || typeof payload !== "object"
|| (payload as { ok?: unknown }).ok !== true
) throw new Error("workflow diagnostics failed");
} finally {
runtime.release();
}
if (
result.code !== 0 || !payload || typeof payload !== "object"
|| (payload as { ok?: unknown }).ok !== true
) throw new Error("workflow diagnostics failed");
}
return { ready: true, workspaces: revisions.length };
});
+9
View File
@@ -11,6 +11,7 @@ import {
validateDeclarativePiConfig,
} from "./managed-config.js";
import type { RuntimeModelCatalog } from "../models/runtime-model-catalog.js";
import { secretValue } from "../config/secret-bundle.js";
export interface PiModel {
provider: string;
@@ -64,6 +65,14 @@ export function createPiModelLister(cfg: AppConfig, opts: Opts = {}): ListModels
}
const env = buildPiChildEnv({});
// Pi's availability enumeration also needs the catalog-owned credentials for built-in
// providers. It must keep working after their obsolete Pi auth entries are removed.
for (const model of opts.modelCatalog?.sessionModels() ?? []) {
const name = model.authentication.mode === "secret_env" ? model.authentication.apiKeyEnv : undefined;
if (!name) continue;
const value = secretValue(cfg, name);
if (value) env[name] = value;
}
delete env.THT_DATA_ROOT;
if (cfg.dataRoot !== undefined) env.THT_DATA_ROOT = cfg.dataRoot;
const child = spawnFn(cfg.piBin, ["--mode", "rpc"], { cwd: cfg.harnessDir, env });
+13 -2
View File
@@ -138,7 +138,7 @@ export interface PiRuntimeAgentSnapshot {
* Bind a session Pi process to the exact managed auth/model bytes validated at spawn time.
* Other agent resources remain live through symlinks, while session storage stays persistent.
*/
export function createPiRuntimeAgentSnapshot(): PiRuntimeAgentSnapshot {
export function createPiRuntimeAgentSnapshot(options: { excludeAuthProvider?: string } = {}): PiRuntimeAgentSnapshot {
const sourceAgentDir = configuredPiAgentDir();
const auth = readPiAgentFile(sourceAgentDir, "auth.json", true);
const models = readPiAgentFile(sourceAgentDir, "models.json", true);
@@ -165,7 +165,18 @@ export function createPiRuntimeAgentSnapshot(): PiRuntimeAgentSnapshot {
);
}
if (auth !== undefined) {
writeFileSync(join(snapshotDir, "auth.json"), auth, { flag: "wx", mode: 0o600 });
let effectiveAuth = auth;
if (options.excludeAuthProvider) {
const parsed = parsePiConfigJson(auth);
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) throw new PiManagedConfigError();
const provider = options.excludeAuthProvider.trim().toLowerCase();
effectiveAuth = JSON.stringify(Object.fromEntries(
Object.entries(parsed).filter(([key]) => key.trim().toLowerCase() !== provider),
));
}
// secret_env is authoritative for this provider. Keep the operator's auth file intact,
// but do not let an old Pi credential override the shared bundle inside this child.
writeFileSync(join(snapshotDir, "auth.json"), effectiveAuth, { flag: "wx", mode: 0o600 });
}
if (models !== undefined) {
writeFileSync(join(snapshotDir, "models.json"), models, { flag: "wx", mode: 0o600 });
+14 -10
View File
@@ -36,6 +36,7 @@ export interface PiInstallationConfig {
}
export interface PiStatus {
hostPlatform: "linux" | "macos" | "windows";
version?: string;
ready: boolean;
credentials: PiCredentialStatus;
@@ -92,6 +93,9 @@ interface PiManagementDeps {
}
export function createPiManagement(config: AppConfig, deps: PiManagementDeps): PiManagementService {
const platform = config.hostPlatform ?? process.platform;
const hostPlatform = platform === "darwin" ? "macos"
: platform === "windows" || platform === "win32" ? "windows" : "linux";
const now = deps.now ?? (() => new Date());
const diagnostics: string[] = [];
const addDiagnostic = (message: string): void => {
@@ -106,23 +110,23 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
});
const credentialStatus = deps.credentialStatus ?? ((provider: string | undefined) => {
try {
const model = deps.modelCatalog.defaultSession
? deps.modelCatalog.sessionModels().find((entry) => entry.id === deps.modelCatalog.defaultSession)
const model = deps.modelCatalog.defaultInteraction
? deps.modelCatalog.sessionModels().find((entry) => entry.id === deps.modelCatalog.defaultInteraction)
: undefined;
const credentialName = model?.authentication.mode === "secret_env"
? model.authentication.apiKeyEnv
: undefined;
const configuredApiKey = configuredPiProviderApiKey(
const configuredApiKey = credentialName ? `$${credentialName}` : configuredPiProviderApiKey(
readConfiguredPiAgentFile("models.json", true),
provider,
) ?? (credentialName ? `$${credentialName}` : undefined);
);
return piProviderCredentialStatus({
provider,
authProviders: loadPiAuthProviders(),
authProviders: credentialName ? new Set() : loadPiAuthProviders(),
resolveCredentialValue: () => credentialName
? secretValue(config, credentialName)
: config.modelCatalogFile ? undefined : secretValue(config, "THT_MODEL_API_KEY"),
credentialFile: config.modelApiKeyFile,
credentialFile: credentialName ? undefined : config.modelApiKeyFile,
configuredApiKey,
});
} catch {
@@ -152,8 +156,8 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
const installationConfig = (): PiInstallationConfig => {
const settings = readSettings();
const reasoning = config.defaults.thinking ?? settings.thinking;
const selected = deps.modelCatalog.defaultSession
? splitCanonicalModelId(deps.modelCatalog.defaultSession)
const selected = deps.modelCatalog.defaultInteraction
? splitCanonicalModelId(deps.modelCatalog.defaultInteraction)
: undefined;
return {
...(selected ? selected : {}),
@@ -169,11 +173,11 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
try {
const currentVersion = await version();
addDiagnostic("Pi version probe succeeded");
return { version: currentVersion, ready: true, credentials, config: current, checkedAt };
return { hostPlatform, version: currentVersion, ready: true, credentials, config: current, checkedAt };
} catch (error) {
const message = stableMessage(error, "Pi runtime is unavailable");
addDiagnostic(message);
return { ready: false, credentials, config: current, checkedAt, message };
return { hostPlatform, ready: false, credentials, config: current, checkedAt, message };
}
},
+30 -19
View File
@@ -25,6 +25,7 @@ export interface SessionRuntime {
}
export interface RuntimeOptions {
interactionLanguage?: string | null;
provider?: string;
model?: string;
thinking?: string;
@@ -48,6 +49,7 @@ export class PiProcessManager {
private spawnFn: (
sessionId: string, author: string, provider: string | undefined, model: string | undefined,
principal?: PrincipalContext, runtimeConfigPath?: string,
interactionLanguage?: string | null,
) => ChildProcessWithoutNullStreams;
private loadAuthProviders: (agentDir: string) => ReadonlySet<string>;
private modelCatalog: RuntimeModelCatalog;
@@ -63,15 +65,15 @@ export class PiProcessManager {
) {
this.modelCatalog = opts?.modelCatalog ?? loadRuntimeModelCatalog(cfg.modelCatalogFile);
this.modelCatalogConfigured = cfg.modelCatalogFile !== undefined
|| this.modelCatalog.defaultSession !== null;
|| this.modelCatalog.defaultInteraction !== null;
this.loadAuthProviders = opts?.authProviders
?? ((agentDir) => loadPiAuthProviders({ agentDir }));
if (opts?.spawnFn) {
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath) =>
this.spawnPi(opts.spawnFn!, sessionId, author, provider, model, principal, runtimeConfigPath);
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath, language) =>
this.spawnPi(opts.spawnFn!, sessionId, author, provider, model, principal, runtimeConfigPath, language);
} else {
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath) =>
this.spawnPi(nodeSpawn, sessionId, author, provider, model, principal, runtimeConfigPath);
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath, language) =>
this.spawnPi(nodeSpawn, sessionId, author, provider, model, principal, runtimeConfigPath, language);
}
}
@@ -85,33 +87,35 @@ export class PiProcessManager {
private spawnPi(
spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined,
model: string | undefined, principal?: PrincipalContext, runtimeConfigPath?: string,
interactionLanguage?: string | null,
): ChildProcessWithoutNullStreams {
// This is the final shared boundary for createFor(), spawnFor(), and resume(). Validate
// before auth-provider inspection, then make Pi consume the exact copied bytes rather than
// reopening mutable mounted auth/models files after this check.
const agent = createPiRuntimeAgentSnapshot();
const catalogModel = provider && model
? this.modelCatalog.sessionModels().find((entry) => entry.provider === provider && entry.model === model)
: undefined;
const credentialName = catalogModel?.authentication.mode === "secret_env"
? catalogModel.authentication.apiKeyEnv : undefined;
const agent = createPiRuntimeAgentSnapshot({ excludeAuthProvider: credentialName ? provider : undefined });
let child: ChildProcessWithoutNullStreams | undefined;
try {
const catalogModel = provider && model
? this.modelCatalog.sessionModels()
.find((entry) => entry.provider === provider && entry.model === model)
: undefined;
const credentialName = catalogModel?.authentication.mode === "secret_env"
? catalogModel.authentication.apiKeyEnv
: undefined;
const projectedApiKey = configuredPiProviderApiKey(agent.models, provider)
?? (credentialName ? `$${credentialName}` : undefined);
const projectedApiKey = credentialName ? `$${credentialName}`
: configuredPiProviderApiKey(agent.models, provider);
const env = buildPiChildEnv({
provider,
authProviders: this.loadAuthProviders(agent.agentDir),
authProviders: credentialName ? new Set() : this.loadAuthProviders(agent.agentDir),
credentialValue: credentialName
? secretValue(this.cfg, credentialName)
: this.modelCatalogConfigured ? undefined : secretValue(this.cfg, "THT_MODEL_API_KEY"),
credentialFile: this.cfg.modelApiKeyFile,
credentialFile: credentialName ? undefined : this.cfg.modelApiKeyFile,
configuredApiKey: projectedApiKey,
additions: { THT_SESSION: sessionId, THT_AUTHOR: author },
});
env.PI_CODING_AGENT_DIR = agent.agentDir;
// A launch hint only: the gate reads the authoritative manifest before each turn.
delete env.THT_INTERACTION_LANGUAGE;
if (interactionLanguage) env.THT_INTERACTION_LANGUAGE = interactionLanguage;
env.PI_CODING_AGENT_SESSION_DIR = agent.sessionDir;
clearPrincipalEnvironment(env);
if (principal) Object.assign(env, principalEnvironment(principal));
@@ -189,7 +193,9 @@ export class PiProcessManager {
const model = o.model ?? this.cfg.defaults.model;
let child: ChildProcessWithoutNullStreams;
try {
child = this.spawnFn(sessionId, author, provider, model, o.principal, o.runtimeConfig?.path);
child = this.spawnFn(
sessionId, author, provider, model, o.principal, o.runtimeConfig?.path, o.interactionLanguage,
);
} catch (error) {
o.runtimeConfig?.release();
throw error;
@@ -298,8 +304,13 @@ export class PiProcessManager {
}
async resume(sessionId: string, tht: ThtRunner): Promise<SessionRuntime> {
const manifest = await tht.sessionShow(sessionId) as { provider?: string; model?: string; thinking?: string } | null;
const manifest = await tht.sessionShow(sessionId) as {
provider?: string; model?: string; thinking?: string; interaction_language?: string | null;
} | null;
const language = manifest?.interaction_language
?? (await tht.ensureInteractionLanguage(sessionId)).interaction_language;
return this.spawnFor(sessionId, {
interactionLanguage: language,
provider: manifest?.provider,
model: manifest?.model,
thinking: manifest?.thinking,
+6 -5
View File
@@ -70,28 +70,29 @@ export function createPiProviderSmoke(
try {
const canonicalProvider = canonicalPiProvider(provider);
if (!canonicalProvider || timeoutMs <= 0) throw providerFailure();
const configuredAuthProviders = authProviders();
const configuredAuthProviders = new Set(authProviders());
const configuredModels = options.readModelsStore
? options.readModelsStore()
: readConfiguredPiAgentFile("models.json", true);
const catalog = options.modelCatalog ?? loadRuntimeModelCatalog(config.modelCatalogFile);
const catalogConfigured = config.modelCatalogFile !== undefined
|| catalog.defaultSession !== null;
|| catalog.defaultInteraction !== null;
const catalogModel = catalog.sessionModels()
.find((entry) => entry.provider === canonicalProvider && entry.model === model);
const upstreamModel = catalogModel?.upstreamModel ?? model;
const credentialName = catalogModel?.authentication.mode === "secret_env"
? catalogModel.authentication.apiKeyEnv
: undefined;
const projectedApiKey = configuredPiProviderApiKey(configuredModels, canonicalProvider)
?? (credentialName ? `$${credentialName}` : undefined);
if (credentialName) configuredAuthProviders.delete(canonicalProvider);
const projectedApiKey = credentialName ? `$${credentialName}`
: configuredPiProviderApiKey(configuredModels, canonicalProvider);
const env = buildPiChildEnv({
provider: canonicalProvider,
authProviders: configuredAuthProviders,
credentialValue: credentialName
? secretValue(config, credentialName)
: catalogConfigured ? undefined : secretValue(config, "THT_MODEL_API_KEY"),
credentialFile: config.modelApiKeyFile,
credentialFile: credentialName ? undefined : config.modelApiKeyFile,
configuredApiKey: projectedApiKey,
});
clearPrincipalEnvironment(env);
+90
View File
@@ -0,0 +1,90 @@
import path from "node:path";
import type { FastifyInstance } from "fastify";
import { z } from "zod";
import { requirePermission, isPrincipalContext } from "../auth/authorization.js";
import type { ThtRunner } from "../tht/tht-runner.js";
import type { WorkspaceRegistry } from "../workspaces/registry.js";
import type { WorkspacePreprocessingService } from "../workspaces/preprocessing-service.js";
const params = z.object({ workspaceId: z.string().regex(/^[a-z][a-z0-9-]{2,62}$/),
evidenceId: z.string().regex(/^evidence:[a-z0-9]+(?:-[a-z0-9]+)*$/).optional() });
const query = z.object({
q: z.string().max(1000).optional(), kind: z.enum(["domain", "glossary", "enum", "example", "mapping", "normalization", "formula", "reference"]).optional(),
purpose: z.enum(["disambiguation", "rewriting", "schema_linking", "sql_generation"]).optional(),
status: z.enum(["new", "modified", "active", "removed", "review_required", "legacy", "invalid"]).optional(),
concept: z.string().max(300).optional(), table: z.string().max(300).optional(), column: z.string().max(300).optional(),
source: z.string().max(300).optional(), language: z.string().max(30).optional(),
sort: z.enum(["title", "id", "kind", "status"]).optional(), direction: z.enum(["asc", "desc"]).optional(),
page: z.coerce.number().int().positive().optional(), page_size: z.coerce.number().int().min(1).max(100).optional(),
}).strict();
const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`;
export function evidenceRoutes(app: FastifyInstance, deps: {
runner: Pick<ThtRunner, "withPrincipal">; registry: Pick<WorkspaceRegistry, "list">;
registryRoot: string; hostRegistryRoot?: string;
service: Partial<Pick<WorkspacePreprocessingService, "consolidateEvidence" | "evidenceSources">>;
}) {
app.post("/workspaces/:workspaceId/evidence/sources", async (request, reply) => {
const principal = requirePermission(request, reply, "evidence.manage");
if (!isPrincipalContext(principal)) return reply;
try {
const { workspaceId } = params.parse(request.params);
const body = z.discriminatedUnion("action", [
z.object({ action: z.literal("refresh") }).strict(),
z.object({ action: z.literal("decide"), sourceId: z.string().regex(/^[a-f0-9]{64}$/),
revision: z.string().regex(/^[a-f0-9]{64}$/), decision: z.enum(["keep", "replace"]) }).strict(),
]).parse(request.body);
if (!(await deps.registry.list()).some(w => w.id === workspaceId)) return reply.code(404).send({ code: "workspace_invalid" });
if (!deps.service.evidenceSources) throw new Error("Evidence service unavailable");
return await deps.service.evidenceSources({ workspaceId, ...body, actor: principal.subject });
} catch (error) {
return reply.code(error instanceof z.ZodError ? 400 : 503).send({ code: "evidence_unavailable",
message: error instanceof z.ZodError ? "Invalid source request." : "Evidence source service is unavailable." });
}
});
app.get<{ Params: { workspaceId: string; evidenceId?: string } }>("/workspaces/:workspaceId/evidence", read);
app.get<{ Params: { workspaceId: string; evidenceId?: string } }>("/workspaces/:workspaceId/evidence/:evidenceId", read);
async function read(request: import("fastify").FastifyRequest, reply: import("fastify").FastifyReply) {
const principal = requirePermission(request, reply, "evidence.manage");
if (!isPrincipalContext(principal)) return reply;
try {
const { workspaceId, evidenceId } = params.parse(request.params);
const filters = query.parse(request.query);
if (!(await deps.registry.list()).some(w => w.id === workspaceId)) return reply.code(404).send({ code: "workspace_invalid" });
const root = path.join(deps.registryRoot, "repo", workspaceId);
const result = await deps.runner.withPrincipal(principal).runWithRuntimeSnapshot(
["evidence", "admin", "--workspace", workspaceId],
JSON.stringify({ root, query: { ...filters, ...(evidenceId ? { id: evidenceId } : {}) } }),
);
const payload = JSON.parse(result.stdout);
if (result.code !== 0) return reply.code(503).send(payload);
if (evidenceId && !payload.item) return reply.code(404).send({ code: "evidence_not_found", message: "Evidence was not found." });
const host = deps.hostRegistryRoot;
const hostPath = host && (path.isAbsolute(host) || path.win32.isAbsolute(host))
? (path.win32.isAbsolute(host) && !path.isAbsolute(host) ? path.win32 : path).join(host, "repo") : null;
const hostJoin = hostPath && path.win32.isAbsolute(hostPath) && !path.isAbsolute(hostPath) ? path.win32.join : path.join;
return { ...payload, location: { repository: hostPath, workspace: hostPath ? hostJoin(hostPath, workspaceId) : null,
runtime_workspace: root, host: "Installation host", command: `tht workspace evidence consolidate --workspace ${workspaceId}`,
git_commands: hostPath ? [
`git -C ${quote(hostPath)} status --short -- ${quote(workspaceId + "/evidence")}`,
`git -C ${quote(hostPath)} diff -- ${quote(workspaceId + "/evidence")}`,
`git -C ${quote(hostPath)} add -A -- ${quote(workspaceId + "/evidence")}`,
`git -C ${quote(hostPath)} commit --only -m 'Curate Evidence' -- ${quote(workspaceId + "/evidence")}`,
`git -C ${quote(hostPath)} push`,
] : [] } };
} catch (error) {
return reply.code(error instanceof z.ZodError ? 400 : 503).send({ code: "evidence_unavailable",
message: error instanceof z.ZodError ? "Evidence filters are invalid." : "Evidence archive is unavailable." });
}
}
app.post("/workspaces/:workspaceId/evidence/consolidate", async (request, reply) => {
const principal = requirePermission(request, reply, "evidence.manage");
if (!isPrincipalContext(principal)) return reply;
try {
const { workspaceId } = params.parse(request.params);
if (!(await deps.registry.list()).some(w => w.id === workspaceId)) return reply.code(404).send({ code: "workspace_invalid" });
if (!deps.service.consolidateEvidence) throw new Error("Evidence service unavailable");
return await deps.service.consolidateEvidence({ workspaceId });
} catch { return reply.code(503).send({ code: "evidence_unavailable", message: "Evidence consolidation is unavailable." }); }
});
}
+94
View File
@@ -0,0 +1,94 @@
import type { FastifyInstance, FastifyReply, FastifyRequest } from "fastify";
import { z } from "zod";
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
import type { ThtRunner } from "../tht/tht-runner.js";
import type { WorkspaceRegistry } from "../workspaces/registry.js";
import type { SemanticRuntimeConfig } from "../workspaces/runtime-renderer.js";
const paramsSchema = z.object({
workspaceId: z.string().regex(/^[a-z][a-z0-9_-]{0,63}$/),
cardId: z.string().regex(/^mem-[0-9a-f-]{36}$/).optional(),
});
const family = z.enum(["domain_clarification", "sql_rule", "solved_question", "explained_error"]);
const cardSchema = z.object({
family, subject: z.string().trim().min(1).max(1000),
detail: z.string().max(50000).default(""), scope: z.string().trim().min(1).max(10000),
rationale: z.string().max(10000).default(""), question: z.string().max(10000).default(""),
sql: z.string().max(100000).default(""),
concepts: z.array(z.string().trim().min(1).max(200)).max(100).default([]),
dependencies: z.array(z.object({
database: z.string().trim().min(1).max(200), schema_name: z.string().max(200).default(""),
table: z.string().max(200).default(""), column: z.string().max(200).default(""),
}).strict()).max(200).default([]),
links: z.array(z.object({
target_id: z.string().min(1).max(100), meaning: z.string().trim().min(1).max(1000),
}).strict()).max(200).default([]),
}).strict();
const querySchema = z.object({
q: z.string().max(1000).optional(), family: family.optional(),
concept: z.string().max(200).optional(), database: z.string().max(200).optional(),
table: z.string().max(200).optional(), column: z.string().max(200).optional(),
origin: z.enum(["manual", "workflow"]).optional(),
updated_after: z.iso.datetime({ offset: true }).optional(),
updated_before: z.iso.datetime({ offset: true }).optional(),
page: z.coerce.number().int().positive().optional(),
page_size: z.coerce.number().int().min(1).max(100).optional(),
sort: z.enum(["updated_at", "created_at", "subject", "family"]).optional(),
direction: z.enum(["asc", "desc"]).optional(),
}).strict();
export function memoryRoutes(app: FastifyInstance, deps: {
runner: Pick<ThtRunner, "withPrincipal">;
registry: Pick<WorkspaceRegistry, "list" | "read">;
runtime: SemanticRuntimeConfig;
}) {
const invoke = (action: string) => async (request: FastifyRequest, reply: FastifyReply) => {
const principal = requirePermission(request, reply, "memory.manage");
if (!isPrincipalContext(principal)) return reply;
try {
const { workspaceId, cardId } = paramsSchema.parse(request.params);
const input = action === "list" ? querySchema.parse(request.query)
: action === "create" || action === "update"
? { card: cardSchema.parse(request.body), ...(cardId ? { id: cardId } : {}) }
: cardId ? { id: cardId } : {};
// Registry identity is enough: administrative browsing must not require a DWH binding.
const workspaces = await deps.registry.list();
if (!workspaces.some(workspace => workspace.id === workspaceId)) {
return reply.code(404).send({ code: "workspace_invalid", message: "Workspace was not found." });
}
const { workspace } = await deps.registry.read(workspaceId);
const result = await deps.runner.withPrincipal(principal).runWithRuntimeSnapshot(
["memory", "admin", "--workspace", workspaceId],
JSON.stringify({ action, request: input, runtime: {
...deps.runtime, memoryLanguage: workspace.workspace.language,
} }),
);
let payload;
try { payload = JSON.parse(result.stdout); } catch {
return reply.code(503).send({ code: "memory_unavailable", message: "Memory archive is unavailable." });
}
if (result.code !== 0) {
const codes: Record<string, number> = {
memory_invalid: 400, memory_forbidden: 403, memory_not_found: 404,
memory_conflict: 409, memory_unavailable: 503,
};
const code = typeof payload?.code === "string" && payload.code in codes
? payload.code : "memory_unavailable";
return reply.code(codes[code]).send({ code, message: "Memory operation could not be completed." });
}
return reply.code(action === "create" ? 201 : 200).send(payload);
} catch (error) {
if (error instanceof z.ZodError) {
return reply.code(400).send({ code: "memory_invalid", message: "Memory request is invalid." });
}
return reply.code(503).send({ code: "memory_unavailable", message: "Memory archive is unavailable." });
}
};
app.get("/workspaces/:workspaceId/memory", invoke("list"));
app.post("/workspaces/:workspaceId/memory", invoke("create"));
app.get("/workspaces/:workspaceId/memory/pending", invoke("pending"));
app.get("/workspaces/:workspaceId/memory/:cardId", invoke("show"));
app.put("/workspaces/:workspaceId/memory/:cardId", invoke("update"));
app.delete("/workspaces/:workspaceId/memory/:cardId", invoke("delete"));
app.post("/workspaces/:workspaceId/memory/:cardId/retry", invoke("retry"));
}
+97 -23
View File
@@ -13,6 +13,7 @@ import type { MaintenanceBarrier } from "../runtime/maintenance-gate.js";
import { hasPermission, isPrincipalContext, requirePermission } from "../auth/authorization.js";
import { splitCanonicalModelId, type RuntimeModelCatalog } from "../models/runtime-model-catalog.js";
import type { CatalogRepository } from "../catalog/types.js";
import { interactionLanguage } from "../tht/interaction-language.js";
const BOOTSTRAP_FAILURE_MESSAGE =
"Session startup failed. Check configuration and connectivity, then Resume the session.";
@@ -150,13 +151,34 @@ export function sessionRoutes(
});
app.addHook("onResponse", async (req) => { admissionLeases.get(req)?.(); });
type SessionRevisionScan = {
revisions: Awaited<ReturnType<typeof d.workspaceRegistry.list>>;
retainedComplete: boolean;
};
let retainedSnapshotWarning: string | null = null;
/** Include retained historical descriptors so removed workspaces remain resumable. */
const sessionRevisions = async () => {
const sessionRevisions = async (): Promise<SessionRevisionScan> => {
const registry = d.workspaceRegistry as Partial<WorkspaceRegistry>;
if (typeof registry.listRetainedSnapshots === "function") {
return await registry.listRetainedSnapshots();
try {
const revisions = await registry.listRetainedSnapshots();
retainedSnapshotWarning = null;
return { revisions, retainedComplete: true };
} catch (error) {
// A legacy/corrupt historical snapshot must not make active sessions (and their SSE
// reviewer gates) unreachable. The registry still rejects that snapshot; this fallback
// exposes only descriptors from the verified active state and deliberately disables
// retention reconciliation because the resulting session view is incomplete.
const detail = error instanceof Error ? error.message : "unknown error";
if (retainedSnapshotWarning !== detail) {
console.warn("[sessions] retained snapshot discovery failed; using active snapshots:", detail);
retainedSnapshotWarning = detail;
}
return { revisions: await d.workspaceRegistry.list(), retainedComplete: false };
}
}
return await d.workspaceRegistry.list();
return { revisions: await d.workspaceRegistry.list(), retainedComplete: true };
};
const isNotFound = (error: unknown) =>
@@ -200,7 +222,7 @@ export function sessionRoutes(
};
let revisions: Awaited<ReturnType<typeof d.workspaceRegistry.list>>;
try {
revisions = await sessionRevisions();
({ revisions } = await sessionRevisions());
} catch (registryError) {
// Sessions created before revision pinning still live under the installation's legacy
// default config. Keep that compatibility path available when a fresh installation has
@@ -360,8 +382,17 @@ export function sessionRoutes(
app.post("/sessions", async (req, reply) => {
const b = req.body as {
question: string; name?: string; workspace?: string; workspaceId?: string;
provider?: string; model?: string; thinking?: string;
provider?: string; model?: string; thinking?: string; interactionLanguage?: unknown;
};
const language = interactionLanguage(b?.interactionLanguage);
if (!language) return reply.code(400).send({
code: "invalid_interaction_language",
error: "interactionLanguage must be a well-formed BCP-47 language tag",
});
if ((b.provider === undefined) !== (b.model === undefined)
|| (b.provider !== undefined && (typeof b.provider !== "string" || typeof b.model !== "string" || !b.provider || !b.model))) {
return reply.code(400).send({ error: "provider and model must be supplied together" });
}
const principal = getPrincipal(req);
let s: Settings;
try { s = await d.getSettings(principal); } catch { return storageFailure(reply); }
@@ -425,11 +456,9 @@ export function sessionRoutes(
}
}
const requestedCanonical = b.provider && b.model ? `${b.provider}/${b.model}` : undefined;
let selectedCanonical = requestedCanonical ?? d.modelCatalog.defaultSession;
let modelWarning: string | undefined;
if (selectedCanonical && d.modelCatalog.defaultSession && !d.modelCatalog.hasSession(selectedCanonical)) {
selectedCanonical = d.modelCatalog.defaultSession;
modelWarning = `Configured model ${requestedCanonical ?? "selection"} is unavailable; using ${selectedCanonical}.`;
const selectedCanonical = requestedCanonical ?? d.modelCatalog.defaultInteraction;
if (selectedCanonical && d.modelCatalog.defaultInteraction && !d.modelCatalog.hasSession(selectedCanonical)) {
return reply.code(503).send({ error: MODEL_UNAVAILABLE_MESSAGE, code: "model_unavailable" });
}
const selected = selectedCanonical ? splitCanonicalModelId(selectedCanonical) : undefined;
const provider = selected?.provider ?? b.provider;
@@ -480,11 +509,15 @@ export function sessionRoutes(
// registry snapshot. The legacy fallback stays available for sessions created before
// the browser-local preference migration.
let id: string;
let sessionLanguage = language;
try {
({ id } = await runner.sessionNew({
const created = await runner.sessionNew({
question: b.question, name: b.name, workspaceConfigPath,
workspaceId, workspaceRevision, provider, model, thinking,
}));
interactionLanguage: language,
});
id = created.id;
sessionLanguage = created.interaction_language ?? language;
manifestPersisted = true;
if (revisionLease) {
await revisionLease.markPersisted().catch((error: unknown) => {
@@ -497,6 +530,7 @@ export function sessionRoutes(
} catch { return storageFailure(reply); }
const options = {
provider, model, thinking,
interactionLanguage: sessionLanguage,
author: principal.displayName ?? principal.subject,
principal,
question: b.question,
@@ -533,7 +567,7 @@ export function sessionRoutes(
),
() => d.mgr.start(id, rt, runtimeOptions),
);
return { id, ...(modelWarning ? { warning: modelWarning } : {}) };
return { id };
} finally {
if (revisionLease && !manifestPersisted) {
await revisionLease.abort().catch((error: unknown) => {
@@ -559,8 +593,8 @@ export function sessionRoutes(
? { ...principal, isAdmin: false }
: ownershipPrincipal(principal, "session.read_all");
const runner = runnerFor(scopedPrincipal);
const revisions = await sessionRevisions();
const lists = await Promise.all(revisions
const revisionScan = await sessionRevisions();
const lists = await Promise.all(revisionScan.revisions
.map((revision) => runner.sessionList(revision.snapshotPath) as Promise<SessionRow[]>));
const sessions = new Map<string, SessionRow>();
for (const row of lists.flat()) {
@@ -570,7 +604,8 @@ export function sessionRoutes(
// Only an administrator-visible complete list (or the single local principal) is safe
// input for retention. A remote per-user view can never discard another principal's pin.
const reconcileSnapshotRetention = (d.workspaceRegistry as Partial<WorkspaceRegistry>).reconcileSnapshotRetention;
const hasCompleteRetentionView = (scope === "all" || principal.issuer === "local")
const hasCompleteRetentionView = revisionScan.retainedComplete
&& (scope === "all" || principal.issuer === "local")
&& hasPermission(principal, "session.read_all");
if (hasCompleteRetentionView && typeof reconcileSnapshotRetention === "function") {
const retained = [...new Set(list
@@ -604,6 +639,23 @@ export function sessionRoutes(
} catch (error) { return lifecycleFailure(reply, error); }
const rt = d.mgr.get(id);
if (!rt) return reply.code(404).send({ error: "sessione non attiva" });
const pending = rt.bridge.pendingWidget?.() as any;
const response = (req.body as any)?.ui_response;
if (pending?.widget === "archive-repair" && response?.id === pending.id && !response.control) {
const choices = response.choices;
if (!Array.isArray(choices) || choices.length !== 1)
return reply.code(400).send({ error: "Select one archive repair choice" });
if (choices[0] !== "continue" && choices[0] !== "reject") {
const option = pending.repair?.options?.find((item: any) => item.id === choices[0]);
if (!option || !["memory", "evidence"].includes(option.archive))
return reply.code(400).send({ error: "Unknown archive repair choice" });
const permission = option.archive === "memory" ? "memory.manage" : "evidence.manage";
if (!principal.isAdmin || !hasPermission(principal, permission))
return reply.code(403).send({ error: "Archive corrections require an administrator" });
if (rt.ownerKey !== `${principal.issuer}\0${principal.subject}`)
return reply.code(409).send({ error: "Resume the session with your account before correcting an archive" });
}
}
if (!rt.bridge.respond((req.body as any).ui_response)) {
return reply.code(409).send({ error: "risposta non corrispondente al gate in attesa" });
}
@@ -623,6 +675,13 @@ export function sessionRoutes(
app.post("/sessions/:id/resume", async (req, reply) => {
const id = (req.params as any).id;
const principal = getPrincipal(req);
if (Object.hasOwn(req.body ?? {}, "interactionLanguage")
|| Object.hasOwn(req.body ?? {}, "interaction_language")) {
return reply.code(400).send({
code: "interaction_language_pinned",
error: "Resume uses the session's persisted interaction language; overrides are not accepted",
});
}
return withSessionLifecycle(id, async () => {
let settings: Settings;
let located: LocatedSession | undefined;
@@ -640,11 +699,19 @@ export function sessionRoutes(
const saved = manifest as {
provider?: string; model?: string; thinking?: string;
workspace_id?: string; workspace_revision?: string;
interaction_language?: string | null;
};
const savedCanonical = saved.provider && saved.model ? `${saved.provider}/${saved.model}` : "";
if (d.modelCatalog.defaultSession && (!savedCanonical || !d.modelCatalog.hasSession(savedCanonical))) {
const requested = (req.body ?? {}) as { provider?: string; model?: string; thinking?: string };
if ((requested.provider === undefined) !== (requested.model === undefined)
|| (requested.provider !== undefined && (typeof requested.provider !== "string" || typeof requested.model !== "string" || !requested.provider || !requested.model))) {
return reply.code(400).send({ error: "provider and model must be supplied together" });
}
const selectedCanonical = requested.provider && requested.model
? `${requested.provider}/${requested.model}` : d.modelCatalog.defaultInteraction;
if (d.modelCatalog.defaultInteraction && (!selectedCanonical || !d.modelCatalog.hasSession(selectedCanonical))) {
return reply.code(503).send({ error: MODEL_UNAVAILABLE_MESSAGE, code: "model_unavailable" });
}
const selected = selectedCanonical ? splitCanonicalModelId(selectedCanonical) : saved;
let workspaceConfigPath: string;
let workspaceDescriptor: WorkspaceDescriptor | undefined;
try {
@@ -671,13 +738,19 @@ export function sessionRoutes(
});
}
try { settings = await d.getSettings(principal); } catch { return storageFailure(reply); }
let language = saved.interaction_language;
if (language == null) {
try {
language = (await runner.ensureInteractionLanguage(id, workspaceConfigPath)).interaction_language;
} catch { return reply.code(503).send({ error: RESUME_FAILURE_MESSAGE }); }
}
// This check belongs inside the per-session lock: a preceding cold Resume may have
// installed a running runtime while this request was waiting.
const existing = d.mgr.get(id);
if (existing) {
const state = existing.bridge.turnState();
if (state === "running" || state === "waiting") {
return reply.code(200).send({ id, alreadyActive: true });
return reply.code(200).send({ id, alreadyActive: true, workspaceId: saved.workspace_id });
}
}
const ensure = await d.readiness.ensure(
@@ -688,8 +761,9 @@ export function sessionRoutes(
...(ensure.code ? { code: ensure.code } : {}),
});
const options = {
provider: saved?.provider,
model: saved?.model,
provider: selected.provider,
interactionLanguage: language,
model: selected.model,
thinking: saved?.thinking ?? settings.thinking,
author: principal.displayName ?? principal.subject,
principal,
@@ -711,7 +785,7 @@ export function sessionRoutes(
if (current) {
const state = current.bridge.turnState();
if (state === "running" || state === "waiting") {
return reply.code(200).send({ id, alreadyActive: true });
return reply.code(200).send({ id, alreadyActive: true, workspaceId: saved.workspace_id });
}
}
@@ -752,7 +826,7 @@ export function sessionRoutes(
d.mgr.configure(rt, runtimeOptions), null,
() => d.mgr.start(id, rt, runtimeOptions),
);
return reply.code(200).send({ id, alreadyActive: false });
return reply.code(200).send({ id, alreadyActive: false, workspaceId: saved.workspace_id });
});
});
app.post("/sessions/:id/close", async (req, reply) => {
+2 -2
View File
@@ -16,8 +16,8 @@ export function effectiveSettings(
modelCatalog?: RuntimeModelCatalog,
): Settings {
const workspaces = listWorkspaces(cfg.harnessDir);
const selected = modelCatalog?.defaultSession
? splitCanonicalModelId(modelCatalog.defaultSession)
const selected = modelCatalog?.defaultInteraction
? splitCanonicalModelId(modelCatalog.defaultInteraction)
: undefined;
return {
workspace: stored.workspace ?? workspaces[0]?.name,
+10
View File
@@ -0,0 +1,10 @@
/** Validate/canonicalize the tag; available translation catalogs belong to the UI. */
export function interactionLanguage(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
try {
const [canonical] = Intl.getCanonicalLocales(value);
return canonical;
} catch {
return undefined;
}
}
+10 -1
View File
@@ -59,6 +59,7 @@ export interface SessionRow {
author: string | null;
workspace_id?: string | null;
workspace_revision?: string | null;
interaction_language?: string | null;
archived?: boolean;
}
@@ -482,6 +483,7 @@ export class ThtRunner {
async sessionNew(o: {
question: string;
interactionLanguage?: string;
provider?: string;
model?: string;
thinking?: string;
@@ -497,6 +499,7 @@ export class ThtRunner {
["--provider", o.provider],
["--model", o.model],
["--thinking", o.thinking],
["--interaction-language", o.interactionLanguage],
["--name", o.name],
["--workspace-id", o.workspaceId],
["--workspace-revision", o.workspaceRevision],
@@ -504,7 +507,7 @@ export class ThtRunner {
if (v) a.push(f, v);
}
a.push("--json");
return this.json<{ id: string }>(a, o.workspaceConfigPath ?? o.workspace);
return this.json<{ id: string; interaction_language?: string }>(a, o.workspaceConfigPath ?? o.workspace);
}
/** Build and persist the deterministic F1 retrieval pack for a new session. */
@@ -533,6 +536,12 @@ export class ThtRunner {
return this.json<unknown>(["session", "show", id, "--json"], workspace);
}
ensureInteractionLanguage(id: string, workspace?: string) {
return this.json<{ interaction_language: string }>(
["session", "ensure-interaction-language", id, "--json"], workspace,
);
}
sqlPreview(id: string, p: { limit?: number; offset?: number }, workspace?: string) {
// No positional FILE: the harness resolves sql_final.sql from the session
// via _session_sql_file(cfg, session_id), which respects the workspace path.
+16 -2
View File
@@ -18,7 +18,7 @@ export interface WorkspaceMaintenanceIo {
writeStderr(value: string): void;
}
type Command = "inspect" | "preprocess-run" | "preprocess-clear";
type Command = "inspect" | "preprocess-run" | "preprocess-clear" | "evidence-consolidate" | "evidence-refresh" | "evidence-decide";
function failureResult(
operation: string,
@@ -65,6 +65,9 @@ function parseRequest(command: string, stdin: string): Record<string, unknown> {
inspect: ["schemaVersion", "workspaceId"],
"preprocess-run": ["schemaVersion", "workspaceId"],
"preprocess-clear": ["schemaVersion", "workspaceId"],
"evidence-consolidate": ["schemaVersion", "workspaceId"],
"evidence-refresh": ["schemaVersion", "workspaceId"],
"evidence-decide": ["schemaVersion", "workspaceId", "sourceId", "revision", "decision"],
};
const allowed = allowedByCommand[command];
if (!allowed) throw new Error("unknown command");
@@ -81,6 +84,16 @@ function exitCodeFor(result: WorkspaceOperationResult): number {
async function dispatch(command: Command, service: WorkspacePreprocessingService, request: Record<string, unknown>): Promise<WorkspaceOperationResult> {
switch (command) {
case "evidence-refresh":
return await service.evidenceSources({ workspaceId: request.workspaceId as string, action: "refresh" });
case "evidence-decide":
if (typeof request.sourceId !== "string" || !/^[a-f0-9]{64}$/.test(request.sourceId)
|| typeof request.revision !== "string" || !/^[a-f0-9]{64}$/.test(request.revision)
|| (request.decision !== "keep" && request.decision !== "replace")) throw new Error("invalid request");
return await service.evidenceSources({ workspaceId: request.workspaceId as string, action: "decide",
sourceId: request.sourceId, revision: request.revision, decision: request.decision });
case "evidence-consolidate":
return await service.consolidateEvidence({ workspaceId: request.workspaceId as string });
case "inspect":
return await service.inspect({ workspaceId: request.workspaceId as string });
case "preprocess-run":
@@ -127,7 +140,8 @@ export async function runWorkspaceMaintenanceCli(
|| message === "unexpected request field"
|| message === "invalid workspace id";
return command in {
inspect: true, "preprocess-run": true, "preprocess-clear": true,
inspect: true, "preprocess-run": true, "preprocess-clear": true, "evidence-consolidate": true,
"evidence-refresh": true, "evidence-decide": true,
} ? (requestError ? 2 : 1) : 2;
}
}
@@ -22,6 +22,7 @@ export interface EvidencePreprocessingRequest {
evidence: EvidenceConfig;
job: EvidenceJobState;
dryRun?: boolean;
consolidate?: boolean;
httpPrivateHostAllowlist?: readonly string[];
}
@@ -56,7 +57,7 @@ function isPrivateHost(hostname: string): boolean {
return hostname.endsWith(".internal");
}
function evidencePolicy(
export function evidencePolicy(
evidence: EvidenceConfig,
httpPrivateHostAllowlist?: readonly string[],
): EvidencePreprocessingOutcome | undefined {
@@ -95,13 +96,14 @@ function jobResult(job: EvidenceJobState): Pick<
};
}
async function runEvidenceStage(
export async function runEvidenceStage(
request: EvidencePreprocessingRequest,
deps: EvidencePreprocessingDependencies,
): Promise<EvidencePreprocessingOutcome> {
const payload = await deps.runStage([
"preprocess",
"evidence",
...(request.consolidate ? ["--consolidate"] : []),
...(request.dryRun ? ["--dry-run"] : []),
...(request.job.childRuns.evidence
? ["--resume", request.job.childRuns.evidence]
@@ -3,6 +3,8 @@ import { renameSync, rmSync, writeFileSync, mkdirSync } from "node:fs";
import { join } from "node:path";
import {
continueEvidencePreprocessing,
evidencePolicy,
runEvidenceStage,
type EvidencePreprocessingDependencies,
type EvidencePreprocessingOutcome,
} from "./evidence/preprocessing.js";
@@ -108,6 +110,72 @@ function baseResult(
export class WorkspacePreprocessingService {
constructor(private readonly deps: WorkspacePreprocessingServiceDeps) {}
async evidenceSources(options: { workspaceId: string; action: "refresh" | "decide";
sourceId?: string; revision?: string; decision?: "keep" | "replace"; actor?: string }): Promise<WorkspaceOperationResult> {
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
try {
if (options.action === "refresh") {
const refused = evidencePolicy(runtime.workspace.evidence, this.deps.httpPrivateHostAllowlist);
if (refused) return baseResult(runtime, "evidence refresh", "failed", refused.code);
}
const argv = ["evidence", "sources", options.action, "--json", "-c", "/dev/fd/3"];
if (options.action === "decide") {
if (!/^[a-f0-9]{64}$/.test(options.sourceId ?? "") || !/^[a-f0-9]{64}$/.test(options.revision ?? "")
|| !["keep", "replace"].includes(options.decision ?? "")) throw new Error("Invalid source decision");
const preflight = await this.deps.evidencePreflight(runtime.workspace);
if (!preflight.ok) return baseResult(runtime, "evidence sources", "failed", preflight.code);
argv.push("--source-id", options.sourceId!, "--revision", options.revision!, "--decision", options.decision!);
}
argv.push("--actor", options.actor ?? "installation operator");
const result = await this.deps.runChild({ argv, configPath: runtime.configLease.path });
const payload = JSON.parse(result.stdout);
if (result.exitCode !== 0 || payload.status !== "succeeded") throw new Error(
typeof payload.error === "string" ? payload.error.slice(0, 1500) : "Evidence source operation failed");
return baseResult(runtime, `evidence ${options.action}`, "succeeded", "ok", {
counts: this.numberRecord(payload.counts),
warnings: [options.action === "refresh" ? "Source comparisons are ready in Evidence management. Active content is unchanged."
: "Source decision activated locally. Commit and push the Evidence tree manually."],
});
} catch (error) {
return baseResult(runtime, `evidence ${options.action}`, "failed", "evidence_materialization_required", {
warnings: [error instanceof Error ? error.message : "Evidence source operation failed"],
});
}
}
async consolidateEvidence(options: { workspaceId: string }): Promise<WorkspaceOperationResult> {
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
const preflight = await this.deps.evidencePreflight(runtime.workspace);
if (!preflight.ok) return baseResult(runtime, "evidence consolidate", "failed", preflight.code);
// Evidence has its own durable archive/corpus jobs. Do not alter Catalog readiness
// or the full preprocessing job when publishing this one component.
try {
const outcome = await runEvidenceStage({ evidence: runtime.workspace.evidence,
consolidate: true, job: { runId: randomBytes(16).toString("hex"), completedStages: [], childRuns: {} },
}, {
runStage: async argv => {
const result = await this.deps.runChild({ argv, configPath: runtime.configLease.path });
const payload = JSON.parse(result.stdout);
if (result.exitCode !== 0 || payload.status !== "succeeded") {
return Promise.reject(new Error(typeof payload.error === "string" ? payload.error.slice(0, 1500) : "Evidence consolidation failed. Retry the command."));
}
return payload;
},
persistJob: () => undefined,
evidencePreflight: async () => preflight,
requireRunId: value => this.requireRunId(value),
numberRecord: value => this.numberRecord(value),
});
return baseResult(runtime, "evidence consolidate", "succeeded", "ok", {
...outcome, warnings: ["Evidence is active locally. Catalog and Schema readiness are unchanged. Commit and push the Evidence files manually."],
});
} catch (error) {
return baseResult(runtime, "evidence consolidate", "failed", "evidence_materialization_required", {
warnings: [error instanceof Error ? error.message : "Evidence consolidation failed. Retry the command."],
});
}
}
async inspect(options: { workspaceId: string }): Promise<WorkspaceOperationResult> {
try {
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
@@ -497,7 +497,10 @@ export async function publishDeterministicRuntimeConfigLease(options: {
rendered.workspaceRevision,
renderedConfigObject,
);
const identitySuffix = inputFingerprintValue.slice(7, 23);
// The Catalog input identity intentionally excludes representation-only changes.
// A lease also identifies its bytes, so a new renderer never collides with an
// immutable config produced by an earlier release for the same Catalog inputs.
const identitySuffix = sha256(inputFingerprintValue + "\n" + publishedConfig).slice(7, 23);
const preprocessingRoot = ensureTrustedDirectory(join(
options.dataRoot,
+2 -1
View File
@@ -1,4 +1,4 @@
import { basename, join } from "node:path";
import { basename, dirname, join } from "node:path";
import { stringify } from "yaml";
import { buildInstallationContract } from "./contracts.js";
import { validateWorkspaceDescriptor, type WorkspaceDescriptor } from "./schema.js";
@@ -179,6 +179,7 @@ function renderEvidence(
return {
evidence: {
...(workspace.evidence.schema_version === 2 ? { schema_version: 2 } : {}),
local_archive_root: join(dirname(dirname(context.revisionContentRoot)), "repo", context.workspaceId),
sources: [renderedSource],
},
vector: {
@@ -0,0 +1,48 @@
import Fastify from "fastify";
import { expect, test, vi } from "vitest";
import { sessionRoutes } from "../src/routes/sessions.js";
import type { PrincipalContext } from "../src/auth/principal.js";
test.each([
[false, "m", 403], [false, "e", 403], [false, "reject", 204],
[true, "m", 204], [true, "e", 204], [true, "forged", 400],
] as const)("archive repair response enforces the responding principal (%s, %s)", async (admin, choice, status) => {
const app = Fastify();
const principal: PrincipalContext = { issuer: "test", subject: "reviewer", isAdmin: admin,
roles: admin ? ["admin"] : ["user"],
permissions: admin ? ["session.use", "memory.manage", "evidence.manage"] : ["session.use"] };
app.addHook("preHandler", async request => { request.principal = principal; });
const respond = vi.fn(() => true);
sessionRoutes(app, {
tht: {}, mgr: { get: () => ({ ownerKey: "test\0reviewer", bridge: {
respond, pendingWidget: () => ({ id: "gate", widget: "archive-repair", repair: { options: [
{ id: "m", archive: "memory" }, { id: "e", archive: "evidence" },
] } }),
} }) },
} as unknown as Parameters<typeof sessionRoutes>[1]);
try {
const result = await app.inject({ method: "POST", url: "/sessions/s/response",
payload: { ui_response: { id: "gate", choices: [choice] } } });
expect(result.statusCode).toBe(status);
expect(respond).toHaveBeenCalledTimes(status === 204 ? 1 : 0);
} finally { await app.close(); }
});
test("an administrator cannot attribute a repair to another runtime's principal", async () => {
const app = Fastify();
app.addHook("preHandler", async request => { request.principal = {
issuer: "test", subject: "second-admin", isAdmin: true, roles: ["admin"],
permissions: ["session.use", "memory.manage"],
}; });
const respond = vi.fn();
sessionRoutes(app, { tht: {}, mgr: { get: () => ({ ownerKey: "test\0first-admin", bridge: {
respond, pendingWidget: () => ({ id: "gate", widget: "archive-repair",
repair: { options: [{ id: "m", archive: "memory" }] } }),
} }) } } as unknown as Parameters<typeof sessionRoutes>[1]);
try {
const result = await app.inject({ method: "POST", url: "/sessions/s/response",
payload: { ui_response: { id: "gate", choices: ["m"] } } });
expect(result.statusCode).toBe(409);
expect(respond).not.toHaveBeenCalled();
} finally { await app.close(); }
});
+2
View File
@@ -188,6 +188,8 @@ test("roles collapse duplicates and admin contains all administrative permission
"workspace.manage",
"workspace.secrets.manage",
"database.manage",
"memory.manage",
"evidence.manage",
"pi.manage",
"auth.diagnostics.read",
]);
+1 -1
View File
@@ -130,7 +130,7 @@ test("local login sets a non-persistent opaque session cookie and exposes only a
roles: ["admin"],
permissions: [
"session.use", "session.read_all", "session.manage_all", "settings.manage",
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
],
isAdmin: true,
csrfToken: expect.stringMatching(/^[A-Za-z0-9_-]{43}$/),
+32 -6
View File
@@ -7,6 +7,8 @@ import { join } from "node:path";
import { expandLocalHome, localPrincipal, upstreamPrincipal } from "../src/auth/principal.js";
import { buildApp } from "../src/app.js";
import { loadConfig } from "../src/config.js";
import { rolesToPermissions } from "../src/auth/config.js";
import type { Permission, Role } from "../src/auth/types.js";
test("server smoke rejects retired trusted claims under OIDC authentication", () => {
const smoke = readFileSync("../scripts/unified-deployment-smoke.sh", "utf8");
@@ -32,7 +34,7 @@ test("local mode resolves a stable local principal", async () => {
roles: ["admin"],
permissions: [
"session.use", "session.read_all", "session.manage_all", "settings.manage",
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
],
isAdmin: true,
});
@@ -74,7 +76,7 @@ test("upstream mode accepts only normalized proxy principal headers", async () =
issuer: "portal", subject: "42", displayName: "Alice", roles: ["user", "admin"],
permissions: [
"session.use", "session.read_all", "session.manage_all", "settings.manage",
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
],
isAdmin: true,
});
@@ -206,15 +208,33 @@ test("the session boundary rejects tht maintenance headers outside exact loopbac
})).statusCode).toBe(503);
});
test("the session boundary touches a valid cookie session through the bounded Task 7 store operation", async () => {
test.each<{
name: string;
roles: Role[];
storedPermissions: Permission[];
}>([
{ name: "current user", roles: ["user"], storedPermissions: ["session.use"] },
{
name: "administrator signed in before archive permissions existed",
roles: ["admin"],
storedPermissions: rolesToPermissions(["admin"]).filter(
(permission) => permission !== "memory.manage" && permission !== "evidence.manage",
),
},
{
name: "user with obsolete administrative permissions",
roles: ["user"],
storedPermissions: ["session.use", "memory.manage", "evidence.manage"],
},
])("the session boundary derives current permissions and touches a valid cookie: $name", async ({ roles, storedPermissions }) => {
const sessions = {
resolve: vi.fn(async () => ({
version: 1,
issuer: "local",
subject: "user-1",
method: "local",
roles: ["user"],
permissions: ["session.use"],
roles,
permissions: storedPermissions,
userAuthRevision: 1,
authConfigRevision: "b".repeat(64),
remembered: false,
@@ -249,7 +269,13 @@ test("the session boundary touches a valid cookie session through the bounded Ta
app.get("/private", async (request) => getPrincipal(request));
const token = "z".repeat(43);
expect((await app.inject({ method: "GET", url: "/private", headers: { cookie: `thothii_session=${token}` } })).statusCode).toBe(200);
const response = await app.inject({ method: "GET", url: "/private", headers: { cookie: `thothii_session=${token}` } });
expect(response.statusCode).toBe(200);
expect(response.json()).toMatchObject({
roles,
permissions: rolesToPermissions(roles),
isAdmin: roles.includes("admin"),
});
expect(sessions.touch).toHaveBeenCalledWith(token);
});
@@ -124,8 +124,17 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
],
relationships: [],
};
expect(await repository.applySchemaSync(created.id, 1, "columns", [], fullColumnsSnapshot))
const memoryCleanupRun = await repository.createSyncRun(created.id, "columns", [], 1);
await repository.updateSyncRun(memoryCleanupRun.id, { state: "applying", phase: "applying" });
await expect(repository.applySchemaSync(created.id, 999, "columns", [], fullColumnsSnapshot, memoryCleanupRun.id))
.resolves.toBeUndefined();
expect((await repository.getSyncRun(memoryCleanupRun.id))?.phase).toBe("applying");
expect(await repository.applySchemaSync(created.id, 1, "columns", [], fullColumnsSnapshot, memoryCleanupRun.id))
.toMatchObject({ created: 4, deleted: 0 });
expect((await repository.getSyncRun(memoryCleanupRun.id))?.phase).toBe("memory_cleanup");
await repository.interruptActiveSyncRuns();
expect(await repository.getSyncRun(memoryCleanupRun.id))
.toMatchObject({ state: "interrupted", phase: "memory_cleanup" });
expect((await repository.listColumns(created.id, patients.id)).map((column) => ({
name: column.name,
sensitive: column.sensitive,
@@ -233,6 +242,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
});
expect(await repository.listSyncRuns(created.id)).toEqual([
expect.objectContaining({ id: syncRun.id, tableIds: [patients.id] }),
expect.objectContaining({ id: memoryCleanupRun.id, phase: "memory_cleanup" }),
]);
const lockedDatabase = await repository.create({
+42 -2
View File
@@ -117,6 +117,7 @@ async function setup(env: Record<string, string> = {}) {
});
const introspector: CatalogSchemaIntrospector = { scan };
const operations = new CatalogOperationCoordinator();
const cleanup = vi.fn(async () => ({ code: 0, stdout: JSON.stringify({ indexed: true, deleted: 0 }), stderr: "" }));
const registry = {
list: vi.fn(async () => [revision]),
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
@@ -124,7 +125,7 @@ async function setup(env: Record<string, string> = {}) {
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
} as unknown as WorkspaceRegistry;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test", ...env }), {
thtRunner: {} as never,
thtRunner: { withPrincipal: () => ({ runWithRuntimeSnapshot: cleanup }) } as never,
workspaceRegistry: registry,
workspaceSecretStore: new WorkspaceSecretStore({ root: secretRoot, runtimeRoot, installationId: "test" }),
catalogRepository: repository,
@@ -133,7 +134,7 @@ async function setup(env: Record<string, string> = {}) {
workspaceDiagnoser: vi.fn(),
});
return {
app, repository, database: (await repository.get(created.id))!, scan, operations,
app, repository, database: (await repository.get(created.id))!, scan, operations, cleanup,
setObserved(next: ObservedSchemaSnapshot) { observed = next; },
};
}
@@ -614,6 +615,45 @@ test("waits for confirmation and rescans before applying destructive changes", a
expect(scan).toHaveBeenCalledTimes(3);
});
test("retries committed Memory cleanup with the original removals without rescanning", async () => {
const { app, repository, database, scan, setObserved, cleanup } = await setup();
await seedCatalog(repository, database);
const observed = snapshot();
observed.tables = observed.tables.filter(t => t.name !== "visits");
observed.columns = observed.columns.filter(c => c.tableName !== "visits");
observed.relationships = [];
setObserved(observed);
cleanup.mockResolvedValueOnce({ code: 0, stdout: JSON.stringify({ indexed: false, deleted: 2 }), stderr: "" });
const started = await app.inject({ method: "POST", url: `/catalog/databases/${database.id}/sync-runs`,
payload: { version: database.version, scope: "all", tableIds: [] } });
const waiting = await waitFor(repository, started.json().id, "awaiting_confirmation");
expect(cleanup).not.toHaveBeenCalled();
await app.inject({ method: "POST", url: `/catalog/sync-runs/${waiting.id}/confirm`,
payload: { confirmationToken: waiting.confirmationToken } });
const failed = await waitFor(repository, waiting.id, "failed");
expect(failed).toMatchObject({ phase: "memory_cleanup", errorCode: "memory_cleanup_pending",
plannedDiff: { deletedTables: ["visits"] } });
expect((await repository.listTables(database.id)).map(t => t.name)).toEqual(["patients"]);
const callsBeforeRetry = scan.mock.calls.length;
const blocked = await app.inject({ method: "POST", url: `/catalog/databases/${database.id}/sync-runs`,
payload: { version: database.version, scope: "all", tableIds: [] } });
expect(blocked.statusCode).toBe(409);
// Simulate startup recovery and an unreachable DWH after schema application.
await repository.interruptActiveSyncRuns();
scan.mockRejectedValue(new Error("DWH unavailable"));
cleanup.mockResolvedValue({ code: 0, stdout: JSON.stringify({ indexed: true, deleted: 2 }), stderr: "" });
const retried = await app.inject({ method: "POST", url: `/catalog/sync-runs/${waiting.id}/retry` });
expect(retried.statusCode).toBe(202);
const completed = await waitFor(repository, waiting.id, "succeeded");
expect(completed.counts).toMatchObject({ memoryDeleted: 2 });
expect(scan.mock.calls.length).toBe(callsBeforeRetry);
expect(cleanup).toHaveBeenCalledTimes(2);
expect(cleanup.mock.calls[1]).toEqual(cleanup.mock.calls[0]);
const request = JSON.parse((cleanup.mock.calls[0] as unknown as string[])[1]!).request;
expect(request).toMatchObject({ sync_id: waiting.id, database: "warehouse",
schema_name: "datawarehouse", removed_tables: ["visits"] });
});
test("deletes every catalog table for multiple selected databases and cascades dependent metadata", async () => {
const { app, repository, database } = await setup();
const second = await repository.create({
+2 -2
View File
@@ -36,7 +36,7 @@ test("loop F1: crea sessione → SSE riceve il widget → risponde → il modell
ollamaEnsure: async () => ({ ok: true }),
searchPack: async () => {},
sessionNew: async () => ({ id: "s1" }),
sessionShow: async (_id: string) => ({ id: "s1", provider: undefined, model: undefined, thinking: undefined }),
sessionShow: async (_id: string) => ({ id: "s1", interaction_language: "en", provider: undefined, model: undefined, thinking: undefined }),
sessionList: async () => [],
} as any,
spawnFn: () => spawn("node", [FAKE, SCRIPT]) as any,
@@ -48,7 +48,7 @@ test("loop F1: crea sessione → SSE riceve il widget → risponde → il modell
const created = await fetch(`${base}/sessions`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ workspace: "w", question: "q" }),
body: JSON.stringify({ workspace: "w", question: "q", interactionLanguage: "en" }),
});
expect(created.status).toBe(200);
+69
View File
@@ -0,0 +1,69 @@
import Fastify from "fastify";
import { afterEach, expect, test, vi } from "vitest";
import { evidenceRoutes } from "../src/routes/evidence.js";
import type { ThtRunner } from "../src/tht/tht-runner.js";
import type { PrincipalContext } from "../src/auth/principal.js";
const apps: ReturnType<typeof Fastify>[] = [];
afterEach(async () => { await Promise.all(apps.splice(0).map(app => app.close())); });
function setup(admin = true) {
const app = Fastify(); apps.push(app);
const principal: PrincipalContext = { issuer: "test", subject: "curator", roles: [admin ? "admin" : "user"],
permissions: admin ? ["evidence.manage"] : ["session.use"], isAdmin: admin };
app.addHook("preHandler", async request => { request.principal = principal; });
const run = vi.fn(async () => ({ code: 0, stdout: JSON.stringify({ items: [], total: 0 }), stderr: "" }));
const withPrincipal = vi.fn(() => ({ runWithRuntimeSnapshot: run }) as unknown as ThtRunner);
const consolidateEvidence = vi.fn();
const evidenceSources = vi.fn().mockResolvedValue({status: "succeeded"});
evidenceRoutes(app, { runner: { withPrincipal }, registryRoot: "/data/registry", hostRegistryRoot: "/srv/Thoth workspaces",
registry: { list: async () => [{ id: "sales", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/missing" }] },
service: { consolidateEvidence, evidenceSources } });
return { app, run, withPrincipal, principal, consolidateEvidence, evidenceSources };
}
test("browses complete local files with host paths, without a database runtime", async () => {
const { app, run, withPrincipal, principal } = setup();
const response = await app.inject("/workspaces/sales/evidence?kind=domain&concept=orders&page=2");
expect(response.statusCode).toBe(200);
expect(withPrincipal).toHaveBeenCalledWith(principal);
const [argv, snapshot] = run.mock.calls[0] as unknown as [string[], string];
expect(argv).toEqual(["evidence", "admin", "--workspace", "sales"]);
expect(JSON.parse(snapshot)).toEqual({ root: "/data/registry/repo/sales", query: { kind: "domain", concept: "orders", page: 2 } });
expect(response.json().location.workspace).toBe("/srv/Thoth workspaces/repo/sales");
expect(response.json().location.git_commands[0]).toContain("git -C '/srv/Thoth workspaces/repo'");
});
test("source actions require admin, bind the actor, and reject cross-workspace or arbitrary inputs", async () => {
const denied = setup(false);
expect((await denied.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload: {action: "refresh"}})).statusCode).toBe(403);
expect(denied.evidenceSources).not.toHaveBeenCalled();
const allowed = setup();
expect((await allowed.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload: {action: "refresh"}})).statusCode).toBe(200);
expect(allowed.evidenceSources).toHaveBeenCalledWith({workspaceId: "sales", action: "refresh", actor: "curator"});
const choice = {action: "decide", sourceId: "a".repeat(64), revision: "b".repeat(64), decision: "keep"};
expect((await allowed.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload: choice})).statusCode).toBe(200);
expect(allowed.evidenceSources).toHaveBeenLastCalledWith({...choice, workspaceId: "sales", actor: "curator"});
for (const payload of [{action: "refresh", url: "http://localhost"}, {...choice, actor: "forged"}, {...choice, revision: "../other"}]) {
expect((await allowed.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload})).statusCode).toBe(400);
}
expect((await allowed.app.inject({method: "POST", url: "/workspaces/other/evidence/sources", payload: choice})).statusCode).toBe(404);
expect(allowed.evidenceSources).toHaveBeenCalledTimes(2);
});
test.each(["/", "/evidence:rule", "/consolidate"])("refuses non-admin access %s", async suffix => {
const { app, run, consolidateEvidence } = setup(false);
const response = await app.inject({ method: suffix === "/consolidate" ? "POST" : "GET", url: `/workspaces/sales/evidence${suffix === "/" ? "" : suffix}` });
expect(response.statusCode).toBe(403); expect(run).not.toHaveBeenCalled(); expect(consolidateEvidence).not.toHaveBeenCalled();
});
test("rejects workspace escape, unknown workspace and unsupported filters", async () => {
const { app, run } = setup();
expect((await app.inject("/workspaces/foreign/evidence")).statusCode).toBe(404);
expect((await app.inject("/workspaces/sales/evidence?root=/private")).statusCode).toBe(400);
expect((await app.inject("/workspaces/sales/evidence?page_size=101")).statusCode).toBe(400);
expect(run).not.toHaveBeenCalled();
});
test("a missing unit is 404 and consolidation preserves the partial outcome", async () => {
const { app, consolidateEvidence } = setup();
expect((await app.inject("/workspaces/sales/evidence/evidence:missing")).statusCode).toBe(404);
consolidateEvidence.mockResolvedValue({ status: "failed", warnings: ["Saved; retry indexing."] });
const response = await app.inject({ method: "POST", url: "/workspaces/sales/evidence/consolidate" });
expect(response.json()).toEqual({ status: "failed", warnings: ["Saved; retry indexing."] });
});
+33 -1
View File
@@ -56,7 +56,7 @@ test("catalog listing translates upstream Pi IDs back to canonical model keys",
session: { reasoning: true, contextWindow: 32768, maxTokens: 8192 },
};
const modelCatalog: RuntimeModelCatalog = {
defaultSession: model.id, defaultMetadataGeneration: null, embedding: null,
defaultInteraction: model.id, embedding: null,
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
};
try {
@@ -75,6 +75,38 @@ test("catalog listing translates upstream Pi IDs back to canonical model keys",
}
});
test("catalog listing supplies the shared DeepSeek key without requiring Pi auth", async () => {
const script = scriptWith([{ provider: "deepseek", id: "deepseek-v4-pro", name: "DeepSeek V4 Pro" }]);
const secret = join(path.dirname(script), "thothii.secrets");
writeFileSync(secret, "DEEPSEEK_API_KEY=shared-key\nOPENAI_API_KEY=unrelated-key\n", { mode: 0o600 });
const model: RuntimeModel = {
id: "deepseek/deepseek-v4-pro", provider: "deepseek", model: "deepseek-v4-pro",
label: "DeepSeek V4 Pro", upstreamModel: "deepseek-v4-pro",
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
sessionAdapter: { mode: "pi_builtin" }, session: { reasoning: false },
};
const modelCatalog: RuntimeModelCatalog = {
defaultInteraction: model.id, embedding: null,
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
};
try {
const lister = createPiModelLister(loadConfig({ THT_SECRETS_FILE: secret }), {
...noManagedModels, modelCatalog, loadEnabledModels: enabled(model.id),
spawnFn: (_command, _args, options) => {
expect(options.env.DEEPSEEK_API_KEY).toBe("shared-key");
expect(options.env).not.toHaveProperty("OPENAI_API_KEY");
expect(options.env).not.toHaveProperty("THT_SECRETS_FILE");
return spawn("node", [FAKE, script], { env: options.env }) as any;
},
});
await expect(lister()).resolves.toEqual([{
provider: model.provider, id: model.model, name: model.label, reasoning: false,
}]);
} finally {
rmSync(path.dirname(script), { recursive: true, force: true });
}
});
test("createPiModelLister caches within ttl (spawns once for two calls)", async () => {
const script = scriptWith([{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }]);
try {
+81
View File
@@ -0,0 +1,81 @@
import Fastify from "fastify";
import { afterEach, expect, test, vi } from "vitest";
import { memoryRoutes } from "../src/routes/memory.js";
import { DEFAULT_SEMANTIC_RUNTIME } from "../src/workspaces/runtime-renderer.js";
import type { ThtRunner } from "../src/tht/tht-runner.js";
import type { PrincipalContext } from "../src/auth/principal.js";
const apps: ReturnType<typeof Fastify>[] = [];
afterEach(async () => { await Promise.all(apps.splice(0).map(app => app.close())); });
const id = "mem-11111111-1111-4111-8111-111111111111";
const input = { family: "domain_clarification", subject: "Order", scope: "Sales" };
function setup(admin = true) {
const app = Fastify(); apps.push(app);
const principal: PrincipalContext = { issuer: "test", subject: "operator", roles: [admin ? "admin" : "user"],
permissions: admin ? ["memory.manage"] : ["session.use"], isAdmin: admin };
app.addHook("preHandler", async request => { request.principal = principal; });
const run = vi.fn(async () => ({ code: 0, stdout: JSON.stringify({ items: [], total: 0 }), stderr: "" }));
const bound = { runWithRuntimeSnapshot: run } as unknown as ThtRunner;
const withPrincipal = vi.fn(() => bound);
memoryRoutes(app, { runner: { withPrincipal }, runtime: DEFAULT_SEMANTIC_RUNTIME,
registry: {
list: async () => [{ id: "sales", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/missing" }],
read: vi.fn().mockResolvedValue({ workspace: { workspace: { language: "it" } } }),
} });
return { app, run, withPrincipal, principal };
}
test("list binds principal and workspace without requiring a DWH runtime", async () => {
const { app, run, withPrincipal, principal } = setup();
const response = await app.inject("/workspaces/sales/memory?q=Order&page=2&column=id");
expect(response.statusCode).toBe(200);
expect(withPrincipal).toHaveBeenCalledWith(principal);
const [args, snapshot] = run.mock.calls[0] as unknown as [string[], string];
expect(args).toEqual(["memory", "admin", "--workspace", "sales"]);
expect(JSON.parse(snapshot)).toMatchObject({ action: "list", request: { q: "Order", page: 2, column: "id" } });
expect(JSON.parse(snapshot).runtime.memoryLanguage).toBe("it");
});
test.each([
["GET", ""], ["POST", ""], ["GET", "/pending"], ["GET", `/${id}`],
["PUT", `/${id}`], ["DELETE", `/${id}`], ["POST", `/${id}/retry`],
] as const)("non-admin cannot %s %s", async (method, suffix) => {
const { app, run } = setup(false);
const response = await app.inject({ method, url: `/workspaces/sales/memory${suffix}`,
...(method === "PUT" || method === "POST" && suffix === "" ? { payload: input } : {}) });
expect(response.statusCode).toBe(403); expect(run).not.toHaveBeenCalled();
});
test("unknown workspace and invalid input do not invoke the harness", async () => {
const { app, run } = setup();
expect((await app.inject("/workspaces/missing/memory")).statusCode).toBe(404);
expect((await app.inject("/workspaces/sales/memory?page_size=101")).statusCode).toBe(400);
expect((await app.inject({ method: "POST", url: "/workspaces/sales/memory",
payload: { ...input, workspace_id: "foreign" } })).statusCode).toBe(400);
expect(run).not.toHaveBeenCalled();
});
test("create preserves saved-but-unindexed outcome, update carries complete related changes", async () => {
const { app, run } = setup();
run.mockResolvedValue({ code: 0, stdout: JSON.stringify({ id, saved: true, indexed: false }), stderr: "" });
const result = await app.inject({ method: "POST", url: "/workspaces/sales/memory", payload: input });
expect(result.statusCode).toBe(201); expect(result.json()).toEqual({ id, saved: true, indexed: false });
await app.inject({ method: "PUT", url: `/workspaces/sales/memory/${id}`, payload: {
...input, links: [{ target_id: id, meaning: "Related" }], dependencies: [{ database: "dwh" }],
} });
const [, snapshot] = run.mock.calls[1] as unknown as [string[], string];
expect(JSON.parse(snapshot)).toMatchObject({ action: "update", request: { id, card: {
links: [{ target_id: id, meaning: "Related" }], dependencies: [{ database: "dwh" }],
} } });
});
test.each([["memory_not_found", 404], ["memory_conflict", 409], ["memory_unavailable", 503]])(
"maps %s without leaking stderr", async (code, status) => {
const { app, run } = setup();
run.mockResolvedValue({ code: 1, stdout: JSON.stringify({ code, message: "sensitive detail" }), stderr: "secret" });
const result = await app.inject("/workspaces/sales/memory");
expect(result.statusCode).toBe(status); expect(result.json().code).toBe(code);
expect(result.body).not.toMatch(/secret|sensitive/);
},
);
@@ -20,9 +20,8 @@ function runtimeCatalog(overrides: Record<string, unknown> = {}, secrets = "OPEN
const catalogFile = join(root, "catalog.json");
const secretsFile = join(root, "thothii.secrets");
const catalog = {
schemaVersion: 1,
defaultSession: "zai/glm-5.3",
defaultMetadataGeneration: "zai/glm-5.3",
schemaVersion: 2,
defaultInteraction: "zai/glm-5.3",
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
models: [
{
@@ -55,8 +54,8 @@ test("loads session default and safe metadata choices from the normalized runtim
const runtime = loadRuntimeModelCatalog(catalogFile);
const metadata = loadMetadataGenerationModels({ catalogFile, secretsFile });
expect(runtime.defaultSession).toBe("zai/glm-5.3");
expect(runtime.hasSession("deepseek/deepseek-v4-pro")).toBe(true);
expect(runtime.defaultInteraction).toBe("zai/glm-5.3");
expect(runtime.hasSession("deepseek/deepseek-v4-pro")).toBe(false);
expect(metadata.catalog()).toEqual({
models: [{ id: "zai/glm-5.3", label: "GLM 5.3" }],
default: "zai/glm-5.3",
@@ -69,14 +68,65 @@ test("loads session default and safe metadata choices from the normalized runtim
expect(() => metadata.resolve("zai/missing")).toThrow(MetadataGenerationModelUnavailableError);
});
test("DeepSeek uses one identity and bundle credential for native Pi and LiteLLM", () => {
const models = ["deepseek-v4-pro", "deepseek-v4-flash"].map((model) => ({
id: `deepseek/${model}`, provider: "deepseek", model, label: model, upstreamModel: model,
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
sessionAdapter: { mode: "pi_builtin" }, metadataAdapter: { litellmProvider: "deepseek" },
session: { reasoning: false }, metadataGeneration: { disableThinking: false },
}));
const files = runtimeCatalog({ defaultInteraction: models[0].id, models }, "DEEPSEEK_API_KEY=shared-key\n");
const runtime = loadRuntimeModelCatalog(files.catalogFile);
const metadata = loadMetadataGenerationModels(files);
expect(runtime.sessionModels().map((model) => model.id)).toEqual(metadata.catalog().models.map((model) => model.id));
expect(metadata.catalog().default).toBe(runtime.defaultInteraction);
for (const model of models) {
expect(metadata.resolve(model.id)).toMatchObject({
id: model.id, provider: "deepseek", model: model.model,
apiKeyEnv: "DEEPSEEK_API_KEY", apiKey: "shared-key",
});
}
expect(() => metadata.resolve("deepseek-metadata/deepseek-v4-pro"))
.toThrow(MetadataGenerationModelUnavailableError);
});
test("returns empty catalogs when no runtime projection is configured", () => {
expect(loadRuntimeModelCatalog().defaultSession).toBeNull();
expect(loadRuntimeModelCatalog().defaultInteraction).toBeNull();
expect(loadMetadataGenerationModels({}).catalog()).toEqual({ models: [], default: null });
});
test.each([
["qwen-chat-template", true, true],
["qwen", true, true],
["qwen-typo", true, false],
["qwen-chat-template", false, false],
])("validates Qwen thinking format %s with reasoning=%s", (thinkingFormat, reasoning, valid) => {
const { catalogFile } = runtimeCatalog({
defaultInteraction: "local/qwen",
models: [{
id: "local/qwen", provider: "local", model: "qwen", label: "Qwen",
upstreamModel: "qwen", endpoint: { baseUrl: "http://localhost:8000/v1" },
authentication: { mode: "none" }, sessionAdapter: { mode: "openai_compatible" },
session: {
reasoning, contextWindow: 32768, maxTokens: 8192,
compatibility: {
supportsDeveloperRole: false, supportsReasoningEffort: false,
supportsStore: false, maxTokensField: "max_tokens", thinkingFormat,
},
},
}],
});
if (valid) {
expect(loadRuntimeModelCatalog(catalogFile).sessionModels()[0].session?.compatibility)
.toMatchObject({ thinkingFormat });
} else {
expect(() => loadRuntimeModelCatalog(catalogFile)).toThrow("runtime model catalog is invalid");
}
});
test("rejects a drifted default and an unprotected projection", () => {
const drifted = runtimeCatalog({ defaultSession: "zai/missing" });
expect(() => loadRuntimeModelCatalog(drifted.catalogFile)).toThrow("session default is invalid");
const drifted = runtimeCatalog({ defaultInteraction: "zai/missing" });
expect(() => loadRuntimeModelCatalog(drifted.catalogFile)).toThrow("interaction default is invalid");
const unprotected = runtimeCatalog();
chmodSync(unprotected.catalogFile, 0o666);
@@ -85,7 +135,7 @@ test("rejects a drifted default and an unprotected projection", () => {
test("rejects authentication semantics that cannot come from the installation catalog", () => {
const invalid = runtimeCatalog({
defaultMetadataGeneration: undefined,
defaultInteraction: undefined,
models: [{
id: "zai/glm-5.3",
provider: "zai",
@@ -112,3 +162,13 @@ test("splits canonical session identities without provider aliases", () => {
expect(splitCanonicalModelId("zai/glm-5.3")).toEqual({ provider: "zai", model: "glm-5.3" });
expect(() => splitCanonicalModelId("glm-5.3")).toThrow("model identity is invalid");
});
test("rejects a default supported by only one configured use", () => {
const { catalogFile } = runtimeCatalog({ defaultInteraction: "deepseek/deepseek-v4-pro" });
expect(() => loadRuntimeModelCatalog(catalogFile)).toThrow("interaction default is invalid");
});
test("requires regenerated runtime schema v2 rather than interpreting two legacy defaults", () => {
const { catalogFile } = runtimeCatalog({ schemaVersion: 1, defaultSession: "zai/glm-5.3" });
expect(() => loadRuntimeModelCatalog(catalogFile)).toThrow("runtime model catalog is invalid");
});
+18
View File
@@ -5,6 +5,8 @@ const fakes = vi.hoisted(() => ({
catalogRepository: { close: vi.fn(async () => {}) },
createCatalogRepository: vi.fn(),
runnerConfig: undefined as Record<string, unknown> | undefined,
acquireWorkspaceRuntime: vi.fn(),
releaseWorkspaceRuntime: vi.fn(),
run: vi.fn(async () => ({
code: 0,
stdout: JSON.stringify({ ok: true }),
@@ -24,6 +26,8 @@ vi.mock("../src/tht/tht-runner.js", () => ({
run = fakes.run;
acquireWorkspaceRuntime = fakes.acquireWorkspaceRuntime;
withPrincipal() {
return this;
}
@@ -67,6 +71,12 @@ beforeEach(() => {
fakes.createCatalogRepository.mockReset();
fakes.createCatalogRepository.mockReturnValue(fakes.catalogRepository);
fakes.run.mockClear();
fakes.acquireWorkspaceRuntime.mockReset();
fakes.acquireWorkspaceRuntime.mockResolvedValue({
path: "/data/workspace-registry/snapshots/runtime/workspace.yaml",
release: fakes.releaseWorkspaceRuntime,
});
fakes.releaseWorkspaceRuntime.mockClear();
fakes.runnerConfig = undefined;
});
@@ -78,5 +88,13 @@ test("workflow doctor gives schema-v4 runtime rendering a live Catalog repositor
expect(fakes.createCatalogRepository).toHaveBeenCalledWith(config.catalogDatabase);
expect(fakes.runnerConfig?.catalogRepository).toBe(fakes.catalogRepository);
expect(fakes.acquireWorkspaceRuntime).toHaveBeenCalledWith(
"/data/workspace-registry/snapshots/revision/workspace.yaml",
);
expect(fakes.run).toHaveBeenCalledWith(
["doctor", "--json"],
"/data/workspace-registry/snapshots/runtime/workspace.yaml",
);
expect(fakes.releaseWorkspaceRuntime).toHaveBeenCalledOnce();
expect(fakes.catalogRepository.close).toHaveBeenCalledOnce();
});
+43 -4
View File
@@ -1,4 +1,4 @@
import { mkdtempSync } from "node:fs";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { expect, test, vi } from "vitest";
@@ -7,7 +7,34 @@ import {
createPiManagement,
type PiExecFile,
} from "../src/pi/management.js";
import type { RuntimeModelCatalog } from "../src/models/runtime-model-catalog.js";
import type { RuntimeModel, RuntimeModelCatalog } from "../src/models/runtime-model-catalog.js";
test.each([false, true])("catalog credential status ignores legacy Pi auth, missing bundle key: %s", async (missingKey) => {
const root = mkdtempSync(join(tmpdir(), "tht-catalog-status-"));
writeFileSync(join(root, "auth.json"), JSON.stringify({ deepseek: { type: "api_key", key: "stale-key" } }), { mode: 0o600 });
const secret = join(root, "thothii.secrets");
writeFileSync(secret, missingKey ? "THT_MODEL_API_KEY=legacy-key\n" : "DEEPSEEK_API_KEY=shared-key\n", { mode: 0o600 });
vi.stubEnv("PI_CODING_AGENT_DIR", root);
const model: RuntimeModel = {
id: "deepseek/deepseek-v4-pro", provider: "deepseek", model: "deepseek-v4-pro",
label: "DeepSeek", upstreamModel: "deepseek-v4-pro",
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
sessionAdapter: { mode: "pi_builtin" }, session: { reasoning: false },
};
try {
const service = createPiManagement(loadConfig({ THT_SECRETS_FILE: secret }), {
modelCatalog: {
defaultInteraction: model.id, embedding: null,
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
},
execute: successfulExec([]), readSettings: () => ({ thinking: "medium" }),
});
expect((await service.status()).credentials).toBe(missingKey ? "missing" : "present");
} finally {
vi.unstubAllEnvs();
rmSync(root, { recursive: true, force: true });
}
});
function configFor(settingsFile = join(mkdtempSync(join(tmpdir(), "tht-pi-management-")), "settings.json")) {
return loadConfig({
@@ -15,12 +42,12 @@ function configFor(settingsFile = join(mkdtempSync(join(tmpdir(), "tht-pi-manage
SETTINGS_FILE: settingsFile,
PI_BIN: "/usr/local/bin/pi",
PI_MANAGEMENT_TIMEOUT_MS: "750",
THT_HOST_PLATFORM: "linux",
});
}
const modelCatalog: RuntimeModelCatalog = {
defaultSession: "zai/glm-5.2",
defaultMetadataGeneration: null,
defaultInteraction: "zai/glm-5.2",
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
sessionModels: () => [],
metadataModels: () => [],
@@ -34,6 +61,16 @@ function successfulExec(calls: Array<{ command: string; args: string[]; timeout:
};
}
test.each([
["linux", "linux"], ["darwin", "macos"], ["windows", "windows"],
])("reports installation host %s independently of the backend container OS", async (host, expected) => {
const service = createPiManagement(loadConfig({ THT_HOST_PLATFORM: host }), {
modelCatalog, execute: successfulExec([]), readSettings: () => ({ thinking: "medium" }),
credentialStatus: () => "missing",
});
expect((await service.status()).hostPlatform).toBe(expected);
});
// Catches a Pi executable that emits unexpected text or is invoked through a shell, which could
// turn a version display into a command-injection or information-disclosure surface.
test("status parses only a Pi version from a fixed execFile argument array", async () => {
@@ -47,6 +84,7 @@ test("status parses only a Pi version from a fixed execFile argument array", asy
});
await expect(service.status()).resolves.toEqual({
hostPlatform: "linux",
version: "0.80.3",
ready: true,
credentials: "missing",
@@ -78,6 +116,7 @@ test.each(["present", "missing"] as const)(
const status = await service.status();
expect(status).toEqual({
hostPlatform: "linux",
version: "0.80.3",
ready: true,
credentials,
+54 -16
View File
@@ -190,6 +190,29 @@ test("Pi receives the leased workspace runtime config and releases it on direct
expect(release).toHaveBeenCalledOnce();
});
test("Pi language launch hint comes from the session options and never ambient environment", () => {
const previous = process.env.THT_INTERACTION_LANGUAGE;
process.env.THT_INTERACTION_LANGUAGE = "it";
const environments: NodeJS.ProcessEnv[] = [];
const mgr = new PiProcessManager(loadConfig({}), {
spawnFn: (_command, _args, options) => {
environments.push(options.env);
return recordingChild() as any;
},
});
try {
mgr.createFor("explicit-language", { interactionLanguage: "en" });
mgr.teardown("explicit-language");
mgr.createFor("manifest-resolved-in-gate");
mgr.teardown("manifest-resolved-in-gate");
expect(environments[0].THT_INTERACTION_LANGUAGE).toBe("en");
expect(environments[1]).not.toHaveProperty("THT_INTERACTION_LANGUAGE");
} finally {
if (previous === undefined) delete process.env.THT_INTERACTION_LANGUAGE;
else process.env.THT_INTERACTION_LANGUAGE = previous;
}
});
test("a close-only child event releases its temporary Pi agent snapshot", () => {
const child = recordingChild();
let snapshotDir: string | undefined;
@@ -642,27 +665,33 @@ test("session Pi spawn reads the single secret bundle and scrubs its path", asyn
}
});
test("session Pi spawn resolves the selected catalog credential from the secret bundle", () => {
test.each([false, true])("session Pi uses the catalog bundle despite stale Pi auth: %s", (missingKey) => {
const root = mkdtempSync(path.join(tmpdir(), "thothii-catalog-credential-"));
const agentDir = path.join(root, "agent");
mkdirSync(agentDir, { mode: 0o700 });
writeFileSync(path.join(agentDir, "auth.json"), "{}\n", { mode: 0o600 });
const originalAuth = JSON.stringify({
deepseek: { type: "api_key", key: "stale-key" },
anthropic: { type: "api_key", key: "unrelated-key" },
});
writeFileSync(path.join(agentDir, "auth.json"), originalAuth, { mode: 0o600 });
writeFileSync(path.join(agentDir, "models.json"), '{"providers":{}}\n', { mode: 0o600 });
const secret = path.join(root, "thothii.secrets");
writeFileSync(secret, "ZAI_API_KEY=catalog-secret\nTHT_MODEL_API_KEY=legacy-secret\n", { mode: 0o600 });
writeFileSync(secret, missingKey ? "THT_MODEL_API_KEY=legacy-secret\n"
: "DEEPSEEK_API_KEY=catalog-secret\nTHT_MODEL_API_KEY=legacy-secret\n", { mode: 0o600 });
const legacyKey = path.join(root, "legacy-key");
writeFileSync(legacyKey, "legacy-file-key", { mode: 0o600 });
const model: RuntimeModel = {
id: "openai/test-model",
provider: "openai",
model: "test-model",
label: "Test model",
upstreamModel: "test-model",
authentication: { mode: "secret_env", apiKeyEnv: "ZAI_API_KEY" },
id: "deepseek/deepseek-v4-pro",
provider: "deepseek",
model: "deepseek-v4-pro",
label: "DeepSeek V4 Pro",
upstreamModel: "deepseek-v4-pro",
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
sessionAdapter: { mode: "pi_builtin" },
session: { reasoning: false },
};
const modelCatalog: RuntimeModelCatalog = {
defaultSession: model.id,
defaultMetadataGeneration: null,
defaultInteraction: model.id,
embedding: null,
sessionModels: () => [model],
metadataModels: () => [],
@@ -672,17 +701,26 @@ test("session Pi spawn resolves the selected catalog credential from the secret
const child = recordingChild();
child.stderr.resume = () => {};
vi.stubEnv("PI_CODING_AGENT_DIR", agentDir);
const mgr = new PiProcessManager(loadConfig({ THT_SECRETS_FILE: secret }), {
const mgr = new PiProcessManager(loadConfig({ THT_SECRETS_FILE: secret, THT_MODEL_API_KEY_FILE: legacyKey }), {
modelCatalog,
authProviders: () => new Set(),
authProviders: () => new Set(["deepseek"]),
spawnFn: (...args: any[]) => { calls.push(args); return child as any; },
});
try {
mgr.createFor("catalog-credential", { provider: "openai", model: "test-model" });
expect(calls[0][2].env.ZAI_API_KEY).toBe("catalog-secret");
const create = () => mgr.createFor("catalog-credential", { provider: model.provider, model: model.model });
if (missingKey) {
expect(create).toThrow("model provider credential is unavailable");
expect(calls).toHaveLength(0);
return;
}
create();
expect(calls[0][2].env.DEEPSEEK_API_KEY).toBe("catalog-secret");
expect(calls[0][2].env).not.toHaveProperty("OPENAI_API_KEY");
expect(calls[0][2].env).not.toHaveProperty("THT_MODEL_API_KEY");
expect(JSON.parse(readFileSync(path.join(calls[0][2].env.PI_CODING_AGENT_DIR, "auth.json"), "utf8")))
.toEqual({ anthropic: { type: "api_key", key: "unrelated-key" } });
} finally {
expect(readFileSync(path.join(agentDir, "auth.json"), "utf8")).toBe(originalAuth);
mgr.teardown("catalog-credential");
vi.unstubAllEnvs();
rmSync(root, { recursive: true, force: true });
@@ -750,7 +788,7 @@ test("set_model translates a canonical catalog key to its upstream Pi model ID",
session: { reasoning: false, contextWindow: 32768, maxTokens: 8192 },
};
const modelCatalog: RuntimeModelCatalog = {
defaultSession: model.id, defaultMetadataGeneration: null, embedding: null,
defaultInteraction: model.id, embedding: null,
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
};
const mgr = new PiProcessManager(loadConfig({ PI_BIN: "/usr/local/bin/pi" }), {
+7 -4
View File
@@ -61,20 +61,22 @@ test("provider smoke resolves the selected catalog credential from the secret bu
session: { reasoning: false },
};
const modelCatalog: RuntimeModelCatalog = {
defaultSession: model.id,
defaultMetadataGeneration: null,
defaultInteraction: model.id,
embedding: null,
sessionModels: () => [model],
metadataModels: () => [],
hasSession: (id) => id === model.id,
};
let spawnEnv: NodeJS.ProcessEnv | undefined;
const readAuthStore = vi.fn(() => JSON.stringify({ openai: { type: "api_key", key: "stale-key" } }));
const smoke = createPiProviderSmoke(loadConfig({ THT_SECRETS_FILE: secret }), {
modelCatalog,
authProviders: () => new Set(),
authProviders: () => new Set(["openai"]),
readAuthStore,
readModelsStore: () => undefined,
spawnFn: (_command, _args, options) => {
spawnEnv = options.env;
expect(existsSync(join(options.env.PI_CODING_AGENT_DIR!, "auth.json"))).toBe(false);
return successfulProviderChild();
},
});
@@ -85,6 +87,7 @@ test("provider smoke resolves the selected catalog credential from the secret bu
expect(spawnEnv?.ZAI_API_KEY).toBe("catalog-secret");
expect(spawnEnv).not.toHaveProperty("OPENAI_API_KEY");
expect(spawnEnv).not.toHaveProperty("THT_MODEL_API_KEY");
expect(readAuthStore).not.toHaveBeenCalled();
} finally {
rmSync(root, { recursive: true, force: true });
}
@@ -306,7 +309,7 @@ test("provider smoke makes one configured request from an isolated no-capability
sessionAdapter: { mode: "pi_builtin" }, session: { reasoning: true },
};
const smokeCatalog: RuntimeModelCatalog = {
defaultSession: smokeModel.id, defaultMetadataGeneration: null, embedding: null,
defaultInteraction: smokeModel.id, embedding: null,
sessionModels: () => [smokeModel], metadataModels: () => [],
hasSession: (id) => id === smokeModel.id,
};
+286 -54
View File
@@ -27,10 +27,9 @@ function operationalWorkspace(id = "default") {
} as const;
}
function sessionCatalog(defaultSession = "zai/glm-5.2", available = [defaultSession]) {
function sessionCatalog(defaultInteraction = "zai/glm-5.2", available = [defaultInteraction]) {
return {
defaultSession,
defaultMetadataGeneration: null,
defaultInteraction,
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
sessionModels: () => [],
metadataModels: () => [],
@@ -54,7 +53,11 @@ const defaultWorkspaceRegistry = {
function buildApp(config: Parameters<typeof buildRealApp>[0], deps: Record<string, unknown> = {}) {
const thtRunner = deps.thtRunner
? { qdrantEnsure: async () => ({ ok: true }), ...(deps.thtRunner as object) }
? {
qdrantEnsure: async () => ({ ok: true }),
ensureInteractionLanguage: async () => ({ interaction_language: "en" }),
...(deps.thtRunner as object),
}
: undefined;
return buildRealApp(config, {
workspaceRuntimeSupport: () => true,
@@ -89,6 +92,156 @@ const aliceHeaders = {
"x-thoth-is-admin": "0",
};
test.each([undefined, null, "", "en--US", "en_US", "en\nIGNORE", "en<script>", 42])(
"new sessions reject invalid interaction language %s before persistence", async (interactionLanguage) => {
const app = mutApp({ sessionNew: async () => { throw new Error("must not create"); } });
try {
const response = await app.inject({
method: "POST", url: "/sessions", payload: { question: "Pazienti", interactionLanguage },
});
expect(response.statusCode).toBe(400);
expect(response.json()).toMatchObject({ code: "invalid_interaction_language" });
} finally { await app.close(); }
},
);
test.each([["en", "en"], ["fr-FR", "fr-FR"], ["FR-fr", "fr-FR"]])(
"new session forwards canonical interaction language %s without changing the question", async (language, canonical) => {
let created: any;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
sessionNew: async (options: any) => { created = options; return { id: "language" }; },
searchPack: async () => {},
},
readiness: { ensure: async () => ({ ok: true }) },
mgr: {
get: () => undefined, createFor: () => ({ bridge: { onClientEvent: () => {} } }),
configure: async () => {}, start: () => {},
},
getSettings: () => ({ workspace: "default" }),
});
try {
const response = await app.inject({
method: "POST", url: "/sessions", payload: { question: "Pazienti", interactionLanguage: language },
});
expect(response.statusCode).toBe(200);
expect(created).toMatchObject({ question: "Pazienti", interactionLanguage: canonical });
} finally { await app.close(); }
});
test("resume rejects browser interaction language overrides", async () => {
const app = mutApp({ sessionShow: async () => ({ status: "open", interaction_language: "it" }) });
try {
const response = await app.inject({
method: "POST", url: "/sessions/s1/resume", payload: { interactionLanguage: "en" },
});
expect(response.statusCode).toBe(400);
expect(response.json()).toMatchObject({ code: "interaction_language_pinned" });
} finally { await app.close(); }
});
test("new session starts Pi with the question language persisted by the harness", async () => {
let runtime: any;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
sessionNew: async () => ({ id: "english-question", interaction_language: "en" }),
searchPack: async () => {},
},
readiness: { ensure: async () => ({ ok: true }) },
mgr: {
get: () => undefined,
createFor: (_id: string, options: any) => {
runtime = options;
return { bridge: { onClientEvent: () => {} } };
},
configure: async () => {}, start: () => {},
},
getSettings: () => ({ workspace: "default" }),
});
try {
const response = await app.inject({ method: "POST", url: "/sessions", payload: {
question: "How many patients were admitted last year?", interactionLanguage: "it",
} });
expect(response.statusCode).toBe(200);
expect(runtime.interactionLanguage).toBe("en");
} finally { await app.close(); }
});
test("resume pins legacy interaction language using the resolved harness workspace", async () => {
let pinnedWorkspace: string | undefined;
let runtime: any;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
sessionShow: async () => ({ id: "legacy", status: "closed" }),
ensureInteractionLanguage: async (_id: string, workspace: string) => {
pinnedWorkspace = workspace;
return { interaction_language: "it" };
},
reopenSession: async () => {},
},
readiness: { ensure: async () => ({ ok: true }) },
mgr: {
get: () => undefined,
createFor: (_id: string, options: any) => {
runtime = options;
return { bridge: { onClientEvent: () => {} } };
},
configure: async () => {}, start: () => {},
},
getSettings: () => ({ workspace: "other-browser-workspace" }),
});
try {
const response = await app.inject({ method: "POST", url: "/sessions/legacy/resume" });
expect(response.statusCode).toBe(200);
expect(pinnedWorkspace).toContain("/default.yaml");
expect(runtime).toMatchObject({ interactionLanguage: "it", mode: "resume" });
} finally { await app.close(); }
});
test("resume retains persisted interaction language despite different browser settings", async () => {
let options: any;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
sessionShow: async () => ({ id: "saved", status: "closed", interaction_language: "it" }),
ensureInteractionLanguage: async () => { throw new Error("language is already pinned"); },
reopenSession: async () => {},
},
readiness: { ensure: async () => ({ ok: true }) },
mgr: {
get: () => undefined,
createFor: (_id: string, value: any) => {
options = value;
return { bridge: { onClientEvent: () => {} } };
},
configure: async () => {}, start: () => {},
},
getSettings: () => ({ workspace: "english-workspace", locale: "en" }),
});
try {
const response = await app.inject({ method: "POST", url: "/sessions/saved/resume" });
expect(response.statusCode).toBe(200);
expect(options.interactionLanguage).toBe("it");
} finally { await app.close(); }
});
test("resume cannot start Pi if legacy interaction language cannot be persisted", async () => {
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
sessionShow: async () => ({ id: "legacy", status: "closed" }),
ensureInteractionLanguage: async () => { throw new Error("private storage details"); },
reopenSession: async () => { throw new Error("must not reopen"); },
},
readiness: { ensure: async () => ({ ok: true }) },
mgr: { createFor: () => { throw new Error("must not spawn"); } },
getSettings: () => ({ workspace: "default" }),
});
try {
const response = await app.inject({ method: "POST", url: "/sessions/legacy/resume" });
expect(response.statusCode).toBe(503);
expect(response.body).not.toContain("private storage details");
} finally { await app.close(); }
});
test("upstream requests without a principal fail before a Pi runtime can be created", async () => {
let created = false;
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
@@ -96,7 +249,7 @@ test("upstream requests without a principal fail before a Pi runtime can be crea
thtRunner: {} as any,
});
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(response.statusCode).toBe(401);
expect(created).toBe(false);
@@ -134,7 +287,7 @@ test("maintenance rejects new and resumed session admission without interrupting
});
const create = await app.inject({
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { question: "q" },
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { interactionLanguage: "en", question: "q" },
});
const resume = await app.inject({ method: "POST", url: "/sessions/open/resume", headers: aliceHeaders });
@@ -155,7 +308,7 @@ test("a durable maintenance marker initializes admission closed after backend re
AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness", THT_MAINTENANCE_FILE: marker,
}), { thtRunner: {} as any });
const response = await app.inject({
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { question: "q" },
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { interactionLanguage: "en", question: "q" },
});
expect(response.statusCode).toBe(503);
expect(response.json()).toMatchObject({ code: "maintenance" });
@@ -364,6 +517,65 @@ test("retention scans a removed workspace's retained snapshot", async () => {
expect(response.json()).toEqual([expect.objectContaining({ id: "resumable" })]);
});
test("active sessions remain available when retained snapshot discovery is unreadable", async () => {
const activeSnapshot = "/registry/snapshots/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/active.yaml";
const retainedFailure = "invalid retained snapshot /registry/snapshots/secret/legacy.yaml";
const reconcileSnapshotRetention = vi.fn(async () => {});
const subscribed = vi.fn();
const warning = vi.spyOn(console, "warn").mockImplementation(() => {});
const manifest = {
id: "live-session", status: "open", archived: false,
workspace_id: "active", workspace_revision: "a".repeat(40),
};
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
withPrincipal: () => ({
sessionList: async (snapshotPath: string) => {
expect(snapshotPath).toBe(activeSnapshot);
return [manifest];
},
sessionShow: async (id: string, snapshotPath?: string) => {
if (id === manifest.id && snapshotPath === activeSnapshot) return manifest;
throw new Error("session not found");
},
}),
} as any,
mgr: { get: () => undefined } as any,
hub: {
subscribe: (_id: string, _send: unknown, options: { close?: () => void }) => {
subscribed();
options.close?.();
return () => {};
},
} as any,
workspaceRegistry: {
listRetainedSnapshots: async () => { throw new Error(retainedFailure); },
list: async () => [{
id: "active", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: activeSnapshot,
}],
reconcileSnapshotRetention,
} as any,
});
try {
const list = await app.inject({ method: "GET", url: "/sessions", headers: aliceHeaders });
const detail = await app.inject({ method: "GET", url: `/sessions/${manifest.id}`, headers: aliceHeaders });
const events = await app.inject({ method: "GET", url: `/sessions/${manifest.id}/events`, headers: aliceHeaders });
expect(list.statusCode).toBe(200);
expect(list.json()).toEqual([expect.objectContaining({ id: manifest.id, active: false })]);
expect(detail.statusCode).toBe(200);
expect(detail.json()).toMatchObject(manifest);
expect(events.statusCode).toBe(200);
expect(subscribed).toHaveBeenCalledOnce();
expect(reconcileSnapshotRetention).not.toHaveBeenCalled();
expect(list.body + detail.body + events.body).not.toContain(retainedFailure);
} finally {
warning.mockRestore();
await app.close();
}
});
test("the single local installation listing reconciles its resumable workspace pins", async () => {
const retained = vi.fn(async () => {});
const retainedRevision = "d".repeat(40);
@@ -434,7 +646,7 @@ test("new sessions are created through the authenticated principal, not a client
const response = await app.inject({
method: "POST", url: "/sessions", headers: aliceHeaders,
payload: { question: "q", owner: "mallory" },
payload: { interactionLanguage: "en", question: "q", owner: "mallory" },
});
expect(response.statusCode).toBe(200);
@@ -451,7 +663,7 @@ test("new sessions reject the client legacy workspace field unless local legacy
});
const response = await app.inject({
method: "POST", url: "/sessions", payload: { question: "q", workspace: "legacy" },
method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q", workspace: "legacy" },
});
expect(response.statusCode).toBe(409);
@@ -471,7 +683,7 @@ test("explicit local legacy mode permits the unpinned client workspace request",
});
const response = await app.inject({
method: "POST", url: "/sessions", payload: { question: "q", workspace: "legacy" },
method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q", workspace: "legacy" },
});
expect(response.statusCode).toBe(200);
@@ -509,7 +721,7 @@ test("creates a session from the active immutable workspace revision", async ()
await app.inject({
method: "POST", url: "/sessions",
payload: { question: "q", workspaceId: "psd-clinical", provider: "zai", model: "glm-5.2", thinking: "low" },
payload: { interactionLanguage: "en", question: "q", workspaceId: "psd-clinical", provider: "zai", model: "glm-5.2", thinking: "low" },
});
expect(sessionNew).toHaveBeenCalledWith(expect.objectContaining({
@@ -553,7 +765,7 @@ test("hands one Catalog-backed runtime to both retrieval and Pi", async () => {
const response = await app.inject({
method: "POST",
url: "/sessions",
payload: { question: "Which users placed orders?", workspaceId: "default" },
payload: { interactionLanguage: "en", question: "Which users placed orders?", workspaceId: "default" },
});
expect(response.statusCode).toBe(200);
@@ -629,7 +841,7 @@ test("refuses core admission when the workspace preprocessing fingerprint is sta
const response = await app.inject({
method: "POST",
url: "/sessions",
payload: { question: "q", workspaceId: "default" },
payload: { interactionLanguage: "en", question: "q", workspaceId: "default" },
});
expect(response.statusCode).toBe(409);
@@ -681,7 +893,7 @@ test("rejects an SSH-only Catalog binding before persisting or starting a sessio
} as any,
});
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(response.statusCode).toBe(409);
expect(response.json()).toMatchObject({ code: "workspace_not_activatable" });
@@ -721,7 +933,7 @@ test("hands a revision lease to retention only after the session manifest is dur
workspaceRegistry: { acquireSessionRevision } as any,
});
const request = app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const request = app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
await new Promise((resolve) => setImmediate(resolve));
expect(markPersisted).not.toHaveBeenCalled();
expect(abort).not.toHaveBeenCalled();
@@ -752,7 +964,7 @@ test("creates a session from the configured default workspace revision when work
workspaceRegistry: registry as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(registry.read).toHaveBeenCalledWith("psd-clinical");
expect(sessionNew).toHaveBeenCalledWith(expect.objectContaining({
@@ -838,7 +1050,7 @@ test("session lifecycle locates a B session when installation default is A", asy
} as any,
});
expect((await app.inject({ method: "POST", url: "/sessions", payload: {
expect((await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en",
question: "B question", workspaceId: "b-workspace", provider: "zai", model: "glm-5.2", thinking: "low",
} })).statusCode).toBe(200);
expect((await app.inject({ method: "GET", url: "/sessions" })).json()).toEqual([
@@ -850,7 +1062,7 @@ test("session lifecycle locates a B session when installation default is A", asy
active = undefined;
expect((await app.inject({ method: "POST", url: "/sessions/session-b/resume" })).json())
.toEqual({ id: "session-b", alreadyActive: false });
.toEqual({ id: "session-b", alreadyActive: false, workspaceId: "b-workspace" });
expect(calls).toContain(`new:${bPath}`);
expect(calls).toContain(`list:${bPath}`);
expect(calls).toContain(`show:${bPath}`);
@@ -881,7 +1093,7 @@ test("POST /sessions uses the catalog default with workspace/thinking settings a
],
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
});
const created = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const created = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(created.json()).toEqual({ id: "s1" });
expect(sessionNewArg.workspaceConfigPath).toContain(`/snapshots/${"e".repeat(40)}/w.yaml`);
expect(sessionNewArg.provider).toBe("zai");
@@ -942,11 +1154,11 @@ test("POST /sessions stops the user's previous open Pi runtime before creating a
getSettings: () => ({ workspace: "local" }) as any,
});
expect((await app.inject({ method: "POST", url: "/sessions", payload: { question: "one" } })).statusCode)
expect((await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "one" } })).statusCode)
.toBe(200);
order.length = 0;
const second = await app.inject({ method: "POST", url: "/sessions", payload: { question: "two" } });
const second = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "two" } });
expect(second.statusCode).toBe(200);
expect(second.json()).toEqual({ id: "s2" });
@@ -967,7 +1179,7 @@ test("POST /sessions refuses to create a session when the local DWH precheck fai
readiness: { ensure: async () => ({ ok: true }) } as any,
getSettings: () => ({ workspace: "w" }) as any,
});
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(res.statusCode).toBe(503);
expect(res.json()).toMatchObject({ code: "dwh_unreachable" });
expect(pinged).toBe(1);
@@ -988,7 +1200,7 @@ test("POST /sessions proceeds past a passing DWH precheck", async () => {
getSettings: () => ({ workspace: "w" }) as any,
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
});
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(res.statusCode).toBe(200);
expect(pinged).toBe(1);
expect(created).toBe(1);
@@ -1007,7 +1219,7 @@ test("POST /sessions skips the DWH precheck when the flag is off (default)", asy
getSettings: () => ({ workspace: "w" }) as any,
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
});
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(res.statusCode).toBe(200);
expect(pinged).toBe(0); // probe never runs without the flag
});
@@ -1041,7 +1253,7 @@ test("POST /sessions configura Pi con il thinking globale selezionato", async ()
],
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
await new Promise((resolve) => setImmediate(resolve));
expect(configured.thinking).toBe("high");
@@ -1120,6 +1332,29 @@ test("POST /sessions/:id/resume uses the manifest's retained workspace revision"
);
});
test.each([undefined, { provider: "local", model: "qwen" }])("resume uses the global model/default, not the historical manifest (%j)", async (payload) => {
const configure = vi.fn(async () => {});
const manifest = { status: "open", archived: false, provider: "retired", model: "old-model" };
const runtime = { bridge: { onClientEvent: () => {}, emitClientEvent: () => {} } };
let current: any;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
mgr: {
get: () => current,
createFor: () => { current = runtime; return runtime; },
configure, start: () => {},
} as any,
thtRunner: { sessionShow: async () => manifest, reopenSession: async () => {} } as any,
readiness: { ensure: async () => ({ ok: true }) } as any,
runtimeModelCatalog: sessionCatalog("zai/glm-5.2", ["zai/glm-5.2", "local/qwen"]),
getSettings: () => ({ workspace: "psd", thinking: "medium" }) as any,
});
const response = await app.inject({ method: "POST", url: "/sessions/model-resume/resume", ...(payload ? { payload } : {}) });
expect(response.statusCode).toBe(200);
await new Promise((resolve) => setImmediate(resolve));
expect(configure).toHaveBeenCalledWith(expect.anything(), expect.objectContaining(payload ?? { provider: "zai", model: "glm-5.2" }));
expect(manifest).toMatchObject({ provider: "retired", model: "old-model" });
});
test("POST /sessions/:id/resume returns a sanitized error when its retained revision is unavailable", async () => {
const rawFailure = "cannot read /data/workspace-registry/snapshots/secret-revision";
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
@@ -1346,7 +1581,7 @@ test("resuming a different session stops the user's previous Pi runtime", async
readiness: { ensure: async () => ({ ok: true }) } as any,
getSettings: () => ({ workspace: "local" }) as any,
});
expect((await app.inject({ method: "POST", url: "/sessions", payload: { question: "one" } })).statusCode)
expect((await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "one" } })).statusCode)
.toBe(200);
const resumed = await app.inject({ method: "POST", url: "/sessions/s2/resume" });
@@ -1734,13 +1969,13 @@ test.each([
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "old" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "old" } });
if (lifecycle === "close") {
await app.inject({ method: "POST", url: "/sessions/s1/close" });
await app.inject({ method: "POST", url: "/sessions/s1/resume" });
} else {
await app.inject({ method: "DELETE", url: "/sessions/s1" });
await app.inject({ method: "POST", url: "/sessions", payload: { question: "replacement" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "replacement" } });
}
await new Promise((resolve) => setImmediate(resolve));
expect(current).toBe(replacement);
@@ -1804,7 +2039,7 @@ test.each(["resolve", "reject"] as const)(
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "old" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "old" } });
await app.inject({ method: "DELETE", url: "/sessions/s1" });
published.length = 0;
@@ -1996,7 +2231,7 @@ test("Close suppresses a bootstrap that settles while close persistence is pendi
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
const closeResponse = app.inject({ method: "POST", url: "/sessions/s1/close" });
await closeStarted;
published.length = 0;
@@ -2070,7 +2305,7 @@ test("bootstrap failure persists once and keeps Resume serialized behind that pe
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
oldConfigure.reject(new Error("configure failed"));
await failureStarted;
const resumeResponse = app.inject({ method: "POST", url: "/sessions/s1/resume" })
@@ -2193,7 +2428,7 @@ test("a replaced runtime cannot publish or fail the newly resumed session", asyn
readiness: { ensure: async () => ({ ok: true }) } as any,
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
await new Promise((resolve) => setImmediate(resolve));
oldBridge.setState("idle");
@@ -2253,7 +2488,7 @@ test("a deleted runtime cannot repopulate or fail the forgotten session", async
readiness: { ensure: async () => ({ ok: true }) } as any,
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
await new Promise((resolve) => setImmediate(resolve));
const response = await app.inject({ method: "DELETE", url: "/sessions/s1" });
@@ -2298,7 +2533,7 @@ test("an unexpectedly exited runtime publishes its terminal sequence then releas
readiness: { ensure: async () => ({ ok: true }) } as any,
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
await new Promise((resolve) => setImmediate(resolve));
published.length = 0;
@@ -2349,7 +2584,7 @@ test("agent_end releases the Pi runtime after the session was finalized", async
readiness: { ensure: async () => ({ ok: true }) } as any,
getSettings: () => ({ workspace: "local" }) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
await new Promise((resolve) => setImmediate(resolve));
bridge.emitClientEvent({ type: "system_event", event: "agent_end" });
@@ -2421,7 +2656,7 @@ test("POST /sessions/:id/response senza gate pendente risponde 409 (risposta sta
getSettings: () => ({ workspace: "w" }),
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
});
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
// The fake Pi never emitted a ui_request: the bridge has no pending descriptor, so a
// response (stale UI, double submit) must be rejected instead of forwarded to Pi.
const res = await app.inject({ method: "POST", url: "/sessions/s1/response",
@@ -2562,7 +2797,7 @@ test("POST /sessions readiness failure returns one fixed public message without
getSettings: () => ({ workspace: "psd" }) as any,
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
});
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(res.statusCode).toBe(503);
expect(res.json()).toEqual({
error: "Session services are not ready. Check configuration and connectivity, then try again.",
@@ -2583,7 +2818,7 @@ test.each(["semantic_index_incompatible", "workspace_not_activatable"] as const)
});
const response = await app.inject({
method: "POST", url: "/sessions", payload: { question: "q" },
method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" },
});
expect(response.statusCode).toBe(503);
@@ -2606,7 +2841,7 @@ test("POST /sessions returns storage 503 before creating a Pi runtime when sessi
mgr: { createFor: () => { piCreated = true; throw new Error("must not spawn"); } } as any,
});
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(response.statusCode).toBe(503);
expect(response.json()).toEqual({ error: "session storage is unavailable" });
@@ -2626,13 +2861,13 @@ test("POST /sessions proceeds when ollamaEnsure succeeds", async () => {
getSettings: () => ({ workspace: "psd" }) as any,
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
});
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(res.json()).toEqual({ id: "s1" });
expect(qdrantEnsure).toHaveBeenCalledWith(operationalWorkspace("psd"), 60, "self_heal");
expect(ensureWs).toContain(`/snapshots/${"e".repeat(40)}/psd.yaml`);
});
test("POST /sessions falls back from a stale requested model to the catalog default", async () => {
test("POST /sessions rejects a stale requested model without silently using the default", async () => {
let created = 0;
let persisted: any;
const runtime = { bridge: { onClientEvent: () => {} } };
@@ -2659,16 +2894,13 @@ test("POST /sessions falls back from a stale requested model to the catalog defa
const res = await app.inject({
method: "POST",
url: "/sessions",
payload: { question: "q", provider: "deepseek", model: "deepseek-v4-pro" },
payload: { interactionLanguage: "en", question: "q", provider: "deepseek", model: "deepseek-v4-pro" },
});
expect(res.statusCode).toBe(200);
expect(res.json()).toEqual({
id: "fallback",
warning: "Configured model deepseek/deepseek-v4-pro is unavailable; using zai/glm-5.2.",
});
expect(persisted).toMatchObject({ provider: "zai", model: "glm-5.2" });
expect(created).toBe(1);
expect(res.statusCode).toBe(503);
expect(res.json()).toMatchObject({ code: "model_unavailable" });
expect(persisted).toBeUndefined();
expect(created).toBe(0);
});
test("POST /sessions marks a persisted session failed when runtime construction throws", async () => {
@@ -2697,7 +2929,7 @@ test("POST /sessions marks a persisted session failed when runtime construction
],
});
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(res.statusCode).toBe(503);
expect(res.json()).toEqual({
@@ -2790,7 +3022,7 @@ test.each([
try {
const response = flow === "new"
? await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } })
? await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } })
: await app.inject({ method: "POST", url: `/sessions/${sessionId}/resume` });
const logs = consoleError.mock.calls.flat().map(String).join(" ");
@@ -2896,7 +3128,7 @@ test("POST /sessions returns after bridge attachment but starts only after retri
getSettings: () => ({ workspace: "psd" }) as any,
});
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(res.json()).toEqual({ id: "s-early" });
expect(bridgeAttached).toBe(true);
expect(started).toBe(false);
@@ -2943,7 +3175,7 @@ test("POST /sessions bootstrap failure emits only a fixed recovery message", asy
const response = await app.inject({
method: "POST",
url: "/sessions",
payload: { question: "q" },
payload: { interactionLanguage: "en", question: "q" },
});
await new Promise((resolve) => setImmediate(resolve));
@@ -3039,7 +3271,7 @@ test.each([
})),
} as any,
});
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
expect(response.statusCode).toBe(expectedStatus);
expect(ensure).toHaveBeenCalledTimes(reachesReadiness ? 1 : 0);
+2 -4
View File
@@ -160,8 +160,7 @@ test("GET /models returns session choices from the installation model catalog",
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {} as any,
runtimeModelCatalog: {
defaultSession: "zai/glm-5.2",
defaultMetadataGeneration: null,
defaultInteraction: "zai/glm-5.2",
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
sessionModels: () => [{
id: "zai/glm-5.2", provider: "zai", model: "glm-5.2", label: "GLM 5.2",
@@ -187,8 +186,7 @@ test("GET /models returns an empty list when the catalog has no session models",
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {} as any,
runtimeModelCatalog: {
defaultSession: null,
defaultMetadataGeneration: null,
defaultInteraction: null,
embedding: null,
sessionModels: () => [],
metadataModels: () => [],
+13
View File
@@ -34,6 +34,19 @@ test("sessionNew parses id from JSON", async () => {
expect(await r.sessionNew({ question: "q" })).toEqual({ id: "2026-06-27-100000-x" });
});
test("session language uses public per-command CLI flags and the selected config", async () => {
const runner = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" });
(spawn as any).mockClear();
await runner.sessionNew({ question: "Pazienti", interactionLanguage: "en" });
expect((spawn as any).mock.calls[0][1]).toEqual([
"session", "new", "Pazienti", "--interaction-language", "en", "--json", "-c", "config/tht.yaml",
]);
await runner.ensureInteractionLanguage("s1");
expect((spawn as any).mock.calls[1][1]).toEqual([
"session", "ensure-interaction-language", "s1", "--json", "-c", "config/tht.yaml",
]);
});
test("searchPack persists retrieval context with session and workspace", async () => {
const calls: any[] = [];
const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" });
@@ -10,6 +10,28 @@ import {
const roots: string[] = [];
test("Evidence consolidation runs only its stage and preserves blocked Catalog readiness", async () => {
const catalog = repository();
const runChild = vi.fn(async () => ({ exitCode: 0, stdout: JSON.stringify({ status: "succeeded", counts: { documents: 2 } }), stderr: "" }));
const result = await service({ repository: catalog, runChild }).consolidateEvidence({ workspaceId: "catalog-workspace" });
expect(result.status).toBe("succeeded");
expect(runChild).toHaveBeenCalledOnce();
expect(runChild.mock.calls[0]?.[0]).toMatchObject({ argv: ["preprocess", "evidence", "--consolidate", "--json", "-c", "/dev/fd/3"] });
expect(catalog.beginPreprocessing).not.toHaveBeenCalled();
expect(catalog.finishPreprocessing).not.toHaveBeenCalled();
expect(catalog.clearPreprocessing).not.toHaveBeenCalled();
});
test("Evidence index failure reports retry without changing Catalog state", async () => {
const catalog = repository();
const runChild = vi.fn(async () => ({ exitCode: 1, stdout: JSON.stringify({ status: "failed", saved: true, error: "Saved; retry indexing." }), stderr: "private provider endpoint" }));
const result = await service({ repository: catalog, runChild }).consolidateEvidence({ workspaceId: "catalog-workspace" });
expect(result.status).toBe("failed");
expect(result.warnings).toEqual(["Saved; retry indexing."]);
expect(catalog.finishPreprocessing).not.toHaveBeenCalled();
expect(JSON.stringify(result)).not.toContain("private provider");
});
afterEach(() => {
roots.splice(0).forEach((root) => rmSync(root, { recursive: true, force: true }));
});
@@ -254,6 +276,24 @@ test("semantic preflight failure is persisted before returning", async () => {
);
});
test.each(["refresh", "decide"] as const)("Evidence %s calls only the source worker and preserves Catalog readiness", async action => {
const catalog = repository();
const runChild = vi.fn(async () => ({exitCode: 0, stdout: JSON.stringify({status: "succeeded", counts: {changed: 1}}), stderr: ""}));
const result = await service({repository: catalog, runChild}).evidenceSources({workspaceId: "catalog-workspace", action,
...(action === "decide" ? {sourceId: "a".repeat(64), revision: "b".repeat(64), decision: "replace" as const} : {}), actor: "Curator"});
expect(result.status).toBe("succeeded");
expect(runChild).toHaveBeenCalledWith({configPath: expect.any(String), argv: expect.arrayContaining(["evidence", "sources", action, "-c", "/dev/fd/3", "--actor", "Curator"])});
expect(catalog.beginPreprocessing).not.toHaveBeenCalled();
expect(catalog.finishPreprocessing).not.toHaveBeenCalled();
expect(catalog.clearPreprocessing).not.toHaveBeenCalled();
});
test("failed source activation reports the saved decision for retry", async () => {
const result = await service({runChild: async () => ({exitCode: 1, stdout: JSON.stringify({status: "failed", saved: true, error: "Decision saved; retry indexing"}), stderr: "secret"})})
.evidenceSources({workspaceId: "catalog-workspace", action: "decide", sourceId: "a".repeat(64), revision: "b".repeat(64), decision: "keep"});
expect(result).toMatchObject({status: "failed", warnings: ["Decision saved; retry indexing"]});
});
test("clear invalidates Catalog readiness before clearing only derived worker data", async () => {
const catalog = repository();
const runChild = vi.fn(async () => ({
@@ -1,4 +1,5 @@
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import {
existsSync,
mkdtempSync,
@@ -202,7 +203,7 @@ test("deterministic operator leases are keyed by logical identity and stable acr
workspaceSecretStore: f.workspaceSecretStore,
});
const suffix = first.inputFingerprint.slice(7, 23);
const suffix = createHash("sha256").update(first.inputFingerprint + "\n" + readFileSync(first.path, "utf8")).digest("hex").slice(0, 16);
expect(second.path).toBe(first.path);
expect(first.path).toBe(join(
f.dataRoot,
@@ -242,6 +242,7 @@ test("separate runtime leases hand off byte-identical revision Evidence configs
source_identity: "workspace://psd-clinical",
});
expect(parse(firstYaml).evidence).toEqual({
local_archive_root: join(f.registryConfig.root, "repo", "psd-clinical"),
sources: [{
type: "filesystem",
root: expectedRoot,
@@ -192,6 +192,7 @@ test("renders filesystem Evidence below the immutable revision content root with
expect(rendered.runtime_identity.workspace_revision).toBe(evidenceRevision);
expect(rendered.evidence).toEqual({
schema_version: 2,
local_archive_root: "/srv/registry/repo/psd-clinical",
sources: [{
type: "filesystem",
root: `/srv/registry/snapshots/${evidenceRevision}/psd-clinical/evidence`,
@@ -203,7 +204,7 @@ test("renders filesystem Evidence below the immutable revision content root with
max_chunk_chars: 4_000,
retain_published_generations: 3,
});
expect(yaml).not.toContain("/srv/registry/repo");
expect(rendered.evidence.sources[0].root).not.toContain("/srv/registry/repo");
});
test("renders public HTTP Evidence with exact fractional-second timeouts and every policy limit", () => {
@@ -223,6 +224,7 @@ test("renders public HTTP Evidence with exact fractional-second timeouts and eve
}));
expect(rendered.evidence).toEqual({
local_archive_root: "/srv/registry/repo/psd-clinical",
sources: [{
type: "http",
urls: ["https://evidence.example.test/guide.md"],
+3 -2
View File
@@ -13,6 +13,7 @@ services:
THT_BIN: /opt/venv/bin/tht
THT_DATA_ROOT: /data
SETTINGS_FILE: /data/settings/settings.json
THT_EVIDENCE_HOST_REGISTRY_ROOT: ${THT_WORKSPACE_REGISTRY_ROOT:-}
THT_MAINTENANCE_FILE: /data/settings/maintenance.json
THT_WORKSPACE_REGISTRY_ROOT: /data/workspace-registry
THT_WORKSPACE_GIT_REMOTE: ${THT_WORKSPACE_GIT_REMOTE:?set THT_WORKSPACE_GIT_REMOTE}
@@ -99,7 +100,7 @@ services:
image: thothii-core:local
profiles: [catalog-maintenance]
pull_policy: never
command: ["node", "/app/backend/dist/catalog/migrate.js"]
command: ["bash", "/app/docker/catalog-migrate.sh"]
environment:
THT_CATALOG_DB_HOST: catalog-db
THT_CATALOG_DB_PORT: "5432"
@@ -153,7 +154,7 @@ services:
- type: volume
source: workspace-registry
target: /data/workspace-registry
read_only: true
read_only: false
- type: volume
source: sessions
target: /data/sessions
+20
View File
@@ -0,0 +1,20 @@
# Optional local profile: expose the existing curator checkout on the installation host.
# Copy the existing registry repo to this path before enabling the override. Snapshot/state
# volumes remain unchanged. THT_EVIDENCE_HOST_REGISTRY_ROOT must be an absolute host path.
services:
core:
environment:
THT_EVIDENCE_HOST_REGISTRY_ROOT: ${THT_EVIDENCE_HOST_REGISTRY_ROOT:?set an absolute host registry path}
volumes:
- type: bind
source: ${THT_EVIDENCE_HOST_REGISTRY_ROOT:?set an absolute host registry path}/repo
target: /data/workspace-registry/repo
bind:
create_host_path: false
workspace-maintenance:
volumes:
- type: bind
source: ${THT_EVIDENCE_HOST_REGISTRY_ROOT:?set an absolute host registry path}/repo
target: /data/workspace-registry/repo
bind:
create_host_path: false
+1 -1
View File
@@ -45,7 +45,7 @@ services:
- type: bind
source: ${THT_WORKSPACE_REGISTRY_ROOT:?set THT_WORKSPACE_REGISTRY_ROOT}
target: /data/workspace-registry
read_only: true
read_only: false
- type: bind
source: ${THT_DATA_ROOT:?set THT_DATA_ROOT}/workspace-secrets
target: /data/workspace-secrets
@@ -0,0 +1,4 @@
# The image is tagged from the running frontend before the visual review is published.
services:
frontend:
image: thothii-frontend:before-visual-review-20260912
+8
View File
@@ -0,0 +1,8 @@
# Local-only review override. Apply after the existing local installation profiles.
# Changes only the frontend image; Core, configuration and volumes remain unchanged.
services:
frontend:
image: thothii-frontend:visual-review-20260912
build:
context: /Users/mp/projects/ThothII-visual-review
dockerfile: docker/frontend.Dockerfile
@@ -0,0 +1,5 @@
# Local-only rollback to the full-shell frontend before visual integration.
# Apply after local installation profiles, with --no-build and --no-deps.
services:
frontend:
image: thothii-frontend:before-visual-shell-merge-20260913
+17
View File
@@ -0,0 +1,17 @@
services:
core:
environment:
THT_EVIDENCE_HOST_REGISTRY_ROOT: /Users/mp/projects/ThothII/deploy/psd/evidence-registry
volumes:
- type: bind
source: /Users/mp/projects/ThothII/deploy/psd/evidence-registry/repo
target: /data/workspace-registry/repo
bind:
create_host_path: false
workspace-maintenance:
volumes:
- type: bind
source: /Users/mp/projects/ThothII/deploy/psd/evidence-registry/repo
target: /data/workspace-registry/repo
bind:
create_host_path: false
Submodule deploy/psd/evidence-registry/repo added at 24fb53236b
+13 -3
View File
@@ -2,6 +2,10 @@
# Replace every absolute path before using this as an advanced reference.
schemaVersion: 2
profile: local
# Mac standalone example; the Omics server requires embedded/upstream separately.
shell:
mode: full
defaultLocale: en
projectDirectory: "<abs>/projects/ThothII"
envFile: "<abs>/projects/ThothII/deploy/psd/operator.env"
workspaceRepository:
@@ -10,22 +14,28 @@ workspaceRepository:
access: ssh
modelCatalog:
defaults:
session: zai/glm-5.3
metadataGeneration: zai/glm-5.3
interaction: zai/glm-5.3
embedding:
id: ollama/qwen3-embedding:0.6b
dimensions: 1024
providers:
deepseek:
authentication:
mode: pi_auth
mode: secret_env
apiKeyEnv: DEEPSEEK_API_KEY
session:
mode: pi_builtin
metadataGeneration:
litellmProvider: deepseek
models:
deepseek-v4-pro:
label: DeepSeek V4 Pro
session: {}
metadataGeneration: {}
deepseek-v4-flash:
label: DeepSeek V4 Flash
session: {}
metadataGeneration: {}
zai:
endpoint:
baseUrl: https://api.z.ai/api/coding/paas/v4
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -euo pipefail
node /app/backend/dist/catalog/migrate.js
/opt/venv/bin/python -m tht.memory.migrate
+1 -1
View File
@@ -131,7 +131,7 @@ ENV PATH="/opt/venv/bin:/usr/local/bin:$PATH" \
HOME=/home/thoth
COPY scripts/verify-line-endings.sh /usr/local/bin/verify-line-endings
COPY docker/core-entrypoint.sh docker/workspace-maintenance-entrypoint.sh docker/session-migrate.sh docker/ensure-pi-trust.mjs docker/embedding-model-init.sh /app/docker/
COPY docker/core-entrypoint.sh docker/workspace-maintenance-entrypoint.sh docker/session-migrate.sh docker/catalog-migrate.sh docker/ensure-pi-trust.mjs docker/embedding-model-init.sh /app/docker/
COPY docker/smoke/core-smoke.sh /app/docker/smoke/core-smoke.sh
RUN /usr/local/bin/verify-line-endings /app/docker \
&& chmod +x /app/docker/core-entrypoint.sh /app/docker/workspace-maintenance-entrypoint.sh /app/docker/session-migrate.sh /app/docker/embedding-model-init.sh /app/docker/smoke/core-smoke.sh
+21 -1
View File
@@ -1,6 +1,26 @@
#!/bin/sh
set -eu
test "$(cat /usr/share/nginx/html/config.js)" = 'window.__THOTHII_CONFIG__ = {};'
# The generic image contains an empty fallback; installed containers mount the generated
# public projection over it. An optional path lets host projection tests use this same check.
config_file=${1:-/usr/share/nginx/html/config.js}
test -f "$config_file" && test -r "$config_file"
config=$(tr -d '[:space:]' < "$config_file")
case "$config" in
'window.__THOTHII_CONFIG__={};')
test -z "${THT_FRONTEND_CONFIG_REVISION:-}"
;;
*)
# Installation Load validates BCP47 syntax. Here check only the public payload shape;
# catalog availability and fallback belong to the frontend, not the generic image.
locale='[A-Za-z][A-Za-z0-9]*(-[A-Za-z0-9]+)*'
full="\"mode\":\"full\",\"defaultLocale\":\"$locale\""
embedded="\"mode\":\"embedded\",\"defaultLocale\":\"$locale\",\"adapter\":\"omics-portal\""
if ! printf '%s\n' "$config" | LC_ALL=C grep -Eq "^window\.__THOTHII_CONFIG__=\{\"backendBaseUrl\":\"/api\",\"shell\":\{($full|$embedded)\}\};$"; then
echo "Invalid frontend public runtime configuration" >&2
exit 1
fi
;;
esac
printf '%s\n' "frontend runtime config smoke: ok"
@@ -0,0 +1,59 @@
---
status: accepted
date: 2026-09-08
---
# Use PostgreSQL for Memory and Qdrant for retrieval
The new Memory module uses the installation's existing PostgreSQL service as the
authority for Memory Cards, their links, and structured schema dependencies.
Qdrant holds a rebuildable search projection. This replaces the JSONL registry and
allows related administrative changes to be coordinated in one database transaction.
The owner accepted this direction in Q9 of the Memory design interview; implementation
is pending.
Memory uses its own tables and remains a separate module from the Metadata Catalog.
Sharing the PostgreSQL service does not transfer Memory ownership to Database
management or make Evidence and Memory one canonical domain.
## Retrieval and graph
The owner also accepted hybrid semantic/lexical Qdrant retrieval with scope filters
and explicit card links traversed in core. Both belong to the planned first version;
there is no dedicated graph database or external Memory framework.
This extends [ADR 0017](0017-separate-reference-vectors-from-runtime-memory.md):
the separate reference and memory collections remain, while Memory gains sparse
lexical indexing in addition to dense vectors. Memory projections become rebuildable
from the module's PostgreSQL authority. Preprocessing Clear still preserves Memory;
this decision does not add it to the preprocessing cleanup scope.
Links support discovery. Finding a card through a link does not approve its use.
The core proposes links with the cards for the same final review; Administration
provides manual creation, editing, and deletion. Deleting a card removes its incident
links without deleting the other linked cards.
## Considered options
- Retaining JSONL preserves the current storage format, but leaves coordinated
card/link/dependency mutations and concurrent administrative writes to application code.
- Using Qdrant as the sole authority is a viable alternative for record storage and
retrieval. PostgreSQL is preferred for the coordinated mutations of the new module,
with Qdrant reserved for its search projection.
- Adding a dedicated graph database would introduce another service; the selected
bounded traversal can be implemented in core over persisted links.
## Consequences
The module needs a persistence contract, PostgreSQL schema, and explicit propagation
of additions, updates, and deletions to Qdrant. Choosing PostgreSQL does not make this
propagation atomic across both systems: failures, retries, and invalidation of stale
search content must be handled and tested. An index rebuild uses the original card
content and cannot resurrect deleted cards.
The decision does not introduce Memory revision history or require compatibility with
existing development sessions. Evidence authoring and publication remain governed by
their own contract until the Evidence management project defines its evolution.
The [Memory management project](../plans/2026-09-08-memory-management.md) records the
approved behavior, scope, and integration work.
@@ -0,0 +1,152 @@
---
status: accepted
date: 2026-09-08
---
# Maintain local Evidence with external editors and manual consolidation
Evidence management must provide complete access to registered Evidence and a
maintenance path, as accepted in Q11–Q15 and revised during simplification. The owner subsequently
clarified that a context specialist writes drafts independently of the installation
and without PostgreSQL access; the system refines and stores them locally for use
and maintenance. Implementation is pending.
The storage mechanics of Q11 were revisited in the
[simplification review](../plans/2026-09-08-memory-evidence-simplification-review.md).
The original choice combined the workspace repository as Evidence authority with
application-managed Git writes. The owner accepted the separation of external draft
files/repositories from a durable local canonical file archive and removes
commit/push from normal CRUD. The management surface must expose discoverable,
editable Markdown for domain specialists, without JSONL editing. The owner chose
external editors for release 0: their preferred Mac/PC editor or vim/nano on the
server. No web editor or content form is required. The page keeps browsing,
filtering, and detail, with actual host file paths and maintenance instructions.
The owner also requested manual consolidation followed by manual Git diff, commit,
and push, relying on operator discipline rather than automation. The owner accepted
the remaining simplifications and requested explicit clarification of the core
format change and the simple terminal-based Git check. The original application Git writer
is not the final implementation prescription. A suggestion
to move Evidence authority into PostgreSQL was withdrawn after the clarification.
Memory remains a distinct module with its own PostgreSQL authority under
[ADR 0018](0018-use-postgres-for-memory-and-qdrant-for-retrieval.md).
## Manual consolidation and Git follow-up
The external-editor decision replaces the form's single Save with saving files and
running one consolidation command. The command checks expected structure, required
fields, types, identifiers, references, and provenance. It reports the affected file
and the necessary correction. Invalid input stops activation; valid input updates
derived metadata, the corpus, and the Evidence index. New and deleted files use the
same path. An inaccessible archive must not be interpreted as deleted Evidence.
Structural validation does not prove semantic correctness or rerun model refinement
to rewrite manually edited rules.
The core reads only successfully consolidated active content, never the editable
working files directly. Candidate validation and activation must preserve the last
valid corpus on failure, or report unavailability if its integrity cannot be
guaranteed. Unconsolidated edits must not mix new text with old retrieval data.
Consolidation reports full local success only once the updated content is available
to the core. If persistence succeeds but activation fails, the command reports the
partial outcome and can be rerun without duplicating units or losing edits.
The operation does not imply atomicity across
authoritative storage and Qdrant; their failure and recovery behavior requires
explicit implementation. An interrupted operation must not leave a partial corpus
advertised as ready for subsequent use.
The maintained archive is a persistent Git working tree containing the local
Evidence; it may reuse the workspace repository. Draft sources and curated units
remain distinct even when in the same repository. Setup and the page identify the
repository, working tree, actual host paths, branch, and configured remote. A
rebuildable runtime snapshot is not the editing location.
Before consolidation, the operator checks Git status, including inspection of new
untracked files. Normal terminal Git diff commands are available for line-level
inspection when needed; no custom viewer or mandatory double review is required.
After consolidation, the operator stages the Evidence and required metadata changes,
commits, and pushes using normal Git commands. No watcher, automatic Git writes, retrying push, pull, merge,
or dedicated web execution control is required. Missing credentials, unconfigured
upstreams, and Git conflicts are handled by the operator.
Consolidation activates local changes before commit/push. If the operator omits the
Git steps or push fails, the local update remains effective while transfer to the
remote is incomplete. Pushing does not automatically update other installations.
The system relies on operator discipline to complete the sequence; it does not add
a separate editorial publication state machine or require a second reviewer.
Approved conflict repairs from the core still call the same persistence and
activation service directly. They do not require an external editor or manual
consolidation before the session can use its own approved correction. Their local
file changes are included in the operator's subsequent Git maintenance.
## Administration without concurrent core work
The owner's simplification instruction replaces the earlier Q12 requirement to
refresh open sessions after administrative edits. Core activity can be assumed
absent during administration, or its overlap can be ignored. No dedicated live
update, session notification, restart, maintenance mode, or reader coordination
is required. Completed administrative changes apply to subsequent work.
Deliberate writes from the workflow itself still exist: approved conflict repairs
and the final Memory summary use the same persistence services and handle their
outcome before proceeding. In particular, a session must be able to use its own
approved Evidence correction. This does not require updating all other sessions
or rewriting previously approved decisions, artifacts, or SQL.
## Manual corrections survive source updates
When an updated source contradicts an administrator's correction, the saved manual
Evidence remains active. Evidence management shows the conflicting content for an
explicit decision and subsequent save. The source's newer content does not
automatically override the correction. The owner accepts that the manual rule can
remain in use until that comparison is resolved.
Deleted Evidence must not silently reappear after preparation or reindexing. The
authoring implementation must preserve both manual corrections and suppression of
deleted units through source refreshes. The current protection for uncommitted Git
changes is insufficient: it does not preserve already committed manual corrections
against regeneration, and retirement currently removes the unit without recording
suppression for later preparation.
## Manual creation and explicit source refresh
Administrators can create Evidence without providing an external document. In R0,
they add Markdown using the documented example and consolidate it. The application
records a manual declaration as the managed source of the current statement.
Explicit consolidation approves that declaration; it does not
claim independent documentary verification.
A correction that changes a unit's meaning uses the same explicit manual origin.
The original document remains linked for provenance and source-change detection,
but its excerpt is not presented as support for a rule it does not contain. The
canonical contract must distinguish the current supporting source from the original
document rather than silently retaining outdated support metadata.
External sources are reacquired only when an administrator requests a source
refresh. Ordinary saves and session lookups use the acquired local content; they
do not poll sources or fetch their current versions. A remote change is therefore
detected at the next requested refresh, when the manual-precedence rule applies.
The revised storage recommendation reads the specialist's repository as an input;
ordinary local edits do not write back to it or refresh its content automatically.
## Implementation consequences
The existing [Evidence lifecycle](../evidence.md) and
[Workspace Evidence v3 contract](../contracts/workspace-evidence-v3.md) require
explicit evolution for editable local Markdown, manual consolidation, and manual
precedence. Source provenance and coherent canonical metadata remain requirements;
manual changes must not be presented as statements supported by an unrelated source
excerpt. Under the accepted local-file direction, curated files are primary
installation data preserved by Clear and backed up separately from rebuildable
indexes. Visible Markdown content must become authoritative for editing; operators
must not maintain hidden duplicate text, hashes, or manifest entries themselves.
This is a core Evidence contract change: parser, renderer, authoring, validation,
normalization, and their integration with indexing and recall must be adapted
together. E1 includes a new Curated unit format version, conversion of existing
Evidence, reindexing, and end-to-end verification that visible edits reach core
consumption. Preserve the internal typed model where possible; the change does
not require redesigning the NL-to-SQL workflow phases.
Local edits do not automatically change external drafts or another installation;
Git versioning and transfer occur through the explicit manual follow-up.
The [Evidence management project](../plans/2026-09-08-evidence-management.md)
records the implementation sequence and acceptance checks.
@@ -0,0 +1,27 @@
# Unify administration pages and use namespaced routes
status: accepted
Workspace and Pi management will become full Administration Pages alongside Evidence, Memory and
Database, using one shared Administrative Page Family. The Embedded Thoth Shell will identify the
active Administration Surface through a namespaced browser route, `thoth_route=administration/<surface>`,
so links remain reloadable and support Back/Forward without requiring new host-portal server routes.
Transient form values and secrets stay outside the route; destructive confirmations may remain
short-lived dialogs because they are actions, not page containers.
We considered React-only state, path-based routes and hash routes. React-only state loses refresh and
deep-link behavior, path routes require a host catch-all route that does not yet exist in Omics Portal,
and hash routes can conflict with host-page fragments. A namespaced query route preserves the current
same-document integration boundary while leaving room for a future path adapter.
The accepted visual direction is A / Workbench for all five surfaces; B and C remain recoverable
prototypes. Workspace owns readiness and preprocessing, consuming the Database catalog as a
prerequisite. Database owns connection/binding, schema synchronization, descriptions and sensitivity;
the preprocessing action remains exclusively in Workspace preparation.
On 2026-09-12, the owner refined this direction in
[Gitea #29](https://git.tylconsulting.it/mptyl/ThothII/issues/29): Database opens at its
catalog list, with its original table space and margins, and no Workspace preparation footer.
This supersedes the earlier requirement for a preparation link in that page. Workspace
remains reachable through the shared Administration navigation; its Preparation tab still
links to Database configuration and schema when catalog work is needed.
@@ -0,0 +1,101 @@
# ADR 0021 — Shell separati e adapter sostituibile per il portale
- Stato: accettato
- Data: 2026-09-13
## Decisione
ThothII espone due modalità di installazione, selezionate da `shell.mode`:
- `embedded` (default): ThothII è ospitato da Omics Portal. Non renderizza alcun header e
riceve dal portale lingua, tema e fullscreen. L'accesso resta verificato dal server.
- `full`: ThothII è autonomo. Renderizza il proprio header, con selettore lingua, tema,
fullscreen e nome utente. Il click sul nome apre il logout. Non mostra mai la rotellina o
altri comandi amministrativi del portale. Mantiene un rail vuoto a sinistra di almeno 20 px.
Il fatto che la shell sia `full` è distinto dallo stato `fullscreen`: la prima decide quale
contenitore viene renderizzato, il secondo indica se è attiva la Fullscreen API del browser.
L'icona passa da “entra in fullscreen” a “torna alla modalità normale”; anche `Esc` aggiorna lo
stato visualizzato.
La configurazione installata resta semplice e retrocompatibile. Sul Mac di sviluppo il profilo
locale userà:
```yaml
shell:
mode: full
defaultLocale: en
```
Il deploy sul server userà invece:
```yaml
shell:
mode: embedded
adapter: omics-portal
```
`defaultLocale` indica la lingua iniziale della shell full; in embedded la fonte autorevole resta
il portale.
L'adapter è l'unico confine tra ThothII e il portale. La sua interfaccia pubblica è volutamente
profonda e minima: consegna solo snapshot dello stato, senza esporre comandi, token, identità o
dettagli di trasporto.
```ts
export type HostShellState = {
locale: string; // BCP-47, inizialmente it/en
theme: "light" | "dark";
fullscreen: boolean;
};
export interface PortalAdapter {
subscribe(
onState: (state: HostShellState) => void,
onError: (error: Error) => void,
): () => void;
}
```
`OmicsPortalAdapter` è l'implementazione corrente. Un adapter per un altro portale potrà
sostituirlo senza modificare shell, i18n o workflow. In `full` l'adapter non viene istanziato:
lo stato è gestito internamente dalla shell.
Per l'integrazione oggi operativa, che monta la SPA direttamente nel DOM di Omics Portal,
l'adapter legge la lingua effettiva dal selettore Omics, osserva l'attributo del tema e ascolta
il fullscreen del documento. Selettori e osservatori restano privati dell'implementazione Omics.
La lingua segue il normale ricaricamento Django; tema e fullscreen cambiano nella pagina aperta.
La revisione approvata del 2026-09-13 elimina il precedente handshake a eventi: non servono
messaggi personalizzati, versioni di trasporto o timeout di avvio. Un futuro adapter potrà usare
un diverso trasporto senza modificare l'interfaccia applicativa.
Se `shell` o l'adapter embedded sono omessi, si usa `embedded` con `omics-portal`. Questo default
supporta il documento Omics esistente; nomi adapter sconosciuti o dati host mancanti producono
un errore esplicito, senza attivare la shell full.
## Confini che restano invariati
L'identità e l'autorizzazione del backend non vengono ricostruite nel browser. In embedded,
Omics Portal continua a gestire login e logout e la catena server-side `auth_request` continua a
fornire i principal header già previsti. Lo stato UI non dichiara l'utente autenticato: il modulo
di accesso usa la verifica backend esistente anche alla riconnessione e al ritorno alla pagina.
Un rifiuto su una singola operazione non equivale automaticamente alla perdita dell'accesso.
La lingua UI e la lingua di interazione con il modello restano separate dalla lingua del
workspace; il relativo contratto è in [ADR 0022](0022-separate-ui-locale-from-session-interaction-language.md).
## Alternative scartate
- Duplicare l'header di Omics in embedded: crea due fonti di stato e incompatibilità visive.
- Spargere controlli `if embedded/full` nei componenti: lega ogni pagina al portale.
- Trasmettere utente o token nel bridge: aumenta superficie e accoppia UI e autenticazione.
- Introdurre un protocollo completo request/response: non aggiunge funzionalità richiesta.
- Usare `profile` per distinguere le shell: `profile` descrive la topologia dell'installazione,
non la sua presentazione.
## Conseguenze
La soluzione richiede un adapter nel frontend che osserva il documento condiviso e mantiene la
logica di shell locale a ThothII. Il backend non necessita di un nuovo protocollo di autenticazione
o di una nuova sessione browser. Un nuovo portale deve soddisfare anche il contratto server di
identità fidata: la sola sostituzione della classe UI non sostituisce quel contratto.
@@ -0,0 +1,51 @@
# Separate UI locale from session interaction language
status: accepted
ThothII distinguishes three language concepts:
- `workspace.language` remains the language of workspace-owned documents, descriptions and Evidence;
- `ui_locale` controls deterministic ThothII chrome such as labels, form help, placeholders, errors,
accessibility text and review-widget chrome;
- `interaction_language` is persisted in a session and controls model-generated questions,
explanations and reviewer proposals.
The initial locale catalog supports Italian and English and uses extensible BCP-47 language tags.
Missing deterministic translations fall back to English. The selected UI locale supplies the default
interaction language when a new session is created. A resumed session always uses its persisted
interaction language; changing the host or full-shell UI locale must not silently rewrite an existing
session or make its model output switch language mid-workflow.
The distinction is required because the current workspace contract already uses `language` for
content and the PSD workspace is Italian. Reusing that field for a browser preference would make a
visual choice mutate domain content semantics. The model receives the session interaction language
through the session/Pi workflow context. SQL, identifiers, database values and other technical
artifacts remain governed by their existing contracts and are not translated as UI strings.
In `full`, the local shell owns `ui_locale` and supplies it when starting a new session. In
`embedded`, the host adapter is authoritative for `ui_locale`; ThothII applies host changes to
deterministic UI immediately while preserving the interaction language of any active session.
We considered using only `workspace.language`, using only a global browser locale, and translating
the model output after generation. The first conflates domain content with UI preference; the second
cannot preserve a session's language or follow the host portal; and the third would be unsafe for
structured reviewer decisions and would not control the model's reasoning or proposal language.
## Considered Options
- One mutable `language` field for workspace, UI and session was rejected because the fields have
different owners and lifecycles.
- Client-only translation of reviewer choices was rejected because choices can be generated by the
model and must be requested in the intended language.
- An English-only deterministic chrome was rejected because embedded and full installations must
follow the selected host/user language.
## Consequences
- Session creation and the persisted manifest gain an explicit interaction-language value.
- Legacy manifests without that value use the workspace language, pinned idempotently on first
resume; the browser locale must not determine this compatibility value.
- Resume must read that value from the manifest and must not accept a new locale as an override.
- The workflow prompt contract and deterministic reviewer-widget builders need a locale-aware input.
- Frontend strings need a catalog and stable keys; backend events should expose stable codes where
the frontend is responsible for localization.
+135
View File
@@ -0,0 +1,135 @@
# Rendering full ed embedded
ThothII ha una sola applicazione React, una sola build Vite e gli stessi servizi
backend. «Doppio rendering» significa due modi di ospitare quella applicazione,
non due versioni delle pagine e non rendering React sul server. Django renderizza
il contenitore Omics; React renderizza ThothII nel browser, dentro `#root`.
## Tre decisioni indipendenti
| Decisione | Configurazione | Effetto |
| --- | --- | --- |
| Distribuzione | `profile: local` oppure `server` | Compose, percorsi e vincoli operativi |
| Presentazione | `shell.mode: full` oppure `embedded` | Proprietario di header e preferenze |
| Autenticazione | `auth.yaml` local/OIDC oppure `AUTH_MODE=upstream` | Chi verifica l'identità, come arriva al backend |
Il Mac usa **full + local**, con lingua iniziale inglese. L'integrazione Omics
usa **embedded + upstream**, con accesso già verificato dal portale. Un server
autonomo può usare **full + oidc**. Cambiare `shell.mode` non abilita un metodo
di autenticazione e non modifica permessi o proprietari delle sessioni.
Full con upstream può visualizzare un'identità già verificata dal proxy, ma non
ha un logout ThothII disponibile: non è il profilo autonomo con login/logout.
Embedded non avvia login locale o OIDC anche se il backend è configurato così;
questa combinazione non realizza il login unico Omics e non va usata come fallback.
## Composizione comune
```mermaid
flowchart TD
CONFIG["config.js pubblico"] --> SHELL["ShellProvider"]
FULL["Preferenze full nel browser"] --> SHELL
HOST["Documento Omics"] --> ADAPTER["OmicsPortalAdapter: solo presentazione"]
ADAPTER --> SHELL
SHELL --> GATE["AuthGate: verifica GET /me"]
GATE --> APP["AppShell: stesse pagine, sessioni e amministrazione"]
AUTH["Backend: cookie locale/OIDC o identità upstream"] --> GATE
```
`ShellProvider` risolve la configurazione, applica lingua/tema e monta i contenuti
solo dopo uno snapshot host valido in embedded. `AuthGate` verifica l'accesso;
`AppShell` e le pagine non devono leggere selettori o eventi specifici di Omics.
Il cambio utente smonta lo stato applicativo della precedente identità.
## Full
- Header ThothII rosso Omics `#CB333B` in entrambi i temi; logo interamente chiaro.
- Selettore EN/IT, tema light/dark, fullscreen e nome verificato dell'utente.
- Menu del nome con logout soltanto per local/OIDC; nessuna rotellina admin.
L'amministrazione resta nella navigazione applicativa, secondo i permessi.
- Margine sinistro vuoto e simmetrico al destro: `max(20px, 1.5rem)`, normalmente
24px con radice a 16px. Non è una seconda sidebar di navigazione.
- Lingua e tema ricordati sullo stesso origin in `localStorage`, nelle chiavi
`thothii:shell:locale` e `thothii:shell:theme`. Non sono preferenze server per
utente. In assenza di preferenze: `defaultLocale` e tema light.
- Fullscreen usa `document.documentElement.requestFullscreen()` e
`document.exitFullscreen()`: nasconde il contorno del browser dove supportato.
L'icona cambia sullo stato reale, anche dopo Esc; un rifiuto mostra un errore.
Non è un semplice ingrandimento CSS e non scatta automaticamente all'accesso.
## Embedded
- Nessun header ThothII, selettore lingua, toggle tema, login o logout autonomo.
I controlli rimangono nell'header generale Omics.
- React è nello stesso documento della pagina `/kokoro/datamart-builder/`, non
in un iframe. Non serve `postMessage` né un secondo protocollo di sessione.
- L'adapter legge la lingua Django già confermata, osserva il tema del documento
e ascolta il fullscreen reale. Le azioni rimangono di proprietà del portale.
- Un contesto Omics mancante o invalido mostra un errore d'integrazione; non
passa silenziosamente a full e non offre un secondo login.
- Il portale assegna l'altezza disponibile sotto il proprio header: catena flex
con `min-height: 0`, root contenuto e altezza applicativa vincolata al contenitore.
Il contratto ThothII espone `--thoth-app-height` (fallback `100dvh`); verificare
il contenitore reale, non presumere che l'intera viewport appartenga a React.
Il template Omics mantiene inoltre i suoi override di compatibilità.
Il reset CSS è limitato al mount React e ai popup dell'applicazione, senza
richiedere CSS `@scope`. I token e i popup seguono il tema applicativo. Questo
non rende indipendenti fogli di stile arbitrari caricati dal portale: la verifica
del documento condiviso rimane necessaria a ogni integrazione.
## Caricamento e configurazione pubblica
Il descrittore installato è la sorgente di verità. Il CLI genera
`generated/frontend/config.js` e il suo mount di sola lettura nella proiezione
`generated/compose.models.yaml`. Il file pubblico contiene solo `backendBaseUrl`
e `shell`, mai identità, token, password o percorsi host. Va caricato **prima** del
modulo React e servito senza cache. Nessuna build separata è richiesta per
cambiare modalità; occorre rigenerare e applicare i mount tramite il lifecycle.
L'ordine Omics è: config pubblico → override del solo prefisso API → asset dal
manifest Vite. L'override deve conservare `shell`; l'adapter non configura il proxy.
Il default completo di shell omessa è embedded/en/omics-portal. Il CLI normalizza
anche singoli campi omessi; un oggetto `shell` scritto manualmente nel browser
deve invece contenere `mode` e `defaultLocale`, altrimenti viene rifiutato.
## Lingua, continuità e dati
La lingua UI traduce il testo dell'applicazione, non i contenuti di dominio.
Alla creazione, la lingua UI viene acquisita come `interactionLanguage` di ripiego.
Il CLI riconosce la lingua della domanda originale e salva `interaction_language`
nel manifest; usa il ripiego solo per input troppo brevi, ambigui o composti da codice.
Questa lingua governa domande, spiegazioni, scelte e controlli HITL. Il gate la
include nei descrittori e il frontend la applica al sottoalbero dei widget,
senza cambiare la lingua della navigazione.
Alla ripresa vale la lingua salvata, non l'ultima scelta dell'header. Per i manifest
precedenti senza campo viene riconosciuta e fissata la lingua della domanda,
con la lingua workspace disponibile alla prima ripresa come ripiego.
Il cambio lingua Omics invia il form Django e ricarica la pagina. ThothII conserva
solo l'ID della selezione in `sessionStorage`, separato per pathname, issuer e
subject. Riapre i documenti, non avvia una generazione. Bozze non inviate e modifiche
non salvate richiedono conferma prima della navigazione; non sono una trascrizione
salvata. La ripresa operativa resta esplicita.
## Punti di implementazione e manutenzione
| Sorgente | Responsabilità |
| --- | --- |
| `tools/tht/internal/config/shell.go` | Normalizzazione e validazione del descrittore |
| `tools/tht/internal/modelprojection/projection.go` | Config pubblico e mount generati |
| `frontend/src/api/runtime-config.ts` | Validazione browser e prefisso API same-origin |
| `frontend/src/shell/host/ShellProvider.tsx` | Composizione, preferenze e tema |
| `frontend/src/shell/host/FullHeader.tsx` | Controlli solo full |
| `frontend/src/shell/host/OmicsPortalAdapter.ts` | Conoscenza del documento Omics |
| `frontend/src/auth/AuthGate.tsx` | Accesso e ricontrolli al ritorno alla pagina |
| `backend/src/auth/auth.ts` e `principal.ts` | Verifica server dell'identità |
Per un altro portale servono un'implementazione del
[PortalAdapter](../contracts/portal-shell-adapter-v1.md), la sua registrazione nei
validatori CLI/browser e nel punto di composizione, oltre al
[contratto di autenticazione server](../install/authentication-upstream.md).
Il nome di una classe non è un plugin caricabile dinamicamente da YAML.
Procedure: [configurazione e deploy](../operations/shell-and-localization.md),
[autenticazione](authentication.md), [accettazione](../testing/authentication-manual-acceptance.md).
+50 -6
View File
@@ -1,18 +1,27 @@
# Authentication architecture
ThothII has two production authentication modes: `local` and generic `oidc`. The host operator
surface is one CLI, `tht`; there is no separate authentication executable. The backend owns
opaque browser sessions and authorization, while `tht` owns protected configuration and local-user
files.
ThothII supports `local` and generic `oidc` through protected `auth.yaml`, plus
the trusted-proxy `upstream` path used by Omics. The host operator surface is
one CLI, `tht`; there is no separate authentication executable. The backend owns
authorization in all paths and opaque browser sessions only in local/OIDC.
`tht` owns protected local/OIDC configuration and local-user files; upstream
identity is supplied per request by the authenticated server proxy.
Presentation is separate: [full/embedded rendering](application-shell.md) does
not select authentication. The Mac uses full/local; Omics uses embedded/upstream;
a standalone server can use full/OIDC. Do not configure a second ThothII OIDC
login simply because Omics itself authenticates users through Authentik.
```mermaid
flowchart TB
BROWSER["Browser"] --> BOUNDARY["Authentication boundary"]
BOUNDARY --> LOCAL["Local users\nArgon2id hashes"]
BOUNDARY --> OIDC["OIDC provider\nAuthorization Code PKCE"]
BOUNDARY --> UPSTREAM["Trusted proxy\nVerified portal session"]
OIDC --> GROUPS["Groups claim\nexact mapping"]
LOCAL --> PRINCIPAL["Thoth principal"]
GROUPS --> PRINCIPAL
UPSTREAM --> PRINCIPAL
PRINCIPAL --> ROLES["Roles"]
ROLES --> PERMISSIONS["Permissions"]
PERMISSIONS --> ROUTES["Protected routes"]
@@ -21,6 +30,13 @@ flowchart TB
## Configuration and trust boundaries
The following protected-file configuration applies to local/OIDC. Upstream uses
`AUTH_MODE=upstream` without a mounted `auth.yaml` or authentication runtime
projection. The backend refuses both authorities together. `AUTH_MODE=none`
and `mock` are development/test modes, not production fallbacks. The exact
upstream setup, header contract, proxy hops and origin checks are in the
[server integration guide](../install/authentication-upstream.md).
The installation descriptor points to an operator-controlled authentication directory. It contains
non-secret `auth.yaml` and, for local mode, `users.yaml`. POSIX installations use a private
directory and owner-only regular files; Windows uses equivalent owner-only ACLs. Secret values are
@@ -33,17 +49,26 @@ The production role expansion from `backend/src/auth/config.ts` is exact:
| Role | Permissions |
|---|---|
| `user` | `session.use` |
| `admin` | `session.use`, `session.read_all`, `session.manage_all`, `settings.manage`, `workspace.manage`, `workspace.secrets.manage`, `database.manage`, `pi.manage`, `auth.diagnostics.read` |
| `admin` | `session.use`, `session.read_all`, `session.manage_all`, `settings.manage`, `workspace.manage`, `workspace.secrets.manage`, `database.manage`, `memory.manage`, `evidence.manage`, `pi.manage`, `auth.diagnostics.read` |
`admin` therefore includes the ordinary `session.use` permission. No other role or permission
label is part of the production catalog.
After validating a browser session, the backend expands its roles through the current permission
catalog on every request. The permissions saved at login are a historical snapshot, so existing
administrator sessions can use newly deployed administration features without signing in again.
Session expiry, revocation and local-user role validation still apply before role expansion.
OIDC is provider-neutral at the browser protocol boundary. Authorization Code + PKCE, issuer,
signature, audience, expiry, state, and nonce are validated before a principal is created.
Authentik is the first certified group-catalog adapter, not a special browser login mode.
## Group authorization
This section describes **ThothII's direct OIDC login**, not the embedded Omics
path. Omics checks its own capability and administrator status and supplies
normalized identity headers; ThothII does not repeat the OIDC groups exchange.
OIDC must return a direct, non-empty `groups` claim whose value is a JSON array of strings.
Missing, malformed, indirect, or overage-style claims fail closed. The browser callback returns
HTTP 401 with the generic code `oidc_callback_failed`; it does not expose the internal reason.
@@ -95,6 +120,10 @@ prerequisite fails.
## Browser sessions
This section applies only to **local and direct OIDC**. Upstream reuses the
portal's authenticated session at the proxy boundary, not a ThothII cookie;
its `/me` response has `session: null` and `csrfToken: null`.
The browser receives only an opaque `HttpOnly`, `SameSite=Lax` cookie named `thothii_session`.
State-changing cookie requests require the in-memory CSRF token, same-origin `Origin`, and Fetch
Metadata checks when present. The frontend never stores bearer tokens or session secrets in Web
@@ -111,5 +140,20 @@ affected sessions. Authentication configuration revision changes invalidate all
reload. Logout deletes the server record. Backup restore excludes active sessions and OIDC state,
recreates empty private auth-state directories, and therefore forces reauthentication.
Full local/OIDC logout calls `POST /auth/logout`, revokes the server session and
clears its cookie. It does not call the identity provider's global logout. If the
provider still has an SSO session, the next OIDC login can complete without
another password prompt. Embedded has no ThothII logout control: use the portal.
## Access revalidation
The frontend treats `/me` as the access authority. In embedded it does not fetch
`/auth/config` or offer local/OIDC login. On focus, pageshow, visibility return
and event-stream reconnection it rechecks access. A 401/403 from this probe clears
protected state; a 403 on one operation is not automatically an app-wide logout.
Neither the DOM adapter nor the proxy's initial SSE check guarantees instantaneous
revocation of streams already open in other tabs.
See the [local guide](../install/authentication-local.md), [generic OIDC guide](../install/authentication-oidc.md),
and [Authentik guide](../install/authentik.md) for operator procedures.
[Authentik guide](../install/authentik.md), [upstream integration](../install/authentication-upstream.md),
and [manual acceptance matrix](../testing/authentication-manual-acceptance.md).
+12 -1
View File
@@ -7,7 +7,9 @@ This page complements the [architecture overview](overview.md) with the module s
The frontend communicates with the backend through REST and SSE. The backend does not own session
persistence: it starts Pi, invokes the `tht` CLI, and forwards events. It does own the separate
installation-local database catalog. The harness contains the workflow, the Python CLI, and
adapters for the DWH and vector store.
adapters for the DWH and vector store. Its Memory module also owns the authoritative
PostgreSQL archive of cards, links, dependencies and pending Qdrant projections.
Administrative API calls use the same harness service as workflow producers and recall.
```mermaid
flowchart LR
@@ -20,6 +22,7 @@ flowchart LR
THT --> FS["Sessions and artifacts\nworkspace repository"]
THT --> DWH["DWH\nread-only"]
THT --> VDB["Qdrant / vector store"]
THT --> MEM["thoth_memory\nPostgreSQL Memory archive"]
BE --> CFG["settings.json\nworkspace + thinking"]
BE --> MODELS["generated runtime catalog\nfrom installation YAML"]
BE --> CAT["catalog-db\nPostgreSQL + Kysely"]
@@ -41,6 +44,14 @@ Dipendenze principali:
## Session sequence
The shared frontend is wrapped by `ShellProvider` (full preferences or a
replaceable portal presentation adapter), then `AuthGate` (backend identity),
then `AppShell`. Omics-specific DOM details belong only to `OmicsPortalAdapter`;
credentials and principal validation belong to the server, never that adapter.
Full/embedded do not duplicate the session workflow below. See
[rendering architecture](application-shell.md) and
[upstream identity](../install/authentication-upstream.md) for both boundaries.
The main path starts with a user question and ends with an SSE event. Reviewer decisions use the same channel and are persisted by the harness.
```mermaid
+27 -15
View File
@@ -4,8 +4,11 @@
ThothII is a **human-in-the-loop datamart builder**. It turns a natural-language question into validated SQL, and optionally a dbt datamart, through a **deterministic eight-phase NL-to-SQL workflow** in which the model *proposes* and a human reviewer *decides* at gates.
Production authentication uses local authentication or generic OIDC. `tht` is the only operator CLI.
For sessions, roles, groups, diagnostics, and recovery, see the [authentication documentation](authentication.md).
Production authentication uses local authentication, generic OIDC, or the
trusted-proxy upstream path used by Omics. `tht` is the operator CLI for the
installation and local/OIDC configuration. For roles and recovery, see
[authentication](authentication.md). One React build supports full and embedded;
[rendering architecture](application-shell.md) separates presentation from identity.
```mermaid
flowchart LR
@@ -78,20 +81,20 @@ The model proposes; a human reviewer decides at gates through widgets:
The frontend renders these widget descriptors (registry in `src/widgets/`). It rebuilds the live transcript in memory from the SSE stream (`src/store/sessionStore.ts`); it is **not persisted**.
## Curated and immutable Evidence
## Evidence sources, local authority and search projections
The workspace repository is the publication boundary. The curator prepares `evidence/source/`,
reviews units in `evidence/curated/`, validates them, and merges them. With `evidence.schema_version: 2`,
the runtime materializes the full `evidence/` tree from the exact Git commit, but the renderer passes
only `curated/**/*.md` from the immutable revision root to preprocessing. Sources, manifests, and
evaluation data remain available for traceability. The runtime never modifies, stages, commits, or
publishes the authoring repository.
The workspace repository publishes revision-pinned workspace identities and Evidence sources.
An initialized editable Evidence archive is maintained locally through external editors and
explicit consolidation; normal preprocessing or source refresh must not overwrite its manual
corrections. File save, active search generation and a later human Git commit/push are separate
outcomes. See [the current Evidence contract](../contracts/curated-evidence-v4.md) and
[source/publication boundaries](../contracts/workspace-evidence-v3.md).
Before indexing, the curated corpus from the pinned revision is validated. Each workspace has two
physical Qdrant collections with different lifecycles. `reference` contains Schema, relationships,
and Evidence and may be replaced or cleared by preprocessing; `memory` contains `memory` and
`solved_question` records and is never preprocessing output. Only the `reference` collection has the
sparse `bm25` vector with `idf`, and only the Evidence stage writes sparse values.
Each workspace has separate Qdrant collections. `reference` contains Schema, relationships and
Evidence and may be replaced or cleared by preprocessing. Memory uses its own dense/BM25
projection, rebuilt from authoritative PostgreSQL cards rather than old vector payloads or
session artifacts. Both collections can contain sparse vectors; their ownership and cleanup
lifecycles remain separate. See [Memory](../gestione-memory.md).
The Administration control can clear the replaceable reference collection, LSH, corpus, and derived
checkpoints. The operation preserves the memory collection and makes preprocessing required before
@@ -101,7 +104,9 @@ the core can admit a new session.
- `tht -c`/`--config` is a **per-command** option. It must follow the subcommand, never precede it (`ThtRunner.buildArgv` enforces this).
- `--json` output must be plain JSON on stdout. It is a machine-readable contract.
- UI strings are in English. Document *content* stays in the workspace language because it is the actual data; only chrome and labels are in English.
- UI strings support English and Italian, with English fallback. Session interaction language
is pinned at creation; document content remains in the workspace language. See
[shell and localization](../operations/shell-and-localization.md).
- Each workspace defines identity and optional Evidence only. The PostgreSQL Metadata Catalog defines
its DWH target and binding; secrets remain in the protected workspace secret store.
- Settings are global (`backend/data/settings.json`: workspace/thinking); provider/model choices are
@@ -110,6 +115,13 @@ the core can admit a new session.
## Runtime composition
PostgreSQL is the current database-metadata authority for all core consumers, not a deferred
catalog-to-core integration. The historical research on a separate catalog service and
`annotations.yaml` publication is superseded by ADRs 0004 and 0016. Preserve read-only DWH access,
installation-local bindings/secrets, revision-pinned workspace identity, and fail-closed readiness
when changing those boundaries. The exact snapshot interface is the
[Catalog Schema Snapshot contract](../contracts/catalog-schema-snapshot.md).
The local stack is started by `./scripts/run-stack.sh` after the installation descriptor and
`deploy/env/local.env` exist. `core` includes Pi. `catalog-db`, Qdrant, and the Ollama embedding
service are internal Compose services; only the DWH and model-provider endpoint remain external.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 132 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 189 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 186 KiB

@@ -1,32 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Archify automated browser evidence · thothii-core-sequence.html</title>
<style>
*{box-sizing:border-box}body{margin:0;padding:24px;background:#e9eef5;color:#172033;font:14px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}header{max-width:1500px;margin:0 auto 18px}h1{margin:0 0 6px;font-size:20px}p{margin:0;color:#526176}.grid{max-width:1500px;margin:auto;display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:18px}figure{margin:0;padding:10px;background:white;border:1px solid #c9d4e3;border-radius:12px;box-shadow:0 10px 30px rgba(15,23,42,.08)}img{display:block;width:100%;height:auto;border:1px solid #e2e8f0}figcaption{padding:9px 4px 2px;color:#526176}@media(max-width:900px){.grid{grid-template-columns:1fr}}
</style>
</head>
<body>
<header><h1>Automated browser evidence</h1><p>thothii-core-sequence.html · visual-check containment pass · perceptual visual review pending</p></header>
<main class="grid">
<figure>
<img src="thothii-core-sequence.visual-check.1440x900.light.png" alt="light 1440 by 900">
<figcaption><strong>LIGHT</strong> · 1440×900 · containment pass</figcaption>
</figure>
<figure>
<img src="thothii-core-sequence.visual-check.1440x900.dark.png" alt="dark 1440 by 900">
<figcaption><strong>DARK</strong> · 1440×900 · containment pass</figcaption>
</figure>
<figure>
<img src="thothii-core-sequence.visual-check.2048x1320.light.png" alt="light 2048 by 1320">
<figcaption><strong>LIGHT</strong> · 2048×1320 · containment pass</figcaption>
</figure>
<figure>
<img src="thothii-core-sequence.visual-check.2048x1320.dark.png" alt="dark 2048 by 1320">
<figcaption><strong>DARK</strong> · 2048×1320 · containment pass</figcaption>
</figure>
</main>
</body>
</html>
@@ -1,548 +0,0 @@
{
"schemaVersion": 1,
"ok": true,
"command": "visual-check",
"evidenceKind": "automated-browser",
"status": "pass",
"visualReview": "pending",
"artifact": {
"path": "/Users/mp/projects/ThothII/docs/architecture/thothii-core-sequence.html",
"sha256": "b521eb942f7889cfc3a5e29010546ba59eeab1cf485c22c533d271ccef9a28d5",
"bytes": 716881
},
"state": {
"detail": "read",
"motion": "still"
},
"chrome": {
"status": "available",
"executable": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
},
"diagnostics": [],
"containment": {
"status": "pass",
"viewports": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1194,
"diagramWidth": 1164,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1600,
"height": 1000,
"theme": "light",
"innerWidth": 1600,
"innerHeight": 1000,
"scrollWidth": 1600,
"scrollHeight": 1000,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1242,
"diagramWidth": 1212,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1920,
"height": 1080,
"theme": "light",
"innerWidth": 1920,
"innerHeight": 1080,
"scrollWidth": 1920,
"scrollHeight": 1080,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1402,
"diagramWidth": 1372,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1818,
"diagramWidth": 1768,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
}
]
},
"readability": {
"status": "pass",
"minimumProjectedNodeTextPx": 6,
"viewports": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1194,
"diagramWidth": 1164,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1600,
"height": 1000,
"theme": "light",
"innerWidth": 1600,
"innerHeight": 1000,
"scrollWidth": 1600,
"scrollHeight": 1000,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1242,
"diagramWidth": 1212,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1920,
"height": 1080,
"theme": "light",
"innerWidth": 1920,
"innerHeight": 1080,
"scrollWidth": 1920,
"scrollHeight": 1080,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1402,
"diagramWidth": 1372,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1818,
"diagramWidth": 1768,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
}
]
},
"viewerChrome": {
"status": "pass",
"viewports": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1194,
"diagramWidth": 1164,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1600,
"height": 1000,
"theme": "light",
"innerWidth": 1600,
"innerHeight": 1000,
"scrollWidth": 1600,
"scrollHeight": 1000,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1242,
"diagramWidth": 1212,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1920,
"height": 1080,
"theme": "light",
"innerWidth": 1920,
"innerHeight": 1080,
"scrollWidth": 1920,
"scrollHeight": 1080,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1402,
"diagramWidth": 1372,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1818,
"diagramWidth": 1768,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
}
]
},
"captures": {
"status": "pass",
"screenshots": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1194,
"diagramWidth": 1164,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light",
"file": "thothii-core-sequence.visual-check.1440x900.light.png"
},
{
"width": 1440,
"height": 900,
"theme": "dark",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1194,
"diagramWidth": 1164,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "dark",
"file": "thothii-core-sequence.visual-check.1440x900.dark.png"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1818,
"diagramWidth": 1768,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light",
"file": "thothii-core-sequence.visual-check.2048x1320.light.png"
},
{
"width": 2048,
"height": 1320,
"theme": "dark",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1818,
"diagramWidth": 1768,
"viewBoxWidth": 1080,
"minimumProjectedNodeTextPx": 7,
"minimumProjectedNodeText": "browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "dark",
"file": "thothii-core-sequence.visual-check.2048x1320.dark.png"
}
],
"contactSheet": "thothii-core-sequence.visual-check.html"
},
"sidecars": {
"receipt": "thothii-core-sequence.visual-check.json",
"contactSheet": "thothii-core-sequence.visual-check.html"
}
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 137 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 183 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 182 KiB

@@ -1,32 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Archify automated browser evidence · thothii-runtime.html</title>
<style>
*{box-sizing:border-box}body{margin:0;padding:24px;background:#e9eef5;color:#172033;font:14px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}header{max-width:1500px;margin:0 auto 18px}h1{margin:0 0 6px;font-size:20px}p{margin:0;color:#526176}.grid{max-width:1500px;margin:auto;display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:18px}figure{margin:0;padding:10px;background:white;border:1px solid #c9d4e3;border-radius:12px;box-shadow:0 10px 30px rgba(15,23,42,.08)}img{display:block;width:100%;height:auto;border:1px solid #e2e8f0}figcaption{padding:9px 4px 2px;color:#526176}@media(max-width:900px){.grid{grid-template-columns:1fr}}
</style>
</head>
<body>
<header><h1>Automated browser evidence</h1><p>thothii-runtime.html · visual-check containment pass · perceptual visual review pending</p></header>
<main class="grid">
<figure>
<img src="thothii-runtime.visual-check.1440x900.light.png" alt="light 1440 by 900">
<figcaption><strong>LIGHT</strong> · 1440×900 · containment pass</figcaption>
</figure>
<figure>
<img src="thothii-runtime.visual-check.1440x900.dark.png" alt="dark 1440 by 900">
<figcaption><strong>DARK</strong> · 1440×900 · containment pass</figcaption>
</figure>
<figure>
<img src="thothii-runtime.visual-check.2048x1320.light.png" alt="light 2048 by 1320">
<figcaption><strong>LIGHT</strong> · 2048×1320 · containment pass</figcaption>
</figure>
<figure>
<img src="thothii-runtime.visual-check.2048x1320.dark.png" alt="dark 2048 by 1320">
<figcaption><strong>DARK</strong> · 2048×1320 · containment pass</figcaption>
</figure>
</main>
</body>
</html>
@@ -1,548 +0,0 @@
{
"schemaVersion": 1,
"ok": true,
"command": "visual-check",
"evidenceKind": "automated-browser",
"status": "pass",
"visualReview": "pending",
"artifact": {
"path": "/Users/mp/projects/ThothII/docs/architecture/thothii-runtime.html",
"sha256": "ad19195852c7c643228663e5bf7f3cb273afb0456fedbe295d4df24bc345b6c7",
"bytes": 724277
},
"state": {
"detail": "read",
"motion": "still"
},
"chrome": {
"status": "available",
"executable": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
},
"diagnostics": [],
"containment": {
"status": "pass",
"viewports": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 968,
"diagramWidth": 938,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.20735294117647,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1600,
"height": 1000,
"theme": "light",
"innerWidth": 1600,
"innerHeight": 1000,
"scrollWidth": 1600,
"scrollHeight": 1000,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1009,
"diagramWidth": 979,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.478676470588235,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1920,
"height": 1080,
"theme": "light",
"innerWidth": 1920,
"innerHeight": 1080,
"scrollWidth": 1920,
"scrollHeight": 1080,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1172,
"diagramWidth": 1142,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 7.557352941176471,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1583,
"diagramWidth": 1533,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 9,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
}
]
},
"readability": {
"status": "pass",
"minimumProjectedNodeTextPx": 6,
"viewports": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 968,
"diagramWidth": 938,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.20735294117647,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1600,
"height": 1000,
"theme": "light",
"innerWidth": 1600,
"innerHeight": 1000,
"scrollWidth": 1600,
"scrollHeight": 1000,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1009,
"diagramWidth": 979,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.478676470588235,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1920,
"height": 1080,
"theme": "light",
"innerWidth": 1920,
"innerHeight": 1080,
"scrollWidth": 1920,
"scrollHeight": 1080,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1172,
"diagramWidth": 1142,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 7.557352941176471,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1583,
"diagramWidth": 1533,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 9,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
}
]
},
"viewerChrome": {
"status": "pass",
"viewports": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 968,
"diagramWidth": 938,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.20735294117647,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1600,
"height": 1000,
"theme": "light",
"innerWidth": 1600,
"innerHeight": 1000,
"scrollWidth": 1600,
"scrollHeight": 1000,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1009,
"diagramWidth": 979,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.478676470588235,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 1920,
"height": 1080,
"theme": "light",
"innerWidth": 1920,
"innerHeight": 1080,
"scrollWidth": 1920,
"scrollHeight": 1080,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1172,
"diagramWidth": 1142,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 7.557352941176471,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1583,
"diagramWidth": 1533,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 9,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light"
}
]
},
"captures": {
"status": "pass",
"screenshots": [
{
"width": 1440,
"height": 900,
"theme": "light",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 968,
"diagramWidth": 938,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.20735294117647,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light",
"file": "thothii-runtime.visual-check.1440x900.light.png"
},
{
"width": 1440,
"height": 900,
"theme": "dark",
"innerWidth": 1440,
"innerHeight": 900,
"scrollWidth": 1440,
"scrollHeight": 900,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 968,
"diagramWidth": 938,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 6.20735294117647,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 51,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "dark",
"file": "thothii-runtime.visual-check.1440x900.dark.png"
},
{
"width": 2048,
"height": 1320,
"theme": "light",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1583,
"diagramWidth": 1533,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 9,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "light",
"file": "thothii-runtime.visual-check.2048x1320.light.png"
},
{
"width": 2048,
"height": 1320,
"theme": "dark",
"innerWidth": 2048,
"innerHeight": 1320,
"scrollWidth": 2048,
"scrollHeight": 1320,
"overflowX": false,
"overflowY": false,
"ok": true,
"readerWidth": 1583,
"diagramWidth": 1533,
"viewBoxWidth": 1360,
"minimumProjectedNodeTextPx": 9,
"minimumProjectedNodeText": "Browser",
"minimumProjectedNodeTextDetail": "context",
"minimumRequiredNodeTextPx": 6,
"readabilityOk": true,
"hasLegend": true,
"hasNavigationDock": true,
"legendDockIntersectionArea": 0,
"dockStageIntersectionArea": 0,
"dockStageGap": 10.21875,
"requiredDockStageGap": 10,
"viewerChromeStageOk": true,
"viewerChromeReserve": 41,
"viewerChromeActive": true,
"viewerChromeOk": true,
"resolvedTheme": "dark",
"file": "thothii-runtime.visual-check.2048x1320.dark.png"
}
],
"contactSheet": "thothii-runtime.visual-check.html"
},
"sidecars": {
"receipt": "thothii-runtime.visual-check.json",
"contactSheet": "thothii-runtime.visual-check.html"
}
}
+65
View File
@@ -0,0 +1,65 @@
# Session corrections to Memory and Evidence
`reviewer_archive_repair` presents one to five closed alternatives for a conflict.
Each alternative updates one existing Memory Card or one existing local Evidence unit,
showing complete current and resulting content. The reviewer chooses one alternative,
rejects all as inadequate, or asks for reformulation. A correction does not approve or
advance the session phase; the ordinary gate still reviews its use in the current question.
The harness coordinator `tht/archive_repair.py` joins two independent domains. It uses
Memory's PostgreSQL repository, workspace mutation lock and projection service, and the
same local Evidence save/consolidation boundary as administration. Evidence activation
runs the existing corpus pipeline without reacquiring configured external sources.
No database binding, schema configuration, other archive, or Git repository is changed.
## Authority and recovery
Migration `004_archive_repairs.sql` stores session-bound proposals and receipts in
`thoth_memory.archive_repairs`, isolated by workspace RLS. Each receipt captures the
session decision context, current target revision, complete resulting content, selected
choice, acting principal and publication outcome. The preparation command changes no
archive content. Application accepts only an option ID from the persisted proposal.
Memory writes its new revision and receipt in one SQL transaction. Projection failures
leave a durable pending operation; retry propagates the saved revision. A later card
edit invalidates replay of the correction.
Evidence records the exact choice before writing its canonical file. Recovery accepts
either the reviewed original revision or the already-written approved result; it never
overwrites a different intervening correction. Before a first proposal, the editable
checkout must match its active snapshot. Other curated files are fingerprinted and
checked again before application/retry so a session decision cannot publish unrelated
external edits. A crash after replacement is recoverable by the same receipt. Original
document provenance is retained as the lineage of the manual correction.
The gate displays these outcomes:
| Status | Meaning |
| --- | --- |
| `proposed` | Waiting for a human choice; no content saved |
| `rejected` | All alternatives declined; reformulation required |
| `applying` | Choice recorded; file write or candidate recovery still required |
| `pending_activation` | Content saved; index activation incomplete |
| `active` | Saved correction matches the currently active revision |
| `superseded` | The target was removed or changed after the saved correction |
The reviewer may retry a pending correction or continue the current question while
leaving activation explicitly pending. The latter is not persistent-resolution success.
`repair-show` refreshes target status; `repairs` lists the historical receipt status.
The final Memory summary must not create a duplicate of a correction already saved here.
## Authorization and commands
Inspection/preparation requires an accessible open session in the same workspace.
Applying either archive correction requires an administrator. Browser responses are
also checked against the responding principal's `memory.manage` or `evidence.manage`
permission and the Pi runtime owner. An administrator using another principal's runtime
must resume it under their own account first, preserving truthful receipt attribution.
Non-administrators may reject proposals or continue question review without changing
the archives. The Pi shell guard blocks `repair-apply`; only the human gate invokes it.
The Python workflow CLI provides `memory repair-target`, `repair-prepare`, `repair-show`,
`repair-apply`, and `repairs`. These are workflow integration commands, not new native
installation commands. All accept `--session` and command-local `-c`; JSON output remains
machine-readable. Proposal bodies are bounded to 1 MB. Durable receipts support both
filesystem and server session storage without a persisted chat transcript.
+248
View File
@@ -0,0 +1,248 @@
# Editable Curated Evidence v4
Curated unit v4 makes the visible Markdown body authoritative. It uses the existing
typed Evidence payloads and stable identifiers. Workspace descriptor v4 and Evidence
descriptor v1/v2 are separate version numbers.
E1 implements the format, explicit conversion, local archive and consolidation API.
E2 connects that API to the installed consolidation command, Evidence management
page and runtime source selection. The local PSD preview now uses 35 converted units.
## Write or edit a file
Place units in `<workspace-root>/evidence/curated/<kind>/<name>.md`. Keep the existing
`id` when editing. The directory must match `kind`. A new manual unit needs no external
document, hash or encoded metadata:
```markdown
---
schema_version: 4
id: evidence:order-key
kind: domain
language: en
purposes: [sql_generation]
applies_to:
tables: [sales.orders]
---
# Order key
## Rule
Join orders using the order number, financial year and company.
```
Required metadata is `schema_version`, `id`, `kind`, `language`, and a nonempty
`purposes` list. Purposes are `disambiguation`, `rewriting`, `schema_linking`, and
`sql_generation`. Optional `applies_to` contains `concepts`, `tables`, and `columns`.
Tables use `schema.table`; columns use `schema.table.column`.
The first H1 is the title. H2 headings identify the payload fields below. Heading
spelling follows `language`: Italian for `it` and its regional variants, English
otherwise. Keep structural headings when editing the text below them.
| Kind | English field headings | Italian field headings |
| --- | --- | --- |
| `domain` | Rule | Regola |
| `glossary` | Definition, Synonyms, Variants | Definizione, Sinonimi, Varianti |
| `enum` | Column, Values | Colonna, Valori |
| `example` | Question, Interpretation | Domanda, Interpretazione |
| `mapping` | Concept, Tables, Columns | Concetto, Tabelle, Colonne |
| `normalization` | Input, Output, Rule | Input, Output, Regola |
| `formula` | Concept, Columns, SQL | Concetto, Colonne, SQL |
| `reference` | URL, Label, Description | URL, Etichetta, Descrizione |
List fields use one `- value` per line. Empty optional lists may be omitted. Quoted
JSON strings within bullets preserve unusual or multiline values during conversion.
Enum values use `### "stored value"`, followed by their meaning; `### ""` represents
an empty stored value. Formula SQL uses a fenced `sql` block containing one PostgreSQL
expression. Whole queries and mutation statements remain invalid.
Nested prose headings and fenced examples are supported inside text fields. An H2
matching a field heading is structural outside a code fence. Duplicate fields, missing
required fields, duplicate metadata keys and malformed payloads are rejected. There
is no second title or payload in frontmatter and no hidden authoritative rule text.
Conversion fails explicitly if a legacy payload cannot be represented losslessly.
## Current provenance and original source
The host supplies document provenance when refining source material: relative source
path, normalized source hash and exact supporting excerpts. A manual unit can omit
`provenance`. Consolidation records `kind: manual` and the supplied curator identity.
A visible change to a previously recorded unit becomes a manual declaration. If that
unit originated from a document, its former document provenance is retained under
`original`. The original excerpts establish lineage; they do not assert that the
source contains the new wording. Retrieval carries this distinction through typed
metadata. Curators edit content; the service updates managed provenance.
Unchanged document declarations still require matching source bytes and excerpts.
Replacing a source requires an explicit refresh through the E3 source-review path. Ordinary preparation
is blocked on an initialized local archive, preventing regenerated source material
from overwriting corrections or restoring deletions. Legacy `resolve` is likewise
blocked there; local file corrections and the archive API own those changes.
## Persistent archive and activation
```text
<workspace-root>/
.evidence-archive.lock
evidence/
source/ # acquired original documents
curated/<kind>/*.md # editable primary content
local-manifest.yaml # derived declarations and deletion records
.local/
state.yaml # baseline, pending and active revisions
snapshots/<revision>/ # immutable units, source bytes and manifest
```
Keep the complete Evidence tree and its managed metadata in backups and the operator's
Git review. Historical source bytes and deletion records are needed to preserve manual
care across subsequent imports. The separate corpus cache and Qdrant index are derived.
The lock file only coordinates local service operations.
`LocalEvidenceArchive.initialize()` records the pre-edit baseline without activation.
`consolidate(actor=..., activate=...)` validates files, derives provenance, records
deletions and source suppression, and creates an immutable candidate. The activation
callback receives that snapshot and must raise if indexing is blocked or fails. Only
successful activation advances `active_snapshot()`. Without a callback the result is
explicitly `pending_activation`; saving a file alone never changes this pointer.
The same candidate can be retried after an index failure. Interrupted managed writes
are replayed only when the operator's file bytes have not changed. A missing curated
directory is an availability error, not proof that all units were deleted. Removing
unit files from an accessible directory records deletion without deleting their sources.
The API also provides `get`, revision-checked `save`, and revision-checked `remove` for
future deliberate workflow corrections. Concurrent external edits produce conflicts.
These boundaries are exercised with the existing corpus pipeline and real Qdrant.
An initialized installation reads only its active local snapshot, including during
ordinary preprocessing. Unconsolidated edits are visible in Administration but do
not enter retrieval. Before initialization, the pinned repository source still works.
## Administration and installed command
Open **Administration → Evidence management**. This independent page requires the
`evidence.manage` permission and no active session. It shows complete units, source
lineage, review items, file errors, filters, and changes relative to the active snapshot.
Edit the displayed Markdown path using an external editor. Refresh files to inspect
the result, then run the command shown by the page:
```sh
tht --installation /absolute/path/thothii-installation.yaml \
workspace evidence consolidate --workspace psd-clinical
```
The installed command accepts `--json`. It validates and activates Evidence using
the existing corpus pipeline and embedding service. It does not scan the DWH, run
full workspace preprocessing, change Catalog/Schema readiness, or execute Git.
If indexing fails after saving, the archive retains the candidate for retry and the
previous active revision remains selected. Validation errors identify corrections
to make in the files. Structural and review checks still apply; initialized archives
do not require the legacy repository's fixed retrieval-evaluation fixture, whose
expected IDs would otherwise prevent deliberate local deletions.
The canonical workspace root is `<workspace-registry-root>/repo/<workspace-id>`.
Snapshots and the derived corpus are separate. For a registry in a Docker volume,
copy its existing `repo` to a persistent host directory before enabling
`deploy/compose.evidence-host.yaml`; set `THT_EVIDENCE_HOST_REGISTRY_ROOT` to that
directory and include the override in the installation descriptor. Core and
workspace-maintenance must mount the same checkout. Keep registry state/snapshots
on their existing volume. The page only presents a host path when configured; it
does not label an internal container path as a usable editor path.
After successful consolidation, inspect and commit the complete workspace Evidence
tree, including managed manifests, snapshots, and deletion records, then push manually.
The page provides quoted POSIX-shell examples for status, diff, add, commit, and push.
Do not commit only the edited Markdown. Ordinary Git operations remain the operator's
responsibility. E3 source decisions use this same activation boundary.
**Clear** removes derived Reference/corpus data while retaining editable files,
snapshots and Memory. It still requires full workspace preprocessing to recreate
Reference/Schema readiness; Evidence consolidation does not satisfy that gate.
## Import drafts and refresh sources
Place externally authored Markdown drafts in `<workspace-root>/evidence/incoming/`.
The specialist needs no installation account or database access to write a draft.
Copying the draft to the installation and choosing **Import or refresh sources** in
Evidence management explicitly starts acquisition and refinement. The equivalent
installed command is:
```sh
tht --installation /absolute/path/thothii-installation.yaml \
workspace evidence refresh --workspace psd-clinical
```
This reads local `incoming/**/*.md` and original `source/**/*.md` files, excluding
managed `source/acquired/` versions. It also reads configured HTTP and S3 sources
through the existing read-only adapters and network policies. After local archive
initialization, filesystem descriptors use this local authoring tree; ordinary
runtime/preprocessing never fetches remote source changes. No source-server write
credential is needed. The existing Pi authoring refiner runs once for each changed
document, without session state or tools. Unchanged source hashes skip refinement.
All acquisitions and proposals must succeed before the new comparison set is
recorded. An access or refinement failure preserves previous comparisons and active
Evidence. A source absent from a successful discovery is marked missing and never
treated as permission to delete units. Restore an accidentally missing local original
file before consolidating its document-derived units, or explicitly retire those units.
Each changed source has a durable comparison showing current local units, complete
proposed content, supporting excerpts, and IDs that replacement would retire:
- **Keep local Evidence** retains the current wording as a manual declaration, with
its original documentary lineage preserved. The changed source is acknowledged;
the next unchanged refresh does not reopen that decision.
- **Use proposed Evidence** adopts the displayed proposal and explicitly retires the
displayed omitted IDs. The source version and provenance change together.
Both choices save and activate through the same local consolidation/index pipeline.
Review items block adoption; correct the original draft and refresh, or keep local
content. There is no automatic merge based on a model's semantic conflict assessment.
Any change to an affected curated file invalidates the comparison and requires a new
refresh. If indexing fails after the decision is saved, use **Retry saved decision**;
this reuses acquired content without fetching sources again. An intervening external
edit is never silently overwritten by recovery.
Headless operators can make the same decision using the source ID and comparison
revision from the local source registry or administration response:
```sh
tht --installation /absolute/path/thothii-installation.yaml \
workspace evidence decide --workspace psd-clinical \
--source-id <64-hex-source-id> --revision <64-hex-comparison-revision> \
--decision keep
```
Use `--decision replace` to adopt the proposal. All installed commands accept `--json`.
The Python `evidence sources` worker is internal to this installed command/API surface.
`evidence/.local/sources.json` stores source identities, comparisons and retry journals.
`evidence/.local/acquisitions/` preserves original acquired bytes and credential-free
remote provenance. Adopted normalized documents live under
`evidence/source/acquired/<source-id>/<content-hash>.md`. Versioned paths let a new
document and an older manual declaration's original source coexist. Include all of
these files in the existing manual Git/backup sequence. Do not edit managed acquired
versions: edit the original local draft or refresh its remote origin.
Deleted IDs remain reserved. Once a source has had a curated deletion, fresh model
IDs from that source are conservatively suppressed as well: changing an ID must not
restore retired knowledge. Existing surviving IDs can still receive reviewed updates;
deliberate new knowledge can be written as a manual Evidence file. Refresh is bounded
to 200 documents and 100 MiB per operation, in addition to each adapter's limits.
## Convert an existing workspace
The existing workflow CLI command `tht evidence migrate <workspace-root>` converts
unit versions 1–3 to 4 deterministically and initializes the archive baseline. It
preserves IDs, typed content, provenance and review items, with no model call or Git
commit. It does not activate a local index. Review items still block consolidation.
The command's existing Git-worktree path check remains in effect.
The first installed consolidation performs this conversion automatically when the
legacy manifest is present, then validates and activates the result. Preserve the
existing checkout in backups before upgrading. The E1 validation used an isolated
copy; E2 also converted and indexed all 35 units on the running local preview.
See the [E1 validation report](../reports/knowledge-archives-release.md) and
[E2 validation report](../reports/knowledge-archives-release.md).
+135
View File
@@ -0,0 +1,135 @@
# Portal Shell Adapter v1
Contratto minimo della presentazione embedded. La revisione approvata il 2026-09-13
sostituisce il precedente trasporto a eventi personalizzati con l'osservazione del
documento condiviso. Non trasferisce identità, token o stato di autenticazione.
## Configurazione
Sul Mac:
```yaml
shell:
mode: full
defaultLocale: en
```
Sul server Omics:
```yaml
shell:
mode: embedded
adapter: omics-portal
```
Se `shell` o `mode` sono omessi, la modalità è embedded. L'adapter embedded
predefinito è `omics-portal`; un nome sconosciuto è un errore di configurazione.
Questi default sono normalizzati dal CLI prima della proiezione. Nel browser,
shell interamente omessa ha gli stessi default, ma un oggetto `shell` parziale
senza `mode` o `defaultLocale` viene rifiutato: non scrivere proiezioni a mano.
Full non istanzia adapter. `defaultLocale` inizializza full; in embedded il locale
proviene dal portale. L'autenticazione si configura separatamente dalla shell.
## API applicativa
```ts
export type PortalSnapshot = {
locale: string;
theme: "light" | "dark";
fullscreen: boolean;
};
export interface PortalAdapter {
subscribe(
onState: (state: PortalSnapshot) => void,
onError: (error: Error) => void,
): () => void;
}
```
Una sottoscrizione installa gli osservatori e consegna lo snapshot iniziale senza
richiedere messaggi all'altro applicativo. Gli aggiornamenti contengono snapshot
completi e validati. La disiscrizione elimina tutti i listener e osservatori.
La lingua viene risolta tramite i cataloghi UI, con fallback inglese.
## Implementazione Omics
L'integrazione monta React nel documento Django, non in un iframe.
| Dato | Fonte privata dell'adapter | Aggiornamento |
| --- | --- | --- |
| Locale | `data-lang` del selettore `.omics-language-select` | nuova pagina Django dopo `set_language` |
| Tema | `data-bs-theme` su `html` | osservazione limitata a quell'attributo |
| Fullscreen | stato effettivo del documento | evento del browser, inclusa uscita con Esc |
Il template Omics aggiornato allinea anche `html lang` alla lingua Django, ma
la fonte dell'adapter rimane `select.omics-language-select[data-lang]`. Leggere
il valore renderizzato evita di anticipare un cambio lingua prima che il form
abbia successo. Cambiare soltanto `select.value` o `data-lang` senza il normale
reload non è un trasporto runtime implementato per la lingua.
L'assenza del contesto host atteso produce un errore di integrazione; non abilita
controlli locali. Non si introducono eventi `ready/state`, handshake, timeout,
versioni dei messaggi o comandi duplicati. Selettori e dettagli Omics non devono
essere letti dai componenti applicativi.
## Proprietà per modalità
| Funzione | Full | Embedded |
| --- | --- | --- |
| Header | ThothII | solo Omics |
| Lingua | selettore locale | selettore Omics, normale reload Django |
| Tema | toggle locale light/dark | stato Omics |
| Fullscreen | controllo locale, stato reale | controllo Omics, stato reale |
| Login/logout | ThothII local/OIDC; upstream non offre logout locale | autenticazione Omics esistente |
| Nome utente | header ThothII | header Omics |
| Rotellina amministrativa | mai | eventuale comando del portale |
## Accesso e continuità
Il server Omics verifica l'accesso a Datamart Builder e il proxy trasmette i
principal header normalizzati al backend ThothII. La UI usa `/me`; non effettua
un secondo login. Un altro portale deve soddisfare anche questo contratto server,
oltre a fornire una nuova implementazione dell'adapter UI.
Il [contratto upstream](../install/authentication-upstream.md) specifica header,
origine, rete e configurazioni incompatibili. Lo snapshot non può contenere
`authenticated`, utente, ruoli, cookie o token; un evento browser non autorizza
una richiesta API. Il prefisso API viene configurato separatamente prima del
caricamento React, non viene dedotto dall'adapter.
Il logout del portale segue la sua navigazione. Una perdita di accesso rilevata
dal server chiude lo stato protetto; un 403 di una singola operazione non equivale
automaticamente a logout. La riconnessione degli eventi e il ritorno alla pagina
ricontrollano l'accesso. Non si garantisce revoca istantanea di una connessione
aperta in un'altra scheda attraverso il solo controllo iniziale del proxy.
Il cambio lingua può ricaricare la pagina: conservare la selezione della sessione,
proteggere le modifiche non salvate e non avviare una nuova generazione al reload.
Non si conserva una trascrizione integrale nel browser. La lingua della sessione
rimane quella registrata nel manifest, secondo ADR 0022.
## Sostituzione e verifiche
Un nuovo adapter può usare un diverso documento o trasporto, ma deve rispettare
la stessa sottoscrizione e mantenere la conoscenza del portale nella propria
implementazione. Oggi `ShellProvider` istanzia direttamente `OmicsPortalAdapter`:
per sostituirlo aggiornare quel punto e i nomi accettati in
`tools/tht/internal/config/shell.go` e `frontend/src/api/runtime-config.ts`.
Non è disponibile il caricamento dinamico di classi da una stringa YAML.
Non occorre implementare ora iframe o un secondo portale.
La nuova implementazione deve pubblicare uno snapshot iniziale completo, poi gli
aggiornamenti; segnalare contesto invalido; liberare tutti i listener alla
disiscrizione. Locale ben formato ma non tradotto significa fallback inglese;
locale assente/malformato e tema diverso da light/dark sono errori di integrazione.
Non cambiare componenti applicativi o workflow per aggiungere selettori specifici
del nuovo portale.
Verificare snapshot prima/dopo il montaggio, tema, fullscreen con Esc, cleanup,
contesto host mancante, assenza di header ThothII embedded, accesso singolo,
locale dopo reload e compatibilità del prefisso API. Full deve funzionare senza
alcun elemento Omics presente.
Vedere anche [architettura del rendering](../architecture/application-shell.md)
e [matrice di accettazione](../testing/authentication-manual-acceptance.md).
+8 -5
View File
@@ -6,8 +6,9 @@ When present, `evidence` is strict: it contains `source` and a defaulted strict
source variant and the policy reject unknown keys.
The version numbers are intentionally separate: the Evidence descriptor supports v1/v2, while the
latest Curated Evidence Unit format is v3. There is no Evidence descriptor v3/v4 and no Curated
Evidence Unit v4.
latest Curated Evidence Unit format is [v4](curated-evidence-v4.md). There is no Evidence
descriptor v3/v4. The editable local archive is implemented in E1; installation integration
is the next increment, E2.
```mermaid
flowchart LR
@@ -67,9 +68,11 @@ ignore it. Domain rules also retain their exact canonical text in an invisible `
comment while presenting long prose as paragraphs, labelled subsections, and semicolon-derived
lists. Runtime chunking reads the parsed canonical rule, not this review-only presentation.
Newly prepared units use v3. `tht evidence migrate <workspace-root>` upgrades v1 and v2 units and
canonicalizes an older v3 presentation locally without a model call, commit, publication, or
semantic change.
The representations above are legacy conversion inputs. Newly prepared units use editable v4:
short YAML metadata, a visible H1 title and typed H2 payload fields, with no hidden content copy.
`tht evidence migrate <workspace-root>` converts v1–v3 to v4 and initializes a local archive
baseline without a model call, commit, activation, or semantic change. See the
[v4 editing and consolidation contract](curated-evidence-v4.md).
### Example: filesystem
+26 -7
View File
@@ -14,12 +14,30 @@ tht --installation <absolute>/thothii-installation.yaml workspace preprocess run
tht --installation <absolute>/thothii-installation.yaml workspace preprocess clear
--workspace <id> [--json]
tht --installation <absolute>/thothii-installation.yaml workspace evidence consolidate
--workspace <id> [--json]
tht --installation <absolute>/thothii-installation.yaml workspace evidence refresh
--workspace <id> [--json]
tht --installation <absolute>/thothii-installation.yaml workspace evidence decide
--workspace <id> --source-id <64-hex> --revision <64-hex>
--decision keep|replace [--json]
```
There are no public partial commands for DWH introspection, LSH, FK suggestions, schema indexing,
Evidence indexing, or Qdrant rebuild. `preprocess run` does not accept `--resume`, `--dry-run`, a
or Qdrant rebuild. The separate Evidence curation command validates editable local files and
activates only their index; it does not satisfy workspace preprocessing readiness or execute Git.
See [Curated Evidence v4](curated-evidence-v4.md#administration-and-installed-command).
Source refresh acquires and refines only on explicit request, saving comparisons without
changing active Evidence. Source decisions activate through the same Evidence-only
pipeline and preserve Catalog readiness. Their envelopes reject arbitrary URLs, paths
and extra flags; source locations and credentials come from installation configuration.
`preprocess run` does not accept `--resume`, `--dry-run`, a
generation identifier, or a rollback option. Re-running it replaces the preceding derived output.
`preprocess clear` removes all replaceable preprocessing output and preserves runtime memory.
`preprocess clear` removes all replaceable preprocessing output and preserves runtime memory
and the canonical local Evidence archive. Full preprocessing remains required afterward.
## Sources of truth
@@ -29,8 +47,10 @@ generation identifier, or a rollback option. Re-running it replaces the precedin
- PostgreSQL Metadata Catalog is the sole database authority. It owns the workspace/database
association, installation-local binding, tables, columns, descriptions, sensitivity flags,
physical foreign keys, and active logical relationships.
- The workspace Git revision remains authoritative for Evidence. Evidence Descriptor v1/v2 and
Curated Evidence Unit v3 are unchanged; there is no Evidence v4.
- Evidence Descriptor v1/v2 still configures the initial source. Initialized archives use
editable Curated Evidence Unit v4 and the last successfully activated local snapshot.
Ordinary preprocessing never imports unconsolidated working-tree edits. Before initialization,
the pinned workspace Git source remains supported.
No metadata is imported from legacy workspace YAML or `physical.yaml`/`annotations.yaml`.
@@ -98,9 +118,8 @@ generation produced for another database or revision.
`workspace preprocess clear` is intentionally narrower than deleting all semantic data. It:
1. copies any `memory` and `solved_question` records still present in the pre-split workspace
collection to `<workspace>-memory`, then retires that legacy collection (a normal preprocessing
write performs the same one-time cutover if clear was not invoked first);
1. leaves `<workspace>-memory` unchanged; Memory projections are reconstructed only from
the authoritative PostgreSQL archive, never imported from legacy vector payloads;
2. deletes `<workspace>-reference`;
3. removes the active LSH generation, Evidence corpus, private Catalog snapshot, and derived job
checkpoints;
+54 -5
View File
@@ -2,7 +2,30 @@
This page describes the complete workspace Evidence lifecycle: where original material lives, how curated units are produced, when they become available at runtime, and what the author and reviewer are responsible for.
## Publication rule
## Editable local Evidence
E1 adds [Curated Evidence v4 and a persistent local archive](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md).
The visible title and payload fields are authoritative Markdown. New manual units need no
external source; consolidation records their curator and distinguishes later corrections
from original documentary provenance. The core archive API creates immutable candidates
and advances its active pointer only after successful indexing.
E2 adds **Administration → Evidence management**, actual host file paths, complete
browsing and filtering, and the installed `tht workspace evidence consolidate
--workspace <id>` command. Edit files externally, consolidate to activate them, then
review and run Git manually. Runtime consumes only the active local snapshot.
The [v4 contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md#administration-and-installed-command)
describes host mounting, first conversion, failure recovery and Clear behavior.
The local PSD preview has 35 converted and indexed units. E3 adds explicit import/refresh
from local drafts and configured HTTP/S3 sources, durable current/proposed comparisons,
and keep/replace decisions with activation and retry. See
[Import drafts and refresh sources](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md#import-drafts-and-refresh-sources).
Workflow gate corrections remain the subsequent shared increment, X1.
## Existing repository publication path
The remainder describes the legacy, uninitialized repository source path. Initialized
v4 local archives use the lifecycle above; manual declarations need no source document.
The workspace repository is the versioned source. ThothII reads it, validates it, and publishes an atomic generation. It does not modify, commit, or push the author's repository.
@@ -46,7 +69,33 @@ The legacy configuration may expose `source_root`, such as `${THT_DOCS_ROOT}` or
HTTP and S3 are separate adapters. They do not use the filesystem structure `source/` and `curated/`, but they must still provide stable provenance, without credentials in URIs, under the adapter-specific contract.
## What a curated unit must contain
## What an editable curated unit contains
New preparation produces unit schema v4. The first H1 contains its title; documented H2
sections contain the typed payload. A minimal manual unit is:
```markdown
---
schema_version: 4
id: evidence:order-key
kind: domain
language: en
purposes: [sql_generation]
---
# Order key
## Rule
Join orders using the order number, financial year and company.
```
Use `tht evidence migrate <workspace-root>` for deterministic legacy conversion. The
conversion preserves typed content and initializes an archive baseline; it does not
activate the local corpus. See the [v4 contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md) for
all eight kinds, provenance, file layout and the E1/E2 boundary.
## Legacy v3 representation
Canonical Curated Evidence v3 hides canonical machine metadata in an HTML comment and renders the
whole review surface as real Markdown. GitHub therefore shows no frontmatter table. The body layout
@@ -94,7 +143,7 @@ La fascia pediatrica comprende i pazienti con età inferiore a 18 anni.
The actual files contain invisible `tht:` comments for canonical metadata and typed-field
boundaries. Removing, duplicating, or desynchronizing them makes validation fail closed instead of
silently ignoring content. Unit schemas v1 and v2 remain readable for compatibility, but newly
prepared units use v3.
prepared units use v4. Unit v3 remains readable as a conversion input.
Curated units must be atomic, readable by a second reviewer, and supported by the source.
Provenance references must lead back to the original file and the passage that supports the claim.
@@ -213,5 +262,5 @@ Formulas use a format distinct from document Evidence. A formula proposed during
## Contract references
- [Workspace Evidence v3 contract](contracts/workspace-evidence-v3.md)
- [Preprocessing CLI contract](contracts/workspace-preprocessing-cli.md)
- [Workspace Evidence v3 contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/workspace-evidence-v3.md)
- [Preprocessing CLI contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/workspace-preprocessing-cli.md)
+130 -9
View File
@@ -8,14 +8,26 @@ declare providers, model allowlists, defaults, embeddings, dimensions, or vector
Do not edit `deploy/pi/models.json`, `deploy/pi/settings.json`, files under `generated/`, or
provider/model environment defaults. Those former sources are retired.
## Host instructions in Administration
The **Pi configuration** navigation button opens **Pi management**. Its Host maintenance
section selects the installation host's Linux, macOS, or Windows tab automatically; the
browser's operating system does not affect it. Selecting another tab is still possible.
The host CLI includes `THT_HOST_PLATFORM` in the generated Compose projection using the OS
on which it runs. Generate projections on the destination host, not on another computer,
and do not edit generated files. Without this projection (older deployments or native
development), the backend reports its own OS; a Linux Docker container cannot discover
whether the physical host is macOS or Windows. Run the updated host CLI's normal
configuration reload on the destination installation to regenerate this information.
## Minimal catalog
```yaml
schemaVersion: 2
modelCatalog:
defaults:
session: zai/glm-5.3
metadataGeneration: zai/glm-5.3
interaction: zai/glm-5.3
embedding:
id: ollama/qwen3-embedding:0.6b
@@ -50,23 +62,63 @@ it contains that use block:
- `metadataGeneration` makes it selectable for description generation;
- `embedding` is a single installation-level model rather than a selectable list.
`defaults.session` is required. `defaults.metadataGeneration` is required exactly when at least
one metadata-generation model exists. A session manifest pins its canonical identity, so removing a
model never silently changes an existing session: resume fails with `model_unavailable`.
`defaults.interaction` is the only LLM default, required once per installation, never per workspace.
Core and Administration share the user's operational model choice. An explicit choice takes priority
over the default and is remembered in this browser for the authenticated user and application mount.
Switching workspace does not change the model. Browser-storage restrictions may limit remembering
to the current visit; preferences do not synchronize across devices or change installation YAML.
An unavailable remembered model is not silently replaced: select another configured model.
When any metadata-generation models are configured, the default and the operational list must
support both `session` and `metadataGeneration`. Entries for just one adapter may remain in the
installation inventory, but are not selectable for global interaction. Core-only installations remain
supported when no metadata-generation model is configured; Admin AI is then unavailable.
The embedding model remains separate and is unaffected by the interaction selector.
New sessions record the selected model in their manifest. Resume retains the session's workspace and
revision, but uses the current global model (the installation default for clients that omit a model).
Historical manifest model fields are not rewritten by resume. Archived/finalized sessions remain read-only.
### Required operator verification for every model
Catalog validation checks configuration, not model behavior. Before offering a model to users, and
after changing its endpoint, adapters, or the Pi/LiteLLM versions, the operator must verify **both**:
1. **Core / Pi:** select the model, start a test session in a prepared test workspace, exercise an
actual tool call and its returned result, a human review gate, and stop/resume. Check streaming,
tool arguments, authentication, and reasoning/token-limit compatibility. A plain chat reply or
`tht pi test` alone is not sufficient.
2. **Administration / LiteLLM:** select the same model and generate descriptions for a small,
non-sensitive test table. Check the structured result is accepted and the generation completes.
Review the output quality before using it on real metadata. This action writes test metadata
and may incur provider charges: use an authorized test database and approved data.
There is no automatic certification flag or startup model probe. The operator owns this verification;
do not infer compatibility from the model label or from success in just one path. Both adapters point
to one catalog identity; Pi does not need to route through a new LiteLLM proxy. Models using only
`pi_auth` cannot serve the current LiteLLM path and are excluded from shared selection.
## Session adapters
Use `pi_builtin` for a model whose technical definition ships with Pi:
Use `pi_builtin` for a model whose technical definition ships with Pi. This does not require
`pi_auth`: a shared bundle credential lets native Pi and LiteLLM use the same provider identity:
```yaml
deepseek:
authentication:
mode: pi_auth
mode: secret_env
apiKeyEnv: DEEPSEEK_API_KEY
session:
mode: pi_builtin
metadataGeneration:
litellmProvider: deepseek
models:
deepseek-v4-pro:
session: {}
metadataGeneration: {}
deepseek-v4-flash:
session: {}
metadataGeneration: {}
```
Use `openai_compatible` for an explicit compatible endpoint. Each eligible session model must then
@@ -76,6 +128,51 @@ endpoint expects a model name different from the catalog key.
Provider integrations remain declarative. Do not register providers from
`harness/.pi/extensions/`; those extensions implement the workflow and human gates only.
### Qwen 3.6 sessions and thinking controls
For Qwen served through a vLLM-compatible chat template, declare the following inside the
model's `session` block, alongside its context and output limits:
```yaml
reasoning: true
compatibility:
supportsDeveloperRole: false
supportsReasoningEffort: false
supportsStore: false
maxTokensField: max_tokens
thinkingFormat: qwen-chat-template
```
This makes Pi send `chat_template_kwargs.enable_thinking` from the selected thinking level,
with `preserve_thinking: true`. Choose **off** to explicitly disable thinking. The alternative
`thinkingFormat: qwen` is for endpoints expecting top-level `enable_thinking`. Both formats
require `reasoning: true`; declaring `reasoning: false` does not tell the server to disable
thinking. Omit `thinkingFormat` to preserve Pi's default behavior for other providers.
Regenerate projections with the updated host CLI and recreate the local core container after
rebuilding it. Do not add these fields directly to generated Pi files. These controls do not
force tool calls or certify the workflow; perform the operator verification above.
```sh
tht --installation /absolute/path/thothii-installation.yaml installation generate
```
Use the model identifier exposed by your endpoint, such as `qwen3.6-35b-a3b`, and limits
supported by that deployment. The thinking format configures the Pi session adapter;
metadata generation continues to use its separate LiteLLM settings.
If a session displays text such as `{"type":"bash","command":"tht session show … --json"}`
and never opens a review widget, that text is not an executed tool call. A verified cause
was the Evidence JSON extension being loaded into interactive sessions and forcing
`response_format: {type: "json_object"}`. Upgrade to the core image containing the fix:
the extension belongs in `.pi/evidence-extensions/` and is loaded explicitly only by
Evidence authoring. It must not also remain in the automatically loaded `.pi/extensions/`
directory. Regenerating model configuration alone does not remove an extension from an old image.
After upgrading, reload the browser and resume the session. Verify that Pi executes
`tht session show` and opens a review widget. This fix does not require changing the Qwen
server, forcing every turn to call a tool, or teaching the model to print tool-call JSON.
## Authentication
Every provider chooses one explicit mode:
@@ -89,6 +186,18 @@ Secret values never belong in installation YAML, generated files, logs, CLI argu
requests. The YAML contains only an environment-variable name or an authentication mode. Pi's
protected credential file remains selected by the installation authentication configuration.
For catalog providers using `secret_env`, the bundle is authoritative in both Core and Admin.
ThothII removes only the selected provider's old auth entry from the temporary Pi session snapshot;
the operator's original Pi auth store and other providers are unchanged. Provider smoke checks use
the same precedence, and model enumeration receives the catalog-declared bundle keys. A missing
declared key is an error, not permission to fall back to Pi auth or the legacy generic key file.
After rotating a bundle key, apply the normal installation lifecycle so processes reload it.
The PSD descriptor now declares only `deepseek/deepseek-v4-pro` and `deepseek/deepseek-v4-flash`
for both uses; it no longer duplicates them under `deepseek-metadata`. Historical records are not
rewritten. A saved obsolete identity must be explicitly reselected from the current catalog;
it is not silently remapped to another model or account.
## Generated runtime projections
Before Compose starts, `tht` validates the installation and atomically writes deterministic files
@@ -133,6 +242,16 @@ configuration command. There is no `tht pi configure` and no separate apply comm
## Migrating a legacy installation
For schema-v2 descriptors with the former `defaults.session` and `defaults.metadataGeneration`,
replace both with `defaults.interaction`. Equal legacy values are accepted and normalized in memory;
the loader never rewrites the descriptor. Different values fail with `migration_required`: explicitly
choose a model supporting both uses, remove both old fields, and set the single new field. Do not mix
new and legacy fields. A Core-only legacy session default can be normalized when Admin AI is absent.
The generated runtime catalog now uses schema version **2** and only `defaultInteraction`. Regenerate
and apply all runtime projections with the matching host/backend release using the normal installation
lifecycle; do not deploy only the backend against an old generated catalog or hand-edit generated JSON.
The migrator reads the former installation `metadataGeneration` block and the two former Pi JSON
files, but never modifies them. Supply the facts that cannot be inferred safely and write a separate
candidate:
@@ -148,13 +267,15 @@ tht --installation /absolute/path/legacy/thothii-installation.yaml installation
Review the candidate, move the legacy source files out of the installation only after approval,
then select the v2 descriptor. Ambiguous aliases, endpoint conflicts, or missing authentication
facts produce field-level errors; the migrator does not guess.
The legacy CLI flag `--session-default` now supplies the unified interaction default in the candidate;
if it conflicts with the legacy metadata default, align that choice explicitly before retrying.
## Troubleshooting
| Symptom | Meaning | Action |
| --- | --- | --- |
| `migration_required` | A retired model source or installation schema is still present | Run the installation migrator and review its candidate |
| Unknown session or metadata default | The canonical ID is missing the corresponding use block | Correct the provider/model key or add the intended use block |
| Invalid interaction default | The canonical ID does not support all configured uses | Correct `defaults.interaction` or the intended adapter blocks |
| Generated projection drift | Runtime files differ from the descriptor-derived bytes | Run `tht start` or `tht pi restart --yes --drain` |
| `model_unavailable` on resume | The session's pinned model is no longer session-eligible | Restore that catalog entry or keep the session unavailable; do not remap it |
| `model_unavailable` on create/resume | The selected global model is no longer eligible | Explicitly choose an eligible model; no fallback is applied |
| Provider smoke failure | Credentials, endpoint, or provider availability is invalid | Correct the protected credential or catalog endpoint, restart, then run `tht pi test` |

Some files were not shown because too many files have changed in this diff Show More