Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2f53512e4d | ||
|
|
497ab84031 | ||
|
|
0d2e573e0d | ||
|
|
bd416f7327 | ||
|
|
23e52c80de | ||
|
|
84084bba37 | ||
|
|
efd7d788d9 | ||
|
|
b1c510a097 | ||
|
|
5f3680a0fb | ||
|
|
4ff91e8d6e | ||
|
|
5f3a7f5975 | ||
|
|
043ffdfad6 | ||
|
|
6a4634dcf1 | ||
|
|
84804be9f8 | ||
|
|
c3caba94dd | ||
|
|
b1723c34c4 | ||
|
|
d6cdffea62 | ||
|
|
49333a2d35 | ||
|
|
b006b94479 | ||
|
|
bdcd8fcd28 | ||
|
|
cf90c1bd51 | ||
|
|
571a4bcaa2 | ||
|
|
9051463654 | ||
|
|
26c5605ff7 | ||
|
|
3535fda958 | ||
|
|
2953f6b608 | ||
|
|
45db3a239b | ||
|
|
7d826e46c0 | ||
|
|
648434a32e | ||
|
|
023b822f83 | ||
|
|
d8a29bfbdd | ||
|
|
a59624a68f | ||
|
|
5af4408194 | ||
|
|
e088abd60a | ||
|
|
803e9e9201 | ||
|
|
8c81996896 | ||
|
|
eed398e569 | ||
|
|
c8d276ddc6 | ||
|
|
2d1b714ebe | ||
|
|
c7e5f295e6 | ||
|
|
f52bf22e05 | ||
|
|
840344706f | ||
|
|
41b9fed4d5 | ||
|
|
36bf659ea9 | ||
|
|
4a67d60233 | ||
|
|
debb63d87b | ||
|
|
3943022a97 | ||
|
|
f5ec2d9313 | ||
|
|
82e2c91f42 | ||
|
|
8fe526dd6e | ||
|
|
e68e80a33d | ||
|
|
818563c408 | ||
|
|
50c546e42d | ||
|
|
651a5c7902 |
@@ -17,6 +17,7 @@ frontend/vite.database-management-prototype.config.ts
|
||||
!deploy/env/*.env.example
|
||||
deploy/thothii.env
|
||||
deploy/secrets/
|
||||
deploy/psd/
|
||||
harness/workspaces/*.yaml
|
||||
!harness/workspaces/local.yaml
|
||||
!harness/workspaces/tht.example.yaml
|
||||
@@ -29,3 +30,7 @@ coverage/
|
||||
data/
|
||||
sessions/
|
||||
workspace-registry/
|
||||
|
||||
.tht/
|
||||
|
||||
deploy/local/
|
||||
|
||||
@@ -7,6 +7,11 @@ on:
|
||||
paths:
|
||||
- "docs/**"
|
||||
- "mkdocs.yml"
|
||||
- "scripts/build-docs.sh"
|
||||
- "scripts/verify-public-docs.py"
|
||||
- "scripts/test-verify-public-docs.py"
|
||||
- "scripts/verify-auth-docs.py"
|
||||
- "scripts/test-verify-auth-docs.py"
|
||||
- "docs/requirements.txt"
|
||||
- ".gitea/workflows/publish-docs.yml"
|
||||
workflow_dispatch:
|
||||
@@ -34,14 +39,25 @@ jobs:
|
||||
with:
|
||||
python-version: "3.x"
|
||||
cache: pip
|
||||
cache-dependency-path: docs/requirements.txt
|
||||
cache-dependency-path: docs/requirements.lock
|
||||
|
||||
- name: Install MkDocs dependencies
|
||||
run: python -m pip install -r docs/requirements.txt
|
||||
run: python -m pip install -r docs/requirements.lock
|
||||
|
||||
- name: Test public documentation boundary
|
||||
run: python scripts/test-verify-public-docs.py
|
||||
|
||||
- name: Test current authentication documentation
|
||||
run: |
|
||||
python scripts/verify-auth-docs.py auth
|
||||
python scripts/verify-auth-docs.py dwh
|
||||
python scripts/test-verify-auth-docs.py auth
|
||||
python scripts/test-verify-auth-docs.py dwh
|
||||
|
||||
- name: Build documentation
|
||||
# Some documented source files intentionally live outside docs/.
|
||||
run: mkdocs build
|
||||
run: |
|
||||
mkdocs build --strict
|
||||
python scripts/verify-public-docs.py
|
||||
|
||||
- name: Publish generated site to the pages branch
|
||||
working-directory: site
|
||||
|
||||
@@ -46,6 +46,7 @@ deploy/secrets/*
|
||||
# Per-installation configuration generated by `tht setup` (examples stay tracked).
|
||||
deploy/*/thothii-installation.yaml
|
||||
deploy/*/operator.env
|
||||
deploy/*/auth/
|
||||
deploy/*/generated/
|
||||
deploy/*/secrets/*
|
||||
!deploy/*/secrets/.gitkeep
|
||||
|
||||
@@ -98,8 +98,18 @@ frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness →
|
||||
- **`tht`'s `-c`/`--config` is a PER-COMMAND option** — it must follow the subcommand, never
|
||||
precede it (`ThtRunner.buildArgv` enforces this; prepending caused live 500s).
|
||||
- **`--json` output must be pristine** (only valid JSON on stdout) — used as a machine contract.
|
||||
- **UI strings are English; document *content* stays the workspace language** (Italian for
|
||||
`psd`) because it's the real data. Only chrome/labels are English.
|
||||
- **Localization:** deterministic UI uses the EN/IT catalogs with English fallback;
|
||||
model interaction uses the session manifest's immutable `interaction_language`.
|
||||
Workspace content, SQL, and identifiers remain unchanged. For shell modes, portal
|
||||
integration, or translations, read `docs/operations/shell-and-localization.md`.
|
||||
- **Server deployment:** for the coordinated ThothII/Omics upgrade, follow
|
||||
`docs/operations/server-codex-handoff.md`; it supersedes earlier Omics delivery
|
||||
instructions. Omics source integration uses GitHub with no repository relay prerequisite.
|
||||
- **Server identity:** for portal login/logout, proxy headers, or Omics deploy, read
|
||||
`docs/install/authentication-upstream.md` before changing authentication. Omics
|
||||
uses embedded/upstream, not a second ThothII OIDC login. Full/embedded rendering
|
||||
is documented in `docs/architecture/application-shell.md`; release acceptance
|
||||
is in `docs/testing/authentication-manual-acceptance.md`.
|
||||
- **Workspace schema v4** defines workspace identity and optional Evidence only. PostgreSQL Metadata
|
||||
Catalog owns database identity, binding, schema, descriptions, sensitivity, and relationships;
|
||||
embedding/model facts come from the installation catalog. The legacy `harness/workspaces/*.yaml` runtime snapshots still use
|
||||
|
||||
@@ -91,21 +91,79 @@ correzione successiva crea una nuova sessione derivata, collegata a quella prece
|
||||
dopo la finalizzazione. Non può modificare il ledger, gli artifact canonici o lo stato
|
||||
terminale della sessione.
|
||||
|
||||
## Memory
|
||||
|
||||
**Memory Module** — Il modulo che possiede le conoscenze ed esperienze curate per
|
||||
migliorare schema linking e generazione SQL di domande future. Le Memory appartengono
|
||||
a un workspace e rimangono distinte dalle Evidence.
|
||||
|
||||
**Memory Card** — L'unità di contenuto gestibile del Memory Module, con identità,
|
||||
ambito di applicazione e provenienza. Il formato è allineato per analogia alle
|
||||
Evidence, senza implicare la stessa origine o lo stesso percorso di pubblicazione.
|
||||
|
||||
**Reusable Memory** — Una Memory Card che esprime un chiarimento di dominio, una
|
||||
regola di costruzione SQL o un errore da evitare con motivo compreso e approvato.
|
||||
La sua validità è circoscritta a un ambito esplicito e non deriva dalla sola
|
||||
approvazione di una scelta occasionale in una domanda.
|
||||
|
||||
**Solved Question** — Una Memory Card che conserva una domanda risolta con la
|
||||
relativa soluzione SQL e il contesto necessario a interpretarla. È un exemplar
|
||||
consultativo: i parametri e le scelte del caso non diventano regole generali.
|
||||
|
||||
**Memory Graph** — L'insieme dei collegamenti espliciti fra card che contribuisce
|
||||
al recupero di conoscenze pertinenti oltre alla somiglianza del contenuto. Il
|
||||
ritrovamento di una card tramite un collegamento non ne implica l'approvazione.
|
||||
|
||||
**Memory Link** — Un collegamento curato fra card, con destinazione e significato
|
||||
espliciti, che contribuisce alla consultazione di contenuti pertinenti. La sua
|
||||
rimozione non comporta la cancellazione delle card collegate.
|
||||
|
||||
## Evidence
|
||||
|
||||
**Context specialist** — La persona competente sul dominio che redige e cura il
|
||||
contenuto delle Evidence. Può essere distinta da chi amministra l'installazione;
|
||||
il suo lavoro di redazione non richiede accesso al database applicativo.
|
||||
|
||||
**Evidence draft** — Il documento iniziale scritto dallo specialista di contesto,
|
||||
che il sistema acquisisce e raffina in Evidence Unit. Può essere redatto e
|
||||
consegnato indipendentemente dall'installazione che userà le Evidence risultanti.
|
||||
|
||||
**Evidence Module** — Il modulo autonomo che possiede la preparazione delle Evidence e
|
||||
la loro consultazione durante il workflow. La preparazione avviene fuori dalle singole
|
||||
sessioni; il workflow usa soltanto contenuti già pubblicati. A runtime contribuisce agli
|
||||
stage semantici esistenti, senza diventare uno stage visibile e senza modificare ledger,
|
||||
artifact o stato del workflow.
|
||||
|
||||
**Source Evidence** — Un documento originale del workspace, conservato senza modifiche
|
||||
come riferimento umano e origine della successiva ristrutturazione.
|
||||
**Source Evidence** — Il documento o la dichiarazione che sostiene il contenuto
|
||||
corrente di una Evidence Unit. Un documento acquisito viene conservato come
|
||||
riferimento umano; una dichiarazione manuale attribuisce il contenuto alla persona
|
||||
che lo ha scritto e approvato.
|
||||
|
||||
**Manual Evidence declaration** — Una dichiarazione esplicita dell'amministratore
|
||||
che sostiene una Evidence creata direttamente o una correzione del suo significato.
|
||||
Non implica una verifica indipendente da parte di una fonte documentale esterna.
|
||||
|
||||
**Evidence origin** — Il documento da cui una Evidence Unit è stata inizialmente
|
||||
derivata. Può restare collegato per provenienza e confronto con gli aggiornamenti
|
||||
anche quando una dichiarazione manuale sostiene il testo corrente. La sola origine
|
||||
non dimostra il supporto semantico di una successiva correzione.
|
||||
|
||||
**Local Evidence archive** — L'insieme delle Evidence curate custodite
|
||||
dall'installazione, distinto dalle draft originali e dai contenuti derivati per
|
||||
la ricerca. Comprende le correzioni manuali e i ritiri deliberati.
|
||||
|
||||
**Consolidated Evidence** — Una versione delle Evidence locali controllata come
|
||||
insieme coerente e pronta per l'attivazione. I file ancora in modifica non ne
|
||||
cambiano il contenuto.
|
||||
|
||||
**Active Evidence** — La versione consolidata disponibile alla consultazione del
|
||||
core. Un tentativo di aggiornamento fallito conserva la versione attiva precedente.
|
||||
|
||||
**Evidence Unit** — La più piccola unità semantica coerente, revisionabile e ricercabile
|
||||
derivata da una sola Source Evidence. Possiede un identificatore stabile indipendente
|
||||
dal kind, assegnato una volta nella forma `evidence:<slug>`; fonti diverse non vengono
|
||||
fuse automaticamente.
|
||||
fondata su una Source Evidence corrente, anche manuale, e con eventuale origine
|
||||
documentale distinta. Possiede un identificatore stabile indipendente dal kind,
|
||||
assegnato una volta nella forma `evidence:<slug>`; fonti diverse non vengono fuse
|
||||
automaticamente.
|
||||
|
||||
**Evidence kind** — La categoria semantica di una Evidence Unit, che ne determina i
|
||||
campi specifici e ne orienta l'uso. Ogni unità ha un solo kind primario; i tipi iniziali
|
||||
@@ -151,10 +209,9 @@ avanzare fino a un retry riuscito.
|
||||
nella sessione: stage semantico, purpose, generazione interrogata e identificatori delle
|
||||
Evidence restituite. Non duplica il contenuto delle Evidence.
|
||||
|
||||
**Curated Evidence** — Una o più Evidence Unit ristrutturate a partire da una Source
|
||||
Evidence e conservate nel repository del workspace come proposte per la revisione
|
||||
umana. Git conserva la versione precedente e rende visibile ogni modifica; una Curated
|
||||
Evidence non è ancora contenuto autorevole del runtime.
|
||||
**Curated Evidence** — Una o più Evidence Unit preparate da documenti o curate
|
||||
manualmente. La presenza nell'archivio curato non implica da sola che il contenuto
|
||||
sia già attivo per il workflow.
|
||||
|
||||
**Published Evidence** — Le Curated Evidence valide appartenenti alla revisione attiva
|
||||
del workspace e alla generazione Evidence pubblicata. L'approvazione umana precede
|
||||
@@ -176,9 +233,17 @@ una Evidence Unit. Il sistema ne verifica deterministicamente la presenza dopo l
|
||||
normalizzazione meccanica; il curatore resta responsabile di verificarne la sufficienza
|
||||
semantica.
|
||||
|
||||
**Evidence resolution** — L'operazione esplicita con cui un curatore ritira una
|
||||
Evidence Unit oppure la ricollega a un Source Evidence esistente. Aggiorna documento e
|
||||
manifest insieme, lascia un diff Git revisionabile e non pubblica né crea commit.
|
||||
**Evidence resolution** — La decisione esplicita con cui un curatore risolve un
|
||||
problema di una Evidence Unit, correggendola, ritirandola oppure ricollegandola a
|
||||
una fonte adeguata.
|
||||
|
||||
**Source update conflict** — Un contrasto fra una fonte aggiornata e una correzione
|
||||
manuale già approvata. La correzione resta in uso fino alla risoluzione esplicita
|
||||
del confronto da parte dell'amministratore.
|
||||
|
||||
**Evidence source refresh** — La riacquisizione delle fonti esterne richiesta
|
||||
dall'amministratore per rilevarne le modifiche. Fra due aggiornamenti il contenuto
|
||||
già acquisito resta il riferimento per preparazione e consultazione.
|
||||
|
||||
**Review item** — Un blocco di revisione descritto da codice stabile, messaggio umano e
|
||||
campo opzionale. Finché viene mantenuto nell'Evidence Unit, ne impedisce la
|
||||
@@ -506,3 +571,73 @@ _Avoid_: Sensitive Data Suggestion Event
|
||||
**Introspection Capability** — Una categoria di struttura fisica che una Database Binding
|
||||
può osservare, come tabelle, colonne, relazioni, indici o enum. Una capability non disponibile
|
||||
è distinta da una capability osservata che non ha restituito elementi.
|
||||
|
||||
## Amministrazione e integrazione
|
||||
|
||||
**Workspace Readiness** — La preparazione di uno specifico Workspace per l'uso nel
|
||||
workflow, comprensiva della disponibilità degli artefatti derivati dai suoi metadati
|
||||
Database e dalle sue Evidence. Il preprocessing appartiene a questa preparazione;
|
||||
la configurazione e la sincronizzazione del catalogo restano responsabilità Database.
|
||||
|
||||
**Administration Surface** — Una superficie amministrativa autonoma per configurare o curare una
|
||||
parte dell'installazione. Workspace, Evidence, Memory, Database e Pi sono superfici peer e non
|
||||
dipendono dall'esistenza di una sessione attiva.
|
||||
|
||||
**Administration Page** — La rappresentazione a pagina intera di una Administration Surface, con
|
||||
una gerarchia condivisa per identità, stato, azioni e contenuto. Un form amministrativo appartiene
|
||||
alla pagina e non a una popup come contenitore principale.
|
||||
_Avoid_: management popup, settings modal
|
||||
|
||||
**Administration Route** — L'identità navigabile di una Administration Surface nel browser. Deve
|
||||
essere ripristinabile con refresh e cronologia e non contiene valori transitori o segreti dei form.
|
||||
|
||||
**Embedded Thoth Shell** — L'esperienza Thoth ospitata dentro il documento e il contesto visuale di
|
||||
un portale host. Conserva la propria gerarchia funzionale, ma deve rispettare la geometria,
|
||||
l'autenticazione e le regole responsive del portale host.
|
||||
|
||||
**Full Thoth Shell** — L'esperienza Thoth autonoma che possiede il proprio header e il proprio
|
||||
layout di pagina. Non replica la navigazione amministrativa del portale host e non dipende dal suo
|
||||
template visuale.
|
||||
|
||||
**Shell mode** — La scelta di installazione fra `embedded` e `full`. Determina chi possiede il
|
||||
chrome globale, i comandi di identità e le integrazioni visuali, ma non cambia il workflow o la
|
||||
persistenza delle sessioni.
|
||||
|
||||
**Fullscreen state** — Lo stato temporaneo in cui il documento applicativo occupa il fullscreen
|
||||
del browser. È distinto da `Shell mode`: una Full Thoth Shell può essere aperta senza fullscreen;
|
||||
il passaggio è attivato da un comando esplicito e può essere annullato con la stessa azione o con
|
||||
il comando nativo del browser.
|
||||
|
||||
**Portal Shell Adapter** — Il confine sostituibile che traduce lo stato e i comandi del chrome di
|
||||
un portale host nel modello semantico usato da Thoth. L'adapter non possiede autorizzazione,
|
||||
sessioni di workflow o contenuti del modello.
|
||||
|
||||
**Host Shell State** — Il minimo stato visuale fornito dal portale host: locale UI, tema e stato
|
||||
fullscreen. In una Embedded Thoth Shell è la fonte autorevole per queste preferenze;
|
||||
non include identità, token o stato di autenticazione, che restano responsabilità dell'accesso.
|
||||
|
||||
**UI locale** — La lingua delle label, dei messaggi, dei tooltip, degli stati e delle istruzioni
|
||||
non generate dal modello nell'interfaccia Thoth. È distinta dalla lingua dei contenuti di un
|
||||
workspace.
|
||||
|
||||
**Interaction language** — La lingua in cui il modello presenta domande, spiegazioni e proposte
|
||||
al revisore durante una sessione. Viene fissata alla creazione della sessione e rimane invariata
|
||||
durante una ripresa, anche se la UI locale corrente cambia.
|
||||
|
||||
**Administrative Page Family** — L'insieme delle cinque Administration Page che condividono shell,
|
||||
navigazione, tipografia e regole responsive, pur mantenendo contenuti e operazioni specifici:
|
||||
Workspace, Evidence, Memory, Database e Pi.
|
||||
|
||||
## Installazione
|
||||
|
||||
**Manual standalone installation** — Una copia di ThothII predisposta per l'uso autonomo da una
|
||||
persona che possiede il computer, con una Full Thoth Shell e servizi applicativi locali. La
|
||||
procedura non implica che DWH o provider LLM siano locali o disponibili offline.
|
||||
|
||||
**Installation bootstrap** — L'insieme delle attività iniziali che rende disponibile una
|
||||
installazione manuale: verifica dell'host, generazione della configurazione, predisposizione
|
||||
delle credenziali protette e avvio dei servizi. Non è un installer dell'applicazione.
|
||||
|
||||
**Platform acceptance** — La verifica che una Manual standalone installation possa essere
|
||||
predisposta e avviata su una specifica combinazione di sistema operativo, architettura e runtime,
|
||||
distinta dalla verifica funzionale del collegamento a DWH e provider LLM.
|
||||
|
||||
@@ -11,50 +11,50 @@ colors:
|
||||
warm-graphite: "oklch(26.78% 0.0097 355.6)"
|
||||
muted-graphite: "oklch(51.33% 0.0088 345.6)"
|
||||
quiet-border: "oklch(90.93% 0.0035 354.7)"
|
||||
success-mint: "oklch(75.77% 0.1581 165)"
|
||||
success-mint: "oklch(46% 0.095 160)"
|
||||
navigation-active: "oklch(92.5% 0.052 23.2)"
|
||||
navigation-active-hover: "oklch(89.5% 0.071 23.2)"
|
||||
navigation-active-foreground: "oklch(36.5% 0.11 23.2)"
|
||||
navigation-active-border: "oklch(60% 0.135 23.2)"
|
||||
warning-amber: "oklch(85.23% 0.1386 78.9)"
|
||||
information-blue: "oklch(70.35% 0.1128 221.3)"
|
||||
warning-amber: "oklch(48% 0.09 70)"
|
||||
information-neutral: "oklch(51.33% 0.0088 345.6)"
|
||||
typography:
|
||||
display:
|
||||
fontFamily: "Fraunces, Source Serif Pro, Georgia, Times New Roman, serif"
|
||||
fontSize: "3rem"
|
||||
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
|
||||
fontSize: "1.5rem"
|
||||
fontWeight: 600
|
||||
lineHeight: 1.03
|
||||
letterSpacing: "-0.025em"
|
||||
headline:
|
||||
fontFamily: "Fraunces, Source Serif Pro, Georgia, Times New Roman, serif"
|
||||
fontSize: "1.875rem"
|
||||
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
|
||||
fontSize: "1.5rem"
|
||||
fontWeight: 600
|
||||
lineHeight: 1.15
|
||||
letterSpacing: "-0.015em"
|
||||
title:
|
||||
fontFamily: "Fraunces, Source Serif Pro, Georgia, Times New Roman, serif"
|
||||
fontSize: "1.2rem"
|
||||
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
|
||||
fontSize: "1.25rem"
|
||||
fontWeight: 600
|
||||
lineHeight: 1.25
|
||||
letterSpacing: "-0.01em"
|
||||
body:
|
||||
fontFamily: "Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
|
||||
fontSize: "0.9375rem"
|
||||
fontFamily: "Manrope Variable, Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
|
||||
fontSize: "1rem"
|
||||
fontWeight: 400
|
||||
lineHeight: 1.65
|
||||
letterSpacing: "normal"
|
||||
control:
|
||||
fontFamily: "Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
|
||||
fontFamily: "Manrope Variable, Manrope, -apple-system, BlinkMacSystemFont, Segoe UI, system-ui, Arial, sans-serif"
|
||||
fontSize: "0.875rem"
|
||||
fontWeight: 600
|
||||
lineHeight: 1.25
|
||||
letterSpacing: "0.005em"
|
||||
label:
|
||||
fontFamily: "ui-monospace, SF Mono, Cascadia Code, Menlo, Consolas, monospace"
|
||||
fontSize: "0.6875rem"
|
||||
fontFamily: "Manrope Variable, Manrope, system-ui, sans-serif"
|
||||
fontSize: "0.75rem"
|
||||
fontWeight: 600
|
||||
lineHeight: 1.25
|
||||
letterSpacing: "0.06em"
|
||||
letterSpacing: "normal"
|
||||
rounded:
|
||||
xs: "4px"
|
||||
sm: "6px"
|
||||
@@ -113,6 +113,16 @@ components:
|
||||
|
||||
# Design System: ThothII
|
||||
|
||||
## Visual review branch, September 2026
|
||||
|
||||
The revision on `codex/ui-visual-review` is approved for implementation and Docker visual review,
|
||||
not yet for adoption on `main`. The previous look remains recoverable from the base commit and
|
||||
the preserved Docker image. Historical prototypes must remain untouched.
|
||||
|
||||
This revision follows Impeccable's product register: one locally bundled Manrope family for the
|
||||
whole UI, five fixed size roles, red as the sole brand accent and additional color only for meaningful
|
||||
state. The primary scene remains an analyst reading data and SQL in a well-lit office.
|
||||
|
||||
## Overview
|
||||
|
||||
**Creative North Star: "The Clinical Workbench"**
|
||||
@@ -134,7 +144,7 @@ disciplined and tactile, never playful, sluggish, or visually unstable.
|
||||
**Key Characteristics:**
|
||||
|
||||
- Warm, restrained surfaces with one scarce red accent.
|
||||
- Editorial headings paired with highly legible operational body text.
|
||||
- One sans-serif family, with hierarchy expressed through size, weight and spacing.
|
||||
- Dense information organized through hierarchy, rhythm, and progressive disclosure.
|
||||
- Persisted artifacts and reviewer decisions presented as the visual source of truth.
|
||||
- Fast state feedback with reduced-motion parity.
|
||||
@@ -150,6 +160,15 @@ users can scan structure without adding nested containers.
|
||||
|
||||
## Colors
|
||||
|
||||
The full-mode application header matches Omics Portal's `--gsd-red-primary`
|
||||
(`#CB333B`) in both themes. Its complete wordmark, including `II`, and controls
|
||||
use a near-white foreground. This header is absent in embedded mode. The sidebar
|
||||
and welcome wordmarks retain their red suffix. Context editing places workspace,
|
||||
model and Done in one desktop row, stacking on narrow containers. Session-scope
|
||||
tabs retain their selected fill and accessible keyboard state with a uniform one-pixel
|
||||
border on every side, gray when inactive and red when active. Their padding is 11px
|
||||
horizontal and 3px vertical, with a 38px minimum height and wrapping labels.
|
||||
|
||||
The palette combines warm porcelain surfaces, warm graphite text, and an instrument red used only
|
||||
for action, focus, and important state. OKLCH values in the frontmatter are normative because the
|
||||
frontend uses OKLCH tokens directly.
|
||||
@@ -178,7 +197,8 @@ frontend uses OKLCH tokens directly.
|
||||
foreground. It shares Instrument Red's hue but uses a lighter, lower-chroma fill, so location is
|
||||
visible without carrying the full weight of a primary action.
|
||||
- **Warning Amber** (`warning-amber`): waiting, attention, and in-progress states.
|
||||
- **Information Blue** (`information-blue`): informational state when red would imply action.
|
||||
- **Information**: neutral text and indicators for dates, protocols and ordinary status. The legacy
|
||||
`--info` token resolves to muted foreground, not an additional blue accent.
|
||||
|
||||
The dark theme keeps the same semantic mapping with neutral near-black surfaces and a slightly
|
||||
lighter red accent. Do not introduce a second visual identity for dark mode.
|
||||
@@ -191,30 +211,33 @@ for their named states. Color is never the only state indicator.
|
||||
|
||||
## Typography
|
||||
|
||||
**Display Font:** Fraunces, with Source Serif Pro, Georgia, and Times New Roman fallbacks
|
||||
**Body Font:** Manrope, with native system sans-serif fallbacks
|
||||
**Label/Mono Font:** SF Mono or Cascadia Code, with Menlo and Consolas fallbacks
|
||||
**UI Font:** locally bundled Manrope Variable, with Manrope and native sans-serif fallbacks.
|
||||
**Technical Font:** SF Mono or Cascadia Code, with Menlo and Consolas fallbacks.
|
||||
|
||||
**Character:** Fraunces gives persisted artifacts and key headings editorial authority. Manrope
|
||||
keeps dense controls and prose calm and readable. The mono register separates machine identity,
|
||||
metadata, SQL, identifiers, and micro-labels from natural-language content.
|
||||
Manrope covers headings, labels, controls, navigation and document reading. Monospace is reserved
|
||||
for SQL, code, paths and machine identifiers, never for ordinary UI labels or status headings.
|
||||
|
||||
### Hierarchy
|
||||
|
||||
- **Display** (600, `3rem`, `1.03`): authentication and exceptional page-level statements only.
|
||||
- **Headline** (600, `1.875rem`, `1.15`): major page or artifact titles.
|
||||
- **Title** (600, `1.2rem`, `1.25`): panel and document section hierarchy.
|
||||
- **Body** (400, `0.9375rem`, `1.65`): operational prose, with a target line length of 65 to 75
|
||||
- **Headline** (600, `1.5rem`, `1.3`): page or artifact titles, `--text-page`.
|
||||
- **Title** (600, `1.25rem`, `1.4`): section hierarchy, `--text-section`.
|
||||
- **Body** (400, `1rem`, `1.6`): operational prose, `--text-body`, with a target line length of 65 to 75
|
||||
characters where the surface controls width.
|
||||
- **Control** (600, `0.875rem`, `1.25`): buttons, inputs, tabs, and compact actions.
|
||||
- **Label** (600, `0.6875rem`, `0.06em` tracking): uppercase micro-labels, state metadata, and panel
|
||||
headers. Labels use the mono family.
|
||||
- **Control** (400–600, `0.875rem`, `1.5`): buttons, inputs, tables, tabs and compact subheadings,
|
||||
`--text-control`.
|
||||
- **Metadata** (400–600, `0.75rem`, `1.5`): secondary status, counts and timestamps, `--text-meta`.
|
||||
Labels use sentence case and normal tracking. Ordinary operational text never falls below 12px.
|
||||
|
||||
Typography uses fixed sizes. Responsive changes happen at structural breakpoints, not through fluid
|
||||
type scaling. Numeric data and identifiers use tabular numerals where comparison matters.
|
||||
|
||||
**The Three Registers Rule.** Serif means authority, sans means interaction and reading, mono means
|
||||
machine identity. Do not exchange these roles for novelty.
|
||||
**Application wordmark:** ThothII is a brand mark, not a page title: use Manrope semibold at
|
||||
48px (`3rem`) in the Core welcome area and 32px (`2rem`) in the session sidebar, with the
|
||||
`II` suffix in brand red. Preserve these sizes across responsive layouts.
|
||||
|
||||
**The One Family Rule.** The UI and document readers use sans-serif throughout. The legacy
|
||||
`--font-heading` alias resolves to `--font-sans`. Preserve technical monospace without turning it
|
||||
into a second decorative hierarchy. Do not shrink text to solve layout constraints.
|
||||
|
||||
**The Read Once Rule.** A heading, label, and body must be distinguishable on first glance through
|
||||
size and weight. Do not repeat headings in explanatory copy.
|
||||
@@ -279,27 +302,70 @@ default, hover, focus, active, disabled, loading, and error behavior where those
|
||||
- **Focus:** three-pixel Instrument Red ring with a clear border shift.
|
||||
- **Error / Disabled:** errors combine destructive color with explanatory text; disabled controls
|
||||
retain readable contrast and use 50 percent opacity.
|
||||
- **Metadata catalog model:** Database Management keeps one compact, installation-level
|
||||
metadata-generation LLM selector in the application header. The selection persists across
|
||||
database, table, column, and relationship views; when no usable profile is configured, the
|
||||
disabled control explains: “No metadata-generation LLM model is configured for this installation.”
|
||||
- **Global context:** the collapsible top shelf is the sole workspace/model selector for Core and
|
||||
Admin. Preserve independent remembered choices, installation defaults, operation locks and unsaved
|
||||
edit guards. Never introduce a separate metadata-generation default or selector.
|
||||
|
||||
### Navigation
|
||||
|
||||
- **Workspace readiness:** the Workspace navigation button carries an 8px dot to
|
||||
the right of its label. Green means a selected workspace with confirmed ready
|
||||
preprocessing and no query error; all other states are red. The button's
|
||||
tooltip and accessible description retain the translated exact state. Do not
|
||||
add a separate readiness text row or change the backend readiness gate.
|
||||
- **Session groups:** one accessible single-open accordion contains Active sessions
|
||||
and Archive, both initially closed. Below the scope tabs, show only their
|
||||
adjacent section headers, without a redundant Sessions heading. Selection and
|
||||
bulk-delete controls belong inside each panel and only appear for nonempty
|
||||
lists. Select all affects that list only, preserves the other list's selection,
|
||||
and exposes a mixed state for partial selection. Preserve the existing archived
|
||||
flag as the grouping rule, independent of whether a Pi process is running.
|
||||
Opening a section closes the other; either can be collapsed, including both.
|
||||
Empty lists show only the translated "No sessions yet." message.
|
||||
The open section uses the rail's remaining height; its list scrolls internally
|
||||
with a cap of `min(18rem, 35dvh)`, while its trigger remains outside that scroll
|
||||
area. The mobile navigation dialog supplies a bounded viewport-height container.
|
||||
Keyboard users can focus and scroll each labelled panel.
|
||||
- **Session entry:** one Session button returns to the current unfinished session,
|
||||
including provisional creation, without resetting or reconnecting it. Otherwise
|
||||
it prepares a new question using the normal readiness and unsaved-work guards.
|
||||
- **Style:** compact session rows use `8px` corners and restrained vertical padding.
|
||||
- **Default / Hover / Active:** porcelain at rest, Sunken Surface on hover, and a muted Navigation
|
||||
Active red with a defined border when current. Exactly one top-level navigation control is current.
|
||||
- **Administrative controls:** the admin-only Administration accordion groups Database management,
|
||||
a structural divider, Workspace management, and Pi management in that order. Its trigger exposes
|
||||
- **Administrative controls:** the admin-only Administration accordion groups Database,
|
||||
Memory, Evidence, a structural divider, Workspace, and Pi configuration in that order. Its trigger exposes
|
||||
expanded state and starts collapsed by default, while non-admin users do not receive the accordion
|
||||
or its navigation actions.
|
||||
- **Responsive:** collapse navigation structurally at the application breakpoint. Do not shrink
|
||||
labels into illegibility.
|
||||
labels into illegibility. Below 768px, Memory and Evidence management use the full content
|
||||
width; a Navigation button opens the shared accessible dialog. Selecting another archive
|
||||
page or pressing Escape closes it. Desktop retains the right session sidebar and its My sessions /
|
||||
All sessions tabs. Core retains question/answer, eight phases, reviewer gates and the left log.
|
||||
In embedded mode the portal owns the red header and left sidebar; ThothII must not duplicate them. Size to the
|
||||
actual application container. Narrow session document panels may use the available width.
|
||||
|
||||
### Session review and confirmations
|
||||
|
||||
Session dialogs use the visible application area, including the portal's header
|
||||
and side rail. Artifact and schema-column review can grow to 80rem wide and the
|
||||
available height; short confirmations use up to 40rem and at least 18rem when
|
||||
space permits. Keep a 24px outer margin on desktop and 8px on small or short
|
||||
screens. Long review content scrolls internally; on very short screens the
|
||||
whole dialog can also scroll so every action remains reachable.
|
||||
|
||||
Session forms and review gates repeat their existing primary confirmation above
|
||||
and below the content, sharing selection, validation, pending state and response
|
||||
handlers. Alternate-response inputs follow the same rule. Reserved navigation
|
||||
controls remain below the review. Stop/delete initially focus Cancel; rename
|
||||
initially focuses the name field. Administration dialogs and forms retain their
|
||||
existing layout and actions.
|
||||
|
||||
### Tabs
|
||||
|
||||
- **Shape:** compact label tabs sit on a shared baseline with rounded top corners and a two-pixel
|
||||
lower edge. Inactive labels retain a complete Quiet Border and Porcelain Card surface, so every
|
||||
lower edge, except session-scope tabs which use a uniform one-pixel border, rounded
|
||||
corners and a 4px gap without a shared border or negative bottom margin.
|
||||
Inactive labels retain a Quiet Border and Porcelain Card surface, so every
|
||||
label reads as a tab before interaction; hover feedback reinforces clickability.
|
||||
- **Current:** the selected tab uses the muted Navigation Active red for its fill, text, and defined border.
|
||||
It must expose `aria-selected`, participate in a labelled `tablist`/`tabpanel`, and be the only
|
||||
@@ -319,9 +385,42 @@ default, hover, focus, active, disabled, loading, and error behavior where those
|
||||
|
||||
### Curated Evidence Documents
|
||||
|
||||
Memory and Evidence share the `thot-knowledge-reader` reading contract. Use locally
|
||||
bundled Manrope with normal tracking for prose and labels, and these fixed roles:
|
||||
|
||||
- Card title: 24px, weight 600, line-height 1.3 (`thot-knowledge-title`).
|
||||
- Field/section heading, including Scope and Provenance: 20px, weight 600,
|
||||
line-height 1.4, 8px clearance below (`thot-knowledge-heading`).
|
||||
- All narrative text, including scope, lists and provenance: 16px, weight 400,
|
||||
line-height 1.65. Do not apply compact UI text sizes to these fields.
|
||||
- Authored Markdown subheadings inside a field: 16px, weight 600, line-height 1.5,
|
||||
24px above/8px below. They remain subordinate to the enclosing field heading;
|
||||
their semantic heading levels and original content are preserved.
|
||||
- Technical metadata labels/values: 14px/1.5, with weight 600 for labels.
|
||||
Only code, paths and machine identifiers use the technical monospace family at
|
||||
14px/1.65, identical for inline and fenced code (never compound `em` shrinkage).
|
||||
|
||||
Separate reading sections by 24px; keep the first Markdown block flush with its
|
||||
field heading's 8px bottom gap. The same typography applies in light/dark and at
|
||||
all responsive widths. Controls and archive indexes retain their compact UI roles.
|
||||
|
||||
Memory and Evidence detail readers use the entire available content width, without
|
||||
the ordinary 72–75ch prose cap. This is the owner's explicit reading-layout choice.
|
||||
Long unstructured paragraphs are split for display at existing sentence/semicolon
|
||||
boundaries outside inline code and links; authored Markdown structure and stored
|
||||
content are unchanged. Paragraph spacing is 1.25em. Scope and provenance share the
|
||||
available width; provenance excerpts render Markdown rather than literal markers.
|
||||
Copy actions use the two-overlapping-sheets icon, an accessible name/tooltip and
|
||||
live success/failure feedback instead of a visible Copy label.
|
||||
|
||||
Memory has four explicitly FAKE formatting examples, one per family, in a separate
|
||||
expandable section. They reuse the real detail reader but never enter persistence,
|
||||
indexing, link search or model recall, and expose no edit/delete/save actions.
|
||||
|
||||
Curated evidence follows a fixed reading order: title, compact type and purpose summary, scope,
|
||||
typed content, supporting excerpts, review items, then collapsed technical provenance. Machine
|
||||
metadata stays in invisible comments so GitHub Preview shows only the reviewable document.
|
||||
typed content, supporting excerpts, review items, then technical provenance. Curated v4 files
|
||||
use short, visible YAML frontmatter for identity and classification. The Markdown title and
|
||||
body are authoritative; hidden payload comments are a legacy format converted on consolidation.
|
||||
|
||||
`applies_to` is rendered as “Ambito di applicazione” with separate bullet lists for concepts,
|
||||
tables, and columns. Enum values also use lists. Tables are forbidden for metadata, scope, or any
|
||||
@@ -329,7 +428,9 @@ one-dimensional collection; reserve tables for genuinely two-dimensional dataset
|
||||
identifiers use inline code. SQL uses fenced code. Supporting excerpts use blockquotes.
|
||||
|
||||
**The Review Surface Rule.** The visible Markdown must be readable without understanding the
|
||||
machine contract. Technical metadata belongs in progressive disclosure, not above the title.
|
||||
machine contract. In Administration, explain current and original provenance separately and
|
||||
keep file-editing templates and Git instructions in progressive disclosure. Show actual host
|
||||
paths with copy controls, never browser file links to container-only locations.
|
||||
|
||||
## Do's and Don'ts
|
||||
|
||||
@@ -341,7 +442,8 @@ machine contract. Technical metadata belongs in progressive disclosure, not abov
|
||||
- **Do** preserve information density with headings, rhythm, and progressive disclosure.
|
||||
- **Do** keep keyboard focus explicit and pair color with text, shape, icon, or position.
|
||||
- **Do** respect `prefers-reduced-motion` while preserving immediate non-kinetic feedback.
|
||||
- **Do** use English for interface chrome and the workspace language for persisted document content.
|
||||
- **Do** use the selected interface language (English by default) for chrome and preserve the
|
||||
workspace language for persisted domain content. Session interaction language remains pinned.
|
||||
- **Do** render curated metadata and scope as Markdown prose or lists, never as a frontmatter table.
|
||||
- **Do** break long curated rules into paragraphs, labelled subsections, and lists at existing
|
||||
punctuation boundaries while preserving the exact canonical text for machines.
|
||||
|
||||
@@ -1,309 +1,101 @@
|
||||
# ThothII — Project State
|
||||
# Project state
|
||||
|
||||
Last updated: 2026-09-06.
|
||||
Updated: 2026-09-27. This is a current snapshot, not a release diary. Stable commands
|
||||
and invariants are in [AGENTS.md](AGENTS.md); prior snapshots remain in Git.
|
||||
|
||||
This file is the short operational snapshot. Stable commands and the architecture mental model
|
||||
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
|
||||
`docs/contracts/`, `docs/adr/`, and `docs/evidence.md`. Superseded plans and reports are
|
||||
available from Git history rather than duplicated in the working tree.
|
||||
## Current contracts
|
||||
|
||||
The guarded server migration from a legacy checkout to the schema-v2 installation, Gitea source,
|
||||
Authentik, internal catalog/embedding services, and the PSD workspace repository is documented in
|
||||
`docs/operations/server-upgrade-gitea-workspace-v2.md`. Treat its operator gates and rollback
|
||||
requirements as mandatory; do not replace the running server stack in place.
|
||||
- React supports full/embedded rendering independently of local/OIDC/upstream auth,
|
||||
with EN/IT UI and immutable session interaction language. See
|
||||
[application shell](docs/architecture/application-shell.md) and
|
||||
[localization](docs/operations/shell-and-localization.md).
|
||||
- PostgreSQL Metadata Catalog owns database identity, binding, schema, descriptions,
|
||||
sensitivity and relationships for all core consumers. Workspace schema v4 contains
|
||||
identity and optional Evidence only. Installation schema v2 is the authored model
|
||||
catalog source. See [overview](docs/architecture/overview.md) and
|
||||
[model configuration](docs/general/pi-configuration.md).
|
||||
- The harness owns workflow persistence; chat is not the durable session record.
|
||||
Memory uses PostgreSQL authority and derived Qdrant dense/BM25 search. Editable
|
||||
Evidence has local archive authority and manual consolidation. File save, search
|
||||
activation and Git publication have distinct outcomes. See
|
||||
[Memory](docs/gestione-memory.md), [Evidence](docs/contracts/curated-evidence-v4.md)
|
||||
and [consolidated release evidence](docs/reports/knowledge-archives-release.md).
|
||||
- Reference preprocessing must not clear Memory. Use the installation-scoped
|
||||
`tht --installation /absolute/path/thothii-installation.yaml workspace preprocess run`
|
||||
and its [contract](docs/contracts/workspace-preprocessing-cli.md).
|
||||
- DWH sessions are read-only. SSH tunnels support database-management diagnostics
|
||||
and metadata synchronization, not NL→SQL session creation; use direct or REST
|
||||
transport for sessions.
|
||||
|
||||
## Current product shape
|
||||
## Installation and workspace boundaries
|
||||
|
||||
ThothII is a human-in-the-loop datamart builder with three independently built layers:
|
||||
Fresh standalone installations follow the manual terminal procedures in
|
||||
[Italian](docs/install/standalone-manual-it.md) or
|
||||
[English](docs/install/standalone-manual-en.md), without an installer or launcher.
|
||||
The [Compose reference](docs/operations/compose-reference.md) is for maintainers,
|
||||
not another quick start. Catalog and Memory migrations are explicit.
|
||||
|
||||
```text
|
||||
frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness → DWH (read-only)
|
||||
```
|
||||
Descriptors, authentication, provider credentials, certificates and runtime bindings
|
||||
stay in protected installation-local paths. Do not copy secrets into examples or
|
||||
workspace Git. Legacy runtime snapshots may use absolute paths and protected
|
||||
`harness/.env`; do not silently relocate them.
|
||||
|
||||
The harness owns the deterministic eight-phase NL→SQL workflow and all session persistence.
|
||||
The backend remains a process/RPC/SSE bridge for sessions and now also owns an isolated PostgreSQL
|
||||
metadata catalog for administrative database configuration. The frontend renders the review gates
|
||||
and keeps the live transcript in memory. See
|
||||
`docs/architecture/components.md` for the detailed component and data-flow map.
|
||||
PSD authoring is separate at `/Users/mp/projects/tht-workspace-psd`. Its GitHub
|
||||
repository was copied to private Gitea
|
||||
[workspace_psd](https://git.tylconsulting.it/mptyl/workspace_psd), preserving both
|
||||
branches. It is a copy, not automatic synchronization. Running installations were
|
||||
not repointed to a different workspace remote.
|
||||
|
||||
## Evidence restructuring — accepted
|
||||
## Recorded deployments and server authority
|
||||
|
||||
The evidence restructuring and PSD migration completed real acceptance on 2026-08-25.
|
||||
Use the ordered [server handoff](docs/operations/server-codex-handoff.md) for
|
||||
coordinated ThothII/Omics upgrades. Omics integration uses GitHub
|
||||
`Dallavilla-Tiziano/omics_portal`, with no Gitea relay prerequisite. Omics uses
|
||||
embedded/upstream identity, not another ThothII OIDC login. Read
|
||||
[upstream authentication](docs/install/authentication-upstream.md) before changes.
|
||||
|
||||
- The curated PSD revision contains 35 approved Evidence units and 60 review items.
|
||||
- The PSD authoring repository publishes all 35 units using Curated unit schema v3. Its table-free
|
||||
presentation uses hidden canonical metadata, wrapping Markdown scope lists, list-based enum
|
||||
values, and collapsed technical provenance. Long domain rules now have a deterministic
|
||||
human-readable presentation while retaining their exact canonical text for vector ingestion.
|
||||
`tht evidence migrate <workspace-root>` performs the deterministic v1/v2 upgrade and older-v3
|
||||
presentation rewrite without model calls. The structured-rule PSD rewrite is currently local and
|
||||
pending commit/publication.
|
||||
- The accepted snapshot is
|
||||
`psd-clinical-675990d90eae51da6f2bd51b1ae2609f245772ef-snapshot`.
|
||||
- The active generation is `gen:f968b3bd7a553dbfef3cf47093698f2bc7f95f11`.
|
||||
- Retrieval acceptance reached 20/20 Hit@10.
|
||||
- A real session, `20301df7-cad7-403d-a4c1-9f35c9d07b66`, completed F1–F8 with five
|
||||
receipts, three CTEs, and a final result of 78 patients.
|
||||
- The durable acceptance record is
|
||||
`docs/testing/evidence/evidence-restructuring-psd-acceptance-2026-08-25.md`.
|
||||
Last recorded application deliveries (not a fresh runtime attestation):
|
||||
|
||||
The canonical authoring, validation, publication, materialization, and preprocessing flow is
|
||||
documented in `docs/evidence.md`. The governing contracts are
|
||||
`docs/contracts/workspace-evidence-v3.md` and
|
||||
`docs/contracts/workspace-preprocessing-cli.md`.
|
||||
The incremental server procedure for the `260906-preprocessing-complete` release is
|
||||
`docs/operations/server-handoff-260906-preprocessing-complete.md`.
|
||||
- [Coordinated ThothII/Omics release](docs/reports/2026-09-26-server-release-execution.md):
|
||||
core/frontend `497ab840-preflight`, Omics proxy fix `928f7e9f` with existing web
|
||||
image retained. Automated acceptance passed; on September 27 the operator confirmed
|
||||
browser access, UI controls and session start/stop/resume. Functional browser
|
||||
acceptance passed; remaining extended checks are handed off in the
|
||||
[server acceptance follow-up](docs/reports/2026-09-27-server-acceptance-handoff.md).
|
||||
- [Session dialogs](docs/reports/2026-09-14-session-dialogs-release.md):
|
||||
`b1723c34-session-dialogs-20260914`, frontend-only.
|
||||
- [Session layout/Memory fix](docs/reports/2026-09-14-session-layout-memory-fix.md):
|
||||
`49333a2d-session-memory-fix`, core/frontend.
|
||||
|
||||
## Workspace preprocessing and configuration
|
||||
Keep those reports and rollback instructions while operator gates remain open.
|
||||
A later deployment does not prove every earlier acceptance item passed.
|
||||
|
||||
The native host CLI `tht` is the operator surface. Workspace preprocessing runs through:
|
||||
## Remaining acceptance and design gates
|
||||
|
||||
```sh
|
||||
tht --installation /absolute/path/thothii-installation.yaml \
|
||||
workspace preprocess run --workspace <workspace-id>
|
||||
```
|
||||
- Fresh-machine Mac, Windows/WSL2 and Linux installation acceptance, including real
|
||||
DWH/model endpoints, remains a separate operator exercise.
|
||||
- Real IdP/portal login, logout, embedded interaction and PSD semantic acceptance
|
||||
follow the [manual matrix](docs/testing/authentication-manual-acceptance.md) and
|
||||
delivery reports; synthetic tests do not close them.
|
||||
- [Security hardening](docs/plans/2026-09-08-security-hardening-prd.md) is a draft.
|
||||
Revalidate SEC01–12 and obtain design approval before implementation or real
|
||||
server/IdP/DWH mutation.
|
||||
- Optional NER remains opt-in; labeled Italian quality, benchmark and licensing
|
||||
acceptance are not implied by document cleanup.
|
||||
- Semantic aliases, value descriptions, synonyms/concepts, dialect and multi-schema
|
||||
extensions remain explicit design work. Current sensitivity delivery follows the
|
||||
Catalog contract; additional policies require their own acceptance.
|
||||
- Legacy database UI fallback (`?db-ui=legacy`, dev/staging) and prototype removal
|
||||
remain subject to owner acceptance.
|
||||
|
||||
This complete one-shot command uses the profile-gated `workspace-maintenance` service. Partial DWH,
|
||||
schema, Evidence, and vector mutation commands are retired.
|
||||
## Documentation maintenance
|
||||
|
||||
The right Administration sidebar invokes that same operation for the selected workspace. It shows
|
||||
only current readiness or the latest bounded failure diagnostic; there is no preprocessing history.
|
||||
Known non-ready state disables **New session**, while backend admission remains authoritative.
|
||||
The same control exposes an inline-confirmed **Clear** action to remove replaceable reference
|
||||
vectors, LSH, corpus, and checkpoints while preserving the separate Memory collection. The host CLI
|
||||
equivalent is `workspace preprocess clear`.
|
||||
MkDocs publishes only 20 product/operator pages and five approved assets.
|
||||
Architecture, contracts, ADRs, plans, research, tests and release evidence are
|
||||
excluded from HTML and search. The repository itself is public: editorial exclusion
|
||||
is not confidentiality.
|
||||
|
||||
Each workspace now uses `<workspace>-reference` for Schema, relationships, and Evidence and
|
||||
`<workspace>-memory` for `memory` and `solved_question`. Clear and preprocessing own only the former.
|
||||
LSH ownership additionally binds the Catalog database ID and Metadata Content Revision, so derived
|
||||
values cannot be reused across database identities or Catalog revisions.
|
||||
|
||||
Workspace descriptors use schema v4 and contain only workspace identity and optional Evidence.
|
||||
PostgreSQL Metadata Catalog owns database identity, binding, schema, descriptions, sensitivity, and
|
||||
relationships; model, provider, embedding, and vector-store configuration is installation-owned. For
|
||||
PSD, workspace content and runtime roots point to the
|
||||
separate uncommitted repository `/Users/mp/projects/tht-workspace-psd`. Secrets remain outside
|
||||
Git and are supplied only through installation-local protected files.
|
||||
|
||||
## Installation Model Catalog
|
||||
|
||||
`thothii-installation.yaml` schema version 2 is the only operator-authored source for session,
|
||||
metadata-generation, and embedding models. The host `tht` lifecycle validates `modelCatalog` and
|
||||
regenerates the backend catalog, Pi `models.json`/`settings.json`, and Compose override under the
|
||||
installation-local `generated/` directory. Those projections are replaceable runtime adapters:
|
||||
they are not edited, backed up, or treated as configuration.
|
||||
|
||||
Session and metadata defaults use canonical `provider/model` IDs. Provider authentication declares
|
||||
one explicit mode (`secret_env`, `pi_auth`, or `none`); `secret_env` names a protected bundle key.
|
||||
The backend settings store now owns only the selected workspace and thinking level. Existing v1
|
||||
installations use the explicit catalog migration command; schema-v3 workspace descriptors are
|
||||
converted deterministically in their curator-owned repository before commit. Strict runtime loading
|
||||
does not silently infer or merge legacy sources. ADR 0013 and
|
||||
`docs/plans/2026-09-02-installation-model-catalog.md` record the decision and implementation.
|
||||
|
||||
## Database management
|
||||
|
||||
The database, table, and authoritative physical-schema catalog slices are implemented. Database
|
||||
Management now opens the Fleet Ledger presentation by default inside `AppShell`, lists every YAML
|
||||
workspace, creates at most one PostgreSQL database configuration per workspace, edits direct
|
||||
PostgreSQL, REST API, or SSH-tunnel installation bindings, replaces write-only encrypted secrets,
|
||||
and tests supported connector bindings. The surface keeps one responsive AG Grid visible at a time:
|
||||
databases lead to tables, tables lead to columns, and relationships are a sibling database view.
|
||||
Parent navigation remains explicit through the breadcrumb and emphasized back control.
|
||||
|
||||
Selection-scoped operations use one action selector plus an explicit **Run** control; ineligible
|
||||
actions remain visible with their disabled reason, while row-scoped actions stay in the pinned final
|
||||
column. The KPI strip reads installation-wide or selected-database aggregates from
|
||||
`GET /catalog/metrics`. Database configuration, metadata editors, synchronization history,
|
||||
description history, and sensitive-field review/history use the production APIs in right-side
|
||||
drawers rather than prototype fixtures; closing a history drawer does not stop its background run.
|
||||
|
||||
Sensitive-field review is now driven by the versioned local `sensitivity-v4` policy, not by a
|
||||
catalog model. The backend reads selected source tables through read-only, database-specific
|
||||
adapters and makes every `sensitive | non_sensitive` draft decision in the TypeScript
|
||||
`SensitivityClassifier`. A single validated match protects the column. Tables up to 1,000 rows are
|
||||
fully scanned; larger tables use breadth-first 300, 1,000, and text-only 3,000-value targets, with a
|
||||
five-second limit per source query and no global request deadline. Source failures fail the run
|
||||
instead of yielding `unknown`; coverage remains visible separately from the proposal. Draft
|
||||
assessments remain transient until an administrator explicitly saves them. Optional GLiNER2
|
||||
evidence is CPU-only, offline, opt-in, and never replaces the deterministic decision point; see
|
||||
`docs/operations/sensitivity-analysis.md`. The earlier v1 PSD shadow comparison kept NER disabled by
|
||||
default; see `docs/reports/2026-09-02-psd-sensitivity-shadow.md`. The v2 comparison completed all
|
||||
2,275 columns: CPU NER added 18 sensitive proposals and increased warm runtime from 50.1 to 61.3
|
||||
seconds; see `docs/reports/2026-09-03-psd-progressive-sensitivity-shadow.md`.
|
||||
Version 4 excludes declared `bigint` primary-key columns and conventionally named `pk bigint`
|
||||
columns before source inspection, reporting both as non-informative structural identifiers while
|
||||
distinguishing declared constraints from inferred roles.
|
||||
|
||||
Physical membership, source
|
||||
comments, column types/default/nullability/PK positions, and constraint-level ordered FK pairs are
|
||||
projections of the external schema. They cannot be created, renamed, or structurally edited by
|
||||
hand, but administrators can explicitly clear catalog tables, columns, or relationships without
|
||||
touching the source database, binding, configuration, or secrets. Table deletion cascades through
|
||||
columns and relationships; table-scoped relationship cleanup includes incoming and outgoing
|
||||
relationships. Curated and generated descriptions are editable; generated descriptions start null
|
||||
and Database Management can generate or consolidate them for selected tables, selected columns,
|
||||
all targets, or only targets whose Generated Description is missing.
|
||||
|
||||
Relationship Management is now reachable directly from each configured Fleet database. One
|
||||
Relationship Map shows read-only Physical Relationships together with Generated and Manual Logical
|
||||
Relationships, with Active, Excluded, and All filters. Administrators can add a single-column
|
||||
relationship, run deterministic name/PK/type inference, exclude or restore a logical relationship,
|
||||
or delete it permanently. Exclusion retains a tombstone that a rebuild cannot reactivate; permanent
|
||||
deletion allows a later rebuild to infer the same endpoints again. Inference uses no LLM, embedding,
|
||||
or source values. It supports normalized table-qualified names, unique non-generic PK names,
|
||||
composite-PK source columns, and the `*time_key -> dim_time.<single PK>` warehouse convention while
|
||||
ignoring bare generic names. Explicit table/column metadata cleanup remains a destructive boundary: it removes
|
||||
the attached logical relationships and exclusions and requires a full schema synchronization before
|
||||
inference or runtime publication can continue.
|
||||
|
||||
The previous Database Management renderer remains a temporary comparison fallback for development
|
||||
and staging only: `?db-ui=legacy` is honored in Vite development or when
|
||||
`VITE_DB_MANAGEMENT_LEGACY=true`; it is not a production presentation. The standalone Fleet Ledger
|
||||
prototype on port `5173` also remains temporary until owner acceptance of the integrated surface,
|
||||
after which both migration aids can be removed.
|
||||
|
||||
Schema refresh is one durable asynchronous engine with database-table, selected-table-column,
|
||||
relationship, and full-database actions. Database-level menus expose only the table, relationship,
|
||||
and full scopes; selecting tables exposes column synchronization plus manual column and relationship cleanup for that subset. Database selections
|
||||
also expose manual table and relationship cleanup. Cleanup selections are atomic and share the
|
||||
one-active-operation-per-database exclusion with synchronization. Runs have leases and
|
||||
restart recovery, atomic apply, destructive-diff confirmation with re-scan, cancellation before
|
||||
apply, retained history, and a live SSE log with polling fallback. Null metadata renders blank
|
||||
rather than as a placeholder.
|
||||
|
||||
Direct PostgreSQL and strict known-host-verified OpenSSH use `pg_catalog`. REST bindings use the
|
||||
typed full-snapshot `POST /rpc/schema_snapshot` contract when available. Servers such as the
|
||||
current PSD endpoint that exposes only `POST /rpc/run_query` use one catalog-owned read-only query
|
||||
to return the exact same strict v1 snapshot in a single round trip. Both paths remain fail-closed:
|
||||
an absent capability, query error, partial result, or invalid snapshot applies no catalog changes.
|
||||
SSH is not yet enabled for NL→SQL session runtime.
|
||||
|
||||
The catalog runs in the internal `catalog-db` PostgreSQL service. Kysely migrations are an explicit
|
||||
one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before local startup. Runtime
|
||||
sessions consume an immutable Catalog JSON snapshot tied to the runtime-config lease. It contains
|
||||
the tables, columns, effective descriptions, sensitivity flags, and active relationships used by
|
||||
the harness; PostgreSQL is the exclusive runtime authority for database metadata. Authored
|
||||
workspace YAML remains limited to workspace identity and optional Evidence configuration. The
|
||||
accepted design is recorded in ADR 0016 and the contracts under `docs/contracts/`.
|
||||
|
||||
Semantic aliases, value descriptions, synonyms, and concepts remain deferred to their dedicated
|
||||
slices.
|
||||
|
||||
AI Description Generation uses the catalog's human-owned Sensitive Data Flag. The flag defaults to
|
||||
`false`, including for newly synchronized columns. An administrator may request a local sensitivity
|
||||
analysis for one selected database, selected tables, or selected columns. One deterministic
|
||||
TypeScript classifier combines metadata, bounded source-content rules, and optional CPU-only NER;
|
||||
no generative model decides the result. Its `sensitive` or `non_sensitive` assessments remain an
|
||||
unsaved draft until the human reviews and saves any chosen flag changes, including a downgrade to
|
||||
non-sensitive. Coverage is reported separately; interrupted history may count unprocessed columns.
|
||||
Each started analysis records a separate Sensitivity Analysis Run with aggregate counters and safe
|
||||
ordered events. The progress drawer opens before the synchronous request completes, polls the run,
|
||||
and displays sanitized source-scan and local-NER phase/batch activity while classification is in
|
||||
progress. This operational history never stores per-column assessments, source values,
|
||||
matched spans, prompts, or free-form diagnostics. Saving a sensitive decision persists a sanitized
|
||||
Sensitivity Reason as column Catalog Metadata alongside the human-owned flag; clearing the flag
|
||||
clears that reason. Reloading still discards an unsaved review draft.
|
||||
For unprotected columns, up to five source rows and five representative non-null values may be sent
|
||||
transiently to the configured model provider. Protected columns are omitted from source reads and
|
||||
replaced in the prompt by deterministic plausible values derived only from their metadata. Existing
|
||||
descriptions are not regenerated when a flag changes.
|
||||
|
||||
The accepted AI-description design is recorded in
|
||||
`docs/plans/2026-08-28-ai-catalog-description-generation.md`, with the formal specification in the
|
||||
adjacent `-spec.md` document and Gitea issue #4. Gitea issues #5–#11 deliver the implementation.
|
||||
The runtime deliberately keeps ThothAI's simple operating model: one installation-wide sequential
|
||||
run owned by the backend, one short-lived Python/LiteLLM completion helper per request, and
|
||||
persistence limited to the run, its safe ordered text events, and each Generated Description as
|
||||
soon as it succeeds. The helper performs at most one provider retry and never falls back to another
|
||||
model. Stop terminates the current helper and retains prior results; three consecutive exhausted
|
||||
technical batches fail the run. Startup marks stale queued/running work interrupted, and Unlock is
|
||||
available only when no local start, worker, or helper is live. Runs remain inspectable through a
|
||||
live SSE log with ordered polling fallback; there is no automatic resume or user-facing generation
|
||||
CLI. ADRs 0009–0010 record the runtime and source-sampling decisions.
|
||||
|
||||
The Installation Model Catalog accepts the protected `DEEPSEEK_API_KEY` and `ZAI_API_KEY`
|
||||
references for metadata-generation providers.
|
||||
It also accepts a model with no secret reference only when its OpenAI-compatible endpoint is
|
||||
explicit; this covers the VPN-only AritmoLab Qwen 3.6 server without creating a fake operator
|
||||
credential. The Python client supplies only its fixed non-secret compatibility placeholder.
|
||||
The AritmoLab entry also sets `disableThinking: true`, mapped to the endpoint's chat-template flag,
|
||||
because its default reasoning prose would violate the worker's exact JSON response contract.
|
||||
|
||||
Logical relationship integration with core schema-linking is complete: session creation and resume
|
||||
materialize the active physical/generated/manual map, retrieval-pack generation and Pi receive the
|
||||
same runtime config, and snapshot validation fails closed on a declared missing, invalid, or orphaned
|
||||
endpoint. Broader publication of other Catalog metadata to schema-linking remains a separate future
|
||||
slice.
|
||||
|
||||
**Deferred follow-up — Sensitive Data Policy in schema-linking.** The policy is first delivered
|
||||
and tested in catalog description generation. Its enforcement for core schema-linking remains
|
||||
out of scope until the current tickets are closed and the owner has completed the acceptance test.
|
||||
At that gate, resume the design: `tht` must receive a read-only projection of the current Sensitive
|
||||
Data Flags and exclude values from columns marked sensitive from every LSH result before it is
|
||||
given to Pi. Do not start this integration before the owner gives final approval after that test.
|
||||
|
||||
## Active deployment work and manual gates
|
||||
|
||||
### PSD server deployment program
|
||||
|
||||
The approved design and executable entry point are:
|
||||
|
||||
- `docs/plans/2026-08-20-psd-server-deployment-program-design.md`
|
||||
- `docs/plans/2026-08-20-psd-server-deployment-program.md`
|
||||
- `docs/plans/2026-08-20-psd-server-survey.md`
|
||||
- `docs/plans/2026-08-20-psd-server-project-a-standalone.md`
|
||||
- `docs/plans/2026-08-20-psd-server-project-b-authentik.md`
|
||||
|
||||
Last recorded state:
|
||||
|
||||
- survey: `SURVEY_NO_GO`;
|
||||
- Project A: `BLOCKED_BY_SURVEY_AND_MUTATION_GATE`;
|
||||
- Project B: `BLOCKED_BY_PROJECT_A_AND_PRE_B_GATE`.
|
||||
|
||||
The deployment is a clean replacement: legacy sessions, indexes, and application configuration
|
||||
are not migration inputs. The existing stack remains intact until its documented mutation and
|
||||
rollback gates are explicitly approved. Shared Omics/LocalLLM networks, ETL Evidence, DWH,
|
||||
`dwh-auth`, Supabase, Authentik, Superset, and Aritmolab are outside cleanup scope.
|
||||
|
||||
Human acceptance guides and sanitized report templates live under `docs/testing/` and
|
||||
`docs/testing/evidence/`. The remediation checklist is
|
||||
`docs/operations/psd-server-survey-remediation-checklist.md`.
|
||||
|
||||
### Authentication
|
||||
|
||||
The local/OIDC authentication remediation passed its automated review on 2026-08-18. Release and
|
||||
PSD mutation gates remain governed by:
|
||||
|
||||
- `docs/architecture/authentication.md`;
|
||||
- `docs/plans/2026-08-18-thothii-authentication-acceptance-and-psd-deployment.md`;
|
||||
- `docs/operations/psd-dwh-auth-rollout.md`;
|
||||
- `docs/testing/authentication-manual-acceptance.md`.
|
||||
|
||||
Do not infer authorization for server, Nginx, Authentik, database, credential, or cutover changes
|
||||
from an automated PASS.
|
||||
|
||||
## Verification status
|
||||
|
||||
- The P1.1 workspace-directory registry and P2–P6 preprocessing workstreams are implemented and
|
||||
have automated coverage.
|
||||
- Evidence restructuring has a real PSD acceptance PASS as recorded above.
|
||||
- AI Description Generation has automated coverage across installation setup, model selection,
|
||||
generation/consolidation scopes, bounded sampling, cancellation/recovery, history, SSE/polling,
|
||||
and the LiteLLM helper boundary.
|
||||
- L2 tests requiring real providers or remote databases remain opt-in.
|
||||
- Server deployment, release, and owner-operated acceptance steps remain pending wherever the
|
||||
referenced runbooks require explicit approval.
|
||||
|
||||
Run the layer-specific checks documented in `AGENTS.md`. For release-sensitive changes, also run
|
||||
the repository contract scripts in `scripts/` and build the MkDocs site.
|
||||
|
||||
## Operational invariants
|
||||
|
||||
- `tht`'s `-c`/`--config` option follows the subcommand; it is not a global option.
|
||||
- `--json` commands write pristine JSON to stdout.
|
||||
- Persisted phase documents and the decision ledger are the source of session truth; chat is not.
|
||||
- UI chrome is English; workspace document content retains the workspace language.
|
||||
- The backend refuses resume for finalized or archived sessions.
|
||||
- A resume must send `/riprendi-sessione <id>`; a new session must send `/nuova-domanda`.
|
||||
- DWH access is read-only.
|
||||
The [cleanup record](docs/maintenance/2026-09-15-documentation-cleanup.md) records
|
||||
retired sources and retained gates. Main contains source; Actions generates the
|
||||
`pages` branch. The live site requires the separate explicit deployment described
|
||||
in [public manual publication](docs/operations/public-docs-publication.md).
|
||||
|
||||
@@ -4,72 +4,51 @@ ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fasti
|
||||
core. The portable deployment runs two application services plus the installation-local metadata
|
||||
catalog; DWH and LLM services remain external. Semantic services are bundled in Compose.
|
||||
|
||||
Authentication is configured through the single host CLI tht: see the [local authentication guide](docs/install/authentication-local.md),
|
||||
[generic OIDC guide](docs/install/authentication-oidc.md), and [manual acceptance matrix](docs/testing/authentication-manual-acceptance.md).
|
||||
The same frontend supports **full** (its own header) and **embedded** (inside a
|
||||
portal). This choice is independent of authentication: the Mac uses full/local,
|
||||
Omics uses embedded/upstream with its existing login, and a standalone server
|
||||
can use full/OIDC. See [rendering architecture](docs/architecture/application-shell.md)
|
||||
and [configuration, Omics delivery and deploy](docs/operations/shell-and-localization.md).
|
||||
|
||||
## Docker Compose: local startup
|
||||
For the current server upgrade with Omics Portal, follow the ordered
|
||||
[Codex server handoff](docs/operations/server-codex-handoff.md), including source
|
||||
integration, embedded/upstream configuration, coordinated rollout and rollback.
|
||||
|
||||
Requirements: Docker Engine with Compose v2. The mandatory stack is `frontend`, `core`,
|
||||
`catalog-db`, `qdrant`, `embedding`, and the one-shot `embedding-model-init`. DWH and LLM remain external,
|
||||
configurable endpoints—even when they are co-located with ThothII.
|
||||
Local/OIDC authentication is configured through the host CLI `tht`; portal
|
||||
authentication is established by the trusted server proxy. See the
|
||||
[local guide](docs/install/authentication-local.md),
|
||||
[OIDC guide](docs/install/authentication-oidc.md),
|
||||
[upstream integration](docs/install/authentication-upstream.md), and
|
||||
[manual acceptance matrix](docs/testing/authentication-manual-acceptance.md).
|
||||
|
||||
From a fresh clone, run these commands from the repository root:
|
||||
For the clone-based manual standalone installation test on macOS, Windows, and Linux, use the
|
||||
[Italian procedure](docs/install/standalone-manual-it.md) or the
|
||||
[English procedure](docs/install/standalone-manual-en.md).
|
||||
|
||||
```sh
|
||||
cp deploy/env/local.env.example deploy/env/local.env
|
||||
# Copy docs/install/examples/thothii-installation.local.yaml to a protected operator path,
|
||||
# replace its placeholders, chmod it 600, and set that exact THT_INSTALLATION_CONFIG_SOURCE.
|
||||
# Edit deploy/env/local.env, including PI_AUTH_FILE, THT_SECRETS_FILE, and external endpoints.
|
||||
./scripts/run-stack.sh
|
||||
```
|
||||
The [public manual](https://git.tylconsulting.it/thothii-docs/) covers the product,
|
||||
installation, use and administration. Developer architecture, contracts, ADRs, tests,
|
||||
plans and release records remain in this repository but are excluded from MkDocs
|
||||
pages and search. This is an editorial boundary, not an access restriction on the
|
||||
public repository. See the [documentation cleanup review](docs/maintenance/2026-09-15-documentation-cleanup.md)
|
||||
for the executed consolidation and the inventory of historical sources retained in Git.
|
||||
|
||||
The launcher builds the core, starts `catalog-db`, runs the explicit one-shot Kysely migrations,
|
||||
then runs the base+local stack in the foreground. Migrations never run implicitly in backend
|
||||
startup. The core image contains its Pi runtime; no host `pi` executable is used. For a server
|
||||
installation, build the image, start the catalog, and run the same migration service before the
|
||||
application rollout:
|
||||
## Docker Compose and installation
|
||||
|
||||
```sh
|
||||
cp deploy/env/server.env.example deploy/env/server.env
|
||||
# Prepare a mode-600 thothii-installation.yaml from the server example and set its exact
|
||||
# path as THT_INSTALLATION_CONFIG_SOURCE. Edit all remaining storage/secret/endpoint paths.
|
||||
sudo scripts/prepare-server-pi-state.sh /srv/thothii/pi-state 10001 10001
|
||||
docker compose --env-file deploy/env/server.env \
|
||||
-f compose.yaml -f deploy/compose.server.yaml \
|
||||
-f deploy/compose.session-server.yaml.example build core
|
||||
docker compose --env-file deploy/env/server.env \
|
||||
-f compose.yaml -f deploy/compose.server.yaml \
|
||||
-f deploy/compose.session-server.yaml.example up -d catalog-db
|
||||
docker compose --env-file deploy/env/server.env \
|
||||
-f compose.yaml -f deploy/compose.server.yaml \
|
||||
-f deploy/compose.session-server.yaml.example run --rm catalog-migrate
|
||||
docker compose --env-file deploy/env/server.env \
|
||||
-f compose.yaml -f deploy/compose.server.yaml \
|
||||
-f deploy/compose.session-server.yaml.example up --build -d
|
||||
```
|
||||
For a fresh installation, follow the complete manual procedure in
|
||||
[Italian](docs/install/standalone-manual-it.md) or
|
||||
[English](docs/install/standalone-manual-en.md). Configure protected files first;
|
||||
then run the documented build, explicit migrations and startup commands with the
|
||||
same installation descriptor and Compose project. There is no installer or launcher.
|
||||
|
||||
The initializer is required for an empty or restored server Pi-state bind. It atomically creates
|
||||
the three regular targets hidden below the writable parent bind; protected Pi auth and tracked
|
||||
model/settings sources remain separate read-only mounts. See the server manual before substituting
|
||||
a root other than `/srv/thothii/pi-state`.
|
||||
The mandatory stack includes frontend, core, PostgreSQL catalog, Qdrant, Ollama
|
||||
and the embedding initializer. DWH and LLM endpoints remain external dependencies.
|
||||
Pi is included in the core image. Credentials and certificates belong in protected
|
||||
installation-local files, never in the workspace repository.
|
||||
|
||||
Workspace descriptors come from the Git remote configured by `THT_WORKSPACE_GIT_REMOTE`; their
|
||||
runtime endpoint and secret bindings remain installation-local. Open
|
||||
<http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in `deploy/env/local.env` to choose another
|
||||
loopback port).
|
||||
|
||||
Credentials and certificates are local protected files. Do not put them in environment examples,
|
||||
workspace YAML, URLs, or Compose interpolation values.
|
||||
|
||||
Application state is split across the named `settings`, `pi-state`, `workspace-registry`,
|
||||
`sessions`, `qdrant-data`, and `embedding-models` volumes. `docker compose down` keeps them.
|
||||
`qdrant-data` is a derived but persistent index store; `embedding-models` is an Ollama model
|
||||
cache for `qwen3-embedding:0.6b` with fixed `1024`-dimension embeddings. Only an explicit destructive command such as `docker compose
|
||||
down --volumes` removes them.
|
||||
|
||||
The frontend depends on the core health check and proxies `/health` and `/api/*` to it. The
|
||||
application health endpoint intentionally checks process readiness only; external dependency
|
||||
diagnostics are exposed by `tht doctor` and do not prevent the UI from starting.
|
||||
For developer topology, overlays and lifecycle details, see the internal
|
||||
[Compose reference](docs/operations/compose-reference.md). Ordinary stop/down keeps
|
||||
persistent data; removing volumes is destructive and is not an upgrade step.
|
||||
Process health is distinct from external dependency checks performed by doctor.
|
||||
|
||||
## Git-backed workspace repository
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ import { fileURLToPath } from "node:url";
|
||||
import { tmpdir } from "node:os";
|
||||
import type { AppConfig } from "./config.js";
|
||||
import { ThtRunner } from "./tht/tht-runner.js";
|
||||
import { createMemoryCleanup } from "./catalog/memory-cleanup.js";
|
||||
import { PiProcessManager } from "./pi/pi-process-manager.js";
|
||||
import { SseHub } from "./sse/sse-hub.js";
|
||||
import { authenticateSession, captureAuthConfigSnapshot, configuredOrigin } from "./auth/auth.js";
|
||||
@@ -80,6 +81,8 @@ import { createProductionWorkspacePreprocessingService } from "./workspace-maint
|
||||
import type { WorkspacePreprocessingService } from "./workspaces/preprocessing-service.js";
|
||||
import { PreprocessingStateStore } from "./workspaces/preprocessing-state.js";
|
||||
import { workspacePreprocessingRoutes } from "./routes/workspace-preprocessing.js";
|
||||
import { memoryRoutes } from "./routes/memory.js";
|
||||
import { evidenceRoutes } from "./routes/evidence.js";
|
||||
|
||||
export interface BuildAppDeps {
|
||||
thtRunner?: ThtRunner;
|
||||
@@ -93,7 +96,7 @@ export interface BuildAppDeps {
|
||||
workspaceDiagnoser?: WorkspaceDiagnoser;
|
||||
workspaceDatabaseTester?: WorkspaceDatabaseTester;
|
||||
workspaceSecretStore?: WorkspaceSecretStore;
|
||||
workspacePreprocessingService?: Pick<WorkspacePreprocessingService, "run" | "clear">;
|
||||
workspacePreprocessingService?: Pick<WorkspacePreprocessingService, "run" | "clear"> & Partial<Pick<WorkspacePreprocessingService, "consolidateEvidence" | "evidenceSources">>;
|
||||
catalogRepository?: CatalogRepository;
|
||||
catalogService?: CatalogService;
|
||||
catalogPostgresAccess?: CatalogPostgresAccess;
|
||||
@@ -265,6 +268,11 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
catalogSchemaIntrospector,
|
||||
catalogOperationCoordinator,
|
||||
config.catalogSyncTimeoutMs,
|
||||
createMemoryCleanup(tht as ThtRunner, {
|
||||
internalQdrantUrl: config.internalQdrantUrl, internalEmbeddingUrl: config.internalEmbeddingUrl,
|
||||
internalEmbeddingId: config.internalEmbeddingId, internalEmbeddingModel: config.internalEmbeddingModel,
|
||||
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
|
||||
}),
|
||||
);
|
||||
app.addHook("onReady", async () => { await catalogSyncWorker.initialize(); });
|
||||
app.addHook("onReady", async () => { await descriptionGenerationWorker.initialize(); });
|
||||
@@ -493,7 +501,16 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
return maintenanceBarrier.status();
|
||||
});
|
||||
sqlRoutes(app, { tht: tht as ThtRunner, getSettings, workspaceRegistry });
|
||||
memoryRoutes(app, { runner: tht as ThtRunner, registry: workspaceRegistry, runtime: {
|
||||
internalQdrantUrl: config.internalQdrantUrl,
|
||||
internalEmbeddingUrl: config.internalEmbeddingUrl,
|
||||
internalEmbeddingModel: config.internalEmbeddingModel,
|
||||
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
|
||||
} });
|
||||
metaRoutes(app, { harnessDir: config.harnessDir, modelCatalog: runtimeModelCatalog });
|
||||
evidenceRoutes(app, { runner: tht as ThtRunner, registry: workspaceRegistry,
|
||||
registryRoot: config.workspaceRegistry.root, hostRegistryRoot: config.evidenceHostRegistryRoot,
|
||||
service: workspacePreprocessingService });
|
||||
workspaceRoutes(app, {
|
||||
registry: workspaceRegistry,
|
||||
config: config.workspaceRegistry,
|
||||
|
||||
@@ -119,7 +119,9 @@ export function authenticateSession(deps: AuthDependencies): preHandlerHookHandl
|
||||
subject: session.subject,
|
||||
...(session.displayName === undefined ? {} : { displayName: session.displayName }),
|
||||
roles: session.roles,
|
||||
permissions: session.permissions,
|
||||
// Sessions can outlive a deployment that changes the role permission catalog.
|
||||
// resolve() has already checked validity, including current local user roles.
|
||||
permissions: rolesToPermissions(session.roles),
|
||||
isAdmin: session.roles.includes("admin"),
|
||||
};
|
||||
if (STATE_CHANGING_METHODS.has(request.method)) {
|
||||
|
||||
@@ -38,7 +38,7 @@ const MAX_MAPPED_GROUPS = 128;
|
||||
const ROLES = ["user", "admin"] as const;
|
||||
export const PERMISSION_CATALOG: readonly Permission[] = [
|
||||
"session.use", "session.read_all", "session.manage_all", "settings.manage",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
|
||||
];
|
||||
|
||||
const invalid = (): Error => new Error("authentication configuration is invalid");
|
||||
|
||||
@@ -33,7 +33,7 @@ const EMPTY_HKDF_SALT = Buffer.alloc(0);
|
||||
const ROLES = ["user", "admin"] as const;
|
||||
const PERMISSIONS = [
|
||||
"session.use", "session.read_all", "session.manage_all", "settings.manage",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
|
||||
] as const satisfies readonly Permission[];
|
||||
|
||||
const invalid = (): Error => new Error("auth_session_store_invalid");
|
||||
|
||||
@@ -5,7 +5,7 @@ export type Role = "user" | "admin";
|
||||
export type Permission =
|
||||
| "session.use" | "session.read_all" | "session.manage_all"
|
||||
| "settings.manage" | "workspace.manage" | "workspace.secrets.manage"
|
||||
| "database.manage" | "pi.manage" | "auth.diagnostics.read";
|
||||
| "database.manage" | "memory.manage" | "evidence.manage" | "pi.manage" | "auth.diagnostics.read";
|
||||
|
||||
export interface AuthenticationSessionConfig {
|
||||
regularTtlSeconds: number;
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { ThtRunner } from "../tht/tht-runner.js";
|
||||
import type { SemanticRuntimeConfig } from "../workspaces/runtime-renderer.js";
|
||||
import type { CatalogSyncRun, WorkspaceDatabase } from "./types.js";
|
||||
|
||||
/** Internal continuation of an applied physical sync, using the harness Memory boundary. */
|
||||
export function createMemoryCleanup(runner: Pick<ThtRunner, "withPrincipal">, runtime: SemanticRuntimeConfig) {
|
||||
return async (database: WorkspaceDatabase, run: CatalogSyncRun): Promise<number> => {
|
||||
if (run.phase !== "memory_cleanup" || !run.plannedDiff) throw new Error("Physical cleanup is not committed");
|
||||
const result = await runner.withPrincipal({ issuer: "installation", subject: "catalog-sync",
|
||||
roles: ["admin"], permissions: ["memory.manage"], isAdmin: true,
|
||||
}).runWithRuntimeSnapshot(["memory", "admin", "--workspace", database.workspaceId], JSON.stringify({
|
||||
action: "cleanup", runtime, request: { sync_id: run.id, database: database.databaseName,
|
||||
schema_name: database.schema, removed_tables: run.plannedDiff.deletedTables,
|
||||
removed_columns: run.plannedDiff.deletedColumns.map(column => ({ table: column.tableName, column: column.columnName })),
|
||||
},
|
||||
}));
|
||||
const payload = JSON.parse(result.stdout);
|
||||
if (result.code !== 0 || payload.indexed !== true || !Number.isInteger(payload.deleted) || payload.deleted < 0) {
|
||||
throw new Error("Memory cleanup is incomplete");
|
||||
}
|
||||
return payload.deleted;
|
||||
};
|
||||
}
|
||||
@@ -926,6 +926,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
scope: CatalogSyncScope,
|
||||
tableIds: readonly string[],
|
||||
snapshot: ObservedSchemaSnapshot,
|
||||
syncRunId?: string,
|
||||
): Promise<CatalogSyncCounts | undefined> {
|
||||
const database = this.records.get(databaseId);
|
||||
if (!database || database.version !== expectedDatabaseVersion) return undefined;
|
||||
@@ -1141,6 +1142,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
if (scope === "all") {
|
||||
this.records.set(databaseId, { ...database, schemaSyncedVersion: expectedDatabaseVersion, schemaSyncedAt: now });
|
||||
}
|
||||
if (syncRunId) await this.updateSyncRun(syncRunId, { phase: "memory_cleanup" });
|
||||
return {
|
||||
tables: (await this.listTables(databaseId)).length,
|
||||
columns: [...this.columns.values()].filter((column) => this.tables.get(column.tableId)?.databaseId === databaseId).length,
|
||||
@@ -1254,7 +1256,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
for (const run of this.syncRuns.values()) {
|
||||
if (["queued", "running", "awaiting_confirmation", "applying"].includes(run.state)) {
|
||||
await this.updateSyncRun(run.id, {
|
||||
state: "interrupted", phase: "completed", finishedAt: new Date().toISOString(),
|
||||
state: "interrupted", phase: run.phase === "memory_cleanup" ? "memory_cleanup" : "completed", finishedAt: new Date().toISOString(),
|
||||
errorCode: "SYNC_INTERRUPTED", errorMessage: "Synchronization was interrupted by a service restart",
|
||||
});
|
||||
}
|
||||
|
||||
@@ -102,7 +102,7 @@ export function loadMetadataGenerationModels(options: {
|
||||
...(apiKeyEnv ? { apiKeyEnv, apiKey } : {}),
|
||||
}));
|
||||
}
|
||||
const result = new RestartLoadedMetadataGenerationModels(models, catalog.defaultMetadataGeneration);
|
||||
const result = new RestartLoadedMetadataGenerationModels(models, models.size ? catalog.defaultInteraction : null);
|
||||
const safe = result.catalog();
|
||||
return {
|
||||
catalog: () => ({
|
||||
|
||||
@@ -1604,6 +1604,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
scope: CatalogSyncScope,
|
||||
tableIds: readonly string[],
|
||||
snapshot: ObservedSchemaSnapshot,
|
||||
syncRunId?: string,
|
||||
): Promise<CatalogSyncCounts | undefined> {
|
||||
return await this.db.transaction().execute(async (trx) => {
|
||||
const database = await trx.selectFrom("workspaceDatabases").select("version")
|
||||
@@ -1779,6 +1780,10 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
schemaSyncedAt: now,
|
||||
}).where("id", "=", databaseId).execute();
|
||||
}
|
||||
if (syncRunId) {
|
||||
await trx.updateTable("catalogSyncRuns").set({ phase: "memory_cleanup" })
|
||||
.where("id", "=", syncRunId).where("databaseId", "=", databaseId).execute();
|
||||
}
|
||||
return {
|
||||
tables: snapshot.tables.length,
|
||||
columns: scope === "tables" ? undefined : snapshot.columns.length,
|
||||
@@ -1882,7 +1887,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
|
||||
async interruptActiveSyncRuns(): Promise<void> {
|
||||
await this.db.updateTable("catalogSyncRuns").set({
|
||||
state: "interrupted", phase: "completed", errorCode: "worker_restarted",
|
||||
state: "interrupted", phase: sql`case when phase='memory_cleanup' then phase else 'completed' end`, errorCode: "worker_restarted",
|
||||
errorMessage: "Synchronization was interrupted by a backend restart.",
|
||||
finishedAt: sql`now()`, updatedAt: sql`now()`,
|
||||
leaseOwner: null, leaseExpiresAt: null,
|
||||
|
||||
@@ -58,6 +58,8 @@ export class CatalogSyncWorker {
|
||||
private readonly introspector: CatalogSchemaIntrospector,
|
||||
private readonly operations: CatalogOperationCoordinator,
|
||||
private readonly timeoutMs: number,
|
||||
private readonly cleanupMemory: (database: WorkspaceDatabase, run: CatalogSyncRun) => Promise<number>
|
||||
= async () => 0,
|
||||
) {}
|
||||
|
||||
async initialize(): Promise<void> {
|
||||
@@ -67,6 +69,9 @@ export class CatalogSyncWorker {
|
||||
}
|
||||
|
||||
async start(database: WorkspaceDatabase, scope: CatalogSyncScope, tableIds: readonly string[]): Promise<CatalogSyncRun> {
|
||||
if ((await this.repository.listSyncRuns(database.id, 100)).some(run => run.phase === "memory_cleanup")) {
|
||||
throw new CatalogConflictError("Retry the pending Memory cleanup before starting another synchronization");
|
||||
}
|
||||
const uniqueTableIds = [...new Set(tableIds)];
|
||||
if (scope === "columns") {
|
||||
const tables = await Promise.all(uniqueTableIds.map((tableId) => this.repository.getTable(database.id, tableId)));
|
||||
@@ -109,7 +114,7 @@ export class CatalogSyncWorker {
|
||||
async cancel(runId: string): Promise<CatalogSyncRun | undefined> {
|
||||
const run = await this.repository.getSyncRun(runId);
|
||||
if (!run) return undefined;
|
||||
if (run.state === "applying" || TERMINAL_STATES.has(run.state)) return run;
|
||||
if (run.phase === "memory_cleanup" || run.state === "applying" || TERMINAL_STATES.has(run.state)) return run;
|
||||
await this.repository.requestSyncRunCancellation(runId);
|
||||
this.controllers.get(runId)?.abort();
|
||||
if (run.state === "queued" || run.state === "awaiting_confirmation") {
|
||||
@@ -138,6 +143,16 @@ export class CatalogSyncWorker {
|
||||
}
|
||||
const database = await this.repository.get(previous.databaseId);
|
||||
if (!database) return undefined;
|
||||
if (previous.phase === "memory_cleanup") {
|
||||
const release = this.operations.reserve(database.id);
|
||||
try {
|
||||
const queued = await this.repository.updateSyncRun(runId, { state: "queued", cancelRequested: false,
|
||||
errorCode: null, errorMessage: null, finishedAt: null, leaseOwner: null, leaseExpiresAt: null });
|
||||
this.reservations.set(runId, release);
|
||||
this.launch(runId);
|
||||
return queued;
|
||||
} catch (error) { release(); throw error; }
|
||||
}
|
||||
return await this.start(database, previous.scope, previous.tableIds);
|
||||
}
|
||||
|
||||
@@ -179,6 +194,10 @@ export class CatalogSyncWorker {
|
||||
if (!database || database.version !== claimed.requestedDatabaseVersion) {
|
||||
throw new CatalogConflictError("Database binding changed before synchronization started");
|
||||
}
|
||||
if (claimed.phase === "memory_cleanup") {
|
||||
await this.finishMemoryCleanup(database, claimed);
|
||||
return;
|
||||
}
|
||||
const progress: CatalogSchemaScanProgress = async (phase, counts) => {
|
||||
await this.checkCancelled(runId);
|
||||
await this.repository.updateSyncRun(runId, {
|
||||
@@ -230,36 +249,30 @@ export class CatalogSyncWorker {
|
||||
claimed.scope,
|
||||
claimed.tableIds,
|
||||
snapshot,
|
||||
runId,
|
||||
);
|
||||
if (!applied) throw new CatalogConflictError("Database binding changed before schema changes were applied");
|
||||
await this.repository.updateSyncRun(runId, {
|
||||
state: "succeeded",
|
||||
phase: "completed",
|
||||
counts: applied,
|
||||
finishedAt: new Date().toISOString(),
|
||||
observedSnapshot: null,
|
||||
plannedDiff: null,
|
||||
confirmationToken: null,
|
||||
heartbeatAt: new Date().toISOString(),
|
||||
leaseOwner: null,
|
||||
leaseExpiresAt: null,
|
||||
});
|
||||
await this.repository.appendSyncEvent(runId, "info", "succeeded", "Synchronization completed.", { ...applied });
|
||||
this.release(runId);
|
||||
const cleanupRun = await this.repository.updateSyncRun(runId, { counts: applied });
|
||||
if (!cleanupRun) throw new Error("Synchronization run disappeared");
|
||||
await this.finishMemoryCleanup(database, cleanupRun);
|
||||
/* Completion is recorded only after the durable Memory cleanup succeeds. */
|
||||
return;
|
||||
} catch (error) {
|
||||
const current = await this.repository.getSyncRun(runId);
|
||||
const cancelled = !timedOut && (error instanceof SyncCancelledError || controller.signal.aborted || current?.cancelRequested);
|
||||
const failure = timedOut
|
||||
const failure = current?.phase === "memory_cleanup"
|
||||
? { code: "memory_cleanup_pending", message: "Catalog synchronized. Memory cleanup is pending; retry this synchronization to complete it." }
|
||||
: timedOut
|
||||
? { code: "schema_sync_timed_out", message: "Schema synchronization timed out." }
|
||||
: safeFailure(error);
|
||||
await this.repository.updateSyncRun(runId, {
|
||||
state: cancelled ? "cancelled" : "failed",
|
||||
phase: "completed",
|
||||
phase: current?.phase === "memory_cleanup" ? "memory_cleanup" : "completed",
|
||||
errorCode: cancelled ? null : failure.code,
|
||||
errorMessage: cancelled ? null : failure.message,
|
||||
finishedAt: new Date().toISOString(),
|
||||
observedSnapshot: null,
|
||||
plannedDiff: null,
|
||||
observedSnapshot: current?.phase === "memory_cleanup" ? current.observedSnapshot : null,
|
||||
plannedDiff: current?.phase === "memory_cleanup" ? current.plannedDiff : null,
|
||||
confirmationToken: null,
|
||||
leaseOwner: null,
|
||||
leaseExpiresAt: null,
|
||||
@@ -278,6 +291,20 @@ export class CatalogSyncWorker {
|
||||
}
|
||||
}
|
||||
|
||||
private async finishMemoryCleanup(database: WorkspaceDatabase, run: CatalogSyncRun): Promise<void> {
|
||||
if (!run.plannedDiff || run.phase !== "memory_cleanup") throw new Error("Missing committed cleanup context");
|
||||
await this.repository.appendSyncEvent(run.id, "info", "memory_cleanup", "Removing Memory cards with deleted physical dependencies.");
|
||||
const memoryDeleted = await this.cleanupMemory(database, run);
|
||||
const counts = { ...run.counts, memoryDeleted };
|
||||
await this.repository.updateSyncRun(run.id, {
|
||||
state: "succeeded", phase: "completed", counts, finishedAt: new Date().toISOString(),
|
||||
observedSnapshot: null, plannedDiff: null, confirmationToken: null,
|
||||
heartbeatAt: new Date().toISOString(), leaseOwner: null, leaseExpiresAt: null,
|
||||
});
|
||||
await this.repository.appendSyncEvent(run.id, "info", "succeeded", "Synchronization completed.", counts);
|
||||
this.release(run.id);
|
||||
}
|
||||
|
||||
private assertCapability(scope: CatalogSyncScope, snapshot: ObservedSchemaSnapshot): void {
|
||||
const required = scope === "all" ? ["tables", "columns", "relationships"] as const : [scope] as const;
|
||||
for (const name of required) {
|
||||
|
||||
@@ -366,7 +366,7 @@ export type CatalogSyncState =
|
||||
export type CatalogSyncPhase =
|
||||
| "queued" | "connecting" | "scanning_tables" | "scanning_columns"
|
||||
| "scanning_relationships" | "planning" | "awaiting_confirmation"
|
||||
| "applying" | "completed";
|
||||
| "applying" | "memory_cleanup" | "completed";
|
||||
|
||||
export interface CatalogSchemaDiff {
|
||||
deletedTables: string[];
|
||||
@@ -375,6 +375,7 @@ export interface CatalogSchemaDiff {
|
||||
}
|
||||
|
||||
export interface CatalogSyncCounts {
|
||||
memoryDeleted?: number;
|
||||
tables?: number;
|
||||
columns?: number;
|
||||
relationships?: number;
|
||||
@@ -587,6 +588,7 @@ export interface CatalogRepository {
|
||||
scope: CatalogSyncScope,
|
||||
tableIds: readonly string[],
|
||||
snapshot: ObservedSchemaSnapshot,
|
||||
syncRunId?: string,
|
||||
): Promise<CatalogSyncCounts | undefined>;
|
||||
createSyncRun(
|
||||
databaseId: string,
|
||||
|
||||
@@ -30,8 +30,11 @@ export interface AppConfig {
|
||||
dataRoot?: string;
|
||||
ollamaEnsureTimeoutMs: number;
|
||||
piManagementTimeoutMs: number;
|
||||
/** Host CLI platform projected into Docker; not the browser or container OS. */
|
||||
hostPlatform?: string;
|
||||
secretsFile?: string;
|
||||
installationConfigFile?: string;
|
||||
evidenceHostRegistryRoot?: string;
|
||||
modelCatalogFile?: string;
|
||||
sensitivityNer?: {
|
||||
pythonExecutable: string;
|
||||
@@ -474,8 +477,10 @@ export function loadConfig(
|
||||
dataRoot: env.THT_DATA_ROOT,
|
||||
ollamaEnsureTimeoutMs: Number(env.OLLAMA_ENSURE_TIMEOUT_MS ?? 60000),
|
||||
piManagementTimeoutMs: piManagementTimeout(env.PI_MANAGEMENT_TIMEOUT_MS),
|
||||
hostPlatform: env.THT_HOST_PLATFORM,
|
||||
secretsFile,
|
||||
installationConfigFile,
|
||||
evidenceHostRegistryRoot: env.THT_EVIDENCE_HOST_REGISTRY_ROOT || undefined,
|
||||
modelCatalogFile,
|
||||
sensitivityNer,
|
||||
piAuthFile,
|
||||
|
||||
@@ -41,15 +41,17 @@ const runtimeModelSchema = z.object({
|
||||
supportsReasoningEffort: z.boolean(),
|
||||
supportsStore: z.boolean(),
|
||||
maxTokensField: z.string().optional(),
|
||||
thinkingFormat: z.enum(["qwen", "qwen-chat-template"]).optional(),
|
||||
}).strict().optional(),
|
||||
}).strict().optional(),
|
||||
}).strict().refine((session) => !session.compatibility?.thinkingFormat || session.reasoning, {
|
||||
message: "thinkingFormat requires reasoning: true",
|
||||
}).optional(),
|
||||
metadataGeneration: z.object({ disableThinking: z.boolean() }).strict().optional(),
|
||||
}).strict();
|
||||
|
||||
const catalogSchema = z.object({
|
||||
schemaVersion: z.literal(1),
|
||||
defaultSession: canonicalId,
|
||||
defaultMetadataGeneration: canonicalId.optional(),
|
||||
schemaVersion: z.literal(2),
|
||||
defaultInteraction: canonicalId,
|
||||
embedding: z.object({ id: canonicalId, dimensions: z.number().int().positive() }).strict(),
|
||||
models: z.array(runtimeModelSchema).max(64),
|
||||
}).strict();
|
||||
@@ -57,8 +59,7 @@ const catalogSchema = z.object({
|
||||
export type RuntimeModel = z.infer<typeof runtimeModelSchema>;
|
||||
|
||||
export interface RuntimeModelCatalog {
|
||||
readonly defaultSession: string | null;
|
||||
readonly defaultMetadataGeneration: string | null;
|
||||
readonly defaultInteraction: string | null;
|
||||
readonly embedding: Readonly<{ id: string; dimensions: number }> | null;
|
||||
sessionModels(): readonly RuntimeModel[];
|
||||
metadataModels(): readonly RuntimeModel[];
|
||||
@@ -66,19 +67,21 @@ export interface RuntimeModelCatalog {
|
||||
}
|
||||
|
||||
class RestartLoadedRuntimeModelCatalog implements RuntimeModelCatalog {
|
||||
readonly defaultSession: string | null;
|
||||
readonly defaultMetadataGeneration: string | null;
|
||||
readonly defaultInteraction: string | null;
|
||||
readonly embedding: Readonly<{ id: string; dimensions: number }> | null;
|
||||
readonly #sessions: readonly RuntimeModel[];
|
||||
readonly #metadata: readonly RuntimeModel[];
|
||||
readonly #sessionIds: ReadonlySet<string>;
|
||||
|
||||
constructor(catalog?: z.infer<typeof catalogSchema>) {
|
||||
this.defaultSession = catalog?.defaultSession ?? null;
|
||||
this.defaultMetadataGeneration = catalog?.defaultMetadataGeneration ?? null;
|
||||
this.defaultInteraction = catalog?.defaultInteraction ?? null;
|
||||
this.embedding = catalog ? Object.freeze({ ...catalog.embedding }) : null;
|
||||
this.#sessions = Object.freeze((catalog?.models ?? []).filter((model) => model.session !== undefined));
|
||||
this.#metadata = Object.freeze((catalog?.models ?? []).filter((model) => model.metadataGeneration !== undefined));
|
||||
// One operational list. Session-only installations can still run Core, but once Admin
|
||||
// LLM models are configured every selectable model must support both adapters.
|
||||
const hasMetadata = catalog?.models.some((model) => model.metadataGeneration !== undefined);
|
||||
this.#sessions = Object.freeze((catalog?.models ?? []).filter((model) =>
|
||||
model.session !== undefined && (!hasMetadata || model.metadataGeneration !== undefined)));
|
||||
this.#metadata = Object.freeze(this.#sessions.filter((model) => model.metadataGeneration !== undefined));
|
||||
this.#sessionIds = new Set(this.#sessions.map((model) => model.id));
|
||||
}
|
||||
|
||||
@@ -129,11 +132,9 @@ export function loadRuntimeModelCatalog(file?: string): RuntimeModelCatalog {
|
||||
if (ids.size !== parsed.data.models.length) throw new Error("runtime model catalog contains duplicate models");
|
||||
const sessions = parsed.data.models.filter((model) => model.session !== undefined).map((model) => model.id);
|
||||
const metadata = parsed.data.models.filter((model) => model.metadataGeneration !== undefined).map((model) => model.id);
|
||||
if (!sessions.includes(parsed.data.defaultSession)) throw new Error("runtime model catalog session default is invalid");
|
||||
if ((metadata.length > 0) !== (parsed.data.defaultMetadataGeneration !== undefined)
|
||||
|| (parsed.data.defaultMetadataGeneration !== undefined
|
||||
&& !metadata.includes(parsed.data.defaultMetadataGeneration))) {
|
||||
throw new Error("runtime model catalog metadata default is invalid");
|
||||
if (!sessions.includes(parsed.data.defaultInteraction)
|
||||
|| (metadata.length > 0 && !metadata.includes(parsed.data.defaultInteraction))) {
|
||||
throw new Error("runtime model catalog interaction default is invalid");
|
||||
}
|
||||
return new RestartLoadedRuntimeModelCatalog(parsed.data);
|
||||
}
|
||||
|
||||
@@ -88,17 +88,22 @@ async function workflowDiagnostics(config: AppConfig): Promise<{ ready: true; wo
|
||||
if (revisions.length === 0) throw new Error("workflow diagnostics unavailable");
|
||||
return await withOperatorRunner(config, async (runner) => {
|
||||
for (const revision of revisions) {
|
||||
const result = await runner.run(["doctor", "--json"], revision.snapshotPath);
|
||||
let payload: unknown;
|
||||
const runtime = await runner.acquireWorkspaceRuntime(revision.snapshotPath);
|
||||
try {
|
||||
payload = JSON.parse(result.stdout);
|
||||
} catch {
|
||||
throw new Error("workflow diagnostics failed");
|
||||
const result = await runner.run(["doctor", "--json"], runtime.path);
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = JSON.parse(result.stdout);
|
||||
} catch {
|
||||
throw new Error("workflow diagnostics failed");
|
||||
}
|
||||
if (
|
||||
result.code !== 0 || !payload || typeof payload !== "object"
|
||||
|| (payload as { ok?: unknown }).ok !== true
|
||||
) throw new Error("workflow diagnostics failed");
|
||||
} finally {
|
||||
runtime.release();
|
||||
}
|
||||
if (
|
||||
result.code !== 0 || !payload || typeof payload !== "object"
|
||||
|| (payload as { ok?: unknown }).ok !== true
|
||||
) throw new Error("workflow diagnostics failed");
|
||||
}
|
||||
return { ready: true, workspaces: revisions.length };
|
||||
});
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
validateDeclarativePiConfig,
|
||||
} from "./managed-config.js";
|
||||
import type { RuntimeModelCatalog } from "../models/runtime-model-catalog.js";
|
||||
import { secretValue } from "../config/secret-bundle.js";
|
||||
|
||||
export interface PiModel {
|
||||
provider: string;
|
||||
@@ -64,6 +65,14 @@ export function createPiModelLister(cfg: AppConfig, opts: Opts = {}): ListModels
|
||||
}
|
||||
|
||||
const env = buildPiChildEnv({});
|
||||
// Pi's availability enumeration also needs the catalog-owned credentials for built-in
|
||||
// providers. It must keep working after their obsolete Pi auth entries are removed.
|
||||
for (const model of opts.modelCatalog?.sessionModels() ?? []) {
|
||||
const name = model.authentication.mode === "secret_env" ? model.authentication.apiKeyEnv : undefined;
|
||||
if (!name) continue;
|
||||
const value = secretValue(cfg, name);
|
||||
if (value) env[name] = value;
|
||||
}
|
||||
delete env.THT_DATA_ROOT;
|
||||
if (cfg.dataRoot !== undefined) env.THT_DATA_ROOT = cfg.dataRoot;
|
||||
const child = spawnFn(cfg.piBin, ["--mode", "rpc"], { cwd: cfg.harnessDir, env });
|
||||
|
||||
@@ -138,7 +138,7 @@ export interface PiRuntimeAgentSnapshot {
|
||||
* Bind a session Pi process to the exact managed auth/model bytes validated at spawn time.
|
||||
* Other agent resources remain live through symlinks, while session storage stays persistent.
|
||||
*/
|
||||
export function createPiRuntimeAgentSnapshot(): PiRuntimeAgentSnapshot {
|
||||
export function createPiRuntimeAgentSnapshot(options: { excludeAuthProvider?: string } = {}): PiRuntimeAgentSnapshot {
|
||||
const sourceAgentDir = configuredPiAgentDir();
|
||||
const auth = readPiAgentFile(sourceAgentDir, "auth.json", true);
|
||||
const models = readPiAgentFile(sourceAgentDir, "models.json", true);
|
||||
@@ -165,7 +165,18 @@ export function createPiRuntimeAgentSnapshot(): PiRuntimeAgentSnapshot {
|
||||
);
|
||||
}
|
||||
if (auth !== undefined) {
|
||||
writeFileSync(join(snapshotDir, "auth.json"), auth, { flag: "wx", mode: 0o600 });
|
||||
let effectiveAuth = auth;
|
||||
if (options.excludeAuthProvider) {
|
||||
const parsed = parsePiConfigJson(auth);
|
||||
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) throw new PiManagedConfigError();
|
||||
const provider = options.excludeAuthProvider.trim().toLowerCase();
|
||||
effectiveAuth = JSON.stringify(Object.fromEntries(
|
||||
Object.entries(parsed).filter(([key]) => key.trim().toLowerCase() !== provider),
|
||||
));
|
||||
}
|
||||
// secret_env is authoritative for this provider. Keep the operator's auth file intact,
|
||||
// but do not let an old Pi credential override the shared bundle inside this child.
|
||||
writeFileSync(join(snapshotDir, "auth.json"), effectiveAuth, { flag: "wx", mode: 0o600 });
|
||||
}
|
||||
if (models !== undefined) {
|
||||
writeFileSync(join(snapshotDir, "models.json"), models, { flag: "wx", mode: 0o600 });
|
||||
|
||||
@@ -36,6 +36,7 @@ export interface PiInstallationConfig {
|
||||
}
|
||||
|
||||
export interface PiStatus {
|
||||
hostPlatform: "linux" | "macos" | "windows";
|
||||
version?: string;
|
||||
ready: boolean;
|
||||
credentials: PiCredentialStatus;
|
||||
@@ -92,6 +93,9 @@ interface PiManagementDeps {
|
||||
}
|
||||
|
||||
export function createPiManagement(config: AppConfig, deps: PiManagementDeps): PiManagementService {
|
||||
const platform = config.hostPlatform ?? process.platform;
|
||||
const hostPlatform = platform === "darwin" ? "macos"
|
||||
: platform === "windows" || platform === "win32" ? "windows" : "linux";
|
||||
const now = deps.now ?? (() => new Date());
|
||||
const diagnostics: string[] = [];
|
||||
const addDiagnostic = (message: string): void => {
|
||||
@@ -106,23 +110,23 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
|
||||
});
|
||||
const credentialStatus = deps.credentialStatus ?? ((provider: string | undefined) => {
|
||||
try {
|
||||
const model = deps.modelCatalog.defaultSession
|
||||
? deps.modelCatalog.sessionModels().find((entry) => entry.id === deps.modelCatalog.defaultSession)
|
||||
const model = deps.modelCatalog.defaultInteraction
|
||||
? deps.modelCatalog.sessionModels().find((entry) => entry.id === deps.modelCatalog.defaultInteraction)
|
||||
: undefined;
|
||||
const credentialName = model?.authentication.mode === "secret_env"
|
||||
? model.authentication.apiKeyEnv
|
||||
: undefined;
|
||||
const configuredApiKey = configuredPiProviderApiKey(
|
||||
const configuredApiKey = credentialName ? `$${credentialName}` : configuredPiProviderApiKey(
|
||||
readConfiguredPiAgentFile("models.json", true),
|
||||
provider,
|
||||
) ?? (credentialName ? `$${credentialName}` : undefined);
|
||||
);
|
||||
return piProviderCredentialStatus({
|
||||
provider,
|
||||
authProviders: loadPiAuthProviders(),
|
||||
authProviders: credentialName ? new Set() : loadPiAuthProviders(),
|
||||
resolveCredentialValue: () => credentialName
|
||||
? secretValue(config, credentialName)
|
||||
: config.modelCatalogFile ? undefined : secretValue(config, "THT_MODEL_API_KEY"),
|
||||
credentialFile: config.modelApiKeyFile,
|
||||
credentialFile: credentialName ? undefined : config.modelApiKeyFile,
|
||||
configuredApiKey,
|
||||
});
|
||||
} catch {
|
||||
@@ -152,8 +156,8 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
|
||||
const installationConfig = (): PiInstallationConfig => {
|
||||
const settings = readSettings();
|
||||
const reasoning = config.defaults.thinking ?? settings.thinking;
|
||||
const selected = deps.modelCatalog.defaultSession
|
||||
? splitCanonicalModelId(deps.modelCatalog.defaultSession)
|
||||
const selected = deps.modelCatalog.defaultInteraction
|
||||
? splitCanonicalModelId(deps.modelCatalog.defaultInteraction)
|
||||
: undefined;
|
||||
return {
|
||||
...(selected ? selected : {}),
|
||||
@@ -169,11 +173,11 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
|
||||
try {
|
||||
const currentVersion = await version();
|
||||
addDiagnostic("Pi version probe succeeded");
|
||||
return { version: currentVersion, ready: true, credentials, config: current, checkedAt };
|
||||
return { hostPlatform, version: currentVersion, ready: true, credentials, config: current, checkedAt };
|
||||
} catch (error) {
|
||||
const message = stableMessage(error, "Pi runtime is unavailable");
|
||||
addDiagnostic(message);
|
||||
return { ready: false, credentials, config: current, checkedAt, message };
|
||||
return { hostPlatform, ready: false, credentials, config: current, checkedAt, message };
|
||||
}
|
||||
},
|
||||
|
||||
|
||||
@@ -25,6 +25,7 @@ export interface SessionRuntime {
|
||||
}
|
||||
|
||||
export interface RuntimeOptions {
|
||||
interactionLanguage?: string | null;
|
||||
provider?: string;
|
||||
model?: string;
|
||||
thinking?: string;
|
||||
@@ -48,6 +49,7 @@ export class PiProcessManager {
|
||||
private spawnFn: (
|
||||
sessionId: string, author: string, provider: string | undefined, model: string | undefined,
|
||||
principal?: PrincipalContext, runtimeConfigPath?: string,
|
||||
interactionLanguage?: string | null,
|
||||
) => ChildProcessWithoutNullStreams;
|
||||
private loadAuthProviders: (agentDir: string) => ReadonlySet<string>;
|
||||
private modelCatalog: RuntimeModelCatalog;
|
||||
@@ -63,15 +65,15 @@ export class PiProcessManager {
|
||||
) {
|
||||
this.modelCatalog = opts?.modelCatalog ?? loadRuntimeModelCatalog(cfg.modelCatalogFile);
|
||||
this.modelCatalogConfigured = cfg.modelCatalogFile !== undefined
|
||||
|| this.modelCatalog.defaultSession !== null;
|
||||
|| this.modelCatalog.defaultInteraction !== null;
|
||||
this.loadAuthProviders = opts?.authProviders
|
||||
?? ((agentDir) => loadPiAuthProviders({ agentDir }));
|
||||
if (opts?.spawnFn) {
|
||||
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath) =>
|
||||
this.spawnPi(opts.spawnFn!, sessionId, author, provider, model, principal, runtimeConfigPath);
|
||||
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath, language) =>
|
||||
this.spawnPi(opts.spawnFn!, sessionId, author, provider, model, principal, runtimeConfigPath, language);
|
||||
} else {
|
||||
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath) =>
|
||||
this.spawnPi(nodeSpawn, sessionId, author, provider, model, principal, runtimeConfigPath);
|
||||
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath, language) =>
|
||||
this.spawnPi(nodeSpawn, sessionId, author, provider, model, principal, runtimeConfigPath, language);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -85,33 +87,35 @@ export class PiProcessManager {
|
||||
private spawnPi(
|
||||
spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined,
|
||||
model: string | undefined, principal?: PrincipalContext, runtimeConfigPath?: string,
|
||||
interactionLanguage?: string | null,
|
||||
): ChildProcessWithoutNullStreams {
|
||||
// This is the final shared boundary for createFor(), spawnFor(), and resume(). Validate
|
||||
// before auth-provider inspection, then make Pi consume the exact copied bytes rather than
|
||||
// reopening mutable mounted auth/models files after this check.
|
||||
const agent = createPiRuntimeAgentSnapshot();
|
||||
const catalogModel = provider && model
|
||||
? this.modelCatalog.sessionModels().find((entry) => entry.provider === provider && entry.model === model)
|
||||
: undefined;
|
||||
const credentialName = catalogModel?.authentication.mode === "secret_env"
|
||||
? catalogModel.authentication.apiKeyEnv : undefined;
|
||||
const agent = createPiRuntimeAgentSnapshot({ excludeAuthProvider: credentialName ? provider : undefined });
|
||||
let child: ChildProcessWithoutNullStreams | undefined;
|
||||
try {
|
||||
const catalogModel = provider && model
|
||||
? this.modelCatalog.sessionModels()
|
||||
.find((entry) => entry.provider === provider && entry.model === model)
|
||||
: undefined;
|
||||
const credentialName = catalogModel?.authentication.mode === "secret_env"
|
||||
? catalogModel.authentication.apiKeyEnv
|
||||
: undefined;
|
||||
const projectedApiKey = configuredPiProviderApiKey(agent.models, provider)
|
||||
?? (credentialName ? `$${credentialName}` : undefined);
|
||||
const projectedApiKey = credentialName ? `$${credentialName}`
|
||||
: configuredPiProviderApiKey(agent.models, provider);
|
||||
const env = buildPiChildEnv({
|
||||
provider,
|
||||
authProviders: this.loadAuthProviders(agent.agentDir),
|
||||
authProviders: credentialName ? new Set() : this.loadAuthProviders(agent.agentDir),
|
||||
credentialValue: credentialName
|
||||
? secretValue(this.cfg, credentialName)
|
||||
: this.modelCatalogConfigured ? undefined : secretValue(this.cfg, "THT_MODEL_API_KEY"),
|
||||
credentialFile: this.cfg.modelApiKeyFile,
|
||||
credentialFile: credentialName ? undefined : this.cfg.modelApiKeyFile,
|
||||
configuredApiKey: projectedApiKey,
|
||||
additions: { THT_SESSION: sessionId, THT_AUTHOR: author },
|
||||
});
|
||||
env.PI_CODING_AGENT_DIR = agent.agentDir;
|
||||
// A launch hint only: the gate reads the authoritative manifest before each turn.
|
||||
delete env.THT_INTERACTION_LANGUAGE;
|
||||
if (interactionLanguage) env.THT_INTERACTION_LANGUAGE = interactionLanguage;
|
||||
env.PI_CODING_AGENT_SESSION_DIR = agent.sessionDir;
|
||||
clearPrincipalEnvironment(env);
|
||||
if (principal) Object.assign(env, principalEnvironment(principal));
|
||||
@@ -189,7 +193,9 @@ export class PiProcessManager {
|
||||
const model = o.model ?? this.cfg.defaults.model;
|
||||
let child: ChildProcessWithoutNullStreams;
|
||||
try {
|
||||
child = this.spawnFn(sessionId, author, provider, model, o.principal, o.runtimeConfig?.path);
|
||||
child = this.spawnFn(
|
||||
sessionId, author, provider, model, o.principal, o.runtimeConfig?.path, o.interactionLanguage,
|
||||
);
|
||||
} catch (error) {
|
||||
o.runtimeConfig?.release();
|
||||
throw error;
|
||||
@@ -298,8 +304,13 @@ export class PiProcessManager {
|
||||
}
|
||||
|
||||
async resume(sessionId: string, tht: ThtRunner): Promise<SessionRuntime> {
|
||||
const manifest = await tht.sessionShow(sessionId) as { provider?: string; model?: string; thinking?: string } | null;
|
||||
const manifest = await tht.sessionShow(sessionId) as {
|
||||
provider?: string; model?: string; thinking?: string; interaction_language?: string | null;
|
||||
} | null;
|
||||
const language = manifest?.interaction_language
|
||||
?? (await tht.ensureInteractionLanguage(sessionId)).interaction_language;
|
||||
return this.spawnFor(sessionId, {
|
||||
interactionLanguage: language,
|
||||
provider: manifest?.provider,
|
||||
model: manifest?.model,
|
||||
thinking: manifest?.thinking,
|
||||
|
||||
@@ -70,28 +70,29 @@ export function createPiProviderSmoke(
|
||||
try {
|
||||
const canonicalProvider = canonicalPiProvider(provider);
|
||||
if (!canonicalProvider || timeoutMs <= 0) throw providerFailure();
|
||||
const configuredAuthProviders = authProviders();
|
||||
const configuredAuthProviders = new Set(authProviders());
|
||||
const configuredModels = options.readModelsStore
|
||||
? options.readModelsStore()
|
||||
: readConfiguredPiAgentFile("models.json", true);
|
||||
const catalog = options.modelCatalog ?? loadRuntimeModelCatalog(config.modelCatalogFile);
|
||||
const catalogConfigured = config.modelCatalogFile !== undefined
|
||||
|| catalog.defaultSession !== null;
|
||||
|| catalog.defaultInteraction !== null;
|
||||
const catalogModel = catalog.sessionModels()
|
||||
.find((entry) => entry.provider === canonicalProvider && entry.model === model);
|
||||
const upstreamModel = catalogModel?.upstreamModel ?? model;
|
||||
const credentialName = catalogModel?.authentication.mode === "secret_env"
|
||||
? catalogModel.authentication.apiKeyEnv
|
||||
: undefined;
|
||||
const projectedApiKey = configuredPiProviderApiKey(configuredModels, canonicalProvider)
|
||||
?? (credentialName ? `$${credentialName}` : undefined);
|
||||
if (credentialName) configuredAuthProviders.delete(canonicalProvider);
|
||||
const projectedApiKey = credentialName ? `$${credentialName}`
|
||||
: configuredPiProviderApiKey(configuredModels, canonicalProvider);
|
||||
const env = buildPiChildEnv({
|
||||
provider: canonicalProvider,
|
||||
authProviders: configuredAuthProviders,
|
||||
credentialValue: credentialName
|
||||
? secretValue(config, credentialName)
|
||||
: catalogConfigured ? undefined : secretValue(config, "THT_MODEL_API_KEY"),
|
||||
credentialFile: config.modelApiKeyFile,
|
||||
credentialFile: credentialName ? undefined : config.modelApiKeyFile,
|
||||
configuredApiKey: projectedApiKey,
|
||||
});
|
||||
clearPrincipalEnvironment(env);
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
import path from "node:path";
|
||||
import type { FastifyInstance } from "fastify";
|
||||
import { z } from "zod";
|
||||
import { requirePermission, isPrincipalContext } from "../auth/authorization.js";
|
||||
import type { ThtRunner } from "../tht/tht-runner.js";
|
||||
import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||
import type { WorkspacePreprocessingService } from "../workspaces/preprocessing-service.js";
|
||||
|
||||
const params = z.object({ workspaceId: z.string().regex(/^[a-z][a-z0-9-]{2,62}$/),
|
||||
evidenceId: z.string().regex(/^evidence:[a-z0-9]+(?:-[a-z0-9]+)*$/).optional() });
|
||||
const query = z.object({
|
||||
q: z.string().max(1000).optional(), kind: z.enum(["domain", "glossary", "enum", "example", "mapping", "normalization", "formula", "reference"]).optional(),
|
||||
purpose: z.enum(["disambiguation", "rewriting", "schema_linking", "sql_generation"]).optional(),
|
||||
status: z.enum(["new", "modified", "active", "removed", "review_required", "legacy", "invalid"]).optional(),
|
||||
concept: z.string().max(300).optional(), table: z.string().max(300).optional(), column: z.string().max(300).optional(),
|
||||
source: z.string().max(300).optional(), language: z.string().max(30).optional(),
|
||||
sort: z.enum(["title", "id", "kind", "status"]).optional(), direction: z.enum(["asc", "desc"]).optional(),
|
||||
page: z.coerce.number().int().positive().optional(), page_size: z.coerce.number().int().min(1).max(100).optional(),
|
||||
}).strict();
|
||||
const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`;
|
||||
|
||||
export function evidenceRoutes(app: FastifyInstance, deps: {
|
||||
runner: Pick<ThtRunner, "withPrincipal">; registry: Pick<WorkspaceRegistry, "list">;
|
||||
registryRoot: string; hostRegistryRoot?: string;
|
||||
service: Partial<Pick<WorkspacePreprocessingService, "consolidateEvidence" | "evidenceSources">>;
|
||||
}) {
|
||||
app.post("/workspaces/:workspaceId/evidence/sources", async (request, reply) => {
|
||||
const principal = requirePermission(request, reply, "evidence.manage");
|
||||
if (!isPrincipalContext(principal)) return reply;
|
||||
try {
|
||||
const { workspaceId } = params.parse(request.params);
|
||||
const body = z.discriminatedUnion("action", [
|
||||
z.object({ action: z.literal("refresh") }).strict(),
|
||||
z.object({ action: z.literal("decide"), sourceId: z.string().regex(/^[a-f0-9]{64}$/),
|
||||
revision: z.string().regex(/^[a-f0-9]{64}$/), decision: z.enum(["keep", "replace"]) }).strict(),
|
||||
]).parse(request.body);
|
||||
if (!(await deps.registry.list()).some(w => w.id === workspaceId)) return reply.code(404).send({ code: "workspace_invalid" });
|
||||
if (!deps.service.evidenceSources) throw new Error("Evidence service unavailable");
|
||||
return await deps.service.evidenceSources({ workspaceId, ...body, actor: principal.subject });
|
||||
} catch (error) {
|
||||
return reply.code(error instanceof z.ZodError ? 400 : 503).send({ code: "evidence_unavailable",
|
||||
message: error instanceof z.ZodError ? "Invalid source request." : "Evidence source service is unavailable." });
|
||||
}
|
||||
});
|
||||
app.get<{ Params: { workspaceId: string; evidenceId?: string } }>("/workspaces/:workspaceId/evidence", read);
|
||||
app.get<{ Params: { workspaceId: string; evidenceId?: string } }>("/workspaces/:workspaceId/evidence/:evidenceId", read);
|
||||
async function read(request: import("fastify").FastifyRequest, reply: import("fastify").FastifyReply) {
|
||||
const principal = requirePermission(request, reply, "evidence.manage");
|
||||
if (!isPrincipalContext(principal)) return reply;
|
||||
try {
|
||||
const { workspaceId, evidenceId } = params.parse(request.params);
|
||||
const filters = query.parse(request.query);
|
||||
if (!(await deps.registry.list()).some(w => w.id === workspaceId)) return reply.code(404).send({ code: "workspace_invalid" });
|
||||
const root = path.join(deps.registryRoot, "repo", workspaceId);
|
||||
const result = await deps.runner.withPrincipal(principal).runWithRuntimeSnapshot(
|
||||
["evidence", "admin", "--workspace", workspaceId],
|
||||
JSON.stringify({ root, query: { ...filters, ...(evidenceId ? { id: evidenceId } : {}) } }),
|
||||
);
|
||||
const payload = JSON.parse(result.stdout);
|
||||
if (result.code !== 0) return reply.code(503).send(payload);
|
||||
if (evidenceId && !payload.item) return reply.code(404).send({ code: "evidence_not_found", message: "Evidence was not found." });
|
||||
const host = deps.hostRegistryRoot;
|
||||
const hostPath = host && (path.isAbsolute(host) || path.win32.isAbsolute(host))
|
||||
? (path.win32.isAbsolute(host) && !path.isAbsolute(host) ? path.win32 : path).join(host, "repo") : null;
|
||||
const hostJoin = hostPath && path.win32.isAbsolute(hostPath) && !path.isAbsolute(hostPath) ? path.win32.join : path.join;
|
||||
return { ...payload, location: { repository: hostPath, workspace: hostPath ? hostJoin(hostPath, workspaceId) : null,
|
||||
runtime_workspace: root, host: "Installation host", command: `tht workspace evidence consolidate --workspace ${workspaceId}`,
|
||||
git_commands: hostPath ? [
|
||||
`git -C ${quote(hostPath)} status --short -- ${quote(workspaceId + "/evidence")}`,
|
||||
`git -C ${quote(hostPath)} diff -- ${quote(workspaceId + "/evidence")}`,
|
||||
`git -C ${quote(hostPath)} add -A -- ${quote(workspaceId + "/evidence")}`,
|
||||
`git -C ${quote(hostPath)} commit --only -m 'Curate Evidence' -- ${quote(workspaceId + "/evidence")}`,
|
||||
`git -C ${quote(hostPath)} push`,
|
||||
] : [] } };
|
||||
} catch (error) {
|
||||
return reply.code(error instanceof z.ZodError ? 400 : 503).send({ code: "evidence_unavailable",
|
||||
message: error instanceof z.ZodError ? "Evidence filters are invalid." : "Evidence archive is unavailable." });
|
||||
}
|
||||
}
|
||||
app.post("/workspaces/:workspaceId/evidence/consolidate", async (request, reply) => {
|
||||
const principal = requirePermission(request, reply, "evidence.manage");
|
||||
if (!isPrincipalContext(principal)) return reply;
|
||||
try {
|
||||
const { workspaceId } = params.parse(request.params);
|
||||
if (!(await deps.registry.list()).some(w => w.id === workspaceId)) return reply.code(404).send({ code: "workspace_invalid" });
|
||||
if (!deps.service.consolidateEvidence) throw new Error("Evidence service unavailable");
|
||||
return await deps.service.consolidateEvidence({ workspaceId });
|
||||
} catch { return reply.code(503).send({ code: "evidence_unavailable", message: "Evidence consolidation is unavailable." }); }
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
import type { FastifyInstance, FastifyReply, FastifyRequest } from "fastify";
|
||||
import { z } from "zod";
|
||||
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||
import type { ThtRunner } from "../tht/tht-runner.js";
|
||||
import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||
import type { SemanticRuntimeConfig } from "../workspaces/runtime-renderer.js";
|
||||
|
||||
const paramsSchema = z.object({
|
||||
workspaceId: z.string().regex(/^[a-z][a-z0-9_-]{0,63}$/),
|
||||
cardId: z.string().regex(/^mem-[0-9a-f-]{36}$/).optional(),
|
||||
});
|
||||
const family = z.enum(["domain_clarification", "sql_rule", "solved_question", "explained_error"]);
|
||||
const cardSchema = z.object({
|
||||
family, subject: z.string().trim().min(1).max(1000),
|
||||
detail: z.string().max(50000).default(""), scope: z.string().trim().min(1).max(10000),
|
||||
rationale: z.string().max(10000).default(""), question: z.string().max(10000).default(""),
|
||||
sql: z.string().max(100000).default(""),
|
||||
concepts: z.array(z.string().trim().min(1).max(200)).max(100).default([]),
|
||||
dependencies: z.array(z.object({
|
||||
database: z.string().trim().min(1).max(200), schema_name: z.string().max(200).default(""),
|
||||
table: z.string().max(200).default(""), column: z.string().max(200).default(""),
|
||||
}).strict()).max(200).default([]),
|
||||
links: z.array(z.object({
|
||||
target_id: z.string().min(1).max(100), meaning: z.string().trim().min(1).max(1000),
|
||||
}).strict()).max(200).default([]),
|
||||
}).strict();
|
||||
const querySchema = z.object({
|
||||
q: z.string().max(1000).optional(), family: family.optional(),
|
||||
concept: z.string().max(200).optional(), database: z.string().max(200).optional(),
|
||||
table: z.string().max(200).optional(), column: z.string().max(200).optional(),
|
||||
origin: z.enum(["manual", "workflow"]).optional(),
|
||||
updated_after: z.iso.datetime({ offset: true }).optional(),
|
||||
updated_before: z.iso.datetime({ offset: true }).optional(),
|
||||
page: z.coerce.number().int().positive().optional(),
|
||||
page_size: z.coerce.number().int().min(1).max(100).optional(),
|
||||
sort: z.enum(["updated_at", "created_at", "subject", "family"]).optional(),
|
||||
direction: z.enum(["asc", "desc"]).optional(),
|
||||
}).strict();
|
||||
|
||||
export function memoryRoutes(app: FastifyInstance, deps: {
|
||||
runner: Pick<ThtRunner, "withPrincipal">;
|
||||
registry: Pick<WorkspaceRegistry, "list" | "read">;
|
||||
runtime: SemanticRuntimeConfig;
|
||||
}) {
|
||||
const invoke = (action: string) => async (request: FastifyRequest, reply: FastifyReply) => {
|
||||
const principal = requirePermission(request, reply, "memory.manage");
|
||||
if (!isPrincipalContext(principal)) return reply;
|
||||
try {
|
||||
const { workspaceId, cardId } = paramsSchema.parse(request.params);
|
||||
const input = action === "list" ? querySchema.parse(request.query)
|
||||
: action === "create" || action === "update"
|
||||
? { card: cardSchema.parse(request.body), ...(cardId ? { id: cardId } : {}) }
|
||||
: cardId ? { id: cardId } : {};
|
||||
// Registry identity is enough: administrative browsing must not require a DWH binding.
|
||||
const workspaces = await deps.registry.list();
|
||||
if (!workspaces.some(workspace => workspace.id === workspaceId)) {
|
||||
return reply.code(404).send({ code: "workspace_invalid", message: "Workspace was not found." });
|
||||
}
|
||||
const { workspace } = await deps.registry.read(workspaceId);
|
||||
const result = await deps.runner.withPrincipal(principal).runWithRuntimeSnapshot(
|
||||
["memory", "admin", "--workspace", workspaceId],
|
||||
JSON.stringify({ action, request: input, runtime: {
|
||||
...deps.runtime, memoryLanguage: workspace.workspace.language,
|
||||
} }),
|
||||
);
|
||||
let payload;
|
||||
try { payload = JSON.parse(result.stdout); } catch {
|
||||
return reply.code(503).send({ code: "memory_unavailable", message: "Memory archive is unavailable." });
|
||||
}
|
||||
if (result.code !== 0) {
|
||||
const codes: Record<string, number> = {
|
||||
memory_invalid: 400, memory_forbidden: 403, memory_not_found: 404,
|
||||
memory_conflict: 409, memory_unavailable: 503,
|
||||
};
|
||||
const code = typeof payload?.code === "string" && payload.code in codes
|
||||
? payload.code : "memory_unavailable";
|
||||
return reply.code(codes[code]).send({ code, message: "Memory operation could not be completed." });
|
||||
}
|
||||
return reply.code(action === "create" ? 201 : 200).send(payload);
|
||||
} catch (error) {
|
||||
if (error instanceof z.ZodError) {
|
||||
return reply.code(400).send({ code: "memory_invalid", message: "Memory request is invalid." });
|
||||
}
|
||||
return reply.code(503).send({ code: "memory_unavailable", message: "Memory archive is unavailable." });
|
||||
}
|
||||
};
|
||||
app.get("/workspaces/:workspaceId/memory", invoke("list"));
|
||||
app.post("/workspaces/:workspaceId/memory", invoke("create"));
|
||||
app.get("/workspaces/:workspaceId/memory/pending", invoke("pending"));
|
||||
app.get("/workspaces/:workspaceId/memory/:cardId", invoke("show"));
|
||||
app.put("/workspaces/:workspaceId/memory/:cardId", invoke("update"));
|
||||
app.delete("/workspaces/:workspaceId/memory/:cardId", invoke("delete"));
|
||||
app.post("/workspaces/:workspaceId/memory/:cardId/retry", invoke("retry"));
|
||||
}
|
||||
@@ -13,6 +13,7 @@ import type { MaintenanceBarrier } from "../runtime/maintenance-gate.js";
|
||||
import { hasPermission, isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||
import { splitCanonicalModelId, type RuntimeModelCatalog } from "../models/runtime-model-catalog.js";
|
||||
import type { CatalogRepository } from "../catalog/types.js";
|
||||
import { interactionLanguage } from "../tht/interaction-language.js";
|
||||
|
||||
const BOOTSTRAP_FAILURE_MESSAGE =
|
||||
"Session startup failed. Check configuration and connectivity, then Resume the session.";
|
||||
@@ -150,13 +151,34 @@ export function sessionRoutes(
|
||||
});
|
||||
app.addHook("onResponse", async (req) => { admissionLeases.get(req)?.(); });
|
||||
|
||||
type SessionRevisionScan = {
|
||||
revisions: Awaited<ReturnType<typeof d.workspaceRegistry.list>>;
|
||||
retainedComplete: boolean;
|
||||
};
|
||||
let retainedSnapshotWarning: string | null = null;
|
||||
|
||||
/** Include retained historical descriptors so removed workspaces remain resumable. */
|
||||
const sessionRevisions = async () => {
|
||||
const sessionRevisions = async (): Promise<SessionRevisionScan> => {
|
||||
const registry = d.workspaceRegistry as Partial<WorkspaceRegistry>;
|
||||
if (typeof registry.listRetainedSnapshots === "function") {
|
||||
return await registry.listRetainedSnapshots();
|
||||
try {
|
||||
const revisions = await registry.listRetainedSnapshots();
|
||||
retainedSnapshotWarning = null;
|
||||
return { revisions, retainedComplete: true };
|
||||
} catch (error) {
|
||||
// A legacy/corrupt historical snapshot must not make active sessions (and their SSE
|
||||
// reviewer gates) unreachable. The registry still rejects that snapshot; this fallback
|
||||
// exposes only descriptors from the verified active state and deliberately disables
|
||||
// retention reconciliation because the resulting session view is incomplete.
|
||||
const detail = error instanceof Error ? error.message : "unknown error";
|
||||
if (retainedSnapshotWarning !== detail) {
|
||||
console.warn("[sessions] retained snapshot discovery failed; using active snapshots:", detail);
|
||||
retainedSnapshotWarning = detail;
|
||||
}
|
||||
return { revisions: await d.workspaceRegistry.list(), retainedComplete: false };
|
||||
}
|
||||
}
|
||||
return await d.workspaceRegistry.list();
|
||||
return { revisions: await d.workspaceRegistry.list(), retainedComplete: true };
|
||||
};
|
||||
|
||||
const isNotFound = (error: unknown) =>
|
||||
@@ -200,7 +222,7 @@ export function sessionRoutes(
|
||||
};
|
||||
let revisions: Awaited<ReturnType<typeof d.workspaceRegistry.list>>;
|
||||
try {
|
||||
revisions = await sessionRevisions();
|
||||
({ revisions } = await sessionRevisions());
|
||||
} catch (registryError) {
|
||||
// Sessions created before revision pinning still live under the installation's legacy
|
||||
// default config. Keep that compatibility path available when a fresh installation has
|
||||
@@ -360,8 +382,17 @@ export function sessionRoutes(
|
||||
app.post("/sessions", async (req, reply) => {
|
||||
const b = req.body as {
|
||||
question: string; name?: string; workspace?: string; workspaceId?: string;
|
||||
provider?: string; model?: string; thinking?: string;
|
||||
provider?: string; model?: string; thinking?: string; interactionLanguage?: unknown;
|
||||
};
|
||||
const language = interactionLanguage(b?.interactionLanguage);
|
||||
if (!language) return reply.code(400).send({
|
||||
code: "invalid_interaction_language",
|
||||
error: "interactionLanguage must be a well-formed BCP-47 language tag",
|
||||
});
|
||||
if ((b.provider === undefined) !== (b.model === undefined)
|
||||
|| (b.provider !== undefined && (typeof b.provider !== "string" || typeof b.model !== "string" || !b.provider || !b.model))) {
|
||||
return reply.code(400).send({ error: "provider and model must be supplied together" });
|
||||
}
|
||||
const principal = getPrincipal(req);
|
||||
let s: Settings;
|
||||
try { s = await d.getSettings(principal); } catch { return storageFailure(reply); }
|
||||
@@ -425,11 +456,9 @@ export function sessionRoutes(
|
||||
}
|
||||
}
|
||||
const requestedCanonical = b.provider && b.model ? `${b.provider}/${b.model}` : undefined;
|
||||
let selectedCanonical = requestedCanonical ?? d.modelCatalog.defaultSession;
|
||||
let modelWarning: string | undefined;
|
||||
if (selectedCanonical && d.modelCatalog.defaultSession && !d.modelCatalog.hasSession(selectedCanonical)) {
|
||||
selectedCanonical = d.modelCatalog.defaultSession;
|
||||
modelWarning = `Configured model ${requestedCanonical ?? "selection"} is unavailable; using ${selectedCanonical}.`;
|
||||
const selectedCanonical = requestedCanonical ?? d.modelCatalog.defaultInteraction;
|
||||
if (selectedCanonical && d.modelCatalog.defaultInteraction && !d.modelCatalog.hasSession(selectedCanonical)) {
|
||||
return reply.code(503).send({ error: MODEL_UNAVAILABLE_MESSAGE, code: "model_unavailable" });
|
||||
}
|
||||
const selected = selectedCanonical ? splitCanonicalModelId(selectedCanonical) : undefined;
|
||||
const provider = selected?.provider ?? b.provider;
|
||||
@@ -480,11 +509,15 @@ export function sessionRoutes(
|
||||
// registry snapshot. The legacy fallback stays available for sessions created before
|
||||
// the browser-local preference migration.
|
||||
let id: string;
|
||||
let sessionLanguage = language;
|
||||
try {
|
||||
({ id } = await runner.sessionNew({
|
||||
const created = await runner.sessionNew({
|
||||
question: b.question, name: b.name, workspaceConfigPath,
|
||||
workspaceId, workspaceRevision, provider, model, thinking,
|
||||
}));
|
||||
interactionLanguage: language,
|
||||
});
|
||||
id = created.id;
|
||||
sessionLanguage = created.interaction_language ?? language;
|
||||
manifestPersisted = true;
|
||||
if (revisionLease) {
|
||||
await revisionLease.markPersisted().catch((error: unknown) => {
|
||||
@@ -497,6 +530,7 @@ export function sessionRoutes(
|
||||
} catch { return storageFailure(reply); }
|
||||
const options = {
|
||||
provider, model, thinking,
|
||||
interactionLanguage: sessionLanguage,
|
||||
author: principal.displayName ?? principal.subject,
|
||||
principal,
|
||||
question: b.question,
|
||||
@@ -533,7 +567,7 @@ export function sessionRoutes(
|
||||
),
|
||||
() => d.mgr.start(id, rt, runtimeOptions),
|
||||
);
|
||||
return { id, ...(modelWarning ? { warning: modelWarning } : {}) };
|
||||
return { id };
|
||||
} finally {
|
||||
if (revisionLease && !manifestPersisted) {
|
||||
await revisionLease.abort().catch((error: unknown) => {
|
||||
@@ -559,8 +593,8 @@ export function sessionRoutes(
|
||||
? { ...principal, isAdmin: false }
|
||||
: ownershipPrincipal(principal, "session.read_all");
|
||||
const runner = runnerFor(scopedPrincipal);
|
||||
const revisions = await sessionRevisions();
|
||||
const lists = await Promise.all(revisions
|
||||
const revisionScan = await sessionRevisions();
|
||||
const lists = await Promise.all(revisionScan.revisions
|
||||
.map((revision) => runner.sessionList(revision.snapshotPath) as Promise<SessionRow[]>));
|
||||
const sessions = new Map<string, SessionRow>();
|
||||
for (const row of lists.flat()) {
|
||||
@@ -570,7 +604,8 @@ export function sessionRoutes(
|
||||
// Only an administrator-visible complete list (or the single local principal) is safe
|
||||
// input for retention. A remote per-user view can never discard another principal's pin.
|
||||
const reconcileSnapshotRetention = (d.workspaceRegistry as Partial<WorkspaceRegistry>).reconcileSnapshotRetention;
|
||||
const hasCompleteRetentionView = (scope === "all" || principal.issuer === "local")
|
||||
const hasCompleteRetentionView = revisionScan.retainedComplete
|
||||
&& (scope === "all" || principal.issuer === "local")
|
||||
&& hasPermission(principal, "session.read_all");
|
||||
if (hasCompleteRetentionView && typeof reconcileSnapshotRetention === "function") {
|
||||
const retained = [...new Set(list
|
||||
@@ -604,6 +639,23 @@ export function sessionRoutes(
|
||||
} catch (error) { return lifecycleFailure(reply, error); }
|
||||
const rt = d.mgr.get(id);
|
||||
if (!rt) return reply.code(404).send({ error: "sessione non attiva" });
|
||||
const pending = rt.bridge.pendingWidget?.() as any;
|
||||
const response = (req.body as any)?.ui_response;
|
||||
if (pending?.widget === "archive-repair" && response?.id === pending.id && !response.control) {
|
||||
const choices = response.choices;
|
||||
if (!Array.isArray(choices) || choices.length !== 1)
|
||||
return reply.code(400).send({ error: "Select one archive repair choice" });
|
||||
if (choices[0] !== "continue" && choices[0] !== "reject") {
|
||||
const option = pending.repair?.options?.find((item: any) => item.id === choices[0]);
|
||||
if (!option || !["memory", "evidence"].includes(option.archive))
|
||||
return reply.code(400).send({ error: "Unknown archive repair choice" });
|
||||
const permission = option.archive === "memory" ? "memory.manage" : "evidence.manage";
|
||||
if (!principal.isAdmin || !hasPermission(principal, permission))
|
||||
return reply.code(403).send({ error: "Archive corrections require an administrator" });
|
||||
if (rt.ownerKey !== `${principal.issuer}\0${principal.subject}`)
|
||||
return reply.code(409).send({ error: "Resume the session with your account before correcting an archive" });
|
||||
}
|
||||
}
|
||||
if (!rt.bridge.respond((req.body as any).ui_response)) {
|
||||
return reply.code(409).send({ error: "risposta non corrispondente al gate in attesa" });
|
||||
}
|
||||
@@ -623,6 +675,13 @@ export function sessionRoutes(
|
||||
app.post("/sessions/:id/resume", async (req, reply) => {
|
||||
const id = (req.params as any).id;
|
||||
const principal = getPrincipal(req);
|
||||
if (Object.hasOwn(req.body ?? {}, "interactionLanguage")
|
||||
|| Object.hasOwn(req.body ?? {}, "interaction_language")) {
|
||||
return reply.code(400).send({
|
||||
code: "interaction_language_pinned",
|
||||
error: "Resume uses the session's persisted interaction language; overrides are not accepted",
|
||||
});
|
||||
}
|
||||
return withSessionLifecycle(id, async () => {
|
||||
let settings: Settings;
|
||||
let located: LocatedSession | undefined;
|
||||
@@ -640,11 +699,19 @@ export function sessionRoutes(
|
||||
const saved = manifest as {
|
||||
provider?: string; model?: string; thinking?: string;
|
||||
workspace_id?: string; workspace_revision?: string;
|
||||
interaction_language?: string | null;
|
||||
};
|
||||
const savedCanonical = saved.provider && saved.model ? `${saved.provider}/${saved.model}` : "";
|
||||
if (d.modelCatalog.defaultSession && (!savedCanonical || !d.modelCatalog.hasSession(savedCanonical))) {
|
||||
const requested = (req.body ?? {}) as { provider?: string; model?: string; thinking?: string };
|
||||
if ((requested.provider === undefined) !== (requested.model === undefined)
|
||||
|| (requested.provider !== undefined && (typeof requested.provider !== "string" || typeof requested.model !== "string" || !requested.provider || !requested.model))) {
|
||||
return reply.code(400).send({ error: "provider and model must be supplied together" });
|
||||
}
|
||||
const selectedCanonical = requested.provider && requested.model
|
||||
? `${requested.provider}/${requested.model}` : d.modelCatalog.defaultInteraction;
|
||||
if (d.modelCatalog.defaultInteraction && (!selectedCanonical || !d.modelCatalog.hasSession(selectedCanonical))) {
|
||||
return reply.code(503).send({ error: MODEL_UNAVAILABLE_MESSAGE, code: "model_unavailable" });
|
||||
}
|
||||
const selected = selectedCanonical ? splitCanonicalModelId(selectedCanonical) : saved;
|
||||
let workspaceConfigPath: string;
|
||||
let workspaceDescriptor: WorkspaceDescriptor | undefined;
|
||||
try {
|
||||
@@ -671,13 +738,19 @@ export function sessionRoutes(
|
||||
});
|
||||
}
|
||||
try { settings = await d.getSettings(principal); } catch { return storageFailure(reply); }
|
||||
let language = saved.interaction_language;
|
||||
if (language == null) {
|
||||
try {
|
||||
language = (await runner.ensureInteractionLanguage(id, workspaceConfigPath)).interaction_language;
|
||||
} catch { return reply.code(503).send({ error: RESUME_FAILURE_MESSAGE }); }
|
||||
}
|
||||
// This check belongs inside the per-session lock: a preceding cold Resume may have
|
||||
// installed a running runtime while this request was waiting.
|
||||
const existing = d.mgr.get(id);
|
||||
if (existing) {
|
||||
const state = existing.bridge.turnState();
|
||||
if (state === "running" || state === "waiting") {
|
||||
return reply.code(200).send({ id, alreadyActive: true });
|
||||
return reply.code(200).send({ id, alreadyActive: true, workspaceId: saved.workspace_id });
|
||||
}
|
||||
}
|
||||
const ensure = await d.readiness.ensure(
|
||||
@@ -688,8 +761,9 @@ export function sessionRoutes(
|
||||
...(ensure.code ? { code: ensure.code } : {}),
|
||||
});
|
||||
const options = {
|
||||
provider: saved?.provider,
|
||||
model: saved?.model,
|
||||
provider: selected.provider,
|
||||
interactionLanguage: language,
|
||||
model: selected.model,
|
||||
thinking: saved?.thinking ?? settings.thinking,
|
||||
author: principal.displayName ?? principal.subject,
|
||||
principal,
|
||||
@@ -711,7 +785,7 @@ export function sessionRoutes(
|
||||
if (current) {
|
||||
const state = current.bridge.turnState();
|
||||
if (state === "running" || state === "waiting") {
|
||||
return reply.code(200).send({ id, alreadyActive: true });
|
||||
return reply.code(200).send({ id, alreadyActive: true, workspaceId: saved.workspace_id });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -752,7 +826,7 @@ export function sessionRoutes(
|
||||
d.mgr.configure(rt, runtimeOptions), null,
|
||||
() => d.mgr.start(id, rt, runtimeOptions),
|
||||
);
|
||||
return reply.code(200).send({ id, alreadyActive: false });
|
||||
return reply.code(200).send({ id, alreadyActive: false, workspaceId: saved.workspace_id });
|
||||
});
|
||||
});
|
||||
app.post("/sessions/:id/close", async (req, reply) => {
|
||||
|
||||
@@ -16,8 +16,8 @@ export function effectiveSettings(
|
||||
modelCatalog?: RuntimeModelCatalog,
|
||||
): Settings {
|
||||
const workspaces = listWorkspaces(cfg.harnessDir);
|
||||
const selected = modelCatalog?.defaultSession
|
||||
? splitCanonicalModelId(modelCatalog.defaultSession)
|
||||
const selected = modelCatalog?.defaultInteraction
|
||||
? splitCanonicalModelId(modelCatalog.defaultInteraction)
|
||||
: undefined;
|
||||
return {
|
||||
workspace: stored.workspace ?? workspaces[0]?.name,
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
/** Validate/canonicalize the tag; available translation catalogs belong to the UI. */
|
||||
export function interactionLanguage(value: unknown): string | undefined {
|
||||
if (typeof value !== "string") return undefined;
|
||||
try {
|
||||
const [canonical] = Intl.getCanonicalLocales(value);
|
||||
return canonical;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
@@ -59,6 +59,7 @@ export interface SessionRow {
|
||||
author: string | null;
|
||||
workspace_id?: string | null;
|
||||
workspace_revision?: string | null;
|
||||
interaction_language?: string | null;
|
||||
archived?: boolean;
|
||||
}
|
||||
|
||||
@@ -482,6 +483,7 @@ export class ThtRunner {
|
||||
|
||||
async sessionNew(o: {
|
||||
question: string;
|
||||
interactionLanguage?: string;
|
||||
provider?: string;
|
||||
model?: string;
|
||||
thinking?: string;
|
||||
@@ -497,6 +499,7 @@ export class ThtRunner {
|
||||
["--provider", o.provider],
|
||||
["--model", o.model],
|
||||
["--thinking", o.thinking],
|
||||
["--interaction-language", o.interactionLanguage],
|
||||
["--name", o.name],
|
||||
["--workspace-id", o.workspaceId],
|
||||
["--workspace-revision", o.workspaceRevision],
|
||||
@@ -504,7 +507,7 @@ export class ThtRunner {
|
||||
if (v) a.push(f, v);
|
||||
}
|
||||
a.push("--json");
|
||||
return this.json<{ id: string }>(a, o.workspaceConfigPath ?? o.workspace);
|
||||
return this.json<{ id: string; interaction_language?: string }>(a, o.workspaceConfigPath ?? o.workspace);
|
||||
}
|
||||
|
||||
/** Build and persist the deterministic F1 retrieval pack for a new session. */
|
||||
@@ -533,6 +536,12 @@ export class ThtRunner {
|
||||
return this.json<unknown>(["session", "show", id, "--json"], workspace);
|
||||
}
|
||||
|
||||
ensureInteractionLanguage(id: string, workspace?: string) {
|
||||
return this.json<{ interaction_language: string }>(
|
||||
["session", "ensure-interaction-language", id, "--json"], workspace,
|
||||
);
|
||||
}
|
||||
|
||||
sqlPreview(id: string, p: { limit?: number; offset?: number }, workspace?: string) {
|
||||
// No positional FILE: the harness resolves sql_final.sql from the session
|
||||
// via _session_sql_file(cfg, session_id), which respects the workspace path.
|
||||
|
||||
@@ -18,7 +18,7 @@ export interface WorkspaceMaintenanceIo {
|
||||
writeStderr(value: string): void;
|
||||
}
|
||||
|
||||
type Command = "inspect" | "preprocess-run" | "preprocess-clear";
|
||||
type Command = "inspect" | "preprocess-run" | "preprocess-clear" | "evidence-consolidate" | "evidence-refresh" | "evidence-decide";
|
||||
|
||||
function failureResult(
|
||||
operation: string,
|
||||
@@ -65,6 +65,9 @@ function parseRequest(command: string, stdin: string): Record<string, unknown> {
|
||||
inspect: ["schemaVersion", "workspaceId"],
|
||||
"preprocess-run": ["schemaVersion", "workspaceId"],
|
||||
"preprocess-clear": ["schemaVersion", "workspaceId"],
|
||||
"evidence-consolidate": ["schemaVersion", "workspaceId"],
|
||||
"evidence-refresh": ["schemaVersion", "workspaceId"],
|
||||
"evidence-decide": ["schemaVersion", "workspaceId", "sourceId", "revision", "decision"],
|
||||
};
|
||||
const allowed = allowedByCommand[command];
|
||||
if (!allowed) throw new Error("unknown command");
|
||||
@@ -81,6 +84,16 @@ function exitCodeFor(result: WorkspaceOperationResult): number {
|
||||
|
||||
async function dispatch(command: Command, service: WorkspacePreprocessingService, request: Record<string, unknown>): Promise<WorkspaceOperationResult> {
|
||||
switch (command) {
|
||||
case "evidence-refresh":
|
||||
return await service.evidenceSources({ workspaceId: request.workspaceId as string, action: "refresh" });
|
||||
case "evidence-decide":
|
||||
if (typeof request.sourceId !== "string" || !/^[a-f0-9]{64}$/.test(request.sourceId)
|
||||
|| typeof request.revision !== "string" || !/^[a-f0-9]{64}$/.test(request.revision)
|
||||
|| (request.decision !== "keep" && request.decision !== "replace")) throw new Error("invalid request");
|
||||
return await service.evidenceSources({ workspaceId: request.workspaceId as string, action: "decide",
|
||||
sourceId: request.sourceId, revision: request.revision, decision: request.decision });
|
||||
case "evidence-consolidate":
|
||||
return await service.consolidateEvidence({ workspaceId: request.workspaceId as string });
|
||||
case "inspect":
|
||||
return await service.inspect({ workspaceId: request.workspaceId as string });
|
||||
case "preprocess-run":
|
||||
@@ -127,7 +140,8 @@ export async function runWorkspaceMaintenanceCli(
|
||||
|| message === "unexpected request field"
|
||||
|| message === "invalid workspace id";
|
||||
return command in {
|
||||
inspect: true, "preprocess-run": true, "preprocess-clear": true,
|
||||
inspect: true, "preprocess-run": true, "preprocess-clear": true, "evidence-consolidate": true,
|
||||
"evidence-refresh": true, "evidence-decide": true,
|
||||
} ? (requestError ? 2 : 1) : 2;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,6 +22,7 @@ export interface EvidencePreprocessingRequest {
|
||||
evidence: EvidenceConfig;
|
||||
job: EvidenceJobState;
|
||||
dryRun?: boolean;
|
||||
consolidate?: boolean;
|
||||
httpPrivateHostAllowlist?: readonly string[];
|
||||
}
|
||||
|
||||
@@ -56,7 +57,7 @@ function isPrivateHost(hostname: string): boolean {
|
||||
return hostname.endsWith(".internal");
|
||||
}
|
||||
|
||||
function evidencePolicy(
|
||||
export function evidencePolicy(
|
||||
evidence: EvidenceConfig,
|
||||
httpPrivateHostAllowlist?: readonly string[],
|
||||
): EvidencePreprocessingOutcome | undefined {
|
||||
@@ -95,13 +96,14 @@ function jobResult(job: EvidenceJobState): Pick<
|
||||
};
|
||||
}
|
||||
|
||||
async function runEvidenceStage(
|
||||
export async function runEvidenceStage(
|
||||
request: EvidencePreprocessingRequest,
|
||||
deps: EvidencePreprocessingDependencies,
|
||||
): Promise<EvidencePreprocessingOutcome> {
|
||||
const payload = await deps.runStage([
|
||||
"preprocess",
|
||||
"evidence",
|
||||
...(request.consolidate ? ["--consolidate"] : []),
|
||||
...(request.dryRun ? ["--dry-run"] : []),
|
||||
...(request.job.childRuns.evidence
|
||||
? ["--resume", request.job.childRuns.evidence]
|
||||
|
||||
@@ -3,6 +3,8 @@ import { renameSync, rmSync, writeFileSync, mkdirSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import {
|
||||
continueEvidencePreprocessing,
|
||||
evidencePolicy,
|
||||
runEvidenceStage,
|
||||
type EvidencePreprocessingDependencies,
|
||||
type EvidencePreprocessingOutcome,
|
||||
} from "./evidence/preprocessing.js";
|
||||
@@ -108,6 +110,72 @@ function baseResult(
|
||||
export class WorkspacePreprocessingService {
|
||||
constructor(private readonly deps: WorkspacePreprocessingServiceDeps) {}
|
||||
|
||||
async evidenceSources(options: { workspaceId: string; action: "refresh" | "decide";
|
||||
sourceId?: string; revision?: string; decision?: "keep" | "replace"; actor?: string }): Promise<WorkspaceOperationResult> {
|
||||
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
|
||||
try {
|
||||
if (options.action === "refresh") {
|
||||
const refused = evidencePolicy(runtime.workspace.evidence, this.deps.httpPrivateHostAllowlist);
|
||||
if (refused) return baseResult(runtime, "evidence refresh", "failed", refused.code);
|
||||
}
|
||||
const argv = ["evidence", "sources", options.action, "--json", "-c", "/dev/fd/3"];
|
||||
if (options.action === "decide") {
|
||||
if (!/^[a-f0-9]{64}$/.test(options.sourceId ?? "") || !/^[a-f0-9]{64}$/.test(options.revision ?? "")
|
||||
|| !["keep", "replace"].includes(options.decision ?? "")) throw new Error("Invalid source decision");
|
||||
const preflight = await this.deps.evidencePreflight(runtime.workspace);
|
||||
if (!preflight.ok) return baseResult(runtime, "evidence sources", "failed", preflight.code);
|
||||
argv.push("--source-id", options.sourceId!, "--revision", options.revision!, "--decision", options.decision!);
|
||||
}
|
||||
argv.push("--actor", options.actor ?? "installation operator");
|
||||
const result = await this.deps.runChild({ argv, configPath: runtime.configLease.path });
|
||||
const payload = JSON.parse(result.stdout);
|
||||
if (result.exitCode !== 0 || payload.status !== "succeeded") throw new Error(
|
||||
typeof payload.error === "string" ? payload.error.slice(0, 1500) : "Evidence source operation failed");
|
||||
return baseResult(runtime, `evidence ${options.action}`, "succeeded", "ok", {
|
||||
counts: this.numberRecord(payload.counts),
|
||||
warnings: [options.action === "refresh" ? "Source comparisons are ready in Evidence management. Active content is unchanged."
|
||||
: "Source decision activated locally. Commit and push the Evidence tree manually."],
|
||||
});
|
||||
} catch (error) {
|
||||
return baseResult(runtime, `evidence ${options.action}`, "failed", "evidence_materialization_required", {
|
||||
warnings: [error instanceof Error ? error.message : "Evidence source operation failed"],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async consolidateEvidence(options: { workspaceId: string }): Promise<WorkspaceOperationResult> {
|
||||
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
|
||||
const preflight = await this.deps.evidencePreflight(runtime.workspace);
|
||||
if (!preflight.ok) return baseResult(runtime, "evidence consolidate", "failed", preflight.code);
|
||||
// Evidence has its own durable archive/corpus jobs. Do not alter Catalog readiness
|
||||
// or the full preprocessing job when publishing this one component.
|
||||
try {
|
||||
const outcome = await runEvidenceStage({ evidence: runtime.workspace.evidence,
|
||||
consolidate: true, job: { runId: randomBytes(16).toString("hex"), completedStages: [], childRuns: {} },
|
||||
}, {
|
||||
runStage: async argv => {
|
||||
const result = await this.deps.runChild({ argv, configPath: runtime.configLease.path });
|
||||
const payload = JSON.parse(result.stdout);
|
||||
if (result.exitCode !== 0 || payload.status !== "succeeded") {
|
||||
return Promise.reject(new Error(typeof payload.error === "string" ? payload.error.slice(0, 1500) : "Evidence consolidation failed. Retry the command."));
|
||||
}
|
||||
return payload;
|
||||
},
|
||||
persistJob: () => undefined,
|
||||
evidencePreflight: async () => preflight,
|
||||
requireRunId: value => this.requireRunId(value),
|
||||
numberRecord: value => this.numberRecord(value),
|
||||
});
|
||||
return baseResult(runtime, "evidence consolidate", "succeeded", "ok", {
|
||||
...outcome, warnings: ["Evidence is active locally. Catalog and Schema readiness are unchanged. Commit and push the Evidence files manually."],
|
||||
});
|
||||
} catch (error) {
|
||||
return baseResult(runtime, "evidence consolidate", "failed", "evidence_materialization_required", {
|
||||
warnings: [error instanceof Error ? error.message : "Evidence consolidation failed. Retry the command."],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async inspect(options: { workspaceId: string }): Promise<WorkspaceOperationResult> {
|
||||
try {
|
||||
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
|
||||
|
||||
@@ -497,7 +497,10 @@ export async function publishDeterministicRuntimeConfigLease(options: {
|
||||
rendered.workspaceRevision,
|
||||
renderedConfigObject,
|
||||
);
|
||||
const identitySuffix = inputFingerprintValue.slice(7, 23);
|
||||
// The Catalog input identity intentionally excludes representation-only changes.
|
||||
// A lease also identifies its bytes, so a new renderer never collides with an
|
||||
// immutable config produced by an earlier release for the same Catalog inputs.
|
||||
const identitySuffix = sha256(inputFingerprintValue + "\n" + publishedConfig).slice(7, 23);
|
||||
|
||||
const preprocessingRoot = ensureTrustedDirectory(join(
|
||||
options.dataRoot,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { basename, join } from "node:path";
|
||||
import { basename, dirname, join } from "node:path";
|
||||
import { stringify } from "yaml";
|
||||
import { buildInstallationContract } from "./contracts.js";
|
||||
import { validateWorkspaceDescriptor, type WorkspaceDescriptor } from "./schema.js";
|
||||
@@ -179,6 +179,7 @@ function renderEvidence(
|
||||
return {
|
||||
evidence: {
|
||||
...(workspace.evidence.schema_version === 2 ? { schema_version: 2 } : {}),
|
||||
local_archive_root: join(dirname(dirname(context.revisionContentRoot)), "repo", context.workspaceId),
|
||||
sources: [renderedSource],
|
||||
},
|
||||
vector: {
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
import Fastify from "fastify";
|
||||
import { expect, test, vi } from "vitest";
|
||||
import { sessionRoutes } from "../src/routes/sessions.js";
|
||||
import type { PrincipalContext } from "../src/auth/principal.js";
|
||||
|
||||
test.each([
|
||||
[false, "m", 403], [false, "e", 403], [false, "reject", 204],
|
||||
[true, "m", 204], [true, "e", 204], [true, "forged", 400],
|
||||
] as const)("archive repair response enforces the responding principal (%s, %s)", async (admin, choice, status) => {
|
||||
const app = Fastify();
|
||||
const principal: PrincipalContext = { issuer: "test", subject: "reviewer", isAdmin: admin,
|
||||
roles: admin ? ["admin"] : ["user"],
|
||||
permissions: admin ? ["session.use", "memory.manage", "evidence.manage"] : ["session.use"] };
|
||||
app.addHook("preHandler", async request => { request.principal = principal; });
|
||||
const respond = vi.fn(() => true);
|
||||
sessionRoutes(app, {
|
||||
tht: {}, mgr: { get: () => ({ ownerKey: "test\0reviewer", bridge: {
|
||||
respond, pendingWidget: () => ({ id: "gate", widget: "archive-repair", repair: { options: [
|
||||
{ id: "m", archive: "memory" }, { id: "e", archive: "evidence" },
|
||||
] } }),
|
||||
} }) },
|
||||
} as unknown as Parameters<typeof sessionRoutes>[1]);
|
||||
try {
|
||||
const result = await app.inject({ method: "POST", url: "/sessions/s/response",
|
||||
payload: { ui_response: { id: "gate", choices: [choice] } } });
|
||||
expect(result.statusCode).toBe(status);
|
||||
expect(respond).toHaveBeenCalledTimes(status === 204 ? 1 : 0);
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
|
||||
test("an administrator cannot attribute a repair to another runtime's principal", async () => {
|
||||
const app = Fastify();
|
||||
app.addHook("preHandler", async request => { request.principal = {
|
||||
issuer: "test", subject: "second-admin", isAdmin: true, roles: ["admin"],
|
||||
permissions: ["session.use", "memory.manage"],
|
||||
}; });
|
||||
const respond = vi.fn();
|
||||
sessionRoutes(app, { tht: {}, mgr: { get: () => ({ ownerKey: "test\0first-admin", bridge: {
|
||||
respond, pendingWidget: () => ({ id: "gate", widget: "archive-repair",
|
||||
repair: { options: [{ id: "m", archive: "memory" }] } }),
|
||||
} }) } } as unknown as Parameters<typeof sessionRoutes>[1]);
|
||||
try {
|
||||
const result = await app.inject({ method: "POST", url: "/sessions/s/response",
|
||||
payload: { ui_response: { id: "gate", choices: ["m"] } } });
|
||||
expect(result.statusCode).toBe(409);
|
||||
expect(respond).not.toHaveBeenCalled();
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
@@ -188,6 +188,8 @@ test("roles collapse duplicates and admin contains all administrative permission
|
||||
"workspace.manage",
|
||||
"workspace.secrets.manage",
|
||||
"database.manage",
|
||||
"memory.manage",
|
||||
"evidence.manage",
|
||||
"pi.manage",
|
||||
"auth.diagnostics.read",
|
||||
]);
|
||||
|
||||
@@ -130,7 +130,7 @@ test("local login sets a non-persistent opaque session cookie and exposes only a
|
||||
roles: ["admin"],
|
||||
permissions: [
|
||||
"session.use", "session.read_all", "session.manage_all", "settings.manage",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
|
||||
],
|
||||
isAdmin: true,
|
||||
csrfToken: expect.stringMatching(/^[A-Za-z0-9_-]{43}$/),
|
||||
|
||||
@@ -7,6 +7,8 @@ import { join } from "node:path";
|
||||
import { expandLocalHome, localPrincipal, upstreamPrincipal } from "../src/auth/principal.js";
|
||||
import { buildApp } from "../src/app.js";
|
||||
import { loadConfig } from "../src/config.js";
|
||||
import { rolesToPermissions } from "../src/auth/config.js";
|
||||
import type { Permission, Role } from "../src/auth/types.js";
|
||||
|
||||
test("server smoke rejects retired trusted claims under OIDC authentication", () => {
|
||||
const smoke = readFileSync("../scripts/unified-deployment-smoke.sh", "utf8");
|
||||
@@ -32,7 +34,7 @@ test("local mode resolves a stable local principal", async () => {
|
||||
roles: ["admin"],
|
||||
permissions: [
|
||||
"session.use", "session.read_all", "session.manage_all", "settings.manage",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
|
||||
],
|
||||
isAdmin: true,
|
||||
});
|
||||
@@ -74,7 +76,7 @@ test("upstream mode accepts only normalized proxy principal headers", async () =
|
||||
issuer: "portal", subject: "42", displayName: "Alice", roles: ["user", "admin"],
|
||||
permissions: [
|
||||
"session.use", "session.read_all", "session.manage_all", "settings.manage",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage", "auth.diagnostics.read",
|
||||
"workspace.manage", "workspace.secrets.manage", "database.manage", "memory.manage", "evidence.manage", "pi.manage", "auth.diagnostics.read",
|
||||
],
|
||||
isAdmin: true,
|
||||
});
|
||||
@@ -206,15 +208,33 @@ test("the session boundary rejects tht maintenance headers outside exact loopbac
|
||||
})).statusCode).toBe(503);
|
||||
});
|
||||
|
||||
test("the session boundary touches a valid cookie session through the bounded Task 7 store operation", async () => {
|
||||
test.each<{
|
||||
name: string;
|
||||
roles: Role[];
|
||||
storedPermissions: Permission[];
|
||||
}>([
|
||||
{ name: "current user", roles: ["user"], storedPermissions: ["session.use"] },
|
||||
{
|
||||
name: "administrator signed in before archive permissions existed",
|
||||
roles: ["admin"],
|
||||
storedPermissions: rolesToPermissions(["admin"]).filter(
|
||||
(permission) => permission !== "memory.manage" && permission !== "evidence.manage",
|
||||
),
|
||||
},
|
||||
{
|
||||
name: "user with obsolete administrative permissions",
|
||||
roles: ["user"],
|
||||
storedPermissions: ["session.use", "memory.manage", "evidence.manage"],
|
||||
},
|
||||
])("the session boundary derives current permissions and touches a valid cookie: $name", async ({ roles, storedPermissions }) => {
|
||||
const sessions = {
|
||||
resolve: vi.fn(async () => ({
|
||||
version: 1,
|
||||
issuer: "local",
|
||||
subject: "user-1",
|
||||
method: "local",
|
||||
roles: ["user"],
|
||||
permissions: ["session.use"],
|
||||
roles,
|
||||
permissions: storedPermissions,
|
||||
userAuthRevision: 1,
|
||||
authConfigRevision: "b".repeat(64),
|
||||
remembered: false,
|
||||
@@ -249,7 +269,13 @@ test("the session boundary touches a valid cookie session through the bounded Ta
|
||||
app.get("/private", async (request) => getPrincipal(request));
|
||||
|
||||
const token = "z".repeat(43);
|
||||
expect((await app.inject({ method: "GET", url: "/private", headers: { cookie: `thothii_session=${token}` } })).statusCode).toBe(200);
|
||||
const response = await app.inject({ method: "GET", url: "/private", headers: { cookie: `thothii_session=${token}` } });
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(response.json()).toMatchObject({
|
||||
roles,
|
||||
permissions: rolesToPermissions(roles),
|
||||
isAdmin: roles.includes("admin"),
|
||||
});
|
||||
expect(sessions.touch).toHaveBeenCalledWith(token);
|
||||
});
|
||||
|
||||
|
||||
@@ -124,8 +124,17 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
|
||||
],
|
||||
relationships: [],
|
||||
};
|
||||
expect(await repository.applySchemaSync(created.id, 1, "columns", [], fullColumnsSnapshot))
|
||||
const memoryCleanupRun = await repository.createSyncRun(created.id, "columns", [], 1);
|
||||
await repository.updateSyncRun(memoryCleanupRun.id, { state: "applying", phase: "applying" });
|
||||
await expect(repository.applySchemaSync(created.id, 999, "columns", [], fullColumnsSnapshot, memoryCleanupRun.id))
|
||||
.resolves.toBeUndefined();
|
||||
expect((await repository.getSyncRun(memoryCleanupRun.id))?.phase).toBe("applying");
|
||||
expect(await repository.applySchemaSync(created.id, 1, "columns", [], fullColumnsSnapshot, memoryCleanupRun.id))
|
||||
.toMatchObject({ created: 4, deleted: 0 });
|
||||
expect((await repository.getSyncRun(memoryCleanupRun.id))?.phase).toBe("memory_cleanup");
|
||||
await repository.interruptActiveSyncRuns();
|
||||
expect(await repository.getSyncRun(memoryCleanupRun.id))
|
||||
.toMatchObject({ state: "interrupted", phase: "memory_cleanup" });
|
||||
expect((await repository.listColumns(created.id, patients.id)).map((column) => ({
|
||||
name: column.name,
|
||||
sensitive: column.sensitive,
|
||||
@@ -233,6 +242,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
|
||||
});
|
||||
expect(await repository.listSyncRuns(created.id)).toEqual([
|
||||
expect.objectContaining({ id: syncRun.id, tableIds: [patients.id] }),
|
||||
expect.objectContaining({ id: memoryCleanupRun.id, phase: "memory_cleanup" }),
|
||||
]);
|
||||
|
||||
const lockedDatabase = await repository.create({
|
||||
|
||||
@@ -117,6 +117,7 @@ async function setup(env: Record<string, string> = {}) {
|
||||
});
|
||||
const introspector: CatalogSchemaIntrospector = { scan };
|
||||
const operations = new CatalogOperationCoordinator();
|
||||
const cleanup = vi.fn(async () => ({ code: 0, stdout: JSON.stringify({ indexed: true, deleted: 0 }), stderr: "" }));
|
||||
const registry = {
|
||||
list: vi.fn(async () => [revision]),
|
||||
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
|
||||
@@ -124,7 +125,7 @@ async function setup(env: Record<string, string> = {}) {
|
||||
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
|
||||
} as unknown as WorkspaceRegistry;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test", ...env }), {
|
||||
thtRunner: {} as never,
|
||||
thtRunner: { withPrincipal: () => ({ runWithRuntimeSnapshot: cleanup }) } as never,
|
||||
workspaceRegistry: registry,
|
||||
workspaceSecretStore: new WorkspaceSecretStore({ root: secretRoot, runtimeRoot, installationId: "test" }),
|
||||
catalogRepository: repository,
|
||||
@@ -133,7 +134,7 @@ async function setup(env: Record<string, string> = {}) {
|
||||
workspaceDiagnoser: vi.fn(),
|
||||
});
|
||||
return {
|
||||
app, repository, database: (await repository.get(created.id))!, scan, operations,
|
||||
app, repository, database: (await repository.get(created.id))!, scan, operations, cleanup,
|
||||
setObserved(next: ObservedSchemaSnapshot) { observed = next; },
|
||||
};
|
||||
}
|
||||
@@ -614,6 +615,45 @@ test("waits for confirmation and rescans before applying destructive changes", a
|
||||
expect(scan).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
test("retries committed Memory cleanup with the original removals without rescanning", async () => {
|
||||
const { app, repository, database, scan, setObserved, cleanup } = await setup();
|
||||
await seedCatalog(repository, database);
|
||||
const observed = snapshot();
|
||||
observed.tables = observed.tables.filter(t => t.name !== "visits");
|
||||
observed.columns = observed.columns.filter(c => c.tableName !== "visits");
|
||||
observed.relationships = [];
|
||||
setObserved(observed);
|
||||
cleanup.mockResolvedValueOnce({ code: 0, stdout: JSON.stringify({ indexed: false, deleted: 2 }), stderr: "" });
|
||||
const started = await app.inject({ method: "POST", url: `/catalog/databases/${database.id}/sync-runs`,
|
||||
payload: { version: database.version, scope: "all", tableIds: [] } });
|
||||
const waiting = await waitFor(repository, started.json().id, "awaiting_confirmation");
|
||||
expect(cleanup).not.toHaveBeenCalled();
|
||||
await app.inject({ method: "POST", url: `/catalog/sync-runs/${waiting.id}/confirm`,
|
||||
payload: { confirmationToken: waiting.confirmationToken } });
|
||||
const failed = await waitFor(repository, waiting.id, "failed");
|
||||
expect(failed).toMatchObject({ phase: "memory_cleanup", errorCode: "memory_cleanup_pending",
|
||||
plannedDiff: { deletedTables: ["visits"] } });
|
||||
expect((await repository.listTables(database.id)).map(t => t.name)).toEqual(["patients"]);
|
||||
const callsBeforeRetry = scan.mock.calls.length;
|
||||
const blocked = await app.inject({ method: "POST", url: `/catalog/databases/${database.id}/sync-runs`,
|
||||
payload: { version: database.version, scope: "all", tableIds: [] } });
|
||||
expect(blocked.statusCode).toBe(409);
|
||||
// Simulate startup recovery and an unreachable DWH after schema application.
|
||||
await repository.interruptActiveSyncRuns();
|
||||
scan.mockRejectedValue(new Error("DWH unavailable"));
|
||||
cleanup.mockResolvedValue({ code: 0, stdout: JSON.stringify({ indexed: true, deleted: 2 }), stderr: "" });
|
||||
const retried = await app.inject({ method: "POST", url: `/catalog/sync-runs/${waiting.id}/retry` });
|
||||
expect(retried.statusCode).toBe(202);
|
||||
const completed = await waitFor(repository, waiting.id, "succeeded");
|
||||
expect(completed.counts).toMatchObject({ memoryDeleted: 2 });
|
||||
expect(scan.mock.calls.length).toBe(callsBeforeRetry);
|
||||
expect(cleanup).toHaveBeenCalledTimes(2);
|
||||
expect(cleanup.mock.calls[1]).toEqual(cleanup.mock.calls[0]);
|
||||
const request = JSON.parse((cleanup.mock.calls[0] as unknown as string[])[1]!).request;
|
||||
expect(request).toMatchObject({ sync_id: waiting.id, database: "warehouse",
|
||||
schema_name: "datawarehouse", removed_tables: ["visits"] });
|
||||
});
|
||||
|
||||
test("deletes every catalog table for multiple selected databases and cascades dependent metadata", async () => {
|
||||
const { app, repository, database } = await setup();
|
||||
const second = await repository.create({
|
||||
|
||||
@@ -36,7 +36,7 @@ test("loop F1: crea sessione → SSE riceve il widget → risponde → il modell
|
||||
ollamaEnsure: async () => ({ ok: true }),
|
||||
searchPack: async () => {},
|
||||
sessionNew: async () => ({ id: "s1" }),
|
||||
sessionShow: async (_id: string) => ({ id: "s1", provider: undefined, model: undefined, thinking: undefined }),
|
||||
sessionShow: async (_id: string) => ({ id: "s1", interaction_language: "en", provider: undefined, model: undefined, thinking: undefined }),
|
||||
sessionList: async () => [],
|
||||
} as any,
|
||||
spawnFn: () => spawn("node", [FAKE, SCRIPT]) as any,
|
||||
@@ -48,7 +48,7 @@ test("loop F1: crea sessione → SSE riceve il widget → risponde → il modell
|
||||
const created = await fetch(`${base}/sessions`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ workspace: "w", question: "q" }),
|
||||
body: JSON.stringify({ workspace: "w", question: "q", interactionLanguage: "en" }),
|
||||
});
|
||||
expect(created.status).toBe(200);
|
||||
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
import Fastify from "fastify";
|
||||
import { afterEach, expect, test, vi } from "vitest";
|
||||
import { evidenceRoutes } from "../src/routes/evidence.js";
|
||||
import type { ThtRunner } from "../src/tht/tht-runner.js";
|
||||
import type { PrincipalContext } from "../src/auth/principal.js";
|
||||
|
||||
const apps: ReturnType<typeof Fastify>[] = [];
|
||||
afterEach(async () => { await Promise.all(apps.splice(0).map(app => app.close())); });
|
||||
function setup(admin = true) {
|
||||
const app = Fastify(); apps.push(app);
|
||||
const principal: PrincipalContext = { issuer: "test", subject: "curator", roles: [admin ? "admin" : "user"],
|
||||
permissions: admin ? ["evidence.manage"] : ["session.use"], isAdmin: admin };
|
||||
app.addHook("preHandler", async request => { request.principal = principal; });
|
||||
const run = vi.fn(async () => ({ code: 0, stdout: JSON.stringify({ items: [], total: 0 }), stderr: "" }));
|
||||
const withPrincipal = vi.fn(() => ({ runWithRuntimeSnapshot: run }) as unknown as ThtRunner);
|
||||
const consolidateEvidence = vi.fn();
|
||||
const evidenceSources = vi.fn().mockResolvedValue({status: "succeeded"});
|
||||
evidenceRoutes(app, { runner: { withPrincipal }, registryRoot: "/data/registry", hostRegistryRoot: "/srv/Thoth workspaces",
|
||||
registry: { list: async () => [{ id: "sales", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/missing" }] },
|
||||
service: { consolidateEvidence, evidenceSources } });
|
||||
return { app, run, withPrincipal, principal, consolidateEvidence, evidenceSources };
|
||||
}
|
||||
test("browses complete local files with host paths, without a database runtime", async () => {
|
||||
const { app, run, withPrincipal, principal } = setup();
|
||||
const response = await app.inject("/workspaces/sales/evidence?kind=domain&concept=orders&page=2");
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(withPrincipal).toHaveBeenCalledWith(principal);
|
||||
const [argv, snapshot] = run.mock.calls[0] as unknown as [string[], string];
|
||||
expect(argv).toEqual(["evidence", "admin", "--workspace", "sales"]);
|
||||
expect(JSON.parse(snapshot)).toEqual({ root: "/data/registry/repo/sales", query: { kind: "domain", concept: "orders", page: 2 } });
|
||||
expect(response.json().location.workspace).toBe("/srv/Thoth workspaces/repo/sales");
|
||||
expect(response.json().location.git_commands[0]).toContain("git -C '/srv/Thoth workspaces/repo'");
|
||||
});
|
||||
|
||||
test("source actions require admin, bind the actor, and reject cross-workspace or arbitrary inputs", async () => {
|
||||
const denied = setup(false);
|
||||
expect((await denied.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload: {action: "refresh"}})).statusCode).toBe(403);
|
||||
expect(denied.evidenceSources).not.toHaveBeenCalled();
|
||||
const allowed = setup();
|
||||
expect((await allowed.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload: {action: "refresh"}})).statusCode).toBe(200);
|
||||
expect(allowed.evidenceSources).toHaveBeenCalledWith({workspaceId: "sales", action: "refresh", actor: "curator"});
|
||||
const choice = {action: "decide", sourceId: "a".repeat(64), revision: "b".repeat(64), decision: "keep"};
|
||||
expect((await allowed.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload: choice})).statusCode).toBe(200);
|
||||
expect(allowed.evidenceSources).toHaveBeenLastCalledWith({...choice, workspaceId: "sales", actor: "curator"});
|
||||
for (const payload of [{action: "refresh", url: "http://localhost"}, {...choice, actor: "forged"}, {...choice, revision: "../other"}]) {
|
||||
expect((await allowed.app.inject({method: "POST", url: "/workspaces/sales/evidence/sources", payload})).statusCode).toBe(400);
|
||||
}
|
||||
expect((await allowed.app.inject({method: "POST", url: "/workspaces/other/evidence/sources", payload: choice})).statusCode).toBe(404);
|
||||
expect(allowed.evidenceSources).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
test.each(["/", "/evidence:rule", "/consolidate"])("refuses non-admin access %s", async suffix => {
|
||||
const { app, run, consolidateEvidence } = setup(false);
|
||||
const response = await app.inject({ method: suffix === "/consolidate" ? "POST" : "GET", url: `/workspaces/sales/evidence${suffix === "/" ? "" : suffix}` });
|
||||
expect(response.statusCode).toBe(403); expect(run).not.toHaveBeenCalled(); expect(consolidateEvidence).not.toHaveBeenCalled();
|
||||
});
|
||||
test("rejects workspace escape, unknown workspace and unsupported filters", async () => {
|
||||
const { app, run } = setup();
|
||||
expect((await app.inject("/workspaces/foreign/evidence")).statusCode).toBe(404);
|
||||
expect((await app.inject("/workspaces/sales/evidence?root=/private")).statusCode).toBe(400);
|
||||
expect((await app.inject("/workspaces/sales/evidence?page_size=101")).statusCode).toBe(400);
|
||||
expect(run).not.toHaveBeenCalled();
|
||||
});
|
||||
test("a missing unit is 404 and consolidation preserves the partial outcome", async () => {
|
||||
const { app, consolidateEvidence } = setup();
|
||||
expect((await app.inject("/workspaces/sales/evidence/evidence:missing")).statusCode).toBe(404);
|
||||
consolidateEvidence.mockResolvedValue({ status: "failed", warnings: ["Saved; retry indexing."] });
|
||||
const response = await app.inject({ method: "POST", url: "/workspaces/sales/evidence/consolidate" });
|
||||
expect(response.json()).toEqual({ status: "failed", warnings: ["Saved; retry indexing."] });
|
||||
});
|
||||
@@ -56,7 +56,7 @@ test("catalog listing translates upstream Pi IDs back to canonical model keys",
|
||||
session: { reasoning: true, contextWindow: 32768, maxTokens: 8192 },
|
||||
};
|
||||
const modelCatalog: RuntimeModelCatalog = {
|
||||
defaultSession: model.id, defaultMetadataGeneration: null, embedding: null,
|
||||
defaultInteraction: model.id, embedding: null,
|
||||
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
|
||||
};
|
||||
try {
|
||||
@@ -75,6 +75,38 @@ test("catalog listing translates upstream Pi IDs back to canonical model keys",
|
||||
}
|
||||
});
|
||||
|
||||
test("catalog listing supplies the shared DeepSeek key without requiring Pi auth", async () => {
|
||||
const script = scriptWith([{ provider: "deepseek", id: "deepseek-v4-pro", name: "DeepSeek V4 Pro" }]);
|
||||
const secret = join(path.dirname(script), "thothii.secrets");
|
||||
writeFileSync(secret, "DEEPSEEK_API_KEY=shared-key\nOPENAI_API_KEY=unrelated-key\n", { mode: 0o600 });
|
||||
const model: RuntimeModel = {
|
||||
id: "deepseek/deepseek-v4-pro", provider: "deepseek", model: "deepseek-v4-pro",
|
||||
label: "DeepSeek V4 Pro", upstreamModel: "deepseek-v4-pro",
|
||||
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
|
||||
sessionAdapter: { mode: "pi_builtin" }, session: { reasoning: false },
|
||||
};
|
||||
const modelCatalog: RuntimeModelCatalog = {
|
||||
defaultInteraction: model.id, embedding: null,
|
||||
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
|
||||
};
|
||||
try {
|
||||
const lister = createPiModelLister(loadConfig({ THT_SECRETS_FILE: secret }), {
|
||||
...noManagedModels, modelCatalog, loadEnabledModels: enabled(model.id),
|
||||
spawnFn: (_command, _args, options) => {
|
||||
expect(options.env.DEEPSEEK_API_KEY).toBe("shared-key");
|
||||
expect(options.env).not.toHaveProperty("OPENAI_API_KEY");
|
||||
expect(options.env).not.toHaveProperty("THT_SECRETS_FILE");
|
||||
return spawn("node", [FAKE, script], { env: options.env }) as any;
|
||||
},
|
||||
});
|
||||
await expect(lister()).resolves.toEqual([{
|
||||
provider: model.provider, id: model.model, name: model.label, reasoning: false,
|
||||
}]);
|
||||
} finally {
|
||||
rmSync(path.dirname(script), { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("createPiModelLister caches within ttl (spawns once for two calls)", async () => {
|
||||
const script = scriptWith([{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }]);
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
import Fastify from "fastify";
|
||||
import { afterEach, expect, test, vi } from "vitest";
|
||||
import { memoryRoutes } from "../src/routes/memory.js";
|
||||
import { DEFAULT_SEMANTIC_RUNTIME } from "../src/workspaces/runtime-renderer.js";
|
||||
import type { ThtRunner } from "../src/tht/tht-runner.js";
|
||||
import type { PrincipalContext } from "../src/auth/principal.js";
|
||||
|
||||
const apps: ReturnType<typeof Fastify>[] = [];
|
||||
afterEach(async () => { await Promise.all(apps.splice(0).map(app => app.close())); });
|
||||
const id = "mem-11111111-1111-4111-8111-111111111111";
|
||||
const input = { family: "domain_clarification", subject: "Order", scope: "Sales" };
|
||||
|
||||
function setup(admin = true) {
|
||||
const app = Fastify(); apps.push(app);
|
||||
const principal: PrincipalContext = { issuer: "test", subject: "operator", roles: [admin ? "admin" : "user"],
|
||||
permissions: admin ? ["memory.manage"] : ["session.use"], isAdmin: admin };
|
||||
app.addHook("preHandler", async request => { request.principal = principal; });
|
||||
const run = vi.fn(async () => ({ code: 0, stdout: JSON.stringify({ items: [], total: 0 }), stderr: "" }));
|
||||
const bound = { runWithRuntimeSnapshot: run } as unknown as ThtRunner;
|
||||
const withPrincipal = vi.fn(() => bound);
|
||||
memoryRoutes(app, { runner: { withPrincipal }, runtime: DEFAULT_SEMANTIC_RUNTIME,
|
||||
registry: {
|
||||
list: async () => [{ id: "sales", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/missing" }],
|
||||
read: vi.fn().mockResolvedValue({ workspace: { workspace: { language: "it" } } }),
|
||||
} });
|
||||
return { app, run, withPrincipal, principal };
|
||||
}
|
||||
|
||||
test("list binds principal and workspace without requiring a DWH runtime", async () => {
|
||||
const { app, run, withPrincipal, principal } = setup();
|
||||
const response = await app.inject("/workspaces/sales/memory?q=Order&page=2&column=id");
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(withPrincipal).toHaveBeenCalledWith(principal);
|
||||
const [args, snapshot] = run.mock.calls[0] as unknown as [string[], string];
|
||||
expect(args).toEqual(["memory", "admin", "--workspace", "sales"]);
|
||||
expect(JSON.parse(snapshot)).toMatchObject({ action: "list", request: { q: "Order", page: 2, column: "id" } });
|
||||
expect(JSON.parse(snapshot).runtime.memoryLanguage).toBe("it");
|
||||
});
|
||||
|
||||
test.each([
|
||||
["GET", ""], ["POST", ""], ["GET", "/pending"], ["GET", `/${id}`],
|
||||
["PUT", `/${id}`], ["DELETE", `/${id}`], ["POST", `/${id}/retry`],
|
||||
] as const)("non-admin cannot %s %s", async (method, suffix) => {
|
||||
const { app, run } = setup(false);
|
||||
const response = await app.inject({ method, url: `/workspaces/sales/memory${suffix}`,
|
||||
...(method === "PUT" || method === "POST" && suffix === "" ? { payload: input } : {}) });
|
||||
expect(response.statusCode).toBe(403); expect(run).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("unknown workspace and invalid input do not invoke the harness", async () => {
|
||||
const { app, run } = setup();
|
||||
expect((await app.inject("/workspaces/missing/memory")).statusCode).toBe(404);
|
||||
expect((await app.inject("/workspaces/sales/memory?page_size=101")).statusCode).toBe(400);
|
||||
expect((await app.inject({ method: "POST", url: "/workspaces/sales/memory",
|
||||
payload: { ...input, workspace_id: "foreign" } })).statusCode).toBe(400);
|
||||
expect(run).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("create preserves saved-but-unindexed outcome, update carries complete related changes", async () => {
|
||||
const { app, run } = setup();
|
||||
run.mockResolvedValue({ code: 0, stdout: JSON.stringify({ id, saved: true, indexed: false }), stderr: "" });
|
||||
const result = await app.inject({ method: "POST", url: "/workspaces/sales/memory", payload: input });
|
||||
expect(result.statusCode).toBe(201); expect(result.json()).toEqual({ id, saved: true, indexed: false });
|
||||
await app.inject({ method: "PUT", url: `/workspaces/sales/memory/${id}`, payload: {
|
||||
...input, links: [{ target_id: id, meaning: "Related" }], dependencies: [{ database: "dwh" }],
|
||||
} });
|
||||
const [, snapshot] = run.mock.calls[1] as unknown as [string[], string];
|
||||
expect(JSON.parse(snapshot)).toMatchObject({ action: "update", request: { id, card: {
|
||||
links: [{ target_id: id, meaning: "Related" }], dependencies: [{ database: "dwh" }],
|
||||
} } });
|
||||
});
|
||||
|
||||
test.each([["memory_not_found", 404], ["memory_conflict", 409], ["memory_unavailable", 503]])(
|
||||
"maps %s without leaking stderr", async (code, status) => {
|
||||
const { app, run } = setup();
|
||||
run.mockResolvedValue({ code: 1, stdout: JSON.stringify({ code, message: "sensitive detail" }), stderr: "secret" });
|
||||
const result = await app.inject("/workspaces/sales/memory");
|
||||
expect(result.statusCode).toBe(status); expect(result.json().code).toBe(code);
|
||||
expect(result.body).not.toMatch(/secret|sensitive/);
|
||||
},
|
||||
);
|
||||
@@ -20,9 +20,8 @@ function runtimeCatalog(overrides: Record<string, unknown> = {}, secrets = "OPEN
|
||||
const catalogFile = join(root, "catalog.json");
|
||||
const secretsFile = join(root, "thothii.secrets");
|
||||
const catalog = {
|
||||
schemaVersion: 1,
|
||||
defaultSession: "zai/glm-5.3",
|
||||
defaultMetadataGeneration: "zai/glm-5.3",
|
||||
schemaVersion: 2,
|
||||
defaultInteraction: "zai/glm-5.3",
|
||||
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||
models: [
|
||||
{
|
||||
@@ -55,8 +54,8 @@ test("loads session default and safe metadata choices from the normalized runtim
|
||||
const runtime = loadRuntimeModelCatalog(catalogFile);
|
||||
const metadata = loadMetadataGenerationModels({ catalogFile, secretsFile });
|
||||
|
||||
expect(runtime.defaultSession).toBe("zai/glm-5.3");
|
||||
expect(runtime.hasSession("deepseek/deepseek-v4-pro")).toBe(true);
|
||||
expect(runtime.defaultInteraction).toBe("zai/glm-5.3");
|
||||
expect(runtime.hasSession("deepseek/deepseek-v4-pro")).toBe(false);
|
||||
expect(metadata.catalog()).toEqual({
|
||||
models: [{ id: "zai/glm-5.3", label: "GLM 5.3" }],
|
||||
default: "zai/glm-5.3",
|
||||
@@ -69,14 +68,65 @@ test("loads session default and safe metadata choices from the normalized runtim
|
||||
expect(() => metadata.resolve("zai/missing")).toThrow(MetadataGenerationModelUnavailableError);
|
||||
});
|
||||
|
||||
test("DeepSeek uses one identity and bundle credential for native Pi and LiteLLM", () => {
|
||||
const models = ["deepseek-v4-pro", "deepseek-v4-flash"].map((model) => ({
|
||||
id: `deepseek/${model}`, provider: "deepseek", model, label: model, upstreamModel: model,
|
||||
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
|
||||
sessionAdapter: { mode: "pi_builtin" }, metadataAdapter: { litellmProvider: "deepseek" },
|
||||
session: { reasoning: false }, metadataGeneration: { disableThinking: false },
|
||||
}));
|
||||
const files = runtimeCatalog({ defaultInteraction: models[0].id, models }, "DEEPSEEK_API_KEY=shared-key\n");
|
||||
const runtime = loadRuntimeModelCatalog(files.catalogFile);
|
||||
const metadata = loadMetadataGenerationModels(files);
|
||||
expect(runtime.sessionModels().map((model) => model.id)).toEqual(metadata.catalog().models.map((model) => model.id));
|
||||
expect(metadata.catalog().default).toBe(runtime.defaultInteraction);
|
||||
for (const model of models) {
|
||||
expect(metadata.resolve(model.id)).toMatchObject({
|
||||
id: model.id, provider: "deepseek", model: model.model,
|
||||
apiKeyEnv: "DEEPSEEK_API_KEY", apiKey: "shared-key",
|
||||
});
|
||||
}
|
||||
expect(() => metadata.resolve("deepseek-metadata/deepseek-v4-pro"))
|
||||
.toThrow(MetadataGenerationModelUnavailableError);
|
||||
});
|
||||
|
||||
test("returns empty catalogs when no runtime projection is configured", () => {
|
||||
expect(loadRuntimeModelCatalog().defaultSession).toBeNull();
|
||||
expect(loadRuntimeModelCatalog().defaultInteraction).toBeNull();
|
||||
expect(loadMetadataGenerationModels({}).catalog()).toEqual({ models: [], default: null });
|
||||
});
|
||||
|
||||
test.each([
|
||||
["qwen-chat-template", true, true],
|
||||
["qwen", true, true],
|
||||
["qwen-typo", true, false],
|
||||
["qwen-chat-template", false, false],
|
||||
])("validates Qwen thinking format %s with reasoning=%s", (thinkingFormat, reasoning, valid) => {
|
||||
const { catalogFile } = runtimeCatalog({
|
||||
defaultInteraction: "local/qwen",
|
||||
models: [{
|
||||
id: "local/qwen", provider: "local", model: "qwen", label: "Qwen",
|
||||
upstreamModel: "qwen", endpoint: { baseUrl: "http://localhost:8000/v1" },
|
||||
authentication: { mode: "none" }, sessionAdapter: { mode: "openai_compatible" },
|
||||
session: {
|
||||
reasoning, contextWindow: 32768, maxTokens: 8192,
|
||||
compatibility: {
|
||||
supportsDeveloperRole: false, supportsReasoningEffort: false,
|
||||
supportsStore: false, maxTokensField: "max_tokens", thinkingFormat,
|
||||
},
|
||||
},
|
||||
}],
|
||||
});
|
||||
if (valid) {
|
||||
expect(loadRuntimeModelCatalog(catalogFile).sessionModels()[0].session?.compatibility)
|
||||
.toMatchObject({ thinkingFormat });
|
||||
} else {
|
||||
expect(() => loadRuntimeModelCatalog(catalogFile)).toThrow("runtime model catalog is invalid");
|
||||
}
|
||||
});
|
||||
|
||||
test("rejects a drifted default and an unprotected projection", () => {
|
||||
const drifted = runtimeCatalog({ defaultSession: "zai/missing" });
|
||||
expect(() => loadRuntimeModelCatalog(drifted.catalogFile)).toThrow("session default is invalid");
|
||||
const drifted = runtimeCatalog({ defaultInteraction: "zai/missing" });
|
||||
expect(() => loadRuntimeModelCatalog(drifted.catalogFile)).toThrow("interaction default is invalid");
|
||||
|
||||
const unprotected = runtimeCatalog();
|
||||
chmodSync(unprotected.catalogFile, 0o666);
|
||||
@@ -85,7 +135,7 @@ test("rejects a drifted default and an unprotected projection", () => {
|
||||
|
||||
test("rejects authentication semantics that cannot come from the installation catalog", () => {
|
||||
const invalid = runtimeCatalog({
|
||||
defaultMetadataGeneration: undefined,
|
||||
defaultInteraction: undefined,
|
||||
models: [{
|
||||
id: "zai/glm-5.3",
|
||||
provider: "zai",
|
||||
@@ -112,3 +162,13 @@ test("splits canonical session identities without provider aliases", () => {
|
||||
expect(splitCanonicalModelId("zai/glm-5.3")).toEqual({ provider: "zai", model: "glm-5.3" });
|
||||
expect(() => splitCanonicalModelId("glm-5.3")).toThrow("model identity is invalid");
|
||||
});
|
||||
|
||||
test("rejects a default supported by only one configured use", () => {
|
||||
const { catalogFile } = runtimeCatalog({ defaultInteraction: "deepseek/deepseek-v4-pro" });
|
||||
expect(() => loadRuntimeModelCatalog(catalogFile)).toThrow("interaction default is invalid");
|
||||
});
|
||||
|
||||
test("requires regenerated runtime schema v2 rather than interpreting two legacy defaults", () => {
|
||||
const { catalogFile } = runtimeCatalog({ schemaVersion: 1, defaultSession: "zai/glm-5.3" });
|
||||
expect(() => loadRuntimeModelCatalog(catalogFile)).toThrow("runtime model catalog is invalid");
|
||||
});
|
||||
|
||||
@@ -5,6 +5,8 @@ const fakes = vi.hoisted(() => ({
|
||||
catalogRepository: { close: vi.fn(async () => {}) },
|
||||
createCatalogRepository: vi.fn(),
|
||||
runnerConfig: undefined as Record<string, unknown> | undefined,
|
||||
acquireWorkspaceRuntime: vi.fn(),
|
||||
releaseWorkspaceRuntime: vi.fn(),
|
||||
run: vi.fn(async () => ({
|
||||
code: 0,
|
||||
stdout: JSON.stringify({ ok: true }),
|
||||
@@ -24,6 +26,8 @@ vi.mock("../src/tht/tht-runner.js", () => ({
|
||||
|
||||
run = fakes.run;
|
||||
|
||||
acquireWorkspaceRuntime = fakes.acquireWorkspaceRuntime;
|
||||
|
||||
withPrincipal() {
|
||||
return this;
|
||||
}
|
||||
@@ -67,6 +71,12 @@ beforeEach(() => {
|
||||
fakes.createCatalogRepository.mockReset();
|
||||
fakes.createCatalogRepository.mockReturnValue(fakes.catalogRepository);
|
||||
fakes.run.mockClear();
|
||||
fakes.acquireWorkspaceRuntime.mockReset();
|
||||
fakes.acquireWorkspaceRuntime.mockResolvedValue({
|
||||
path: "/data/workspace-registry/snapshots/runtime/workspace.yaml",
|
||||
release: fakes.releaseWorkspaceRuntime,
|
||||
});
|
||||
fakes.releaseWorkspaceRuntime.mockClear();
|
||||
fakes.runnerConfig = undefined;
|
||||
});
|
||||
|
||||
@@ -78,5 +88,13 @@ test("workflow doctor gives schema-v4 runtime rendering a live Catalog repositor
|
||||
|
||||
expect(fakes.createCatalogRepository).toHaveBeenCalledWith(config.catalogDatabase);
|
||||
expect(fakes.runnerConfig?.catalogRepository).toBe(fakes.catalogRepository);
|
||||
expect(fakes.acquireWorkspaceRuntime).toHaveBeenCalledWith(
|
||||
"/data/workspace-registry/snapshots/revision/workspace.yaml",
|
||||
);
|
||||
expect(fakes.run).toHaveBeenCalledWith(
|
||||
["doctor", "--json"],
|
||||
"/data/workspace-registry/snapshots/runtime/workspace.yaml",
|
||||
);
|
||||
expect(fakes.releaseWorkspaceRuntime).toHaveBeenCalledOnce();
|
||||
expect(fakes.catalogRepository.close).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { expect, test, vi } from "vitest";
|
||||
@@ -7,7 +7,34 @@ import {
|
||||
createPiManagement,
|
||||
type PiExecFile,
|
||||
} from "../src/pi/management.js";
|
||||
import type { RuntimeModelCatalog } from "../src/models/runtime-model-catalog.js";
|
||||
import type { RuntimeModel, RuntimeModelCatalog } from "../src/models/runtime-model-catalog.js";
|
||||
|
||||
test.each([false, true])("catalog credential status ignores legacy Pi auth, missing bundle key: %s", async (missingKey) => {
|
||||
const root = mkdtempSync(join(tmpdir(), "tht-catalog-status-"));
|
||||
writeFileSync(join(root, "auth.json"), JSON.stringify({ deepseek: { type: "api_key", key: "stale-key" } }), { mode: 0o600 });
|
||||
const secret = join(root, "thothii.secrets");
|
||||
writeFileSync(secret, missingKey ? "THT_MODEL_API_KEY=legacy-key\n" : "DEEPSEEK_API_KEY=shared-key\n", { mode: 0o600 });
|
||||
vi.stubEnv("PI_CODING_AGENT_DIR", root);
|
||||
const model: RuntimeModel = {
|
||||
id: "deepseek/deepseek-v4-pro", provider: "deepseek", model: "deepseek-v4-pro",
|
||||
label: "DeepSeek", upstreamModel: "deepseek-v4-pro",
|
||||
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
|
||||
sessionAdapter: { mode: "pi_builtin" }, session: { reasoning: false },
|
||||
};
|
||||
try {
|
||||
const service = createPiManagement(loadConfig({ THT_SECRETS_FILE: secret }), {
|
||||
modelCatalog: {
|
||||
defaultInteraction: model.id, embedding: null,
|
||||
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
|
||||
},
|
||||
execute: successfulExec([]), readSettings: () => ({ thinking: "medium" }),
|
||||
});
|
||||
expect((await service.status()).credentials).toBe(missingKey ? "missing" : "present");
|
||||
} finally {
|
||||
vi.unstubAllEnvs();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
function configFor(settingsFile = join(mkdtempSync(join(tmpdir(), "tht-pi-management-")), "settings.json")) {
|
||||
return loadConfig({
|
||||
@@ -15,12 +42,12 @@ function configFor(settingsFile = join(mkdtempSync(join(tmpdir(), "tht-pi-manage
|
||||
SETTINGS_FILE: settingsFile,
|
||||
PI_BIN: "/usr/local/bin/pi",
|
||||
PI_MANAGEMENT_TIMEOUT_MS: "750",
|
||||
THT_HOST_PLATFORM: "linux",
|
||||
});
|
||||
}
|
||||
|
||||
const modelCatalog: RuntimeModelCatalog = {
|
||||
defaultSession: "zai/glm-5.2",
|
||||
defaultMetadataGeneration: null,
|
||||
defaultInteraction: "zai/glm-5.2",
|
||||
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||
sessionModels: () => [],
|
||||
metadataModels: () => [],
|
||||
@@ -34,6 +61,16 @@ function successfulExec(calls: Array<{ command: string; args: string[]; timeout:
|
||||
};
|
||||
}
|
||||
|
||||
test.each([
|
||||
["linux", "linux"], ["darwin", "macos"], ["windows", "windows"],
|
||||
])("reports installation host %s independently of the backend container OS", async (host, expected) => {
|
||||
const service = createPiManagement(loadConfig({ THT_HOST_PLATFORM: host }), {
|
||||
modelCatalog, execute: successfulExec([]), readSettings: () => ({ thinking: "medium" }),
|
||||
credentialStatus: () => "missing",
|
||||
});
|
||||
expect((await service.status()).hostPlatform).toBe(expected);
|
||||
});
|
||||
|
||||
// Catches a Pi executable that emits unexpected text or is invoked through a shell, which could
|
||||
// turn a version display into a command-injection or information-disclosure surface.
|
||||
test("status parses only a Pi version from a fixed execFile argument array", async () => {
|
||||
@@ -47,6 +84,7 @@ test("status parses only a Pi version from a fixed execFile argument array", asy
|
||||
});
|
||||
|
||||
await expect(service.status()).resolves.toEqual({
|
||||
hostPlatform: "linux",
|
||||
version: "0.80.3",
|
||||
ready: true,
|
||||
credentials: "missing",
|
||||
@@ -78,6 +116,7 @@ test.each(["present", "missing"] as const)(
|
||||
|
||||
const status = await service.status();
|
||||
expect(status).toEqual({
|
||||
hostPlatform: "linux",
|
||||
version: "0.80.3",
|
||||
ready: true,
|
||||
credentials,
|
||||
|
||||
@@ -190,6 +190,29 @@ test("Pi receives the leased workspace runtime config and releases it on direct
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
test("Pi language launch hint comes from the session options and never ambient environment", () => {
|
||||
const previous = process.env.THT_INTERACTION_LANGUAGE;
|
||||
process.env.THT_INTERACTION_LANGUAGE = "it";
|
||||
const environments: NodeJS.ProcessEnv[] = [];
|
||||
const mgr = new PiProcessManager(loadConfig({}), {
|
||||
spawnFn: (_command, _args, options) => {
|
||||
environments.push(options.env);
|
||||
return recordingChild() as any;
|
||||
},
|
||||
});
|
||||
try {
|
||||
mgr.createFor("explicit-language", { interactionLanguage: "en" });
|
||||
mgr.teardown("explicit-language");
|
||||
mgr.createFor("manifest-resolved-in-gate");
|
||||
mgr.teardown("manifest-resolved-in-gate");
|
||||
expect(environments[0].THT_INTERACTION_LANGUAGE).toBe("en");
|
||||
expect(environments[1]).not.toHaveProperty("THT_INTERACTION_LANGUAGE");
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.THT_INTERACTION_LANGUAGE;
|
||||
else process.env.THT_INTERACTION_LANGUAGE = previous;
|
||||
}
|
||||
});
|
||||
|
||||
test("a close-only child event releases its temporary Pi agent snapshot", () => {
|
||||
const child = recordingChild();
|
||||
let snapshotDir: string | undefined;
|
||||
@@ -642,27 +665,33 @@ test("session Pi spawn reads the single secret bundle and scrubs its path", asyn
|
||||
}
|
||||
});
|
||||
|
||||
test("session Pi spawn resolves the selected catalog credential from the secret bundle", () => {
|
||||
test.each([false, true])("session Pi uses the catalog bundle despite stale Pi auth: %s", (missingKey) => {
|
||||
const root = mkdtempSync(path.join(tmpdir(), "thothii-catalog-credential-"));
|
||||
const agentDir = path.join(root, "agent");
|
||||
mkdirSync(agentDir, { mode: 0o700 });
|
||||
writeFileSync(path.join(agentDir, "auth.json"), "{}\n", { mode: 0o600 });
|
||||
const originalAuth = JSON.stringify({
|
||||
deepseek: { type: "api_key", key: "stale-key" },
|
||||
anthropic: { type: "api_key", key: "unrelated-key" },
|
||||
});
|
||||
writeFileSync(path.join(agentDir, "auth.json"), originalAuth, { mode: 0o600 });
|
||||
writeFileSync(path.join(agentDir, "models.json"), '{"providers":{}}\n', { mode: 0o600 });
|
||||
const secret = path.join(root, "thothii.secrets");
|
||||
writeFileSync(secret, "ZAI_API_KEY=catalog-secret\nTHT_MODEL_API_KEY=legacy-secret\n", { mode: 0o600 });
|
||||
writeFileSync(secret, missingKey ? "THT_MODEL_API_KEY=legacy-secret\n"
|
||||
: "DEEPSEEK_API_KEY=catalog-secret\nTHT_MODEL_API_KEY=legacy-secret\n", { mode: 0o600 });
|
||||
const legacyKey = path.join(root, "legacy-key");
|
||||
writeFileSync(legacyKey, "legacy-file-key", { mode: 0o600 });
|
||||
const model: RuntimeModel = {
|
||||
id: "openai/test-model",
|
||||
provider: "openai",
|
||||
model: "test-model",
|
||||
label: "Test model",
|
||||
upstreamModel: "test-model",
|
||||
authentication: { mode: "secret_env", apiKeyEnv: "ZAI_API_KEY" },
|
||||
id: "deepseek/deepseek-v4-pro",
|
||||
provider: "deepseek",
|
||||
model: "deepseek-v4-pro",
|
||||
label: "DeepSeek V4 Pro",
|
||||
upstreamModel: "deepseek-v4-pro",
|
||||
authentication: { mode: "secret_env", apiKeyEnv: "DEEPSEEK_API_KEY" },
|
||||
sessionAdapter: { mode: "pi_builtin" },
|
||||
session: { reasoning: false },
|
||||
};
|
||||
const modelCatalog: RuntimeModelCatalog = {
|
||||
defaultSession: model.id,
|
||||
defaultMetadataGeneration: null,
|
||||
defaultInteraction: model.id,
|
||||
embedding: null,
|
||||
sessionModels: () => [model],
|
||||
metadataModels: () => [],
|
||||
@@ -672,17 +701,26 @@ test("session Pi spawn resolves the selected catalog credential from the secret
|
||||
const child = recordingChild();
|
||||
child.stderr.resume = () => {};
|
||||
vi.stubEnv("PI_CODING_AGENT_DIR", agentDir);
|
||||
const mgr = new PiProcessManager(loadConfig({ THT_SECRETS_FILE: secret }), {
|
||||
const mgr = new PiProcessManager(loadConfig({ THT_SECRETS_FILE: secret, THT_MODEL_API_KEY_FILE: legacyKey }), {
|
||||
modelCatalog,
|
||||
authProviders: () => new Set(),
|
||||
authProviders: () => new Set(["deepseek"]),
|
||||
spawnFn: (...args: any[]) => { calls.push(args); return child as any; },
|
||||
});
|
||||
try {
|
||||
mgr.createFor("catalog-credential", { provider: "openai", model: "test-model" });
|
||||
expect(calls[0][2].env.ZAI_API_KEY).toBe("catalog-secret");
|
||||
const create = () => mgr.createFor("catalog-credential", { provider: model.provider, model: model.model });
|
||||
if (missingKey) {
|
||||
expect(create).toThrow("model provider credential is unavailable");
|
||||
expect(calls).toHaveLength(0);
|
||||
return;
|
||||
}
|
||||
create();
|
||||
expect(calls[0][2].env.DEEPSEEK_API_KEY).toBe("catalog-secret");
|
||||
expect(calls[0][2].env).not.toHaveProperty("OPENAI_API_KEY");
|
||||
expect(calls[0][2].env).not.toHaveProperty("THT_MODEL_API_KEY");
|
||||
expect(JSON.parse(readFileSync(path.join(calls[0][2].env.PI_CODING_AGENT_DIR, "auth.json"), "utf8")))
|
||||
.toEqual({ anthropic: { type: "api_key", key: "unrelated-key" } });
|
||||
} finally {
|
||||
expect(readFileSync(path.join(agentDir, "auth.json"), "utf8")).toBe(originalAuth);
|
||||
mgr.teardown("catalog-credential");
|
||||
vi.unstubAllEnvs();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
@@ -750,7 +788,7 @@ test("set_model translates a canonical catalog key to its upstream Pi model ID",
|
||||
session: { reasoning: false, contextWindow: 32768, maxTokens: 8192 },
|
||||
};
|
||||
const modelCatalog: RuntimeModelCatalog = {
|
||||
defaultSession: model.id, defaultMetadataGeneration: null, embedding: null,
|
||||
defaultInteraction: model.id, embedding: null,
|
||||
sessionModels: () => [model], metadataModels: () => [], hasSession: (id) => id === model.id,
|
||||
};
|
||||
const mgr = new PiProcessManager(loadConfig({ PI_BIN: "/usr/local/bin/pi" }), {
|
||||
|
||||
@@ -61,20 +61,22 @@ test("provider smoke resolves the selected catalog credential from the secret bu
|
||||
session: { reasoning: false },
|
||||
};
|
||||
const modelCatalog: RuntimeModelCatalog = {
|
||||
defaultSession: model.id,
|
||||
defaultMetadataGeneration: null,
|
||||
defaultInteraction: model.id,
|
||||
embedding: null,
|
||||
sessionModels: () => [model],
|
||||
metadataModels: () => [],
|
||||
hasSession: (id) => id === model.id,
|
||||
};
|
||||
let spawnEnv: NodeJS.ProcessEnv | undefined;
|
||||
const readAuthStore = vi.fn(() => JSON.stringify({ openai: { type: "api_key", key: "stale-key" } }));
|
||||
const smoke = createPiProviderSmoke(loadConfig({ THT_SECRETS_FILE: secret }), {
|
||||
modelCatalog,
|
||||
authProviders: () => new Set(),
|
||||
authProviders: () => new Set(["openai"]),
|
||||
readAuthStore,
|
||||
readModelsStore: () => undefined,
|
||||
spawnFn: (_command, _args, options) => {
|
||||
spawnEnv = options.env;
|
||||
expect(existsSync(join(options.env.PI_CODING_AGENT_DIR!, "auth.json"))).toBe(false);
|
||||
return successfulProviderChild();
|
||||
},
|
||||
});
|
||||
@@ -85,6 +87,7 @@ test("provider smoke resolves the selected catalog credential from the secret bu
|
||||
expect(spawnEnv?.ZAI_API_KEY).toBe("catalog-secret");
|
||||
expect(spawnEnv).not.toHaveProperty("OPENAI_API_KEY");
|
||||
expect(spawnEnv).not.toHaveProperty("THT_MODEL_API_KEY");
|
||||
expect(readAuthStore).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
@@ -306,7 +309,7 @@ test("provider smoke makes one configured request from an isolated no-capability
|
||||
sessionAdapter: { mode: "pi_builtin" }, session: { reasoning: true },
|
||||
};
|
||||
const smokeCatalog: RuntimeModelCatalog = {
|
||||
defaultSession: smokeModel.id, defaultMetadataGeneration: null, embedding: null,
|
||||
defaultInteraction: smokeModel.id, embedding: null,
|
||||
sessionModels: () => [smokeModel], metadataModels: () => [],
|
||||
hasSession: (id) => id === smokeModel.id,
|
||||
};
|
||||
|
||||
@@ -27,10 +27,9 @@ function operationalWorkspace(id = "default") {
|
||||
} as const;
|
||||
}
|
||||
|
||||
function sessionCatalog(defaultSession = "zai/glm-5.2", available = [defaultSession]) {
|
||||
function sessionCatalog(defaultInteraction = "zai/glm-5.2", available = [defaultInteraction]) {
|
||||
return {
|
||||
defaultSession,
|
||||
defaultMetadataGeneration: null,
|
||||
defaultInteraction,
|
||||
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||
sessionModels: () => [],
|
||||
metadataModels: () => [],
|
||||
@@ -54,7 +53,11 @@ const defaultWorkspaceRegistry = {
|
||||
|
||||
function buildApp(config: Parameters<typeof buildRealApp>[0], deps: Record<string, unknown> = {}) {
|
||||
const thtRunner = deps.thtRunner
|
||||
? { qdrantEnsure: async () => ({ ok: true }), ...(deps.thtRunner as object) }
|
||||
? {
|
||||
qdrantEnsure: async () => ({ ok: true }),
|
||||
ensureInteractionLanguage: async () => ({ interaction_language: "en" }),
|
||||
...(deps.thtRunner as object),
|
||||
}
|
||||
: undefined;
|
||||
return buildRealApp(config, {
|
||||
workspaceRuntimeSupport: () => true,
|
||||
@@ -89,6 +92,156 @@ const aliceHeaders = {
|
||||
"x-thoth-is-admin": "0",
|
||||
};
|
||||
|
||||
test.each([undefined, null, "", "en--US", "en_US", "en\nIGNORE", "en<script>", 42])(
|
||||
"new sessions reject invalid interaction language %s before persistence", async (interactionLanguage) => {
|
||||
const app = mutApp({ sessionNew: async () => { throw new Error("must not create"); } });
|
||||
try {
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions", payload: { question: "Pazienti", interactionLanguage },
|
||||
});
|
||||
expect(response.statusCode).toBe(400);
|
||||
expect(response.json()).toMatchObject({ code: "invalid_interaction_language" });
|
||||
} finally { await app.close(); }
|
||||
},
|
||||
);
|
||||
|
||||
test.each([["en", "en"], ["fr-FR", "fr-FR"], ["FR-fr", "fr-FR"]])(
|
||||
"new session forwards canonical interaction language %s without changing the question", async (language, canonical) => {
|
||||
let created: any;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
sessionNew: async (options: any) => { created = options; return { id: "language" }; },
|
||||
searchPack: async () => {},
|
||||
},
|
||||
readiness: { ensure: async () => ({ ok: true }) },
|
||||
mgr: {
|
||||
get: () => undefined, createFor: () => ({ bridge: { onClientEvent: () => {} } }),
|
||||
configure: async () => {}, start: () => {},
|
||||
},
|
||||
getSettings: () => ({ workspace: "default" }),
|
||||
});
|
||||
try {
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions", payload: { question: "Pazienti", interactionLanguage: language },
|
||||
});
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(created).toMatchObject({ question: "Pazienti", interactionLanguage: canonical });
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
|
||||
test("resume rejects browser interaction language overrides", async () => {
|
||||
const app = mutApp({ sessionShow: async () => ({ status: "open", interaction_language: "it" }) });
|
||||
try {
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions/s1/resume", payload: { interactionLanguage: "en" },
|
||||
});
|
||||
expect(response.statusCode).toBe(400);
|
||||
expect(response.json()).toMatchObject({ code: "interaction_language_pinned" });
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
|
||||
test("new session starts Pi with the question language persisted by the harness", async () => {
|
||||
let runtime: any;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
sessionNew: async () => ({ id: "english-question", interaction_language: "en" }),
|
||||
searchPack: async () => {},
|
||||
},
|
||||
readiness: { ensure: async () => ({ ok: true }) },
|
||||
mgr: {
|
||||
get: () => undefined,
|
||||
createFor: (_id: string, options: any) => {
|
||||
runtime = options;
|
||||
return { bridge: { onClientEvent: () => {} } };
|
||||
},
|
||||
configure: async () => {}, start: () => {},
|
||||
},
|
||||
getSettings: () => ({ workspace: "default" }),
|
||||
});
|
||||
try {
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: {
|
||||
question: "How many patients were admitted last year?", interactionLanguage: "it",
|
||||
} });
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(runtime.interactionLanguage).toBe("en");
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
|
||||
test("resume pins legacy interaction language using the resolved harness workspace", async () => {
|
||||
let pinnedWorkspace: string | undefined;
|
||||
let runtime: any;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
sessionShow: async () => ({ id: "legacy", status: "closed" }),
|
||||
ensureInteractionLanguage: async (_id: string, workspace: string) => {
|
||||
pinnedWorkspace = workspace;
|
||||
return { interaction_language: "it" };
|
||||
},
|
||||
reopenSession: async () => {},
|
||||
},
|
||||
readiness: { ensure: async () => ({ ok: true }) },
|
||||
mgr: {
|
||||
get: () => undefined,
|
||||
createFor: (_id: string, options: any) => {
|
||||
runtime = options;
|
||||
return { bridge: { onClientEvent: () => {} } };
|
||||
},
|
||||
configure: async () => {}, start: () => {},
|
||||
},
|
||||
getSettings: () => ({ workspace: "other-browser-workspace" }),
|
||||
});
|
||||
try {
|
||||
const response = await app.inject({ method: "POST", url: "/sessions/legacy/resume" });
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(pinnedWorkspace).toContain("/default.yaml");
|
||||
expect(runtime).toMatchObject({ interactionLanguage: "it", mode: "resume" });
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
|
||||
test("resume retains persisted interaction language despite different browser settings", async () => {
|
||||
let options: any;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
sessionShow: async () => ({ id: "saved", status: "closed", interaction_language: "it" }),
|
||||
ensureInteractionLanguage: async () => { throw new Error("language is already pinned"); },
|
||||
reopenSession: async () => {},
|
||||
},
|
||||
readiness: { ensure: async () => ({ ok: true }) },
|
||||
mgr: {
|
||||
get: () => undefined,
|
||||
createFor: (_id: string, value: any) => {
|
||||
options = value;
|
||||
return { bridge: { onClientEvent: () => {} } };
|
||||
},
|
||||
configure: async () => {}, start: () => {},
|
||||
},
|
||||
getSettings: () => ({ workspace: "english-workspace", locale: "en" }),
|
||||
});
|
||||
try {
|
||||
const response = await app.inject({ method: "POST", url: "/sessions/saved/resume" });
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(options.interactionLanguage).toBe("it");
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
|
||||
test("resume cannot start Pi if legacy interaction language cannot be persisted", async () => {
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
sessionShow: async () => ({ id: "legacy", status: "closed" }),
|
||||
ensureInteractionLanguage: async () => { throw new Error("private storage details"); },
|
||||
reopenSession: async () => { throw new Error("must not reopen"); },
|
||||
},
|
||||
readiness: { ensure: async () => ({ ok: true }) },
|
||||
mgr: { createFor: () => { throw new Error("must not spawn"); } },
|
||||
getSettings: () => ({ workspace: "default" }),
|
||||
});
|
||||
try {
|
||||
const response = await app.inject({ method: "POST", url: "/sessions/legacy/resume" });
|
||||
expect(response.statusCode).toBe(503);
|
||||
expect(response.body).not.toContain("private storage details");
|
||||
} finally { await app.close(); }
|
||||
});
|
||||
|
||||
test("upstream requests without a principal fail before a Pi runtime can be created", async () => {
|
||||
let created = false;
|
||||
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
|
||||
@@ -96,7 +249,7 @@ test("upstream requests without a principal fail before a Pi runtime can be crea
|
||||
thtRunner: {} as any,
|
||||
});
|
||||
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
|
||||
expect(response.statusCode).toBe(401);
|
||||
expect(created).toBe(false);
|
||||
@@ -134,7 +287,7 @@ test("maintenance rejects new and resumed session admission without interrupting
|
||||
});
|
||||
|
||||
const create = await app.inject({
|
||||
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { question: "q" },
|
||||
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { interactionLanguage: "en", question: "q" },
|
||||
});
|
||||
const resume = await app.inject({ method: "POST", url: "/sessions/open/resume", headers: aliceHeaders });
|
||||
|
||||
@@ -155,7 +308,7 @@ test("a durable maintenance marker initializes admission closed after backend re
|
||||
AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness", THT_MAINTENANCE_FILE: marker,
|
||||
}), { thtRunner: {} as any });
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { question: "q" },
|
||||
method: "POST", url: "/sessions", headers: aliceHeaders, payload: { interactionLanguage: "en", question: "q" },
|
||||
});
|
||||
expect(response.statusCode).toBe(503);
|
||||
expect(response.json()).toMatchObject({ code: "maintenance" });
|
||||
@@ -364,6 +517,65 @@ test("retention scans a removed workspace's retained snapshot", async () => {
|
||||
expect(response.json()).toEqual([expect.objectContaining({ id: "resumable" })]);
|
||||
});
|
||||
|
||||
test("active sessions remain available when retained snapshot discovery is unreadable", async () => {
|
||||
const activeSnapshot = "/registry/snapshots/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/active.yaml";
|
||||
const retainedFailure = "invalid retained snapshot /registry/snapshots/secret/legacy.yaml";
|
||||
const reconcileSnapshotRetention = vi.fn(async () => {});
|
||||
const subscribed = vi.fn();
|
||||
const warning = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||
const manifest = {
|
||||
id: "live-session", status: "open", archived: false,
|
||||
workspace_id: "active", workspace_revision: "a".repeat(40),
|
||||
};
|
||||
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
withPrincipal: () => ({
|
||||
sessionList: async (snapshotPath: string) => {
|
||||
expect(snapshotPath).toBe(activeSnapshot);
|
||||
return [manifest];
|
||||
},
|
||||
sessionShow: async (id: string, snapshotPath?: string) => {
|
||||
if (id === manifest.id && snapshotPath === activeSnapshot) return manifest;
|
||||
throw new Error("session not found");
|
||||
},
|
||||
}),
|
||||
} as any,
|
||||
mgr: { get: () => undefined } as any,
|
||||
hub: {
|
||||
subscribe: (_id: string, _send: unknown, options: { close?: () => void }) => {
|
||||
subscribed();
|
||||
options.close?.();
|
||||
return () => {};
|
||||
},
|
||||
} as any,
|
||||
workspaceRegistry: {
|
||||
listRetainedSnapshots: async () => { throw new Error(retainedFailure); },
|
||||
list: async () => [{
|
||||
id: "active", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: activeSnapshot,
|
||||
}],
|
||||
reconcileSnapshotRetention,
|
||||
} as any,
|
||||
});
|
||||
|
||||
try {
|
||||
const list = await app.inject({ method: "GET", url: "/sessions", headers: aliceHeaders });
|
||||
const detail = await app.inject({ method: "GET", url: `/sessions/${manifest.id}`, headers: aliceHeaders });
|
||||
const events = await app.inject({ method: "GET", url: `/sessions/${manifest.id}/events`, headers: aliceHeaders });
|
||||
|
||||
expect(list.statusCode).toBe(200);
|
||||
expect(list.json()).toEqual([expect.objectContaining({ id: manifest.id, active: false })]);
|
||||
expect(detail.statusCode).toBe(200);
|
||||
expect(detail.json()).toMatchObject(manifest);
|
||||
expect(events.statusCode).toBe(200);
|
||||
expect(subscribed).toHaveBeenCalledOnce();
|
||||
expect(reconcileSnapshotRetention).not.toHaveBeenCalled();
|
||||
expect(list.body + detail.body + events.body).not.toContain(retainedFailure);
|
||||
} finally {
|
||||
warning.mockRestore();
|
||||
await app.close();
|
||||
}
|
||||
});
|
||||
|
||||
test("the single local installation listing reconciles its resumable workspace pins", async () => {
|
||||
const retained = vi.fn(async () => {});
|
||||
const retainedRevision = "d".repeat(40);
|
||||
@@ -434,7 +646,7 @@ test("new sessions are created through the authenticated principal, not a client
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions", headers: aliceHeaders,
|
||||
payload: { question: "q", owner: "mallory" },
|
||||
payload: { interactionLanguage: "en", question: "q", owner: "mallory" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
@@ -451,7 +663,7 @@ test("new sessions reject the client legacy workspace field unless local legacy
|
||||
});
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions", payload: { question: "q", workspace: "legacy" },
|
||||
method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q", workspace: "legacy" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(409);
|
||||
@@ -471,7 +683,7 @@ test("explicit local legacy mode permits the unpinned client workspace request",
|
||||
});
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions", payload: { question: "q", workspace: "legacy" },
|
||||
method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q", workspace: "legacy" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
@@ -509,7 +721,7 @@ test("creates a session from the active immutable workspace revision", async ()
|
||||
|
||||
await app.inject({
|
||||
method: "POST", url: "/sessions",
|
||||
payload: { question: "q", workspaceId: "psd-clinical", provider: "zai", model: "glm-5.2", thinking: "low" },
|
||||
payload: { interactionLanguage: "en", question: "q", workspaceId: "psd-clinical", provider: "zai", model: "glm-5.2", thinking: "low" },
|
||||
});
|
||||
|
||||
expect(sessionNew).toHaveBeenCalledWith(expect.objectContaining({
|
||||
@@ -553,7 +765,7 @@ test("hands one Catalog-backed runtime to both retrieval and Pi", async () => {
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: "/sessions",
|
||||
payload: { question: "Which users placed orders?", workspaceId: "default" },
|
||||
payload: { interactionLanguage: "en", question: "Which users placed orders?", workspaceId: "default" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
@@ -629,7 +841,7 @@ test("refuses core admission when the workspace preprocessing fingerprint is sta
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: "/sessions",
|
||||
payload: { question: "q", workspaceId: "default" },
|
||||
payload: { interactionLanguage: "en", question: "q", workspaceId: "default" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(409);
|
||||
@@ -681,7 +893,7 @@ test("rejects an SSH-only Catalog binding before persisting or starting a sessio
|
||||
} as any,
|
||||
});
|
||||
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
|
||||
expect(response.statusCode).toBe(409);
|
||||
expect(response.json()).toMatchObject({ code: "workspace_not_activatable" });
|
||||
@@ -721,7 +933,7 @@ test("hands a revision lease to retention only after the session manifest is dur
|
||||
workspaceRegistry: { acquireSessionRevision } as any,
|
||||
});
|
||||
|
||||
const request = app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const request = app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
expect(markPersisted).not.toHaveBeenCalled();
|
||||
expect(abort).not.toHaveBeenCalled();
|
||||
@@ -752,7 +964,7 @@ test("creates a session from the configured default workspace revision when work
|
||||
workspaceRegistry: registry as any,
|
||||
});
|
||||
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
|
||||
expect(registry.read).toHaveBeenCalledWith("psd-clinical");
|
||||
expect(sessionNew).toHaveBeenCalledWith(expect.objectContaining({
|
||||
@@ -838,7 +1050,7 @@ test("session lifecycle locates a B session when installation default is A", asy
|
||||
} as any,
|
||||
});
|
||||
|
||||
expect((await app.inject({ method: "POST", url: "/sessions", payload: {
|
||||
expect((await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en",
|
||||
question: "B question", workspaceId: "b-workspace", provider: "zai", model: "glm-5.2", thinking: "low",
|
||||
} })).statusCode).toBe(200);
|
||||
expect((await app.inject({ method: "GET", url: "/sessions" })).json()).toEqual([
|
||||
@@ -850,7 +1062,7 @@ test("session lifecycle locates a B session when installation default is A", asy
|
||||
|
||||
active = undefined;
|
||||
expect((await app.inject({ method: "POST", url: "/sessions/session-b/resume" })).json())
|
||||
.toEqual({ id: "session-b", alreadyActive: false });
|
||||
.toEqual({ id: "session-b", alreadyActive: false, workspaceId: "b-workspace" });
|
||||
expect(calls).toContain(`new:${bPath}`);
|
||||
expect(calls).toContain(`list:${bPath}`);
|
||||
expect(calls).toContain(`show:${bPath}`);
|
||||
@@ -881,7 +1093,7 @@ test("POST /sessions uses the catalog default with workspace/thinking settings a
|
||||
],
|
||||
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
|
||||
});
|
||||
const created = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const created = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
expect(created.json()).toEqual({ id: "s1" });
|
||||
expect(sessionNewArg.workspaceConfigPath).toContain(`/snapshots/${"e".repeat(40)}/w.yaml`);
|
||||
expect(sessionNewArg.provider).toBe("zai");
|
||||
@@ -942,11 +1154,11 @@ test("POST /sessions stops the user's previous open Pi runtime before creating a
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
|
||||
expect((await app.inject({ method: "POST", url: "/sessions", payload: { question: "one" } })).statusCode)
|
||||
expect((await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "one" } })).statusCode)
|
||||
.toBe(200);
|
||||
order.length = 0;
|
||||
|
||||
const second = await app.inject({ method: "POST", url: "/sessions", payload: { question: "two" } });
|
||||
const second = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "two" } });
|
||||
|
||||
expect(second.statusCode).toBe(200);
|
||||
expect(second.json()).toEqual({ id: "s2" });
|
||||
@@ -967,7 +1179,7 @@ test("POST /sessions refuses to create a session when the local DWH precheck fai
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
getSettings: () => ({ workspace: "w" }) as any,
|
||||
});
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
expect(res.statusCode).toBe(503);
|
||||
expect(res.json()).toMatchObject({ code: "dwh_unreachable" });
|
||||
expect(pinged).toBe(1);
|
||||
@@ -988,7 +1200,7 @@ test("POST /sessions proceeds past a passing DWH precheck", async () => {
|
||||
getSettings: () => ({ workspace: "w" }) as any,
|
||||
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
|
||||
});
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
expect(res.statusCode).toBe(200);
|
||||
expect(pinged).toBe(1);
|
||||
expect(created).toBe(1);
|
||||
@@ -1007,7 +1219,7 @@ test("POST /sessions skips the DWH precheck when the flag is off (default)", asy
|
||||
getSettings: () => ({ workspace: "w" }) as any,
|
||||
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
|
||||
});
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
expect(res.statusCode).toBe(200);
|
||||
expect(pinged).toBe(0); // probe never runs without the flag
|
||||
});
|
||||
@@ -1041,7 +1253,7 @@ test("POST /sessions configura Pi con il thinking globale selezionato", async ()
|
||||
],
|
||||
});
|
||||
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
expect(configured.thinking).toBe("high");
|
||||
@@ -1120,6 +1332,29 @@ test("POST /sessions/:id/resume uses the manifest's retained workspace revision"
|
||||
);
|
||||
});
|
||||
|
||||
test.each([undefined, { provider: "local", model: "qwen" }])("resume uses the global model/default, not the historical manifest (%j)", async (payload) => {
|
||||
const configure = vi.fn(async () => {});
|
||||
const manifest = { status: "open", archived: false, provider: "retired", model: "old-model" };
|
||||
const runtime = { bridge: { onClientEvent: () => {}, emitClientEvent: () => {} } };
|
||||
let current: any;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
mgr: {
|
||||
get: () => current,
|
||||
createFor: () => { current = runtime; return runtime; },
|
||||
configure, start: () => {},
|
||||
} as any,
|
||||
thtRunner: { sessionShow: async () => manifest, reopenSession: async () => {} } as any,
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
runtimeModelCatalog: sessionCatalog("zai/glm-5.2", ["zai/glm-5.2", "local/qwen"]),
|
||||
getSettings: () => ({ workspace: "psd", thinking: "medium" }) as any,
|
||||
});
|
||||
const response = await app.inject({ method: "POST", url: "/sessions/model-resume/resume", ...(payload ? { payload } : {}) });
|
||||
expect(response.statusCode).toBe(200);
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
expect(configure).toHaveBeenCalledWith(expect.anything(), expect.objectContaining(payload ?? { provider: "zai", model: "glm-5.2" }));
|
||||
expect(manifest).toMatchObject({ provider: "retired", model: "old-model" });
|
||||
});
|
||||
|
||||
test("POST /sessions/:id/resume returns a sanitized error when its retained revision is unavailable", async () => {
|
||||
const rawFailure = "cannot read /data/workspace-registry/snapshots/secret-revision";
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
@@ -1346,7 +1581,7 @@ test("resuming a different session stops the user's previous Pi runtime", async
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
expect((await app.inject({ method: "POST", url: "/sessions", payload: { question: "one" } })).statusCode)
|
||||
expect((await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "one" } })).statusCode)
|
||||
.toBe(200);
|
||||
|
||||
const resumed = await app.inject({ method: "POST", url: "/sessions/s2/resume" });
|
||||
@@ -1734,13 +1969,13 @@ test.each([
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "old" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "old" } });
|
||||
if (lifecycle === "close") {
|
||||
await app.inject({ method: "POST", url: "/sessions/s1/close" });
|
||||
await app.inject({ method: "POST", url: "/sessions/s1/resume" });
|
||||
} else {
|
||||
await app.inject({ method: "DELETE", url: "/sessions/s1" });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "replacement" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "replacement" } });
|
||||
}
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
expect(current).toBe(replacement);
|
||||
@@ -1804,7 +2039,7 @@ test.each(["resolve", "reject"] as const)(
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "old" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "old" } });
|
||||
await app.inject({ method: "DELETE", url: "/sessions/s1" });
|
||||
published.length = 0;
|
||||
|
||||
@@ -1996,7 +2231,7 @@ test("Close suppresses a bootstrap that settles while close persistence is pendi
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
const closeResponse = app.inject({ method: "POST", url: "/sessions/s1/close" });
|
||||
await closeStarted;
|
||||
published.length = 0;
|
||||
@@ -2070,7 +2305,7 @@ test("bootstrap failure persists once and keeps Resume serialized behind that pe
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
oldConfigure.reject(new Error("configure failed"));
|
||||
await failureStarted;
|
||||
const resumeResponse = app.inject({ method: "POST", url: "/sessions/s1/resume" })
|
||||
@@ -2193,7 +2428,7 @@ test("a replaced runtime cannot publish or fail the newly resumed session", asyn
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
oldBridge.setState("idle");
|
||||
|
||||
@@ -2253,7 +2488,7 @@ test("a deleted runtime cannot repopulate or fail the forgotten session", async
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
const response = await app.inject({ method: "DELETE", url: "/sessions/s1" });
|
||||
@@ -2298,7 +2533,7 @@ test("an unexpectedly exited runtime publishes its terminal sequence then releas
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
published.length = 0;
|
||||
|
||||
@@ -2349,7 +2584,7 @@ test("agent_end releases the Pi runtime after the session was finalized", async
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
getSettings: () => ({ workspace: "local" }) as any,
|
||||
});
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
bridge.emitClientEvent({ type: "system_event", event: "agent_end" });
|
||||
@@ -2421,7 +2656,7 @@ test("POST /sessions/:id/response senza gate pendente risponde 409 (risposta sta
|
||||
getSettings: () => ({ workspace: "w" }),
|
||||
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
|
||||
});
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
// The fake Pi never emitted a ui_request: the bridge has no pending descriptor, so a
|
||||
// response (stale UI, double submit) must be rejected instead of forwarded to Pi.
|
||||
const res = await app.inject({ method: "POST", url: "/sessions/s1/response",
|
||||
@@ -2562,7 +2797,7 @@ test("POST /sessions readiness failure returns one fixed public message without
|
||||
getSettings: () => ({ workspace: "psd" }) as any,
|
||||
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
|
||||
});
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
expect(res.statusCode).toBe(503);
|
||||
expect(res.json()).toEqual({
|
||||
error: "Session services are not ready. Check configuration and connectivity, then try again.",
|
||||
@@ -2583,7 +2818,7 @@ test.each(["semantic_index_incompatible", "workspace_not_activatable"] as const)
|
||||
});
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST", url: "/sessions", payload: { question: "q" },
|
||||
method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(503);
|
||||
@@ -2606,7 +2841,7 @@ test("POST /sessions returns storage 503 before creating a Pi runtime when sessi
|
||||
mgr: { createFor: () => { piCreated = true; throw new Error("must not spawn"); } } as any,
|
||||
});
|
||||
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
|
||||
expect(response.statusCode).toBe(503);
|
||||
expect(response.json()).toEqual({ error: "session storage is unavailable" });
|
||||
@@ -2626,13 +2861,13 @@ test("POST /sessions proceeds when ollamaEnsure succeeds", async () => {
|
||||
getSettings: () => ({ workspace: "psd" }) as any,
|
||||
spawnFn: () => nodeSpawn("node", [FAKE, SCRIPT]) as any,
|
||||
});
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
expect(res.json()).toEqual({ id: "s1" });
|
||||
expect(qdrantEnsure).toHaveBeenCalledWith(operationalWorkspace("psd"), 60, "self_heal");
|
||||
expect(ensureWs).toContain(`/snapshots/${"e".repeat(40)}/psd.yaml`);
|
||||
});
|
||||
|
||||
test("POST /sessions falls back from a stale requested model to the catalog default", async () => {
|
||||
test("POST /sessions rejects a stale requested model without silently using the default", async () => {
|
||||
let created = 0;
|
||||
let persisted: any;
|
||||
const runtime = { bridge: { onClientEvent: () => {} } };
|
||||
@@ -2659,16 +2894,13 @@ test("POST /sessions falls back from a stale requested model to the catalog defa
|
||||
const res = await app.inject({
|
||||
method: "POST",
|
||||
url: "/sessions",
|
||||
payload: { question: "q", provider: "deepseek", model: "deepseek-v4-pro" },
|
||||
payload: { interactionLanguage: "en", question: "q", provider: "deepseek", model: "deepseek-v4-pro" },
|
||||
});
|
||||
|
||||
expect(res.statusCode).toBe(200);
|
||||
expect(res.json()).toEqual({
|
||||
id: "fallback",
|
||||
warning: "Configured model deepseek/deepseek-v4-pro is unavailable; using zai/glm-5.2.",
|
||||
});
|
||||
expect(persisted).toMatchObject({ provider: "zai", model: "glm-5.2" });
|
||||
expect(created).toBe(1);
|
||||
expect(res.statusCode).toBe(503);
|
||||
expect(res.json()).toMatchObject({ code: "model_unavailable" });
|
||||
expect(persisted).toBeUndefined();
|
||||
expect(created).toBe(0);
|
||||
});
|
||||
|
||||
test("POST /sessions marks a persisted session failed when runtime construction throws", async () => {
|
||||
@@ -2697,7 +2929,7 @@ test("POST /sessions marks a persisted session failed when runtime construction
|
||||
],
|
||||
});
|
||||
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
|
||||
expect(res.statusCode).toBe(503);
|
||||
expect(res.json()).toEqual({
|
||||
@@ -2790,7 +3022,7 @@ test.each([
|
||||
|
||||
try {
|
||||
const response = flow === "new"
|
||||
? await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } })
|
||||
? await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } })
|
||||
: await app.inject({ method: "POST", url: `/sessions/${sessionId}/resume` });
|
||||
const logs = consoleError.mock.calls.flat().map(String).join(" ");
|
||||
|
||||
@@ -2896,7 +3128,7 @@ test("POST /sessions returns after bridge attachment but starts only after retri
|
||||
getSettings: () => ({ workspace: "psd" }) as any,
|
||||
});
|
||||
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
expect(res.json()).toEqual({ id: "s-early" });
|
||||
expect(bridgeAttached).toBe(true);
|
||||
expect(started).toBe(false);
|
||||
@@ -2943,7 +3175,7 @@ test("POST /sessions bootstrap failure emits only a fixed recovery message", asy
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: "/sessions",
|
||||
payload: { question: "q" },
|
||||
payload: { interactionLanguage: "en", question: "q" },
|
||||
});
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
@@ -3039,7 +3271,7 @@ test.each([
|
||||
})),
|
||||
} as any,
|
||||
});
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
||||
const response = await app.inject({ method: "POST", url: "/sessions", payload: { interactionLanguage: "en", question: "q" } });
|
||||
|
||||
expect(response.statusCode).toBe(expectedStatus);
|
||||
expect(ensure).toHaveBeenCalledTimes(reachesReadiness ? 1 : 0);
|
||||
|
||||
@@ -160,8 +160,7 @@ test("GET /models returns session choices from the installation model catalog",
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {} as any,
|
||||
runtimeModelCatalog: {
|
||||
defaultSession: "zai/glm-5.2",
|
||||
defaultMetadataGeneration: null,
|
||||
defaultInteraction: "zai/glm-5.2",
|
||||
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||
sessionModels: () => [{
|
||||
id: "zai/glm-5.2", provider: "zai", model: "glm-5.2", label: "GLM 5.2",
|
||||
@@ -187,8 +186,7 @@ test("GET /models returns an empty list when the catalog has no session models",
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {} as any,
|
||||
runtimeModelCatalog: {
|
||||
defaultSession: null,
|
||||
defaultMetadataGeneration: null,
|
||||
defaultInteraction: null,
|
||||
embedding: null,
|
||||
sessionModels: () => [],
|
||||
metadataModels: () => [],
|
||||
|
||||
@@ -34,6 +34,19 @@ test("sessionNew parses id from JSON", async () => {
|
||||
expect(await r.sessionNew({ question: "q" })).toEqual({ id: "2026-06-27-100000-x" });
|
||||
});
|
||||
|
||||
test("session language uses public per-command CLI flags and the selected config", async () => {
|
||||
const runner = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" });
|
||||
(spawn as any).mockClear();
|
||||
await runner.sessionNew({ question: "Pazienti", interactionLanguage: "en" });
|
||||
expect((spawn as any).mock.calls[0][1]).toEqual([
|
||||
"session", "new", "Pazienti", "--interaction-language", "en", "--json", "-c", "config/tht.yaml",
|
||||
]);
|
||||
await runner.ensureInteractionLanguage("s1");
|
||||
expect((spawn as any).mock.calls[1][1]).toEqual([
|
||||
"session", "ensure-interaction-language", "s1", "--json", "-c", "config/tht.yaml",
|
||||
]);
|
||||
});
|
||||
|
||||
test("searchPack persists retrieval context with session and workspace", async () => {
|
||||
const calls: any[] = [];
|
||||
const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" });
|
||||
|
||||
@@ -10,6 +10,28 @@ import {
|
||||
|
||||
const roots: string[] = [];
|
||||
|
||||
test("Evidence consolidation runs only its stage and preserves blocked Catalog readiness", async () => {
|
||||
const catalog = repository();
|
||||
const runChild = vi.fn(async () => ({ exitCode: 0, stdout: JSON.stringify({ status: "succeeded", counts: { documents: 2 } }), stderr: "" }));
|
||||
const result = await service({ repository: catalog, runChild }).consolidateEvidence({ workspaceId: "catalog-workspace" });
|
||||
expect(result.status).toBe("succeeded");
|
||||
expect(runChild).toHaveBeenCalledOnce();
|
||||
expect(runChild.mock.calls[0]?.[0]).toMatchObject({ argv: ["preprocess", "evidence", "--consolidate", "--json", "-c", "/dev/fd/3"] });
|
||||
expect(catalog.beginPreprocessing).not.toHaveBeenCalled();
|
||||
expect(catalog.finishPreprocessing).not.toHaveBeenCalled();
|
||||
expect(catalog.clearPreprocessing).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("Evidence index failure reports retry without changing Catalog state", async () => {
|
||||
const catalog = repository();
|
||||
const runChild = vi.fn(async () => ({ exitCode: 1, stdout: JSON.stringify({ status: "failed", saved: true, error: "Saved; retry indexing." }), stderr: "private provider endpoint" }));
|
||||
const result = await service({ repository: catalog, runChild }).consolidateEvidence({ workspaceId: "catalog-workspace" });
|
||||
expect(result.status).toBe("failed");
|
||||
expect(result.warnings).toEqual(["Saved; retry indexing."]);
|
||||
expect(catalog.finishPreprocessing).not.toHaveBeenCalled();
|
||||
expect(JSON.stringify(result)).not.toContain("private provider");
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
roots.splice(0).forEach((root) => rmSync(root, { recursive: true, force: true }));
|
||||
});
|
||||
@@ -254,6 +276,24 @@ test("semantic preflight failure is persisted before returning", async () => {
|
||||
);
|
||||
});
|
||||
|
||||
test.each(["refresh", "decide"] as const)("Evidence %s calls only the source worker and preserves Catalog readiness", async action => {
|
||||
const catalog = repository();
|
||||
const runChild = vi.fn(async () => ({exitCode: 0, stdout: JSON.stringify({status: "succeeded", counts: {changed: 1}}), stderr: ""}));
|
||||
const result = await service({repository: catalog, runChild}).evidenceSources({workspaceId: "catalog-workspace", action,
|
||||
...(action === "decide" ? {sourceId: "a".repeat(64), revision: "b".repeat(64), decision: "replace" as const} : {}), actor: "Curator"});
|
||||
expect(result.status).toBe("succeeded");
|
||||
expect(runChild).toHaveBeenCalledWith({configPath: expect.any(String), argv: expect.arrayContaining(["evidence", "sources", action, "-c", "/dev/fd/3", "--actor", "Curator"])});
|
||||
expect(catalog.beginPreprocessing).not.toHaveBeenCalled();
|
||||
expect(catalog.finishPreprocessing).not.toHaveBeenCalled();
|
||||
expect(catalog.clearPreprocessing).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("failed source activation reports the saved decision for retry", async () => {
|
||||
const result = await service({runChild: async () => ({exitCode: 1, stdout: JSON.stringify({status: "failed", saved: true, error: "Decision saved; retry indexing"}), stderr: "secret"})})
|
||||
.evidenceSources({workspaceId: "catalog-workspace", action: "decide", sourceId: "a".repeat(64), revision: "b".repeat(64), decision: "keep"});
|
||||
expect(result).toMatchObject({status: "failed", warnings: ["Decision saved; retry indexing"]});
|
||||
});
|
||||
|
||||
test("clear invalidates Catalog readiness before clearing only derived worker data", async () => {
|
||||
const catalog = repository();
|
||||
const runChild = vi.fn(async () => ({
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
existsSync,
|
||||
mkdtempSync,
|
||||
@@ -202,7 +203,7 @@ test("deterministic operator leases are keyed by logical identity and stable acr
|
||||
workspaceSecretStore: f.workspaceSecretStore,
|
||||
});
|
||||
|
||||
const suffix = first.inputFingerprint.slice(7, 23);
|
||||
const suffix = createHash("sha256").update(first.inputFingerprint + "\n" + readFileSync(first.path, "utf8")).digest("hex").slice(0, 16);
|
||||
expect(second.path).toBe(first.path);
|
||||
expect(first.path).toBe(join(
|
||||
f.dataRoot,
|
||||
|
||||
@@ -242,6 +242,7 @@ test("separate runtime leases hand off byte-identical revision Evidence configs
|
||||
source_identity: "workspace://psd-clinical",
|
||||
});
|
||||
expect(parse(firstYaml).evidence).toEqual({
|
||||
local_archive_root: join(f.registryConfig.root, "repo", "psd-clinical"),
|
||||
sources: [{
|
||||
type: "filesystem",
|
||||
root: expectedRoot,
|
||||
|
||||
@@ -192,6 +192,7 @@ test("renders filesystem Evidence below the immutable revision content root with
|
||||
expect(rendered.runtime_identity.workspace_revision).toBe(evidenceRevision);
|
||||
expect(rendered.evidence).toEqual({
|
||||
schema_version: 2,
|
||||
local_archive_root: "/srv/registry/repo/psd-clinical",
|
||||
sources: [{
|
||||
type: "filesystem",
|
||||
root: `/srv/registry/snapshots/${evidenceRevision}/psd-clinical/evidence`,
|
||||
@@ -203,7 +204,7 @@ test("renders filesystem Evidence below the immutable revision content root with
|
||||
max_chunk_chars: 4_000,
|
||||
retain_published_generations: 3,
|
||||
});
|
||||
expect(yaml).not.toContain("/srv/registry/repo");
|
||||
expect(rendered.evidence.sources[0].root).not.toContain("/srv/registry/repo");
|
||||
});
|
||||
|
||||
test("renders public HTTP Evidence with exact fractional-second timeouts and every policy limit", () => {
|
||||
@@ -223,6 +224,7 @@ test("renders public HTTP Evidence with exact fractional-second timeouts and eve
|
||||
}));
|
||||
|
||||
expect(rendered.evidence).toEqual({
|
||||
local_archive_root: "/srv/registry/repo/psd-clinical",
|
||||
sources: [{
|
||||
type: "http",
|
||||
urls: ["https://evidence.example.test/guide.md"],
|
||||
|
||||
@@ -13,6 +13,7 @@ services:
|
||||
THT_BIN: /opt/venv/bin/tht
|
||||
THT_DATA_ROOT: /data
|
||||
SETTINGS_FILE: /data/settings/settings.json
|
||||
THT_EVIDENCE_HOST_REGISTRY_ROOT: ${THT_WORKSPACE_REGISTRY_ROOT:-}
|
||||
THT_MAINTENANCE_FILE: /data/settings/maintenance.json
|
||||
THT_WORKSPACE_REGISTRY_ROOT: /data/workspace-registry
|
||||
THT_WORKSPACE_GIT_REMOTE: ${THT_WORKSPACE_GIT_REMOTE:?set THT_WORKSPACE_GIT_REMOTE}
|
||||
@@ -99,7 +100,7 @@ services:
|
||||
image: thothii-core:local
|
||||
profiles: [catalog-maintenance]
|
||||
pull_policy: never
|
||||
command: ["node", "/app/backend/dist/catalog/migrate.js"]
|
||||
command: ["bash", "/app/docker/catalog-migrate.sh"]
|
||||
environment:
|
||||
THT_CATALOG_DB_HOST: catalog-db
|
||||
THT_CATALOG_DB_PORT: "5432"
|
||||
@@ -153,7 +154,7 @@ services:
|
||||
- type: volume
|
||||
source: workspace-registry
|
||||
target: /data/workspace-registry
|
||||
read_only: true
|
||||
read_only: false
|
||||
- type: volume
|
||||
source: sessions
|
||||
target: /data/sessions
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# Optional local profile: expose the existing curator checkout on the installation host.
|
||||
# Copy the existing registry repo to this path before enabling the override. Snapshot/state
|
||||
# volumes remain unchanged. THT_EVIDENCE_HOST_REGISTRY_ROOT must be an absolute host path.
|
||||
services:
|
||||
core:
|
||||
environment:
|
||||
THT_EVIDENCE_HOST_REGISTRY_ROOT: ${THT_EVIDENCE_HOST_REGISTRY_ROOT:?set an absolute host registry path}
|
||||
volumes:
|
||||
- type: bind
|
||||
source: ${THT_EVIDENCE_HOST_REGISTRY_ROOT:?set an absolute host registry path}/repo
|
||||
target: /data/workspace-registry/repo
|
||||
bind:
|
||||
create_host_path: false
|
||||
workspace-maintenance:
|
||||
volumes:
|
||||
- type: bind
|
||||
source: ${THT_EVIDENCE_HOST_REGISTRY_ROOT:?set an absolute host registry path}/repo
|
||||
target: /data/workspace-registry/repo
|
||||
bind:
|
||||
create_host_path: false
|
||||
@@ -45,7 +45,7 @@ services:
|
||||
- type: bind
|
||||
source: ${THT_WORKSPACE_REGISTRY_ROOT:?set THT_WORKSPACE_REGISTRY_ROOT}
|
||||
target: /data/workspace-registry
|
||||
read_only: true
|
||||
read_only: false
|
||||
- type: bind
|
||||
source: ${THT_DATA_ROOT:?set THT_DATA_ROOT}/workspace-secrets
|
||||
target: /data/workspace-secrets
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
# The image is tagged from the running frontend before the visual review is published.
|
||||
services:
|
||||
frontend:
|
||||
image: thothii-frontend:before-visual-review-20260912
|
||||
@@ -0,0 +1,8 @@
|
||||
# Local-only review override. Apply after the existing local installation profiles.
|
||||
# Changes only the frontend image; Core, configuration and volumes remain unchanged.
|
||||
services:
|
||||
frontend:
|
||||
image: thothii-frontend:visual-review-20260912
|
||||
build:
|
||||
context: /Users/mp/projects/ThothII-visual-review
|
||||
dockerfile: docker/frontend.Dockerfile
|
||||
@@ -0,0 +1,5 @@
|
||||
# Local-only rollback to the full-shell frontend before visual integration.
|
||||
# Apply after local installation profiles, with --no-build and --no-deps.
|
||||
services:
|
||||
frontend:
|
||||
image: thothii-frontend:before-visual-shell-merge-20260913
|
||||
@@ -0,0 +1,17 @@
|
||||
services:
|
||||
core:
|
||||
environment:
|
||||
THT_EVIDENCE_HOST_REGISTRY_ROOT: /Users/mp/projects/ThothII/deploy/psd/evidence-registry
|
||||
volumes:
|
||||
- type: bind
|
||||
source: /Users/mp/projects/ThothII/deploy/psd/evidence-registry/repo
|
||||
target: /data/workspace-registry/repo
|
||||
bind:
|
||||
create_host_path: false
|
||||
workspace-maintenance:
|
||||
volumes:
|
||||
- type: bind
|
||||
source: /Users/mp/projects/ThothII/deploy/psd/evidence-registry/repo
|
||||
target: /data/workspace-registry/repo
|
||||
bind:
|
||||
create_host_path: false
|
||||
@@ -2,6 +2,10 @@
|
||||
# Replace every absolute path before using this as an advanced reference.
|
||||
schemaVersion: 2
|
||||
profile: local
|
||||
# Mac standalone example; the Omics server requires embedded/upstream separately.
|
||||
shell:
|
||||
mode: full
|
||||
defaultLocale: en
|
||||
projectDirectory: "<abs>/projects/ThothII"
|
||||
envFile: "<abs>/projects/ThothII/deploy/psd/operator.env"
|
||||
workspaceRepository:
|
||||
@@ -10,22 +14,28 @@ workspaceRepository:
|
||||
access: ssh
|
||||
modelCatalog:
|
||||
defaults:
|
||||
session: zai/glm-5.3
|
||||
metadataGeneration: zai/glm-5.3
|
||||
interaction: zai/glm-5.3
|
||||
embedding:
|
||||
id: ollama/qwen3-embedding:0.6b
|
||||
dimensions: 1024
|
||||
providers:
|
||||
deepseek:
|
||||
authentication:
|
||||
mode: pi_auth
|
||||
mode: secret_env
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
session:
|
||||
mode: pi_builtin
|
||||
metadataGeneration:
|
||||
litellmProvider: deepseek
|
||||
models:
|
||||
deepseek-v4-pro:
|
||||
label: DeepSeek V4 Pro
|
||||
session: {}
|
||||
metadataGeneration: {}
|
||||
deepseek-v4-flash:
|
||||
label: DeepSeek V4 Flash
|
||||
session: {}
|
||||
metadataGeneration: {}
|
||||
zai:
|
||||
endpoint:
|
||||
baseUrl: https://api.z.ai/api/coding/paas/v4
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
node /app/backend/dist/catalog/migrate.js
|
||||
/opt/venv/bin/python -m tht.memory.migrate
|
||||
@@ -131,7 +131,7 @@ ENV PATH="/opt/venv/bin:/usr/local/bin:$PATH" \
|
||||
HOME=/home/thoth
|
||||
|
||||
COPY scripts/verify-line-endings.sh /usr/local/bin/verify-line-endings
|
||||
COPY docker/core-entrypoint.sh docker/workspace-maintenance-entrypoint.sh docker/session-migrate.sh docker/ensure-pi-trust.mjs docker/embedding-model-init.sh /app/docker/
|
||||
COPY docker/core-entrypoint.sh docker/workspace-maintenance-entrypoint.sh docker/session-migrate.sh docker/catalog-migrate.sh docker/ensure-pi-trust.mjs docker/embedding-model-init.sh /app/docker/
|
||||
COPY docker/smoke/core-smoke.sh /app/docker/smoke/core-smoke.sh
|
||||
RUN /usr/local/bin/verify-line-endings /app/docker \
|
||||
&& chmod +x /app/docker/core-entrypoint.sh /app/docker/workspace-maintenance-entrypoint.sh /app/docker/session-migrate.sh /app/docker/embedding-model-init.sh /app/docker/smoke/core-smoke.sh
|
||||
|
||||
@@ -1,6 +1,26 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
test "$(cat /usr/share/nginx/html/config.js)" = 'window.__THOTHII_CONFIG__ = {};'
|
||||
# The generic image contains an empty fallback; installed containers mount the generated
|
||||
# public projection over it. An optional path lets host projection tests use this same check.
|
||||
config_file=${1:-/usr/share/nginx/html/config.js}
|
||||
test -f "$config_file" && test -r "$config_file"
|
||||
config=$(tr -d '[:space:]' < "$config_file")
|
||||
case "$config" in
|
||||
'window.__THOTHII_CONFIG__={};')
|
||||
test -z "${THT_FRONTEND_CONFIG_REVISION:-}"
|
||||
;;
|
||||
*)
|
||||
# Installation Load validates BCP47 syntax. Here check only the public payload shape;
|
||||
# catalog availability and fallback belong to the frontend, not the generic image.
|
||||
locale='[A-Za-z][A-Za-z0-9]*(-[A-Za-z0-9]+)*'
|
||||
full="\"mode\":\"full\",\"defaultLocale\":\"$locale\""
|
||||
embedded="\"mode\":\"embedded\",\"defaultLocale\":\"$locale\",\"adapter\":\"omics-portal\""
|
||||
if ! printf '%s\n' "$config" | LC_ALL=C grep -Eq "^window\.__THOTHII_CONFIG__=\{\"backendBaseUrl\":\"/api\",\"shell\":\{($full|$embedded)\}\};$"; then
|
||||
echo "Invalid frontend public runtime configuration" >&2
|
||||
exit 1
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
printf '%s\n' "frontend runtime config smoke: ok"
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-09-08
|
||||
---
|
||||
|
||||
# Use PostgreSQL for Memory and Qdrant for retrieval
|
||||
|
||||
The new Memory module uses the installation's existing PostgreSQL service as the
|
||||
authority for Memory Cards, their links, and structured schema dependencies.
|
||||
Qdrant holds a rebuildable search projection. This replaces the JSONL registry and
|
||||
allows related administrative changes to be coordinated in one database transaction.
|
||||
The owner accepted this direction in Q9 of the Memory design interview; implementation
|
||||
is pending.
|
||||
|
||||
Memory uses its own tables and remains a separate module from the Metadata Catalog.
|
||||
Sharing the PostgreSQL service does not transfer Memory ownership to Database
|
||||
management or make Evidence and Memory one canonical domain.
|
||||
|
||||
## Retrieval and graph
|
||||
|
||||
The owner also accepted hybrid semantic/lexical Qdrant retrieval with scope filters
|
||||
and explicit card links traversed in core. Both belong to the planned first version;
|
||||
there is no dedicated graph database or external Memory framework.
|
||||
|
||||
This extends [ADR 0017](0017-separate-reference-vectors-from-runtime-memory.md):
|
||||
the separate reference and memory collections remain, while Memory gains sparse
|
||||
lexical indexing in addition to dense vectors. Memory projections become rebuildable
|
||||
from the module's PostgreSQL authority. Preprocessing Clear still preserves Memory;
|
||||
this decision does not add it to the preprocessing cleanup scope.
|
||||
|
||||
Links support discovery. Finding a card through a link does not approve its use.
|
||||
The core proposes links with the cards for the same final review; Administration
|
||||
provides manual creation, editing, and deletion. Deleting a card removes its incident
|
||||
links without deleting the other linked cards.
|
||||
|
||||
## Considered options
|
||||
|
||||
- Retaining JSONL preserves the current storage format, but leaves coordinated
|
||||
card/link/dependency mutations and concurrent administrative writes to application code.
|
||||
- Using Qdrant as the sole authority is a viable alternative for record storage and
|
||||
retrieval. PostgreSQL is preferred for the coordinated mutations of the new module,
|
||||
with Qdrant reserved for its search projection.
|
||||
- Adding a dedicated graph database would introduce another service; the selected
|
||||
bounded traversal can be implemented in core over persisted links.
|
||||
|
||||
## Consequences
|
||||
|
||||
The module needs a persistence contract, PostgreSQL schema, and explicit propagation
|
||||
of additions, updates, and deletions to Qdrant. Choosing PostgreSQL does not make this
|
||||
propagation atomic across both systems: failures, retries, and invalidation of stale
|
||||
search content must be handled and tested. An index rebuild uses the original card
|
||||
content and cannot resurrect deleted cards.
|
||||
|
||||
The decision does not introduce Memory revision history or require compatibility with
|
||||
existing development sessions. Evidence authoring and publication remain governed by
|
||||
their own contract until the Evidence management project defines its evolution.
|
||||
|
||||
The [Memory management project](../plans/2026-09-08-memory-management.md) records the
|
||||
approved behavior, scope, and integration work.
|
||||
@@ -0,0 +1,152 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-09-08
|
||||
---
|
||||
|
||||
# Maintain local Evidence with external editors and manual consolidation
|
||||
|
||||
Evidence management must provide complete access to registered Evidence and a
|
||||
maintenance path, as accepted in Q11–Q15 and revised during simplification. The owner subsequently
|
||||
clarified that a context specialist writes drafts independently of the installation
|
||||
and without PostgreSQL access; the system refines and stores them locally for use
|
||||
and maintenance. Implementation is pending.
|
||||
|
||||
The storage mechanics of Q11 were revisited in the
|
||||
[simplification review](../plans/2026-09-08-memory-evidence-simplification-review.md).
|
||||
The original choice combined the workspace repository as Evidence authority with
|
||||
application-managed Git writes. The owner accepted the separation of external draft
|
||||
files/repositories from a durable local canonical file archive and removes
|
||||
commit/push from normal CRUD. The management surface must expose discoverable,
|
||||
editable Markdown for domain specialists, without JSONL editing. The owner chose
|
||||
external editors for release 0: their preferred Mac/PC editor or vim/nano on the
|
||||
server. No web editor or content form is required. The page keeps browsing,
|
||||
filtering, and detail, with actual host file paths and maintenance instructions.
|
||||
The owner also requested manual consolidation followed by manual Git diff, commit,
|
||||
and push, relying on operator discipline rather than automation. The owner accepted
|
||||
the remaining simplifications and requested explicit clarification of the core
|
||||
format change and the simple terminal-based Git check. The original application Git writer
|
||||
is not the final implementation prescription. A suggestion
|
||||
to move Evidence authority into PostgreSQL was withdrawn after the clarification.
|
||||
Memory remains a distinct module with its own PostgreSQL authority under
|
||||
[ADR 0018](0018-use-postgres-for-memory-and-qdrant-for-retrieval.md).
|
||||
|
||||
## Manual consolidation and Git follow-up
|
||||
|
||||
The external-editor decision replaces the form's single Save with saving files and
|
||||
running one consolidation command. The command checks expected structure, required
|
||||
fields, types, identifiers, references, and provenance. It reports the affected file
|
||||
and the necessary correction. Invalid input stops activation; valid input updates
|
||||
derived metadata, the corpus, and the Evidence index. New and deleted files use the
|
||||
same path. An inaccessible archive must not be interpreted as deleted Evidence.
|
||||
Structural validation does not prove semantic correctness or rerun model refinement
|
||||
to rewrite manually edited rules.
|
||||
|
||||
The core reads only successfully consolidated active content, never the editable
|
||||
working files directly. Candidate validation and activation must preserve the last
|
||||
valid corpus on failure, or report unavailability if its integrity cannot be
|
||||
guaranteed. Unconsolidated edits must not mix new text with old retrieval data.
|
||||
|
||||
Consolidation reports full local success only once the updated content is available
|
||||
to the core. If persistence succeeds but activation fails, the command reports the
|
||||
partial outcome and can be rerun without duplicating units or losing edits.
|
||||
The operation does not imply atomicity across
|
||||
authoritative storage and Qdrant; their failure and recovery behavior requires
|
||||
explicit implementation. An interrupted operation must not leave a partial corpus
|
||||
advertised as ready for subsequent use.
|
||||
|
||||
The maintained archive is a persistent Git working tree containing the local
|
||||
Evidence; it may reuse the workspace repository. Draft sources and curated units
|
||||
remain distinct even when in the same repository. Setup and the page identify the
|
||||
repository, working tree, actual host paths, branch, and configured remote. A
|
||||
rebuildable runtime snapshot is not the editing location.
|
||||
|
||||
Before consolidation, the operator checks Git status, including inspection of new
|
||||
untracked files. Normal terminal Git diff commands are available for line-level
|
||||
inspection when needed; no custom viewer or mandatory double review is required.
|
||||
After consolidation, the operator stages the Evidence and required metadata changes,
|
||||
commits, and pushes using normal Git commands. No watcher, automatic Git writes, retrying push, pull, merge,
|
||||
or dedicated web execution control is required. Missing credentials, unconfigured
|
||||
upstreams, and Git conflicts are handled by the operator.
|
||||
|
||||
Consolidation activates local changes before commit/push. If the operator omits the
|
||||
Git steps or push fails, the local update remains effective while transfer to the
|
||||
remote is incomplete. Pushing does not automatically update other installations.
|
||||
The system relies on operator discipline to complete the sequence; it does not add
|
||||
a separate editorial publication state machine or require a second reviewer.
|
||||
|
||||
Approved conflict repairs from the core still call the same persistence and
|
||||
activation service directly. They do not require an external editor or manual
|
||||
consolidation before the session can use its own approved correction. Their local
|
||||
file changes are included in the operator's subsequent Git maintenance.
|
||||
|
||||
## Administration without concurrent core work
|
||||
|
||||
The owner's simplification instruction replaces the earlier Q12 requirement to
|
||||
refresh open sessions after administrative edits. Core activity can be assumed
|
||||
absent during administration, or its overlap can be ignored. No dedicated live
|
||||
update, session notification, restart, maintenance mode, or reader coordination
|
||||
is required. Completed administrative changes apply to subsequent work.
|
||||
|
||||
Deliberate writes from the workflow itself still exist: approved conflict repairs
|
||||
and the final Memory summary use the same persistence services and handle their
|
||||
outcome before proceeding. In particular, a session must be able to use its own
|
||||
approved Evidence correction. This does not require updating all other sessions
|
||||
or rewriting previously approved decisions, artifacts, or SQL.
|
||||
|
||||
## Manual corrections survive source updates
|
||||
|
||||
When an updated source contradicts an administrator's correction, the saved manual
|
||||
Evidence remains active. Evidence management shows the conflicting content for an
|
||||
explicit decision and subsequent save. The source's newer content does not
|
||||
automatically override the correction. The owner accepts that the manual rule can
|
||||
remain in use until that comparison is resolved.
|
||||
|
||||
Deleted Evidence must not silently reappear after preparation or reindexing. The
|
||||
authoring implementation must preserve both manual corrections and suppression of
|
||||
deleted units through source refreshes. The current protection for uncommitted Git
|
||||
changes is insufficient: it does not preserve already committed manual corrections
|
||||
against regeneration, and retirement currently removes the unit without recording
|
||||
suppression for later preparation.
|
||||
|
||||
## Manual creation and explicit source refresh
|
||||
|
||||
Administrators can create Evidence without providing an external document. In R0,
|
||||
they add Markdown using the documented example and consolidate it. The application
|
||||
records a manual declaration as the managed source of the current statement.
|
||||
Explicit consolidation approves that declaration; it does not
|
||||
claim independent documentary verification.
|
||||
|
||||
A correction that changes a unit's meaning uses the same explicit manual origin.
|
||||
The original document remains linked for provenance and source-change detection,
|
||||
but its excerpt is not presented as support for a rule it does not contain. The
|
||||
canonical contract must distinguish the current supporting source from the original
|
||||
document rather than silently retaining outdated support metadata.
|
||||
|
||||
External sources are reacquired only when an administrator requests a source
|
||||
refresh. Ordinary saves and session lookups use the acquired local content; they
|
||||
do not poll sources or fetch their current versions. A remote change is therefore
|
||||
detected at the next requested refresh, when the manual-precedence rule applies.
|
||||
The revised storage recommendation reads the specialist's repository as an input;
|
||||
ordinary local edits do not write back to it or refresh its content automatically.
|
||||
|
||||
## Implementation consequences
|
||||
|
||||
The existing [Evidence lifecycle](../evidence.md) and
|
||||
[Workspace Evidence v3 contract](../contracts/workspace-evidence-v3.md) require
|
||||
explicit evolution for editable local Markdown, manual consolidation, and manual
|
||||
precedence. Source provenance and coherent canonical metadata remain requirements;
|
||||
manual changes must not be presented as statements supported by an unrelated source
|
||||
excerpt. Under the accepted local-file direction, curated files are primary
|
||||
installation data preserved by Clear and backed up separately from rebuildable
|
||||
indexes. Visible Markdown content must become authoritative for editing; operators
|
||||
must not maintain hidden duplicate text, hashes, or manifest entries themselves.
|
||||
This is a core Evidence contract change: parser, renderer, authoring, validation,
|
||||
normalization, and their integration with indexing and recall must be adapted
|
||||
together. E1 includes a new Curated unit format version, conversion of existing
|
||||
Evidence, reindexing, and end-to-end verification that visible edits reach core
|
||||
consumption. Preserve the internal typed model where possible; the change does
|
||||
not require redesigning the NL-to-SQL workflow phases.
|
||||
Local edits do not automatically change external drafts or another installation;
|
||||
Git versioning and transfer occur through the explicit manual follow-up.
|
||||
The [Evidence management project](../plans/2026-09-08-evidence-management.md)
|
||||
records the implementation sequence and acceptance checks.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Unify administration pages and use namespaced routes
|
||||
|
||||
status: accepted
|
||||
|
||||
Workspace and Pi management will become full Administration Pages alongside Evidence, Memory and
|
||||
Database, using one shared Administrative Page Family. The Embedded Thoth Shell will identify the
|
||||
active Administration Surface through a namespaced browser route, `thoth_route=administration/<surface>`,
|
||||
so links remain reloadable and support Back/Forward without requiring new host-portal server routes.
|
||||
Transient form values and secrets stay outside the route; destructive confirmations may remain
|
||||
short-lived dialogs because they are actions, not page containers.
|
||||
|
||||
We considered React-only state, path-based routes and hash routes. React-only state loses refresh and
|
||||
deep-link behavior, path routes require a host catch-all route that does not yet exist in Omics Portal,
|
||||
and hash routes can conflict with host-page fragments. A namespaced query route preserves the current
|
||||
same-document integration boundary while leaving room for a future path adapter.
|
||||
|
||||
The accepted visual direction is A / Workbench for all five surfaces; B and C remain recoverable
|
||||
prototypes. Workspace owns readiness and preprocessing, consuming the Database catalog as a
|
||||
prerequisite. Database owns connection/binding, schema synchronization, descriptions and sensitivity;
|
||||
the preprocessing action remains exclusively in Workspace preparation.
|
||||
|
||||
On 2026-09-12, the owner refined this direction in
|
||||
[Gitea #29](https://git.tylconsulting.it/mptyl/ThothII/issues/29): Database opens at its
|
||||
catalog list, with its original table space and margins, and no Workspace preparation footer.
|
||||
This supersedes the earlier requirement for a preparation link in that page. Workspace
|
||||
remains reachable through the shared Administration navigation; its Preparation tab still
|
||||
links to Database configuration and schema when catalog work is needed.
|
||||
@@ -0,0 +1,101 @@
|
||||
# ADR 0021 — Shell separati e adapter sostituibile per il portale
|
||||
|
||||
- Stato: accettato
|
||||
- Data: 2026-09-13
|
||||
|
||||
## Decisione
|
||||
|
||||
ThothII espone due modalità di installazione, selezionate da `shell.mode`:
|
||||
|
||||
- `embedded` (default): ThothII è ospitato da Omics Portal. Non renderizza alcun header e
|
||||
riceve dal portale lingua, tema e fullscreen. L'accesso resta verificato dal server.
|
||||
- `full`: ThothII è autonomo. Renderizza il proprio header, con selettore lingua, tema,
|
||||
fullscreen e nome utente. Il click sul nome apre il logout. Non mostra mai la rotellina o
|
||||
altri comandi amministrativi del portale. Mantiene un rail vuoto a sinistra di almeno 20 px.
|
||||
|
||||
Il fatto che la shell sia `full` è distinto dallo stato `fullscreen`: la prima decide quale
|
||||
contenitore viene renderizzato, il secondo indica se è attiva la Fullscreen API del browser.
|
||||
L'icona passa da “entra in fullscreen” a “torna alla modalità normale”; anche `Esc` aggiorna lo
|
||||
stato visualizzato.
|
||||
|
||||
La configurazione installata resta semplice e retrocompatibile. Sul Mac di sviluppo il profilo
|
||||
locale userà:
|
||||
|
||||
```yaml
|
||||
shell:
|
||||
mode: full
|
||||
defaultLocale: en
|
||||
```
|
||||
|
||||
Il deploy sul server userà invece:
|
||||
|
||||
```yaml
|
||||
shell:
|
||||
mode: embedded
|
||||
adapter: omics-portal
|
||||
```
|
||||
|
||||
`defaultLocale` indica la lingua iniziale della shell full; in embedded la fonte autorevole resta
|
||||
il portale.
|
||||
|
||||
L'adapter è l'unico confine tra ThothII e il portale. La sua interfaccia pubblica è volutamente
|
||||
profonda e minima: consegna solo snapshot dello stato, senza esporre comandi, token, identità o
|
||||
dettagli di trasporto.
|
||||
|
||||
```ts
|
||||
export type HostShellState = {
|
||||
locale: string; // BCP-47, inizialmente it/en
|
||||
theme: "light" | "dark";
|
||||
fullscreen: boolean;
|
||||
};
|
||||
|
||||
export interface PortalAdapter {
|
||||
subscribe(
|
||||
onState: (state: HostShellState) => void,
|
||||
onError: (error: Error) => void,
|
||||
): () => void;
|
||||
}
|
||||
```
|
||||
|
||||
`OmicsPortalAdapter` è l'implementazione corrente. Un adapter per un altro portale potrà
|
||||
sostituirlo senza modificare shell, i18n o workflow. In `full` l'adapter non viene istanziato:
|
||||
lo stato è gestito internamente dalla shell.
|
||||
|
||||
Per l'integrazione oggi operativa, che monta la SPA direttamente nel DOM di Omics Portal,
|
||||
l'adapter legge la lingua effettiva dal selettore Omics, osserva l'attributo del tema e ascolta
|
||||
il fullscreen del documento. Selettori e osservatori restano privati dell'implementazione Omics.
|
||||
La lingua segue il normale ricaricamento Django; tema e fullscreen cambiano nella pagina aperta.
|
||||
La revisione approvata del 2026-09-13 elimina il precedente handshake a eventi: non servono
|
||||
messaggi personalizzati, versioni di trasporto o timeout di avvio. Un futuro adapter potrà usare
|
||||
un diverso trasporto senza modificare l'interfaccia applicativa.
|
||||
|
||||
Se `shell` o l'adapter embedded sono omessi, si usa `embedded` con `omics-portal`. Questo default
|
||||
supporta il documento Omics esistente; nomi adapter sconosciuti o dati host mancanti producono
|
||||
un errore esplicito, senza attivare la shell full.
|
||||
|
||||
## Confini che restano invariati
|
||||
|
||||
L'identità e l'autorizzazione del backend non vengono ricostruite nel browser. In embedded,
|
||||
Omics Portal continua a gestire login e logout e la catena server-side `auth_request` continua a
|
||||
fornire i principal header già previsti. Lo stato UI non dichiara l'utente autenticato: il modulo
|
||||
di accesso usa la verifica backend esistente anche alla riconnessione e al ritorno alla pagina.
|
||||
Un rifiuto su una singola operazione non equivale automaticamente alla perdita dell'accesso.
|
||||
|
||||
La lingua UI e la lingua di interazione con il modello restano separate dalla lingua del
|
||||
workspace; il relativo contratto è in [ADR 0022](0022-separate-ui-locale-from-session-interaction-language.md).
|
||||
|
||||
## Alternative scartate
|
||||
|
||||
- Duplicare l'header di Omics in embedded: crea due fonti di stato e incompatibilità visive.
|
||||
- Spargere controlli `if embedded/full` nei componenti: lega ogni pagina al portale.
|
||||
- Trasmettere utente o token nel bridge: aumenta superficie e accoppia UI e autenticazione.
|
||||
- Introdurre un protocollo completo request/response: non aggiunge funzionalità richiesta.
|
||||
- Usare `profile` per distinguere le shell: `profile` descrive la topologia dell'installazione,
|
||||
non la sua presentazione.
|
||||
|
||||
## Conseguenze
|
||||
|
||||
La soluzione richiede un adapter nel frontend che osserva il documento condiviso e mantiene la
|
||||
logica di shell locale a ThothII. Il backend non necessita di un nuovo protocollo di autenticazione
|
||||
o di una nuova sessione browser. Un nuovo portale deve soddisfare anche il contratto server di
|
||||
identità fidata: la sola sostituzione della classe UI non sostituisce quel contratto.
|
||||
@@ -0,0 +1,51 @@
|
||||
# Separate UI locale from session interaction language
|
||||
|
||||
status: accepted
|
||||
|
||||
ThothII distinguishes three language concepts:
|
||||
|
||||
- `workspace.language` remains the language of workspace-owned documents, descriptions and Evidence;
|
||||
- `ui_locale` controls deterministic ThothII chrome such as labels, form help, placeholders, errors,
|
||||
accessibility text and review-widget chrome;
|
||||
- `interaction_language` is persisted in a session and controls model-generated questions,
|
||||
explanations and reviewer proposals.
|
||||
|
||||
The initial locale catalog supports Italian and English and uses extensible BCP-47 language tags.
|
||||
Missing deterministic translations fall back to English. The selected UI locale supplies the default
|
||||
interaction language when a new session is created. A resumed session always uses its persisted
|
||||
interaction language; changing the host or full-shell UI locale must not silently rewrite an existing
|
||||
session or make its model output switch language mid-workflow.
|
||||
|
||||
The distinction is required because the current workspace contract already uses `language` for
|
||||
content and the PSD workspace is Italian. Reusing that field for a browser preference would make a
|
||||
visual choice mutate domain content semantics. The model receives the session interaction language
|
||||
through the session/Pi workflow context. SQL, identifiers, database values and other technical
|
||||
artifacts remain governed by their existing contracts and are not translated as UI strings.
|
||||
|
||||
In `full`, the local shell owns `ui_locale` and supplies it when starting a new session. In
|
||||
`embedded`, the host adapter is authoritative for `ui_locale`; ThothII applies host changes to
|
||||
deterministic UI immediately while preserving the interaction language of any active session.
|
||||
|
||||
We considered using only `workspace.language`, using only a global browser locale, and translating
|
||||
the model output after generation. The first conflates domain content with UI preference; the second
|
||||
cannot preserve a session's language or follow the host portal; and the third would be unsafe for
|
||||
structured reviewer decisions and would not control the model's reasoning or proposal language.
|
||||
|
||||
## Considered Options
|
||||
|
||||
- One mutable `language` field for workspace, UI and session was rejected because the fields have
|
||||
different owners and lifecycles.
|
||||
- Client-only translation of reviewer choices was rejected because choices can be generated by the
|
||||
model and must be requested in the intended language.
|
||||
- An English-only deterministic chrome was rejected because embedded and full installations must
|
||||
follow the selected host/user language.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Session creation and the persisted manifest gain an explicit interaction-language value.
|
||||
- Legacy manifests without that value use the workspace language, pinned idempotently on first
|
||||
resume; the browser locale must not determine this compatibility value.
|
||||
- Resume must read that value from the manifest and must not accept a new locale as an override.
|
||||
- The workflow prompt contract and deterministic reviewer-widget builders need a locale-aware input.
|
||||
- Frontend strings need a catalog and stable keys; backend events should expose stable codes where
|
||||
the frontend is responsible for localization.
|
||||
@@ -0,0 +1,135 @@
|
||||
# Rendering full ed embedded
|
||||
|
||||
ThothII ha una sola applicazione React, una sola build Vite e gli stessi servizi
|
||||
backend. «Doppio rendering» significa due modi di ospitare quella applicazione,
|
||||
non due versioni delle pagine e non rendering React sul server. Django renderizza
|
||||
il contenitore Omics; React renderizza ThothII nel browser, dentro `#root`.
|
||||
|
||||
## Tre decisioni indipendenti
|
||||
|
||||
| Decisione | Configurazione | Effetto |
|
||||
| --- | --- | --- |
|
||||
| Distribuzione | `profile: local` oppure `server` | Compose, percorsi e vincoli operativi |
|
||||
| Presentazione | `shell.mode: full` oppure `embedded` | Proprietario di header e preferenze |
|
||||
| Autenticazione | `auth.yaml` local/OIDC oppure `AUTH_MODE=upstream` | Chi verifica l'identità, come arriva al backend |
|
||||
|
||||
Il Mac usa **full + local**, con lingua iniziale inglese. L'integrazione Omics
|
||||
usa **embedded + upstream**, con accesso già verificato dal portale. Un server
|
||||
autonomo può usare **full + oidc**. Cambiare `shell.mode` non abilita un metodo
|
||||
di autenticazione e non modifica permessi o proprietari delle sessioni.
|
||||
|
||||
Full con upstream può visualizzare un'identità già verificata dal proxy, ma non
|
||||
ha un logout ThothII disponibile: non è il profilo autonomo con login/logout.
|
||||
Embedded non avvia login locale o OIDC anche se il backend è configurato così;
|
||||
questa combinazione non realizza il login unico Omics e non va usata come fallback.
|
||||
|
||||
## Composizione comune
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
CONFIG["config.js pubblico"] --> SHELL["ShellProvider"]
|
||||
FULL["Preferenze full nel browser"] --> SHELL
|
||||
HOST["Documento Omics"] --> ADAPTER["OmicsPortalAdapter: solo presentazione"]
|
||||
ADAPTER --> SHELL
|
||||
SHELL --> GATE["AuthGate: verifica GET /me"]
|
||||
GATE --> APP["AppShell: stesse pagine, sessioni e amministrazione"]
|
||||
AUTH["Backend: cookie locale/OIDC o identità upstream"] --> GATE
|
||||
```
|
||||
|
||||
`ShellProvider` risolve la configurazione, applica lingua/tema e monta i contenuti
|
||||
solo dopo uno snapshot host valido in embedded. `AuthGate` verifica l'accesso;
|
||||
`AppShell` e le pagine non devono leggere selettori o eventi specifici di Omics.
|
||||
Il cambio utente smonta lo stato applicativo della precedente identità.
|
||||
|
||||
## Full
|
||||
|
||||
- Header ThothII rosso Omics `#CB333B` in entrambi i temi; logo interamente chiaro.
|
||||
- Selettore EN/IT, tema light/dark, fullscreen e nome verificato dell'utente.
|
||||
- Menu del nome con logout soltanto per local/OIDC; nessuna rotellina admin.
|
||||
L'amministrazione resta nella navigazione applicativa, secondo i permessi.
|
||||
- Margine sinistro vuoto e simmetrico al destro: `max(20px, 1.5rem)`, normalmente
|
||||
24px con radice a 16px. Non è una seconda sidebar di navigazione.
|
||||
- Lingua e tema ricordati sullo stesso origin in `localStorage`, nelle chiavi
|
||||
`thothii:shell:locale` e `thothii:shell:theme`. Non sono preferenze server per
|
||||
utente. In assenza di preferenze: `defaultLocale` e tema light.
|
||||
- Fullscreen usa `document.documentElement.requestFullscreen()` e
|
||||
`document.exitFullscreen()`: nasconde il contorno del browser dove supportato.
|
||||
L'icona cambia sullo stato reale, anche dopo Esc; un rifiuto mostra un errore.
|
||||
Non è un semplice ingrandimento CSS e non scatta automaticamente all'accesso.
|
||||
|
||||
## Embedded
|
||||
|
||||
- Nessun header ThothII, selettore lingua, toggle tema, login o logout autonomo.
|
||||
I controlli rimangono nell'header generale Omics.
|
||||
- React è nello stesso documento della pagina `/kokoro/datamart-builder/`, non
|
||||
in un iframe. Non serve `postMessage` né un secondo protocollo di sessione.
|
||||
- L'adapter legge la lingua Django già confermata, osserva il tema del documento
|
||||
e ascolta il fullscreen reale. Le azioni rimangono di proprietà del portale.
|
||||
- Un contesto Omics mancante o invalido mostra un errore d'integrazione; non
|
||||
passa silenziosamente a full e non offre un secondo login.
|
||||
- Il portale assegna l'altezza disponibile sotto il proprio header: catena flex
|
||||
con `min-height: 0`, root contenuto e altezza applicativa vincolata al contenitore.
|
||||
Il contratto ThothII espone `--thoth-app-height` (fallback `100dvh`); verificare
|
||||
il contenitore reale, non presumere che l'intera viewport appartenga a React.
|
||||
Il template Omics mantiene inoltre i suoi override di compatibilità.
|
||||
|
||||
Il reset CSS è limitato al mount React e ai popup dell'applicazione, senza
|
||||
richiedere CSS `@scope`. I token e i popup seguono il tema applicativo. Questo
|
||||
non rende indipendenti fogli di stile arbitrari caricati dal portale: la verifica
|
||||
del documento condiviso rimane necessaria a ogni integrazione.
|
||||
|
||||
## Caricamento e configurazione pubblica
|
||||
|
||||
Il descrittore installato è la sorgente di verità. Il CLI genera
|
||||
`generated/frontend/config.js` e il suo mount di sola lettura nella proiezione
|
||||
`generated/compose.models.yaml`. Il file pubblico contiene solo `backendBaseUrl`
|
||||
e `shell`, mai identità, token, password o percorsi host. Va caricato **prima** del
|
||||
modulo React e servito senza cache. Nessuna build separata è richiesta per
|
||||
cambiare modalità; occorre rigenerare e applicare i mount tramite il lifecycle.
|
||||
|
||||
L'ordine Omics è: config pubblico → override del solo prefisso API → asset dal
|
||||
manifest Vite. L'override deve conservare `shell`; l'adapter non configura il proxy.
|
||||
Il default completo di shell omessa è embedded/en/omics-portal. Il CLI normalizza
|
||||
anche singoli campi omessi; un oggetto `shell` scritto manualmente nel browser
|
||||
deve invece contenere `mode` e `defaultLocale`, altrimenti viene rifiutato.
|
||||
|
||||
## Lingua, continuità e dati
|
||||
|
||||
La lingua UI traduce il testo dell'applicazione, non i contenuti di dominio.
|
||||
Alla creazione, la lingua UI viene acquisita come `interactionLanguage` di ripiego.
|
||||
Il CLI riconosce la lingua della domanda originale e salva `interaction_language`
|
||||
nel manifest; usa il ripiego solo per input troppo brevi, ambigui o composti da codice.
|
||||
Questa lingua governa domande, spiegazioni, scelte e controlli HITL. Il gate la
|
||||
include nei descrittori e il frontend la applica al sottoalbero dei widget,
|
||||
senza cambiare la lingua della navigazione.
|
||||
Alla ripresa vale la lingua salvata, non l'ultima scelta dell'header. Per i manifest
|
||||
precedenti senza campo viene riconosciuta e fissata la lingua della domanda,
|
||||
con la lingua workspace disponibile alla prima ripresa come ripiego.
|
||||
|
||||
Il cambio lingua Omics invia il form Django e ricarica la pagina. ThothII conserva
|
||||
solo l'ID della selezione in `sessionStorage`, separato per pathname, issuer e
|
||||
subject. Riapre i documenti, non avvia una generazione. Bozze non inviate e modifiche
|
||||
non salvate richiedono conferma prima della navigazione; non sono una trascrizione
|
||||
salvata. La ripresa operativa resta esplicita.
|
||||
|
||||
## Punti di implementazione e manutenzione
|
||||
|
||||
| Sorgente | Responsabilità |
|
||||
| --- | --- |
|
||||
| `tools/tht/internal/config/shell.go` | Normalizzazione e validazione del descrittore |
|
||||
| `tools/tht/internal/modelprojection/projection.go` | Config pubblico e mount generati |
|
||||
| `frontend/src/api/runtime-config.ts` | Validazione browser e prefisso API same-origin |
|
||||
| `frontend/src/shell/host/ShellProvider.tsx` | Composizione, preferenze e tema |
|
||||
| `frontend/src/shell/host/FullHeader.tsx` | Controlli solo full |
|
||||
| `frontend/src/shell/host/OmicsPortalAdapter.ts` | Conoscenza del documento Omics |
|
||||
| `frontend/src/auth/AuthGate.tsx` | Accesso e ricontrolli al ritorno alla pagina |
|
||||
| `backend/src/auth/auth.ts` e `principal.ts` | Verifica server dell'identità |
|
||||
|
||||
Per un altro portale servono un'implementazione del
|
||||
[PortalAdapter](../contracts/portal-shell-adapter-v1.md), la sua registrazione nei
|
||||
validatori CLI/browser e nel punto di composizione, oltre al
|
||||
[contratto di autenticazione server](../install/authentication-upstream.md).
|
||||
Il nome di una classe non è un plugin caricabile dinamicamente da YAML.
|
||||
|
||||
Procedure: [configurazione e deploy](../operations/shell-and-localization.md),
|
||||
[autenticazione](authentication.md), [accettazione](../testing/authentication-manual-acceptance.md).
|
||||
@@ -1,18 +1,27 @@
|
||||
# Authentication architecture
|
||||
|
||||
ThothII has two production authentication modes: `local` and generic `oidc`. The host operator
|
||||
surface is one CLI, `tht`; there is no separate authentication executable. The backend owns
|
||||
opaque browser sessions and authorization, while `tht` owns protected configuration and local-user
|
||||
files.
|
||||
ThothII supports `local` and generic `oidc` through protected `auth.yaml`, plus
|
||||
the trusted-proxy `upstream` path used by Omics. The host operator surface is
|
||||
one CLI, `tht`; there is no separate authentication executable. The backend owns
|
||||
authorization in all paths and opaque browser sessions only in local/OIDC.
|
||||
`tht` owns protected local/OIDC configuration and local-user files; upstream
|
||||
identity is supplied per request by the authenticated server proxy.
|
||||
|
||||
Presentation is separate: [full/embedded rendering](application-shell.md) does
|
||||
not select authentication. The Mac uses full/local; Omics uses embedded/upstream;
|
||||
a standalone server can use full/OIDC. Do not configure a second ThothII OIDC
|
||||
login simply because Omics itself authenticates users through Authentik.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
BROWSER["Browser"] --> BOUNDARY["Authentication boundary"]
|
||||
BOUNDARY --> LOCAL["Local users\nArgon2id hashes"]
|
||||
BOUNDARY --> OIDC["OIDC provider\nAuthorization Code PKCE"]
|
||||
BOUNDARY --> UPSTREAM["Trusted proxy\nVerified portal session"]
|
||||
OIDC --> GROUPS["Groups claim\nexact mapping"]
|
||||
LOCAL --> PRINCIPAL["Thoth principal"]
|
||||
GROUPS --> PRINCIPAL
|
||||
UPSTREAM --> PRINCIPAL
|
||||
PRINCIPAL --> ROLES["Roles"]
|
||||
ROLES --> PERMISSIONS["Permissions"]
|
||||
PERMISSIONS --> ROUTES["Protected routes"]
|
||||
@@ -21,6 +30,13 @@ flowchart TB
|
||||
|
||||
## Configuration and trust boundaries
|
||||
|
||||
The following protected-file configuration applies to local/OIDC. Upstream uses
|
||||
`AUTH_MODE=upstream` without a mounted `auth.yaml` or authentication runtime
|
||||
projection. The backend refuses both authorities together. `AUTH_MODE=none`
|
||||
and `mock` are development/test modes, not production fallbacks. The exact
|
||||
upstream setup, header contract, proxy hops and origin checks are in the
|
||||
[server integration guide](../install/authentication-upstream.md).
|
||||
|
||||
The installation descriptor points to an operator-controlled authentication directory. It contains
|
||||
non-secret `auth.yaml` and, for local mode, `users.yaml`. POSIX installations use a private
|
||||
directory and owner-only regular files; Windows uses equivalent owner-only ACLs. Secret values are
|
||||
@@ -33,17 +49,26 @@ The production role expansion from `backend/src/auth/config.ts` is exact:
|
||||
| Role | Permissions |
|
||||
|---|---|
|
||||
| `user` | `session.use` |
|
||||
| `admin` | `session.use`, `session.read_all`, `session.manage_all`, `settings.manage`, `workspace.manage`, `workspace.secrets.manage`, `database.manage`, `pi.manage`, `auth.diagnostics.read` |
|
||||
| `admin` | `session.use`, `session.read_all`, `session.manage_all`, `settings.manage`, `workspace.manage`, `workspace.secrets.manage`, `database.manage`, `memory.manage`, `evidence.manage`, `pi.manage`, `auth.diagnostics.read` |
|
||||
|
||||
`admin` therefore includes the ordinary `session.use` permission. No other role or permission
|
||||
label is part of the production catalog.
|
||||
|
||||
After validating a browser session, the backend expands its roles through the current permission
|
||||
catalog on every request. The permissions saved at login are a historical snapshot, so existing
|
||||
administrator sessions can use newly deployed administration features without signing in again.
|
||||
Session expiry, revocation and local-user role validation still apply before role expansion.
|
||||
|
||||
OIDC is provider-neutral at the browser protocol boundary. Authorization Code + PKCE, issuer,
|
||||
signature, audience, expiry, state, and nonce are validated before a principal is created.
|
||||
Authentik is the first certified group-catalog adapter, not a special browser login mode.
|
||||
|
||||
## Group authorization
|
||||
|
||||
This section describes **ThothII's direct OIDC login**, not the embedded Omics
|
||||
path. Omics checks its own capability and administrator status and supplies
|
||||
normalized identity headers; ThothII does not repeat the OIDC groups exchange.
|
||||
|
||||
OIDC must return a direct, non-empty `groups` claim whose value is a JSON array of strings.
|
||||
Missing, malformed, indirect, or overage-style claims fail closed. The browser callback returns
|
||||
HTTP 401 with the generic code `oidc_callback_failed`; it does not expose the internal reason.
|
||||
@@ -95,6 +120,10 @@ prerequisite fails.
|
||||
|
||||
## Browser sessions
|
||||
|
||||
This section applies only to **local and direct OIDC**. Upstream reuses the
|
||||
portal's authenticated session at the proxy boundary, not a ThothII cookie;
|
||||
its `/me` response has `session: null` and `csrfToken: null`.
|
||||
|
||||
The browser receives only an opaque `HttpOnly`, `SameSite=Lax` cookie named `thothii_session`.
|
||||
State-changing cookie requests require the in-memory CSRF token, same-origin `Origin`, and Fetch
|
||||
Metadata checks when present. The frontend never stores bearer tokens or session secrets in Web
|
||||
@@ -111,5 +140,20 @@ affected sessions. Authentication configuration revision changes invalidate all
|
||||
reload. Logout deletes the server record. Backup restore excludes active sessions and OIDC state,
|
||||
recreates empty private auth-state directories, and therefore forces reauthentication.
|
||||
|
||||
Full local/OIDC logout calls `POST /auth/logout`, revokes the server session and
|
||||
clears its cookie. It does not call the identity provider's global logout. If the
|
||||
provider still has an SSO session, the next OIDC login can complete without
|
||||
another password prompt. Embedded has no ThothII logout control: use the portal.
|
||||
|
||||
## Access revalidation
|
||||
|
||||
The frontend treats `/me` as the access authority. In embedded it does not fetch
|
||||
`/auth/config` or offer local/OIDC login. On focus, pageshow, visibility return
|
||||
and event-stream reconnection it rechecks access. A 401/403 from this probe clears
|
||||
protected state; a 403 on one operation is not automatically an app-wide logout.
|
||||
Neither the DOM adapter nor the proxy's initial SSE check guarantees instantaneous
|
||||
revocation of streams already open in other tabs.
|
||||
|
||||
See the [local guide](../install/authentication-local.md), [generic OIDC guide](../install/authentication-oidc.md),
|
||||
and [Authentik guide](../install/authentik.md) for operator procedures.
|
||||
[Authentik guide](../install/authentik.md), [upstream integration](../install/authentication-upstream.md),
|
||||
and [manual acceptance matrix](../testing/authentication-manual-acceptance.md).
|
||||
|
||||
@@ -7,7 +7,9 @@ This page complements the [architecture overview](overview.md) with the module s
|
||||
The frontend communicates with the backend through REST and SSE. The backend does not own session
|
||||
persistence: it starts Pi, invokes the `tht` CLI, and forwards events. It does own the separate
|
||||
installation-local database catalog. The harness contains the workflow, the Python CLI, and
|
||||
adapters for the DWH and vector store.
|
||||
adapters for the DWH and vector store. Its Memory module also owns the authoritative
|
||||
PostgreSQL archive of cards, links, dependencies and pending Qdrant projections.
|
||||
Administrative API calls use the same harness service as workflow producers and recall.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -20,6 +22,7 @@ flowchart LR
|
||||
THT --> FS["Sessions and artifacts\nworkspace repository"]
|
||||
THT --> DWH["DWH\nread-only"]
|
||||
THT --> VDB["Qdrant / vector store"]
|
||||
THT --> MEM["thoth_memory\nPostgreSQL Memory archive"]
|
||||
BE --> CFG["settings.json\nworkspace + thinking"]
|
||||
BE --> MODELS["generated runtime catalog\nfrom installation YAML"]
|
||||
BE --> CAT["catalog-db\nPostgreSQL + Kysely"]
|
||||
@@ -41,6 +44,14 @@ Dipendenze principali:
|
||||
|
||||
## Session sequence
|
||||
|
||||
The shared frontend is wrapped by `ShellProvider` (full preferences or a
|
||||
replaceable portal presentation adapter), then `AuthGate` (backend identity),
|
||||
then `AppShell`. Omics-specific DOM details belong only to `OmicsPortalAdapter`;
|
||||
credentials and principal validation belong to the server, never that adapter.
|
||||
Full/embedded do not duplicate the session workflow below. See
|
||||
[rendering architecture](application-shell.md) and
|
||||
[upstream identity](../install/authentication-upstream.md) for both boundaries.
|
||||
|
||||
The main path starts with a user question and ends with an SSE event. Reviewer decisions use the same channel and are persisted by the harness.
|
||||
|
||||
```mermaid
|
||||
|
||||
@@ -4,8 +4,11 @@
|
||||
|
||||
ThothII is a **human-in-the-loop datamart builder**. It turns a natural-language question into validated SQL, and optionally a dbt datamart, through a **deterministic eight-phase NL-to-SQL workflow** in which the model *proposes* and a human reviewer *decides* at gates.
|
||||
|
||||
Production authentication uses local authentication or generic OIDC. `tht` is the only operator CLI.
|
||||
For sessions, roles, groups, diagnostics, and recovery, see the [authentication documentation](authentication.md).
|
||||
Production authentication uses local authentication, generic OIDC, or the
|
||||
trusted-proxy upstream path used by Omics. `tht` is the operator CLI for the
|
||||
installation and local/OIDC configuration. For roles and recovery, see
|
||||
[authentication](authentication.md). One React build supports full and embedded;
|
||||
[rendering architecture](application-shell.md) separates presentation from identity.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -78,20 +81,20 @@ The model proposes; a human reviewer decides at gates through widgets:
|
||||
|
||||
The frontend renders these widget descriptors (registry in `src/widgets/`). It rebuilds the live transcript in memory from the SSE stream (`src/store/sessionStore.ts`); it is **not persisted**.
|
||||
|
||||
## Curated and immutable Evidence
|
||||
## Evidence sources, local authority and search projections
|
||||
|
||||
The workspace repository is the publication boundary. The curator prepares `evidence/source/`,
|
||||
reviews units in `evidence/curated/`, validates them, and merges them. With `evidence.schema_version: 2`,
|
||||
the runtime materializes the full `evidence/` tree from the exact Git commit, but the renderer passes
|
||||
only `curated/**/*.md` from the immutable revision root to preprocessing. Sources, manifests, and
|
||||
evaluation data remain available for traceability. The runtime never modifies, stages, commits, or
|
||||
publishes the authoring repository.
|
||||
The workspace repository publishes revision-pinned workspace identities and Evidence sources.
|
||||
An initialized editable Evidence archive is maintained locally through external editors and
|
||||
explicit consolidation; normal preprocessing or source refresh must not overwrite its manual
|
||||
corrections. File save, active search generation and a later human Git commit/push are separate
|
||||
outcomes. See [the current Evidence contract](../contracts/curated-evidence-v4.md) and
|
||||
[source/publication boundaries](../contracts/workspace-evidence-v3.md).
|
||||
|
||||
Before indexing, the curated corpus from the pinned revision is validated. Each workspace has two
|
||||
physical Qdrant collections with different lifecycles. `reference` contains Schema, relationships,
|
||||
and Evidence and may be replaced or cleared by preprocessing; `memory` contains `memory` and
|
||||
`solved_question` records and is never preprocessing output. Only the `reference` collection has the
|
||||
sparse `bm25` vector with `idf`, and only the Evidence stage writes sparse values.
|
||||
Each workspace has separate Qdrant collections. `reference` contains Schema, relationships and
|
||||
Evidence and may be replaced or cleared by preprocessing. Memory uses its own dense/BM25
|
||||
projection, rebuilt from authoritative PostgreSQL cards rather than old vector payloads or
|
||||
session artifacts. Both collections can contain sparse vectors; their ownership and cleanup
|
||||
lifecycles remain separate. See [Memory](../gestione-memory.md).
|
||||
|
||||
The Administration control can clear the replaceable reference collection, LSH, corpus, and derived
|
||||
checkpoints. The operation preserves the memory collection and makes preprocessing required before
|
||||
@@ -101,7 +104,9 @@ the core can admit a new session.
|
||||
|
||||
- `tht -c`/`--config` is a **per-command** option. It must follow the subcommand, never precede it (`ThtRunner.buildArgv` enforces this).
|
||||
- `--json` output must be plain JSON on stdout. It is a machine-readable contract.
|
||||
- UI strings are in English. Document *content* stays in the workspace language because it is the actual data; only chrome and labels are in English.
|
||||
- UI strings support English and Italian, with English fallback. Session interaction language
|
||||
is pinned at creation; document content remains in the workspace language. See
|
||||
[shell and localization](../operations/shell-and-localization.md).
|
||||
- Each workspace defines identity and optional Evidence only. The PostgreSQL Metadata Catalog defines
|
||||
its DWH target and binding; secrets remain in the protected workspace secret store.
|
||||
- Settings are global (`backend/data/settings.json`: workspace/thinking); provider/model choices are
|
||||
@@ -110,6 +115,13 @@ the core can admit a new session.
|
||||
|
||||
## Runtime composition
|
||||
|
||||
PostgreSQL is the current database-metadata authority for all core consumers, not a deferred
|
||||
catalog-to-core integration. The historical research on a separate catalog service and
|
||||
`annotations.yaml` publication is superseded by ADRs 0004 and 0016. Preserve read-only DWH access,
|
||||
installation-local bindings/secrets, revision-pinned workspace identity, and fail-closed readiness
|
||||
when changing those boundaries. The exact snapshot interface is the
|
||||
[Catalog Schema Snapshot contract](../contracts/catalog-schema-snapshot.md).
|
||||
|
||||
The local stack is started by `./scripts/run-stack.sh` after the installation descriptor and
|
||||
`deploy/env/local.env` exist. `core` includes Pi. `catalog-db`, Qdrant, and the Ollama embedding
|
||||
service are internal Compose services; only the DWH and model-provider endpoint remain external.
|
||||
|
||||
|
Before Width: | Height: | Size: 132 KiB |
|
Before Width: | Height: | Size: 131 KiB |
|
Before Width: | Height: | Size: 189 KiB |
|
Before Width: | Height: | Size: 186 KiB |
@@ -1,32 +0,0 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>Archify automated browser evidence · thothii-core-sequence.html</title>
|
||||
<style>
|
||||
*{box-sizing:border-box}body{margin:0;padding:24px;background:#e9eef5;color:#172033;font:14px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}header{max-width:1500px;margin:0 auto 18px}h1{margin:0 0 6px;font-size:20px}p{margin:0;color:#526176}.grid{max-width:1500px;margin:auto;display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:18px}figure{margin:0;padding:10px;background:white;border:1px solid #c9d4e3;border-radius:12px;box-shadow:0 10px 30px rgba(15,23,42,.08)}img{display:block;width:100%;height:auto;border:1px solid #e2e8f0}figcaption{padding:9px 4px 2px;color:#526176}@media(max-width:900px){.grid{grid-template-columns:1fr}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header><h1>Automated browser evidence</h1><p>thothii-core-sequence.html · visual-check containment pass · perceptual visual review pending</p></header>
|
||||
<main class="grid">
|
||||
<figure>
|
||||
<img src="thothii-core-sequence.visual-check.1440x900.light.png" alt="light 1440 by 900">
|
||||
<figcaption><strong>LIGHT</strong> · 1440×900 · containment pass</figcaption>
|
||||
</figure>
|
||||
<figure>
|
||||
<img src="thothii-core-sequence.visual-check.1440x900.dark.png" alt="dark 1440 by 900">
|
||||
<figcaption><strong>DARK</strong> · 1440×900 · containment pass</figcaption>
|
||||
</figure>
|
||||
<figure>
|
||||
<img src="thothii-core-sequence.visual-check.2048x1320.light.png" alt="light 2048 by 1320">
|
||||
<figcaption><strong>LIGHT</strong> · 2048×1320 · containment pass</figcaption>
|
||||
</figure>
|
||||
<figure>
|
||||
<img src="thothii-core-sequence.visual-check.2048x1320.dark.png" alt="dark 2048 by 1320">
|
||||
<figcaption><strong>DARK</strong> · 2048×1320 · containment pass</figcaption>
|
||||
</figure>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,548 +0,0 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"ok": true,
|
||||
"command": "visual-check",
|
||||
"evidenceKind": "automated-browser",
|
||||
"status": "pass",
|
||||
"visualReview": "pending",
|
||||
"artifact": {
|
||||
"path": "/Users/mp/projects/ThothII/docs/architecture/thothii-core-sequence.html",
|
||||
"sha256": "b521eb942f7889cfc3a5e29010546ba59eeab1cf485c22c533d271ccef9a28d5",
|
||||
"bytes": 716881
|
||||
},
|
||||
"state": {
|
||||
"detail": "read",
|
||||
"motion": "still"
|
||||
},
|
||||
"chrome": {
|
||||
"status": "available",
|
||||
"executable": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
|
||||
},
|
||||
"diagnostics": [],
|
||||
"containment": {
|
||||
"status": "pass",
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1194,
|
||||
"diagramWidth": 1164,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1600,
|
||||
"height": 1000,
|
||||
"theme": "light",
|
||||
"innerWidth": 1600,
|
||||
"innerHeight": 1000,
|
||||
"scrollWidth": 1600,
|
||||
"scrollHeight": 1000,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1242,
|
||||
"diagramWidth": 1212,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"theme": "light",
|
||||
"innerWidth": 1920,
|
||||
"innerHeight": 1080,
|
||||
"scrollWidth": 1920,
|
||||
"scrollHeight": 1080,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1402,
|
||||
"diagramWidth": 1372,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1818,
|
||||
"diagramWidth": 1768,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
}
|
||||
]
|
||||
},
|
||||
"readability": {
|
||||
"status": "pass",
|
||||
"minimumProjectedNodeTextPx": 6,
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1194,
|
||||
"diagramWidth": 1164,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1600,
|
||||
"height": 1000,
|
||||
"theme": "light",
|
||||
"innerWidth": 1600,
|
||||
"innerHeight": 1000,
|
||||
"scrollWidth": 1600,
|
||||
"scrollHeight": 1000,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1242,
|
||||
"diagramWidth": 1212,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"theme": "light",
|
||||
"innerWidth": 1920,
|
||||
"innerHeight": 1080,
|
||||
"scrollWidth": 1920,
|
||||
"scrollHeight": 1080,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1402,
|
||||
"diagramWidth": 1372,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1818,
|
||||
"diagramWidth": 1768,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
}
|
||||
]
|
||||
},
|
||||
"viewerChrome": {
|
||||
"status": "pass",
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1194,
|
||||
"diagramWidth": 1164,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1600,
|
||||
"height": 1000,
|
||||
"theme": "light",
|
||||
"innerWidth": 1600,
|
||||
"innerHeight": 1000,
|
||||
"scrollWidth": 1600,
|
||||
"scrollHeight": 1000,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1242,
|
||||
"diagramWidth": 1212,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"theme": "light",
|
||||
"innerWidth": 1920,
|
||||
"innerHeight": 1080,
|
||||
"scrollWidth": 1920,
|
||||
"scrollHeight": 1080,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1402,
|
||||
"diagramWidth": 1372,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1818,
|
||||
"diagramWidth": 1768,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
}
|
||||
]
|
||||
},
|
||||
"captures": {
|
||||
"status": "pass",
|
||||
"screenshots": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1194,
|
||||
"diagramWidth": 1164,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light",
|
||||
"file": "thothii-core-sequence.visual-check.1440x900.light.png"
|
||||
},
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "dark",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1194,
|
||||
"diagramWidth": 1164,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "dark",
|
||||
"file": "thothii-core-sequence.visual-check.1440x900.dark.png"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1818,
|
||||
"diagramWidth": 1768,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light",
|
||||
"file": "thothii-core-sequence.visual-check.2048x1320.light.png"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "dark",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1818,
|
||||
"diagramWidth": 1768,
|
||||
"viewBoxWidth": 1080,
|
||||
"minimumProjectedNodeTextPx": 7,
|
||||
"minimumProjectedNodeText": "browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "dark",
|
||||
"file": "thothii-core-sequence.visual-check.2048x1320.dark.png"
|
||||
}
|
||||
],
|
||||
"contactSheet": "thothii-core-sequence.visual-check.html"
|
||||
},
|
||||
"sidecars": {
|
||||
"receipt": "thothii-core-sequence.visual-check.json",
|
||||
"contactSheet": "thothii-core-sequence.visual-check.html"
|
||||
}
|
||||
}
|
||||
|
Before Width: | Height: | Size: 137 KiB |
|
Before Width: | Height: | Size: 136 KiB |
|
Before Width: | Height: | Size: 183 KiB |
|
Before Width: | Height: | Size: 182 KiB |
@@ -1,32 +0,0 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>Archify automated browser evidence · thothii-runtime.html</title>
|
||||
<style>
|
||||
*{box-sizing:border-box}body{margin:0;padding:24px;background:#e9eef5;color:#172033;font:14px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}header{max-width:1500px;margin:0 auto 18px}h1{margin:0 0 6px;font-size:20px}p{margin:0;color:#526176}.grid{max-width:1500px;margin:auto;display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:18px}figure{margin:0;padding:10px;background:white;border:1px solid #c9d4e3;border-radius:12px;box-shadow:0 10px 30px rgba(15,23,42,.08)}img{display:block;width:100%;height:auto;border:1px solid #e2e8f0}figcaption{padding:9px 4px 2px;color:#526176}@media(max-width:900px){.grid{grid-template-columns:1fr}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header><h1>Automated browser evidence</h1><p>thothii-runtime.html · visual-check containment pass · perceptual visual review pending</p></header>
|
||||
<main class="grid">
|
||||
<figure>
|
||||
<img src="thothii-runtime.visual-check.1440x900.light.png" alt="light 1440 by 900">
|
||||
<figcaption><strong>LIGHT</strong> · 1440×900 · containment pass</figcaption>
|
||||
</figure>
|
||||
<figure>
|
||||
<img src="thothii-runtime.visual-check.1440x900.dark.png" alt="dark 1440 by 900">
|
||||
<figcaption><strong>DARK</strong> · 1440×900 · containment pass</figcaption>
|
||||
</figure>
|
||||
<figure>
|
||||
<img src="thothii-runtime.visual-check.2048x1320.light.png" alt="light 2048 by 1320">
|
||||
<figcaption><strong>LIGHT</strong> · 2048×1320 · containment pass</figcaption>
|
||||
</figure>
|
||||
<figure>
|
||||
<img src="thothii-runtime.visual-check.2048x1320.dark.png" alt="dark 2048 by 1320">
|
||||
<figcaption><strong>DARK</strong> · 2048×1320 · containment pass</figcaption>
|
||||
</figure>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,548 +0,0 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"ok": true,
|
||||
"command": "visual-check",
|
||||
"evidenceKind": "automated-browser",
|
||||
"status": "pass",
|
||||
"visualReview": "pending",
|
||||
"artifact": {
|
||||
"path": "/Users/mp/projects/ThothII/docs/architecture/thothii-runtime.html",
|
||||
"sha256": "ad19195852c7c643228663e5bf7f3cb273afb0456fedbe295d4df24bc345b6c7",
|
||||
"bytes": 724277
|
||||
},
|
||||
"state": {
|
||||
"detail": "read",
|
||||
"motion": "still"
|
||||
},
|
||||
"chrome": {
|
||||
"status": "available",
|
||||
"executable": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
|
||||
},
|
||||
"diagnostics": [],
|
||||
"containment": {
|
||||
"status": "pass",
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 968,
|
||||
"diagramWidth": 938,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.20735294117647,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1600,
|
||||
"height": 1000,
|
||||
"theme": "light",
|
||||
"innerWidth": 1600,
|
||||
"innerHeight": 1000,
|
||||
"scrollWidth": 1600,
|
||||
"scrollHeight": 1000,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1009,
|
||||
"diagramWidth": 979,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.478676470588235,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"theme": "light",
|
||||
"innerWidth": 1920,
|
||||
"innerHeight": 1080,
|
||||
"scrollWidth": 1920,
|
||||
"scrollHeight": 1080,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1172,
|
||||
"diagramWidth": 1142,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 7.557352941176471,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1583,
|
||||
"diagramWidth": 1533,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 9,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
}
|
||||
]
|
||||
},
|
||||
"readability": {
|
||||
"status": "pass",
|
||||
"minimumProjectedNodeTextPx": 6,
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 968,
|
||||
"diagramWidth": 938,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.20735294117647,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1600,
|
||||
"height": 1000,
|
||||
"theme": "light",
|
||||
"innerWidth": 1600,
|
||||
"innerHeight": 1000,
|
||||
"scrollWidth": 1600,
|
||||
"scrollHeight": 1000,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1009,
|
||||
"diagramWidth": 979,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.478676470588235,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"theme": "light",
|
||||
"innerWidth": 1920,
|
||||
"innerHeight": 1080,
|
||||
"scrollWidth": 1920,
|
||||
"scrollHeight": 1080,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1172,
|
||||
"diagramWidth": 1142,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 7.557352941176471,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1583,
|
||||
"diagramWidth": 1533,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 9,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
}
|
||||
]
|
||||
},
|
||||
"viewerChrome": {
|
||||
"status": "pass",
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 968,
|
||||
"diagramWidth": 938,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.20735294117647,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1600,
|
||||
"height": 1000,
|
||||
"theme": "light",
|
||||
"innerWidth": 1600,
|
||||
"innerHeight": 1000,
|
||||
"scrollWidth": 1600,
|
||||
"scrollHeight": 1000,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1009,
|
||||
"diagramWidth": 979,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.478676470588235,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"theme": "light",
|
||||
"innerWidth": 1920,
|
||||
"innerHeight": 1080,
|
||||
"scrollWidth": 1920,
|
||||
"scrollHeight": 1080,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1172,
|
||||
"diagramWidth": 1142,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 7.557352941176471,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1583,
|
||||
"diagramWidth": 1533,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 9,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light"
|
||||
}
|
||||
]
|
||||
},
|
||||
"captures": {
|
||||
"status": "pass",
|
||||
"screenshots": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "light",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 968,
|
||||
"diagramWidth": 938,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.20735294117647,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light",
|
||||
"file": "thothii-runtime.visual-check.1440x900.light.png"
|
||||
},
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900,
|
||||
"theme": "dark",
|
||||
"innerWidth": 1440,
|
||||
"innerHeight": 900,
|
||||
"scrollWidth": 1440,
|
||||
"scrollHeight": 900,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 968,
|
||||
"diagramWidth": 938,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 6.20735294117647,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 51,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "dark",
|
||||
"file": "thothii-runtime.visual-check.1440x900.dark.png"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "light",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1583,
|
||||
"diagramWidth": 1533,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 9,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "light",
|
||||
"file": "thothii-runtime.visual-check.2048x1320.light.png"
|
||||
},
|
||||
{
|
||||
"width": 2048,
|
||||
"height": 1320,
|
||||
"theme": "dark",
|
||||
"innerWidth": 2048,
|
||||
"innerHeight": 1320,
|
||||
"scrollWidth": 2048,
|
||||
"scrollHeight": 1320,
|
||||
"overflowX": false,
|
||||
"overflowY": false,
|
||||
"ok": true,
|
||||
"readerWidth": 1583,
|
||||
"diagramWidth": 1533,
|
||||
"viewBoxWidth": 1360,
|
||||
"minimumProjectedNodeTextPx": 9,
|
||||
"minimumProjectedNodeText": "Browser",
|
||||
"minimumProjectedNodeTextDetail": "context",
|
||||
"minimumRequiredNodeTextPx": 6,
|
||||
"readabilityOk": true,
|
||||
"hasLegend": true,
|
||||
"hasNavigationDock": true,
|
||||
"legendDockIntersectionArea": 0,
|
||||
"dockStageIntersectionArea": 0,
|
||||
"dockStageGap": 10.21875,
|
||||
"requiredDockStageGap": 10,
|
||||
"viewerChromeStageOk": true,
|
||||
"viewerChromeReserve": 41,
|
||||
"viewerChromeActive": true,
|
||||
"viewerChromeOk": true,
|
||||
"resolvedTheme": "dark",
|
||||
"file": "thothii-runtime.visual-check.2048x1320.dark.png"
|
||||
}
|
||||
],
|
||||
"contactSheet": "thothii-runtime.visual-check.html"
|
||||
},
|
||||
"sidecars": {
|
||||
"receipt": "thothii-runtime.visual-check.json",
|
||||
"contactSheet": "thothii-runtime.visual-check.html"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
# Session corrections to Memory and Evidence
|
||||
|
||||
`reviewer_archive_repair` presents one to five closed alternatives for a conflict.
|
||||
Each alternative updates one existing Memory Card or one existing local Evidence unit,
|
||||
showing complete current and resulting content. The reviewer chooses one alternative,
|
||||
rejects all as inadequate, or asks for reformulation. A correction does not approve or
|
||||
advance the session phase; the ordinary gate still reviews its use in the current question.
|
||||
|
||||
The harness coordinator `tht/archive_repair.py` joins two independent domains. It uses
|
||||
Memory's PostgreSQL repository, workspace mutation lock and projection service, and the
|
||||
same local Evidence save/consolidation boundary as administration. Evidence activation
|
||||
runs the existing corpus pipeline without reacquiring configured external sources.
|
||||
No database binding, schema configuration, other archive, or Git repository is changed.
|
||||
|
||||
## Authority and recovery
|
||||
|
||||
Migration `004_archive_repairs.sql` stores session-bound proposals and receipts in
|
||||
`thoth_memory.archive_repairs`, isolated by workspace RLS. Each receipt captures the
|
||||
session decision context, current target revision, complete resulting content, selected
|
||||
choice, acting principal and publication outcome. The preparation command changes no
|
||||
archive content. Application accepts only an option ID from the persisted proposal.
|
||||
|
||||
Memory writes its new revision and receipt in one SQL transaction. Projection failures
|
||||
leave a durable pending operation; retry propagates the saved revision. A later card
|
||||
edit invalidates replay of the correction.
|
||||
|
||||
Evidence records the exact choice before writing its canonical file. Recovery accepts
|
||||
either the reviewed original revision or the already-written approved result; it never
|
||||
overwrites a different intervening correction. Before a first proposal, the editable
|
||||
checkout must match its active snapshot. Other curated files are fingerprinted and
|
||||
checked again before application/retry so a session decision cannot publish unrelated
|
||||
external edits. A crash after replacement is recoverable by the same receipt. Original
|
||||
document provenance is retained as the lineage of the manual correction.
|
||||
|
||||
The gate displays these outcomes:
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `proposed` | Waiting for a human choice; no content saved |
|
||||
| `rejected` | All alternatives declined; reformulation required |
|
||||
| `applying` | Choice recorded; file write or candidate recovery still required |
|
||||
| `pending_activation` | Content saved; index activation incomplete |
|
||||
| `active` | Saved correction matches the currently active revision |
|
||||
| `superseded` | The target was removed or changed after the saved correction |
|
||||
|
||||
The reviewer may retry a pending correction or continue the current question while
|
||||
leaving activation explicitly pending. The latter is not persistent-resolution success.
|
||||
`repair-show` refreshes target status; `repairs` lists the historical receipt status.
|
||||
The final Memory summary must not create a duplicate of a correction already saved here.
|
||||
|
||||
## Authorization and commands
|
||||
|
||||
Inspection/preparation requires an accessible open session in the same workspace.
|
||||
Applying either archive correction requires an administrator. Browser responses are
|
||||
also checked against the responding principal's `memory.manage` or `evidence.manage`
|
||||
permission and the Pi runtime owner. An administrator using another principal's runtime
|
||||
must resume it under their own account first, preserving truthful receipt attribution.
|
||||
Non-administrators may reject proposals or continue question review without changing
|
||||
the archives. The Pi shell guard blocks `repair-apply`; only the human gate invokes it.
|
||||
|
||||
The Python workflow CLI provides `memory repair-target`, `repair-prepare`, `repair-show`,
|
||||
`repair-apply`, and `repairs`. These are workflow integration commands, not new native
|
||||
installation commands. All accept `--session` and command-local `-c`; JSON output remains
|
||||
machine-readable. Proposal bodies are bounded to 1 MB. Durable receipts support both
|
||||
filesystem and server session storage without a persisted chat transcript.
|
||||
@@ -0,0 +1,248 @@
|
||||
# Editable Curated Evidence v4
|
||||
|
||||
Curated unit v4 makes the visible Markdown body authoritative. It uses the existing
|
||||
typed Evidence payloads and stable identifiers. Workspace descriptor v4 and Evidence
|
||||
descriptor v1/v2 are separate version numbers.
|
||||
|
||||
E1 implements the format, explicit conversion, local archive and consolidation API.
|
||||
E2 connects that API to the installed consolidation command, Evidence management
|
||||
page and runtime source selection. The local PSD preview now uses 35 converted units.
|
||||
|
||||
## Write or edit a file
|
||||
|
||||
Place units in `<workspace-root>/evidence/curated/<kind>/<name>.md`. Keep the existing
|
||||
`id` when editing. The directory must match `kind`. A new manual unit needs no external
|
||||
document, hash or encoded metadata:
|
||||
|
||||
```markdown
|
||||
---
|
||||
schema_version: 4
|
||||
id: evidence:order-key
|
||||
kind: domain
|
||||
language: en
|
||||
purposes: [sql_generation]
|
||||
applies_to:
|
||||
tables: [sales.orders]
|
||||
---
|
||||
|
||||
# Order key
|
||||
|
||||
## Rule
|
||||
|
||||
Join orders using the order number, financial year and company.
|
||||
```
|
||||
|
||||
Required metadata is `schema_version`, `id`, `kind`, `language`, and a nonempty
|
||||
`purposes` list. Purposes are `disambiguation`, `rewriting`, `schema_linking`, and
|
||||
`sql_generation`. Optional `applies_to` contains `concepts`, `tables`, and `columns`.
|
||||
Tables use `schema.table`; columns use `schema.table.column`.
|
||||
|
||||
The first H1 is the title. H2 headings identify the payload fields below. Heading
|
||||
spelling follows `language`: Italian for `it` and its regional variants, English
|
||||
otherwise. Keep structural headings when editing the text below them.
|
||||
|
||||
| Kind | English field headings | Italian field headings |
|
||||
| --- | --- | --- |
|
||||
| `domain` | Rule | Regola |
|
||||
| `glossary` | Definition, Synonyms, Variants | Definizione, Sinonimi, Varianti |
|
||||
| `enum` | Column, Values | Colonna, Valori |
|
||||
| `example` | Question, Interpretation | Domanda, Interpretazione |
|
||||
| `mapping` | Concept, Tables, Columns | Concetto, Tabelle, Colonne |
|
||||
| `normalization` | Input, Output, Rule | Input, Output, Regola |
|
||||
| `formula` | Concept, Columns, SQL | Concetto, Colonne, SQL |
|
||||
| `reference` | URL, Label, Description | URL, Etichetta, Descrizione |
|
||||
|
||||
List fields use one `- value` per line. Empty optional lists may be omitted. Quoted
|
||||
JSON strings within bullets preserve unusual or multiline values during conversion.
|
||||
Enum values use `### "stored value"`, followed by their meaning; `### ""` represents
|
||||
an empty stored value. Formula SQL uses a fenced `sql` block containing one PostgreSQL
|
||||
expression. Whole queries and mutation statements remain invalid.
|
||||
|
||||
Nested prose headings and fenced examples are supported inside text fields. An H2
|
||||
matching a field heading is structural outside a code fence. Duplicate fields, missing
|
||||
required fields, duplicate metadata keys and malformed payloads are rejected. There
|
||||
is no second title or payload in frontmatter and no hidden authoritative rule text.
|
||||
Conversion fails explicitly if a legacy payload cannot be represented losslessly.
|
||||
|
||||
## Current provenance and original source
|
||||
|
||||
The host supplies document provenance when refining source material: relative source
|
||||
path, normalized source hash and exact supporting excerpts. A manual unit can omit
|
||||
`provenance`. Consolidation records `kind: manual` and the supplied curator identity.
|
||||
|
||||
A visible change to a previously recorded unit becomes a manual declaration. If that
|
||||
unit originated from a document, its former document provenance is retained under
|
||||
`original`. The original excerpts establish lineage; they do not assert that the
|
||||
source contains the new wording. Retrieval carries this distinction through typed
|
||||
metadata. Curators edit content; the service updates managed provenance.
|
||||
|
||||
Unchanged document declarations still require matching source bytes and excerpts.
|
||||
Replacing a source requires an explicit refresh through the E3 source-review path. Ordinary preparation
|
||||
is blocked on an initialized local archive, preventing regenerated source material
|
||||
from overwriting corrections or restoring deletions. Legacy `resolve` is likewise
|
||||
blocked there; local file corrections and the archive API own those changes.
|
||||
|
||||
## Persistent archive and activation
|
||||
|
||||
```text
|
||||
<workspace-root>/
|
||||
.evidence-archive.lock
|
||||
evidence/
|
||||
source/ # acquired original documents
|
||||
curated/<kind>/*.md # editable primary content
|
||||
local-manifest.yaml # derived declarations and deletion records
|
||||
.local/
|
||||
state.yaml # baseline, pending and active revisions
|
||||
snapshots/<revision>/ # immutable units, source bytes and manifest
|
||||
```
|
||||
|
||||
Keep the complete Evidence tree and its managed metadata in backups and the operator's
|
||||
Git review. Historical source bytes and deletion records are needed to preserve manual
|
||||
care across subsequent imports. The separate corpus cache and Qdrant index are derived.
|
||||
The lock file only coordinates local service operations.
|
||||
|
||||
`LocalEvidenceArchive.initialize()` records the pre-edit baseline without activation.
|
||||
`consolidate(actor=..., activate=...)` validates files, derives provenance, records
|
||||
deletions and source suppression, and creates an immutable candidate. The activation
|
||||
callback receives that snapshot and must raise if indexing is blocked or fails. Only
|
||||
successful activation advances `active_snapshot()`. Without a callback the result is
|
||||
explicitly `pending_activation`; saving a file alone never changes this pointer.
|
||||
|
||||
The same candidate can be retried after an index failure. Interrupted managed writes
|
||||
are replayed only when the operator's file bytes have not changed. A missing curated
|
||||
directory is an availability error, not proof that all units were deleted. Removing
|
||||
unit files from an accessible directory records deletion without deleting their sources.
|
||||
The API also provides `get`, revision-checked `save`, and revision-checked `remove` for
|
||||
future deliberate workflow corrections. Concurrent external edits produce conflicts.
|
||||
|
||||
These boundaries are exercised with the existing corpus pipeline and real Qdrant.
|
||||
An initialized installation reads only its active local snapshot, including during
|
||||
ordinary preprocessing. Unconsolidated edits are visible in Administration but do
|
||||
not enter retrieval. Before initialization, the pinned repository source still works.
|
||||
|
||||
## Administration and installed command
|
||||
|
||||
Open **Administration → Evidence management**. This independent page requires the
|
||||
`evidence.manage` permission and no active session. It shows complete units, source
|
||||
lineage, review items, file errors, filters, and changes relative to the active snapshot.
|
||||
Edit the displayed Markdown path using an external editor. Refresh files to inspect
|
||||
the result, then run the command shown by the page:
|
||||
|
||||
```sh
|
||||
tht --installation /absolute/path/thothii-installation.yaml \
|
||||
workspace evidence consolidate --workspace psd-clinical
|
||||
```
|
||||
|
||||
The installed command accepts `--json`. It validates and activates Evidence using
|
||||
the existing corpus pipeline and embedding service. It does not scan the DWH, run
|
||||
full workspace preprocessing, change Catalog/Schema readiness, or execute Git.
|
||||
If indexing fails after saving, the archive retains the candidate for retry and the
|
||||
previous active revision remains selected. Validation errors identify corrections
|
||||
to make in the files. Structural and review checks still apply; initialized archives
|
||||
do not require the legacy repository's fixed retrieval-evaluation fixture, whose
|
||||
expected IDs would otherwise prevent deliberate local deletions.
|
||||
|
||||
The canonical workspace root is `<workspace-registry-root>/repo/<workspace-id>`.
|
||||
Snapshots and the derived corpus are separate. For a registry in a Docker volume,
|
||||
copy its existing `repo` to a persistent host directory before enabling
|
||||
`deploy/compose.evidence-host.yaml`; set `THT_EVIDENCE_HOST_REGISTRY_ROOT` to that
|
||||
directory and include the override in the installation descriptor. Core and
|
||||
workspace-maintenance must mount the same checkout. Keep registry state/snapshots
|
||||
on their existing volume. The page only presents a host path when configured; it
|
||||
does not label an internal container path as a usable editor path.
|
||||
|
||||
After successful consolidation, inspect and commit the complete workspace Evidence
|
||||
tree, including managed manifests, snapshots, and deletion records, then push manually.
|
||||
The page provides quoted POSIX-shell examples for status, diff, add, commit, and push.
|
||||
Do not commit only the edited Markdown. Ordinary Git operations remain the operator's
|
||||
responsibility. E3 source decisions use this same activation boundary.
|
||||
|
||||
**Clear** removes derived Reference/corpus data while retaining editable files,
|
||||
snapshots and Memory. It still requires full workspace preprocessing to recreate
|
||||
Reference/Schema readiness; Evidence consolidation does not satisfy that gate.
|
||||
|
||||
## Import drafts and refresh sources
|
||||
|
||||
Place externally authored Markdown drafts in `<workspace-root>/evidence/incoming/`.
|
||||
The specialist needs no installation account or database access to write a draft.
|
||||
Copying the draft to the installation and choosing **Import or refresh sources** in
|
||||
Evidence management explicitly starts acquisition and refinement. The equivalent
|
||||
installed command is:
|
||||
|
||||
```sh
|
||||
tht --installation /absolute/path/thothii-installation.yaml \
|
||||
workspace evidence refresh --workspace psd-clinical
|
||||
```
|
||||
|
||||
This reads local `incoming/**/*.md` and original `source/**/*.md` files, excluding
|
||||
managed `source/acquired/` versions. It also reads configured HTTP and S3 sources
|
||||
through the existing read-only adapters and network policies. After local archive
|
||||
initialization, filesystem descriptors use this local authoring tree; ordinary
|
||||
runtime/preprocessing never fetches remote source changes. No source-server write
|
||||
credential is needed. The existing Pi authoring refiner runs once for each changed
|
||||
document, without session state or tools. Unchanged source hashes skip refinement.
|
||||
|
||||
All acquisitions and proposals must succeed before the new comparison set is
|
||||
recorded. An access or refinement failure preserves previous comparisons and active
|
||||
Evidence. A source absent from a successful discovery is marked missing and never
|
||||
treated as permission to delete units. Restore an accidentally missing local original
|
||||
file before consolidating its document-derived units, or explicitly retire those units.
|
||||
|
||||
Each changed source has a durable comparison showing current local units, complete
|
||||
proposed content, supporting excerpts, and IDs that replacement would retire:
|
||||
|
||||
- **Keep local Evidence** retains the current wording as a manual declaration, with
|
||||
its original documentary lineage preserved. The changed source is acknowledged;
|
||||
the next unchanged refresh does not reopen that decision.
|
||||
- **Use proposed Evidence** adopts the displayed proposal and explicitly retires the
|
||||
displayed omitted IDs. The source version and provenance change together.
|
||||
|
||||
Both choices save and activate through the same local consolidation/index pipeline.
|
||||
Review items block adoption; correct the original draft and refresh, or keep local
|
||||
content. There is no automatic merge based on a model's semantic conflict assessment.
|
||||
Any change to an affected curated file invalidates the comparison and requires a new
|
||||
refresh. If indexing fails after the decision is saved, use **Retry saved decision**;
|
||||
this reuses acquired content without fetching sources again. An intervening external
|
||||
edit is never silently overwritten by recovery.
|
||||
|
||||
Headless operators can make the same decision using the source ID and comparison
|
||||
revision from the local source registry or administration response:
|
||||
|
||||
```sh
|
||||
tht --installation /absolute/path/thothii-installation.yaml \
|
||||
workspace evidence decide --workspace psd-clinical \
|
||||
--source-id <64-hex-source-id> --revision <64-hex-comparison-revision> \
|
||||
--decision keep
|
||||
```
|
||||
|
||||
Use `--decision replace` to adopt the proposal. All installed commands accept `--json`.
|
||||
The Python `evidence sources` worker is internal to this installed command/API surface.
|
||||
|
||||
`evidence/.local/sources.json` stores source identities, comparisons and retry journals.
|
||||
`evidence/.local/acquisitions/` preserves original acquired bytes and credential-free
|
||||
remote provenance. Adopted normalized documents live under
|
||||
`evidence/source/acquired/<source-id>/<content-hash>.md`. Versioned paths let a new
|
||||
document and an older manual declaration's original source coexist. Include all of
|
||||
these files in the existing manual Git/backup sequence. Do not edit managed acquired
|
||||
versions: edit the original local draft or refresh its remote origin.
|
||||
|
||||
Deleted IDs remain reserved. Once a source has had a curated deletion, fresh model
|
||||
IDs from that source are conservatively suppressed as well: changing an ID must not
|
||||
restore retired knowledge. Existing surviving IDs can still receive reviewed updates;
|
||||
deliberate new knowledge can be written as a manual Evidence file. Refresh is bounded
|
||||
to 200 documents and 100 MiB per operation, in addition to each adapter's limits.
|
||||
|
||||
## Convert an existing workspace
|
||||
|
||||
The existing workflow CLI command `tht evidence migrate <workspace-root>` converts
|
||||
unit versions 1–3 to 4 deterministically and initializes the archive baseline. It
|
||||
preserves IDs, typed content, provenance and review items, with no model call or Git
|
||||
commit. It does not activate a local index. Review items still block consolidation.
|
||||
The command's existing Git-worktree path check remains in effect.
|
||||
|
||||
The first installed consolidation performs this conversion automatically when the
|
||||
legacy manifest is present, then validates and activates the result. Preserve the
|
||||
existing checkout in backups before upgrading. The E1 validation used an isolated
|
||||
copy; E2 also converted and indexed all 35 units on the running local preview.
|
||||
See the [E1 validation report](../reports/knowledge-archives-release.md) and
|
||||
[E2 validation report](../reports/knowledge-archives-release.md).
|
||||
@@ -0,0 +1,135 @@
|
||||
# Portal Shell Adapter v1
|
||||
|
||||
Contratto minimo della presentazione embedded. La revisione approvata il 2026-09-13
|
||||
sostituisce il precedente trasporto a eventi personalizzati con l'osservazione del
|
||||
documento condiviso. Non trasferisce identità, token o stato di autenticazione.
|
||||
|
||||
## Configurazione
|
||||
|
||||
Sul Mac:
|
||||
|
||||
```yaml
|
||||
shell:
|
||||
mode: full
|
||||
defaultLocale: en
|
||||
```
|
||||
|
||||
Sul server Omics:
|
||||
|
||||
```yaml
|
||||
shell:
|
||||
mode: embedded
|
||||
adapter: omics-portal
|
||||
```
|
||||
|
||||
Se `shell` o `mode` sono omessi, la modalità è embedded. L'adapter embedded
|
||||
predefinito è `omics-portal`; un nome sconosciuto è un errore di configurazione.
|
||||
Questi default sono normalizzati dal CLI prima della proiezione. Nel browser,
|
||||
shell interamente omessa ha gli stessi default, ma un oggetto `shell` parziale
|
||||
senza `mode` o `defaultLocale` viene rifiutato: non scrivere proiezioni a mano.
|
||||
Full non istanzia adapter. `defaultLocale` inizializza full; in embedded il locale
|
||||
proviene dal portale. L'autenticazione si configura separatamente dalla shell.
|
||||
|
||||
## API applicativa
|
||||
|
||||
```ts
|
||||
export type PortalSnapshot = {
|
||||
locale: string;
|
||||
theme: "light" | "dark";
|
||||
fullscreen: boolean;
|
||||
};
|
||||
|
||||
export interface PortalAdapter {
|
||||
subscribe(
|
||||
onState: (state: PortalSnapshot) => void,
|
||||
onError: (error: Error) => void,
|
||||
): () => void;
|
||||
}
|
||||
```
|
||||
|
||||
Una sottoscrizione installa gli osservatori e consegna lo snapshot iniziale senza
|
||||
richiedere messaggi all'altro applicativo. Gli aggiornamenti contengono snapshot
|
||||
completi e validati. La disiscrizione elimina tutti i listener e osservatori.
|
||||
La lingua viene risolta tramite i cataloghi UI, con fallback inglese.
|
||||
|
||||
## Implementazione Omics
|
||||
|
||||
L'integrazione monta React nel documento Django, non in un iframe.
|
||||
|
||||
| Dato | Fonte privata dell'adapter | Aggiornamento |
|
||||
| --- | --- | --- |
|
||||
| Locale | `data-lang` del selettore `.omics-language-select` | nuova pagina Django dopo `set_language` |
|
||||
| Tema | `data-bs-theme` su `html` | osservazione limitata a quell'attributo |
|
||||
| Fullscreen | stato effettivo del documento | evento del browser, inclusa uscita con Esc |
|
||||
|
||||
Il template Omics aggiornato allinea anche `html lang` alla lingua Django, ma
|
||||
la fonte dell'adapter rimane `select.omics-language-select[data-lang]`. Leggere
|
||||
il valore renderizzato evita di anticipare un cambio lingua prima che il form
|
||||
abbia successo. Cambiare soltanto `select.value` o `data-lang` senza il normale
|
||||
reload non è un trasporto runtime implementato per la lingua.
|
||||
|
||||
L'assenza del contesto host atteso produce un errore di integrazione; non abilita
|
||||
controlli locali. Non si introducono eventi `ready/state`, handshake, timeout,
|
||||
versioni dei messaggi o comandi duplicati. Selettori e dettagli Omics non devono
|
||||
essere letti dai componenti applicativi.
|
||||
|
||||
## Proprietà per modalità
|
||||
|
||||
| Funzione | Full | Embedded |
|
||||
| --- | --- | --- |
|
||||
| Header | ThothII | solo Omics |
|
||||
| Lingua | selettore locale | selettore Omics, normale reload Django |
|
||||
| Tema | toggle locale light/dark | stato Omics |
|
||||
| Fullscreen | controllo locale, stato reale | controllo Omics, stato reale |
|
||||
| Login/logout | ThothII local/OIDC; upstream non offre logout locale | autenticazione Omics esistente |
|
||||
| Nome utente | header ThothII | header Omics |
|
||||
| Rotellina amministrativa | mai | eventuale comando del portale |
|
||||
|
||||
## Accesso e continuità
|
||||
|
||||
Il server Omics verifica l'accesso a Datamart Builder e il proxy trasmette i
|
||||
principal header normalizzati al backend ThothII. La UI usa `/me`; non effettua
|
||||
un secondo login. Un altro portale deve soddisfare anche questo contratto server,
|
||||
oltre a fornire una nuova implementazione dell'adapter UI.
|
||||
|
||||
Il [contratto upstream](../install/authentication-upstream.md) specifica header,
|
||||
origine, rete e configurazioni incompatibili. Lo snapshot non può contenere
|
||||
`authenticated`, utente, ruoli, cookie o token; un evento browser non autorizza
|
||||
una richiesta API. Il prefisso API viene configurato separatamente prima del
|
||||
caricamento React, non viene dedotto dall'adapter.
|
||||
|
||||
Il logout del portale segue la sua navigazione. Una perdita di accesso rilevata
|
||||
dal server chiude lo stato protetto; un 403 di una singola operazione non equivale
|
||||
automaticamente a logout. La riconnessione degli eventi e il ritorno alla pagina
|
||||
ricontrollano l'accesso. Non si garantisce revoca istantanea di una connessione
|
||||
aperta in un'altra scheda attraverso il solo controllo iniziale del proxy.
|
||||
|
||||
Il cambio lingua può ricaricare la pagina: conservare la selezione della sessione,
|
||||
proteggere le modifiche non salvate e non avviare una nuova generazione al reload.
|
||||
Non si conserva una trascrizione integrale nel browser. La lingua della sessione
|
||||
rimane quella registrata nel manifest, secondo ADR 0022.
|
||||
|
||||
## Sostituzione e verifiche
|
||||
|
||||
Un nuovo adapter può usare un diverso documento o trasporto, ma deve rispettare
|
||||
la stessa sottoscrizione e mantenere la conoscenza del portale nella propria
|
||||
implementazione. Oggi `ShellProvider` istanzia direttamente `OmicsPortalAdapter`:
|
||||
per sostituirlo aggiornare quel punto e i nomi accettati in
|
||||
`tools/tht/internal/config/shell.go` e `frontend/src/api/runtime-config.ts`.
|
||||
Non è disponibile il caricamento dinamico di classi da una stringa YAML.
|
||||
Non occorre implementare ora iframe o un secondo portale.
|
||||
|
||||
La nuova implementazione deve pubblicare uno snapshot iniziale completo, poi gli
|
||||
aggiornamenti; segnalare contesto invalido; liberare tutti i listener alla
|
||||
disiscrizione. Locale ben formato ma non tradotto significa fallback inglese;
|
||||
locale assente/malformato e tema diverso da light/dark sono errori di integrazione.
|
||||
Non cambiare componenti applicativi o workflow per aggiungere selettori specifici
|
||||
del nuovo portale.
|
||||
|
||||
Verificare snapshot prima/dopo il montaggio, tema, fullscreen con Esc, cleanup,
|
||||
contesto host mancante, assenza di header ThothII embedded, accesso singolo,
|
||||
locale dopo reload e compatibilità del prefisso API. Full deve funzionare senza
|
||||
alcun elemento Omics presente.
|
||||
|
||||
Vedere anche [architettura del rendering](../architecture/application-shell.md)
|
||||
e [matrice di accettazione](../testing/authentication-manual-acceptance.md).
|
||||
@@ -6,8 +6,9 @@ When present, `evidence` is strict: it contains `source` and a defaulted strict
|
||||
source variant and the policy reject unknown keys.
|
||||
|
||||
The version numbers are intentionally separate: the Evidence descriptor supports v1/v2, while the
|
||||
latest Curated Evidence Unit format is v3. There is no Evidence descriptor v3/v4 and no Curated
|
||||
Evidence Unit v4.
|
||||
latest Curated Evidence Unit format is [v4](curated-evidence-v4.md). There is no Evidence
|
||||
descriptor v3/v4. The editable local archive is implemented in E1; installation integration
|
||||
is the next increment, E2.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -67,9 +68,11 @@ ignore it. Domain rules also retain their exact canonical text in an invisible `
|
||||
comment while presenting long prose as paragraphs, labelled subsections, and semicolon-derived
|
||||
lists. Runtime chunking reads the parsed canonical rule, not this review-only presentation.
|
||||
|
||||
Newly prepared units use v3. `tht evidence migrate <workspace-root>` upgrades v1 and v2 units and
|
||||
canonicalizes an older v3 presentation locally without a model call, commit, publication, or
|
||||
semantic change.
|
||||
The representations above are legacy conversion inputs. Newly prepared units use editable v4:
|
||||
short YAML metadata, a visible H1 title and typed H2 payload fields, with no hidden content copy.
|
||||
`tht evidence migrate <workspace-root>` converts v1–v3 to v4 and initializes a local archive
|
||||
baseline without a model call, commit, activation, or semantic change. See the
|
||||
[v4 editing and consolidation contract](curated-evidence-v4.md).
|
||||
|
||||
### Example: filesystem
|
||||
|
||||
|
||||
@@ -14,12 +14,30 @@ tht --installation <absolute>/thothii-installation.yaml workspace preprocess run
|
||||
|
||||
tht --installation <absolute>/thothii-installation.yaml workspace preprocess clear
|
||||
--workspace <id> [--json]
|
||||
|
||||
tht --installation <absolute>/thothii-installation.yaml workspace evidence consolidate
|
||||
--workspace <id> [--json]
|
||||
|
||||
tht --installation <absolute>/thothii-installation.yaml workspace evidence refresh
|
||||
--workspace <id> [--json]
|
||||
|
||||
tht --installation <absolute>/thothii-installation.yaml workspace evidence decide
|
||||
--workspace <id> --source-id <64-hex> --revision <64-hex>
|
||||
--decision keep|replace [--json]
|
||||
```
|
||||
|
||||
There are no public partial commands for DWH introspection, LSH, FK suggestions, schema indexing,
|
||||
Evidence indexing, or Qdrant rebuild. `preprocess run` does not accept `--resume`, `--dry-run`, a
|
||||
or Qdrant rebuild. The separate Evidence curation command validates editable local files and
|
||||
activates only their index; it does not satisfy workspace preprocessing readiness or execute Git.
|
||||
See [Curated Evidence v4](curated-evidence-v4.md#administration-and-installed-command).
|
||||
Source refresh acquires and refines only on explicit request, saving comparisons without
|
||||
changing active Evidence. Source decisions activate through the same Evidence-only
|
||||
pipeline and preserve Catalog readiness. Their envelopes reject arbitrary URLs, paths
|
||||
and extra flags; source locations and credentials come from installation configuration.
|
||||
`preprocess run` does not accept `--resume`, `--dry-run`, a
|
||||
generation identifier, or a rollback option. Re-running it replaces the preceding derived output.
|
||||
`preprocess clear` removes all replaceable preprocessing output and preserves runtime memory.
|
||||
`preprocess clear` removes all replaceable preprocessing output and preserves runtime memory
|
||||
and the canonical local Evidence archive. Full preprocessing remains required afterward.
|
||||
|
||||
## Sources of truth
|
||||
|
||||
@@ -29,8 +47,10 @@ generation identifier, or a rollback option. Re-running it replaces the precedin
|
||||
- PostgreSQL Metadata Catalog is the sole database authority. It owns the workspace/database
|
||||
association, installation-local binding, tables, columns, descriptions, sensitivity flags,
|
||||
physical foreign keys, and active logical relationships.
|
||||
- The workspace Git revision remains authoritative for Evidence. Evidence Descriptor v1/v2 and
|
||||
Curated Evidence Unit v3 are unchanged; there is no Evidence v4.
|
||||
- Evidence Descriptor v1/v2 still configures the initial source. Initialized archives use
|
||||
editable Curated Evidence Unit v4 and the last successfully activated local snapshot.
|
||||
Ordinary preprocessing never imports unconsolidated working-tree edits. Before initialization,
|
||||
the pinned workspace Git source remains supported.
|
||||
|
||||
No metadata is imported from legacy workspace YAML or `physical.yaml`/`annotations.yaml`.
|
||||
|
||||
@@ -98,9 +118,8 @@ generation produced for another database or revision.
|
||||
|
||||
`workspace preprocess clear` is intentionally narrower than deleting all semantic data. It:
|
||||
|
||||
1. copies any `memory` and `solved_question` records still present in the pre-split workspace
|
||||
collection to `<workspace>-memory`, then retires that legacy collection (a normal preprocessing
|
||||
write performs the same one-time cutover if clear was not invoked first);
|
||||
1. leaves `<workspace>-memory` unchanged; Memory projections are reconstructed only from
|
||||
the authoritative PostgreSQL archive, never imported from legacy vector payloads;
|
||||
2. deletes `<workspace>-reference`;
|
||||
3. removes the active LSH generation, Evidence corpus, private Catalog snapshot, and derived job
|
||||
checkpoints;
|
||||
|
||||
@@ -2,7 +2,30 @@
|
||||
|
||||
This page describes the complete workspace Evidence lifecycle: where original material lives, how curated units are produced, when they become available at runtime, and what the author and reviewer are responsible for.
|
||||
|
||||
## Publication rule
|
||||
## Editable local Evidence
|
||||
|
||||
E1 adds [Curated Evidence v4 and a persistent local archive](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md).
|
||||
The visible title and payload fields are authoritative Markdown. New manual units need no
|
||||
external source; consolidation records their curator and distinguishes later corrections
|
||||
from original documentary provenance. The core archive API creates immutable candidates
|
||||
and advances its active pointer only after successful indexing.
|
||||
|
||||
E2 adds **Administration → Evidence management**, actual host file paths, complete
|
||||
browsing and filtering, and the installed `tht workspace evidence consolidate
|
||||
--workspace <id>` command. Edit files externally, consolidate to activate them, then
|
||||
review and run Git manually. Runtime consumes only the active local snapshot.
|
||||
The [v4 contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md#administration-and-installed-command)
|
||||
describes host mounting, first conversion, failure recovery and Clear behavior.
|
||||
The local PSD preview has 35 converted and indexed units. E3 adds explicit import/refresh
|
||||
from local drafts and configured HTTP/S3 sources, durable current/proposed comparisons,
|
||||
and keep/replace decisions with activation and retry. See
|
||||
[Import drafts and refresh sources](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md#import-drafts-and-refresh-sources).
|
||||
Workflow gate corrections remain the subsequent shared increment, X1.
|
||||
|
||||
## Existing repository publication path
|
||||
|
||||
The remainder describes the legacy, uninitialized repository source path. Initialized
|
||||
v4 local archives use the lifecycle above; manual declarations need no source document.
|
||||
|
||||
The workspace repository is the versioned source. ThothII reads it, validates it, and publishes an atomic generation. It does not modify, commit, or push the author's repository.
|
||||
|
||||
@@ -46,7 +69,33 @@ The legacy configuration may expose `source_root`, such as `${THT_DOCS_ROOT}` or
|
||||
|
||||
HTTP and S3 are separate adapters. They do not use the filesystem structure `source/` and `curated/`, but they must still provide stable provenance, without credentials in URIs, under the adapter-specific contract.
|
||||
|
||||
## What a curated unit must contain
|
||||
## What an editable curated unit contains
|
||||
|
||||
New preparation produces unit schema v4. The first H1 contains its title; documented H2
|
||||
sections contain the typed payload. A minimal manual unit is:
|
||||
|
||||
```markdown
|
||||
---
|
||||
schema_version: 4
|
||||
id: evidence:order-key
|
||||
kind: domain
|
||||
language: en
|
||||
purposes: [sql_generation]
|
||||
---
|
||||
|
||||
# Order key
|
||||
|
||||
## Rule
|
||||
|
||||
Join orders using the order number, financial year and company.
|
||||
```
|
||||
|
||||
Use `tht evidence migrate <workspace-root>` for deterministic legacy conversion. The
|
||||
conversion preserves typed content and initializes an archive baseline; it does not
|
||||
activate the local corpus. See the [v4 contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/curated-evidence-v4.md) for
|
||||
all eight kinds, provenance, file layout and the E1/E2 boundary.
|
||||
|
||||
## Legacy v3 representation
|
||||
|
||||
Canonical Curated Evidence v3 hides canonical machine metadata in an HTML comment and renders the
|
||||
whole review surface as real Markdown. GitHub therefore shows no frontmatter table. The body layout
|
||||
@@ -94,7 +143,7 @@ La fascia pediatrica comprende i pazienti con età inferiore a 18 anni.
|
||||
The actual files contain invisible `tht:` comments for canonical metadata and typed-field
|
||||
boundaries. Removing, duplicating, or desynchronizing them makes validation fail closed instead of
|
||||
silently ignoring content. Unit schemas v1 and v2 remain readable for compatibility, but newly
|
||||
prepared units use v3.
|
||||
prepared units use v4. Unit v3 remains readable as a conversion input.
|
||||
|
||||
Curated units must be atomic, readable by a second reviewer, and supported by the source.
|
||||
Provenance references must lead back to the original file and the passage that supports the claim.
|
||||
@@ -213,5 +262,5 @@ Formulas use a format distinct from document Evidence. A formula proposed during
|
||||
|
||||
## Contract references
|
||||
|
||||
- [Workspace Evidence v3 contract](contracts/workspace-evidence-v3.md)
|
||||
- [Preprocessing CLI contract](contracts/workspace-preprocessing-cli.md)
|
||||
- [Workspace Evidence v3 contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/workspace-evidence-v3.md)
|
||||
- [Preprocessing CLI contract](https://git.tylconsulting.it/mptyl/ThothII/src/branch/main/docs/contracts/workspace-preprocessing-cli.md)
|
||||
|
||||
@@ -8,14 +8,26 @@ declare providers, model allowlists, defaults, embeddings, dimensions, or vector
|
||||
Do not edit `deploy/pi/models.json`, `deploy/pi/settings.json`, files under `generated/`, or
|
||||
provider/model environment defaults. Those former sources are retired.
|
||||
|
||||
## Host instructions in Administration
|
||||
|
||||
The **Pi configuration** navigation button opens **Pi management**. Its Host maintenance
|
||||
section selects the installation host's Linux, macOS, or Windows tab automatically; the
|
||||
browser's operating system does not affect it. Selecting another tab is still possible.
|
||||
|
||||
The host CLI includes `THT_HOST_PLATFORM` in the generated Compose projection using the OS
|
||||
on which it runs. Generate projections on the destination host, not on another computer,
|
||||
and do not edit generated files. Without this projection (older deployments or native
|
||||
development), the backend reports its own OS; a Linux Docker container cannot discover
|
||||
whether the physical host is macOS or Windows. Run the updated host CLI's normal
|
||||
configuration reload on the destination installation to regenerate this information.
|
||||
|
||||
## Minimal catalog
|
||||
|
||||
```yaml
|
||||
schemaVersion: 2
|
||||
modelCatalog:
|
||||
defaults:
|
||||
session: zai/glm-5.3
|
||||
metadataGeneration: zai/glm-5.3
|
||||
interaction: zai/glm-5.3
|
||||
|
||||
embedding:
|
||||
id: ollama/qwen3-embedding:0.6b
|
||||
@@ -50,23 +62,63 @@ it contains that use block:
|
||||
- `metadataGeneration` makes it selectable for description generation;
|
||||
- `embedding` is a single installation-level model rather than a selectable list.
|
||||
|
||||
`defaults.session` is required. `defaults.metadataGeneration` is required exactly when at least
|
||||
one metadata-generation model exists. A session manifest pins its canonical identity, so removing a
|
||||
model never silently changes an existing session: resume fails with `model_unavailable`.
|
||||
`defaults.interaction` is the only LLM default, required once per installation, never per workspace.
|
||||
Core and Administration share the user's operational model choice. An explicit choice takes priority
|
||||
over the default and is remembered in this browser for the authenticated user and application mount.
|
||||
Switching workspace does not change the model. Browser-storage restrictions may limit remembering
|
||||
to the current visit; preferences do not synchronize across devices or change installation YAML.
|
||||
An unavailable remembered model is not silently replaced: select another configured model.
|
||||
|
||||
When any metadata-generation models are configured, the default and the operational list must
|
||||
support both `session` and `metadataGeneration`. Entries for just one adapter may remain in the
|
||||
installation inventory, but are not selectable for global interaction. Core-only installations remain
|
||||
supported when no metadata-generation model is configured; Admin AI is then unavailable.
|
||||
The embedding model remains separate and is unaffected by the interaction selector.
|
||||
|
||||
New sessions record the selected model in their manifest. Resume retains the session's workspace and
|
||||
revision, but uses the current global model (the installation default for clients that omit a model).
|
||||
Historical manifest model fields are not rewritten by resume. Archived/finalized sessions remain read-only.
|
||||
|
||||
### Required operator verification for every model
|
||||
|
||||
Catalog validation checks configuration, not model behavior. Before offering a model to users, and
|
||||
after changing its endpoint, adapters, or the Pi/LiteLLM versions, the operator must verify **both**:
|
||||
|
||||
1. **Core / Pi:** select the model, start a test session in a prepared test workspace, exercise an
|
||||
actual tool call and its returned result, a human review gate, and stop/resume. Check streaming,
|
||||
tool arguments, authentication, and reasoning/token-limit compatibility. A plain chat reply or
|
||||
`tht pi test` alone is not sufficient.
|
||||
2. **Administration / LiteLLM:** select the same model and generate descriptions for a small,
|
||||
non-sensitive test table. Check the structured result is accepted and the generation completes.
|
||||
Review the output quality before using it on real metadata. This action writes test metadata
|
||||
and may incur provider charges: use an authorized test database and approved data.
|
||||
|
||||
There is no automatic certification flag or startup model probe. The operator owns this verification;
|
||||
do not infer compatibility from the model label or from success in just one path. Both adapters point
|
||||
to one catalog identity; Pi does not need to route through a new LiteLLM proxy. Models using only
|
||||
`pi_auth` cannot serve the current LiteLLM path and are excluded from shared selection.
|
||||
|
||||
## Session adapters
|
||||
|
||||
Use `pi_builtin` for a model whose technical definition ships with Pi:
|
||||
Use `pi_builtin` for a model whose technical definition ships with Pi. This does not require
|
||||
`pi_auth`: a shared bundle credential lets native Pi and LiteLLM use the same provider identity:
|
||||
|
||||
```yaml
|
||||
deepseek:
|
||||
authentication:
|
||||
mode: pi_auth
|
||||
mode: secret_env
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
session:
|
||||
mode: pi_builtin
|
||||
metadataGeneration:
|
||||
litellmProvider: deepseek
|
||||
models:
|
||||
deepseek-v4-pro:
|
||||
session: {}
|
||||
metadataGeneration: {}
|
||||
deepseek-v4-flash:
|
||||
session: {}
|
||||
metadataGeneration: {}
|
||||
```
|
||||
|
||||
Use `openai_compatible` for an explicit compatible endpoint. Each eligible session model must then
|
||||
@@ -76,6 +128,51 @@ endpoint expects a model name different from the catalog key.
|
||||
Provider integrations remain declarative. Do not register providers from
|
||||
`harness/.pi/extensions/`; those extensions implement the workflow and human gates only.
|
||||
|
||||
### Qwen 3.6 sessions and thinking controls
|
||||
|
||||
For Qwen served through a vLLM-compatible chat template, declare the following inside the
|
||||
model's `session` block, alongside its context and output limits:
|
||||
|
||||
```yaml
|
||||
reasoning: true
|
||||
compatibility:
|
||||
supportsDeveloperRole: false
|
||||
supportsReasoningEffort: false
|
||||
supportsStore: false
|
||||
maxTokensField: max_tokens
|
||||
thinkingFormat: qwen-chat-template
|
||||
```
|
||||
|
||||
This makes Pi send `chat_template_kwargs.enable_thinking` from the selected thinking level,
|
||||
with `preserve_thinking: true`. Choose **off** to explicitly disable thinking. The alternative
|
||||
`thinkingFormat: qwen` is for endpoints expecting top-level `enable_thinking`. Both formats
|
||||
require `reasoning: true`; declaring `reasoning: false` does not tell the server to disable
|
||||
thinking. Omit `thinkingFormat` to preserve Pi's default behavior for other providers.
|
||||
|
||||
Regenerate projections with the updated host CLI and recreate the local core container after
|
||||
rebuilding it. Do not add these fields directly to generated Pi files. These controls do not
|
||||
force tool calls or certify the workflow; perform the operator verification above.
|
||||
|
||||
```sh
|
||||
tht --installation /absolute/path/thothii-installation.yaml installation generate
|
||||
```
|
||||
|
||||
Use the model identifier exposed by your endpoint, such as `qwen3.6-35b-a3b`, and limits
|
||||
supported by that deployment. The thinking format configures the Pi session adapter;
|
||||
metadata generation continues to use its separate LiteLLM settings.
|
||||
|
||||
If a session displays text such as `{"type":"bash","command":"tht session show … --json"}`
|
||||
and never opens a review widget, that text is not an executed tool call. A verified cause
|
||||
was the Evidence JSON extension being loaded into interactive sessions and forcing
|
||||
`response_format: {type: "json_object"}`. Upgrade to the core image containing the fix:
|
||||
the extension belongs in `.pi/evidence-extensions/` and is loaded explicitly only by
|
||||
Evidence authoring. It must not also remain in the automatically loaded `.pi/extensions/`
|
||||
directory. Regenerating model configuration alone does not remove an extension from an old image.
|
||||
|
||||
After upgrading, reload the browser and resume the session. Verify that Pi executes
|
||||
`tht session show` and opens a review widget. This fix does not require changing the Qwen
|
||||
server, forcing every turn to call a tool, or teaching the model to print tool-call JSON.
|
||||
|
||||
## Authentication
|
||||
|
||||
Every provider chooses one explicit mode:
|
||||
@@ -89,6 +186,18 @@ Secret values never belong in installation YAML, generated files, logs, CLI argu
|
||||
requests. The YAML contains only an environment-variable name or an authentication mode. Pi's
|
||||
protected credential file remains selected by the installation authentication configuration.
|
||||
|
||||
For catalog providers using `secret_env`, the bundle is authoritative in both Core and Admin.
|
||||
ThothII removes only the selected provider's old auth entry from the temporary Pi session snapshot;
|
||||
the operator's original Pi auth store and other providers are unchanged. Provider smoke checks use
|
||||
the same precedence, and model enumeration receives the catalog-declared bundle keys. A missing
|
||||
declared key is an error, not permission to fall back to Pi auth or the legacy generic key file.
|
||||
After rotating a bundle key, apply the normal installation lifecycle so processes reload it.
|
||||
|
||||
The PSD descriptor now declares only `deepseek/deepseek-v4-pro` and `deepseek/deepseek-v4-flash`
|
||||
for both uses; it no longer duplicates them under `deepseek-metadata`. Historical records are not
|
||||
rewritten. A saved obsolete identity must be explicitly reselected from the current catalog;
|
||||
it is not silently remapped to another model or account.
|
||||
|
||||
## Generated runtime projections
|
||||
|
||||
Before Compose starts, `tht` validates the installation and atomically writes deterministic files
|
||||
@@ -133,6 +242,16 @@ configuration command. There is no `tht pi configure` and no separate apply comm
|
||||
|
||||
## Migrating a legacy installation
|
||||
|
||||
For schema-v2 descriptors with the former `defaults.session` and `defaults.metadataGeneration`,
|
||||
replace both with `defaults.interaction`. Equal legacy values are accepted and normalized in memory;
|
||||
the loader never rewrites the descriptor. Different values fail with `migration_required`: explicitly
|
||||
choose a model supporting both uses, remove both old fields, and set the single new field. Do not mix
|
||||
new and legacy fields. A Core-only legacy session default can be normalized when Admin AI is absent.
|
||||
|
||||
The generated runtime catalog now uses schema version **2** and only `defaultInteraction`. Regenerate
|
||||
and apply all runtime projections with the matching host/backend release using the normal installation
|
||||
lifecycle; do not deploy only the backend against an old generated catalog or hand-edit generated JSON.
|
||||
|
||||
The migrator reads the former installation `metadataGeneration` block and the two former Pi JSON
|
||||
files, but never modifies them. Supply the facts that cannot be inferred safely and write a separate
|
||||
candidate:
|
||||
@@ -148,13 +267,15 @@ tht --installation /absolute/path/legacy/thothii-installation.yaml installation
|
||||
Review the candidate, move the legacy source files out of the installation only after approval,
|
||||
then select the v2 descriptor. Ambiguous aliases, endpoint conflicts, or missing authentication
|
||||
facts produce field-level errors; the migrator does not guess.
|
||||
The legacy CLI flag `--session-default` now supplies the unified interaction default in the candidate;
|
||||
if it conflicts with the legacy metadata default, align that choice explicitly before retrying.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Meaning | Action |
|
||||
| --- | --- | --- |
|
||||
| `migration_required` | A retired model source or installation schema is still present | Run the installation migrator and review its candidate |
|
||||
| Unknown session or metadata default | The canonical ID is missing the corresponding use block | Correct the provider/model key or add the intended use block |
|
||||
| Invalid interaction default | The canonical ID does not support all configured uses | Correct `defaults.interaction` or the intended adapter blocks |
|
||||
| Generated projection drift | Runtime files differ from the descriptor-derived bytes | Run `tht start` or `tht pi restart --yes --drain` |
|
||||
| `model_unavailable` on resume | The session's pinned model is no longer session-eligible | Restore that catalog entry or keep the session unavailable; do not remap it |
|
||||
| `model_unavailable` on create/resume | The selected global model is no longer eligible | Explicitly choose an eligible model; no fallback is applied |
|
||||
| Provider smoke failure | Credentials, endpoint, or provider availability is invalid | Correct the protected credential or catalog endpoint, restart, then run `tht pi test` |
|
||||
|
||||