Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7630762927 |
+23
-96
@@ -244,29 +244,6 @@ incompatibile o non aggiornato produce nessuna Evidence e un avviso esplicito. I
|
||||
workflow può continuare, ma non usa mai silenziosamente contenuti di una revisione
|
||||
precedente o di un altro workspace.
|
||||
|
||||
## Configurazione dei modelli
|
||||
|
||||
**Workspace Descriptor** — La dichiarazione versionata dell'identità e dello scope del
|
||||
Workspace Database e delle Evidence di un workspace. Non definisce modelli, selezioni di
|
||||
modello o Database Binding specifiche di un'installazione.
|
||||
|
||||
**Installation Model Catalog** — L'insieme dichiarativo, proprio di un'installazione, dei
|
||||
modelli disponibili, dei loro Model Usage e dei relativi default. È l'unica autorità per i
|
||||
modelli di sessione, generazione dei metadati ed embedding e non appartiene a un workspace.
|
||||
_Avoid_: Model Catalog, Metadata Generation Model Configuration
|
||||
|
||||
**Model Usage** — Lo scopo per cui un modello dell'Installation Model Catalog può essere
|
||||
usato: `session`, `metadata_generation` oppure `embedding`. L'ammissibilità e il default
|
||||
dipendono dall'uso, non dal workspace.
|
||||
|
||||
**Model Selection** — La scelta runtime, a livello di installazione, di un modello del
|
||||
catalogo per uno specifico Model Usage. Riferisce l'identità canonica del modello senza
|
||||
ridefinirne provider, endpoint o capacità.
|
||||
|
||||
**Model Runtime Projection** — La rappresentazione derivata e non autoritativa
|
||||
dell'Installation Model Catalog richiesta da uno specifico runtime. Può essere rigenerata
|
||||
integralmente dalla configurazione dell'installazione.
|
||||
|
||||
## Catalogo dei metadati
|
||||
|
||||
**Workspace Database** — Il database che appartiene a un solo workspace e non può essere
|
||||
@@ -319,40 +296,8 @@ Metadata Catalog. Non è creata o modificata manualmente, ma può essere rimossa
|
||||
Metadata Cleanup.
|
||||
_Avoid_: denormalized FK, relationship string
|
||||
|
||||
**Logical Relationship** — Una relazione modificabile fra due Catalog Column che non corrisponde
|
||||
necessariamente a un vincolo fisico. Può essere Generated o Manual e rimane distinta dalla Catalog
|
||||
Relationship osservata nel database.
|
||||
|
||||
**Generated Relationship** — Una Logical Relationship ricavata dai nomi delle colonne, dalle
|
||||
primary key e dalla compatibilità dei tipi mediante regole deterministiche, senza LLM, embedding o
|
||||
campionamento dei dati. Una ricostruzione non riattiva una Generated Relationship cancellata
|
||||
logicamente, ma può ricrearne una cancellata fisicamente.
|
||||
|
||||
**Manual Relationship** — Una Logical Relationship aggiunta dall'utente. La ricostruzione delle
|
||||
Generated Relationship non la modifica.
|
||||
|
||||
**Logical Relationship Deletion** — L'esclusione persistente di una Logical Relationship che ne
|
||||
conserva l'identità per impedirne la ricreazione automatica finché esistono entrambe le Catalog
|
||||
Column alle quali è collegata.
|
||||
|
||||
**Permanent Relationship Deletion** — La rimozione completa di una Logical Relationship. Una
|
||||
ricostruzione successiva può ricrearla quando soddisfa nuovamente le regole di inferenza. Anche il
|
||||
cleanup distruttivo di una tabella o colonna endpoint rimuove permanentemente le relative esclusioni.
|
||||
|
||||
**Relationship Reconstruction** — L'operazione amministrativa esplicita che scopre e aggiunge le
|
||||
Generated Relationship mancanti. Conserva le Manual Relationship e le relationship già presenti e
|
||||
non riattiva quelle cancellate logicamente.
|
||||
|
||||
**Relationship Restore** — La riattivazione esplicita di una Logical Relationship cancellata
|
||||
logicamente.
|
||||
|
||||
**Effective Relationship Map** — La vista unificata delle Catalog Relationship fisiche e delle
|
||||
Logical Relationship, con origine e stato espliciti. È l'interfaccia usata dall'amministrazione e
|
||||
dalla comprensione dello schema, non un ulteriore modello persistito.
|
||||
|
||||
**Effective Relationship Snapshot** — La proiezione runtime immutabile delle relationship attive
|
||||
contenute nell'Effective Relationship Map. È derivata dal Metadata Catalog per una singola sessione
|
||||
e viene eliminata insieme alla relativa configurazione runtime.
|
||||
**Logical Relationship** — Una relazione semantica curata o inferita che non corrisponde
|
||||
necessariamente a un vincolo fisico. Ha ownership e lifecycle distinti da Catalog Relationship.
|
||||
|
||||
**Description** — Il testo curato e consolidato che descrive una Catalog Table o Catalog Column
|
||||
per gli usi downstream.
|
||||
@@ -408,56 +353,38 @@ con la binding corrente.
|
||||
distinti dai fatti strutturali governati dalla sincronizzazione. Possono essere popolati dall'AI,
|
||||
da un'importazione o da una modifica amministrativa senza cambiare il database esterno.
|
||||
|
||||
**Metadata Generation Model Configuration** — La configurazione a livello di setup applicativo
|
||||
che elenca i modelli selezionabili, il default e i riferimenti agli eventuali segreti per la sola
|
||||
generazione dei metadati. Un modello keyless è ammesso solo con un endpoint esplicito che non
|
||||
richiede autenticazione. Non appartiene al workspace ed è indipendente dalla configurazione Pi.
|
||||
|
||||
**Model Completion Helper** — Il processo Python interno ed effimero che esegue una singola
|
||||
richiesta LiteLLM per conto del backend. Non è un servizio HTTP, non possiede il lifecycle della
|
||||
Description Generation Run e non è una CLI esposta agli utenti.
|
||||
|
||||
**Catalog Sample** — Un input transitorio composto da un massimo di cinque righe e da valori di
|
||||
esempio bounded di una Catalog Table per la generazione delle descrizioni. Può contenere valori
|
||||
reali oppure sintetici in base alla Source Value Disclosure Decision; non viene persistito e non
|
||||
diventa Catalog Metadata.
|
||||
reali oppure sintetici in base al Sensitive Data Flag della Catalog Column; non viene persistito
|
||||
e non diventa Catalog Metadata.
|
||||
|
||||
**Sensitive Data Flag** — La classificazione binaria umana applicata a una Catalog Column. Può
|
||||
essere impostata liberamente dall'amministratore anche in contrasto con una valutazione automatica.
|
||||
**Sensitive Data Flag** — La scelta binaria umana applicata a una Catalog Column: `true` protegge
|
||||
i valori sorgente e `false` ne consente l'invio al modello. Il valore predefinito è `false`, anche
|
||||
per le nuove colonne.
|
||||
|
||||
**Local Sensitivity Assessment** — La valutazione locale, non autoritativa e priva di LLM di una
|
||||
Catalog Column, basata su metadati e contenuto sorgente, con esito `sensitive`, `non_sensitive`
|
||||
oppure `unknown`.
|
||||
_Avoid_: AI suggestion, automatic flag
|
||||
|
||||
**Local NER Detector** — Il componente NLP opzionale e CPU-only che esamina soltanto testo ancora
|
||||
ambiguo e restituisce evidenze al Local Sensitivity Assessment. Non decide lo stato della colonna,
|
||||
non usa un LLM generativo e non persiste valori sorgente.
|
||||
_Avoid_: AI classifier, local LLM fallback
|
||||
|
||||
**Model Data Boundary** — La qualificazione amministrativa di un modello come `internal` oppure
|
||||
`external` rispetto al confine entro cui i valori sorgente possono essere comunicati.
|
||||
_Avoid_: local model, remote model
|
||||
|
||||
**Source Value Disclosure Decision** — L'unica decisione effettiva che stabilisce se un modello
|
||||
riceve valori sorgente reali oppure sostituti sintetici, combinando Model Data Boundary e Sensitive
|
||||
Data Flag.
|
||||
_Avoid_: sample filter, export flag
|
||||
|
||||
**Sensitive Data Policy** — L'insieme versionato di regole locali generali e specifiche che produce
|
||||
una Local Sensitivity Assessment. Un singolo riscontro blocca l'intera colonna e qualsiasi valore
|
||||
testuale più lungo di 500 caratteri rende sensibile la colonna.
|
||||
**Sensitive Data Policy** — La regola che applica il Sensitive Data Flag ai Catalog Sample:
|
||||
valori sintetici per una colonna protetta, valori reali per una colonna non protetta. L'AI può
|
||||
suggerire il flag dai soli metadati tecnici di un database, delle tabelle o delle colonne
|
||||
esplicitamente selezionate; le richieste ampie vengono divise in batch bounded, ma soltanto
|
||||
l'utente imposta i flag dopo aver rivisto la proposta completa.
|
||||
_Avoid_: PII filter, sample filter
|
||||
|
||||
**Sensitivity Analysis Run** — Il tentativo amministrativo esplicito e tracciato che valuta una
|
||||
selezione di colonne mediante la Sensitive Data Policy. Conserva stato, copertura e conteggi
|
||||
aggregati, ma non valori sorgente né esiti per colonna.
|
||||
_Avoid_: Sensitive Data Suggestion Run, AI analysis
|
||||
**Sensitive Data Suggestion Run** — Il tentativo amministrativo tracciato con cui il modello
|
||||
propone Sensitive Data Flag dai soli metadati strutturali. Conserva stato e conteggi aggregati,
|
||||
ma non i suggerimenti per colonna, che restano una proposta transitoria fino al salvataggio umano.
|
||||
|
||||
**Sensitivity Review Draft** — La proposta transitoria che associa alle colonne selezionate una
|
||||
Local Sensitivity Assessment e le relative evidenze sanificate. Non modifica il Sensitive Data Flag
|
||||
finché l'amministratore non salva le proprie decisioni e viene scartata al reload.
|
||||
_Avoid_: automatic flag
|
||||
|
||||
**Sensitivity Analysis Event** — Una riga testuale ordinata e sanificata che registra l'avvio,
|
||||
l'esito o l'errore di una Sensitivity Analysis Run senza conservare contenuti sorgente, output grezzi
|
||||
del detector o proposte per colonna.
|
||||
_Avoid_: Sensitive Data Suggestion Event
|
||||
**Sensitive Data Suggestion Event** — Una riga testuale ordinata e sanitizzata che registra
|
||||
l'avvio, l'esito o l'errore di una Sensitive Data Suggestion Run senza conservare prompt,
|
||||
risposte grezze del provider o proposte per colonna.
|
||||
|
||||
**Introspection Capability** — Una categoria di struttura fisica che una Database Binding
|
||||
può osservare, come tabelle, colonne, relazioni, indici o enum. Una capability non disponibile
|
||||
|
||||
@@ -12,10 +12,6 @@ colors:
|
||||
muted-graphite: "oklch(51.33% 0.0088 345.6)"
|
||||
quiet-border: "oklch(90.93% 0.0035 354.7)"
|
||||
success-mint: "oklch(75.77% 0.1581 165)"
|
||||
navigation-active: "oklch(92.5% 0.052 23.2)"
|
||||
navigation-active-hover: "oklch(89.5% 0.071 23.2)"
|
||||
navigation-active-foreground: "oklch(36.5% 0.11 23.2)"
|
||||
navigation-active-border: "oklch(60% 0.135 23.2)"
|
||||
warning-amber: "oklch(85.23% 0.1386 78.9)"
|
||||
information-blue: "oklch(70.35% 0.1128 221.3)"
|
||||
typography:
|
||||
@@ -156,8 +152,8 @@ frontend uses OKLCH tokens directly.
|
||||
|
||||
### Primary
|
||||
|
||||
- **Instrument Red** (`instrument-red`): primary actions, focus identity, and destructive meaning
|
||||
where the context already makes the action explicit.
|
||||
- **Instrument Red** (`instrument-red`): primary actions, current selection, focus identity, and
|
||||
destructive meaning where the context already makes the action explicit.
|
||||
- **Instrument Red Pressed** (`instrument-red-hover`): hover and active emphasis for the primary
|
||||
action family.
|
||||
|
||||
@@ -174,9 +170,6 @@ frontend uses OKLCH tokens directly.
|
||||
### Semantic
|
||||
|
||||
- **Success Mint** (`success-mint`): completed and ready states.
|
||||
- **Navigation Active** (`navigation-active`): the one application surface currently in the
|
||||
foreground. It shares Instrument Red's hue but uses a lighter, lower-chroma fill, so location is
|
||||
visible without carrying the full weight of a primary action.
|
||||
- **Warning Amber** (`warning-amber`): waiting, attention, and in-progress states.
|
||||
- **Information Blue** (`information-blue`): informational state when red would imply action.
|
||||
|
||||
@@ -279,44 +272,15 @@ default, hover, focus, active, disabled, loading, and error behavior where those
|
||||
- **Focus:** three-pixel Instrument Red ring with a clear border shift.
|
||||
- **Error / Disabled:** errors combine destructive color with explanatory text; disabled controls
|
||||
retain readable contrast and use 50 percent opacity.
|
||||
- **Metadata catalog model:** Database Management keeps one compact, installation-level
|
||||
metadata-generation LLM selector in the application header. The selection persists across
|
||||
database, table, column, and relationship views; when no usable profile is configured, the
|
||||
disabled control explains: “No metadata-generation LLM model is configured for this installation.”
|
||||
|
||||
### Navigation
|
||||
|
||||
- **Style:** compact session rows use `8px` corners and restrained vertical padding.
|
||||
- **Default / Hover / Active:** porcelain at rest, Sunken Surface on hover, and a muted Navigation
|
||||
Active red with a defined border when current. Exactly one top-level navigation control is current.
|
||||
- **Administrative controls:** the admin-only Administration accordion groups Database management,
|
||||
a structural divider, Workspace management, and Pi management in that order. Its trigger exposes
|
||||
expanded state and starts collapsed by default, while non-admin users do not receive the accordion
|
||||
or its navigation actions.
|
||||
- **Default / Hover / Active:** transparent at rest, Sunken Surface on hover, and the same surface
|
||||
with stronger text weight when active.
|
||||
- **Responsive:** collapse navigation structurally at the application breakpoint. Do not shrink
|
||||
labels into illegibility.
|
||||
|
||||
### Tabs
|
||||
|
||||
- **Shape:** compact label tabs sit on a shared baseline with rounded top corners and a two-pixel
|
||||
lower edge. Inactive labels retain a complete Quiet Border and Porcelain Card surface, so every
|
||||
label reads as a tab before interaction; hover feedback reinforces clickability.
|
||||
- **Current:** the selected tab uses the muted Navigation Active red for its fill, text, and defined border.
|
||||
It must expose `aria-selected`, participate in a labelled `tablist`/`tabpanel`, and be the only
|
||||
tab in the roving keyboard tab order.
|
||||
- **Keyboard:** Left/Right move between adjacent tabs with wrapping; Home/End select the first or
|
||||
last tab.
|
||||
|
||||
### Tooltips
|
||||
|
||||
- **Row actions:** icon-action tooltips open three pixels below the trigger and align to its trailing
|
||||
edge, so they never cover the icon row. They use a dark slate surface, porcelain text, and a
|
||||
defined border rather than the light popover treatment.
|
||||
- **Interaction:** tooltip layers never receive pointer events. They appear on hover and keyboard
|
||||
focus with a short ease-out transition, while the icon button keeps its complete accessible name.
|
||||
- **Scope:** this treatment is shared by database, table, column, and relationship row actions.
|
||||
Toolbar and navigation hints may use separate collision-aware placement.
|
||||
|
||||
### Curated Evidence Documents
|
||||
|
||||
Curated evidence follows a fixed reading order: title, compact type and purpose summary, scope,
|
||||
|
||||
+15
-28
@@ -1,6 +1,6 @@
|
||||
# ThothII — Project State
|
||||
|
||||
Last updated: 2026-08-31.
|
||||
Last updated: 2026-08-27.
|
||||
|
||||
This file is the short operational snapshot. Stable commands and the architecture mental model
|
||||
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
|
||||
@@ -90,27 +90,16 @@ relationships. Curated and generated descriptions are editable; generated descri
|
||||
and Database Management can generate or consolidate them for selected tables, selected columns,
|
||||
all targets, or only targets whose Generated Description is missing.
|
||||
|
||||
Relationship Management is now reachable directly from each configured Fleet database. One
|
||||
Relationship Map shows read-only Physical Relationships together with Generated and Manual Logical
|
||||
Relationships, with Active, Excluded, and All filters. Administrators can add a single-column
|
||||
relationship, run deterministic name/PK/type inference, exclude or restore a logical relationship,
|
||||
or delete it permanently. Exclusion retains a tombstone that a rebuild cannot reactivate; permanent
|
||||
deletion allows a later rebuild to infer the same endpoints again. Inference uses no LLM, embedding,
|
||||
or source values. It supports normalized table-qualified names, unique non-generic PK names,
|
||||
composite-PK source columns, and the `*time_key -> dim_time.<single PK>` warehouse convention while
|
||||
ignoring bare generic names. Explicit table/column metadata cleanup remains a destructive boundary: it removes
|
||||
the attached logical relationships and exclusions and requires a full schema synchronization before
|
||||
inference or runtime publication can continue.
|
||||
|
||||
The previous Database Management renderer remains a temporary comparison fallback for development
|
||||
and staging only: `?db-ui=legacy` is honored in Vite development or when
|
||||
`VITE_DB_MANAGEMENT_LEGACY=true`; it is not a production presentation. The standalone Fleet Ledger
|
||||
prototype on port `5173` also remains temporary until owner acceptance of the integrated surface,
|
||||
after which both migration aids can be removed.
|
||||
|
||||
Schema refresh is one durable asynchronous engine with database-table, selected-table-column,
|
||||
relationship, and full-database actions. Database-level menus expose only the table, relationship,
|
||||
and full scopes; selecting tables exposes column synchronization plus manual column and relationship cleanup for that subset. Database selections
|
||||
Schema refresh is one durable asynchronous engine with database-table, database-column,
|
||||
selected-table-column, relationship, and full-database actions. Database-level menus expose the
|
||||
table, all-column, relationship, and full scopes separately; selecting tables exposes column
|
||||
synchronization plus manual column and relationship cleanup for that subset. Database selections
|
||||
also expose manual table and relationship cleanup. Cleanup selections are atomic and share the
|
||||
one-active-operation-per-database exclusion with synchronization. Runs have leases and
|
||||
restart recovery, atomic apply, destructive-diff confirmation with re-scan, cancellation before
|
||||
@@ -126,15 +115,13 @@ SSH is not yet enabled for NL→SQL session runtime.
|
||||
|
||||
The catalog runs in the internal `catalog-db` PostgreSQL service. Kysely migrations are an explicit
|
||||
one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before local startup. Runtime
|
||||
sessions now consume the Catalog's active effective relationship map through an immutable JSON
|
||||
snapshot tied to the runtime-config lease. The harness uses that snapshot as its exclusive
|
||||
relationship source while retaining Git-pinned annotations for descriptive metadata; legacy
|
||||
runtimes without a snapshot keep the previous merge behavior. The accepted design is recorded in
|
||||
sessions still consume the existing workspace configuration in this slice: database-management
|
||||
records do not yet change the NL→SQL handoff. The accepted design is recorded in
|
||||
`docs/plans/2026-08-26-metadata-catalog-from-thothai.md`, the snapshot contract under
|
||||
`docs/contracts/`, and ADRs 0001–0012.
|
||||
`docs/contracts/`, and ADRs 0001–0011.
|
||||
|
||||
Semantic aliases, value descriptions, synonyms, and concepts remain deferred to their dedicated
|
||||
slices.
|
||||
Semantic aliases, value descriptions, synonyms, concepts, and logical relationships remain
|
||||
deferred to their dedicated slices.
|
||||
|
||||
AI Description Generation uses the catalog's human-owned Sensitive Data Flag. The flag defaults to
|
||||
`false`, including for newly synchronized columns. An administrator may request an AI proposal based
|
||||
@@ -171,11 +158,11 @@ credential. The Python client supplies only its fixed non-secret compatibility p
|
||||
The AritmoLab entry also sets `disableThinking: true`, mapped to the endpoint's chat-template flag,
|
||||
because its default reasoning prose would violate the worker's exact JSON response contract.
|
||||
|
||||
Logical relationship integration with core schema-linking is complete: session creation and resume
|
||||
materialize the active physical/generated/manual map, retrieval-pack generation and Pi receive the
|
||||
same runtime config, and snapshot validation fails closed on a declared missing, invalid, or orphaned
|
||||
endpoint. Broader publication of other Catalog metadata to schema-linking remains a separate future
|
||||
slice.
|
||||
Integration of the completed metadata catalog with core schema-linking is explicitly deferred
|
||||
until the database, table, column, relationship, and synchronization slices are complete. At that
|
||||
point the next required design gate is to compare the catalog snapshot with the current DWH
|
||||
preprocessing/schema-linking contracts and plan the cutover; this follow-up must not be treated as
|
||||
optional cleanup or silently omitted.
|
||||
|
||||
**Deferred follow-up — Sensitive Data Policy in schema-linking.** The policy is first delivered
|
||||
and tested in catalog description generation. Its enforcement for core schema-linking remains
|
||||
|
||||
+2
-22
@@ -59,13 +59,10 @@ import { DescriptionGenerationWorker } from "./catalog/description-generation-wo
|
||||
import { SensitiveDataSuggester } from "./catalog/sensitive-data-suggester.js";
|
||||
import { SensitiveDataSuggestionRunner } from "./catalog/sensitive-data-suggestion-runner.js";
|
||||
import {
|
||||
ConcreteDescriptionSourceSampler,
|
||||
PostgresDescriptionSourceSampler,
|
||||
type DescriptionSourceSampler,
|
||||
} from "./catalog/description-source-sampler.js";
|
||||
import { catalogDescriptionGenerationRoutes } from "./routes/catalog-description-generation.js";
|
||||
import { CatalogLogicalRelationshipService } from "./catalog/logical-relationship-service.js";
|
||||
import { catalogLogicalRelationshipRoutes } from "./routes/catalog-logical-relationships.js";
|
||||
import { EffectiveRelationshipSnapshotProvider } from "./catalog/effective-relationship-snapshot.js";
|
||||
|
||||
export interface BuildAppDeps {
|
||||
thtRunner?: ThtRunner;
|
||||
@@ -82,8 +79,6 @@ export interface BuildAppDeps {
|
||||
catalogService?: CatalogService;
|
||||
catalogPostgresAccess?: CatalogPostgresAccess;
|
||||
catalogTableService?: CatalogTableService;
|
||||
catalogLogicalRelationshipService?: CatalogLogicalRelationshipService;
|
||||
effectiveRelationshipSnapshotProvider?: EffectiveRelationshipSnapshotProvider;
|
||||
catalogSchemaIntrospector?: CatalogSchemaIntrospector;
|
||||
catalogSyncWorker?: CatalogSyncWorker;
|
||||
catalogOperationCoordinator?: CatalogOperationCoordinator;
|
||||
@@ -177,7 +172,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
{ connectTimeoutMs: config.workspaceDiagnosticTimeoutMs },
|
||||
);
|
||||
const descriptionSourceSampler = deps?.descriptionSourceSampler
|
||||
?? new ConcreteDescriptionSourceSampler(catalogPostgresAccess, workspaceSecretStore);
|
||||
?? new PostgresDescriptionSourceSampler(catalogPostgresAccess);
|
||||
const descriptionGenerationWorker = new DescriptionGenerationWorker(
|
||||
catalogRepository,
|
||||
workspaceRegistry,
|
||||
@@ -205,16 +200,6 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
catalogOperationCoordinator,
|
||||
);
|
||||
const catalogTableService = deps?.catalogTableService ?? new CatalogTableService(catalogRepository);
|
||||
const catalogLogicalRelationshipService = deps?.catalogLogicalRelationshipService
|
||||
?? new CatalogLogicalRelationshipService(catalogRepository);
|
||||
const effectiveRelationships = deps?.effectiveRelationshipSnapshotProvider
|
||||
?? (config.catalogDatabase === undefined
|
||||
? undefined
|
||||
: new EffectiveRelationshipSnapshotProvider(
|
||||
catalogRepository,
|
||||
catalogLogicalRelationshipService,
|
||||
catalogOperationCoordinator,
|
||||
));
|
||||
const catalogSchemaIntrospector = deps?.catalogSchemaIntrospector ?? new ConcreteCatalogSchemaIntrospector(
|
||||
catalogPostgresAccess,
|
||||
workspaceSecretStore,
|
||||
@@ -409,7 +394,6 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
legacyWorkspaceMode: config.legacyWorkspaceMode,
|
||||
workspaceRuntimeSupport,
|
||||
maintenanceBarrier,
|
||||
effectiveRelationships,
|
||||
});
|
||||
app.post("/internal/maintenance/activate", async (req, reply) => {
|
||||
try {
|
||||
@@ -458,10 +442,6 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
worker: catalogSyncWorker,
|
||||
operations: catalogOperationCoordinator,
|
||||
});
|
||||
catalogLogicalRelationshipRoutes(app, {
|
||||
service: catalogLogicalRelationshipService,
|
||||
operations: catalogOperationCoordinator,
|
||||
});
|
||||
catalogDescriptionConsolidationRoutes(app, {
|
||||
repository: catalogRepository,
|
||||
operations: catalogOperationCoordinator,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { z } from "zod";
|
||||
import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||
import type { MetadataGenerationModels, ResolvedMetadataGenerationModel } from "./metadata-generation-models.js";
|
||||
import type { ModelCompleter, ModelCompletionMessage, ModelCompletionResult } from "./model-completer.js";
|
||||
import type { ModelCompleter, ModelCompletionMessage } from "./model-completer.js";
|
||||
import {
|
||||
ModelCompletionCancelledError,
|
||||
ModelCompletionProviderError,
|
||||
@@ -38,7 +38,6 @@ const MAX_SAMPLE_FIELDS_PER_ROW = 4;
|
||||
const MAX_SAMPLE_COLUMNS = 4;
|
||||
const MAX_REPRESENTATIVE_VALUES_PER_REQUEST = 5;
|
||||
const MAX_TARGET_SAMPLE_JSON_BYTES = 8 * 1024;
|
||||
const MAX_COMPLETION_ATTEMPTS_PER_BATCH = 2;
|
||||
const generatedOutcomeSchema = z.object({
|
||||
targetId: z.uuid(),
|
||||
outcome: z.literal("generated"),
|
||||
@@ -56,9 +55,7 @@ const completionResponseSchema = z.object({
|
||||
results: z.array(outcomeSchema).min(1).max(MAX_TARGETS_PER_BATCH),
|
||||
}).strict();
|
||||
|
||||
class InvalidModelJsonError extends Error {}
|
||||
class InvalidModelSchemaError extends Error {}
|
||||
class MissingModelTargetsError extends Error {}
|
||||
class InvalidModelOutcomeError extends Error {}
|
||||
class DescriptionGenerationBatchError extends Error {
|
||||
constructor(
|
||||
readonly failure: unknown,
|
||||
@@ -146,9 +143,6 @@ interface DescriptionGenerationCounters {
|
||||
generated: number;
|
||||
nonGeneratable: number;
|
||||
failed: number;
|
||||
inputTokens: number;
|
||||
cacheReadTokens: number;
|
||||
outputTokens: number;
|
||||
consecutiveTechnicalFailures: number;
|
||||
}
|
||||
|
||||
@@ -158,9 +152,6 @@ function persistedCounters(counters: DescriptionGenerationCounters) {
|
||||
generated: counters.generated,
|
||||
nonGeneratable: counters.nonGeneratable,
|
||||
failed: counters.failed,
|
||||
inputTokens: counters.inputTokens,
|
||||
cacheReadTokens: counters.cacheReadTokens,
|
||||
outputTokens: counters.outputTokens,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -634,47 +625,36 @@ function messagesFor(
|
||||
}
|
||||
|
||||
function parseOutcomes(content: string, expectedTargetIds: readonly string[]): Map<string, ParsedOutcome> {
|
||||
const trimmed = content.trim();
|
||||
const fenced = /^```(?:json)?[ \t]*\r?\n([\s\S]*?)\r?\n```$/iu.exec(trimmed);
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(fenced?.[1] ?? trimmed);
|
||||
} catch {
|
||||
throw new InvalidModelJsonError();
|
||||
}
|
||||
|
||||
const response = completionResponseSchema.safeParse(parsed);
|
||||
if (!response.success) throw new InvalidModelSchemaError();
|
||||
|
||||
const outcomes = response.data.results;
|
||||
const expected = new Set(expectedTargetIds);
|
||||
if (outcomes.length !== expectedTargetIds.length || expected.size !== expectedTargetIds.length) {
|
||||
throw new MissingModelTargetsError();
|
||||
}
|
||||
const mapped = new Map<string, ParsedOutcome>();
|
||||
for (const outcome of outcomes) {
|
||||
if (!expected.has(outcome.targetId) || mapped.has(outcome.targetId)) {
|
||||
throw new MissingModelTargetsError();
|
||||
const trimmed = content.trim();
|
||||
const fenced = /^```(?:json)?[ \t]*\r?\n([\s\S]*?)\r?\n```$/iu.exec(trimmed);
|
||||
const outcomes = completionResponseSchema.parse(JSON.parse(fenced?.[1] ?? trimmed)).results;
|
||||
const expected = new Set(expectedTargetIds);
|
||||
if (outcomes.length !== expectedTargetIds.length || expected.size !== expectedTargetIds.length) {
|
||||
throw new InvalidModelOutcomeError();
|
||||
}
|
||||
mapped.set(outcome.targetId, outcome.outcome === "generated"
|
||||
? { ...outcome, description: outcome.description.trim() }
|
||||
: outcome);
|
||||
const mapped = new Map<string, ParsedOutcome>();
|
||||
for (const outcome of outcomes) {
|
||||
if (!expected.has(outcome.targetId) || mapped.has(outcome.targetId)) {
|
||||
throw new InvalidModelOutcomeError();
|
||||
}
|
||||
mapped.set(outcome.targetId, outcome.outcome === "generated"
|
||||
? { ...outcome, description: outcome.description.trim() }
|
||||
: outcome);
|
||||
}
|
||||
if (mapped.size !== expected.size) throw new InvalidModelOutcomeError();
|
||||
return mapped;
|
||||
} catch (error) {
|
||||
if (error instanceof InvalidModelOutcomeError) throw error;
|
||||
throw new InvalidModelOutcomeError();
|
||||
}
|
||||
if (mapped.size !== expected.size) throw new MissingModelTargetsError();
|
||||
return mapped;
|
||||
}
|
||||
|
||||
function safeFailure(error: unknown): string {
|
||||
if (error instanceof DescriptionGenerationFailureStreakError) return error.message;
|
||||
const failure = error instanceof DescriptionGenerationBatchError ? error.failure : error;
|
||||
if (failure instanceof ModelCompletionProviderError) return "The model provider request failed.";
|
||||
if (failure instanceof InvalidModelJsonError) return "The model response was not valid JSON.";
|
||||
if (failure instanceof InvalidModelSchemaError) {
|
||||
return "The model response did not match the required schema.";
|
||||
}
|
||||
if (failure instanceof MissingModelTargetsError) {
|
||||
return "The model response was missing one or more requested targets.";
|
||||
}
|
||||
if (failure instanceof InvalidModelOutcomeError) return "The model response was invalid.";
|
||||
return "Description generation failed.";
|
||||
}
|
||||
|
||||
@@ -927,9 +907,6 @@ export class DescriptionGenerationWorker {
|
||||
generated: 0,
|
||||
nonGeneratable: 0,
|
||||
failed: 0,
|
||||
inputTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
outputTokens: 0,
|
||||
consecutiveTechnicalFailures: 0,
|
||||
};
|
||||
await this.processTargets(run, database, plan.columnTargets, model, counters, signal);
|
||||
@@ -996,41 +973,20 @@ export class DescriptionGenerationWorker {
|
||||
);
|
||||
}
|
||||
throwIfCancelled(signal);
|
||||
const expectedTargetIds = batch.map((target) => (
|
||||
target.kind === "column" ? target.column.id : target.table.id
|
||||
));
|
||||
const messages = messagesFor(database, batch, run.language, sourceSamples);
|
||||
let outcomes: Map<string, ParsedOutcome> | undefined;
|
||||
let terminalFailure: unknown;
|
||||
for (let attempt = 1; attempt <= MAX_COMPLETION_ATTEMPTS_PER_BATCH; attempt += 1) {
|
||||
try {
|
||||
const completion = await this.completer.complete({ model, messages, signal });
|
||||
const result: ModelCompletionResult = typeof completion === "string"
|
||||
? { content: completion, usage: { input: 0, cacheRead: 0, output: 0 } }
|
||||
: completion;
|
||||
counters.inputTokens += result.usage.input;
|
||||
counters.cacheReadTokens += result.usage.cacheRead;
|
||||
counters.outputTokens += result.usage.output;
|
||||
throwIfCancelled(signal);
|
||||
outcomes = parseOutcomes(result.content, expectedTargetIds);
|
||||
break;
|
||||
} catch (error) {
|
||||
if (error instanceof ModelCompletionCancelledError) throw error;
|
||||
terminalFailure = error;
|
||||
if (attempt < MAX_COMPLETION_ATTEMPTS_PER_BATCH) {
|
||||
await this.appendEvent(
|
||||
run.id,
|
||||
"warning",
|
||||
`${safeFailure(error)} Retrying batch (attempt ${attempt + 1} of ${MAX_COMPLETION_ATTEMPTS_PER_BATCH}).`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!outcomes) {
|
||||
const batchError = new DescriptionGenerationBatchError(
|
||||
terminalFailure,
|
||||
batch.map(failureTarget),
|
||||
);
|
||||
let outcomes: Map<string, ParsedOutcome>;
|
||||
try {
|
||||
const content = await this.completer.complete({
|
||||
model,
|
||||
messages: messagesFor(database, batch, run.language, sourceSamples),
|
||||
signal,
|
||||
});
|
||||
throwIfCancelled(signal);
|
||||
outcomes = parseOutcomes(content, batch.map((target) => (
|
||||
target.kind === "column" ? target.column.id : target.table.id
|
||||
)));
|
||||
} catch (error) {
|
||||
if (error instanceof ModelCompletionCancelledError) throw error;
|
||||
const batchError = new DescriptionGenerationBatchError(error, batch.map(failureTarget));
|
||||
counters.processed += batch.length;
|
||||
counters.failed += batch.length;
|
||||
counters.consecutiveTechnicalFailures += 1;
|
||||
@@ -1053,10 +1009,7 @@ export class DescriptionGenerationWorker {
|
||||
const targetId = target.kind === "column" ? target.column.id : target.table.id;
|
||||
const outcome = outcomes.get(targetId);
|
||||
if (!outcome) {
|
||||
throw new DescriptionGenerationBatchError(
|
||||
new MissingModelTargetsError(),
|
||||
[failureTarget(target)],
|
||||
);
|
||||
throw new DescriptionGenerationBatchError(new InvalidModelOutcomeError(), [failureTarget(target)]);
|
||||
}
|
||||
const generatedDescription = outcome.outcome === "generated"
|
||||
? outcome.description
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import type { WorkspaceSecretStore } from "../workspaces/secret-store.js";
|
||||
import type { CatalogPostgresAccess } from "./postgres-access.js";
|
||||
import { CATALOG_SECRET_IDS } from "./secrets.js";
|
||||
import { CatalogConnectorError, type WorkspaceDatabase } from "./types.js";
|
||||
import type { WorkspaceDatabase } from "./types.js";
|
||||
|
||||
const MAX_SOURCE_ROWS = 5;
|
||||
const MAX_REPRESENTATIVE_VALUES = 5;
|
||||
@@ -82,82 +79,15 @@ function distinctKey(value: Exclude<DescriptionSourceSampleValue, null>): string
|
||||
return `${typeof value}:${String(value)}`;
|
||||
}
|
||||
|
||||
function columnsFor(target: DescriptionSourceSamplingTarget): string[] {
|
||||
return [...new Set(target.columnNames)].slice(0, MAX_SOURCE_COLUMNS_PER_TARGET);
|
||||
}
|
||||
|
||||
function normalizedSample(
|
||||
target: DescriptionSourceSamplingTarget,
|
||||
columnNames: readonly string[],
|
||||
sourceRows: readonly Record<string, unknown>[],
|
||||
): DescriptionTargetSourceSample {
|
||||
const rows = sourceRows.slice(0, MAX_SOURCE_ROWS).map((row) => ({
|
||||
fields: columnNames.flatMap((name) => {
|
||||
const value = normalizeValue(row[name]);
|
||||
return value === undefined ? [] : [{ name, value }];
|
||||
}),
|
||||
}));
|
||||
const valuesByColumn = new Map<string, Exclude<DescriptionSourceSampleValue, null>[]>();
|
||||
const seenByColumn = new Map<string, Set<string>>();
|
||||
let representativeValueCount = 0;
|
||||
for (const row of rows) {
|
||||
for (const field of row.fields) {
|
||||
if (representativeValueCount === MAX_REPRESENTATIVE_VALUES) break;
|
||||
if (field.value === null) continue;
|
||||
const seen = seenByColumn.get(field.name) ?? new Set<string>();
|
||||
const key = distinctKey(field.value);
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
seenByColumn.set(field.name, seen);
|
||||
const values = valuesByColumn.get(field.name) ?? [];
|
||||
values.push(field.value);
|
||||
valuesByColumn.set(field.name, values);
|
||||
representativeValueCount += 1;
|
||||
}
|
||||
if (representativeValueCount === MAX_REPRESENTATIVE_VALUES) break;
|
||||
}
|
||||
return {
|
||||
targetId: target.targetId,
|
||||
tableName: target.tableName,
|
||||
rows,
|
||||
representativeValues: columnNames.flatMap((column) => {
|
||||
const values = valuesByColumn.get(column);
|
||||
return values && values.length > 0 ? [{ column, values }] : [];
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function samplingSql(
|
||||
database: WorkspaceDatabase,
|
||||
target: DescriptionSourceSamplingTarget,
|
||||
columns: readonly string[],
|
||||
): string {
|
||||
const projections = columns.map((columnName) => {
|
||||
const identifier = quoteIdentifier(columnName);
|
||||
return `LEFT((${identifier})::text, ${MAX_SOURCE_VALUE_BYTES}) AS ${identifier}`;
|
||||
});
|
||||
return [
|
||||
`SELECT ${projections.join(", ")}`,
|
||||
`FROM ${quoteIdentifier(database.schema)}.${quoteIdentifier(target.tableName)}`,
|
||||
`LIMIT ${MAX_SOURCE_ROWS}`,
|
||||
].join(" ");
|
||||
}
|
||||
|
||||
/** Bounded source sampler that follows the database's PostgreSQL-wire or REST binding. */
|
||||
export class ConcreteDescriptionSourceSampler implements DescriptionSourceSampler {
|
||||
constructor(
|
||||
private readonly access: CatalogPostgresAccess,
|
||||
private readonly secretStore?: Pick<WorkspaceSecretStore, "materialize">,
|
||||
) {}
|
||||
/** PostgreSQL-wire sampler. REST bindings remain unsupported by CatalogPostgresAccess. */
|
||||
export class PostgresDescriptionSourceSampler implements DescriptionSourceSampler {
|
||||
constructor(private readonly access: CatalogPostgresAccess) {}
|
||||
|
||||
async sample(
|
||||
database: WorkspaceDatabase,
|
||||
targets: readonly DescriptionSourceSamplingTarget[],
|
||||
signal: AbortSignal,
|
||||
): Promise<readonly DescriptionTargetSourceSample[]> {
|
||||
if (database.binding.transport === "rest_api") {
|
||||
return await this.sampleRest(database, targets, signal);
|
||||
}
|
||||
const client = await this.access.connect(database, signal);
|
||||
let transactionOpen = false;
|
||||
try {
|
||||
@@ -165,7 +95,7 @@ export class ConcreteDescriptionSourceSampler implements DescriptionSourceSample
|
||||
transactionOpen = true;
|
||||
const samples: DescriptionTargetSourceSample[] = [];
|
||||
for (const target of targets) {
|
||||
const columnNames = columnsFor(target);
|
||||
const columnNames = [...new Set(target.columnNames)].slice(0, MAX_SOURCE_COLUMNS_PER_TARGET);
|
||||
if (columnNames.length === 0) {
|
||||
samples.push({
|
||||
targetId: target.targetId,
|
||||
@@ -185,7 +115,44 @@ export class ConcreteDescriptionSourceSampler implements DescriptionSourceSample
|
||||
"LIMIT $2",
|
||||
].join(" ");
|
||||
const result = await client.query(sql, [MAX_SOURCE_VALUE_BYTES, MAX_SOURCE_ROWS]);
|
||||
samples.push(normalizedSample(target, columnNames, result.rows));
|
||||
const rows = result.rows.slice(0, MAX_SOURCE_ROWS).map((row) => ({
|
||||
fields: columnNames.flatMap((name) => {
|
||||
const value = normalizeValue(row[name]);
|
||||
return value === undefined ? [] : [{ name, value }];
|
||||
}),
|
||||
}));
|
||||
const valuesByColumn = new Map<
|
||||
string,
|
||||
Exclude<DescriptionSourceSampleValue, null>[]
|
||||
>();
|
||||
const seenByColumn = new Map<string, Set<string>>();
|
||||
let representativeValueCount = 0;
|
||||
for (const row of rows) {
|
||||
for (const field of row.fields) {
|
||||
if (representativeValueCount === MAX_REPRESENTATIVE_VALUES) break;
|
||||
if (field.value === null) continue;
|
||||
const seen = seenByColumn.get(field.name) ?? new Set<string>();
|
||||
const key = distinctKey(field.value);
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
seenByColumn.set(field.name, seen);
|
||||
const values = valuesByColumn.get(field.name) ?? [];
|
||||
values.push(field.value);
|
||||
valuesByColumn.set(field.name, values);
|
||||
representativeValueCount += 1;
|
||||
}
|
||||
if (representativeValueCount === MAX_REPRESENTATIVE_VALUES) break;
|
||||
}
|
||||
const representativeValues = columnNames.flatMap((column) => {
|
||||
const values = valuesByColumn.get(column);
|
||||
return values && values.length > 0 ? [{ column, values }] : [];
|
||||
});
|
||||
samples.push({
|
||||
targetId: target.targetId,
|
||||
tableName: target.tableName,
|
||||
rows,
|
||||
representativeValues,
|
||||
});
|
||||
}
|
||||
return samples;
|
||||
} finally {
|
||||
@@ -193,60 +160,4 @@ export class ConcreteDescriptionSourceSampler implements DescriptionSourceSample
|
||||
await client.end().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
private async sampleRest(
|
||||
database: WorkspaceDatabase,
|
||||
targets: readonly DescriptionSourceSamplingTarget[],
|
||||
signal: AbortSignal,
|
||||
): Promise<readonly DescriptionTargetSourceSample[]> {
|
||||
if (!this.secretStore) throw new CatalogConnectorError("REST source sampling is not configured");
|
||||
const auth = database.binding.restAuth ?? "bearer";
|
||||
const materialized = this.secretStore.materialize(
|
||||
database.workspaceId,
|
||||
auth === "none" ? [] : [CATALOG_SECRET_IDS.apiKey],
|
||||
);
|
||||
try {
|
||||
const headers: Record<string, string> = { "content-type": "application/json" };
|
||||
if (auth !== "none") {
|
||||
const credentialFile = materialized.files.get(CATALOG_SECRET_IDS.apiKey);
|
||||
if (!credentialFile) throw new CatalogConnectorError("REST API key is not configured");
|
||||
const credential = (await readFile(credentialFile, "utf8")).trim();
|
||||
if (auth === "bearer") headers.authorization = `Bearer ${credential}`;
|
||||
else headers["x-api-key"] = credential;
|
||||
}
|
||||
const baseUrl = database.binding.baseUrl?.replace(/\/+$/, "");
|
||||
if (!baseUrl) throw new CatalogConnectorError("Database binding is incomplete");
|
||||
const samples: DescriptionTargetSourceSample[] = [];
|
||||
for (const target of targets) {
|
||||
const columnNames = columnsFor(target);
|
||||
if (columnNames.length === 0) {
|
||||
samples.push(normalizedSample(target, columnNames, []));
|
||||
continue;
|
||||
}
|
||||
const response = await fetch(`${baseUrl}/rpc/run_query`, {
|
||||
method: "POST",
|
||||
headers,
|
||||
body: JSON.stringify({ query_text: samplingSql(database, target, columnNames) }),
|
||||
signal,
|
||||
});
|
||||
if (!response.ok) throw new CatalogConnectorError("REST source sampling failed");
|
||||
const body: unknown = await response.json();
|
||||
if (!Array.isArray(body)
|
||||
|| body.some((row) => !row || typeof row !== "object" || Array.isArray(row))) {
|
||||
throw new CatalogConnectorError("REST source sampling response is invalid");
|
||||
}
|
||||
samples.push(normalizedSample(
|
||||
target,
|
||||
columnNames,
|
||||
body as Array<Record<string, unknown>>,
|
||||
));
|
||||
}
|
||||
return samples;
|
||||
} catch (error) {
|
||||
if (error instanceof CatalogConnectorError) throw error;
|
||||
throw new CatalogConnectorError("REST source sampling failed");
|
||||
} finally {
|
||||
materialized.release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,93 +0,0 @@
|
||||
import type { CatalogRelationship, CatalogRepository } from "./types.js";
|
||||
|
||||
export interface EffectiveRelationshipSnapshotReader {
|
||||
list(databaseId: string): Promise<CatalogRelationship[]>;
|
||||
}
|
||||
|
||||
export interface EffectiveRelationshipSnapshotCoordinator {
|
||||
run<T>(databaseId: string, operation: () => Promise<T>): Promise<T>;
|
||||
}
|
||||
|
||||
export class EffectiveRelationshipSnapshotStaleError extends Error {}
|
||||
|
||||
interface EffectiveRelationship {
|
||||
sourceTable: string;
|
||||
sourceColumns: string[];
|
||||
targetTable: string;
|
||||
targetColumns: string[];
|
||||
origin: CatalogRelationship["origin"];
|
||||
}
|
||||
|
||||
interface EffectiveRelationshipSnapshot {
|
||||
schemaVersion: 1;
|
||||
workspaceId: string;
|
||||
relationships: EffectiveRelationship[];
|
||||
}
|
||||
|
||||
const originRank: Record<CatalogRelationship["origin"], number> = {
|
||||
physical: 0,
|
||||
manual: 1,
|
||||
generated: 2,
|
||||
};
|
||||
|
||||
function endpointKey(relationship: EffectiveRelationship): string {
|
||||
return [
|
||||
relationship.sourceTable,
|
||||
relationship.sourceColumns.join("\u0000"),
|
||||
relationship.targetTable,
|
||||
relationship.targetColumns.join("\u0000"),
|
||||
].join("\u0001");
|
||||
}
|
||||
|
||||
function compareRelationships(left: EffectiveRelationship, right: EffectiveRelationship): number {
|
||||
return endpointKey(left).localeCompare(endpointKey(right))
|
||||
|| originRank[left.origin] - originRank[right.origin];
|
||||
}
|
||||
|
||||
/**
|
||||
* Adapter from the mutable Catalog model to the immutable relationship contract consumed by the
|
||||
* harness. The returned JSON is a deterministic projection, never an authored second store.
|
||||
*/
|
||||
export class EffectiveRelationshipSnapshotProvider {
|
||||
constructor(
|
||||
private readonly repository: Pick<CatalogRepository, "get" | "getByWorkspace">,
|
||||
private readonly relationships: EffectiveRelationshipSnapshotReader,
|
||||
private readonly operations: EffectiveRelationshipSnapshotCoordinator,
|
||||
) {}
|
||||
|
||||
async render(workspaceId: string): Promise<string | undefined> {
|
||||
const database = await this.repository.getByWorkspace(workspaceId);
|
||||
if (!database) return undefined;
|
||||
return await this.operations.run(database.id, async () => {
|
||||
const current = await this.repository.get(database.id);
|
||||
if (!current || current.schemaSyncedVersion !== current.version) {
|
||||
throw new EffectiveRelationshipSnapshotStaleError(
|
||||
"effective relationship snapshot requires a current full schema synchronization",
|
||||
);
|
||||
}
|
||||
const projected = (await this.relationships.list(database.id))
|
||||
.filter((relationship) => relationship.status === "active")
|
||||
.map((relationship): EffectiveRelationship => ({
|
||||
sourceTable: relationship.sourceTableName,
|
||||
sourceColumns: relationship.columns.map((column) => column.sourceColumnName),
|
||||
targetTable: relationship.targetTableName,
|
||||
targetColumns: relationship.columns.map((column) => column.targetColumnName),
|
||||
origin: relationship.origin,
|
||||
}))
|
||||
.sort(compareRelationships);
|
||||
|
||||
const seen = new Set<string>();
|
||||
const snapshot: EffectiveRelationshipSnapshot = {
|
||||
schemaVersion: 1,
|
||||
workspaceId,
|
||||
relationships: projected.filter((relationship) => {
|
||||
const key = endpointKey(relationship);
|
||||
if (seen.has(key)) return false;
|
||||
seen.add(key);
|
||||
return true;
|
||||
}),
|
||||
};
|
||||
return `${JSON.stringify(snapshot, null, 2)}\n`;
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,260 +0,0 @@
|
||||
import type {
|
||||
CatalogLogicalRelationship,
|
||||
CatalogLogicalRelationshipCandidate,
|
||||
CatalogLogicalRelationshipContext,
|
||||
CatalogLogicalRelationshipEndpoint,
|
||||
CatalogRelationship,
|
||||
CatalogRepository,
|
||||
} from "./types.js";
|
||||
|
||||
export class LogicalRelationshipDatabaseNotFoundError extends Error {}
|
||||
export class LogicalRelationshipDuplicateError extends Error {}
|
||||
export class LogicalRelationshipNotFoundError extends Error {}
|
||||
export class LogicalRelationshipReadOnlyError extends Error {}
|
||||
export class LogicalRelationshipSchemaStaleError extends Error {}
|
||||
export class LogicalRelationshipTargetNotUniqueError extends Error {}
|
||||
export class LogicalRelationshipTypeIncompatibleError extends Error {}
|
||||
export class LogicalRelationshipColumnNotFoundError extends Error {
|
||||
constructor(readonly field: "sourceColumnId" | "targetColumnId") {
|
||||
super(`Catalog column '${field}' was not found`);
|
||||
}
|
||||
}
|
||||
|
||||
export interface RebuildGeneratedRelationshipsResult {
|
||||
added: number;
|
||||
alreadyPresent: number;
|
||||
excluded: number;
|
||||
ambiguous: number;
|
||||
}
|
||||
|
||||
function identifierTokens(value: string): string[] {
|
||||
return value
|
||||
.replace(/([a-z0-9])([A-Z])/g, "$1_$2")
|
||||
.toLowerCase()
|
||||
.split(/[^a-z0-9]+/)
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
function singularWord(value: string): string {
|
||||
if (value.length > 4 && value.endsWith("ies")) return `${value.slice(0, -3)}y`;
|
||||
if (value.length > 4 && /(ches|shes|xes|zes|ses)$/.test(value)) return value.slice(0, -2);
|
||||
if (value.length > 3 && value.endsWith("s") && !/(ss|us)$/.test(value)) return value.slice(0, -1);
|
||||
return value;
|
||||
}
|
||||
|
||||
function tableAliases(tableName: string): string[] {
|
||||
const tokens = identifierTokens(tableName);
|
||||
if (tokens.length === 0) return [];
|
||||
const normalized = tokens.join("_");
|
||||
const singular = [...tokens];
|
||||
singular[singular.length - 1] = singularWord(singular[singular.length - 1]);
|
||||
return [...new Set([normalized, singular.join("_")])];
|
||||
}
|
||||
|
||||
const GENERIC_PRIMARY_KEY_NAMES = new Set(["id", "key", "code", "pk"]);
|
||||
|
||||
function nameMatches(
|
||||
source: CatalogLogicalRelationshipEndpoint,
|
||||
target: CatalogLogicalRelationshipEndpoint,
|
||||
): boolean {
|
||||
const sourceName = identifierTokens(source.columnName).join("_");
|
||||
const targetName = identifierTokens(target.columnName).join("_");
|
||||
if (!sourceName || !targetName) return false;
|
||||
const expected = new Set<string>();
|
||||
if (!GENERIC_PRIMARY_KEY_NAMES.has(targetName)) expected.add(targetName);
|
||||
for (const alias of tableAliases(target.tableName)) {
|
||||
expected.add(`${alias}_${targetName}`);
|
||||
expected.add(`${alias.replaceAll("_", "")}${targetName.replaceAll("_", "")}`);
|
||||
if (targetName === "id" || targetName === "pk") expected.add(alias);
|
||||
}
|
||||
return expected.has(sourceName);
|
||||
}
|
||||
|
||||
function canonicalDataType(value: string): string {
|
||||
const normalized = value.trim().toLowerCase().replace(/\s+/g, " ");
|
||||
const arraySuffix = normalized.endsWith("[]") ? "[]" : "";
|
||||
const base = arraySuffix ? normalized.slice(0, -2) : normalized;
|
||||
const withoutModifier = base.replace(/\([^)]*\)/g, "").trim();
|
||||
const aliases: Record<string, string> = {
|
||||
int2: "smallint",
|
||||
smallserial: "smallint",
|
||||
int4: "integer",
|
||||
int: "integer",
|
||||
serial: "integer",
|
||||
int8: "bigint",
|
||||
bigserial: "bigint",
|
||||
decimal: "numeric",
|
||||
varchar: "text",
|
||||
"character varying": "text",
|
||||
bool: "boolean",
|
||||
"timestamp without time zone": "timestamp",
|
||||
"timestamp with time zone": "timestamptz",
|
||||
"time without time zone": "time",
|
||||
"time with time zone": "timetz",
|
||||
};
|
||||
return `${aliases[withoutModifier] ?? withoutModifier}${arraySuffix}`;
|
||||
}
|
||||
|
||||
function typesCompatible(left: string, right: string): boolean {
|
||||
return canonicalDataType(left) === canonicalDataType(right);
|
||||
}
|
||||
|
||||
function pairKey(sourceColumnId: string, targetColumnId: string): string {
|
||||
return `${sourceColumnId}\u0000${targetColumnId}`;
|
||||
}
|
||||
|
||||
function relationshipPair(relationship: CatalogLogicalRelationship): CatalogLogicalRelationshipCandidate {
|
||||
return {
|
||||
sourceColumnId: relationship.columns[0].sourceColumnId,
|
||||
targetColumnId: relationship.columns[0].targetColumnId,
|
||||
};
|
||||
}
|
||||
|
||||
function relationshipSortKey(relationship: CatalogRelationship): string {
|
||||
const sourceColumns = relationship.columns.map((column) => column.sourceColumnName).join(",");
|
||||
const targetColumns = relationship.columns.map((column) => column.targetColumnName).join(",");
|
||||
return [
|
||||
relationship.sourceTableName,
|
||||
sourceColumns,
|
||||
relationship.targetTableName,
|
||||
targetColumns,
|
||||
relationship.origin,
|
||||
].join("\u0000");
|
||||
}
|
||||
|
||||
export class CatalogLogicalRelationshipService {
|
||||
constructor(private readonly repository: CatalogRepository) {}
|
||||
|
||||
async list(databaseId: string): Promise<CatalogRelationship[]> {
|
||||
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
const relationships: CatalogRelationship[] = [
|
||||
...await this.repository.listRelationships(databaseId),
|
||||
...await this.repository.listLogicalRelationships(databaseId),
|
||||
];
|
||||
return relationships.sort((left, right) => relationshipSortKey(left).localeCompare(relationshipSortKey(right)));
|
||||
}
|
||||
|
||||
async addManual(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
): Promise<CatalogLogicalRelationship> {
|
||||
const context = await this.requiredContext(databaseId);
|
||||
const source = context.endpoints.find((endpoint) => endpoint.columnId === sourceColumnId);
|
||||
if (!source) throw new LogicalRelationshipColumnNotFoundError("sourceColumnId");
|
||||
const target = context.endpoints.find((endpoint) => endpoint.columnId === targetColumnId);
|
||||
if (!target) throw new LogicalRelationshipColumnNotFoundError("targetColumnId");
|
||||
if (sourceColumnId === targetColumnId
|
||||
|| target.primaryKeyPosition === null
|
||||
|| target.tablePrimaryKeyColumnCount !== 1) {
|
||||
throw new LogicalRelationshipTargetNotUniqueError();
|
||||
}
|
||||
if (!typesCompatible(source.dataType, target.dataType)) {
|
||||
throw new LogicalRelationshipTypeIncompatibleError();
|
||||
}
|
||||
const key = pairKey(sourceColumnId, targetColumnId);
|
||||
if (context.physicalPairs.some((pair) => pairKey(pair.sourceColumnId, pair.targetColumnId) === key)
|
||||
|| context.logicalRelationships.some((relationship) => {
|
||||
const pair = relationshipPair(relationship);
|
||||
return pairKey(pair.sourceColumnId, pair.targetColumnId) === key;
|
||||
})) throw new LogicalRelationshipDuplicateError();
|
||||
const created = await this.repository.insertLogicalRelationship(
|
||||
databaseId,
|
||||
sourceColumnId,
|
||||
targetColumnId,
|
||||
false,
|
||||
);
|
||||
if (!created) throw new LogicalRelationshipDuplicateError();
|
||||
return created;
|
||||
}
|
||||
|
||||
async rebuildGenerated(databaseId: string): Promise<RebuildGeneratedRelationshipsResult> {
|
||||
const context = await this.requiredContext(databaseId);
|
||||
const targets = context.endpoints.filter((endpoint) => (
|
||||
endpoint.primaryKeyPosition !== null && endpoint.tablePrimaryKeyColumnCount === 1
|
||||
));
|
||||
const physical = new Set(context.physicalPairs.map((pair) => pairKey(pair.sourceColumnId, pair.targetColumnId)));
|
||||
const active = new Set<string>();
|
||||
const excluded = new Set<string>();
|
||||
for (const relationship of context.logicalRelationships) {
|
||||
const pair = relationshipPair(relationship);
|
||||
(relationship.status === "excluded" ? excluded : active)
|
||||
.add(pairKey(pair.sourceColumnId, pair.targetColumnId));
|
||||
}
|
||||
|
||||
const pending: CatalogLogicalRelationshipCandidate[] = [];
|
||||
let alreadyPresent = 0;
|
||||
let excludedCount = 0;
|
||||
let ambiguous = 0;
|
||||
const dimTimeTargets = targets.filter((target) => (
|
||||
identifierTokens(target.tableName).join("_") === "dim_time"
|
||||
));
|
||||
const sources = context.endpoints.filter((endpoint) => (
|
||||
endpoint.primaryKeyPosition === null || endpoint.tablePrimaryKeyColumnCount > 1
|
||||
));
|
||||
for (const source of sources) {
|
||||
const sourceName = identifierTokens(source.columnName).join("_");
|
||||
const isTimeKey = sourceName.endsWith("time_key")
|
||||
&& identifierTokens(source.tableName).join("_") !== "dim_time";
|
||||
const candidates = isTimeKey ? dimTimeTargets : targets;
|
||||
const matches = candidates.filter((target) => (
|
||||
target.columnId !== source.columnId
|
||||
&& typesCompatible(source.dataType, target.dataType)
|
||||
&& (isTimeKey || nameMatches(source, target))
|
||||
));
|
||||
if (matches.length > 1) {
|
||||
ambiguous += 1;
|
||||
continue;
|
||||
}
|
||||
if (matches.length === 0) continue;
|
||||
const candidate = { sourceColumnId: source.columnId, targetColumnId: matches[0].columnId };
|
||||
const key = pairKey(candidate.sourceColumnId, candidate.targetColumnId);
|
||||
if (physical.has(key) || active.has(key)) {
|
||||
alreadyPresent += 1;
|
||||
} else if (excluded.has(key)) {
|
||||
excludedCount += 1;
|
||||
} else {
|
||||
pending.push(candidate);
|
||||
}
|
||||
}
|
||||
const added = await this.repository.insertGeneratedLogicalRelationships(databaseId, pending);
|
||||
alreadyPresent += pending.length - added;
|
||||
return { added, alreadyPresent, excluded: excludedCount, ambiguous };
|
||||
}
|
||||
|
||||
async setStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship> {
|
||||
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
const updated = await this.repository.setLogicalRelationshipStatus(databaseId, relationshipId, status);
|
||||
if (updated) return updated;
|
||||
await this.assertNotPhysical(databaseId, relationshipId);
|
||||
throw new LogicalRelationshipNotFoundError();
|
||||
}
|
||||
|
||||
async deletePermanently(databaseId: string, relationshipId: string): Promise<void> {
|
||||
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
if (await this.repository.deleteLogicalRelationship(databaseId, relationshipId)) return;
|
||||
await this.assertNotPhysical(databaseId, relationshipId);
|
||||
throw new LogicalRelationshipNotFoundError();
|
||||
}
|
||||
|
||||
private async requiredContext(databaseId: string): Promise<CatalogLogicalRelationshipContext> {
|
||||
const database = await this.repository.get(databaseId);
|
||||
if (!database) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
if (database.schemaSyncedVersion !== database.version) {
|
||||
throw new LogicalRelationshipSchemaStaleError();
|
||||
}
|
||||
const context = await this.repository.getLogicalRelationshipContext(databaseId);
|
||||
if (!context) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
return context;
|
||||
}
|
||||
|
||||
private async assertNotPhysical(databaseId: string, relationshipId: string): Promise<void> {
|
||||
if ((await this.repository.listRelationships(databaseId)).some((relationship) => relationship.id === relationshipId)) {
|
||||
throw new LogicalRelationshipReadOnlyError();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -14,10 +14,7 @@ import {
|
||||
type CatalogDatabaseMetadataDeleteTarget,
|
||||
type CatalogMetadataDeleteCounts,
|
||||
type CatalogMetrics,
|
||||
type CatalogLogicalRelationship,
|
||||
type CatalogLogicalRelationshipCandidate,
|
||||
type CatalogLogicalRelationshipContext,
|
||||
type CatalogPhysicalRelationship,
|
||||
type CatalogRelationship,
|
||||
type CatalogSchemaDiff,
|
||||
type CatalogSyncCounts,
|
||||
type CatalogSyncEvent,
|
||||
@@ -52,8 +49,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
private readonly records = new Map<string, WorkspaceDatabase>();
|
||||
private readonly tables = new Map<string, CatalogTable>();
|
||||
private readonly columns = new Map<string, CatalogColumn>();
|
||||
private readonly relationships = new Map<string, CatalogPhysicalRelationship>();
|
||||
private readonly logicalRelationships = new Map<string, CatalogLogicalRelationship>();
|
||||
private readonly relationships = new Map<string, CatalogRelationship>();
|
||||
private readonly descriptionGenerationRuns = new Map<string, DescriptionGenerationRun>();
|
||||
private readonly descriptionGenerationEvents = new Map<string, DescriptionGenerationEvent[]>();
|
||||
private readonly sensitiveDataSuggestionRuns = new Map<string, SensitiveDataSuggestionRun>();
|
||||
@@ -175,9 +171,6 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
for (const [relationshipId, relationship] of this.relationships) {
|
||||
if (relationship.databaseId === id) this.relationships.delete(relationshipId);
|
||||
}
|
||||
for (const [relationshipId, relationship] of this.logicalRelationships) {
|
||||
if (relationship.databaseId === id) this.logicalRelationships.delete(relationshipId);
|
||||
}
|
||||
for (const [runId, run] of this.descriptionGenerationRuns) {
|
||||
if (run.databaseId !== id) continue;
|
||||
this.descriptionGenerationRuns.delete(runId);
|
||||
@@ -335,10 +328,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
processed: 0,
|
||||
generated: 0,
|
||||
nonGeneratable: 0,
|
||||
failed: 0,
|
||||
inputTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
outputTokens: 0,
|
||||
failed: 0,
|
||||
createdAt: now,
|
||||
startedAt: null,
|
||||
updatedAt: now,
|
||||
@@ -447,10 +437,7 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
status: "running",
|
||||
total: 0,
|
||||
suggestedSensitive: 0,
|
||||
suggestedNonSensitive: 0,
|
||||
inputTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
outputTokens: 0,
|
||||
suggestedNonSensitive: 0,
|
||||
createdAt: now,
|
||||
startedAt: now,
|
||||
updatedAt: now,
|
||||
@@ -540,138 +527,12 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
.map((event) => structuredClone(event));
|
||||
}
|
||||
|
||||
async listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]> {
|
||||
async listRelationships(databaseId: string): Promise<CatalogRelationship[]> {
|
||||
return [...this.relationships.values()].filter((relationship) => relationship.databaseId === databaseId)
|
||||
.sort((a, b) => `${a.sourceTableName}.${a.constraintName}`.localeCompare(`${b.sourceTableName}.${b.constraintName}`))
|
||||
.map((relationship) => structuredClone(relationship));
|
||||
}
|
||||
|
||||
async listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]> {
|
||||
return [...this.logicalRelationships.values()]
|
||||
.filter((relationship) => relationship.databaseId === databaseId)
|
||||
.sort((a, b) => {
|
||||
const left = `${a.sourceTableName}.${a.columns[0].sourceColumnName}.${a.targetTableName}.${a.columns[0].targetColumnName}`;
|
||||
const right = `${b.sourceTableName}.${b.columns[0].sourceColumnName}.${b.targetTableName}.${b.columns[0].targetColumnName}`;
|
||||
return left.localeCompare(right);
|
||||
})
|
||||
.map((relationship) => structuredClone(relationship));
|
||||
}
|
||||
|
||||
async getLogicalRelationshipContext(
|
||||
databaseId: string,
|
||||
): Promise<CatalogLogicalRelationshipContext | undefined> {
|
||||
if (!this.records.has(databaseId)) return undefined;
|
||||
const tables = [...this.tables.values()].filter((table) => table.databaseId === databaseId);
|
||||
const tableById = new Map(tables.map((table) => [table.id, table]));
|
||||
const columns = [...this.columns.values()].filter((column) => tableById.has(column.tableId));
|
||||
const primaryKeyCounts = new Map<string, number>();
|
||||
for (const column of columns) {
|
||||
if (column.primaryKeyPosition !== null) {
|
||||
primaryKeyCounts.set(column.tableId, (primaryKeyCounts.get(column.tableId) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
return {
|
||||
endpoints: columns.map((column) => ({
|
||||
columnId: column.id,
|
||||
columnName: column.name,
|
||||
tableId: column.tableId,
|
||||
tableName: tableById.get(column.tableId)!.name,
|
||||
dataType: column.dataType,
|
||||
primaryKeyPosition: column.primaryKeyPosition,
|
||||
tablePrimaryKeyColumnCount: primaryKeyCounts.get(column.tableId) ?? 0,
|
||||
})),
|
||||
physicalPairs: [...this.relationships.values()]
|
||||
.filter((relationship) => relationship.databaseId === databaseId)
|
||||
.flatMap((relationship) => relationship.columns.map((column) => ({
|
||||
sourceColumnId: column.sourceColumnId,
|
||||
targetColumnId: column.targetColumnId,
|
||||
}))),
|
||||
logicalRelationships: await this.listLogicalRelationships(databaseId),
|
||||
};
|
||||
}
|
||||
|
||||
async insertLogicalRelationship(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
generated: boolean,
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
if ([...this.logicalRelationships.values()].some((relationship) => (
|
||||
relationship.databaseId === databaseId
|
||||
&& relationship.columns[0].sourceColumnId === sourceColumnId
|
||||
&& relationship.columns[0].targetColumnId === targetColumnId
|
||||
))) return undefined;
|
||||
const sourceColumn = this.columns.get(sourceColumnId);
|
||||
const targetColumn = this.columns.get(targetColumnId);
|
||||
const sourceTable = sourceColumn ? this.tables.get(sourceColumn.tableId) : undefined;
|
||||
const targetTable = targetColumn ? this.tables.get(targetColumn.tableId) : undefined;
|
||||
if (!sourceColumn || !targetColumn || !sourceTable || !targetTable
|
||||
|| sourceTable.databaseId !== databaseId || targetTable.databaseId !== databaseId
|
||||
|| sourceColumnId === targetColumnId) return undefined;
|
||||
const now = new Date().toISOString();
|
||||
const relationship: CatalogLogicalRelationship = {
|
||||
id: randomUUID(),
|
||||
databaseId,
|
||||
constraintName: null,
|
||||
sourceTableId: sourceTable.id,
|
||||
sourceTableName: sourceTable.name,
|
||||
targetTableId: targetTable.id,
|
||||
targetTableName: targetTable.name,
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId,
|
||||
sourceColumnName: sourceColumn.name,
|
||||
targetColumnId,
|
||||
targetColumnName: targetColumn.name,
|
||||
}],
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
origin: generated ? "generated" : "manual",
|
||||
status: "active",
|
||||
};
|
||||
this.logicalRelationships.set(relationship.id, relationship);
|
||||
return structuredClone(relationship);
|
||||
}
|
||||
|
||||
async insertGeneratedLogicalRelationships(
|
||||
databaseId: string,
|
||||
candidates: readonly CatalogLogicalRelationshipCandidate[],
|
||||
): Promise<number> {
|
||||
let added = 0;
|
||||
for (const candidate of candidates) {
|
||||
if (await this.insertLogicalRelationship(
|
||||
databaseId,
|
||||
candidate.sourceColumnId,
|
||||
candidate.targetColumnId,
|
||||
true,
|
||||
)) added += 1;
|
||||
}
|
||||
return added;
|
||||
}
|
||||
|
||||
async setLogicalRelationshipStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
const current = this.logicalRelationships.get(relationshipId);
|
||||
if (!current || current.databaseId !== databaseId) return undefined;
|
||||
const updated = { ...current, status, updatedAt: new Date().toISOString() };
|
||||
this.logicalRelationships.set(relationshipId, updated);
|
||||
return structuredClone(updated);
|
||||
}
|
||||
|
||||
async deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean> {
|
||||
const current = this.logicalRelationships.get(relationshipId);
|
||||
return Boolean(current?.databaseId === databaseId && this.logicalRelationships.delete(relationshipId));
|
||||
}
|
||||
|
||||
async deleteDatabaseMetadata(
|
||||
databaseIds: readonly string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
@@ -713,12 +574,6 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
const columns = [...this.columns.values()].filter((column) => selected.has(column.tableId));
|
||||
const deletedColumnIds = new Set(columns.map((column) => column.id));
|
||||
for (const column of columns) this.columns.delete(column.id);
|
||||
for (const relationship of [...this.logicalRelationships.values()]) {
|
||||
const pair = relationship.columns[0];
|
||||
if (deletedColumnIds.has(pair.sourceColumnId) || deletedColumnIds.has(pair.targetColumnId)) {
|
||||
this.logicalRelationships.delete(relationship.id);
|
||||
}
|
||||
}
|
||||
for (const [relationshipId, relationship] of this.relationships) {
|
||||
if (relationship.databaseId !== databaseId) continue;
|
||||
this.relationships.set(relationshipId, {
|
||||
@@ -1002,8 +857,6 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
lastSyncedAt: now,
|
||||
createdAt: current?.createdAt ?? now,
|
||||
updatedAt: comparable === nextComparable ? (current?.updatedAt ?? now) : now,
|
||||
origin: "physical",
|
||||
status: "active",
|
||||
});
|
||||
if (!current) created += 1;
|
||||
else if (comparable !== nextComparable) updated += 1;
|
||||
@@ -1220,12 +1073,6 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
this.relationships.delete(relationship.id);
|
||||
}
|
||||
}
|
||||
for (const relationship of [...this.logicalRelationships.values()]) {
|
||||
const pair = relationship.columns[0];
|
||||
if (pair.sourceColumnId === columnId || pair.targetColumnId === columnId) {
|
||||
this.logicalRelationships.delete(relationship.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private markCatalogIncomplete(databaseIds: readonly string[]): void {
|
||||
|
||||
@@ -10,8 +10,6 @@ import * as catalogRuntimeSequencePrivilegesMigration from "./migrations/004_cat
|
||||
import * as descriptionGenerationRunsMigration from "./migrations/005_description_generation_runs.js";
|
||||
import * as sensitiveDataFlagMigration from "./migrations/006_sensitive_data_flag.js";
|
||||
import * as sensitiveDataSuggestionRunsMigration from "./migrations/007_sensitive_data_suggestion_runs.js";
|
||||
import * as catalogLogicalRelationshipsMigration from "./migrations/008_catalog_logical_relationships.js";
|
||||
import * as aiTokenUsageMigration from "./migrations/009_ai_token_usage.js";
|
||||
|
||||
const connectionString = process.env.THT_CATALOG_MIGRATOR_DATABASE_URL;
|
||||
const host = process.env.THT_CATALOG_DB_HOST;
|
||||
@@ -44,8 +42,6 @@ const provider: MigrationProvider = {
|
||||
"005_description_generation_runs": descriptionGenerationRunsMigration,
|
||||
"006_sensitive_data_flag": sensitiveDataFlagMigration,
|
||||
"007_sensitive_data_suggestion_runs": sensitiveDataSuggestionRunsMigration,
|
||||
"008_catalog_logical_relationships": catalogLogicalRelationshipsMigration,
|
||||
"009_ai_token_usage": aiTokenUsageMigration,
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
import { type Kysely, sql } from "kysely";
|
||||
import type { CatalogDatabase } from "../repository.js";
|
||||
|
||||
export async function up(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||
await db.schema.createTable("catalog_logical_relationships")
|
||||
.addColumn("id", "uuid", (column) => column.primaryKey())
|
||||
.addColumn("database_id", "uuid", (column) => column.notNull()
|
||||
.references("workspace_databases.id").onDelete("cascade"))
|
||||
.addColumn("source_column_id", "uuid", (column) => column.notNull()
|
||||
.references("catalog_columns.id").onDelete("cascade"))
|
||||
.addColumn("target_column_id", "uuid", (column) => column.notNull()
|
||||
.references("catalog_columns.id").onDelete("cascade"))
|
||||
.addColumn("generated", "boolean", (column) => column.notNull().defaultTo(false))
|
||||
.addColumn("deleted_at", "timestamptz")
|
||||
.addColumn("created_at", "timestamptz", (column) => column.notNull().defaultTo(sql`now()`))
|
||||
.addColumn("updated_at", "timestamptz", (column) => column.notNull().defaultTo(sql`now()`))
|
||||
.addUniqueConstraint(
|
||||
"catalog_logical_relationships_endpoint_key",
|
||||
["database_id", "source_column_id", "target_column_id"],
|
||||
)
|
||||
.addCheckConstraint(
|
||||
"catalog_logical_relationships_distinct_columns_check",
|
||||
sql`source_column_id <> target_column_id`,
|
||||
)
|
||||
.execute();
|
||||
|
||||
await db.schema.createIndex("catalog_logical_relationships_database_deleted_idx")
|
||||
.on("catalog_logical_relationships")
|
||||
.columns(["database_id", "deleted_at"])
|
||||
.execute();
|
||||
}
|
||||
|
||||
export async function down(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||
await db.schema.dropTable("catalog_logical_relationships").execute();
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
import type { Kysely } from "kysely";
|
||||
import type { CatalogDatabase } from "../repository.js";
|
||||
|
||||
export async function up(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||
for (const table of ["description_generation_runs", "sensitive_data_suggestion_runs"] as const) {
|
||||
await db.schema.alterTable(table)
|
||||
.addColumn("input_tokens", "integer", (col) => col.notNull().defaultTo(0))
|
||||
.addColumn("cache_read_tokens", "integer", (col) => col.notNull().defaultTo(0))
|
||||
.addColumn("output_tokens", "integer", (col) => col.notNull().defaultTo(0))
|
||||
.execute();
|
||||
}
|
||||
}
|
||||
|
||||
export async function down(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||
for (const table of ["sensitive_data_suggestion_runs", "description_generation_runs"] as const) {
|
||||
await db.schema.alterTable(table)
|
||||
.dropColumn("input_tokens").dropColumn("cache_read_tokens").dropColumn("output_tokens")
|
||||
.execute();
|
||||
}
|
||||
}
|
||||
@@ -4,7 +4,7 @@ import type { ResolvedMetadataGenerationModel } from "./metadata-generation-mode
|
||||
|
||||
const MAX_HELPER_OUTPUT_BYTES = 64 * 1024;
|
||||
const helperOutputSchema = z.discriminatedUnion("ok", [
|
||||
z.object({ ok: z.literal(true), content: z.string(), usage: z.object({ input: z.number().int().nonnegative(), cacheRead: z.number().int().nonnegative(), output: z.number().int().nonnegative() }).strict().optional() }).strict(),
|
||||
z.object({ ok: z.literal(true), content: z.string() }).strict(),
|
||||
z.object({ ok: z.literal(false), error: z.literal("provider_failure") }).strict(),
|
||||
]);
|
||||
|
||||
@@ -18,12 +18,10 @@ export interface ModelCompletionRequest {
|
||||
messages: readonly ModelCompletionMessage[];
|
||||
signal: AbortSignal;
|
||||
}
|
||||
export interface ModelCompletionUsage { input: number; cacheRead: number; output: number; }
|
||||
export interface ModelCompletionResult { content: string; usage: ModelCompletionUsage; }
|
||||
|
||||
/** The provider boundary used by Description Generation. */
|
||||
export interface ModelCompleter {
|
||||
complete(request: ModelCompletionRequest): Promise<string | ModelCompletionResult>;
|
||||
complete(request: ModelCompletionRequest): Promise<string>;
|
||||
}
|
||||
|
||||
export class ModelCompletionProviderError extends Error {
|
||||
@@ -49,7 +47,7 @@ export class PythonModelCompleter implements ModelCompleter {
|
||||
terminationGraceMs?: number;
|
||||
}) {}
|
||||
|
||||
async complete(request: ModelCompletionRequest): Promise<ModelCompletionResult> {
|
||||
async complete(request: ModelCompletionRequest): Promise<string> {
|
||||
if (request.signal.aborted) throw new ModelCompletionCancelledError();
|
||||
const payload = {
|
||||
model: `${request.model.provider}/${request.model.model}`,
|
||||
@@ -64,7 +62,7 @@ export class PythonModelCompleter implements ModelCompleter {
|
||||
...(request.model.disableThinking === true ? { disable_thinking: true } : {}),
|
||||
};
|
||||
|
||||
return await new Promise<ModelCompletionResult>((resolve, reject) => {
|
||||
return await new Promise<string>((resolve, reject) => {
|
||||
const child = spawn(
|
||||
this.options.pythonExecutable,
|
||||
["-m", this.options.helperModule ?? "tht.internal.litellm_completion"],
|
||||
@@ -137,7 +135,7 @@ export class PythonModelCompleter implements ModelCompleter {
|
||||
if (!output.ok) return fail();
|
||||
settled = true;
|
||||
cleanup();
|
||||
resolve({ content: output.content, usage: output.usage ?? { input: 0, cacheRead: 0, output: 0 } });
|
||||
resolve(output.content);
|
||||
} catch {
|
||||
fail();
|
||||
}
|
||||
|
||||
@@ -23,10 +23,7 @@ import {
|
||||
type CatalogDatabaseMetadataDeleteTarget,
|
||||
type CatalogMetadataDeleteCounts,
|
||||
type CatalogMetrics,
|
||||
type CatalogLogicalRelationship,
|
||||
type CatalogLogicalRelationshipCandidate,
|
||||
type CatalogLogicalRelationshipContext,
|
||||
type CatalogPhysicalRelationship,
|
||||
type CatalogRelationship,
|
||||
type CatalogSchemaDiff,
|
||||
type CatalogSyncCounts,
|
||||
type CatalogSyncEvent,
|
||||
@@ -148,17 +145,6 @@ interface CatalogRelationshipColumnTable {
|
||||
targetColumnId: string;
|
||||
}
|
||||
|
||||
interface CatalogLogicalRelationshipTable {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
sourceColumnId: string;
|
||||
targetColumnId: string;
|
||||
generated: Generated<boolean>;
|
||||
deletedAt: Timestamp | null;
|
||||
createdAt: Timestamp;
|
||||
updatedAt: Timestamp;
|
||||
}
|
||||
|
||||
interface DescriptionGenerationRunTable {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
@@ -171,9 +157,6 @@ interface DescriptionGenerationRunTable {
|
||||
generated: number;
|
||||
nonGeneratable: number;
|
||||
failed: number;
|
||||
inputTokens: number;
|
||||
cacheReadTokens: number;
|
||||
outputTokens: number;
|
||||
createdAt: Timestamp;
|
||||
startedAt: Timestamp | null;
|
||||
updatedAt: Timestamp;
|
||||
@@ -198,9 +181,6 @@ interface SensitiveDataSuggestionRunTable {
|
||||
total: number;
|
||||
suggestedSensitive: number;
|
||||
suggestedNonSensitive: number;
|
||||
inputTokens: number;
|
||||
cacheReadTokens: number;
|
||||
outputTokens: number;
|
||||
createdAt: Timestamp;
|
||||
startedAt: Timestamp;
|
||||
updatedAt: Timestamp;
|
||||
@@ -261,7 +241,6 @@ export interface CatalogDatabase {
|
||||
catalogColumns: CatalogColumnTable;
|
||||
catalogRelationships: CatalogRelationshipTable;
|
||||
catalogRelationshipColumns: CatalogRelationshipColumnTable;
|
||||
catalogLogicalRelationships: CatalogLogicalRelationshipTable;
|
||||
descriptionGenerationRuns: DescriptionGenerationRunTable;
|
||||
descriptionGenerationEvents: DescriptionGenerationEventTable;
|
||||
sensitiveDataSuggestionRuns: SensitiveDataSuggestionRunTable;
|
||||
@@ -822,9 +801,6 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
generated: 0,
|
||||
nonGeneratable: 0,
|
||||
failed: 0,
|
||||
inputTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
outputTokens: 0,
|
||||
startedAt: null,
|
||||
finishedAt: null,
|
||||
errorSummary: null,
|
||||
@@ -952,9 +928,6 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
total: 0,
|
||||
suggestedSensitive: 0,
|
||||
suggestedNonSensitive: 0,
|
||||
inputTokens: 0,
|
||||
cacheReadTokens: 0,
|
||||
outputTokens: 0,
|
||||
finishedAt: null,
|
||||
errorSummary: null,
|
||||
}).returningAll().executeTakeFirstOrThrow();
|
||||
@@ -1049,7 +1022,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
return rows.map(serializeSensitiveDataSuggestionEvent);
|
||||
}
|
||||
|
||||
async listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]> {
|
||||
async listRelationships(databaseId: string): Promise<CatalogRelationship[]> {
|
||||
const rows = await this.db.selectFrom("catalogRelationships as relationship")
|
||||
.innerJoin("catalogTables as sourceTable", "sourceTable.id", "relationship.sourceTableId")
|
||||
.innerJoin("catalogTables as targetTable", "targetTable.id", "relationship.targetTableId")
|
||||
@@ -1076,7 +1049,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
.where("pair.relationshipId", "in", rows.map((row) => row.id))
|
||||
.orderBy("pair.relationshipId").orderBy("pair.position")
|
||||
.execute();
|
||||
const byRelationship = new Map<string, CatalogPhysicalRelationship["columns"]>();
|
||||
const byRelationship = new Map<string, CatalogRelationship["columns"]>();
|
||||
for (const pair of pairs) {
|
||||
const items = byRelationship.get(pair.relationshipId) ?? [];
|
||||
items.push(pair);
|
||||
@@ -1088,160 +1061,9 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
lastSyncedAt: row.lastSyncedAt === null ? null : new Date(row.lastSyncedAt).toISOString(),
|
||||
createdAt: new Date(row.createdAt).toISOString(),
|
||||
updatedAt: new Date(row.updatedAt).toISOString(),
|
||||
origin: "physical" as const,
|
||||
status: "active" as const,
|
||||
}));
|
||||
}
|
||||
|
||||
async listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]> {
|
||||
const rows = await this.db.selectFrom("catalogLogicalRelationships as relationship")
|
||||
.innerJoin("catalogColumns as sourceColumn", "sourceColumn.id", "relationship.sourceColumnId")
|
||||
.innerJoin("catalogTables as sourceTable", "sourceTable.id", "sourceColumn.tableId")
|
||||
.innerJoin("catalogColumns as targetColumn", "targetColumn.id", "relationship.targetColumnId")
|
||||
.innerJoin("catalogTables as targetTable", "targetTable.id", "targetColumn.tableId")
|
||||
.select([
|
||||
"relationship.id", "relationship.databaseId", "relationship.generated",
|
||||
"relationship.deletedAt", "relationship.createdAt", "relationship.updatedAt",
|
||||
"sourceTable.id as sourceTableId", "sourceTable.name as sourceTableName",
|
||||
"sourceColumn.id as sourceColumnId", "sourceColumn.name as sourceColumnName",
|
||||
"targetTable.id as targetTableId", "targetTable.name as targetTableName",
|
||||
"targetColumn.id as targetColumnId", "targetColumn.name as targetColumnName",
|
||||
])
|
||||
.where("relationship.databaseId", "=", databaseId)
|
||||
.orderBy("sourceTable.name")
|
||||
.orderBy("sourceColumn.name")
|
||||
.orderBy("targetTable.name")
|
||||
.orderBy("targetColumn.name")
|
||||
.execute();
|
||||
return rows.map((row) => ({
|
||||
id: row.id,
|
||||
databaseId: row.databaseId,
|
||||
constraintName: null,
|
||||
sourceTableId: row.sourceTableId,
|
||||
sourceTableName: row.sourceTableName,
|
||||
targetTableId: row.targetTableId,
|
||||
targetTableName: row.targetTableName,
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId: row.sourceColumnId,
|
||||
sourceColumnName: row.sourceColumnName,
|
||||
targetColumnId: row.targetColumnId,
|
||||
targetColumnName: row.targetColumnName,
|
||||
}],
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
createdAt: new Date(row.createdAt).toISOString(),
|
||||
updatedAt: new Date(row.updatedAt).toISOString(),
|
||||
origin: row.generated ? "generated" : "manual",
|
||||
status: row.deletedAt === null ? "active" : "excluded",
|
||||
}));
|
||||
}
|
||||
|
||||
async getLogicalRelationshipContext(
|
||||
databaseId: string,
|
||||
): Promise<CatalogLogicalRelationshipContext | undefined> {
|
||||
if (!(await selectOne(this.db, databaseId))) return undefined;
|
||||
const columns = await this.db.selectFrom("catalogColumns as column")
|
||||
.innerJoin("catalogTables as table", "table.id", "column.tableId")
|
||||
.select([
|
||||
"column.id as columnId", "column.name as columnName", "column.dataType",
|
||||
"column.primaryKeyPosition", "table.id as tableId", "table.name as tableName",
|
||||
])
|
||||
.where("table.databaseId", "=", databaseId)
|
||||
.orderBy("table.name")
|
||||
.orderBy("column.ordinalPosition")
|
||||
.execute();
|
||||
const primaryKeyCounts = new Map<string, number>();
|
||||
for (const column of columns) {
|
||||
if (column.primaryKeyPosition !== null) {
|
||||
primaryKeyCounts.set(column.tableId, (primaryKeyCounts.get(column.tableId) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
const physicalPairs = await this.db.selectFrom("catalogRelationshipColumns as pair")
|
||||
.innerJoin("catalogRelationships as relationship", "relationship.id", "pair.relationshipId")
|
||||
.select(["pair.sourceColumnId", "pair.targetColumnId"])
|
||||
.where("relationship.databaseId", "=", databaseId)
|
||||
.execute();
|
||||
return {
|
||||
endpoints: columns.map((column) => ({
|
||||
...column,
|
||||
tablePrimaryKeyColumnCount: primaryKeyCounts.get(column.tableId) ?? 0,
|
||||
})),
|
||||
physicalPairs,
|
||||
logicalRelationships: await this.listLogicalRelationships(databaseId),
|
||||
};
|
||||
}
|
||||
|
||||
async insertLogicalRelationship(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
generated: boolean,
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
const id = randomUUID();
|
||||
const inserted = await this.db.insertInto("catalogLogicalRelationships").values({
|
||||
id, databaseId, sourceColumnId, targetColumnId, generated,
|
||||
}).onConflict((conflict) => conflict
|
||||
.columns(["databaseId", "sourceColumnId", "targetColumnId"])
|
||||
.doNothing())
|
||||
.returning("id")
|
||||
.executeTakeFirst();
|
||||
if (!inserted) return undefined;
|
||||
return (await this.listLogicalRelationships(databaseId)).find((item) => item.id === id);
|
||||
}
|
||||
|
||||
async insertGeneratedLogicalRelationships(
|
||||
databaseId: string,
|
||||
candidates: readonly CatalogLogicalRelationshipCandidate[],
|
||||
): Promise<number> {
|
||||
if (candidates.length === 0) return 0;
|
||||
return await this.db.transaction().execute(async (trx) => {
|
||||
let added = 0;
|
||||
for (const candidate of candidates) {
|
||||
const inserted = await trx.insertInto("catalogLogicalRelationships").values({
|
||||
id: randomUUID(),
|
||||
databaseId,
|
||||
sourceColumnId: candidate.sourceColumnId,
|
||||
targetColumnId: candidate.targetColumnId,
|
||||
generated: true,
|
||||
}).onConflict((conflict) => conflict
|
||||
.columns(["databaseId", "sourceColumnId", "targetColumnId"])
|
||||
.doNothing())
|
||||
.returning("id")
|
||||
.executeTakeFirst();
|
||||
if (inserted) added += 1;
|
||||
}
|
||||
return added;
|
||||
});
|
||||
}
|
||||
|
||||
async setLogicalRelationshipStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
const updated = await this.db.updateTable("catalogLogicalRelationships")
|
||||
.set({ deletedAt: status === "excluded" ? sql`now()` : null, updatedAt: sql`now()` })
|
||||
.where("databaseId", "=", databaseId)
|
||||
.where("id", "=", relationshipId)
|
||||
.returning("id")
|
||||
.executeTakeFirst();
|
||||
if (!updated) return undefined;
|
||||
return (await this.listLogicalRelationships(databaseId)).find((item) => item.id === relationshipId);
|
||||
}
|
||||
|
||||
async deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean> {
|
||||
const result = await this.db.deleteFrom("catalogLogicalRelationships")
|
||||
.where("databaseId", "=", databaseId)
|
||||
.where("id", "=", relationshipId)
|
||||
.executeTakeFirst();
|
||||
return result.numDeletedRows > 0n;
|
||||
}
|
||||
|
||||
async deleteDatabaseMetadata(
|
||||
databaseIds: readonly string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
@@ -1799,13 +1621,7 @@ export class UnavailableCatalogRepository implements CatalogRepository {
|
||||
async updateSensitiveDataSuggestionRun(): Promise<SensitiveDataSuggestionRun | undefined> { return this.fail(); }
|
||||
async appendSensitiveDataSuggestionEvent(): Promise<SensitiveDataSuggestionEvent> { return this.fail(); }
|
||||
async listSensitiveDataSuggestionEvents(): Promise<SensitiveDataSuggestionEvent[]> { return this.fail(); }
|
||||
async listRelationships(): Promise<CatalogPhysicalRelationship[]> { return this.fail(); }
|
||||
async listLogicalRelationships(): Promise<CatalogLogicalRelationship[]> { return this.fail(); }
|
||||
async getLogicalRelationshipContext(): Promise<CatalogLogicalRelationshipContext | undefined> { return this.fail(); }
|
||||
async insertLogicalRelationship(): Promise<CatalogLogicalRelationship | undefined> { return this.fail(); }
|
||||
async insertGeneratedLogicalRelationships(): Promise<number> { return this.fail(); }
|
||||
async setLogicalRelationshipStatus(): Promise<CatalogLogicalRelationship | undefined> { return this.fail(); }
|
||||
async deleteLogicalRelationship(): Promise<boolean> { return this.fail(); }
|
||||
async listRelationships(): Promise<CatalogRelationship[]> { return this.fail(); }
|
||||
async deleteDatabaseMetadata(): Promise<CatalogMetadataDeleteCounts | undefined> { return this.fail(); }
|
||||
async deleteTableMetadata(): Promise<CatalogMetadataDeleteCounts | undefined> { return this.fail(); }
|
||||
async planSchemaSync(): Promise<CatalogSchemaDiff> { return this.fail(); }
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { z } from "zod";
|
||||
import type { MetadataGenerationModels } from "./metadata-generation-models.js";
|
||||
import type { ModelCompleter, ModelCompletionMessage, ModelCompletionResult, ModelCompletionUsage } from "./model-completer.js";
|
||||
import type { ModelCompleter, ModelCompletionMessage } from "./model-completer.js";
|
||||
import type {
|
||||
CatalogColumn,
|
||||
CatalogRepository,
|
||||
@@ -201,8 +201,6 @@ export class SensitiveDataSuggester {
|
||||
targetIds: readonly string[],
|
||||
signal: AbortSignal,
|
||||
onPrepared?: (total: number) => void | Promise<void>,
|
||||
onProgress?: (processed: number, suggestions: readonly SensitiveDataSuggestion[]) => void | Promise<void>,
|
||||
onUsage?: (usage: ModelCompletionUsage) => void | Promise<void>,
|
||||
): Promise<readonly SensitiveDataSuggestion[]> {
|
||||
const database = await this.repository.get(databaseId);
|
||||
if (!database) throw new SensitiveDataSuggestionTargetNotFoundError("database");
|
||||
@@ -214,16 +212,11 @@ export class SensitiveDataSuggester {
|
||||
for (const batch of batchesFor(database, columns)) {
|
||||
let received: Map<string, { columnId: string; sensitive: boolean }> | undefined;
|
||||
for (let attempt = 0; attempt < 2 && !received; attempt += 1) {
|
||||
const completion = await this.completer.complete({
|
||||
const content = await this.completer.complete({
|
||||
model,
|
||||
signal,
|
||||
messages: [systemMessage, { role: "user", content: userContent(database, batch) }],
|
||||
});
|
||||
const result: ModelCompletionResult = typeof completion === "string"
|
||||
? { content: completion, usage: { input: 0, cacheRead: 0, output: 0 } }
|
||||
: completion;
|
||||
await onUsage?.(result.usage);
|
||||
const content = result.content;
|
||||
try {
|
||||
const parsed = responseSchema.parse(JSON.parse(content));
|
||||
const expected = new Set(batch.map((column) => column.columnId));
|
||||
@@ -247,7 +240,6 @@ export class SensitiveDataSuggester {
|
||||
currentSensitive: column.currentSensitive,
|
||||
sensitive: received!.get(column.columnId)!.sensitive,
|
||||
})));
|
||||
await onProgress?.(suggestions.length, suggestions.slice(-batch.length));
|
||||
}
|
||||
return suggestions;
|
||||
}
|
||||
|
||||
@@ -10,7 +10,6 @@ import type {
|
||||
SensitiveDataSuggestionRun,
|
||||
SensitiveDataSuggestionScope,
|
||||
} from "./types.js";
|
||||
import type { ModelCompletionUsage } from "./model-completer.js";
|
||||
|
||||
const interruptedMessage = "Sensitive-field suggestion generation was interrupted by backend restart.";
|
||||
const failedMessage = "Sensitive-field suggestion generation failed.";
|
||||
@@ -74,31 +73,6 @@ export class SensitiveDataSuggestionRunner {
|
||||
});
|
||||
if (!prepared) throw new Error("Sensitive Data Suggestion Run disappeared");
|
||||
},
|
||||
async (processed, batch) => {
|
||||
const suggestedSensitive = batch.filter((suggestion) => suggestion.sensitive).length;
|
||||
const suggestedNonSensitive = batch.length - suggestedSensitive;
|
||||
const current = await this.repository.getSensitiveDataSuggestionRun(started.id);
|
||||
if (!current) throw new Error("Sensitive Data Suggestion Run disappeared");
|
||||
const progress = await this.repository.updateSensitiveDataSuggestionRun(started.id, {
|
||||
suggestedSensitive: current.suggestedSensitive + suggestedSensitive,
|
||||
suggestedNonSensitive: current.suggestedNonSensitive + suggestedNonSensitive,
|
||||
});
|
||||
if (!progress) throw new Error("Sensitive Data Suggestion Run disappeared");
|
||||
await this.repository.appendSensitiveDataSuggestionEvent(
|
||||
started.id,
|
||||
"info",
|
||||
`Classified ${processed} of ${progress.total} columns.`,
|
||||
);
|
||||
},
|
||||
async (usage: ModelCompletionUsage) => {
|
||||
const current = await this.repository.getSensitiveDataSuggestionRun(started.id);
|
||||
if (!current) throw new Error("Sensitive Data Suggestion Run disappeared");
|
||||
await this.repository.updateSensitiveDataSuggestionRun(started.id, {
|
||||
inputTokens: current.inputTokens + usage.input,
|
||||
cacheReadTokens: current.cacheReadTokens + usage.cacheRead,
|
||||
outputTokens: current.outputTokens + usage.output,
|
||||
});
|
||||
},
|
||||
);
|
||||
const suggestedSensitive = suggestions.filter((suggestion) => suggestion.sensitive).length;
|
||||
const suggestedNonSensitive = suggestions.length - suggestedSensitive;
|
||||
|
||||
@@ -2,7 +2,6 @@ import { buildInstallationContract } from "../workspaces/contracts.js";
|
||||
import { resolveBinding } from "../workspaces/bindings.js";
|
||||
import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||
import type { WorkspaceDescriptor } from "../workspaces/schema.js";
|
||||
import { discoverWorkspaceSecretRequirements } from "../workspaces/secret-requirements.js";
|
||||
import type { WorkspaceSecretStore } from "../workspaces/secret-store.js";
|
||||
import { createConcreteDiagnosticAdapters } from "../workspaces/diagnostics.js";
|
||||
import { CatalogOperationCoordinator } from "./operation-coordinator.js";
|
||||
@@ -26,21 +25,6 @@ export interface CatalogListItem extends Omit<WorkspaceDatabase, "id"> {
|
||||
workspaceName: string;
|
||||
workspaceDescription?: string;
|
||||
workspaceAvailable: boolean;
|
||||
workspaceRevision: { commit: string; blob: string } | null;
|
||||
workspaceEvidence: {
|
||||
sourceType: "filesystem" | "http" | "s3" | null;
|
||||
state:
|
||||
| "not_declared"
|
||||
| "materialized_current_revision"
|
||||
| "configuration_required"
|
||||
| "configured_unverified"
|
||||
| "workspace_unavailable";
|
||||
};
|
||||
runtimeBinding: {
|
||||
transport: DatabaseBinding["transport"];
|
||||
configurationState: "ready" | "configuration_required";
|
||||
sessionTransportSupported: boolean;
|
||||
} | null;
|
||||
configured: boolean;
|
||||
secrets: Record<CatalogSecretName, boolean>;
|
||||
}
|
||||
@@ -84,43 +68,6 @@ function secretState(store: WorkspaceSecretStore, workspaceId: string): Record<C
|
||||
))) as Record<CatalogSecretName, boolean>;
|
||||
}
|
||||
|
||||
function workspaceRuntimeState(
|
||||
store: WorkspaceSecretStore,
|
||||
workspace: WorkspaceDescriptor,
|
||||
secretRoots: readonly string[],
|
||||
): NonNullable<CatalogListItem["runtimeBinding"]> {
|
||||
const requirements = discoverWorkspaceSecretRequirements(workspace, process.env);
|
||||
const secretVariables = new Set(requirements.map(({ variable }) => variable));
|
||||
const effective = resolveBinding(workspace, "DWH", process.env, secretRoots);
|
||||
const configurationRequired = requirements.some(({ id, required }) => (
|
||||
required && !store.has(workspace.workspace.id, id)
|
||||
)) || effective.missing.some((variable) => !secretVariables.has(variable));
|
||||
return {
|
||||
transport: effective.transport,
|
||||
configurationState: configurationRequired ? "configuration_required" : "ready",
|
||||
sessionTransportSupported: effective.transport !== "ssh_tunnel",
|
||||
};
|
||||
}
|
||||
|
||||
function workspaceEvidenceState(
|
||||
store: WorkspaceSecretStore,
|
||||
workspace: WorkspaceDescriptor,
|
||||
): CatalogListItem["workspaceEvidence"] {
|
||||
const source = workspace.evidence?.source;
|
||||
if (!source) return { sourceType: null, state: "not_declared" };
|
||||
if (source.type === "filesystem") {
|
||||
return { sourceType: source.type, state: "materialized_current_revision" };
|
||||
}
|
||||
const configurationRequired = discoverWorkspaceSecretRequirements(workspace, process.env)
|
||||
.some(({ connector, id, required }) => (
|
||||
connector === "evidence" && required && !store.has(workspace.workspace.id, id)
|
||||
));
|
||||
return {
|
||||
sourceType: source.type,
|
||||
state: configurationRequired ? "configuration_required" : "configured_unverified",
|
||||
};
|
||||
}
|
||||
|
||||
export class CatalogService {
|
||||
private readonly adapters = createConcreteDiagnosticAdapters();
|
||||
|
||||
@@ -144,8 +91,7 @@ export class CatalogService {
|
||||
const byWorkspace = new Map(configured.map((database) => [database.workspaceId, database]));
|
||||
const active = await Promise.all(workspaces.map(async (entry) => {
|
||||
const database = byWorkspace.get(entry.id);
|
||||
const { workspace } = await this.registry.readPinned(entry.id, entry.revision.commit);
|
||||
const runtimeDatabaseBinding = yamlBinding(workspace, this.secretRoots);
|
||||
const { workspace } = await this.registry.read(entry.id);
|
||||
const base = database ?? {
|
||||
workspaceId: entry.id,
|
||||
engine: "postgres" as const,
|
||||
@@ -154,7 +100,7 @@ export class CatalogService {
|
||||
version: 0,
|
||||
createdAt: "",
|
||||
updatedAt: "",
|
||||
binding: runtimeDatabaseBinding,
|
||||
binding: yamlBinding(workspace, this.secretRoots),
|
||||
connectionStatus: "untested" as const,
|
||||
};
|
||||
return {
|
||||
@@ -162,12 +108,6 @@ export class CatalogService {
|
||||
workspaceName: entry.name,
|
||||
workspaceDescription: entry.description,
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: {
|
||||
commit: entry.revision.commit,
|
||||
blob: entry.revision.blob,
|
||||
},
|
||||
workspaceEvidence: workspaceEvidenceState(this.secretStore, workspace),
|
||||
runtimeBinding: workspaceRuntimeState(this.secretStore, workspace, this.secretRoots),
|
||||
configured: database !== undefined,
|
||||
secrets: secretState(this.secretStore, entry.id),
|
||||
};
|
||||
@@ -180,9 +120,6 @@ export class CatalogService {
|
||||
workspaceName: database.workspaceId,
|
||||
workspaceDescription: "Workspace is no longer present in the repository catalog.",
|
||||
workspaceAvailable: false,
|
||||
workspaceRevision: null,
|
||||
workspaceEvidence: { sourceType: null, state: "workspace_unavailable" },
|
||||
runtimeBinding: null,
|
||||
configured: true,
|
||||
secrets: secretState(this.secretStore, database.workspaceId),
|
||||
}));
|
||||
|
||||
@@ -128,7 +128,7 @@ export interface CatalogRelationshipColumn {
|
||||
targetColumnName: string;
|
||||
}
|
||||
|
||||
export interface CatalogPhysicalRelationship {
|
||||
export interface CatalogRelationship {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
constraintName: string;
|
||||
@@ -145,52 +145,6 @@ export interface CatalogPhysicalRelationship {
|
||||
lastSyncedAt: string | null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
origin: "physical";
|
||||
status: "active";
|
||||
}
|
||||
|
||||
export interface CatalogLogicalRelationship {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
constraintName: null;
|
||||
sourceTableId: string;
|
||||
sourceTableName: string;
|
||||
targetTableId: string;
|
||||
targetTableName: string;
|
||||
updateRule: null;
|
||||
deleteRule: null;
|
||||
deferrable: false;
|
||||
initiallyDeferred: false;
|
||||
columns: [CatalogRelationshipColumn];
|
||||
lastSyncedDatabaseVersion: null;
|
||||
lastSyncedAt: null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
origin: "generated" | "manual";
|
||||
status: "active" | "excluded";
|
||||
}
|
||||
|
||||
export type CatalogRelationship = CatalogPhysicalRelationship | CatalogLogicalRelationship;
|
||||
|
||||
export interface CatalogLogicalRelationshipEndpoint {
|
||||
columnId: string;
|
||||
columnName: string;
|
||||
tableId: string;
|
||||
tableName: string;
|
||||
dataType: string;
|
||||
primaryKeyPosition: number | null;
|
||||
tablePrimaryKeyColumnCount: number;
|
||||
}
|
||||
|
||||
export interface CatalogLogicalRelationshipContext {
|
||||
endpoints: CatalogLogicalRelationshipEndpoint[];
|
||||
physicalPairs: Array<{ sourceColumnId: string; targetColumnId: string }>;
|
||||
logicalRelationships: CatalogLogicalRelationship[];
|
||||
}
|
||||
|
||||
export interface CatalogLogicalRelationshipCandidate {
|
||||
sourceColumnId: string;
|
||||
targetColumnId: string;
|
||||
}
|
||||
|
||||
export type CatalogDatabaseMetadataDeleteTarget = "tables" | "relationships";
|
||||
@@ -234,9 +188,6 @@ export interface DescriptionGenerationRun {
|
||||
generated: number;
|
||||
nonGeneratable: number;
|
||||
failed: number;
|
||||
inputTokens: number;
|
||||
cacheReadTokens: number;
|
||||
outputTokens: number;
|
||||
createdAt: string;
|
||||
startedAt: string | null;
|
||||
updatedAt: string;
|
||||
@@ -253,9 +204,6 @@ export interface DescriptionGenerationRunUpdate {
|
||||
startedAt?: string | null;
|
||||
finishedAt?: string | null;
|
||||
errorSummary?: string | null;
|
||||
inputTokens?: number;
|
||||
cacheReadTokens?: number;
|
||||
outputTokens?: number;
|
||||
}
|
||||
|
||||
export interface DescriptionGenerationEvent {
|
||||
@@ -278,9 +226,6 @@ export interface SensitiveDataSuggestionRun {
|
||||
total: number;
|
||||
suggestedSensitive: number;
|
||||
suggestedNonSensitive: number;
|
||||
inputTokens: number;
|
||||
cacheReadTokens: number;
|
||||
outputTokens: number;
|
||||
createdAt: string;
|
||||
startedAt: string;
|
||||
updatedAt: string;
|
||||
@@ -295,9 +240,6 @@ export interface SensitiveDataSuggestionRunUpdate {
|
||||
suggestedNonSensitive?: number;
|
||||
finishedAt?: string | null;
|
||||
errorSummary?: string | null;
|
||||
inputTokens?: number;
|
||||
cacheReadTokens?: number;
|
||||
outputTokens?: number;
|
||||
}
|
||||
|
||||
export interface SensitiveDataSuggestionEvent {
|
||||
@@ -514,25 +456,7 @@ export interface CatalogRepository {
|
||||
runId: string,
|
||||
afterSequence?: number,
|
||||
): Promise<SensitiveDataSuggestionEvent[]>;
|
||||
listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]>;
|
||||
listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]>;
|
||||
getLogicalRelationshipContext(databaseId: string): Promise<CatalogLogicalRelationshipContext | undefined>;
|
||||
insertLogicalRelationship(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
generated: boolean,
|
||||
): Promise<CatalogLogicalRelationship | undefined>;
|
||||
insertGeneratedLogicalRelationships(
|
||||
databaseId: string,
|
||||
candidates: readonly CatalogLogicalRelationshipCandidate[],
|
||||
): Promise<number>;
|
||||
setLogicalRelationshipStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship | undefined>;
|
||||
deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean>;
|
||||
listRelationships(databaseId: string): Promise<CatalogRelationship[]>;
|
||||
deleteDatabaseMetadata(
|
||||
databaseIds: readonly string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
|
||||
@@ -101,9 +101,6 @@ function publicRun(run: DescriptionGenerationRun) {
|
||||
generated: run.generated,
|
||||
nonGeneratable: run.nonGeneratable,
|
||||
failed: run.failed,
|
||||
inputTokens: run.inputTokens,
|
||||
cacheReadTokens: run.cacheReadTokens,
|
||||
outputTokens: run.outputTokens,
|
||||
createdAt: run.createdAt,
|
||||
startedAt: run.startedAt,
|
||||
updatedAt: run.updatedAt,
|
||||
@@ -132,9 +129,6 @@ function publicSensitiveDataSuggestionRun(run: SensitiveDataSuggestionRun) {
|
||||
total: run.total,
|
||||
suggestedSensitive: run.suggestedSensitive,
|
||||
suggestedNonSensitive: run.suggestedNonSensitive,
|
||||
inputTokens: run.inputTokens,
|
||||
cacheReadTokens: run.cacheReadTokens,
|
||||
outputTokens: run.outputTokens,
|
||||
createdAt: run.createdAt,
|
||||
startedAt: run.startedAt,
|
||||
updatedAt: run.updatedAt,
|
||||
|
||||
@@ -1,154 +0,0 @@
|
||||
import type { FastifyInstance, FastifyReply, FastifyRequest } from "fastify";
|
||||
import { z } from "zod";
|
||||
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||
import {
|
||||
CatalogLogicalRelationshipService,
|
||||
LogicalRelationshipColumnNotFoundError,
|
||||
LogicalRelationshipDatabaseNotFoundError,
|
||||
LogicalRelationshipDuplicateError,
|
||||
LogicalRelationshipNotFoundError,
|
||||
LogicalRelationshipReadOnlyError,
|
||||
LogicalRelationshipSchemaStaleError,
|
||||
LogicalRelationshipTargetNotUniqueError,
|
||||
LogicalRelationshipTypeIncompatibleError,
|
||||
} from "../catalog/logical-relationship-service.js";
|
||||
import type { CatalogOperationCoordinator } from "../catalog/operation-coordinator.js";
|
||||
import { CatalogOperationInProgressError, CatalogUnavailableError } from "../catalog/types.js";
|
||||
|
||||
const idSchema = z.uuid();
|
||||
const createSchema = z.object({
|
||||
sourceColumnId: idSchema,
|
||||
targetColumnId: idSchema,
|
||||
}).strict();
|
||||
const statusSchema = z.object({ status: z.enum(["active", "excluded"]) }).strict();
|
||||
const emptySchema = z.object({}).strict();
|
||||
|
||||
function manage(request: FastifyRequest, reply: FastifyReply) {
|
||||
return isPrincipalContext(requirePermission(request, reply, "database.manage"));
|
||||
}
|
||||
|
||||
function safeError(reply: FastifyReply, error: unknown) {
|
||||
if (error instanceof CatalogUnavailableError) {
|
||||
return reply.code(503).send({ code: "catalog_unavailable", message: "Database catalog is unavailable." });
|
||||
}
|
||||
if (error instanceof CatalogOperationInProgressError) {
|
||||
return reply.code(409).send({
|
||||
code: "database_operation_in_progress",
|
||||
message: "A database operation is already in progress.",
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipDatabaseNotFoundError) {
|
||||
return reply.code(404).send({ code: "database_not_found", message: "Database configuration was not found." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipColumnNotFoundError) {
|
||||
return reply.code(404).send({
|
||||
code: "column_not_found",
|
||||
message: "Catalog column was not found.",
|
||||
field: error.field,
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipNotFoundError) {
|
||||
return reply.code(404).send({ code: "relationship_not_found", message: "Relationship was not found." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipDuplicateError) {
|
||||
return reply.code(409).send({ code: "relationship_duplicate", message: "Relationship already exists." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipReadOnlyError) {
|
||||
return reply.code(409).send({ code: "relationship_read_only", message: "Physical relationships are read-only." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipSchemaStaleError) {
|
||||
return reply.code(409).send({
|
||||
code: "relationship_schema_stale",
|
||||
message: "Synchronize the current database schema before managing logical relationships.",
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipTargetNotUniqueError) {
|
||||
return reply.code(422).send({
|
||||
code: "relationship_target_not_unique",
|
||||
message: "Target column must be the only primary-key column of its table.",
|
||||
field: "targetColumnId",
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipTypeIncompatibleError) {
|
||||
return reply.code(422).send({
|
||||
code: "relationship_type_incompatible",
|
||||
message: "Source and target column types are incompatible.",
|
||||
field: "targetColumnId",
|
||||
});
|
||||
}
|
||||
if (error instanceof z.ZodError) {
|
||||
return reply.code(400).send({
|
||||
code: "relationship_request_invalid",
|
||||
message: "Relationship request is invalid.",
|
||||
});
|
||||
}
|
||||
return reply.code(500).send({
|
||||
code: "relationship_operation_failed",
|
||||
message: "Relationship operation failed.",
|
||||
});
|
||||
}
|
||||
|
||||
export function catalogLogicalRelationshipRoutes(
|
||||
app: FastifyInstance,
|
||||
deps: {
|
||||
service: CatalogLogicalRelationshipService;
|
||||
operations: CatalogOperationCoordinator;
|
||||
},
|
||||
): void {
|
||||
app.get("/catalog/databases/:databaseId/relationships", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
return await deps.service.list(databaseId);
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.post("/catalog/databases/:databaseId/relationships", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
const input = createSchema.parse(request.body);
|
||||
const relationship = await deps.operations.run(databaseId, async () => (
|
||||
await deps.service.addManual(databaseId, input.sourceColumnId, input.targetColumnId)
|
||||
));
|
||||
return reply.code(201).send(relationship);
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.post("/catalog/databases/:databaseId/relationships/rebuild-generated", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
emptySchema.parse(request.body ?? {});
|
||||
return await deps.operations.run(databaseId, async () => (
|
||||
await deps.service.rebuildGenerated(databaseId)
|
||||
));
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.patch("/catalog/databases/:databaseId/relationships/:relationshipId", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const params = request.params as { databaseId?: unknown; relationshipId?: unknown };
|
||||
const databaseId = idSchema.parse(params.databaseId);
|
||||
const relationshipId = idSchema.parse(params.relationshipId);
|
||||
const input = statusSchema.parse(request.body);
|
||||
return await deps.operations.run(databaseId, async () => (
|
||||
await deps.service.setStatus(databaseId, relationshipId, input.status)
|
||||
));
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.delete("/catalog/databases/:databaseId/relationships/:relationshipId", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const params = request.params as { databaseId?: unknown; relationshipId?: unknown };
|
||||
const databaseId = idSchema.parse(params.databaseId);
|
||||
const relationshipId = idSchema.parse(params.relationshipId);
|
||||
await deps.operations.run(databaseId, async () => {
|
||||
await deps.service.deletePermanently(databaseId, relationshipId);
|
||||
});
|
||||
return reply.code(204).send();
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
}
|
||||
@@ -131,6 +131,17 @@ export function catalogSchemaRoutes(
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.get("/catalog/databases/:databaseId/relationships", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
if (!(await deps.repository.get(databaseId))) {
|
||||
return reply.code(404).send({ code: "database_not_found", message: "Database configuration was not found." });
|
||||
}
|
||||
return await deps.repository.listRelationships(databaseId);
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.post("/catalog/databases/metadata-cleanup", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
|
||||
@@ -11,7 +11,6 @@ import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||
import { validateOperationalWorkspace, type WorkspaceDescriptor } from "../workspaces/schema.js";
|
||||
import type { MaintenanceBarrier } from "../runtime/maintenance-gate.js";
|
||||
import { hasPermission, isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||
import type { EffectiveRelationshipSnapshotProvider } from "../catalog/effective-relationship-snapshot.js";
|
||||
|
||||
const BOOTSTRAP_FAILURE_MESSAGE =
|
||||
"Session startup failed. Check configuration and connectivity, then Resume the session.";
|
||||
@@ -41,8 +40,6 @@ export function sessionRoutes(
|
||||
/** Fail-closed installation/runtime transport capability check. */
|
||||
workspaceRuntimeSupport: (workspace: WorkspaceDescriptor) => boolean;
|
||||
maintenanceBarrier: MaintenanceBarrier;
|
||||
/** Optional only for narrow route-test stubs and installations without a Catalog database. */
|
||||
effectiveRelationships?: EffectiveRelationshipSnapshotProvider;
|
||||
},
|
||||
) {
|
||||
const lifecycleTails = new Map<string, Promise<void>>();
|
||||
@@ -87,21 +84,11 @@ export function sessionRoutes(
|
||||
isAdmin: hasPermission(principal, permission),
|
||||
});
|
||||
|
||||
const optionsWithRuntimeConfig = async (
|
||||
runner: any,
|
||||
workspaceConfigPath: string | undefined,
|
||||
workspaceId: string | undefined,
|
||||
options: any,
|
||||
) => {
|
||||
if (!workspaceConfigPath || typeof runner.acquireWorkspaceRuntime !== "function") return options;
|
||||
const effectiveRelationships = workspaceId && d.effectiveRelationships
|
||||
? await d.effectiveRelationships.render(workspaceId)
|
||||
: undefined;
|
||||
return {
|
||||
...options,
|
||||
runtimeConfig: runner.acquireWorkspaceRuntime(workspaceConfigPath, effectiveRelationships),
|
||||
};
|
||||
};
|
||||
const optionsWithRuntimeConfig = (runner: any, workspaceConfigPath: string | undefined, options: any) => (
|
||||
workspaceConfigPath && typeof runner.acquireWorkspaceRuntime === "function"
|
||||
? { ...options, runtimeConfig: runner.acquireWorkspaceRuntime(workspaceConfigPath) }
|
||||
: options
|
||||
);
|
||||
|
||||
const maintenanceReply = (reply: any) => reply.code(503).send({
|
||||
code: "maintenance",
|
||||
@@ -461,12 +448,7 @@ export function sessionRoutes(
|
||||
let runtimeOptions = options;
|
||||
let rt: ReturnType<PiProcessManager["createFor"]> | undefined;
|
||||
try {
|
||||
runtimeOptions = await optionsWithRuntimeConfig(
|
||||
runner,
|
||||
workspaceConfigPath,
|
||||
workspaceId,
|
||||
options,
|
||||
);
|
||||
runtimeOptions = optionsWithRuntimeConfig(runner, workspaceConfigPath, options);
|
||||
rt = d.mgr.createFor(id, runtimeOptions);
|
||||
bindRuntime(id, rt, runner, workspaceConfigPath);
|
||||
} catch (error) {
|
||||
@@ -483,11 +465,7 @@ export function sessionRoutes(
|
||||
info(id, "Session created");
|
||||
bootstrap(
|
||||
id, rt, runner, workspaceConfigPath, d.mgr.configure(rt, runtimeOptions),
|
||||
runner.searchPack(
|
||||
b.question,
|
||||
id,
|
||||
(runtimeOptions as any).runtimeConfig?.path ?? workspaceConfigPath,
|
||||
),
|
||||
runner.searchPack(b.question, id, workspaceConfigPath),
|
||||
() => d.mgr.start(id, rt, runtimeOptions),
|
||||
);
|
||||
return { id };
|
||||
@@ -661,12 +639,7 @@ export function sessionRoutes(
|
||||
if (boundRuntimes.get(id) === current) boundRuntimes.delete(id);
|
||||
d.mgr.teardownIfCurrent(id, current);
|
||||
}
|
||||
runtimeOptions = await optionsWithRuntimeConfig(
|
||||
runner,
|
||||
workspaceConfigPath,
|
||||
saved.workspace_id,
|
||||
options,
|
||||
);
|
||||
runtimeOptions = optionsWithRuntimeConfig(runner, workspaceConfigPath, options);
|
||||
rt = d.mgr.createFor(id, runtimeOptions);
|
||||
bindRuntime(id, rt, runner, workspaceConfigPath);
|
||||
} catch {
|
||||
|
||||
@@ -213,37 +213,23 @@ export class ThtRunner {
|
||||
}
|
||||
|
||||
/** Render one immutable canonical registry revision into a backend-owned harness config. */
|
||||
acquireWorkspaceRuntime(
|
||||
workspaceConfigPath: string,
|
||||
effectiveRelationships?: string,
|
||||
): RuntimeConfigLease {
|
||||
const effectiveRelationshipsPath = effectiveRelationships === undefined
|
||||
? undefined
|
||||
: this.createRuntimeSnapshot(effectiveRelationships);
|
||||
let rendered: ReturnType<typeof renderWorkspaceRuntimeFromSnapshotPath>;
|
||||
try {
|
||||
rendered = renderWorkspaceRuntimeFromSnapshotPath({
|
||||
snapshotPath: workspaceConfigPath,
|
||||
harnessDir: this.cfg.harnessDir,
|
||||
configPath: this.cfg.configPath,
|
||||
dataRoot: this.cfg.dataRoot ?? (() => {
|
||||
throw new Error("registry workspace runtime requires an absolute data root");
|
||||
})(),
|
||||
secretRoots: this.cfg.secretRoots ?? [],
|
||||
semanticRuntime: this.cfg.semanticRuntime ?? DEFAULT_SEMANTIC_RUNTIME,
|
||||
workspaceSecretStore: this.cfg.workspaceSecretStore,
|
||||
effectiveRelationshipsPath,
|
||||
});
|
||||
} catch (error) {
|
||||
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
|
||||
throw error;
|
||||
}
|
||||
acquireWorkspaceRuntime(workspaceConfigPath: string): RuntimeConfigLease {
|
||||
const rendered = renderWorkspaceRuntimeFromSnapshotPath({
|
||||
snapshotPath: workspaceConfigPath,
|
||||
harnessDir: this.cfg.harnessDir,
|
||||
configPath: this.cfg.configPath,
|
||||
dataRoot: this.cfg.dataRoot ?? (() => {
|
||||
throw new Error("registry workspace runtime requires an absolute data root");
|
||||
})(),
|
||||
secretRoots: this.cfg.secretRoots ?? [],
|
||||
semanticRuntime: this.cfg.semanticRuntime ?? DEFAULT_SEMANTIC_RUNTIME,
|
||||
workspaceSecretStore: this.cfg.workspaceSecretStore,
|
||||
});
|
||||
let path: string;
|
||||
try {
|
||||
path = this.createRuntimeSnapshot(rendered.renderedConfig);
|
||||
} catch (error) {
|
||||
rendered.releaseSecrets();
|
||||
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
|
||||
throw error;
|
||||
}
|
||||
let released = false;
|
||||
@@ -255,7 +241,6 @@ export class ThtRunner {
|
||||
if (released) return;
|
||||
released = true;
|
||||
this.cleanupRuntimeSnapshot(path);
|
||||
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
|
||||
rendered.releaseSecrets();
|
||||
},
|
||||
};
|
||||
|
||||
@@ -243,12 +243,7 @@ function readSnapshotWorkspace(snapshotPath: string): {
|
||||
}
|
||||
}
|
||||
|
||||
function runtimePaths(
|
||||
dataRoot: string,
|
||||
workspaceId: string,
|
||||
workspaceRevision?: string,
|
||||
effectiveRelationshipsPath?: string,
|
||||
): RuntimePaths {
|
||||
function runtimePaths(dataRoot: string, workspaceId: string, workspaceRevision?: string): RuntimePaths {
|
||||
if (!isAbsolute(dataRoot)) throw new Error("registry workspace runtime requires an absolute data root");
|
||||
const root = join(dataRoot, "sessions", workspaceId);
|
||||
return {
|
||||
@@ -259,9 +254,6 @@ function runtimePaths(
|
||||
...(workspaceRevision === undefined
|
||||
? {}
|
||||
: { annotations_root: join(dataRoot, "sessions", workspaceId, "revisions", workspaceRevision, "artifacts") }),
|
||||
...(effectiveRelationshipsPath === undefined
|
||||
? {}
|
||||
: { effective_relationships: effectiveRelationshipsPath }),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -309,7 +301,6 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
|
||||
dataRoot: string;
|
||||
secretRoots: readonly string[];
|
||||
semanticRuntime: SemanticRuntimeConfig;
|
||||
effectiveRelationshipsPath?: string;
|
||||
workspaceSecretStore?: WorkspaceSecretStore;
|
||||
}): RenderedWorkspaceRuntime {
|
||||
const secretLease = options.workspaceSecretStore === undefined
|
||||
@@ -334,12 +325,7 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
|
||||
workspaceId: options.workspaceId,
|
||||
workspaceRevision: options.workspaceRevision,
|
||||
revisionContentRoot: options.revisionContentRoot,
|
||||
runtimePaths: runtimePaths(
|
||||
options.dataRoot,
|
||||
options.workspaceId,
|
||||
options.workspaceRevision,
|
||||
options.effectiveRelationshipsPath,
|
||||
),
|
||||
runtimePaths: runtimePaths(options.dataRoot, options.workspaceId, options.workspaceRevision),
|
||||
installationOverlay: overlay,
|
||||
bindings,
|
||||
bindingDigest: stableBindingDigest(bindings),
|
||||
@@ -348,12 +334,7 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
|
||||
renderedConfig: renderRuntimeConfig(
|
||||
options.workspace,
|
||||
bindings,
|
||||
runtimePaths(
|
||||
options.dataRoot,
|
||||
options.workspaceId,
|
||||
options.workspaceRevision,
|
||||
options.effectiveRelationshipsPath,
|
||||
),
|
||||
runtimePaths(options.dataRoot, options.workspaceId, options.workspaceRevision),
|
||||
context,
|
||||
overlay,
|
||||
options.semanticRuntime,
|
||||
@@ -372,7 +353,6 @@ export function renderWorkspaceRuntimeFromSnapshotPath(options: {
|
||||
dataRoot: string;
|
||||
secretRoots: readonly string[];
|
||||
semanticRuntime: SemanticRuntimeConfig;
|
||||
effectiveRelationshipsPath?: string;
|
||||
workspaceSecretStore?: WorkspaceSecretStore;
|
||||
}): RenderedWorkspaceRuntime {
|
||||
const snapshot = readSnapshotWorkspace(options.snapshotPath);
|
||||
@@ -386,7 +366,6 @@ export function renderWorkspaceRuntimeFromSnapshotPath(options: {
|
||||
dataRoot: options.dataRoot,
|
||||
secretRoots: options.secretRoots,
|
||||
semanticRuntime: options.semanticRuntime,
|
||||
effectiveRelationshipsPath: options.effectiveRelationshipsPath,
|
||||
workspaceSecretStore: options.workspaceSecretStore,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -12,8 +12,6 @@ export interface RuntimePaths {
|
||||
memory: string;
|
||||
/** Revision-qualified root for curated FK annotations (P5); optional for legacy callers. */
|
||||
annotations_root?: string;
|
||||
/** Immutable Catalog projection used as the exclusive runtime relationship source. */
|
||||
effective_relationships?: string;
|
||||
}
|
||||
|
||||
export interface RuntimeIdentity {
|
||||
|
||||
@@ -13,10 +13,7 @@ import type { WorkspaceRegistry, WorkspaceRevision } from "../src/workspaces/reg
|
||||
import type { WorkspaceDescriptor } from "../src/workspaces/schema.js";
|
||||
|
||||
const roots: string[] = [];
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
|
||||
|
||||
const workspace: WorkspaceDescriptor = {
|
||||
workspace: { schema_version: 3, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
||||
@@ -39,7 +36,6 @@ function setup(
|
||||
catalogOperationCoordinator?: CatalogOperationCoordinator;
|
||||
catalogPostgresAccess?: CatalogPostgresAccess;
|
||||
} = {},
|
||||
workspaceDescriptor: WorkspaceDescriptor = workspace,
|
||||
) {
|
||||
const secretRoot = mkdtempSync(join(tmpdir(), "catalog-secret-"));
|
||||
const runtimeRoot = mkdtempSync(join(tmpdir(), "catalog-secret-runtime-"));
|
||||
@@ -49,8 +45,7 @@ function setup(
|
||||
const registry = {
|
||||
list: vi.fn(async () => [revision]),
|
||||
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
|
||||
read: vi.fn(async () => ({ workspace: workspaceDescriptor, revision })),
|
||||
readPinned: vi.fn(async () => ({ workspace: workspaceDescriptor, workspaceConfigPath: revision.snapshotPath })),
|
||||
read: vi.fn(async () => ({ workspace, revision })),
|
||||
} as unknown as WorkspaceRegistry;
|
||||
const app = buildApp(loadConfig({
|
||||
THT_HARNESS_DIR: "/missing",
|
||||
@@ -101,31 +96,10 @@ const fleetSnapshot: ObservedSchemaSnapshot = {
|
||||
};
|
||||
|
||||
test("lists every YAML workspace and creates its one database configuration", async () => {
|
||||
const { app, secretStore } = setup();
|
||||
const { app } = setup();
|
||||
const initial = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(initial.statusCode).toBe(200);
|
||||
expect(initial.json()).toMatchObject([{
|
||||
workspaceId: "psd-clinical",
|
||||
configured: false,
|
||||
databaseName: "warehouse",
|
||||
workspaceRevision: { commit: revision.commit, blob: revision.blob },
|
||||
workspaceEvidence: { sourceType: null, state: "not_declared" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "configuration_required",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
}]);
|
||||
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_HOST", "runtime-db.internal");
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_PORT", "5432");
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_USER", "runtime-reader");
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_TRANSPORT", "postgres_direct");
|
||||
secretStore.putMany("psd-clinical", { "dwh.password": "runtime-password" });
|
||||
const runtimeReady = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(runtimeReady.json()).toMatchObject([{
|
||||
runtimeBinding: { configurationState: "ready", sessionTransportSupported: true },
|
||||
}]);
|
||||
expect(initial.json()).toMatchObject([{ workspaceId: "psd-clinical", configured: false, databaseName: "warehouse" }]);
|
||||
|
||||
const created = await app.inject({ method: "POST", url: "/catalog/databases", payload: direct });
|
||||
expect(created.statusCode).toBe(201);
|
||||
@@ -136,39 +110,6 @@ test("lists every YAML workspace and creates its one database configuration", as
|
||||
expect(listed.json()).toMatchObject([{ configured: true, binding: { transport: "postgres_direct", host: "db.internal" } }]);
|
||||
});
|
||||
|
||||
test("projects remote Evidence credential state without conflating catalog secrets", async () => {
|
||||
const evidenceWorkspace: WorkspaceDescriptor = {
|
||||
...workspace,
|
||||
evidence: {
|
||||
schema_version: 2,
|
||||
source: {
|
||||
type: "http",
|
||||
uris: ["https://evidence.example.test/guide.md"],
|
||||
authentication: "signed_urls_file",
|
||||
connect_timeout_ms: 5_000,
|
||||
read_timeout_ms: 30_000,
|
||||
max_bytes: 10 * 1024 * 1024,
|
||||
max_redirects: 5,
|
||||
allow_private_hosts: false,
|
||||
max_cache_bytes: 64 * 1024 * 1024,
|
||||
},
|
||||
policy: { max_chunk_chars: 4_000, retain_published_generations: 3 },
|
||||
},
|
||||
};
|
||||
const { app, secretStore } = setup({}, {}, evidenceWorkspace);
|
||||
|
||||
const missing = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(missing.json()).toMatchObject([{
|
||||
workspaceEvidence: { sourceType: "http", state: "configuration_required" },
|
||||
}]);
|
||||
|
||||
secretStore.putMany("psd-clinical", { "evidence.signed_urls": "https://signed.example.test/evidence" });
|
||||
const configured = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(configured.json()).toMatchObject([{
|
||||
workspaceEvidence: { sourceType: "http", state: "configured_unverified" },
|
||||
}]);
|
||||
});
|
||||
|
||||
test("lists orphaned records and takes the REST diagnostic path from workspace YAML", async () => {
|
||||
const { app, repository } = setup();
|
||||
await repository.create({
|
||||
|
||||
@@ -218,12 +218,6 @@ test("suggests sensitive flags from structural metadata without persisting them"
|
||||
runId: responseBody.run.id,
|
||||
sequence: 2,
|
||||
level: "info",
|
||||
message: "Classified 1 of 1 columns.",
|
||||
},
|
||||
{
|
||||
runId: responseBody.run.id,
|
||||
sequence: 3,
|
||||
level: "info",
|
||||
message: "Sensitive-field suggestion generation completed for 1 column.",
|
||||
},
|
||||
]);
|
||||
@@ -677,7 +671,6 @@ test("generates one selected Catalog Column from a single JSON code fence", asyn
|
||||
errorSummary: null,
|
||||
});
|
||||
expect(Object.keys(start.json()).sort()).toEqual([
|
||||
"cacheReadTokens",
|
||||
"createdAt",
|
||||
"databaseId",
|
||||
"errorSummary",
|
||||
@@ -685,11 +678,9 @@ test("generates one selected Catalog Column from a single JSON code fence", asyn
|
||||
"finishedAt",
|
||||
"generated",
|
||||
"id",
|
||||
"inputTokens",
|
||||
"language",
|
||||
"modelId",
|
||||
"nonGeneratable",
|
||||
"outputTokens",
|
||||
"processed",
|
||||
"scope",
|
||||
"startedAt",
|
||||
@@ -1281,7 +1272,7 @@ test("Stop aborts source sampling before any model request", async () => {
|
||||
test("an isolated exhausted technical batch failure allows completion with errors", async () => {
|
||||
const modelCompleter: ModelCompleter = {
|
||||
complete: vi.fn(async (request) => {
|
||||
if (vi.mocked(modelCompleter.complete).mock.calls.length <= 2) {
|
||||
if (vi.mocked(modelCompleter.complete).mock.calls.length === 1) {
|
||||
throw new ModelCompletionProviderError();
|
||||
}
|
||||
const context = JSON.parse(request.messages[1]!.content.split("\n").slice(1).join("\n"));
|
||||
@@ -1329,7 +1320,7 @@ test("an isolated exhausted technical batch failure allows completion with error
|
||||
failed: 10,
|
||||
errorSummary: "Description generation completed with errors.",
|
||||
});
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(3);
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(2);
|
||||
const updated = new Map(
|
||||
(await repository.listColumns(database.id, table.id)).map((column) => [column.id, column]),
|
||||
);
|
||||
@@ -1355,10 +1346,10 @@ test("success resets the technical-failure streak and the third later failure st
|
||||
const modelCompleter: ModelCompleter = {
|
||||
complete: vi.fn(async (request) => {
|
||||
const call = vi.mocked(modelCompleter.complete).mock.calls.length;
|
||||
if ([1, 2, 4, 5, 6, 7, 8, 9].includes(call)) {
|
||||
if ([1, 2, 4, 5, 6].includes(call)) {
|
||||
throw Object.assign(new ModelCompletionProviderError(), { message: sensitiveDiagnostic });
|
||||
}
|
||||
if (call > 9) throw new Error("a later batch must not start");
|
||||
if (call > 6) throw new Error("a later batch must not start");
|
||||
const context = JSON.parse(request.messages[1]!.content.split("\n").slice(1).join("\n"));
|
||||
return JSON.stringify({
|
||||
results: context.targets.map((target: { targetId: string }) => ({
|
||||
@@ -1398,25 +1389,25 @@ test("success resets the technical-failure streak and the third later failure st
|
||||
expect(run).toMatchObject({
|
||||
status: "failed",
|
||||
total: 61,
|
||||
processed: 50,
|
||||
processed: 60,
|
||||
generated: 10,
|
||||
nonGeneratable: 0,
|
||||
failed: 40,
|
||||
failed: 50,
|
||||
errorSummary: "Description generation stopped after three consecutive technical batch failures.",
|
||||
});
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(9);
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(6);
|
||||
const requests = vi.mocked(modelCompleter.complete).mock.calls.map(([request]) => request);
|
||||
expect(new Set(requests.map((request) => request.signal)).size).toBe(1);
|
||||
expect(new Set(requests.map((request) => request.model.id))).toEqual(new Set([configuredModel.id]));
|
||||
const updated = new Map(
|
||||
(await repository.listColumns(database.id, table.id)).map((column) => [column.id, column]),
|
||||
);
|
||||
targetIds.slice(10, 20).forEach((targetId) => {
|
||||
targetIds.slice(20, 30).forEach((targetId) => {
|
||||
expect(updated.get(targetId)?.generatedDescription).toBe("Successful reset batch.");
|
||||
});
|
||||
expect(updated.get(targetIds[50]!)?.generatedDescription).toBeNull();
|
||||
expect(updated.get(targetIds[60]!)?.generatedDescription).toBeNull();
|
||||
const events = await repository.listDescriptionGenerationEvents(run.id);
|
||||
expect(events.filter((event) => event.message.includes("model provider request failed"))).toHaveLength(8);
|
||||
expect(events.filter((event) => event.message.includes("model provider request failed"))).toHaveLength(5);
|
||||
expect(events.at(-1)).toEqual(expect.objectContaining({
|
||||
level: "error",
|
||||
message: "Description generation stopped after three consecutive technical batch failures.",
|
||||
@@ -1669,7 +1660,6 @@ test("Description Generation history is newest-first, bounded, and exposes only
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(response.json().map((run: { id: string }) => run.id)).toEqual(ids.slice(1).reverse());
|
||||
expect(Object.keys(response.json()[0]).sort()).toEqual([
|
||||
"cacheReadTokens",
|
||||
"createdAt",
|
||||
"databaseId",
|
||||
"errorSummary",
|
||||
@@ -1677,11 +1667,9 @@ test("Description Generation history is newest-first, bounded, and exposes only
|
||||
"finishedAt",
|
||||
"generated",
|
||||
"id",
|
||||
"inputTokens",
|
||||
"language",
|
||||
"modelId",
|
||||
"nonGeneratable",
|
||||
"outputTokens",
|
||||
"processed",
|
||||
"scope",
|
||||
"startedAt",
|
||||
@@ -1957,7 +1945,7 @@ test("retains completed batch writes when a later batch response is malformed",
|
||||
});
|
||||
const { run } = await waitForTerminalRun(app, start.json().id);
|
||||
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(3);
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(2);
|
||||
expect(run).toMatchObject({
|
||||
status: "completed_with_errors",
|
||||
total: 11,
|
||||
@@ -1984,14 +1972,9 @@ test("retains completed batch writes when a later batch response is malformed",
|
||||
expect(events.slice(2, 12).map((event) => event.message)).toEqual(
|
||||
orderedIds.slice(0, 10).map((targetId) => `Generated description for Catalog Column ${targetId}.`),
|
||||
);
|
||||
expect(events.find((event) => event.level === "warning" && event.message.includes("Retrying batch"))).toEqual(
|
||||
expect.objectContaining({
|
||||
message: "The model response did not match the required schema. Retrying batch (attempt 2 of 2).",
|
||||
}),
|
||||
);
|
||||
expect(events.find((event) => event.level === "error")).toEqual(expect.objectContaining({
|
||||
level: "error",
|
||||
message: `The model response did not match the required schema. Affected Catalog Column target: ${orderedIds[10]}.`,
|
||||
message: `The model response was invalid. Affected Catalog Column target: ${orderedIds[10]}.`,
|
||||
}));
|
||||
} finally {
|
||||
await app.close();
|
||||
@@ -2203,7 +2186,7 @@ test("Generate Missing skips prior partial results and includes null, empty, and
|
||||
const metadata = JSON.parse(request.messages[1]!.content.split("\n").slice(1).join("\n"));
|
||||
if (mode === "partial") {
|
||||
partialCall += 1;
|
||||
if (partialCall === 2 || partialCall === 3) throw new ModelCompletionProviderError();
|
||||
if (partialCall === 2) throw new ModelCompletionProviderError();
|
||||
return JSON.stringify({
|
||||
results: metadata.targets.map((target: { targetId: string }) => ({
|
||||
targetId: target.targetId,
|
||||
@@ -2529,58 +2512,6 @@ test("localizes valid non-generatable Catalog Column results in English", async
|
||||
}
|
||||
});
|
||||
|
||||
test("retries invalid JSON once and completes the batch when the second response is valid", async () => {
|
||||
const modelCompleter: ModelCompleter = {
|
||||
complete: vi.fn(async (request) => {
|
||||
if (vi.mocked(modelCompleter.complete).mock.calls.length === 1) {
|
||||
return { content: "not-json", usage: { input: 11, cacheRead: 3, output: 2 } };
|
||||
}
|
||||
const context = JSON.parse(request.messages[1]!.content.split("\n").slice(1).join("\n"));
|
||||
return {
|
||||
content: JSON.stringify({
|
||||
results: context.targets.map((target: { targetId: string }) => ({
|
||||
targetId: target.targetId,
|
||||
outcome: "generated",
|
||||
description: "Generated after the application retry.",
|
||||
})),
|
||||
}),
|
||||
usage: { input: 7, cacheRead: 1, output: 5 },
|
||||
};
|
||||
}),
|
||||
};
|
||||
const { app, repository, database, table, column } = await setup(modelCompleter);
|
||||
try {
|
||||
const start = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
||||
payload: { modelId: configuredModel.id, scope: "selected_columns", targetIds: [column.id] },
|
||||
});
|
||||
const { run } = await waitForTerminalRun(app, start.json().id);
|
||||
|
||||
expect(run).toMatchObject({
|
||||
status: "completed",
|
||||
processed: 1,
|
||||
generated: 1,
|
||||
failed: 0,
|
||||
inputTokens: 18,
|
||||
cacheReadTokens: 4,
|
||||
outputTokens: 7,
|
||||
});
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(2);
|
||||
expect(await repository.getColumn(database.id, table.id, column.id)).toMatchObject({
|
||||
generatedDescription: "Generated after the application retry.",
|
||||
});
|
||||
expect(await repository.listDescriptionGenerationEvents(run.id)).toEqual(expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
level: "warning",
|
||||
message: "The model response was not valid JSON. Retrying batch (attempt 2 of 2).",
|
||||
}),
|
||||
]));
|
||||
} finally {
|
||||
await app.close();
|
||||
}
|
||||
});
|
||||
|
||||
test("fails safely when the provider fails and redacts provider diagnostics", async () => {
|
||||
const sensitiveDiagnostic = "test-provider-secret private prompt raw provider payload";
|
||||
const modelCompleter: ModelCompleter = {
|
||||
@@ -2597,7 +2528,6 @@ test("fails safely when the provider fails and redacts provider diagnostics", as
|
||||
});
|
||||
const { run } = await waitForTerminalRun(app, start.json().id);
|
||||
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(2);
|
||||
expect(run).toMatchObject({
|
||||
status: "completed_with_errors",
|
||||
processed: 1,
|
||||
@@ -2635,7 +2565,7 @@ test.each([
|
||||
["duplicate mappings", (targetIds: readonly string[]) => ({ results: [
|
||||
{ targetId: targetIds[0], outcome: "generated", description: "First valid value" },
|
||||
{ targetId: targetIds[0], outcome: "non_generatable" },
|
||||
] }), "The model response was missing one or more requested targets."],
|
||||
] })],
|
||||
["unknown mappings", (targetIds: readonly string[]) => ({ results: [
|
||||
{ targetId: targetIds[0], outcome: "non_generatable" },
|
||||
{
|
||||
@@ -2643,15 +2573,15 @@ test.each([
|
||||
outcome: "generated",
|
||||
description: "Unknown target value",
|
||||
},
|
||||
] }), "The model response was missing one or more requested targets."],
|
||||
] })],
|
||||
["missing mappings", (targetIds: readonly string[]) => ({ results: [
|
||||
{ targetId: targetIds[0], outcome: "generated", description: "Only one result" },
|
||||
] }), "The model response was missing one or more requested targets."],
|
||||
] })],
|
||||
["malformed mappings", (targetIds: readonly string[]) => ({ results: [
|
||||
{ targetId: targetIds[0], outcome: "generated", description: "First valid value" },
|
||||
{ targetId: targetIds[1], outcome: "generated", description: " " },
|
||||
] }), "The model response did not match the required schema."],
|
||||
] as const)("rejects %s without applying any result from the batch", async (_name, responseFor, failureMessage) => {
|
||||
] })],
|
||||
] as const)("rejects %s without applying any result from the batch", async (_name, responseFor) => {
|
||||
let selectedColumnIds: string[] = [];
|
||||
const modelCompleter: ModelCompleter = {
|
||||
complete: vi.fn(async () => JSON.stringify(responseFor(selectedColumnIds))),
|
||||
@@ -2695,7 +2625,6 @@ test.each([
|
||||
});
|
||||
const { run } = await waitForTerminalRun(app, start.json().id);
|
||||
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(2);
|
||||
expect(run).toMatchObject({
|
||||
status: "completed_with_errors",
|
||||
processed: 2,
|
||||
@@ -2715,7 +2644,7 @@ test.each([
|
||||
expect((await repository.listDescriptionGenerationEvents(run.id)).find((event) => event.level === "error")).toEqual(
|
||||
expect.objectContaining({
|
||||
level: "error",
|
||||
message: `${failureMessage} Affected Catalog Column targets: ${selectedColumnIds.join(", ")}.`,
|
||||
message: `The model response was invalid. Affected Catalog Column targets: ${selectedColumnIds.join(", ")}.`,
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
|
||||
@@ -13,7 +13,6 @@ import { up as upSchemaSync } from "../src/catalog/migrations/003_catalog_schema
|
||||
import { up as upDescriptionGeneration } from "../src/catalog/migrations/005_description_generation_runs.js";
|
||||
import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js";
|
||||
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
|
||||
import { up as upAiTokenUsage } from "../src/catalog/migrations/009_ai_token_usage.js";
|
||||
import { KyselyCatalogRepository, type CatalogDatabase } from "../src/catalog/repository.js";
|
||||
import { loadConfig } from "../src/config.js";
|
||||
import type { WorkspaceRegistry } from "../src/workspaces/registry.js";
|
||||
@@ -49,7 +48,6 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
||||
await upSensitiveDataFlag(db);
|
||||
await upDescriptionGeneration(db);
|
||||
await upSensitiveSuggestionRuns(db);
|
||||
await upAiTokenUsage(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const database = await repository.create({
|
||||
workspaceId: "psd-clinical",
|
||||
@@ -126,7 +124,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
||||
description: "Elenco dei pazienti e dei loro dati clinici.",
|
||||
}] });
|
||||
}
|
||||
if (call === 5) {
|
||||
if (call === 4) {
|
||||
return JSON.stringify({ results: [
|
||||
{
|
||||
targetId: birthDate.id,
|
||||
@@ -140,14 +138,14 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
||||
},
|
||||
] });
|
||||
}
|
||||
if (call === 6) {
|
||||
if (call === 5) {
|
||||
return JSON.stringify({ results: [{
|
||||
targetId: table.id,
|
||||
outcome: "generated",
|
||||
description: "Descrizione rigenerata della tabella pazienti.",
|
||||
}] });
|
||||
}
|
||||
if (call === 7) {
|
||||
if (call === 6) {
|
||||
return JSON.stringify({ results: [{
|
||||
targetId: birthDate.id,
|
||||
outcome: "generated",
|
||||
@@ -342,7 +340,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
||||
expect(await repository.getColumn(database.id, table.id, birthDate.id)).toMatchObject({
|
||||
generatedDescription: "Descrizione recuperata della data di nascita.",
|
||||
});
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(7);
|
||||
expect(modelCompleter.complete).toHaveBeenCalledTimes(6);
|
||||
expect(JSON.stringify(vi.mocked(modelCompleter.complete).mock.calls)).toContain(persistedSampleSecret);
|
||||
|
||||
const runIds = [
|
||||
|
||||
@@ -1,9 +1,6 @@
|
||||
import { expect, test, vi } from "vitest";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import {
|
||||
ConcreteDescriptionSourceSampler,
|
||||
PostgresDescriptionSourceSampler,
|
||||
type DescriptionSourceSamplingTarget,
|
||||
} from "../src/catalog/description-source-sampler.js";
|
||||
import type {
|
||||
@@ -11,8 +8,6 @@ import type {
|
||||
CatalogPostgresAccess,
|
||||
} from "../src/catalog/postgres-access.js";
|
||||
import type { WorkspaceDatabase } from "../src/catalog/types.js";
|
||||
import type { WorkspaceSecretStore } from "../src/workspaces/secret-store.js";
|
||||
import { CATALOG_SECRET_IDS } from "../src/catalog/secrets.js";
|
||||
|
||||
const database: WorkspaceDatabase = {
|
||||
id: "11111111-1111-4111-8111-111111111111",
|
||||
@@ -56,7 +51,7 @@ test("samples at most five source rows and five distinct non-null examples in a
|
||||
const access: CatalogPostgresAccess = {
|
||||
connect: vi.fn(async () => ({ query, end }) as CatalogDatabaseClient),
|
||||
};
|
||||
const sampler = new ConcreteDescriptionSourceSampler(access);
|
||||
const sampler = new PostgresDescriptionSourceSampler(access);
|
||||
const controller = new AbortController();
|
||||
|
||||
const samples = await sampler.sample(database, [target], controller.signal);
|
||||
@@ -97,71 +92,13 @@ test("samples at most five source rows and five distinct non-null examples in a
|
||||
expect(end).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
test("samples source rows through the configured REST run_query binding", async () => {
|
||||
const root = mkdtempSync(join(tmpdir(), "tht-source-rest-"));
|
||||
const credentialFile = join(root, "api-key");
|
||||
writeFileSync(credentialFile, "test-api-key\n", { mode: 0o600 });
|
||||
const release = vi.fn();
|
||||
const secretStore = {
|
||||
materialize: vi.fn(() => ({
|
||||
files: new Map([[CATALOG_SECRET_IDS.apiKey, credentialFile]]),
|
||||
release,
|
||||
})),
|
||||
} as unknown as WorkspaceSecretStore;
|
||||
const fetchMock = vi.fn(async () => new Response(JSON.stringify([
|
||||
{ 'status"code': "active", ward: null },
|
||||
{ 'status"code': "pending", ward: "A" },
|
||||
]), { status: 200, headers: { "content-type": "application/json" } }));
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
const access: CatalogPostgresAccess = {
|
||||
connect: vi.fn(async () => { throw new Error("PostgreSQL access must not be used"); }),
|
||||
};
|
||||
const sampler = new ConcreteDescriptionSourceSampler(access, secretStore);
|
||||
const restDatabase: WorkspaceDatabase = {
|
||||
...database,
|
||||
binding: {
|
||||
transport: "rest_api",
|
||||
baseUrl: "https://dwh.example.test/root/",
|
||||
restPath: "/health",
|
||||
restAuth: "x-api-key",
|
||||
},
|
||||
};
|
||||
|
||||
try {
|
||||
await expect(sampler.sample(restDatabase, [target], new AbortController().signal)).resolves.toEqual([{
|
||||
targetId: target.targetId,
|
||||
tableName: target.tableName,
|
||||
rows: [
|
||||
{ fields: [{ name: 'status"code', value: "active" }, { name: "ward", value: null }] },
|
||||
{ fields: [{ name: 'status"code', value: "pending" }, { name: "ward", value: "A" }] },
|
||||
],
|
||||
representativeValues: [
|
||||
{ column: 'status"code', values: ["active", "pending"] },
|
||||
{ column: "ward", values: ["A"] },
|
||||
],
|
||||
}]);
|
||||
expect(access.connect).not.toHaveBeenCalled();
|
||||
expect(fetchMock).toHaveBeenCalledWith("https://dwh.example.test/root/rpc/run_query", expect.objectContaining({
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json", "x-api-key": "test-api-key" },
|
||||
body: JSON.stringify({
|
||||
query_text: 'SELECT LEFT(("status""code")::text, 256) AS "status""code", LEFT(("ward")::text, 256) AS "ward" FROM "clinical""data"."patient""facts" LIMIT 5',
|
||||
}),
|
||||
}));
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("does not issue a SELECT when a protected target has no source columns", async () => {
|
||||
const query = vi.fn(async () => ({ rows: [] }));
|
||||
const end = vi.fn(async () => undefined);
|
||||
const access: CatalogPostgresAccess = {
|
||||
connect: vi.fn(async () => ({ query, end }) as CatalogDatabaseClient),
|
||||
};
|
||||
const sampler = new ConcreteDescriptionSourceSampler(access);
|
||||
const sampler = new PostgresDescriptionSourceSampler(access);
|
||||
|
||||
const samples = await sampler.sample(database, [{
|
||||
targetId: target.targetId,
|
||||
@@ -192,7 +129,7 @@ test("rolls back and closes the source connection when sampling fails", async ()
|
||||
const access: CatalogPostgresAccess = {
|
||||
connect: vi.fn(async () => ({ query, end }) as CatalogDatabaseClient),
|
||||
};
|
||||
const sampler = new ConcreteDescriptionSourceSampler(access);
|
||||
const sampler = new PostgresDescriptionSourceSampler(access);
|
||||
const controller = new AbortController();
|
||||
|
||||
await expect(sampler.sample(database, [target], controller.signal)).rejects.toThrow();
|
||||
|
||||
@@ -1,207 +0,0 @@
|
||||
import { expect, test } from "vitest";
|
||||
import {
|
||||
CatalogLogicalRelationshipService,
|
||||
LogicalRelationshipDuplicateError,
|
||||
LogicalRelationshipSchemaStaleError,
|
||||
LogicalRelationshipTargetNotUniqueError,
|
||||
LogicalRelationshipTypeIncompatibleError,
|
||||
} from "../src/catalog/logical-relationship-service.js";
|
||||
import { MemoryCatalogRepository } from "../src/catalog/memory-repository.js";
|
||||
import type { ObservedSchemaSnapshot } from "../src/catalog/types.js";
|
||||
|
||||
const column = (
|
||||
tableName: string,
|
||||
name: string,
|
||||
ordinalPosition: number,
|
||||
dataType: string,
|
||||
primaryKeyPosition: number | null,
|
||||
) => ({
|
||||
tableName, name, ordinalPosition, dataType, primaryKeyPosition,
|
||||
isNullable: false, defaultExpression: null, sourceComment: null,
|
||||
});
|
||||
|
||||
function schema(
|
||||
tableNames: string[],
|
||||
columns: ObservedSchemaSnapshot["columns"],
|
||||
relationships: ObservedSchemaSnapshot["relationships"] = [],
|
||||
): ObservedSchemaSnapshot {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
capabilities: { tables: "available", columns: "available", relationships: "available" },
|
||||
tables: tableNames.map((name) => ({ name, sourceComment: null })),
|
||||
columns,
|
||||
relationships,
|
||||
};
|
||||
}
|
||||
|
||||
async function setup(snapshot: ObservedSchemaSnapshot) {
|
||||
const repository = new MemoryCatalogRepository();
|
||||
const database = await repository.create({
|
||||
workspaceId: "relationships", engine: "postgres", databaseName: "warehouse", schema: "public",
|
||||
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
|
||||
});
|
||||
await repository.applySchemaSync(database.id, database.version, "all", [], snapshot);
|
||||
const service = new CatalogLogicalRelationshipService(repository);
|
||||
const context = (await repository.getLogicalRelationshipContext(database.id))!;
|
||||
const endpoint = (tableName: string, columnName: string) => context.endpoints.find((item) => (
|
||||
item.tableName === tableName && item.columnName === columnName
|
||||
))!;
|
||||
return { repository, database, service, endpoint };
|
||||
}
|
||||
|
||||
test("keeps excluded relationships across rebuild and recreates hard-deleted relationships", async () => {
|
||||
const { database, service, endpoint } = await setup(schema(
|
||||
["users", "orders"],
|
||||
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
|
||||
column("orders", "user_id", 2, "bigint", null)],
|
||||
));
|
||||
const source = endpoint("orders", "user_id");
|
||||
const target = endpoint("users", "id");
|
||||
const manual = await service.addManual(database.id, source.columnId, target.columnId);
|
||||
expect(manual).toMatchObject({ origin: "manual", status: "active" });
|
||||
|
||||
expect(await service.setStatus(database.id, manual.id, "excluded"))
|
||||
.toMatchObject({ status: "excluded" });
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 0, excluded: 1, ambiguous: 0,
|
||||
});
|
||||
expect((await service.list(database.id)).filter((item) => item.origin !== "physical"))
|
||||
.toMatchObject([{ id: manual.id, origin: "manual", status: "excluded" }]);
|
||||
|
||||
await service.deletePermanently(database.id, manual.id);
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
expect((await service.list(database.id)).filter((item) => item.origin !== "physical"))
|
||||
.toMatchObject([{ origin: "generated", status: "active" }]);
|
||||
});
|
||||
|
||||
test("normalizes snake, kebab, and camel names without fuzzy matching", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["users", "events"],
|
||||
[column("users", "id", 1, "bigint", 1), column("events", "id", 1, "bigint", 1),
|
||||
column("events", "user_id", 2, "bigint", null),
|
||||
column("events", "user-id", 3, "bigint", null),
|
||||
column("events", "userId", 4, "bigint", null),
|
||||
column("events", "userid", 5, "bigint", null),
|
||||
column("events", "unrelated", 6, "bigint", null)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 4, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("skips generic bare primary-key names while retaining table-qualified matches", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["users", "accounts", "events"],
|
||||
[column("users", "id", 1, "bigint", 1), column("accounts", "id", 1, "bigint", 1),
|
||||
column("events", "event_key", 1, "bigint", 1), column("events", "id", 2, "bigint", null),
|
||||
column("events", "user_id", 3, "bigint", null)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("infers foreign keys from composite-primary-key sources", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["users", "groups", "memberships"],
|
||||
[column("users", "id", 1, "bigint", 1), column("groups", "id", 1, "bigint", 1),
|
||||
column("memberships", "user_id", 1, "bigint", 1),
|
||||
column("memberships", "group_id", 2, "bigint", 2)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 2, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("maps time-key columns to the single primary key of dim_time", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["dim_time", "admissions"],
|
||||
[column("dim_time", "day_key", 1, "integer", 1),
|
||||
column("admissions", "id", 1, "bigint", 1),
|
||||
column("admissions", "admission_time_key", 2, "integer", null),
|
||||
column("admissions", "discharge_time_key", 3, "integer", null)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 2, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("skips ambiguous targets and physical foreign-key pairs", async () => {
|
||||
const ambiguous = await setup(schema(
|
||||
["user", "users", "events"],
|
||||
[column("user", "id", 1, "bigint", 1), column("users", "id", 1, "bigint", 1),
|
||||
column("events", "id", 1, "bigint", 1), column("events", "user_id", 2, "bigint", null)],
|
||||
));
|
||||
await expect(ambiguous.service.rebuildGenerated(ambiguous.database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 1,
|
||||
});
|
||||
|
||||
const physical = await setup(schema(
|
||||
["users", "orders"],
|
||||
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
|
||||
column("orders", "user_id", 2, "bigint", null)],
|
||||
[{
|
||||
constraintName: "orders_user_id_fkey", sourceTableName: "orders", targetTableName: "users",
|
||||
updateRule: "NO ACTION", deleteRule: "NO ACTION", deferrable: false, initiallyDeferred: false,
|
||||
columns: [{ position: 1, sourceColumnName: "user_id", targetColumnName: "id" }],
|
||||
}],
|
||||
));
|
||||
await expect(physical.service.rebuildGenerated(physical.database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 1, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
await expect(physical.service.addManual(
|
||||
physical.database.id,
|
||||
physical.endpoint("orders", "user_id").columnId,
|
||||
physical.endpoint("users", "id").columnId,
|
||||
)).rejects.toBeInstanceOf(LogicalRelationshipDuplicateError);
|
||||
});
|
||||
|
||||
test("validates target uniqueness and canonical type compatibility for manual relationships", async () => {
|
||||
const { database, service, endpoint } = await setup(schema(
|
||||
["users", "composite", "events"],
|
||||
[column("users", "id", 1, "bigint", 1),
|
||||
column("composite", "left_id", 1, "bigint", 1), column("composite", "right_id", 2, "bigint", 2),
|
||||
column("events", "id", 1, "bigint", 1), column("events", "user_id", 2, "integer", null),
|
||||
column("events", "composite_id", 3, "bigint", null)],
|
||||
));
|
||||
await expect(service.addManual(
|
||||
database.id, endpoint("events", "composite_id").columnId, endpoint("composite", "left_id").columnId,
|
||||
)).rejects.toBeInstanceOf(LogicalRelationshipTargetNotUniqueError);
|
||||
await expect(service.addManual(
|
||||
database.id, endpoint("events", "user_id").columnId, endpoint("users", "id").columnId,
|
||||
)).rejects.toBeInstanceOf(LogicalRelationshipTypeIncompatibleError);
|
||||
});
|
||||
|
||||
test("rebuild is additive when an existing generated relationship stops matching", async () => {
|
||||
const initial = schema(
|
||||
["users", "orders"],
|
||||
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
|
||||
column("orders", "user_id", 2, "bigint", null)],
|
||||
);
|
||||
const { repository, database, service } = await setup(initial);
|
||||
await service.rebuildGenerated(database.id);
|
||||
const changed = structuredClone(initial);
|
||||
changed.columns.find((item) => item.tableName === "orders" && item.name === "user_id")!.dataType = "text";
|
||||
await repository.applySchemaSync(database.id, database.version, "columns", [], changed);
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
expect((await service.list(database.id)).filter((item) => item.origin === "generated")).toHaveLength(1);
|
||||
});
|
||||
|
||||
test("refuses inference until the current database version has a full schema sync", async () => {
|
||||
const repository = new MemoryCatalogRepository();
|
||||
const database = await repository.create({
|
||||
workspaceId: "unsynced-relationships",
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
schema: "public",
|
||||
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
|
||||
});
|
||||
const service = new CatalogLogicalRelationshipService(repository);
|
||||
|
||||
await expect(service.rebuildGenerated(database.id))
|
||||
.rejects.toBeInstanceOf(LogicalRelationshipSchemaStaleError);
|
||||
});
|
||||
@@ -12,8 +12,6 @@ import { up as upRuntimeSequencePrivileges } from "../src/catalog/migrations/004
|
||||
import { up as upDescriptionGeneration } from "../src/catalog/migrations/005_description_generation_runs.js";
|
||||
import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js";
|
||||
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
|
||||
import { up as upLogicalRelationships } from "../src/catalog/migrations/008_catalog_logical_relationships.js";
|
||||
import { up as upAiTokenUsage } from "../src/catalog/migrations/009_ai_token_usage.js";
|
||||
|
||||
const dockerAvailable = spawnSync("docker", ["info"], { stdio: "ignore" }).status === 0;
|
||||
|
||||
@@ -28,10 +26,6 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
await upDescriptionGeneration(db);
|
||||
await upSensitiveSuggestionRuns(db);
|
||||
await upAiTokenUsage(db);
|
||||
await sql`CREATE ROLE thothii_catalog_runtime`.execute(db);
|
||||
await upRuntimeSequencePrivileges(db);
|
||||
const sequencePrivilege = await sql<{ allowed: boolean }>`
|
||||
@@ -208,7 +202,6 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository performs scoped metadata cl
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const database = await repository.create({
|
||||
workspaceId: "cleanup-test",
|
||||
@@ -287,7 +280,6 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository atomically consolidates sel
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const database = await repository.create({
|
||||
workspaceId: "consolidation-test",
|
||||
@@ -387,10 +379,8 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
await upDescriptionGeneration(db);
|
||||
await upSensitiveSuggestionRuns(db);
|
||||
await upAiTokenUsage(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const firstDatabase = await repository.create({
|
||||
workspaceId: "generation-one",
|
||||
@@ -620,63 +610,3 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
||||
await container.stop();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test.skipIf(!dockerAvailable)("PostgreSQL repository persists logical relationship lifecycle and tombstones", async () => {
|
||||
const container = await new PostgreSqlContainer("postgres:17.6-bookworm").start();
|
||||
const db = new Kysely<CatalogDatabase>({
|
||||
dialect: new PostgresDialect({ pool: new Pool({ connectionString: container.getConnectionUri() }) }),
|
||||
plugins: [new CamelCasePlugin()],
|
||||
});
|
||||
try {
|
||||
await upDatabases(db);
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const database = await repository.create({
|
||||
workspaceId: "logical-relationships",
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
schema: "public",
|
||||
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
|
||||
});
|
||||
await repository.applySchemaSync(database.id, database.version, "all", [], {
|
||||
schemaVersion: 1,
|
||||
capabilities: { tables: "available", columns: "available", relationships: "available" },
|
||||
tables: [{ name: "users", sourceComment: null }, { name: "orders", sourceComment: null }],
|
||||
columns: [
|
||||
{ tableName: "users", name: "id", ordinalPosition: 1, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: 1, sourceComment: null },
|
||||
{ tableName: "orders", name: "id", ordinalPosition: 1, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: 1, sourceComment: null },
|
||||
{ tableName: "orders", name: "user_id", ordinalPosition: 2, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: null, sourceComment: null },
|
||||
],
|
||||
relationships: [],
|
||||
});
|
||||
const context = (await repository.getLogicalRelationshipContext(database.id))!;
|
||||
const source = context.endpoints.find((item) => item.tableName === "orders" && item.columnName === "user_id")!;
|
||||
const target = context.endpoints.find((item) => item.tableName === "users" && item.columnName === "id")!;
|
||||
const created = await repository.insertLogicalRelationship(database.id, source.columnId, target.columnId, false);
|
||||
expect(created).toMatchObject({ origin: "manual", status: "active" });
|
||||
await expect(repository.insertLogicalRelationship(database.id, source.columnId, target.columnId, true))
|
||||
.resolves.toBeUndefined();
|
||||
|
||||
expect(await repository.setLogicalRelationshipStatus(database.id, created!.id, "excluded"))
|
||||
.toMatchObject({ status: "excluded" });
|
||||
await expect(repository.insertGeneratedLogicalRelationships(database.id, [{
|
||||
sourceColumnId: source.columnId, targetColumnId: target.columnId,
|
||||
}])).resolves.toBe(0);
|
||||
|
||||
await expect(repository.deleteLogicalRelationship(database.id, created!.id)).resolves.toBe(true);
|
||||
await expect(repository.insertGeneratedLogicalRelationships(database.id, [{
|
||||
sourceColumnId: source.columnId, targetColumnId: target.columnId,
|
||||
}])).resolves.toBe(1);
|
||||
expect(await repository.listLogicalRelationships(database.id))
|
||||
.toMatchObject([{ origin: "generated", status: "active" }]);
|
||||
|
||||
await db.deleteFrom("catalogColumns").where("id", "=", source.columnId).execute();
|
||||
expect(await repository.listLogicalRelationships(database.id)).toEqual([]);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
await container.stop();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
@@ -122,7 +122,6 @@ async function setup(env: Record<string, string> = {}) {
|
||||
list: vi.fn(async () => [revision]),
|
||||
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
|
||||
read: vi.fn(async () => ({ workspace, revision })),
|
||||
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
|
||||
} as unknown as WorkspaceRegistry;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test", ...env }), {
|
||||
thtRunner: {} as never,
|
||||
@@ -606,125 +605,3 @@ test("rejects a missing database without partially cleaning valid selections", a
|
||||
expect(response.statusCode).toBe(404);
|
||||
expect(await repository.listTables(database.id)).toHaveLength(2);
|
||||
});
|
||||
|
||||
test("serves one relationship map and supports the manual relationship lifecycle", async () => {
|
||||
const { app, repository, database } = await setup();
|
||||
await seedCatalog(repository, database);
|
||||
const tables = await repository.listTables(database.id);
|
||||
const patients = tables.find((table) => table.name === "patients")!;
|
||||
const visits = tables.find((table) => table.name === "visits")!;
|
||||
const patientId = (await repository.listColumns(database.id, patients.id)).find((item) => item.name === "id")!;
|
||||
const visitId = (await repository.listColumns(database.id, visits.id)).find((item) => item.name === "id")!;
|
||||
|
||||
const created = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships`,
|
||||
payload: { sourceColumnId: visitId.id, targetColumnId: patientId.id },
|
||||
});
|
||||
expect(created.statusCode).toBe(201);
|
||||
expect(created.json()).toMatchObject({ origin: "manual", status: "active", constraintName: null });
|
||||
|
||||
const listed = (await app.inject({
|
||||
method: "GET", url: `/catalog/databases/${database.id}/relationships`,
|
||||
})).json();
|
||||
expect(listed.map((item: { origin: string }) => item.origin).sort()).toEqual(["manual", "physical"]);
|
||||
|
||||
const relationshipId = created.json().id;
|
||||
const excluded = await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
|
||||
payload: { status: "excluded" },
|
||||
});
|
||||
expect(excluded.statusCode).toBe(200);
|
||||
expect(excluded.json()).toMatchObject({ origin: "manual", status: "excluded" });
|
||||
|
||||
const restored = await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
|
||||
payload: { status: "active" },
|
||||
});
|
||||
expect(restored.json()).toMatchObject({ status: "active" });
|
||||
|
||||
expect((await app.inject({
|
||||
method: "DELETE", url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
|
||||
})).statusCode).toBe(204);
|
||||
});
|
||||
|
||||
test("rejects generated inference while the Catalog schema is not current", async () => {
|
||||
const { app, database } = await setup();
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(409);
|
||||
expect(response.json()).toEqual({
|
||||
code: "relationship_schema_stale",
|
||||
message: "Synchronize the current database schema before managing logical relationships.",
|
||||
});
|
||||
});
|
||||
|
||||
test("rebuilds generated relationships and returns the exact summary", async () => {
|
||||
const { app, repository, database } = await setup();
|
||||
const observed = snapshot();
|
||||
observed.relationships = [];
|
||||
await seedCatalog(repository, database, observed);
|
||||
|
||||
const first = await app.inject({
|
||||
method: "POST", url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
});
|
||||
expect(first.statusCode).toBe(200);
|
||||
expect(first.json()).toEqual({ added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0 });
|
||||
const generated = (await repository.listLogicalRelationships(database.id))[0]!;
|
||||
|
||||
await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${generated.id}`,
|
||||
payload: { status: "excluded" },
|
||||
});
|
||||
const second = await app.inject({
|
||||
method: "POST", url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
});
|
||||
expect(second.json()).toEqual({ added: 0, alreadyPresent: 0, excluded: 1, ambiguous: 0 });
|
||||
});
|
||||
|
||||
test("validates relationship requests and keeps physical relationships read-only", async () => {
|
||||
const { app, repository, database } = await setup();
|
||||
await seedCatalog(repository, database);
|
||||
const physical = (await repository.listRelationships(database.id))[0]!;
|
||||
|
||||
const invalid = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships`,
|
||||
payload: { sourceColumnId: physical.columns[0].sourceColumnId, targetColumnId: physical.columns[0].targetColumnId, generated: true },
|
||||
});
|
||||
expect(invalid.statusCode).toBe(400);
|
||||
expect(invalid.json()).toMatchObject({ code: "relationship_request_invalid" });
|
||||
|
||||
const readOnly = await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${physical.id}`,
|
||||
payload: { status: "excluded" },
|
||||
});
|
||||
expect(readOnly.statusCode).toBe(409);
|
||||
expect(readOnly.json()).toMatchObject({ code: "relationship_read_only" });
|
||||
});
|
||||
|
||||
test("requires database.manage for relationship map mutations", async () => {
|
||||
const { app, repository, database } = await setup({ AUTH_MODE: "upstream" });
|
||||
const observed = snapshot();
|
||||
observed.relationships = [];
|
||||
await seedCatalog(repository, database, observed);
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
headers: {
|
||||
"x-thoth-principal-issuer": "portal",
|
||||
"x-thoth-principal-subject": "catalog-reader",
|
||||
"x-thoth-is-admin": "0",
|
||||
},
|
||||
});
|
||||
expect(response.statusCode).toBe(403);
|
||||
expect(response.json()).toEqual({ code: "auth_forbidden", error: "This operation is not permitted" });
|
||||
});
|
||||
|
||||
@@ -54,7 +54,6 @@ async function setup() {
|
||||
list: vi.fn(async () => [revision]),
|
||||
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
|
||||
read: vi.fn(async () => ({ workspace, revision })),
|
||||
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
|
||||
} as unknown as WorkspaceRegistry;
|
||||
const secretStore = new WorkspaceSecretStore({ root: secretRoot, runtimeRoot, installationId: "test" });
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test" }), {
|
||||
|
||||
@@ -1,196 +0,0 @@
|
||||
import { describe, expect, test, vi } from "vitest";
|
||||
import { EffectiveRelationshipSnapshotProvider } from "../src/catalog/effective-relationship-snapshot.js";
|
||||
import type { CatalogRelationship, WorkspaceDatabase } from "../src/catalog/types.js";
|
||||
|
||||
const timestamp = "2026-08-31T10:00:00.000Z";
|
||||
|
||||
function database(): WorkspaceDatabase {
|
||||
return {
|
||||
id: "database-1",
|
||||
workspaceId: "psd",
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
schema: "public",
|
||||
version: 1,
|
||||
createdAt: timestamp,
|
||||
updatedAt: timestamp,
|
||||
binding: { transport: "postgres_direct" },
|
||||
connectionStatus: "reachable",
|
||||
schemaSyncedVersion: 1,
|
||||
schemaSyncedAt: timestamp,
|
||||
};
|
||||
}
|
||||
|
||||
const operations = {
|
||||
async run<T>(_databaseId: string, operation: () => Promise<T>): Promise<T> {
|
||||
return await operation();
|
||||
},
|
||||
};
|
||||
|
||||
function relationship(options: {
|
||||
id: string;
|
||||
origin: "physical" | "generated" | "manual";
|
||||
status?: "active" | "excluded";
|
||||
sourceTable?: string;
|
||||
sourceColumns?: string[];
|
||||
targetTable?: string;
|
||||
targetColumns?: string[];
|
||||
}): CatalogRelationship {
|
||||
const sourceColumns = options.sourceColumns ?? ["user_id"];
|
||||
const targetColumns = options.targetColumns ?? ["id"];
|
||||
const columns = sourceColumns.map((sourceColumnName, position) => ({
|
||||
position: position + 1,
|
||||
sourceColumnId: `source-${position}`,
|
||||
sourceColumnName,
|
||||
targetColumnId: `target-${position}`,
|
||||
targetColumnName: targetColumns[position],
|
||||
}));
|
||||
const common = {
|
||||
id: options.id,
|
||||
databaseId: "database-1",
|
||||
sourceTableId: "source-table",
|
||||
sourceTableName: options.sourceTable ?? "orders",
|
||||
targetTableId: "target-table",
|
||||
targetTableName: options.targetTable ?? "users",
|
||||
columns,
|
||||
createdAt: timestamp,
|
||||
updatedAt: timestamp,
|
||||
};
|
||||
if (options.origin === "physical") {
|
||||
return {
|
||||
...common,
|
||||
constraintName: `fk_${options.id}`,
|
||||
updateRule: "NO ACTION",
|
||||
deleteRule: "NO ACTION",
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
lastSyncedDatabaseVersion: 1,
|
||||
lastSyncedAt: timestamp,
|
||||
origin: "physical",
|
||||
status: "active",
|
||||
};
|
||||
}
|
||||
return {
|
||||
...common,
|
||||
constraintName: null,
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
origin: options.origin,
|
||||
status: options.status ?? "active",
|
||||
columns: [columns[0]!],
|
||||
};
|
||||
}
|
||||
|
||||
describe("EffectiveRelationshipSnapshotProvider", () => {
|
||||
test("returns no projection for a workspace without a Catalog database", async () => {
|
||||
const list = vi.fn();
|
||||
const provider = new EffectiveRelationshipSnapshotProvider(
|
||||
{
|
||||
get: vi.fn().mockResolvedValue(undefined),
|
||||
getByWorkspace: vi.fn().mockResolvedValue(undefined),
|
||||
},
|
||||
{ list },
|
||||
operations,
|
||||
);
|
||||
|
||||
await expect(provider.render("legacy")).resolves.toBeUndefined();
|
||||
expect(list).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("renders a deterministic active map with physical precedence", async () => {
|
||||
const duplicateGenerated = relationship({ id: "generated", origin: "generated" });
|
||||
const physical = relationship({ id: "physical", origin: "physical" });
|
||||
const compositePhysical = relationship({
|
||||
id: "physical-composite",
|
||||
origin: "physical",
|
||||
sourceTable: "order_lines",
|
||||
sourceColumns: ["order_id", "tenant_id"],
|
||||
targetTable: "orders",
|
||||
targetColumns: ["id", "tenant_id"],
|
||||
});
|
||||
const manual = relationship({
|
||||
id: "manual",
|
||||
origin: "manual",
|
||||
sourceTable: "invoices",
|
||||
sourceColumns: ["customer_id"],
|
||||
targetTable: "customers",
|
||||
targetColumns: ["id"],
|
||||
});
|
||||
const excluded = relationship({
|
||||
id: "excluded",
|
||||
origin: "generated",
|
||||
status: "excluded",
|
||||
sourceTable: "invoices",
|
||||
});
|
||||
const list = vi.fn().mockResolvedValue([
|
||||
manual,
|
||||
duplicateGenerated,
|
||||
excluded,
|
||||
physical,
|
||||
compositePhysical,
|
||||
]);
|
||||
const provider = new EffectiveRelationshipSnapshotProvider(
|
||||
{
|
||||
get: vi.fn().mockResolvedValue(database()),
|
||||
getByWorkspace: vi.fn().mockResolvedValue(database()),
|
||||
},
|
||||
{ list },
|
||||
operations,
|
||||
);
|
||||
|
||||
const rendered = await provider.render("psd");
|
||||
expect(rendered?.endsWith("\n")).toBe(true);
|
||||
expect(JSON.parse(rendered!)).toEqual({
|
||||
schemaVersion: 1,
|
||||
workspaceId: "psd",
|
||||
relationships: [
|
||||
{
|
||||
sourceTable: "invoices",
|
||||
sourceColumns: ["customer_id"],
|
||||
targetTable: "customers",
|
||||
targetColumns: ["id"],
|
||||
origin: "manual",
|
||||
},
|
||||
{
|
||||
sourceTable: "order_lines",
|
||||
sourceColumns: ["order_id", "tenant_id"],
|
||||
targetTable: "orders",
|
||||
targetColumns: ["id", "tenant_id"],
|
||||
origin: "physical",
|
||||
},
|
||||
{
|
||||
sourceTable: "orders",
|
||||
sourceColumns: ["user_id"],
|
||||
targetTable: "users",
|
||||
targetColumns: ["id"],
|
||||
origin: "physical",
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(list).toHaveBeenCalledWith("database-1");
|
||||
});
|
||||
|
||||
test("fails closed when the Catalog schema is absent or stale", async () => {
|
||||
const stale = database();
|
||||
delete stale.schemaSyncedVersion;
|
||||
delete stale.schemaSyncedAt;
|
||||
const list = vi.fn();
|
||||
const provider = new EffectiveRelationshipSnapshotProvider(
|
||||
{
|
||||
get: vi.fn().mockResolvedValue(stale),
|
||||
getByWorkspace: vi.fn().mockResolvedValue(stale),
|
||||
},
|
||||
{ list },
|
||||
operations,
|
||||
);
|
||||
|
||||
await expect(provider.render("psd")).rejects.toThrow(
|
||||
"effective relationship snapshot requires a current full schema synchronization",
|
||||
);
|
||||
expect(list).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -127,7 +127,7 @@ test("resolves only a configured selection for the later generation boundary", (
|
||||
expect(() => models.resolve("unknown-model")).toThrow(MetadataGenerationModelUnavailableError);
|
||||
});
|
||||
|
||||
test("loads DeepSeek models, GLM, and an explicit keyless Qwen endpoint from installation setup", () => {
|
||||
test("loads DeepSeek, GLM, and an explicit keyless Qwen endpoint from installation setup", () => {
|
||||
const { installationFile, secretsFile } = metadataConfiguration(`metadataGeneration:
|
||||
default: glm-53
|
||||
models:
|
||||
@@ -135,10 +135,6 @@ test("loads DeepSeek models, GLM, and an explicit keyless Qwen endpoint from ins
|
||||
label: DeepSeek V4 Pro
|
||||
litellm: {provider: deepseek, model: deepseek-v4-pro}
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
- id: deepseek-v4-flash
|
||||
label: DeepSeek V4 Flash
|
||||
litellm: {provider: deepseek, model: deepseek-v4-flash}
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
- id: glm-53
|
||||
label: GLM 5.3
|
||||
litellm:
|
||||
@@ -160,7 +156,6 @@ test("loads DeepSeek models, GLM, and an explicit keyless Qwen endpoint from ins
|
||||
expect(models.catalog()).toEqual({
|
||||
models: [
|
||||
{ id: "deepseek-v4-pro", label: "DeepSeek V4 Pro" },
|
||||
{ id: "deepseek-v4-flash", label: "DeepSeek V4 Flash" },
|
||||
{ id: "glm-53", label: "GLM 5.3" },
|
||||
{ id: "qwen-36", label: "Qwen 3.6" },
|
||||
],
|
||||
|
||||
@@ -64,7 +64,7 @@ sys.stdout.write(json.dumps({"ok": True, "content": "Descrizione italiana"}))
|
||||
signal: new AbortController().signal,
|
||||
});
|
||||
|
||||
expect(content).toEqual({ content: "Descrizione italiana", usage: { input: 0, cacheRead: 0, output: 0 } });
|
||||
expect(content).toBe("Descrizione italiana");
|
||||
const captured = JSON.parse(readFileSync(join(roots[0]!, "request.json"), "utf8"));
|
||||
expect(captured.request).toEqual({
|
||||
model: "openai/gpt-4.1-mini",
|
||||
@@ -100,7 +100,7 @@ sys.stdout.write(json.dumps({"ok": True, "content": "Descrizione Qwen"}))
|
||||
},
|
||||
messages: [{ role: "user", content: "Describe invented metadata." }],
|
||||
signal: new AbortController().signal,
|
||||
})).resolves.toEqual({ content: "Descrizione Qwen", usage: { input: 0, cacheRead: 0, output: 0 } });
|
||||
})).resolves.toBe("Descrizione Qwen");
|
||||
|
||||
expect(JSON.parse(readFileSync(join(roots[0]!, "request.json"), "utf8"))).toEqual({
|
||||
model: "openai/qwen3.6-35b-a3b",
|
||||
|
||||
@@ -517,63 +517,6 @@ test("creates a session from the active immutable workspace revision", async ()
|
||||
}));
|
||||
});
|
||||
|
||||
test("hands one effective relationship snapshot to both retrieval and Pi", async () => {
|
||||
const effective = JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
workspaceId: "default",
|
||||
relationships: [],
|
||||
});
|
||||
const render = vi.fn(async () => effective);
|
||||
const acquireWorkspaceRuntime = vi.fn((_workspace: string, relationships?: string) => ({
|
||||
path: "/runtime/with-relationships.yaml",
|
||||
workspaceId: "default",
|
||||
workspaceRevision: "e".repeat(40),
|
||||
release: vi.fn(),
|
||||
}));
|
||||
const searchPack = vi.fn(async () => {});
|
||||
const createFor = vi.fn(() => ({ bridge: { onClientEvent: () => {} } }));
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
sessionNew: async () => ({ id: "effective-map" }),
|
||||
acquireWorkspaceRuntime,
|
||||
searchPack,
|
||||
} as any,
|
||||
effectiveRelationshipSnapshotProvider: { render } as any,
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
mgr: {
|
||||
get: () => undefined,
|
||||
createFor,
|
||||
configure: async () => {},
|
||||
start: () => {},
|
||||
} as any,
|
||||
getSettings: () => ({ workspace: "default", thinking: "low" }) as any,
|
||||
});
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: "/sessions",
|
||||
payload: { question: "Which users placed orders?", workspaceId: "default" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(render).toHaveBeenCalledWith("default");
|
||||
expect(acquireWorkspaceRuntime).toHaveBeenCalledWith(
|
||||
expect.stringContaining("/default.yaml"),
|
||||
effective,
|
||||
);
|
||||
expect(createFor).toHaveBeenCalledWith(
|
||||
"effective-map",
|
||||
expect.objectContaining({
|
||||
runtimeConfig: expect.objectContaining({ path: "/runtime/with-relationships.yaml" }),
|
||||
}),
|
||||
);
|
||||
expect(searchPack).toHaveBeenCalledWith(
|
||||
"Which users placed orders?",
|
||||
"effective-map",
|
||||
"/runtime/with-relationships.yaml",
|
||||
);
|
||||
});
|
||||
|
||||
test("rejects an SSH-only workspace before persisting or starting a session", async () => {
|
||||
const sessionNew = vi.fn(async () => ({ id: "must-not-exist" }));
|
||||
const ensure = vi.fn(async () => ({ ok: true }));
|
||||
|
||||
@@ -197,25 +197,6 @@ test("ThtRunner uses a vault secret only for the lifetime of its runtime lease",
|
||||
expect(existsSync(rendered.database.password_file)).toBe(false);
|
||||
});
|
||||
|
||||
test("ThtRunner binds and cleans the effective relationship snapshot with its runtime lease", async () => {
|
||||
const f = await fixture();
|
||||
const runner = runnerFor(f);
|
||||
const relationships = JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
workspaceId: "psd-clinical",
|
||||
relationships: [],
|
||||
});
|
||||
|
||||
const lease = runner.acquireWorkspaceRuntime(f.revision.snapshotPath, relationships);
|
||||
const rendered = parse(readFileSync(lease.path, "utf8")) as {
|
||||
paths: { effective_relationships: string };
|
||||
};
|
||||
|
||||
expect(readFileSync(rendered.paths.effective_relationships, "utf8")).toBe(relationships);
|
||||
lease.release();
|
||||
expect(existsSync(rendered.paths.effective_relationships)).toBe(false);
|
||||
});
|
||||
|
||||
test("separate runtime leases hand off byte-identical revision Evidence configs accepted by tht", async () => {
|
||||
const f = await fixture();
|
||||
const runner = runnerFor(f);
|
||||
|
||||
@@ -17,12 +17,6 @@ metadataGeneration:
|
||||
provider: deepseek
|
||||
model: deepseek-v4-pro
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
- id: deepseek-v4-flash
|
||||
label: DeepSeek V4 Flash
|
||||
litellm:
|
||||
provider: deepseek
|
||||
model: deepseek-v4-flash
|
||||
apiKeyEnv: DEEPSEEK_API_KEY
|
||||
- id: glm-53
|
||||
label: GLM 5.3
|
||||
litellm:
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
# Use the Catalog as the logical relationship authority
|
||||
|
||||
ThothII stores database-declared foreign keys and user-managed Logical Relationships in separate
|
||||
Catalog models, as required by ADR-0006, but exposes them through one effective relationship map.
|
||||
Physical relationships remain read-only and are refreshed from the database. Logical relationships
|
||||
are either Generated by deterministic column-name inference or Manual; inference does not call an
|
||||
AI model and does not inspect source values.
|
||||
|
||||
A generated rebuild is additive. It preserves active and Manual relationships, never reactivates a
|
||||
logically deleted relationship, and may recreate a relationship only after permanent deletion. A
|
||||
logical deletion is therefore represented by retaining the relationship with an exclusion marker;
|
||||
a permanent deletion removes it. Inference accepts only unambiguous, type-compatible matches to a
|
||||
single-column primary key and skips all other candidates. It recognizes normalized table-qualified
|
||||
names such as `user_id -> users.id`, exact non-generic primary-key names with one owner, and the
|
||||
warehouse convention `*time_key -> dim_time.<single PK>`. A source column may participate in a
|
||||
composite primary key; bare generic names such as `id`, `key`, `code`, and `pk` are not evidence by
|
||||
themselves.
|
||||
|
||||
An exclusion is durable while both Catalog Column endpoints exist. Explicit metadata cleanup of an
|
||||
endpoint table or column is a destructive boundary: it permanently removes every relationship and
|
||||
exclusion attached to that endpoint, invalidates the synchronized-catalog marker, and requires a
|
||||
full schema synchronization. The newly imported endpoints may then be inferred again. Preserving an
|
||||
exclusion across endpoint destruction would require a second denormalized name-based identity, which
|
||||
this design deliberately avoids.
|
||||
|
||||
The installation-local Catalog is the sole writable authority for Logical Relationships. Git-pinned
|
||||
workspace annotations remain authoritative for descriptive metadata but their embedded foreign keys
|
||||
are legacy compatibility data. The backend materializes the active effective map as an immutable,
|
||||
deterministic runtime snapshot when a session starts or resumes. When that snapshot is present, the
|
||||
harness uses it as the exclusive relationship source and ignores relationships embedded in both the
|
||||
physical schema artifact and workspace annotations; a missing or invalid declared snapshot fails
|
||||
closed. Legacy runtimes without a snapshot retain the previous merge behavior.
|
||||
|
||||
The snapshot is a projection, not another authored store. Its lifetime is tied to the runtime-config
|
||||
lease, physical relationships take precedence over duplicate Logical Relationships, composite-key
|
||||
column order is retained, and one Pi process observes one stable map for its complete lifetime.
|
||||
@@ -1,67 +0,0 @@
|
||||
# Use one Installation Model Catalog with runtime projections
|
||||
|
||||
ThothII currently declares model availability independently in installation
|
||||
`metadataGeneration`, Pi configuration, and workspace `llm_policy` and embedding settings. The
|
||||
Installation Model Catalog in `thothii-installation.yaml` becomes the sole authored authority for
|
||||
session, metadata-generation, and embedding models, including their allowed usages and per-usage
|
||||
defaults. Workspace descriptors retain database identity and scope plus Evidence concerns, but no
|
||||
model policy or selection; installation-local Database Bindings remain separate from them.
|
||||
|
||||
Pi, the backend metadata-generation helper, and the embedding runtime consume generated Model
|
||||
Runtime Projections of that catalog. Runtime Model Selection stores only a canonical catalog model
|
||||
identity and use-specific controls such as thinking level; endpoint, provider, capabilities, and
|
||||
credential references remain catalog facts, while secret values remain in protected secret stores.
|
||||
The metadata-generation helper continues to use LiteLLM independently of Pi, as established by
|
||||
ADR-0009: unifying model declaration does not unify execution lifecycles.
|
||||
|
||||
The migration is intentionally fail-closed. Workspace schema v4 removes `llm_policy` and the
|
||||
entire redundant `semantic_index`; collection identity is derived from the workspace identity,
|
||||
while vector-store and embedding facts come from the installation. A deterministic migration
|
||||
rewrites existing descriptors. The
|
||||
installation loader replaces `metadataGeneration` with `modelCatalog` and rejects the legacy form
|
||||
with an actionable migration error rather than keeping two live sources. Existing Pi
|
||||
`models.json` and enabled-model settings become generated artifacts and are never edited as
|
||||
authoritative configuration.
|
||||
|
||||
Catalog identities use the canonical `provider/model` form; runtime-specific upstream names are
|
||||
adapter facts, not additional ThothII identities. Installation descriptors use schema version 2
|
||||
and model-free workspace descriptors use schema version 4. Removing a model never substitutes it
|
||||
inside an existing session: an unresolvable resume fails explicitly. Published semantic indexes
|
||||
record the embedding identity and dimensions that produced them and require explicit
|
||||
reprocessing when those facts change.
|
||||
|
||||
Model eligibility is expressed by the presence of a `session` or `metadataGeneration` block,
|
||||
without a duplicate usages list. The installation declares one active embedding identity and its
|
||||
dimensions rather than a selectable embedding catalog. Runtime projections are regenerated
|
||||
deterministically and atomically at start, so they require no persisted digest and are excluded
|
||||
from installation backups; restore regenerates them from the validated installation descriptor.
|
||||
|
||||
A provider owns one endpoint, one explicit authentication mode, and only the runtime adapters it
|
||||
needs. Authentication is either a protected secret-environment reference, Pi-owned authentication
|
||||
for session-only built-in models, or explicit keyless operation for an explicit endpoint; models
|
||||
cannot override it. Pi built-in model facts are not copied into the installation. A session default
|
||||
and the single embedding definition are required, while metadata generation and its default may be
|
||||
omitted together. The schema deliberately excludes unused abstractions and future properties until
|
||||
runtime behavior requires them.
|
||||
|
||||
The catalog session default replaces `PI_PROVIDER`, `PI_MODEL`, and persisted installation model
|
||||
defaults as configuration sources. A user or session selection is only a canonical catalog
|
||||
reference, and an existing session keeps that reference without silently switching models. A
|
||||
generated Compose projection supplies the catalog-derived embedding values and projection mounts
|
||||
to every affected service, so neither the base Compose files nor `operator.env` repeat model facts.
|
||||
|
||||
Workspace v3-to-v4 migration is a deterministic removal of `llm_policy` and `semantic_index`.
|
||||
Installation migration instead inspects the legacy installation metadata block and both Pi model
|
||||
files: it emits a v2 candidate only when their identities and settings can be reconciled without
|
||||
guessing. Conflicts produce an actionable report and leave every source untouched.
|
||||
|
||||
## Considered Options
|
||||
|
||||
- A separate catalog file referenced by the installation was rejected because it adds path,
|
||||
permission, backup, and atomic-update coordination without a current need for cross-installation
|
||||
sharing.
|
||||
- Pi `models.json` was rejected as the authority because it is a Pi-specific projection that does
|
||||
not express all ThothII usages, built-in providers, metadata-generation controls, or embedding
|
||||
facts.
|
||||
- Transitional dual reading was rejected because it would preserve the configuration discrepancy
|
||||
this decision is intended to eliminate.
|
||||
@@ -1,52 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
---
|
||||
|
||||
# Assess sensitive columns locally from source content
|
||||
|
||||
The Sensitive Data Flag remains a human-owned boolean. An explicit, selection-scoped sensitivity
|
||||
analysis may propose changes by inspecting both catalog metadata and source values, but it never
|
||||
writes the flag. The administrator may accept, reject, or reverse every proposal.
|
||||
|
||||
One TypeScript `SensitivityClassifier` is the only component allowed to produce the column-level
|
||||
assessment `sensitive`, `non_sensitive`, or `unknown`. It applies a versioned Sensitive Data Policy
|
||||
and consumes values through database-independent streaming adapters. Database-specific code may
|
||||
read and normalize bounded values, but it may not decide sensitivity.
|
||||
|
||||
The classifier first applies deterministic metadata rules, value validators, checksums,
|
||||
dictionaries, and length rules. A single validated sensitive match makes the whole column
|
||||
`sensitive`; any textual value longer than 500 characters is such a match. It attempts a complete
|
||||
scan, but after five seconds per table it continues by sampling within the remaining run budget. A
|
||||
completed scan with no finding may produce `non_sensitive`; an incomplete scan with no finding
|
||||
produces `unknown`.
|
||||
|
||||
Ambiguous text may additionally be sent to an optional local NER detector only while time remains.
|
||||
The detector runs on CPU, receives no tools or network access, does not persist source values, and
|
||||
returns evidence rather than the column decision. The initial supported detector is
|
||||
[`fastino/gliner2-privacy-filter-PII-multi`](https://huggingface.co/fastino/gliner2-privacy-filter-PII-multi),
|
||||
used through the Apache-2.0 GLiNER2 Python library with a pinned model revision. Its model weights
|
||||
and GLiNER2 code are Apache-2.0, and its mDeBERTa base model is MIT. It is trained for seven
|
||||
languages including Italian and can run on CPU without using the installation's GPUs.
|
||||
|
||||
The NER detector is an optional installation asset because its weights and runtime are materially
|
||||
larger than the deterministic TypeScript engine. It is invoked only for otherwise unresolved text,
|
||||
never for values already classified by a decisive rule. If it is disabled, unavailable, times out,
|
||||
or returns no qualifying evidence before the deadline, the classifier follows the same coverage
|
||||
rule and may return `unknown`.
|
||||
|
||||
No generative LLM, internal or external, participates in sensitivity assessment. A fallback to an
|
||||
installation-local LLM is unnecessary while a permissively licensed local NER implementation is
|
||||
available, and would reintroduce queue latency, non-deterministic judgments, prompt-injection
|
||||
surface, and contention with normal inference. Introducing such a fallback would require a new
|
||||
decision based on evidence that the NER path is unusable.
|
||||
|
||||
This decision supersedes ADR-0011 only where that ADR assigns draft sensitivity suggestions to an
|
||||
AI using structural metadata. ADR-0011's human authority, transient draft, explicit scope, and
|
||||
Sensitive Data Flag remain in force. How description generation uses the flag is outside this
|
||||
decision.
|
||||
|
||||
The selected model's published evaluation is not an Italian production acceptance test. Before
|
||||
enabling the NER profile by default, ThothII must pin the artifacts, generate a dependency/license
|
||||
inventory, and pass a CPU benchmark plus a labeled Italian corpus representative of the target
|
||||
databases. Failure of those gates disables NER; it does not silently select another model or an
|
||||
LLM.
|
||||
@@ -108,7 +108,6 @@ not supported by this installation until the declarative contract is extended ge
|
||||
| Model | Level | Why | Visibility |
|
||||
|---|---|---|---|
|
||||
| `deepseek/deepseek-v4-pro` | Built-in Pi | Known public API, already in the build | All projects |
|
||||
| `deepseek/deepseek-v4-flash` | Built-in Pi | Known public API, already in the build | All projects |
|
||||
| `zai/glm-5.3` | `deploy/pi/models.json` | Custom OpenAI-compatible endpoint | ThothII installation |
|
||||
| `local-qwen/qwen3.6-35b-a3b` | `deploy/pi/models.json` | Locally configured OpenAI-compatible endpoint | ThothII installation |
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ session workflow.
|
||||
|
||||
## What the catalog owns
|
||||
|
||||
For each YAML workspace, an administrator may configure at most one Metadata Catalog binding. It holds
|
||||
For each YAML workspace, an administrator may create at most one database configuration. It holds
|
||||
the database name, schema, connection binding, write-only encrypted secrets, observed physical
|
||||
schema, optional curated descriptions, generated descriptions, and durable operation history.
|
||||
|
||||
@@ -27,18 +27,6 @@ It uses `GET /catalog/metrics` without `databaseId` for installation totals and
|
||||
the current database. Choose a selection-scoped operation from the action selector and then press
|
||||
**Run**; unavailable operations remain listed with an explanation. Row-specific actions are the icon
|
||||
controls in the final column, and each navigation or action icon has an immediate conceptual tooltip.
|
||||
An unconfigured workspace exposes **Configure catalog** directly on its row; there is no global
|
||||
database-creation action and the selected workspace cannot be changed in the configuration form.
|
||||
|
||||
The master grid keeps three independent states visible:
|
||||
|
||||
- **Revision / Evidence** comes from the active immutable workspace revision. Filesystem Evidence is
|
||||
materialized with that revision; remote Evidence is reported as configured-but-unverified or as
|
||||
requiring credentials.
|
||||
- **NL→SQL runtime** is calculated from the workspace DWH/Evidence requirements and runtime secret
|
||||
store. It also reports transports, such as SSH, that are diagnostic-only and unsupported by sessions.
|
||||
- **Metadata Catalog** reports whether the installation-local catalog configuration exists, then shows
|
||||
its separately versioned connection-test or synchronization state.
|
||||
|
||||
Configuration, object details, metadata editors, synchronization history, description history,
|
||||
sensitive-field review, and suggestion-run history open in right-side drawers backed by the
|
||||
@@ -54,9 +42,8 @@ remains available only until the integrated Fleet Ledger surface passes owner ac
|
||||
|
||||
## Configure and test a database
|
||||
|
||||
1. Open **Database Management** and find the repository workspace marked **Not configured**.
|
||||
2. Choose **Configure catalog** on that row. Configure its PostgreSQL catalog binding with
|
||||
`postgres_direct`, `rest_api`, or `ssh_tunnel` and
|
||||
1. Open **Database Management** and choose a workspace.
|
||||
2. Create its PostgreSQL configuration. Choose `postgres_direct`, `rest_api`, or `ssh_tunnel` and
|
||||
complete the binding fields that the chosen transport requires.
|
||||
3. Enter secrets only when replacing them. They remain write-only and are never returned by the
|
||||
application.
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
# ThothII Docker refresh
|
||||
|
||||
The active local `psd-local` stack uses the operator environment generated for the installation:
|
||||
|
||||
`deploy/psd/operator.env`
|
||||
|
||||
It is not `deploy/thothii.env` (that file is empty) and `deploy/env/local.env` is only an example
|
||||
path referenced by the generic launcher. The running stack also uses these Compose overlays:
|
||||
|
||||
```text
|
||||
compose.yaml
|
||||
deploy/compose.local.yaml
|
||||
deploy/compose.git-ssh.yaml
|
||||
deploy/psd/connector-secrets.yaml
|
||||
```
|
||||
|
||||
Refresh the stack from the repository root with:
|
||||
|
||||
```bash
|
||||
compose_psd=(
|
||||
docker compose
|
||||
--env-file deploy/psd/operator.env
|
||||
-p thothii-18998cca7b0a
|
||||
-f compose.yaml
|
||||
-f deploy/compose.local.yaml
|
||||
-f deploy/compose.git-ssh.yaml
|
||||
-f deploy/psd/connector-secrets.yaml
|
||||
)
|
||||
|
||||
"${compose_psd[@]}" config --quiet
|
||||
"${compose_psd[@]}" build core frontend
|
||||
"${compose_psd[@]}" stop core frontend
|
||||
"${compose_psd[@]}" up -d catalog-db
|
||||
"${compose_psd[@]}" run --rm catalog-migrate
|
||||
"${compose_psd[@]}" up -d --remove-orphans
|
||||
```
|
||||
|
||||
Building before the stop keeps the existing application available if an image fails to compile.
|
||||
Stopping only `core` and `frontend` prevents the old backend from using a newly migrated catalog;
|
||||
the database, Qdrant, embedding service, named volumes, and installation state remain in place.
|
||||
|
||||
The installation secret files referenced by that env file live under `deploy/psd/secrets/` and
|
||||
must never be committed or printed.
|
||||
@@ -109,13 +109,12 @@ canonico; i percorsi e i comportamenti descrivono il sorgente disponibile il 202
|
||||
conserva il valore esistente e la sostituzione è un'azione esplicita. Delete rimuove anche i
|
||||
segreti associati.
|
||||
34. La pagina usa AG Grid come master e un form React come detail, con sezioni Database, Connection
|
||||
e TLS/SSH condizionali. Non esiste un'azione globale `Add database`: ogni riga `unconfigured`
|
||||
offre `Configure catalog`, apre il form già vincolato a quello specifico workspace YAML e crea il
|
||||
record soltanto al Save; `workspace_id` non è selezionabile né modificabile.
|
||||
35. La grid mostra separatamente revisione/Evidence del workspace, binding runtime NL→SQL e
|
||||
configurazione del Metadata Catalog, oltre a database, schema, endpoint e ultimo aggiornamento.
|
||||
Su schermi piccoli il dettaglio occupa il pannello completo. Il cambio riga con modifiche non
|
||||
salvate e Delete richiedono conferma, senza conferma testuale tipizzata.
|
||||
e TLS/SSH condizionali. La toolbar offre `Add database`; le righe `unconfigured` offrono
|
||||
`Configure`. Entrambe selezionano esclusivamente workspace YAML senza un database e creano il
|
||||
record soltanto al Save; `workspace_id` diventa immutabile dopo la creazione.
|
||||
35. La grid mostra workspace, database, schema, transport, endpoint, stato connessione e ultimo
|
||||
aggiornamento. Su schermi piccoli il dettaglio occupa il pannello completo. Il cambio riga con
|
||||
modifiche non salvate e Delete richiedono conferma, senza conferma testuale tipizzata.
|
||||
36. La Database Binding conserva `connection_status`, `tested_version`, `last_tested_at`, un codice
|
||||
errore e un messaggio breve sanificato. Non conserva stack trace, DSN, credenziali o output grezzo
|
||||
del driver.
|
||||
@@ -179,12 +178,12 @@ canonico; i percorsi e i comportamenti descrivono il sorgente disponibile il 202
|
||||
e Schema Sync. Il renderer e il runtime delle sessioni NL→SQL restano fuori scope e continuano a
|
||||
rifiutarla finché non verrà deciso il relativo cutover.
|
||||
55. I menu di azione a livello Workspace Database espongono separatamente `Synchronize tables`,
|
||||
`Synchronize relationships` e `Synchronize all`. Su una selezione di
|
||||
`Synchronize all columns`, `Synchronize relationships` e `Synchronize all`. Su una selezione di
|
||||
database lo scope scelto viene avviato per ogni database idoneo; non viene sostituito
|
||||
implicitamente con una sincronizzazione completa.
|
||||
56. Lo scope Columns è disponibile dalla grid Tables e limita la riconciliazione alle tabelle
|
||||
selezionate; la pagina Columns non espone azioni di sincronizzazione. La grid Tables espone
|
||||
`Synchronize columns` sulle tabelle selezionate.
|
||||
56. Per lo scope Columns, `tableIds` vuoto significa tutte le Catalog Table correnti del Workspace
|
||||
Database; `tableIds` valorizzato limita invece la riconciliazione alle tabelle indicate. La grid
|
||||
Tables espone `Synchronize columns` sulle tabelle selezionate.
|
||||
|
||||
## Correzione del modello mentale corrente
|
||||
|
||||
@@ -570,9 +569,9 @@ Le griglie che dispongono di azioni massive usano checkbox e una toolbar contest
|
||||
menu `Actions` e cancellazione della selezione. La selezione identifica ID espliciti, può essere
|
||||
accumulata attraverso i filtri e viene azzerata dopo successo, nuova sincronizzazione o uscita
|
||||
dalla pagina; un'azione è all-or-nothing se un elemento non è idoneo. I menu a livello database
|
||||
espongono gli scope fisici come azioni distinte: `Synchronize tables`, `Synchronize relationships`
|
||||
e `Synchronize all`. La grid Tables espone invece `Synchronize columns` per le tabelle selezionate;
|
||||
la pagina Columns non espone sincronizzazione. Le selezioni database aggiungono `Delete all tables` e
|
||||
espongono gli scope fisici come azioni distinte: `Synchronize tables`, `Synchronize all columns`,
|
||||
`Synchronize relationships` e `Synchronize all`. La grid Tables espone invece `Synchronize
|
||||
columns` per le tabelle selezionate. Le selezioni database aggiungono `Delete all tables` e
|
||||
`Delete all relationships`; le selezioni tabelle aggiungono `Delete all columns` e `Delete all
|
||||
relationships`. Queste operazioni sono atomiche, richiedono conferma e non modificano database
|
||||
esterno, binding, configurazione o segreti. Test connection resta un'azione distinta; griglie senza
|
||||
|
||||
@@ -1,245 +0,0 @@
|
||||
# Installation Model Catalog
|
||||
|
||||
Status: accepted design; implementation not started.
|
||||
|
||||
## Outcome
|
||||
|
||||
`thothii-installation.yaml` is the only operator-authored source for models used by interactive
|
||||
sessions, metadata generation, and embedding. Runtime-specific files are deterministic projections,
|
||||
not additional configuration sources. Workspace descriptors contain database and Evidence concerns
|
||||
and no model, provider, allowlist, default, embedding, or vector-store configuration.
|
||||
|
||||
This design does not merge execution lifecycles. Pi continues to run interactive sessions, the
|
||||
short-lived LiteLLM helper continues to perform metadata generation, and the internal Ollama service
|
||||
continues to provide embeddings. They share model declaration, not execution machinery.
|
||||
|
||||
## Canonical installation shape
|
||||
|
||||
The following example covers all currently required cases: a Pi built-in model, an authenticated
|
||||
custom endpoint, a keyless internal endpoint, metadata generation, and the single embedding model.
|
||||
|
||||
```yaml
|
||||
schemaVersion: 2
|
||||
profile: server
|
||||
projectDirectory: /srv/thothii
|
||||
envFile: /srv/thothii/operator.env
|
||||
|
||||
workspaceRepository:
|
||||
remote: git@git.example.com:organization/workspaces.git
|
||||
branch: main
|
||||
access: ssh
|
||||
|
||||
modelCatalog:
|
||||
defaults:
|
||||
session: zai/glm-5.3
|
||||
metadataGeneration: local-qwen/qwen3.6-35b-a3b
|
||||
|
||||
embedding:
|
||||
id: ollama/qwen3-embedding:0.6b
|
||||
dimensions: 1024
|
||||
|
||||
providers:
|
||||
deepseek:
|
||||
authentication:
|
||||
mode: pi_auth
|
||||
session:
|
||||
mode: pi_builtin
|
||||
models:
|
||||
deepseek-v4-pro:
|
||||
session: {}
|
||||
deepseek-v4-flash:
|
||||
session: {}
|
||||
|
||||
zai:
|
||||
endpoint:
|
||||
baseUrl: https://api.z.ai/api/coding/paas/v4
|
||||
authentication:
|
||||
mode: secret_env
|
||||
apiKeyEnv: ZAI_API_KEY
|
||||
session:
|
||||
mode: openai_compatible
|
||||
metadataGeneration:
|
||||
litellmProvider: openai
|
||||
models:
|
||||
glm-5.3:
|
||||
label: GLM-5.3
|
||||
session:
|
||||
reasoning: true
|
||||
contextWindow: 200000
|
||||
maxTokens: 131072
|
||||
metadataGeneration: {}
|
||||
|
||||
local-qwen:
|
||||
endpoint:
|
||||
baseUrl: https://ml-aritmolab.policlinicosandonato.it/v1
|
||||
authentication:
|
||||
mode: none
|
||||
session:
|
||||
mode: openai_compatible
|
||||
metadataGeneration:
|
||||
litellmProvider: openai
|
||||
models:
|
||||
qwen3.6-35b-a3b:
|
||||
label: Qwen3.6 35B A3B
|
||||
session:
|
||||
reasoning: false
|
||||
contextWindow: 131072
|
||||
maxTokens: 16384
|
||||
compatibility:
|
||||
supportsDeveloperRole: false
|
||||
supportsReasoningEffort: false
|
||||
supportsStore: false
|
||||
maxTokensField: max_tokens
|
||||
metadataGeneration:
|
||||
disableThinking: true
|
||||
|
||||
authentication:
|
||||
configDirectory: /srv/thothii/auth-canonical
|
||||
runtimeProjection:
|
||||
directory: /srv/thothii/auth-runtime
|
||||
uid: 10001
|
||||
gid: 10001
|
||||
```
|
||||
|
||||
The catalog uses maps instead of repeated IDs. The canonical identity of a model is always derived
|
||||
as `<provider-key>/<model-key>`. `upstreamModel` may be added to a model only when the endpoint uses
|
||||
a different identifier. `label` is optional and falls back to the canonical identity.
|
||||
|
||||
Model eligibility is not repeated in an `usages` array. A `session` block makes the model eligible
|
||||
for sessions; a `metadataGeneration` block makes it eligible for metadata generation. The embedding
|
||||
is a single required installation value rather than a list plus default.
|
||||
|
||||
## Provider and authentication rules
|
||||
|
||||
A provider owns one endpoint, one authentication mode, and zero or one adapter for each runtime.
|
||||
Model entries cannot override provider endpoint or credentials. If the same upstream service needs
|
||||
different endpoints or credentials, the installation declares two provider identities.
|
||||
|
||||
Supported session modes are intentionally closed:
|
||||
|
||||
- `pi_builtin`: Pi already owns the model's technical descriptor; the model's `session` block is
|
||||
empty and ThothII does not copy context-window or compatibility facts.
|
||||
- `openai_compatible`: ThothII generates a Pi custom-provider descriptor; each session model supplies
|
||||
the technical values required by Pi.
|
||||
|
||||
Metadata generation uses the provider-level `litellmProvider`. A model-level
|
||||
`metadataGeneration.disableThinking: true` is permitted only for an explicit compatible endpoint.
|
||||
There is no generic adapter or plugin abstraction in schema version 2.
|
||||
|
||||
Exactly one provider authentication mode is allowed:
|
||||
|
||||
- `secret_env` requires an approved API-key environment reference present in the protected secret
|
||||
bundle. Secret values never enter YAML, generated files, logs, arguments, or API responses.
|
||||
- `pi_auth` is valid only for session-only `pi_builtin` providers and resolves through Pi's protected
|
||||
authentication projection.
|
||||
- `none` is valid only for an explicit endpoint. Runtime projections may supply a fixed non-secret
|
||||
compatibility placeholder when a client library requires a non-empty key.
|
||||
|
||||
## Defaults and selections
|
||||
|
||||
`defaults.session` and `embedding` are required. `defaults.metadataGeneration` is required exactly
|
||||
when at least one model has a `metadataGeneration` block; metadata generation may otherwise be
|
||||
absent and its UI controls are disabled.
|
||||
|
||||
`modelCatalog.defaults.session` is the only configured session-model default. `PI_PROVIDER`,
|
||||
`PI_MODEL`, and provider/model fields in installation-default settings are removed. A user choice is
|
||||
a Model Selection containing only the canonical model identity and runtime controls such as thinking
|
||||
level. A session manifest pins the selected canonical identity.
|
||||
|
||||
Removing the currently selected model causes new-session selection to fall back to the catalog
|
||||
default with an explicit administrative warning. An existing session is never silently moved to a
|
||||
different model; resume fails with `model_unavailable` when its pinned identity can no longer be
|
||||
resolved.
|
||||
|
||||
## Generated runtime projections
|
||||
|
||||
Before Compose starts, `tht` strictly validates schema version 2 and generates installation-local
|
||||
artifacts below `deploy/<installation-id>/generated/`:
|
||||
|
||||
- a normalized catalog JSON consumed defensively by the backend;
|
||||
- Pi `models.json` for custom providers;
|
||||
- Pi `settings.json`, combining fixed product settings with the session-eligible canonical IDs;
|
||||
- a Compose override that mounts the projections and supplies embedding identity and dimensions to
|
||||
core, preprocessing, and `embedding-model-init`.
|
||||
|
||||
Generation is deterministic and published only after every candidate artifact validates. A failed
|
||||
generation aborts start before Compose is invoked. `tht doctor` recomputes expected bytes and reports
|
||||
differences; no digest manifest or separate apply command exists. When projection bytes change,
|
||||
`tht start` recreates the affected services so they cannot continue with an older bind mount.
|
||||
|
||||
Generated projections are not backed up. Restore validates the canonical installation descriptor,
|
||||
regenerates every projection, and only then starts services. Base Compose files and `operator.env`
|
||||
must contain no model identities, defaults, endpoints, or dimensions.
|
||||
|
||||
## Workspace schema v4
|
||||
|
||||
Workspace schema v4 removes both top-level `llm_policy` and `semantic_index`. The entire latter
|
||||
block is redundant today: its engine and distance are product constants, its collection duplicates
|
||||
the workspace ID, and its model and dimensions are installation facts.
|
||||
|
||||
The runtime derives:
|
||||
|
||||
- Qdrant collection identity from the workspace ID;
|
||||
- engine and distance from the supported product contract;
|
||||
- embedding identity and dimensions from the Installation Model Catalog.
|
||||
|
||||
The published index generation records the canonical embedding identity and dimensions that created
|
||||
it. A mismatch makes the index explicitly incompatible and requires operator-triggered
|
||||
preprocessing. No existing index is deleted or rebuilt automatically.
|
||||
|
||||
The v3-to-v4 workspace migration is deterministic: set `workspace.schema_version` to `4`, remove
|
||||
`llm_policy`, and remove `semantic_index`. It does not alter database, Evidence, diagnostics, or
|
||||
binding data.
|
||||
|
||||
## Installation migration
|
||||
|
||||
Legacy installation migration must inspect all three former sources:
|
||||
|
||||
1. `metadataGeneration` in `thothii-installation.yaml`;
|
||||
2. `deploy/pi/models.json`;
|
||||
3. `deploy/pi/settings.json`.
|
||||
|
||||
The migrator emits a version-2 candidate only when it can reconcile identities, endpoints,
|
||||
credentials, and runtime-specific facts without guessing. Ambiguous aliases such as `glm-53`,
|
||||
`zai/glm-5.3`, and `openai/glm-5.3` are not silently equated. A conflict produces a field-level
|
||||
report and leaves every input unchanged for operator resolution.
|
||||
|
||||
After migration, the strict loader rejects `metadataGeneration`, workspace `llm_policy`, workspace
|
||||
`semantic_index`, legacy Pi source files, unknown fields, duplicate YAML keys, invalid defaults, and
|
||||
incompatible authentication/adapter combinations with an actionable `migration_required` or
|
||||
validation error.
|
||||
|
||||
## Final simplicity audit
|
||||
|
||||
The accepted design removes every configuration duplication that can be removed without inference:
|
||||
|
||||
- one authored installation file instead of an installation block plus two Pi files;
|
||||
- one canonical `provider/model` identity instead of display IDs and runtime IDs;
|
||||
- per-use blocks instead of a duplicated usages list;
|
||||
- one embedding entry instead of a selectable embedding catalog;
|
||||
- one catalog session default instead of environment and settings defaults;
|
||||
- no model or vector-store fields in workspace descriptors;
|
||||
- provider-level credentials instead of per-model credentials;
|
||||
- no generic runtime-plugin abstraction;
|
||||
- no persisted digest, apply command, or backup of generated projections.
|
||||
|
||||
The remaining generated files are necessary boundary adapters, not configuration concepts. Making
|
||||
the backend parse the authoring YAML independently would remove one file but restore two semantic
|
||||
validators. Hard-coding embedding values in Compose would remove one projection but restore a model
|
||||
source outside the catalog. Inferring authentication from missing fields would save one YAML key but
|
||||
turn a safe explicit choice into ambiguity. These apparent simplifications are therefore rejected.
|
||||
|
||||
No further reduction was found that preserves one authority, strict validation, explicit security,
|
||||
session determinism, and model-free workspaces.
|
||||
|
||||
## Implementation surface
|
||||
|
||||
Implementation must update the host `tht` installation loader, setup and lifecycle projection,
|
||||
doctor, backup/restore, Compose mounts and embedding inputs, backend catalog/settings/session model
|
||||
resolution, workspace schema and migration, runtime rendering and diagnostics, frontend workspace
|
||||
drafts and model filtering, examples, fixtures, and documentation. Existing session manifests remain
|
||||
readable and keep their pinned provider/model identity; only resume resolution changes to the new
|
||||
catalog.
|
||||
|
||||
This document authorizes design only. Software implementation begins only after a separate explicit
|
||||
request.
|
||||
@@ -1,299 +0,0 @@
|
||||
# Visualizzatore ERD nel browser: proposta solo open source
|
||||
|
||||
Data dell'audit: 31 agosto 2026.
|
||||
|
||||
## Decisione
|
||||
|
||||
Per ThothII sceglierei **AntV X6 + ELK.js** come fondazione del visualizzatore ERD.
|
||||
|
||||
- **AntV X6** è MIT, non ha una distinta edizione Professional e include nel progetto OSS
|
||||
le funzioni che servono davvero a un diagramma complesso: nodi e porte personalizzabili,
|
||||
pan/zoom, selezione, minimappa, history, clipboard, tastiera, routing, virtual rendering ed
|
||||
export SVG/PNG/JPEG.
|
||||
- **ELK.js** è il motore di layout automatico, EPL-2.0 e senza tier commerciale. Il layout
|
||||
layered gestisce porte, vincoli sull'ordine delle porte, archi multipli, self-loop e routing
|
||||
ortogonale: proprietà importanti per foreign key composite e schemi affollati.
|
||||
- La combinazione non dipende da esempi o componenti a pagamento. Il lavoro applicativo che
|
||||
resta da fare — semantica ER, livelli di dettaglio, filtri e adattatore X6/ELK — appartiene
|
||||
davvero al dominio di ThothII e non è una funzione nascosta dietro un piano Pro.
|
||||
|
||||
La seconda scelta è **React Flow + ELK.js**. È probabilmente l'integrazione più naturale con
|
||||
l'attuale frontend React e ha una migliore storia documentata per accessibilità. Il runtime è
|
||||
MIT e non viene tecnicamente depotenziato, ma diversi pattern avanzati già implementati
|
||||
(auto-layout, edge routing, undo/redo, copy/paste, expand/collapse) sono pubblicati come esempi
|
||||
con licenza React Flow Pro. Non è quindi la scelta più lineare se il criterio prioritario è
|
||||
evitare anche una dipendenza progettuale da materiale Professional.
|
||||
|
||||
## Cosa significa «open source senza limitazioni Professional»
|
||||
|
||||
L'audit distingue tre casi:
|
||||
|
||||
1. **OSS completo**: licenza OSI e nessun tier commerciale che trattenga funzioni essenziali.
|
||||
2. **OSS con caveat**: il motore è aperto, ma esistono esempi premium, API instabili o limiti
|
||||
architetturali rilevanti. Può essere usato, purché il limite sia accettato esplicitamente.
|
||||
3. **Escluso**: licenza proprietaria oppure core aperto con funzioni necessarie al viewer
|
||||
complesso riservate alla versione commerciale.
|
||||
|
||||
La valutazione riguarda sia la licenza sia ciò che si può effettivamente costruire senza
|
||||
acquistare un prodotto complementare. Le note sulle licenze sono tecniche, non consulenza legale.
|
||||
|
||||
## Matrice di selezione
|
||||
|
||||
| Soluzione | Licenza OSS | Tier Pro rilevante | Adeguatezza a ERD complessi | Verdetto |
|
||||
|---|---|---|---|---|
|
||||
| **AntV X6 + ELK.js** | MIT + EPL-2.0 | Nessuno individuato | Porte per colonna, minimappa, routing, export, virtual rendering e layout avanzato | **Consigliata** |
|
||||
| **React Flow + ELK.js** | MIT + EPL-2.0 | Esempi/pattern avanzati Pro, non runtime separato | Ottima UX React, nodi HTML, minimappa e accessibilità; export SVG non nativo | **Valida con caveat** |
|
||||
| **maxGraph** | Apache-2.0 | Nessuno | Molto completo, SVG, porte, folding e layout; API imperativa e integrazione React costosa | **Valida con caveat** |
|
||||
| **Cytoscape.js + ELK** | MIT | Nessuno | Molto scalabile come grafo, ma meno adatto a tabelle ricche e porte per campo | **Alternativa di nicchia** |
|
||||
| **Liam ERD/CLI** | Apache-2.0 | Nessuno | Viewer ERD già rifinito e self-hostable; il package embeddable è interno e instabile | **Reference o app separata** |
|
||||
| **Graphviz + Viz.js** | EPL-2.0 + MIT | Nessuno | Layout ed SVG eccellenti; non offre da solo un explorer applicativo | **Renderer di export** |
|
||||
| **Mermaid.js + Panzoom** | MIT + MIT/ISC | Mermaid Chart è separato | Facile, ma insufficiente per interazioni e modelli ER ricchi | **Solo preview semplice** |
|
||||
| **Sprotty/GLSP** | EPL-2.0 | Nessuno | Potente e completo, ma è un framework di modeling più pesante del necessario | **Non prioritario** |
|
||||
| **JointJS** | Core MPL-2.0 | Molte funzioni utili sono in JointJS+ | Lo split commerciale intercetta proprio le necessità del viewer | **Esclusa** |
|
||||
| **GoJS** | Proprietaria | Licenza di deployment | Completa tecnicamente, ma non open source | **Esclusa** |
|
||||
| **yFiles** | Proprietaria | Licenza commerciale | Completa tecnicamente, ma non open source | **Esclusa** |
|
||||
|
||||
## 1. Scelta principale: AntV X6 + ELK.js
|
||||
|
||||
### Funzioni disponibili nell'open source
|
||||
|
||||
X6 offre un canvas diagrammatico basato su SVG con supporto anche a nodi HTML/React. Il core
|
||||
espone porte, archi personalizzabili, self-loop e multiedge. I plugin distribuiti dal progetto
|
||||
comprendono Scroller, MiniMap, Selection, History, Clipboard, Keyboard, DnD, Stencil, Snapline,
|
||||
Transform ed Export. La documentazione copre inoltre router `orth`, `manhattan` ed `er`, zoom e
|
||||
panning. Non è emersa un'edizione X6 Pro che renda a pagamento queste capacità.
|
||||
|
||||
Fonti primarie: [repository e licenza X6](https://github.com/antvis/X6),
|
||||
[porte](https://x6.antv.antgroup.com/en/tutorial/basic/port),
|
||||
[router](https://x6.antv.antgroup.com/en/api/registry/router),
|
||||
[minimappa](https://x6.antv.antgroup.com/en/tutorial/plugins/minimap),
|
||||
[scroller](https://x6.antv.antgroup.com/en/tutorial/plugins/scroller) ed
|
||||
[export](https://x6.antv.antgroup.com/en/tutorial/plugins/export).
|
||||
|
||||
ELK non è un renderer: calcola la geometria del grafo. L'algoritmo layered supporta porte e
|
||||
relativi vincoli, archi ortogonali, self-loop e archi multipli. Va eseguito in un Web Worker per
|
||||
non bloccare l'interfaccia sui modelli grandi. L'adattatore dovrà passare a ELK le porte delle
|
||||
colonne e usare anche le `edge sections` e i bend point restituiti, non soltanto le coordinate
|
||||
dei nodi.
|
||||
|
||||
Fonti primarie: [ELK.js e licenza](https://github.com/kieler/elkjs) e
|
||||
[ELK layered](https://eclipse.dev/elk/reference/algorithms/org-eclipse-elk-layered.html).
|
||||
|
||||
### Limiti reali
|
||||
|
||||
- X6 è più imperativo di React Flow: conviene incapsularlo in un singolo componente React con
|
||||
un adapter stabile, evitando di spargere istanze e listener nel resto dell'applicazione.
|
||||
- L'accessibilità per screen reader non è documentata al livello di React Flow; tab order,
|
||||
focus, descrizioni ARIA e navigazione da tastiera richiedono test e lavoro applicativo.
|
||||
- Un nodo React/HTML può introdurre `foreignObject` nell'SVG. Per un export portabile e
|
||||
stampabile è meglio produrre un SVG separato dallo stesso modello e dalla geometria ELK.
|
||||
|
||||
Questi sono costi tecnici, non limitazioni commerciali.
|
||||
|
||||
## 2. Seconda scelta: React Flow + ELK.js
|
||||
|
||||
`@xyflow/react` è MIT. Custom nodes React, handle multipli, pan/zoom, fit view, Controls,
|
||||
MiniMap, selezione e funzioni di accessibilità sono nel runtime OSS. React Flow Pro vende
|
||||
supporto, template ed esempi con relativo codice sorgente; non esiste una libreria runtime Pro
|
||||
che sblocchi il canvas.
|
||||
|
||||
Il caveat è comunque concreto: auto-layout, edge routing, undo/redo, copy/paste ed
|
||||
expand/collapse compaiono nel catalogo degli esempi Pro e quel codice usa la **xyflow Pro
|
||||
License**, non la MIT del core. Le stesse funzioni possono essere sviluppate sopra le API OSS,
|
||||
ma non si può considerare tutto il materiale ufficiale liberamente riutilizzabile.
|
||||
|
||||
Fonti primarie: [package React Flow](https://github.com/xyflow/xyflow/blob/main/packages/react/package.json),
|
||||
[funzioni del core](https://reactflow.dev/index),
|
||||
[catalogo degli esempi Pro](https://reactflow.dev/examples/pro-examples),
|
||||
[auto-layout](https://reactflow.dev/examples/layout/auto-layout) e
|
||||
[termini Pro](https://reactflow.dev/pro).
|
||||
|
||||
È una buona scelta se si privilegiano velocità d'integrazione React, componenti HTML e
|
||||
accessibilità. Non è la prima scelta di questo audit perché l'utente ha chiesto espressamente
|
||||
di minimizzare la distanza fra ciò che è open source e l'offerta Professional.
|
||||
|
||||
Altro limite: React Flow combina nodi DOM e archi SVG. L'esempio ufficiale di download usa
|
||||
`html-to-image`, quindi il diagramma interattivo non si traduce automaticamente in un SVG puro.
|
||||
|
||||
## 3. Altre soluzioni interamente open source
|
||||
|
||||
### maxGraph
|
||||
|
||||
maxGraph, successore TypeScript di mxGraph, è Apache-2.0 e non ha un piano Pro. Offre rendering
|
||||
SVG, connection constraints/porte, loop e multigraph, layout, routing, outline, grouping e
|
||||
folding. Può quindi sostenere un editor ERD ricco.
|
||||
|
||||
Lo terrei come terza scelta: l'API è imperativa, non esiste un wrapper React ufficiale, la
|
||||
virtualizzazione non è documentata e gran parte dell'accessibilità va costruita. Il progetto è
|
||||
ancora pre-1.0. Il costo di integrazione e manutenzione sarebbe maggiore di X6.
|
||||
|
||||
Fonti: [licenza e repository](https://github.com/maxGraph/maxGraph),
|
||||
[guida Vite/TypeScript](https://maxgraph.github.io/maxGraph/docs/getting-started/) e
|
||||
[gestione della complessità](https://maxgraph.github.io/maxGraph/docs/usage/group-and-complexity-management/).
|
||||
|
||||
### Cytoscape.js
|
||||
|
||||
Cytoscape.js e molte estensioni sono MIT, senza tier professionale. È ottimo per grandi grafi,
|
||||
filtri, selezioni e algoritmi di rete. Il rendering Canvas è però meno naturale per schede-tabella
|
||||
ricche, testo selezionabile e handle collegati alle singole colonne. Inoltre l'adapter
|
||||
`cytoscape.js-elk` non passa le porte a ELK né usa le route degli archi: non risolve da solo il
|
||||
problema delle foreign key per colonna.
|
||||
|
||||
È appropriato per una vista di dipendenze ad alto livello, non come renderer ERD principale.
|
||||
L'estensione `cytoscape-svg` è GPL-3.0; in un prodotto che non vuole assorbire quel vincolo è
|
||||
preferibile una pipeline Graphviz/Viz.js o un generatore SVG proprio.
|
||||
|
||||
Fonti: [Cytoscape.js](https://github.com/cytoscape/cytoscape.js),
|
||||
[adapter ELK](https://github.com/cytoscape/cytoscape.js-elk) e
|
||||
[cytoscape-svg](https://github.com/kinimesi/cytoscape-svg).
|
||||
|
||||
### Liam ERD
|
||||
|
||||
Liam ERD è Apache-2.0, self-hostable e già orientato al problema: pan/zoom, ricerca, filtro,
|
||||
command palette e modalità `TABLE_NAME`, `KEY_ONLY`, `ALL_FIELDS`. La documentazione dichiara
|
||||
supporto a schemi con oltre cento tabelle. La CLI può generare un'app Vite statica.
|
||||
|
||||
Non userei però direttamente `@liam-hq/erd-core` dentro ThothII: il maintainer lo definisce una
|
||||
dipendenza interna, ne sconsiglia l'uso diretto e avverte che l'API può cambiare; il package è
|
||||
ancora 0.x. Inoltre occorre verificare la compatibilità con la versione React di ThothII e la
|
||||
fedeltà delle foreign key composite. Liam è quindi un ottimo benchmark UX, una CLI per una vista
|
||||
separata o una base da forkare consapevolmente, non l'interfaccia stabile su cui fondare il
|
||||
prodotto.
|
||||
|
||||
Fonti: [repository Liam](https://github.com/liam-hq/liam),
|
||||
[README di erd-core](https://github.com/liam-hq/liam/blob/main/frontend/packages/erd-core/README.md),
|
||||
[funzioni UI](https://liambx.com/docs/ui-features) e [CLI](https://liambx.com/docs/cli).
|
||||
|
||||
### Sprotty/GLSP
|
||||
|
||||
Sprotty e Eclipse GLSP sono EPL-2.0 e offrono un'infrastruttura seria per diagrammi SVG,
|
||||
layout e protocolli client/server. Sono pensati per strumenti di modeling estensibili, con
|
||||
dependency injection e un'architettura più ampia di un viewer. Restano una soluzione OSS valida,
|
||||
ma sproporzionata per questa esigenza salvo che ThothII evolva in un vero editor di modelli.
|
||||
|
||||
Fonti: [Sprotty](https://github.com/eclipse-sprotty/sprotty) ed
|
||||
[Eclipse GLSP](https://eclipse.dev/glsp/documentation/overview/).
|
||||
|
||||
## 4. Renderer complementari, non fondazioni del viewer
|
||||
|
||||
### Graphviz e Viz.js
|
||||
|
||||
Graphviz è EPL-2.0; `@viz-js/viz`, il port WebAssembly utilizzabile nel browser, è MIT. Graphviz
|
||||
produce SVG di alta qualità, supporta label HTML-like e porte e rimane molto utile per export,
|
||||
stampa o una vista read-only. Non fornisce però da solo ricerca, filtri, livelli di dettaglio,
|
||||
editing o gestione dello stato applicativo.
|
||||
|
||||
Lo userei come renderer di export alternativo, non come UI primaria. Se il layout a schermo
|
||||
deve corrispondere esattamente all'export, è preferibile generare l'SVG direttamente dalla
|
||||
geometria ELK invece di mantenere due motori di layout indipendenti.
|
||||
|
||||
Fonti: [licenza Graphviz](https://graphviz.org/license/),
|
||||
[formati SVG](https://graphviz.org/docs/outputs/svg/) e
|
||||
[Viz.js](https://github.com/mdaines/viz-js).
|
||||
|
||||
### Mermaid.js e librerie di pan/zoom
|
||||
|
||||
Mermaid.js è MIT e non ha feature del renderer open source bloccate. **Mermaid Chart** è invece
|
||||
un servizio commerciale distinto e il suo piano gratuito ha limiti: non va confuso con la
|
||||
libreria self-hosted.
|
||||
|
||||
Si può aggiungere navigazione a un SVG Mermaid con `@panzoom/panzoom` (MIT) o `d3-zoom` (ISC),
|
||||
entrambi senza tier Pro. Questo migliora l'esperienza corrente, ma non supera i limiti strutturali
|
||||
del diagramma ER Mermaid: interazioni a livello di tabella, controllo ridotto delle porte e del
|
||||
routing, assenza di semantic zoom e difficoltà crescente su schemi molto grandi.
|
||||
|
||||
Mermaid resta adatto a preview e documentazione, non al viewer finale.
|
||||
|
||||
Fonti: [Mermaid.js](https://github.com/mermaid-js/mermaid),
|
||||
[distinzione da Mermaid Chart](https://mermaid.ai/open-source/ecosystem/mermaid-chart.html),
|
||||
[Panzoom](https://github.com/timmywil/panzoom) e [d3-zoom](https://github.com/d3/d3-zoom).
|
||||
|
||||
### Python
|
||||
|
||||
SchemaSpy ed ERAlchemy sono open source e possono generare documentazione o grafi attraverso
|
||||
Graphviz, ma non sostituiscono il componente interattivo React. Python è utile lato backend per
|
||||
normalizzare metadati o produrre artefatti batch; pan/zoom, selezione, ricerca e dettagli restano
|
||||
responsabilità del browser. Aggiungere un servizio Python solo per disegnare l'ERD non porta un
|
||||
vantaggio architetturale a ThothII.
|
||||
|
||||
## 5. Soluzioni escluse
|
||||
|
||||
### JointJS / JointJS+
|
||||
|
||||
Il core JointJS è MPL-2.0, ma JointJS+ è commerciale e comprende proprio molte funzioni che
|
||||
servirebbero qui: PaperScroller, Navigator/minimappa, selection, clipboard, keyboard, undo/redo,
|
||||
toolbar, export e layout aggiuntivi. Sarebbe tecnicamente possibile ricostruirle sul core, ma la
|
||||
distanza fra OSS e Professional è troppo grande rispetto al criterio richiesto.
|
||||
|
||||
Fonti: [licenza](https://www.jointjs.com/license),
|
||||
[confronto delle funzioni](https://www.jointjs.com/features) e
|
||||
[prezzi](https://www.jointjs.com/pricing).
|
||||
|
||||
### GoJS e yFiles
|
||||
|
||||
Sono prodotti completi e maturi, ma non open source. GoJS richiede una licenza per il deployment
|
||||
e mostra una filigrana senza chiave; yFiles usa licenze proprietarie, con evaluation temporanea e
|
||||
licenza necessaria per la distribuzione. Non soddisfano il requisito, quindi non entrano nella
|
||||
shortlist.
|
||||
|
||||
Fonti: [licensing GoJS](https://gojs.net/latest/intro/deployment.html) e
|
||||
[licensing yFiles](https://docs.yworks.com/yfiles-html/dguide/deployment/licensing.html).
|
||||
|
||||
## 6. Architettura proposta per ThothII
|
||||
|
||||
Il frontend usa React 18 e oggi `SchemaLinkingViewer.tsx` genera Mermaid con un limite di 45
|
||||
elementi. La resa corrente perde informazione: accorpa più foreign key fra la stessa coppia di
|
||||
tabelle, omette i self-reference e usa cardinalità generiche. Il catalogo espone già tabelle,
|
||||
colonne, chiavi primarie e relazioni con coppie ordinate di colonne, quindi può alimentare un
|
||||
modello più fedele.
|
||||
|
||||
Propongo questa separazione:
|
||||
|
||||
1. **Modello renderer-neutral**: `TableNode`, `ColumnPort` e `RelationEdge` con nome del
|
||||
constraint e lista ordinata delle coppie sorgente/destinazione. Non appiattire le FK composite.
|
||||
2. **Snapshot API**: un endpoint coerente, per esempio
|
||||
`GET /catalog/databases/:databaseId/schema-diagram`, che restituisca tabelle, colonne,
|
||||
relazioni e versione del catalogo in una sola lettura, evitando la richiesta colonne N+1.
|
||||
3. **Layout worker**: ELK.js in Web Worker, con porte per colonna, port constraints, route degli
|
||||
archi e cache per versione del database e filtri correnti.
|
||||
4. **Renderer interattivo**: X6 incapsulato in `DatabaseRelationshipDiagram.tsx`, affiancato
|
||||
alla vista tabellare con un toggle List/Diagram e con riuso del drawer dei dettagli.
|
||||
5. **Export**: generatore SVG separato basato sullo stesso modello e sulla geometria ELK;
|
||||
Graphviz/Viz.js può essere un fallback per layout alternativi o documentazione batch.
|
||||
|
||||
Il catalogo non registra oggi tutti i vincoli UNIQUE né il tipo MATCH delle FK. Senza questi
|
||||
dati non sempre è possibile distinguere correttamente 1:1 da 1:N. Il renderer non deve inventare
|
||||
la cardinalità: deve mostrare una notazione neutra finché il metadato necessario non viene
|
||||
raccolto.
|
||||
|
||||
### Funzioni necessarie per schemi complessi
|
||||
|
||||
- semantic zoom: solo nomi tabella, poi PK/FK, infine tutti i campi;
|
||||
- ricerca e filtri per schema, tabella, colonna e tipo di relazione;
|
||||
- modalità focus con vicinato a uno o due hop;
|
||||
- evidenziazione della relazione e delle due colonne al passaggio/selezione;
|
||||
- minimappa, fit selection, navigazione da tastiera e pannello dettagli;
|
||||
- distinzione visiva di PK, FK, nullable, composite key, self-loop e relazioni parallele;
|
||||
- layout automatico ricalcolabile, ma posizioni manuali persistibili;
|
||||
- rendering progressivo o virtuale e fallback tabellare sempre disponibile;
|
||||
- export dell'intero schema e dell'area filtrata in SVG/PNG.
|
||||
|
||||
## 7. Proof of concept consigliato
|
||||
|
||||
Un POC breve dovrebbe usare **X6 + ELK.js** su tre dataset sintetici: circa 30, 150 e 500
|
||||
tabelle, includendo FK composite, più FK tra la stessa coppia, self-loop, tabelle isolate e hub
|
||||
ad alto grado.
|
||||
|
||||
I criteri di accettazione dovrebbero misurare:
|
||||
|
||||
- tempo di layout nel worker e tempo al primo frame interattivo;
|
||||
- fluidità di pan/zoom e uso memoria sul caso da 500 tabelle;
|
||||
- correttezza di porte, archi paralleli, self-loop e FK composite;
|
||||
- leggibilità nelle tre soglie di semantic zoom;
|
||||
- navigazione completa da tastiera e comportamento con screen reader;
|
||||
- qualità e portabilità dell'SVG esportato;
|
||||
- assenza di codice, esempi o componenti soggetti a licenza commerciale.
|
||||
|
||||
Se X6 non raggiunge il livello di accessibilità richiesto senza un costo eccessivo, il confronto
|
||||
finale va fatto con React Flow + lo stesso adapter ELK e lo stesso modello dati. In questo modo
|
||||
si cambia renderer senza rifare API, semantica delle relazioni o pipeline di export.
|
||||
@@ -1,246 +0,0 @@
|
||||
# Gestione delle relationship: da ThothAI a ThothII
|
||||
|
||||
**Stato:** analisi e direzione funzionale/UX confermate nel *grill with docs*; non costituisce ancora un piano di implementazione.
|
||||
**Revisione esaminata:** commit ThothII `f586152636b1fd653b0bc1d40b54be7f89bbd2bb`. I sorgenti legacy di ThothAI citati sotto sono versionati nello stesso repository, sotto `Thoth/ThothAI/`.
|
||||
**Ambito:** import delle foreign key fisiche, inferenza di relationship logiche, modifica umana, pubblicazione verso il workflow NL→SQL e principali gap tra i due sistemi.
|
||||
|
||||
## Sintesi fattuale
|
||||
|
||||
1. In ThothAI le foreign key dichiarate nel database venivano importate e una procedura separata proponeva relationship basate sul nome dei campi e su una verifica dei valori. Entrambi i percorsi scrivevano però nello stesso modello `Relationship` (`Thoth/ThothAI/backend/thoth_core/dbmanagement.py:484-640`; `Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:571-826`; `Thoth/ThothAI/backend/thoth_core/models.py:488-530`).
|
||||
2. Il ricordo di una relationship inferita persistita come `generated` non trova riscontro nel modello esaminato: `Relationship` contiene solo quattro foreign key verso tabelle e colonne, senza provenienza, stato, confidenza o flag `generated`. La procedura di inferenza calcola localmente pattern e tasso di validazione, ma non li salva (`Thoth/ThothAI/backend/thoth_core/models.py:488-530`; `Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-801`).
|
||||
3. L'amministratore Django di ThothAI permetteva CRUD manuale sul medesimo insieme di relationship e verificava che gli endpoint appartenessero allo stesso database e alle tabelle selezionate; non distingueva visivamente o semanticamente relationship fisiche, inferite e manuali (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:19-313`).
|
||||
4. ThothII oggi separa già i due concetti: il catalogo backend conserva solo le foreign key fisiche dichiarate, mentre il core/harness possiede annotazioni di foreign key logiche curate e un comando che suggerisce candidate per nome, primary key e SQL osservato (`CONTEXT.md:291-300`; `docs/adr/0006-separate-physical-and-logical-relationships.md:3-8`; `harness/tht/cli/schema_cmd.py:203-299`).
|
||||
5. I due mondi ThothII non sono ancora integrati: i record di Database Management non modificano il workflow NL→SQL e l'integrazione catalogo→core/schema-linking è ancora un gate di design esplicitamente differito (`PROJECT_STATE.md:116-124`; `PROJECT_STATE.md:161-165`).
|
||||
|
||||
## Evidenze ThothAI
|
||||
|
||||
### Foreign key ufficiali
|
||||
|
||||
Il percorso di import legge le foreign key dal database, limita l'import alle tabelle già presenti nel catalogo, crea le colonne mancanti, crea o riusa un record `Relationship` e aggiorna anche le stringhe denormalizzate `pk_field`/`fk_field` sulle colonne (`Thoth/ThothAI/backend/thoth_core/dbmanagement.py:484-640`, in particolare `:501-513`, `:526-591` e `:606-607`).
|
||||
|
||||
**Fatto:** una foreign key fisica diventa quindi un record applicativo, non rimane soltanto un fatto letto al momento dal database.
|
||||
|
||||
### Relationship inferite
|
||||
|
||||
La routine legacy dichiara sei famiglie di confronto tra il nome della colonna candidata e quello della primary key: corrispondenza esatta, snake case, kebab case, camel case, concatenazione e solo nome tabella (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:571-689`).
|
||||
|
||||
Per una candidata, la routine:
|
||||
|
||||
- legge fino a 20 valori distinti e non nulli dalla colonna candidata;
|
||||
- verifica ogni valore contro la primary key bersaglio;
|
||||
- accetta la candidata quando almeno il 70% dei valori esaminati trova riscontro;
|
||||
- crea o recupera un normale `Relationship` e aggiorna i campi denormalizzati delle tabelle (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-801`).
|
||||
|
||||
**Correzione documentale:** un commento parla di campionamento casuale, ma la query mostrata non contiene un ordinamento casuale; il comportamento verificabile dal codice è “fino a 20 valori distinti e non nulli”, non un campione statisticamente casuale (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-716`).
|
||||
|
||||
**Rischio legacy:** nomi di tabelle e colonne sono interpolati direttamente in SQL in questo percorso. Portare la logica letteralmente in ThothII riprodurrebbe un problema di quoting/sicurezza e richiederebbe inoltre una policy esplicita per l'accesso ai valori del DWH (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:709-750`).
|
||||
|
||||
### Un solo modello per tre origini
|
||||
|
||||
Il modello `Relationship` legacy contiene soltanto `source_table`, `target_table`, `source_column` e `target_column`, più metodi di rappresentazione/aggiornamento. Non contiene campi per origine, algoritmo, evidenza, confidenza, approvazione o disabilitazione, né un vincolo di unicità dichiarato nel modello (`Thoth/ThothAI/backend/thoth_core/models.py:488-530`).
|
||||
|
||||
L'admin consente aggiunta, modifica e cancellazione ordinarie e valida la coerenza tra database, tabelle e colonne (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:30-38`, `:125-157`, `:218-245`).
|
||||
|
||||
**Conclusione fattuale:** relationship importate, inferite e create manualmente convergono nello stesso tipo persistito. Dal record finale non è possibile ricostruirne con certezza l'origine. Il termine `generated`, se usato nell'interfaccia o nel linguaggio operativo dell'epoca, non era una qualificazione persistita dal modello esaminato.
|
||||
|
||||
### Uso nella comprensione dello schema
|
||||
|
||||
La generazione M-Schema conserva colonne PK/FK e ricostruisce la sezione `【Foreign keys】` analizzando le stringhe denormalizzate `fk_field` (`Thoth/ThothAI/frontend/sql_generator/helpers/main_helpers/main_generate_mschema.py:25-61`, `:94-101`, `:160-187`).
|
||||
|
||||
**Fatto:** le relationship curate nel catalogo legacy influenzavano la rappresentazione dello schema consumata dal generatore SQL, anche se tramite una proiezione denormalizzata.
|
||||
|
||||
## Stato attuale di ThothII
|
||||
|
||||
### Catalogo backend: relationship fisiche
|
||||
|
||||
Il modello di dominio corrente distingue esplicitamente:
|
||||
|
||||
- **Physical Relationship:** vincolo dichiarato nel database;
|
||||
- **Catalog Relationship:** copia persistita di quel fatto fisico, non creabile o modificabile manualmente ma eliminabile per cleanup;
|
||||
- **Logical Relationship:** relazione semantica curata o inferita, con ownership e lifecycle separati (`CONTEXT.md:291-300`; `docs/adr/0006-separate-physical-and-logical-relationships.md:3-8`).
|
||||
|
||||
La migrazione del catalogo crea `catalog_relationships` e le coppie ordinate in `catalog_relationship_columns`. L'identità della relazione fisica è `(source_table_id, constraint_name)`; non esistono campi di origine, stato o confidenza (`backend/src/catalog/migrations/003_catalog_schema_sync.ts:42-74`). Il tipo TypeScript è solo strutturale e le descrizioni generate riguardano esclusivamente tabelle o colonne (`backend/src/catalog/types.ts:123-152`).
|
||||
|
||||
L'introspezione PostgreSQL legge soltanto `pg_constraint` con `contype = 'f'` e mantiene l'ordine delle coppie per le chiavi composite (`backend/src/catalog/schema-introspector.ts:110-171`; `backend/src/catalog/schema-introspector.ts:360-416`; `docs/contracts/catalog-schema-snapshot.md:40-53`). La sincronizzazione autoritativa crea, aggiorna o elimina le copie fisiche in base allo snapshot (`backend/src/catalog/repository.ts:1165-1218`; `backend/src/catalog/repository.ts:1316-1393`).
|
||||
|
||||
Le API e la UI espongono lettura, sync e cleanup in massa, non CRUD logico per singola relationship (`backend/src/routes/catalog-schema.ts:25-37`; `backend/src/routes/catalog-schema.ts:134-150`; `frontend/src/api/catalog-databases.ts:452-489`; `frontend/src/shell/database-management/DatabaseRelationships.tsx:114-123`; `frontend/src/shell/database-management/DatabaseRelationships.tsx:160-225`).
|
||||
|
||||
**Effetto rilevante:** il cleanup è intenzionalmente reversibile tramite una sync successiva e può lasciare temporaneamente un catalogo incompleto (`docs/adr/0008-allow-manual-catalog-metadata-cleanup.md:7-16`). Un test di integrazione ammette anche una relationship rimasta senza coppie di colonne dopo la cancellazione delle colonne catalogate (`backend/test/catalog-repository.integration.test.ts:194-264`, in particolare `:246-252`). Un futuro consumer non può quindi assumere che ogni stato intermedio del catalogo sia pubblicabile così com'è.
|
||||
|
||||
### Core/harness: relationship logiche e suggerimenti
|
||||
|
||||
Il modello M-Schema del core possiede già `TableAnnotation.foreign_keys`, descritte come foreign key logiche curate e unite alle foreign key fisiche (`harness/tht/mschema/models.py:35-39`; `harness/tht/mschema/models.py:76-84`). Il renderer fonde i due insiemi e li presenta insieme nella sezione `【Foreign keys】` (`harness/tht/mschema/render.py:9-20`; `harness/tht/mschema/render.py:46-80`).
|
||||
|
||||
Il comando `schema suggest-fks` costruisce candidate da:
|
||||
|
||||
- uguaglianze trovate in SQL, quando esattamente un lato è una primary key;
|
||||
- convenzione speciale `*time_key → dim_time.<PK singola>`;
|
||||
- stesso nome tra colonna e primary key a proprietario univoco;
|
||||
- assunzioni esplicite per disambiguare;
|
||||
- esclusione di nomi PK generici come `id`, `key` e `code` e dei proprietari ambigui (`harness/tht/cli/schema_cmd.py:203-299`; `harness/tht/mschema/fkmine.py:1-58`).
|
||||
|
||||
Il comando può produrre un documento candidato e, con `--write`, aggiungere annotazioni mancanti in modo idempotente; il messaggio stesso chiede revisione manuale (`harness/tht/cli/schema_cmd.py:406-429`; `harness/tht/cli/schema_cmd.py:519-540`; `harness/tests/test_schema_fk_annotations.py:131-146`; `harness/tests/test_schema_fk_annotations.py:454-470`).
|
||||
|
||||
**Fatto:** ThothII non parte da zero sull'inferenza. Possiede già un motore più conservativo, basato su PK univoche e SQL osservato, ma non conserva per ogni relazione origine, evidenza, frequenza o confidenza. Il miner conta le occorrenze internamente, ma il modello candidato non promuove quel conteggio a lifecycle persistito (`harness/tht/mschema/fkmine.py:1-58`; `harness/tht/mschema/models.py:35-39`).
|
||||
|
||||
**Rischi strutturali già visibili:**
|
||||
|
||||
- `ForeignKey` ammette liste di colonne, ma non valida che source e target abbiano la stessa cardinalità; il renderer usa `zip`, quindi una relazione malformata può essere troncata silenziosamente (`harness/tht/mschema/models.py:35-39`; `harness/tht/mschema/render.py:76-78`).
|
||||
- il merge evita duplicati rispetto alle FK fisiche iniziali, ma non aggiorna l'insieme `seen` dopo aver aggiunto un'annotazione; due annotazioni logiche uguali possono sopravvivere al merge (`harness/tht/mschema/render.py:9-20`).
|
||||
- il catalogo fisico supporta coppie composite ordinate, mentre le euristiche correnti e legacy sono sostanzialmente unary. La semantica delle candidate composite resta da decidere (`backend/src/catalog/migrations/003_catalog_schema_sync.ts:56-74`; `harness/tht/cli/schema_cmd.py:203-299`).
|
||||
|
||||
### Revisione e pubblicazione correnti
|
||||
|
||||
Le annotazioni canoniche del workspace sono un blob Git. Il flusso pubblico produce candidate, verifica le annotazioni e richiede un'accettazione umana esplicita dopo commit/push/pull; l'accettazione registra digest del candidato e delle annotazioni, revisione e blob (`docs/contracts/workspace-preprocessing-cli.md:89-106`; `backend/src/workspaces/preprocessing-service.ts:202-297`). Il preprocessing successivo procede solo se il digest accettato coincide con le annotazioni correnti (`backend/src/workspaces/preprocessing-service.ts:335-388`).
|
||||
|
||||
**Limite fattuale:** il record di review conserva digest, revisione e blob, ma non attore, motivazione o decisioni per singola candidata (`backend/src/workspaces/preprocessing-state.ts:67-73`).
|
||||
|
||||
Il runtime usa le annotazioni quando renderizza lo schema, ma la ricerca vettoriale indicizza record di tabelle e colonne senza contenuto esplicito delle relationship (`harness/tht/vectorstore/records.py:106-138`; `harness/tht/cli/search_cmd.py:233-273`). Inoltre la vista colonne usata in F4 continua a leggere i commenti fisici, non le annotazioni (`harness/tht/cli/schema_cmd.py:592-621`).
|
||||
|
||||
La review dei join durante una sessione è distinta dalla curatela globale: il reviewer conferma l'insieme dei join oppure chiede una revisione completa; non modifica la mappa canonica delle relationship (`harness/.pi/skills/tht-sessione/SKILL.md:301-341`; `frontend/src/widgets/JoinReviewWidget.tsx:5-84`). Il modello di sessione registra join come due stringhe e una decisione opzionale, senza ID stabile della relationship o coppie ordinate strutturate (`harness/tht/session/models.py:147-170`).
|
||||
|
||||
## Delta e rischi da sottoporre al grill
|
||||
|
||||
| Tema | Fatto documentato | Delta/rischio aperto |
|
||||
|---|---|---|
|
||||
| Origine | ThothAI perdeva l'origine; ThothII separa fisico e logico a livello concettuale | Decidere quale provenienza debba essere persistita per manuale, euristica, SQL osservato e import fisico |
|
||||
| `generated` | Non era un flag del modello ThothAI; in ThothII “Generated Description” è già un termine del catalogo (`CONTEXT.md:302-311`) | Usare `generated` anche per relationship potrebbe creare ambiguità terminologica |
|
||||
| Cancellazione utente | In ThothAI era CRUD sul record unico; in ThothII il cleanup fisico viene ricostruito dalla sync | “Eliminare” può significare cancellare una relazione logica, sopprimere un fatto fisico per il core oppure pulire temporaneamente la copia catalogata: sono operazioni diverse |
|
||||
| Inferenza | ThothAI usava sei pattern e valori DWH; ThothII usa PK univoche, nomi e SQL osservato | Stabilire se sostituire, integrare o non portare il campionamento dei valori; servono policy di dati sensibili, query read-only, quoting e limiti |
|
||||
| Approvazione | ThothII dispone di review Git/digest dell'intero artefatto | Mancano decisioni per candidata, motivazione, attore, sticky rejection, versione algoritmo e gestione dello stale |
|
||||
| Compositi | Il catalogo fisico conserva coppie ordinate; le annotazioni accettano liste | Mancano invarianti forti e una strategia di inferenza/review per join compositi |
|
||||
| Pubblicazione | Catalogo management e runtime core sono oggi separati | Va stabilito se pubblicare tutto, una selezione esplicita o una revisione immutabile; il catalogo può essere incompleto durante cleanup/sync |
|
||||
| UI e ownership | UI catalogo fisico read-only; curatela logica via CLI/Git; review join per sessione separata | Va scelto chi cura la mappa e in quale superficie, senza confondere amministrazione globale e correzione della singola sessione |
|
||||
| Retrieval | Le relationship entrano nel render M-Schema ma non nei record vettoriali | Va deciso se e come influenzano selezione tabelle, ranking e descrizioni, oltre al rendering finale |
|
||||
| Drift | Sync fisica è autoritativa; annotazioni sono una revisione separata | Servono semantiche per endpoint rinominati/eliminati, candidate stale, orphan e riapprovazione |
|
||||
|
||||
Le analisi già presenti nel repository trattano l'integrazione catalogo→core, la selezione pubblicabile e la riparazione degli indici come questioni ancora aperte; le loro proposte non sono decisioni implementate (`docs/research/2026-08-23-thothii-metadata-publication-qdrant-seams.md:20-43`; `docs/research/2026-08-23-thothii-metadata-publication-qdrant-seams.md:239-301`). Anche il piano di migrazione rinvia il lifecycle/admin delle relationship logiche (`docs/plans/2026-08-26-metadata-catalog-from-thothai.md:681-691`).
|
||||
|
||||
## Direzione confermata nel grill
|
||||
|
||||
Il 31 agosto 2026 è stato concordato il seguente flusso minimo:
|
||||
|
||||
1. Le foreign key dichiarate continuano a essere sincronizzate come Catalog Relationship fisiche.
|
||||
2. Le foreign key ipotetiche sono salvate una sola volta come Logical Relationship fra colonna
|
||||
sorgente e colonna destinazione; le tabelle sono ricavate dalle colonne e la UI le presenta nel
|
||||
relativo contesto tabella.
|
||||
3. Una relationship inferita è marcata `generated`; una relationship aggiunta dall'utente non lo è.
|
||||
4. La cancellazione logica conserva il record e impedisce a una ricostruzione di riattivarlo.
|
||||
5. La cancellazione fisica rimuove il record; una ricostruzione può ricrearlo se viene nuovamente
|
||||
inferito.
|
||||
6. La ricostruzione è additiva: conserva le relationship attive già presenti, non rimuove quelle
|
||||
non più inferibili e non modifica le relationship manuali.
|
||||
7. L'inferenza usa nomi, primary key e compatibilità dei tipi. Non campiona valori del DWH.
|
||||
8. La relationship è l'unica fonte di verità: non viene duplicata in stringhe `fk_field` sulle
|
||||
colonne.
|
||||
9. I nomi vengono confrontati senza distinzione fra maiuscole/minuscole e normalizzando snake case,
|
||||
kebab case e camel case. La regola riconosce anche casi come `user_id → users.id`, richiede tipi
|
||||
compatibili e una sola destinazione possibile; riconosce inoltre un nome PK non generico con un
|
||||
unico proprietario e la convenzione `*time_key → dim_time.<PK singola>`. Le colonne sorgenti
|
||||
possono appartenere a PK composite, mentre nomi generici isolati come `id`, `key`, `code` e `pk`
|
||||
non costituiscono evidenza. I casi ambigui vengono ignorati e non si usa fuzzy matching o un LLM.
|
||||
10. La ricostruzione parte da un'azione amministrativa esplicita `Rebuild generated relationships`,
|
||||
separata dalla sincronizzazione dello schema.
|
||||
11. Le relationship cancellate logicamente restano consultabili tramite filtro e possono essere
|
||||
riattivate con un'azione `Restore`.
|
||||
|
||||
Non restano decisioni di dominio aperte per il flusso minimo. La progettazione UX e il seam tecnico
|
||||
sono descritti nelle sezioni seguenti.
|
||||
|
||||
## Lacuna UX emersa nel grill
|
||||
|
||||
La vista Fleet `Relationships` esiste, ma nella UI di produzione non ha oggi un punto di ingresso
|
||||
raggiungibile. La riga del database espone sincronizzazione, tabelle, dettagli, modifica e rimozione,
|
||||
ma non le relationship; inoltre i tab `Overview / Tables / Relationships` appartengono soltanto alla
|
||||
presentazione legacy. Il test di navigazione esistente esercita anch'esso la presentazione legacy,
|
||||
non quella Fleet (`frontend/src/shell/database-management/DatabaseGrid.tsx:122-180`;
|
||||
`frontend/src/shell/database-management/DatabaseForm.tsx:442-475`;
|
||||
`frontend/src/shell/database-management/DatabaseForm.tsx:517-546`;
|
||||
`frontend/src/shell/DatabaseManagementPage.test.tsx:2635-2703`).
|
||||
|
||||
Se aperta programmaticamente, la vista mostra soltanto `Physical relationships` in sola lettura. La
|
||||
toolbar contiene ricerca, conteggio, `Refresh`, un selettore azione con esecuzione esplicita e
|
||||
`Sync history`; la griglia offre unicamente `Details`, che apre il drawer della relationship fisica.
|
||||
Sono assenti ingresso visibile, aggiunta manuale, ricostruzione delle generated relationship, origine,
|
||||
stato attivo/cancellato, cancellazione logica, cancellazione fisica e ripristino
|
||||
(`frontend/src/shell/database-management/DatabaseRelationships.tsx:114-225`).
|
||||
|
||||
I pattern Fleet già consolidati da riutilizzare sono:
|
||||
|
||||
- una sola griglia nel livello corrente, con breadcrumb e controllo Back;
|
||||
- azioni di pagina nel selettore `Choose an action…` con `Run action` e motivo visibile quando
|
||||
indisponibili;
|
||||
- azioni della singola riga nella colonna finale fissata a destra;
|
||||
- form e dettagli in un drawer modeless che restituisce il focus al controllo di origine;
|
||||
- conferme distruttive inline o nel drawer, non tramite una nuova pagina;
|
||||
- toast per accettazione o errore e feedback persistente soltanto per le operazioni lunghe;
|
||||
- card di errore con Retry ed empty state che indica la prossima azione possibile.
|
||||
|
||||
ThothAI non offre un modello UX da copiare. L'inferenza è nascosta fra 21 bulk action della lista
|
||||
database, non mostra avanzamento in tempo reale e restituisce soltanto messaggi a fine richiesta. Il
|
||||
CRUD manuale vive in un'altra schermata Django Admin, non distingue origine o stato, offre soltanto
|
||||
la cancellazione fisica e il form di aggiunta ha una validazione server strutturalmente incoerente
|
||||
con le select popolate dal browser
|
||||
(`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:252-274`;
|
||||
`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:58-119`;
|
||||
`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:218-313`).
|
||||
|
||||
## UX confermata
|
||||
|
||||
Il 31 agosto 2026 sono state confermate le seguenti scelte:
|
||||
|
||||
1. La colonna Actions della riga database espone un accesso diretto `Relationships`, accanto a
|
||||
`Tables`. La vista conserva breadcrumb e controllo `Back to databases` esistenti.
|
||||
2. La pagina presenta una sola griglia `Relationship map`, contenente relationship `Physical`,
|
||||
`Generated` e `Manual`. Le colonne sono Source table, Source column, Target table, Target column,
|
||||
Origin, Status e Actions. Constraint e regole update/delete rimangono nel drawer delle FK fisiche.
|
||||
3. Un filtro visibile seleziona `Active`, `Excluded` o `All`; il valore predefinito è `Active`. Le FK
|
||||
fisiche sono consultabili ma non modificabili da questa vista.
|
||||
4. `Add relationship` è il pulsante primario visibile nella toolbar. Apre il drawer standard con i
|
||||
quattro campi Source table, Source column, Target table e Target column e i comandi `Cancel` e
|
||||
`Add relationship`. Il flusso minimo gestisce una sola coppia di colonne e mostra la validazione
|
||||
accanto al campo interessato.
|
||||
5. `Rebuild generated relationships` entra nell'attuale selettore `Choose an action…`, insieme a
|
||||
`Synchronize physical relationships` e `Synchronize full schema`, con esecuzione esplicita tramite
|
||||
`Run action`. `Refresh` ricarica la griglia; `Sync history` resta riservato alla sincronizzazione
|
||||
fisica.
|
||||
6. Il drawer di dettaglio contiene le azioni sulle relationship logiche. `Exclude` realizza la
|
||||
cancellazione logica, `Delete permanently` quella fisica e `Restore` riattiva una relationship
|
||||
esclusa. La conferma avviene nel drawer e spiega rispettivamente che la ricostruzione non
|
||||
riattiverà un record escluso e potrà invece ricreare un record eliminato definitivamente.
|
||||
7. La ricostruzione non introduce un nuovo sistema di job o di storico. Durante l'esecuzione mostra
|
||||
`Rebuilding…`; al termine un toast riporta added, already present, excluded e ambiguous. Add,
|
||||
Exclude, Delete permanently e Restore producono toast specifici; gli errori mantengono aperto il
|
||||
contesto corrente. L'empty state propone di ricostruire dai nomi o aggiungere manualmente.
|
||||
8. L'inferenza non usa AI. È codice deterministico nel backend del catalogo, basato su normalizzazione
|
||||
dei nomi, primary key/unicità, compatibilità dei tipi e assenza di ambiguità. Non usa LLM,
|
||||
embedding o campionamento dei dati. L'AI consuma la mappa risultante per comprendere lo schema,
|
||||
ma non la costruisce.
|
||||
|
||||
## Seam tecnico confermato
|
||||
|
||||
Il Catalog PostgreSQL è l'unica fonte modificabile delle Logical Relationship. Le relationship
|
||||
fisiche e logiche restano in modelli distinti, coerentemente con ADR-0006, mentre un servizio profondo
|
||||
espone a API e UI una sola mappa discriminata per origine e stato.
|
||||
|
||||
All'avvio o alla ripresa di una sessione, il backend materializza le sole relationship attive in una
|
||||
snapshot JSON canonica e immutabile, collegata alla stessa lease della configurazione runtime. La
|
||||
snapshot comprende anche le FK fisiche e conserva l'ordine delle coppie composite. Se due record hanno
|
||||
gli stessi endpoint, la FK fisica ha precedenza.
|
||||
|
||||
Quando la snapshot è presente, l'harness la usa come fonte esclusiva delle relationship e continua a
|
||||
leggere dalle annotazioni Git-pinned soltanto descrizioni, sinonimi, concetti e altri metadati. Non
|
||||
scrive `annotations.yaml` e non interroga direttamente il Catalog. Una snapshot dichiarata ma assente,
|
||||
invalida o incoerente con lo schema fisico fallisce esplicitamente; un runtime legacy che non dichiara
|
||||
la snapshot mantiene il precedente comportamento di compatibilità.
|
||||
|
||||
Questa proiezione non è un secondo store: non può essere modificata, viene eliminata insieme alla
|
||||
configurazione runtime e una sessione Pi vede una mappa stabile per tutta la propria vita. La decisione
|
||||
duratura è registrata in ADR-0012.
|
||||
|
||||
La proiezione e la ricostruzione sono ammesse soltanto dopo una sincronizzazione completa della
|
||||
versione corrente del database e vengono serializzate con le mutazioni del catalogo. Un catalogo mai
|
||||
sincronizzato, reso stale da una modifica della configurazione o invalidato da metadata cleanup non
|
||||
può quindi diventare accidentalmente la fonte esclusiva del runtime. Il cleanup esplicito di una
|
||||
tabella o colonna endpoint è anche il confine distruttivo del tombstone: rimuove definitivamente la
|
||||
relationship esclusa, perché conservarla richiederebbe una seconda identità testuale denormalizzata.
|
||||
@@ -1,219 +0,0 @@
|
||||
# Licenza del core e compatibilità del repository
|
||||
|
||||
Data della verifica: 2026-09-02
|
||||
Ticket: [Valutare licenza del core e compatibilità legale del repository](https://git.tylconsulting.it/mptyl/ThothII/issues/22)
|
||||
|
||||
> Questa è un'analisi tecnica delle licenze e della provenienza osservabile nel repository, non
|
||||
> consulenza legale. Prima della prima distribuzione pubblica o commerciale, un professionista
|
||||
> qualificato nella giurisdizione di TYL Consulting e dei clienti deve validare confini dei lavori
|
||||
> derivati, titolarità, brevetti, marchi, EULA e obblighi di redistribuzione.
|
||||
|
||||
## Decisione proposta
|
||||
|
||||
Adottare **Apache License 2.0** per la Community Edition e una licenza commerciale separata per
|
||||
l'Enterprise Edition. Apache-2.0 è preferibile a MIT per un prodotto enterprise perché contiene
|
||||
una concessione brevettuale esplicita con clausola di cessazione, disciplina i contributi, tutela i
|
||||
marchi e descrive esplicitamente come non derivati i lavori separabili o che si limitano a collegarsi
|
||||
alle interfacce. Rimane una licenza permissiva: consente uso commerciale, modifica e distribuzione
|
||||
proprietaria delle modifiche nel rispetto degli obblighi della licenza.
|
||||
|
||||
**La pubblicazione non è ancora pronta.** Due problemi devono essere risolti prima di applicare
|
||||
Apache-2.0 al repository:
|
||||
|
||||
1. `yake==0.7.3` è importato direttamente dal runtime Python e il file `LICENSE` dell'artefatto
|
||||
ufficiale contiene AGPL-3.0, benché i metadati PyPI dichiarino in modo incoerente LGPL/GPL.
|
||||
Va sostituito con una dipendenza permissiva, rimosso, oppure coperto da una licenza commerciale
|
||||
verificata. L'isolamento va considerato solo dopo parere legale; nel codice attuale non c'è
|
||||
isolamento, ma un normale `import yake`.
|
||||
2. [`harness/tht/vendor/VENDORED.md`](../../harness/tht/vendor/VENDORED.md) dichiara Apache-2.0 per
|
||||
`thoth_lsh.py`, mentre l'artefatto upstream `thoth-dbmanager==0.7.4` su PyPI contiene una licenza
|
||||
MIT e il relativo copyright. Occorre correggere la provenienza e conservare il testo MIT, oppure
|
||||
documentare formalmente una nuova licenza da parte di tutti i titolari effettivi.
|
||||
|
||||
## MIT, Apache-2.0 e copyleft
|
||||
|
||||
| Tema | MIT | Apache-2.0 | GPLv3 / AGPLv3 |
|
||||
| --- | --- | --- | --- |
|
||||
| Uso commerciale e codice proprietario | Sì; obbligo essenziale di conservare copyright e licenza | Sì; consente termini diversi sulle modifiche, preservando gli obblighi Apache | Il codice coperto e le opere combinate distribuite devono rispettare il copyleft |
|
||||
| Brevetti | Nessuna concessione brevettuale espressa nel testo | Concessione espressa sui claim necessariamente violati dal contributo; termina in caso di determinata causa brevettuale | GPLv3/AGPLv3 contengono condizioni brevettuali proprie |
|
||||
| Redistribuzione | Conservare copyright e testo MIT nelle copie o porzioni sostanziali | Fornire la licenza, marcare i file modificati, conservare notice pertinenti e riprodurre `NOTICE` quando presente | Fornire il Corresponding Source e non imporre ulteriori restrizioni incompatibili |
|
||||
| Moduli proprietari | Consentiti; il testo non definisce il confine fra opere | Consentiti; la definizione esclude lavori separabili o che si limitano a collegarsi alle interfacce | Confine fattuale e giuridico. L'aggregazione di opere indipendenti è distinta da un programma combinato |
|
||||
| Uso via rete | Nessun obbligo specifico | Nessun obbligo specifico | AGPLv3 §13 richiede offrire il Corresponding Source agli utenti remoti di una versione modificata |
|
||||
| Contributi | Nessuna disciplina esplicita nel breve testo | §5: contributi intenzionalmente sottoposti sono Apache-2.0 salvo dichiarazione o accordo separato | Contributi al lavoro coperto devono poter essere distribuiti sotto la licenza applicabile |
|
||||
| Marchi | Non disciplinati | §6 non concede diritti sui marchi salvo uso descrittivo e `NOTICE` | Non sono automaticamente una licenza del marchio |
|
||||
|
||||
Fonti primarie: [testo MIT dell'OSI](https://opensource.org/license/mit/),
|
||||
[Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0),
|
||||
[GPLv3](https://www.gnu.org/licenses/gpl-3.0.html),
|
||||
[AGPLv3](https://www.gnu.org/licenses/agpl-3.0.html) e
|
||||
[FAQ GNU su plugin e aggregazione](https://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins).
|
||||
|
||||
### Perché non scegliere MIT
|
||||
|
||||
MIT renderebbe semplice l'adozione e il riuso proprietario, ma non dà a compratori e contributori
|
||||
la stessa chiarezza di Apache-2.0 sui brevetti, sui contributi e sui marchi. La brevità non compensa
|
||||
questa minore precisione per un prodotto venduto a IT department enterprise. MIT resta compatibile
|
||||
con l'obiettivo open-core, ma Apache-2.0 gestisce meglio il rischio del progetto.
|
||||
|
||||
### Perché non scegliere copyleft per il core
|
||||
|
||||
Il copyleft aumenterebbe la reciprocità, ma rende più delicato il confine con l'Enterprise Edition e
|
||||
può creare una revisione legale più onerosa per i clienti. AGPL è inoltre poco allineata alla scelta
|
||||
commerciale già fissata: installazione interamente nel trust boundary del cliente e funzionalità
|
||||
Enterprise proprietarie. Se in futuro la priorità diventasse impedire fork proprietari del core,
|
||||
GPL/AGPL andrebbero valutate in una decisione distinta, non introdotte accidentalmente da una
|
||||
dipendenza.
|
||||
|
||||
## Confine Community / Enterprise
|
||||
|
||||
Il confine raccomandato è tecnico oltre che contrattuale:
|
||||
|
||||
- Community Edition in repository pubblico, licenza Apache-2.0, workflow fondamentale completo e
|
||||
interfacce di estensione pubbliche;
|
||||
- Enterprise Edition in repository, package e immagini separati, con EULA commerciale e SBOM
|
||||
autonomo;
|
||||
- integrazione attraverso API/versioned contracts o process boundary. Anche se Apache-2.0 consente
|
||||
linking e opere separabili, questa separazione riduce errori di packaging e rende auditabile il
|
||||
perimetro venduto;
|
||||
- una distribuzione Enterprise può aggregare componenti Community, ma deve continuare a fornire
|
||||
Apache-2.0 e tutti i notice della Community. L'EULA non deve revocare ai clienti i diritti
|
||||
Apache sui componenti Community;
|
||||
- marchio e logo `ThothII` richiedono una policy separata: la licenza del codice non deve essere
|
||||
usata come licenza del brand.
|
||||
|
||||
Il giudizio finale sul fatto che un plugin concreto sia un'opera indipendente o derivata dipende
|
||||
dalla sua implementazione e dalla legge applicabile: è un punto da sottoporre al legale quando
|
||||
esisterà il primo modulo Enterprise.
|
||||
|
||||
## Inventario del repository
|
||||
|
||||
L'inventario deriva dai file tracciati nel commit `ae053961`, dai lockfile e, per i casi anomali,
|
||||
dagli artefatti ufficiali dei registry. I metadati dei registry non sostituiscono i testi di licenza
|
||||
inclusi nelle distribuzioni: l'incoerenza di YAKE lo dimostra.
|
||||
|
||||
### Stato della licenza propria
|
||||
|
||||
- Non esistono `LICENSE`, `NOTICE`, `COPYING`, `AUTHORS` o `THIRD_PARTY_NOTICES` alla radice.
|
||||
- I manifest `backend/package.json`, `frontend/package.json`, `harness/package.json`,
|
||||
`docker/pi-runtime/package.json`, `tools/html-to-pptx/package.json` e
|
||||
`harness/pyproject.toml` non dichiarano la licenza del progetto.
|
||||
- La cronologia Git contiene più identità (`mptyl`, `Marco Pancotti`, `User`, `Codex`, Gitea
|
||||
Actions) e trailer `Co-authored-by` di strumenti AI, ma nessun `Signed-off-by`. La dichiarazione
|
||||
del proprietario è che il codice appartiene interamente a TYL Consulting; prima del rilascio
|
||||
occorre collegare documentalmente gli alias al titolare e conservare i termini applicabili agli
|
||||
output generati.
|
||||
|
||||
### JavaScript / TypeScript
|
||||
|
||||
Cinque lockfile npm sono presenti e nessun manifest dichiara la licenza del package ThothII.
|
||||
|
||||
| Superficie | Package bloccati | Risultato dei metadati nel lock |
|
||||
| --- | ---: | --- |
|
||||
| backend | 366 | MIT 296, ISC 27, Apache-2.0 19, BSD-3-Clause 16, BlueOak 4, senza campo 3, Unlicense 1 |
|
||||
| runtime Pi | 140 | MIT 67, Apache-2.0 46, BSD-3-Clause 13, ISC 8, BlueOak 5, 0BSD 1 |
|
||||
| frontend | 874 | prevalentemente MIT; inoltre OFL-1.1, CC-BY-4.0, Python-2.0, BSD, ISC e licenze alternative |
|
||||
| harness (dev) | 1 | MIT |
|
||||
| exporter PPTX | 22 | prevalentemente MIT; `jszip` offre l'alternativa MIT oppure GPL-3.0-or-later |
|
||||
|
||||
Elementi che richiedono notice o verifica esplicita:
|
||||
|
||||
- `@fontsource-variable/geist@5.2.9` è OFL-1.1 e viene distribuito nel frontend; OFL richiede che
|
||||
copyright e licenza accompagnino ogni copia del font. Vedi
|
||||
[OFL 1.1](https://openfontlicense.org/open-font-license-official-text/).
|
||||
- `dompurify@3.4.11` offre `MPL-2.0 OR Apache-2.0`: documentare la scelta Apache-2.0.
|
||||
- `jszip@3.10.1` offre `MIT OR GPL-3.0-or-later`: documentare la scelta MIT.
|
||||
- `khroma`, `buildcheck`, `cpu-features` e `ssh2` non hanno il campo `license` nei lock/registry
|
||||
esaminati: lo scanner di release deve leggere la licenza inclusa nei tarball, non assumere MIT.
|
||||
- `@earendil-works/pi-coding-agent@0.80.3`, AG Grid Community, Fastify, Kysely e OpenID Client
|
||||
risultano MIT nei metadati ufficiali npm.
|
||||
|
||||
Fonti: lockfile locali e metadata del [registry npm](https://registry.npmjs.org/).
|
||||
|
||||
### Python
|
||||
|
||||
`harness/uv.lock` blocca 85 package, incluso il workspace `tht`; i metadata ufficiali PyPI degli 84
|
||||
package esterni riportano soprattutto MIT, Apache e BSD. Le eccezioni sono:
|
||||
|
||||
- **YAKE 0.7.3: blocker.** Il file `LICENSE` nel source distribution ufficiale è AGPL-3.0 e offre
|
||||
separatamente una licenza commerciale. I metadata dichiarano invece `LGPLv3` e classificano
|
||||
GPLv3. Il codice lo importa in
|
||||
[`harness/tht/session/store.py`](../../harness/tht/session/store.py). Fonti:
|
||||
[tag upstream v0.7.3](https://github.com/INESCTEC/yake/blob/v0.7.3/LICENSE) e
|
||||
[release PyPI](https://pypi.org/project/yake/0.7.3/).
|
||||
- `psycopg2-binary==2.9.12`: LGPLv3 con eccezioni specifiche nel file `LICENSE` dell'artefatto.
|
||||
- `certifi==2026.7.22`: MPL-2.0 per il bundle di CA modificato.
|
||||
- `tqdm==4.70.0`: licenza composita con parti MPL-2.0 e MIT, non una semplice alternativa;
|
||||
preservare il suo `LICENCE`.
|
||||
- `fsspec==2026.7.0`: il metadata PyPI è vuoto, ma il source distribution include BSD-3-Clause.
|
||||
|
||||
`docs/requirements.lock` contiene 31 dipendenze di build documentale; fra queste `certifi` e
|
||||
`pathspec` riportano MPL-2.0. Sono build-time, ma vanno comunque incluse nella scansione del processo
|
||||
di release se vengono ridistribuite in immagini o toolchain.
|
||||
|
||||
### Go
|
||||
|
||||
`tools/tht/go.mod` ha 12 dipendenze dichiarate e il grafo scaricato contiene solo famiglie
|
||||
permissive osservate nei testi root: Apache-2.0, MIT, BSD-2-Clause, BSD-3-Clause e ISC. Alcuni moduli
|
||||
Apache includono anche `NOTICE`, che deve essere propagato nell'immagine/binario distribuito.
|
||||
`tools/dwh-auth/go.mod` non dichiara dipendenze esterne. `go.sum` è presente solo per `tools/tht`.
|
||||
|
||||
### Codice vendorizzato e asset
|
||||
|
||||
- `harness/tht/vendor/thoth_lsh.py` è copiato e modificato da `thoth-dbmanager==0.7.4`.
|
||||
L'artefatto PyPI ufficiale contiene MIT, mentre il repository lo marca Apache-2.0. È necessario
|
||||
risolvere la discrepanza e includere l'attribuzione corretta. Fonti:
|
||||
[metadata PyPI](https://pypi.org/pypi/thoth-dbmanager/0.7.4/json) e
|
||||
[`VENDORED.md`](../../harness/tht/vendor/VENDORED.md).
|
||||
- `frontend/prototypes/database-management/assets/thoth-cosmic-archive.png` è documentato come
|
||||
output del tool ImageGen. Prima di pubblicarlo sotto Apache-2.0, conservare evidenza del contratto
|
||||
applicabile e della decisione del titolare di includerlo nella Community Edition.
|
||||
- `presentations/thothii-congresso.pptx` è generato dalle sorgenti HTML/JSON locali e contiene PNG
|
||||
rasterizzati. L'HTML carica Fraunces e Manrope da Google Fonts: conservare le rispettive licenze
|
||||
font e decidere se la presentazione appartiene davvero al source release del prodotto.
|
||||
- Il solo font binario individuato nel bundle applicativo arriva da `@fontsource-variable/geist` ed
|
||||
è OFL-1.1. Non sono presenti altri font, PDF, archivi o WASM tracciati; è presente un PNG e un PPTX.
|
||||
|
||||
### Immagini container e modelli
|
||||
|
||||
Il deploy usa immagini pin per Node, Python, Go, nginx-unprivileged, PostgreSQL, Qdrant e Ollama.
|
||||
Il bootstrap scarica dinamicamente `qwen3-embedding:0.6b`. I lockfile applicativi non inventariano i
|
||||
package OS delle immagini né la licenza dei pesi scaricati. Prima della distribuzione occorre
|
||||
generare un SBOM per ogni immagine finale, archiviare i notice dei base image e acquisire la licenza
|
||||
del modello esatto; il nome del modello senza digest non è una prova sufficiente.
|
||||
|
||||
## Contributi e copyright
|
||||
|
||||
Per l'open-core proposto non è necessario acquisire il copyright di ogni contributore se
|
||||
l'Enterprise resta un'opera separata. Apache-2.0 §5 fornisce un default inbound=outbound, ma non è
|
||||
un trasferimento di titolarità.
|
||||
|
||||
Raccomandazione iniziale:
|
||||
|
||||
1. `CONTRIBUTING.md` con Apache-2.0 inbound=outbound e Developer Certificate of Origin;
|
||||
2. sign-off obbligatorio e verifica automatica;
|
||||
3. CLA soltanto se TYL vuole in futuro dual-license **gli stessi file** Community, cambiare licenza
|
||||
unilateralmente o ottenere concessioni brevettuali/di relicensing ulteriori;
|
||||
4. registro di provenienza per codice, asset, modelli e dipendenze vendorizzate;
|
||||
5. policy marchio separata dalla licenza del codice.
|
||||
|
||||
Fonti primarie: [Apache-2.0 §5](https://www.apache.org/licenses/LICENSE-2.0),
|
||||
[Developer Certificate of Origin](https://developercertificate.org/) e
|
||||
[Apache Contributor Agreements](https://www.apache.org/licenses/contributor-agreements.html).
|
||||
|
||||
## Gate prima della pubblicazione
|
||||
|
||||
1. Sostituire YAKE o ottenere e verificare una licenza commerciale compatibile con entrambe le
|
||||
edizioni; la sostituzione è preferibile perché YAKE serve soltanto a derivare il nome sessione.
|
||||
2. Risolvere la provenienza MIT/Apache di `thoth_lsh.py` e correggere `VENDORED.md`.
|
||||
3. Fissare per iscritto l'elenco di directory Community; spostare Enterprise in artefatti separati.
|
||||
4. Aggiungere `LICENSE` Apache-2.0, copyright TYL, `NOTICE`, `THIRD_PARTY_NOTICES` e campi SPDX nei
|
||||
manifest pubblicati.
|
||||
5. Generare SBOM e license report dai lockfile e dalle immagini; fallire la release su licenze non
|
||||
ammesse, metadata mancanti o modelli senza licenza acquisita.
|
||||
6. Verificare titolarità degli alias Git e condizioni degli output AI; decidere se prototype e
|
||||
presentazione fanno parte del rilascio.
|
||||
7. Far validare da un legale: EULA Enterprise, separazione dei moduli, politica contributi/CLA,
|
||||
marchio, notice e dipendenze LGPL/MPL/OFL.
|
||||
|
||||
Con questi gate chiusi, Apache-2.0 offre il miglior equilibrio fra reputazione, adozione enterprise,
|
||||
chiarezza brevettuale e possibilità di vendere capacità superiori in moduli proprietari separati.
|
||||
@@ -1,218 +0,0 @@
|
||||
# Local sensitive-column classifier: TypeScript or Python
|
||||
|
||||
**Date:** 2026-09-02
|
||||
**Scope:** library/runtime choice for a local classifier without a generative LLM. Synthetic data generation and
|
||||
model-boundary policy for description generation are deliberately out of scope.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Implement the mandatory classifier **inside the TypeScript backend**, with no learned model in its
|
||||
mandatory path. The agreed policy is dominated by deterministic work:
|
||||
type/declared-size rules, bounded string normalization, patterns, checksums, exact dictionaries,
|
||||
context terms, and "one match makes the column sensitive". TypeScript is fully adequate for that
|
||||
work and keeps the classifier in the process which already owns the catalog, source connectors,
|
||||
run lifecycle, cancellation, and the human override.
|
||||
|
||||
Use small, focused dependencies rather than a general NLP framework:
|
||||
|
||||
- `validator` for syntax/checksum-oriented recognizers such as email, IP, MAC, IBAN, BIC, payment
|
||||
cards, and locale-aware Italian tax IDs, passports, identity cards and VAT numbers; its official
|
||||
API exposes these validators individually, so unused validators need not be imported
|
||||
([validator.js source and API](https://github.com/validatorjs/validator.js/)).
|
||||
- `libphonenumber-js/max` for international telephone parsing and digit-pattern validation. The
|
||||
project's `max` metadata is intentionally more precise than its default/minimal metadata
|
||||
([libphonenumber-js validation documentation](https://github.com/catamphetamine/libphonenumber-js/blob/master/README.md#isvalid-boolean)).
|
||||
- ThothII-owned recognizers for identifiers not covered adequately by a selected validator (for
|
||||
example the Italian driving licence) and organization-specific identifiers, each with a version,
|
||||
tests, context words, and checksum/semantic validation where the identifier defines one.
|
||||
- A safe regular-expression boundary for administrator-authored rules. `node-re2` offers a mostly
|
||||
`RegExp`-compatible, ReDoS-resistant engine and a set API for matching many patterns, but is a
|
||||
native addon that downloads or builds a binary during installation. That packaging cost must be
|
||||
tested against the project images before adoption
|
||||
([node-re2 README](https://github.com/uhop/node-re2/blob/master/README.md)). If arbitrary regular
|
||||
expressions are not exposed, ThothII can instead keep a reviewed built-in pattern set and bound
|
||||
every inspected value.
|
||||
|
||||
Do **not** add NLP.js, spaCy, Presidio, or a transformer merely to implement regexes and dictionaries.
|
||||
They do not make deterministic identifiers more semantically understandable. In particular, NLP.js
|
||||
documents useful Italian tokenization plus enum/regex/built-in entities, but its advertised built-ins
|
||||
are mostly the same formatted values already covered above; it does not advertise a ready-made
|
||||
Italian PII model for arbitrary people or clinical concepts
|
||||
([NLP.js official README](https://github.com/axa-group/nlp.js/)).
|
||||
|
||||
Add an **optional, non-generative statistical NER pack** based on
|
||||
[`fastino/gliner2-privacy-filter-PII-multi`](https://huggingface.co/fastino/gliner2-privacy-filter-PII-multi).
|
||||
The selected model and its GLiNER2 runtime are Apache-2.0, its mDeBERTa base is MIT, and the model
|
||||
is trained across seven languages including Italian. Run the official implementation in one warmed,
|
||||
CPU-only Python worker; do not spawn it once per column or table. It returns entity evidence only.
|
||||
The TypeScript classifier remains the sole owner of the final column assessment and invokes NER
|
||||
only for bounded text that deterministic rules did not resolve and only while the scan deadline has
|
||||
time remaining.
|
||||
|
||||
This is a positive Q29 decision, not a placeholder for later library selection. The production gate
|
||||
is limited to pinning artifacts, producing an SBOM/license inventory, and measuring the selected
|
||||
model on PSD fixtures and target CPUs. If the optional pack is absent or misses its deadline, the
|
||||
normal coverage rule produces `unknown`; ThothII does not substitute a generative LLM.
|
||||
|
||||
## Why the TypeScript baseline is sufficient
|
||||
|
||||
The classifier is not being asked to infer an open-ended privacy judgment from prose. It executes a
|
||||
versioned policy and produces evidence. Its mandatory recognizers can be expressed as:
|
||||
|
||||
| Rule family | Implementation | Model needed? |
|
||||
| --- | --- | --- |
|
||||
| declared `text`/CLOB/unbounded string or declared maximum `>500` | catalog metadata rule | no |
|
||||
| an observed value longer than 500 characters | length rule, ideally detected in SQL before transferring the value | no |
|
||||
| email, IP/MAC, URL, UUID, payment card, IBAN/BIC, phone | pattern plus format/checksum validator | no |
|
||||
| Italian fiscal code/VAT/passport/identity card/driving licence | locale-aware validator where available, otherwise a reviewed country recognizer; pattern, context and checksum where defined | no |
|
||||
| credentials and technical secrets | anchored prefixes, token shapes, entropy/character-class heuristics, and organization allow/deny rules | no |
|
||||
| hospital- or customer-specific terms/codes | normalized exact dictionary or phrase matching | no |
|
||||
| person/location/organization in otherwise unstructured short text | statistical NER is useful but probabilistic | yes, optional |
|
||||
| clinical meaning in unstructured short Italian text | domain NER or a governed terminology; generic NER is not enough | optional and domain-specific |
|
||||
|
||||
This is not speculative parity with a Python implementation. Presidio's own supported-entity table
|
||||
shows that its global email, IBAN, credit-card and similar recognizers use pattern matching,
|
||||
validation, context and checksums, while its Italian fiscal-code/VAT/passport/identity-card/licence
|
||||
coverage is likewise rule-based
|
||||
([Presidio supported entities](https://presidio.dataprivacystack.org/supported_entities/)). Those
|
||||
mechanisms are portable; Presidio provides tested implementations and orchestration, not a unique
|
||||
Python-language capability.
|
||||
|
||||
For large dictionaries, exact normalized token/phrase matching is still deterministic NLP and does
|
||||
not require a learned model. The policy format should describe the rule rather than the library,
|
||||
for example `kind: email`, `kind: checksum_id`, `kind: regex`, `kind: phrase_set`, and
|
||||
`kind: max_length`. This makes a future engine change possible without changing persisted policy or
|
||||
assessment evidence.
|
||||
|
||||
## What Python/Presidio would materially add
|
||||
|
||||
Presidio is the strongest Python option if ThothII later needs a broader recognizer ecosystem. Its
|
||||
Analyzer composes rule-based recognizers, regexes, validation, NER and contextual enhancement; it
|
||||
also exposes custom recognizers, batch processing and decision tracing
|
||||
([Presidio Analyzer architecture](https://presidio.dataprivacystack.org/analyzer/)). Its current
|
||||
catalog includes Italian identifiers and its YAML registry supports custom patterns and language-
|
||||
specific context. Recent source also contains a `NoOpNlpEngine`, so a deterministic-only Presidio
|
||||
deployment can avoid loading spaCy; this removes model cost but not the Python runtime/process
|
||||
boundary
|
||||
([Presidio `NoOpNlpEngine` integration](https://github.com/data-privacy-stack/presidio/blob/main/presidio-analyzer/presidio_analyzer/recognizer_registry/recognizer_registry.py)).
|
||||
|
||||
`presidio-structured` is conceptually close because it maps tabular columns/JSON keys to detected
|
||||
entities, but its documented strategies select the most common, highest-confidence, or a mixed
|
||||
entity. ThothII would still need its own deadline/sampling/`unknown` semantics and human-override
|
||||
lifecycle; Presidio also lists improved mixed free-text/structured support as future work
|
||||
([Presidio Structured](https://presidio.dataprivacystack.org/structured/)).
|
||||
|
||||
That benefit does not currently justify using it as ThothII's mandatory engine:
|
||||
|
||||
- The backend currently has no PII/NLP dependencies and is an ESM TypeScript service
|
||||
(`backend/package.json`); the existing Python environment belongs to the separate harness/core
|
||||
layer and also has no Presidio or spaCy dependency (`harness/pyproject.toml`). A Python analyzer therefore means a new
|
||||
ownership and deployment boundary, not simply another import.
|
||||
- Presidio is Python-based and can be run as a package or REST container. Its REST API intentionally
|
||||
has no built-in authentication, so a sidecar would have to remain on a protected internal network
|
||||
and be guarded by infrastructure
|
||||
([Presidio FAQ, deployment](https://presidio.dataprivacystack.org/faq/#how-can-i-deploy-presidio-into-my-environment)).
|
||||
- Presidio defaults to English. Adding Italian requires both an Italian NLP pipeline and adapted
|
||||
language-specific recognizers/context; language-agnostic regexes alone do not solve that setup
|
||||
([Presidio multilingual guidance](https://github.com/data-privacy-stack/presidio/blob/main/docs/analyzer/languages.md)).
|
||||
- Presidio itself warns that automated detection cannot guarantee finding all sensitive data and
|
||||
documents an unavoidable false-positive/false-negative trade-off
|
||||
([Presidio FAQ](https://presidio.dataprivacystack.org/faq/#what-is-presidio)). It cannot turn a
|
||||
negative sample into proof that a column is safe.
|
||||
|
||||
## Learned NLP is not an LLM, but it is still a model
|
||||
|
||||
A spaCy NER pipeline does not generate text and is not an LLM. It is a statistical token classifier:
|
||||
it predicts spans such as `PERSON`, `LOCATION` and `ORGANIZATION`. spaCy explicitly notes that the
|
||||
predictions depend on the training examples and will not always be correct
|
||||
([spaCy NER documentation](https://spacy.io/usage/linguistic-features#named-entities)). That makes it
|
||||
a useful additional **positive detector**, not the authority which declares a sampled column safe.
|
||||
|
||||
spaCy publishes Italian CPU pipelines with an NER component, but they are trained on news/media
|
||||
data rather than clinical notes. More importantly, all three official packages (`it_core_news_sm`,
|
||||
`md`, and `lg`) are licensed **CC BY-NC-SA 3.0**, so they are a no-go for a product which can be
|
||||
used commercially; using the MIT-licensed spaCy runtime does not change the weights' license
|
||||
([spaCy Italian pipelines](https://spacy.io/models/it),
|
||||
[`sm` metadata](https://raw.githubusercontent.com/explosion/spacy-models/master/meta/it_core_news_sm-3.8.0.json),
|
||||
[`md` metadata](https://raw.githubusercontent.com/explosion/spacy-models/master/meta/it_core_news_md-3.8.0.json),
|
||||
[`lg` metadata](https://raw.githubusercontent.com/explosion/spacy-models/master/meta/it_core_news_lg-3.8.0.json)). A general Italian model also recognizes
|
||||
names and places, not all sensitive clinical meaning. Training a suitable model would require a
|
||||
labeled, representative and legally usable corpus; Presidio's own model guidance likewise states
|
||||
that a labeled PII dataset is required to train a new model
|
||||
([Presidio spaCy/Stanza guidance](https://github.com/data-privacy-stack/presidio/blob/main/docs/analyzer/nlp_engines/spacy_stanza.md#training-your-own-model)).
|
||||
|
||||
Consequently, the absence of a local LLM is irrelevant to sensitivity assessment. Installations
|
||||
with a local LLM may let description generation receive broader source content under a separate
|
||||
model-boundary policy, but **the sensitivity classifier does not call that LLM**. The optional NER
|
||||
pack is a distinct local classifier capability and is never inferred from the description-model
|
||||
catalog.
|
||||
|
||||
## Licensing and redistribution gate
|
||||
|
||||
Framework code, model weights, and training data are three separate licensing layers. A permissive
|
||||
runtime does not grant ThothII permission to bundle weights trained from restricted data. For an
|
||||
open-source product that may be deployed commercially, the default rule should therefore be:
|
||||
**reject `NC`, research-only, custom restrictive, missing, or ambiguous model licenses**. Pinning an
|
||||
approved artifact must also pin its model card, upstream model, training datasets, required notices,
|
||||
and checksums. This is a technical compatibility screen, not legal advice.
|
||||
|
||||
| Candidate | Framework code | Weights and training-data evidence | Commercial bundling in ThothII | Decision |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Presidio, deterministic recognizers only | MIT; its license permits use and redistribution with the notice ([Presidio license](https://github.com/data-privacy-stack/presidio/blob/main/LICENSE)) | No general Italian model is required with `NoOpNlpEngine`; third-party dependencies still retain their notices | Compatible, but adds a Python process without adding unique detection capability for the agreed baseline | **GO legally; NO-GO architecturally for v1** |
|
||||
| spaCy runtime | MIT ([spaCy license](https://github.com/explosion/spaCy/blob/master/LICENSE)) | Runtime only; no permission is conferred on downloaded pipelines | Compatible if used with separately approved weights | **GO for code only** |
|
||||
| Official spaCy Italian `sm`/`md`/`lg` | spaCy code is MIT | Every current package declares CC BY-NC-SA 3.0; the metadata traces part of training to UD Italian ISDT under the same non-commercial license ([`sm`](https://raw.githubusercontent.com/explosion/spacy-models/master/meta/it_core_news_sm-3.8.0.json), [`md`](https://raw.githubusercontent.com/explosion/spacy-models/master/meta/it_core_news_md-3.8.0.json), [`lg`](https://raw.githubusercontent.com/explosion/spacy-models/master/meta/it_core_news_lg-3.8.0.json)) | `NC` excludes the intended commercial deployments; `SA` adds redistribution obligations | **NO-GO** |
|
||||
| Stanza runtime | Apache-2.0 ([Stanza license](https://github.com/stanfordnlp/stanza/blob/main/LICENSE)) | Stanford says its language packs are ODC-By only to the extent it owns the rights and explicitly tells users to inspect training-data licenses. Italian NER uses FBK's KIND data; KIND annotations are CC BY-NC 4.0 or CC BY-NC-SA 4.0 ([Stanza model licensing note](https://stanfordnlp.github.io/stanza/performance.html), [Italian NER source](https://stanfordnlp.github.io/stanza/ner_models.html), [KIND license](https://github.com/dhfbk/KIND#license)) | The Italian model's non-commercial source data fails the product requirement | **GO for code; NO-GO for Italian weights** |
|
||||
| Flair runtime and official multilingual NER | MIT ([Flair repository and license](https://github.com/flairNLP/flair)) | The official `ner-multi` model card neither includes Italian among its four trained languages nor declares a model license; a maintainer issue asking whether bundled weights are MIT was closed without an answer ([model card](https://huggingface.co/flair/ner-multi), [license issue](https://github.com/flairNLP/flair/issues/1487)) | Runtime is usable, but these weights are unsuitable and their redistribution terms are not established | **GO for code; NO-GO for these weights** |
|
||||
| `osiria/flare-it-ner` | Runs in Transformers/PyTorch; the card labels the weights MIT | Trained on CC BY 4.0 WikiNER plus an additional manually annotated custom dataset whose redistribution terms are not identified in the model card ([model card](https://huggingface.co/osiria/flare-it-ner), [WikiNER dataset](https://figshare.com/articles/dataset/Learning_multilingual_named_entity_recognition_from_Wikipedia/5462500)) | The weights look promising, but an MIT label alone does not resolve the undocumented custom training-data rights | **NO-GO until provenance is explicit** |
|
||||
| Transformers.js / ONNX runtime | Apache-2.0 ([Transformers.js license](https://github.com/huggingface/transformers.js/blob/main/LICENSE)) | ONNX conversion does not replace the source model's license or cure training-data restrictions; each exact model needs its own audit | Compatible runtime, but there is no blanket approval for arbitrary Hub/ONNX weights | **GO for code; weights case by case** |
|
||||
| `Laibniz/italian-ner-pii-browser-uncased` | Packaged for Transformers.js as quantized ONNX; model card says Apache-2.0 | It is a conversion of an Osiria model and therefore inherits the same undocumented custom-data provenance ([model card](https://huggingface.co/Laibniz/italian-ner-pii-browser-uncased)) | CPU/browser packaging does not cure the upstream licensing gap | **NO-GO** |
|
||||
| `Ar86Bat/multilang-pii-ner` | XLM-R/Transformers; model card labels weights MIT | Trained on `ai4privacy/open-pii-masking-500k-ai4privacy`, whose card declares CC BY 4.0 ([model card](https://huggingface.co/Ar86Bat/multilang-pii-ner), [dataset card](https://huggingface.co/datasets/ai4privacy/open-pii-masking-500k-ai4privacy)) | Commercial redistribution appears compatible with MIT notice plus CC BY attribution; an ONNX export, dependency audit, and PSD evaluation are still required | **CONDITIONAL GO as optional PII NER** |
|
||||
| `urchade/gliner_multi_pii-v1` | GLiNER code and model are Apache-2.0 | Its published multilingual synthetic dataset, including Italian, is Apache-2.0 ([model card](https://huggingface.co/urchade/gliner_multi_pii-v1), [dataset card](https://huggingface.co/datasets/urchade/synthetic-pii-ner-mistral-v1)) | The cleanest publicly inspectable license chain, but the published SPY comparison reports substantially lower recall than the selected Fastino model | **GO legally; reserve candidate** |
|
||||
| `fastino/gliner2-privacy-filter-PII-multi` | GLiNER2 code is Apache-2.0; the direct local dependencies are permissively licensed | Weights are Apache-2.0 and the mDeBERTa base is MIT. The authors report a purpose-built synthetic corpus spanning seven languages, including Italian; the corpus itself is not published, so training is not fully reproducible ([model card](https://huggingface.co/fastino/gliner2-privacy-filter-PII-multi), [paper](https://arxiv.org/abs/2605.09973), [base model](https://huggingface.co/microsoft/mdeberta-v3-base)) | Free local/commercial use and redistribution are allowed subject to Apache notices. It has the best reported exact-span F1 and recall among the compared open detectors on SPY, although that evaluation is not Italian-specific | **SELECTED optional NER pack** |
|
||||
| `openai/privacy-filter` | Code and weights are Apache-2.0 and it runs locally through Python or Transformers.js | The publisher explicitly permits commercial deployment, but documents the model as primarily English and warns that non-English performance may drop ([model card](https://huggingface.co/openai/privacy-filter), [source](https://github.com/openai/privacy-filter)) | Clean license and direct TypeScript path, but insufficient Italian evidence and a larger one-billion-parameter artifact make it a weaker fit | **GO legally; NO-GO as primary Italian detector** |
|
||||
| OpenMed Italian PII models | Cards label the weights Apache-2.0 | They are trained on `ai4privacy/pii-masking-400k`, whose license limits use to academic/non-commercial purposes unless a commercial agreement is obtained ([representative model](https://huggingface.co/OpenMed/OpenMed-PII-Italian-BioClinicalBERT-Base-110M-v1), [dataset license](https://huggingface.co/datasets/ai4privacy/pii-masking-400k/blob/main/LICENSE)) | An Apache label on derivative weights does not remove the explicit upstream non-commercial restriction | **NO-GO** |
|
||||
|
||||
The search therefore found a usable, free-of-charge, permissively licensed solution. Fastino
|
||||
GLiNER2-PII is selected because it combines an Apache code/weight grant, explicit Italian training,
|
||||
CPU operation, configurable entity labels, and materially higher published recall than the older
|
||||
Urchade model. This does not prove Italian clinical accuracy: the published SPY evaluation is
|
||||
English and the training corpus is synthetic. Those are quality and reproducibility limitations,
|
||||
not a reason to reintroduce a generative LLM into the classifier.
|
||||
|
||||
## Fit with current ThothII design
|
||||
|
||||
The current backend already owns source sampling and the human-owned `Sensitive Data Flag`; source
|
||||
values are transient and existing description sampling is bounded (`PROJECT_STATE.md` and
|
||||
`backend/src/catalog/description-source-sampler.ts`). The new
|
||||
classifier should therefore be a backend module behind the single effective sensitivity-decision
|
||||
point already being designed, while preserving these agreed semantics:
|
||||
|
||||
1. `sensitive`, `non_sensitive`, and `unknown` are assessment outcomes; `unknown` leaves the
|
||||
human-owned Sensitive Data Flag unchanged. Its effect on later description generation is out of
|
||||
scope here.
|
||||
2. Any positive rule match stops the column scan and yields `sensitive`.
|
||||
3. A negative time-bounded sample yields `unknown`, never `non_sensitive`.
|
||||
4. The administrator may set the final binary flag either way; the automated result remains a
|
||||
proposal with rule/version/coverage evidence and no stored source value.
|
||||
5. Learned NER, if present, can add positive evidence but cannot convert `unknown` to
|
||||
`non_sensitive`.
|
||||
|
||||
The five-second scan budget should be enforced at the database-read orchestration layer and not
|
||||
inside a recognizer library. An in-process deterministic engine maximizes the portion of that budget
|
||||
available to database reads and avoids model cold-start/IPC costs. It should stream bounded batches,
|
||||
check cancellation/deadline between batches, and stop at the first match.
|
||||
|
||||
## Acceptance gate for enabling the selected NER pack
|
||||
|
||||
Build a versioned, value-free fixture corpus representing at least: Italian/general identifiers,
|
||||
false-positive lookalikes, bounded short notes with people/places, credential formats, null/high-
|
||||
cardinality columns, and PSD-specific clinical codes. Measure per category rather than one aggregate
|
||||
score. Pin the GLiNER2 package and model revision/checksum, build an SBOM including transitive
|
||||
dependencies, retain required notices, prohibit model downloads at inference time, and measure warm
|
||||
latency and memory on the target CPU. The detector is enabled only where those checks pass.
|
||||
|
||||
The acceptance test may tune label descriptions and per-label confidence thresholds, but it does
|
||||
not reopen the architecture or silently select another Hub model. If quality or performance is
|
||||
unacceptable, the optional NER pack remains disabled and unresolved sampled columns remain
|
||||
`unknown`. Replacing the chosen model or adding a local-LLM fallback requires a new documented
|
||||
decision.
|
||||
@@ -0,0 +1,256 @@
|
||||
# Prodotti open-core comparabili e meccanismi di conversione per ThothII
|
||||
|
||||
Ricerca originale: 2026-09-02
|
||||
|
||||
Issue: [Studiare prodotti open-core comparabili e i loro meccanismi di conversione](https://git.tylconsulting.it/mptyl/ThothII/issues/25)
|
||||
|
||||
## Domanda
|
||||
|
||||
Come separano core gratuito e capacità Enterprise prodotti open-core comparabili nei data tool e
|
||||
developer tool, quale bisogno induce la conversione e quali anti-pattern hanno danneggiato adozione
|
||||
o fiducia? Quali lezioni sono applicabili a ThothII come Customer-Hosted Installation?
|
||||
|
||||
## Risposta in breve
|
||||
|
||||
Il pattern più solido non è «far pagare il risultato migliore», ma lasciare aperto un prodotto che
|
||||
completa davvero il lavoro fondamentale e far pagare il costo organizzativo del suo uso su scala:
|
||||
identità e autorizzazioni, policy, audit di conformità, esercizio production-grade, integrazioni
|
||||
governate e responsabilità contrattuale.
|
||||
|
||||
Metabase è il confronto più vicino a ThothII: nell'edizione open include interrogazione, generazione
|
||||
SQL, funzioni AI, modello portato dal cliente, MCP, Agent API e CLI; Pro aggiunge permessi granulari,
|
||||
SSO e auditing, mentre Enterprise differenzia soprattutto deployment air-gapped, procurement e
|
||||
SLA. GitLab e Grafana confermano lo stesso schema nei developer tool: il lavoro quotidiano resta
|
||||
nel core e la conversione è indotta dalla crescita dell'organizzazione o dal rischio operativo.
|
||||
|
||||
Per ThothII ne segue una separazione consigliata:
|
||||
|
||||
- **Community Edition open source:** percorso completo domanda → artifact SQL revisionato, Evidence,
|
||||
catalogo, gate umani, BYOM/embedding, connettori fondamentali, CLI e contratti di estensione,
|
||||
installazione singola sicura e utilizzabile in produzione;
|
||||
- **Enterprise Edition commerciale:** federazione delle identità e group sync, RBAC/policy avanzati,
|
||||
audit esportabile e resistente alle manomissioni, segregazione dei ruoli, HA/DR/backup e upgrade
|
||||
governati, pacchetti air-gap, integrazioni ETL/CI/CD e secret manager certificate, supporto e SLA.
|
||||
|
||||
La qualità del workflow F1–F8, la leggibilità degli artifact e la sicurezza minima non devono essere
|
||||
paywall. Una Community Edition incapace di provare il vantaggio distintivo di ThothII non costruisce
|
||||
né reputazione né un futuro funnel commerciale.
|
||||
|
||||
## Metodo e limiti
|
||||
|
||||
La ricerca usa documentazione, licenze e pagine di prezzo ufficiali correnti al 2 settembre 2026.
|
||||
Le aziende non pubblicano dati di funnel sufficienti a dimostrare quali singole feature causino una
|
||||
conversione. I «trigger» sotto sono quindi inferenze esplicite ricavate dal confine dei piani, dal
|
||||
buyer descritto e dal meccanismo di prezzo; non sono tassi di conversione osservati.
|
||||
|
||||
## Confronto
|
||||
|
||||
| Prodotto | Core gratuito | Confine commerciale | Trigger di conversione inferito | Lezione per ThothII |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| **Metabase** | Open Source Edition in AGPL. Il piano open include query builder, editor SQL, AI, BYOM, MCP, Agent API e CLI. | Pro aggiunge SSO, permessi row/column, controlli e auditing; Enterprise aggiunge air-gap, procurement, success engineer e SLA. | Dati condivisi fra più gruppi o tenant, requisiti normativi, deployment mission-critical. | Analogo più forte: aprire il motore AI/SQL completo e monetizzare governo e affidabilità. |
|
||||
| **GitLab** | Community Edition MIT; il prodotto Free conserva repository e workflow DevOps fondamentali. | Enterprise Edition ha codice dedicato e feature Premium/Ultimate; Premium punta a organizzazioni in crescita, Ultimate a sicurezza e compliance. | Coordinamento su scala, controlli di processo, security e compliance. | Tenere stabile il core e far emergere il valore Enterprise quando crescono attori, ambienti e rischio. |
|
||||
| **Grafana** | Grafana OSS è AGPL e resta un prodotto completo di visualizzazione e dashboard. | Enterprise aggiunge RBAC, SAML/team sync, permessi sui data source, audit, Vault, plugin premium e supporto 24/7. | Accesso a dati sensibili, integrazione con stack aziendale, conformità e responsabilità operativa. | Vendere controllo e integrazioni certificate, non la capacità fondamentale di interrogare e capire i dati. |
|
||||
| **Mattermost** | Team Edition è MIT e self-hosted; offre il lavoro collaborativo fondamentale. | Un'edizione commerciale self-hosted abilita feature tramite licenza; Entry è il fallback gratuito con limiti e omissioni. | SSO, compliance, HA, supporto e dimensione del deployment. | Il singolo artefatto installabile può semplificare upgrade e trial, ma i limiti quantitativi e il riposizionamento della Community creano confusione. |
|
||||
| **Airbyte** | Il protocollo è MIT; Core e connector sono oggi ELv2, utilizzabili internamente ma con restrizione alla rivendita come servizio. Il motore di replica resta gratuito. | Offerte commerciali aggiungono governance, supporto/SLA e deployment controllati; l'offerta corrente Enterprise Flex comprende anche un control plane gestito. | Sovranità, governance multi-workspace, secret manager, operatività e supporto. | Utile per il confine funzionale, non come modello di licenza o architettura: ELv2 non è una licenza open source OSI e Flex non è interamente customer-hosted. |
|
||||
|
||||
### Metabase: il comparabile più vicino
|
||||
|
||||
Metabase pubblica l'Open Source Edition in AGPL e distribuisce la parte Enterprise con licenza
|
||||
commerciale; nel repository le due famiglie di codice sono separate anche per directory
|
||||
([licenze Metabase](https://www.metabase.com/license/),
|
||||
[LICENSE del repository](https://github.com/metabase/metabase/blob/master/LICENSE.txt)). La
|
||||
[matrice corrente dei piani](https://www.metabase.com/pricing/compare-plans) mantiene in Open Source
|
||||
le funzioni che dimostrano il valore del prodotto: domande AI, BYOM, MCP, Agent API, CLI, query
|
||||
builder ed editor SQL. Sposta invece in Pro permessi row/column, SSO, controlli AI, analytics d'uso
|
||||
e auditing.
|
||||
|
||||
La [pagina prezzi](https://www.metabase.com/pricing) esplicita anche due scelte economiche utili:
|
||||
Open Source ha utenti illimitati; Pro combina un minimo mensile con prezzo per utente; Enterprise
|
||||
parte da 20.000 USD/anno e risponde a procurement, air-gap e SLA. Inoltre Pro ed Enterprise hanno
|
||||
lo stesso insieme di feature principali: il salto Enterprise vende soprattutto modalità di
|
||||
fornitura e responsabilità. Il pricing non va copiato, ma il segnale è pertinente: il cliente
|
||||
regolato non paga per una query «più intelligente», paga perché il sistema può essere adottato e
|
||||
presidiato secondo le regole dell'organizzazione.
|
||||
|
||||
### GitLab: separazione tecnica e crescita organizzativa
|
||||
|
||||
GitLab dichiara la Community Edition sotto MIT e la Enterprise Edition sotto licenza più
|
||||
restrittiva ([licensing](https://docs.gitlab.com/development/licensing/)). In un unico codebase, il
|
||||
codice Enterprise risiede in `ee/`; senza licenza, una build EE deve comportarsi come CE
|
||||
([linee guida EE](https://docs.gitlab.com/development/ee_features/)). Questo riduce il rischio che
|
||||
l'installazione perda la funzione fondamentale quando una licenza scade e consente di testare
|
||||
esplicitamente sia la modalità Community sia quella Enterprise.
|
||||
|
||||
Il [pricing corrente](https://about.gitlab.com/pricing/) presenta Premium a 29 USD per utente/mese
|
||||
per organizzazioni in crescita e Ultimate a prezzo personalizzato per sicurezza e compliance. Il
|
||||
trigger non è l'accesso al repository o alla CI di base: è il passaggio da un team che produce a
|
||||
un'organizzazione che deve coordinare, dimostrare controlli e ridurre rischio. Per ThothII il pattern
|
||||
tecnico è replicabile, mentre il prezzo per seat lo è meno: pochi specialisti possono produrre un
|
||||
datamart di valore per un'intera organizzazione, quindi il numero degli utenti non misura bene il
|
||||
valore generato.
|
||||
|
||||
### Grafana: integrazioni, governance e supporto
|
||||
|
||||
Grafana OSS è passato da Apache-2.0 ad AGPLv3 per le versioni correnti
|
||||
([FAQ di licensing](https://grafana.com/licensing/)). Grafana Enterprise è una build commerciale
|
||||
che aggiunge RBAC, permessi sui data source, SAML e sincronizzazione dei team, audit, integrazione
|
||||
Vault, reporting, plugin premium e supporto continuativo
|
||||
([feature Enterprise](https://grafana.com/docs/grafana/latest/introduction/grafana-enterprise/)).
|
||||
Sono capacità che acquistano valore quando dashboard e data source diventano infrastruttura
|
||||
aziendale condivisa.
|
||||
|
||||
L'[attivazione Enterprise](https://grafana.com/docs/grafana/latest/administration/enterprise-licensing/)
|
||||
usa una licenza legata all'istanza e agli utenti attivi; per ambienti senza Internet sono previste
|
||||
modalità concordate con il vendor. La stessa documentazione mostra un buon principio di scadenza:
|
||||
diverse configurazioni di sicurezza già applicate continuano a funzionare, mentre non è possibile
|
||||
crearene o modificarne di nuove. ThothII dovrebbe essere ancora più conservativo: una scadenza non
|
||||
deve rendere illeggibili gli artifact, invalidare decisioni persistite o interrompere un workflow
|
||||
Community già consentito.
|
||||
|
||||
### Mattermost: distribuzione semplice, ma confine instabile
|
||||
|
||||
Mattermost usa due pattern contemporaneamente: Team Edition è una piattaforma self-hosted MIT;
|
||||
l'Enterprise Edition è un binario commerciale che può funzionare gratuitamente in modalità Entry
|
||||
e sblocca le funzioni a pagamento con una licenza
|
||||
([edizioni](https://docs.mattermost.com/product-overview/editions-and-offerings.html),
|
||||
[subscription self-hosted](https://docs.mattermost.com/product-overview/self-hosted-subscriptions.html)).
|
||||
È un precedente utile per consegnare a un cliente un solo pacchetto Enterprise che degrada in modo
|
||||
prevedibile a un set gratuito.
|
||||
|
||||
La documentazione corrente racconta però anche il costo di un confine non stabile: Team Edition era
|
||||
stata usata molto oltre il pubblico previsto e alcune integrazioni SSO sovrapponevano capacità
|
||||
commerciali; il prodotto ha quindi rimosso SSO dalla Team Edition e l'ha riposizionata per piccoli
|
||||
team. Per ThothII è preferibile fissare prima del lancio una promessa Community durevole e non
|
||||
ritirare in seguito capacità che gli utenti hanno ragionevolmente considerato parte del core.
|
||||
|
||||
### Airbyte: completezza del motore, ma ambiguità terminologica
|
||||
|
||||
Airbyte lascia gratuito il motore di replica e un ampio ecosistema di connector; l'offerta
|
||||
Enterprise ha aggiunto nel tempo multi-tenancy, RBAC, PII masking, secret manager esterni, air-gap,
|
||||
API/Terraform e supporto ([annuncio Self-Managed Enterprise](https://airbyte.com/blog/1-0-enterprise-ga),
|
||||
[pricing corrente](https://airbyte.com/pricing)). È un altro esempio di core che svolge davvero il
|
||||
lavoro e di conversione legata all'industrializzazione.
|
||||
|
||||
Il limite come modello per ThothII è duplice. La [pagina licensing corrente](https://airbyte.com/why-open-source)
|
||||
definisce MIT il protocollo, ma ELv2 Core e connector: ELv2 vieta di rivendere il prodotto come
|
||||
servizio e quindi introduce una restrizione di campo d'uso. Secondo la
|
||||
[Open Source Definition](https://opensource.org/osd), una licenza open source non può imporre tale
|
||||
restrizione. Inoltre Enterprise Flex oggi comprende un control plane gestito, incompatibile con la
|
||||
decisione ThothII che domande, prompt, metadati e risultati non escano dal trust boundary del
|
||||
cliente. Airbyte è quindi un comparabile di packaging, non una licenza o topologia da copiare.
|
||||
|
||||
## Anti-pattern osservabili
|
||||
|
||||
### 1. Rendere incompleto il lavoro fondamentale
|
||||
|
||||
I comparabili più credibili non riservano all'Enterprise l'azione che genera la prima prova di
|
||||
valore: Metabase non blocca AI/SQL/BYOM; GitLab non blocca source control e CI di base; Grafana non
|
||||
blocca dashboard e interrogazione; Airbyte non blocca il motore di replica. Per ThothII, mettere a
|
||||
pagamento fasi F1–F8, qualità SQL, Evidence o review fondamentale trasformerebbe la Community in una
|
||||
demo e ridurrebbe la reputazione che l'apertura deve costruire.
|
||||
|
||||
### 2. Mettere dietro licenza la sicurezza minima
|
||||
|
||||
SSO federato, group sync, ruoli personalizzati e audit di conformità sono boundary Enterprise
|
||||
ricorrenti. Non ne segue che una Community possa essere insicura. ThothII Community deve conservare
|
||||
autenticazione e autorizzazione di base, secret references, accesso DWH read-only, decision ledger,
|
||||
provenienza degli artifact e log operativi minimi. Enterprise può aggiungere segregazione dei
|
||||
compiti, policy obbligatorie, retention, export SIEM e audit resistente alle manomissioni.
|
||||
|
||||
### 3. Usare «open source» per una licenza source-available
|
||||
|
||||
MIT, Apache-2.0, AGPL e le altre licenze approvate OSI permettono uso per qualsiasi scopo. Una
|
||||
clausola che proibisce di offrire il software come servizio può essere commercialmente legittima,
|
||||
ma cambia la categoria. Presentare ELv2 o BSL come semplice open source crea ambiguità proprio sul
|
||||
diritto che utenti e integratori devono poter valutare rapidamente. Se ThothII sceglie MIT o
|
||||
Apache-2.0 per il core, deve accettare anche fork e offerte commerciali concorrenti; la protezione
|
||||
economica deve venire da marchio, roadmap, moduli Enterprise, certificazioni e servizio.
|
||||
|
||||
### 4. Cambiare licenza dopo aver chiesto investimento all'ecosistema
|
||||
|
||||
HashiCorp passò i prodotti core da MPL-2.0 a BSL nel 2023, descrivendo BSL come source-available
|
||||
([annuncio ufficiale](https://www.hashicorp.com/en/blog/hashicorp-adopts-business-source-license)).
|
||||
Quattro settimane dopo, OpenTofu pubblicò il fork di Terraform e avviò l'ingresso nella Linux
|
||||
Foundation ([cronologia OpenTofu](https://opentofu.org/blog/the-opentofu-fork-is-now-available/)).
|
||||
Elastic cambiò da Apache-2.0 a SSPL/ELv2 nel 2021 e nel 2024 aggiunse nuovamente AGPL, una licenza
|
||||
OSI, per la parte gratuita ([FAQ Elastic](https://www.elastic.co/pricing/faq/licensing/)); OpenSearch
|
||||
documenta di essere nato dal precedente codice Apache-2.0
|
||||
([FAQ OpenSearch](https://opensearch.org/faq/)). Questi fatti non misurano il danno commerciale, ma
|
||||
dimostrano un esito verificabile: un cambio tardivo può frammentare progetto, nome ed ecosistema.
|
||||
|
||||
### 5. Applicare metriche di prezzo non allineate al valore
|
||||
|
||||
GitLab, Metabase e Grafana possono usare utenti o utenti attivi perché la platea operativa cresce
|
||||
con l'adozione. ThothII ha invece pochi operatori IT/data e artifact consumati da molti processi
|
||||
aziendali. Un prezzo puramente per seat può ostacolare collaborazione e sottovalutare installazioni
|
||||
ad alto valore; un prezzo per token è ancora meno coerente quando modello ed embedding sono BYOM.
|
||||
La ricerca suggerisce di valutare installazione, ambienti/workspace governati e livello di supporto
|
||||
come unità principali, lasciando la decisione economica definitiva al ticket di packaging/prezzo.
|
||||
|
||||
### 6. Far fallire il core alla scadenza della licenza
|
||||
|
||||
Un contratto annuale deve disabilitare soltanto capacità commerciali future o modificabili. Dati,
|
||||
artifact, revisioni e funzioni Community devono restare leggibili ed esportabili; non devono esistere
|
||||
ransom lock-in o dipendenze online obbligatorie in un ambiente air-gapped. Questo riduce il rischio
|
||||
per il buyer e rende credibile la promessa customer-hosted.
|
||||
|
||||
## Boundary raccomandato per ThothII
|
||||
|
||||
### Community Edition open source
|
||||
|
||||
- workflow deterministico F1–F8 completo e gate human-in-the-loop;
|
||||
- catalogo, Evidence, schema linking e generazione del Transformation Package;
|
||||
- configurazione BYOM e provider di embedding scelti dal cliente;
|
||||
- connettori fondamentali e accesso read-only;
|
||||
- persistenza, provenance e decision ledger necessari a riprendere e verificare il lavoro;
|
||||
- frontend, backend, CLI, contratti degli artifact e SDK/extension points;
|
||||
- installazione customer-hosted singola, backup/export manuale documentato e aggiornamenti
|
||||
Community;
|
||||
- sicurezza di base sufficiente a un uso reale, senza limiti artificiali a utenti, domande, token o
|
||||
qualità del risultato.
|
||||
|
||||
### Enterprise Edition commerciale
|
||||
|
||||
- SAML/OIDC enterprise, directory/group sync, SCIM e integrazione con identity manager;
|
||||
- RBAC personalizzato, segregazione dei ruoli, approval policy e policy pack obbligatori;
|
||||
- audit immutabile/esportabile, retention, integrazione SIEM e report di conformità;
|
||||
- HA, disaster recovery, backup/restore automatizzati, LTS, upgrade orchestrati e immagini firmate;
|
||||
- pacchetti air-gap, mirror degli artifact e gestione offline delle licenze;
|
||||
- secret manager e connettori certificati, con manutenzione/SLA;
|
||||
- promozione degli artifact fra ambienti e integrazioni ETL, CI/CD e cataloghi aziendali;
|
||||
- fleet/multi-installation management dentro il boundary del cliente;
|
||||
- onboarding, training, supporto con SLA e responsabilità contrattuale.
|
||||
|
||||
### Separazione tecnica iniziale
|
||||
|
||||
Se TYL Consulting vuole che il codice Enterprise rimanga proprietario, il modello più chiaro è:
|
||||
|
||||
1. repository pubblico Community con build e test che non dipendono da codice privato;
|
||||
2. contratti stabili per moduli, policy, identity provider, connettori e operator tooling;
|
||||
3. repository privato Enterprise che produce moduli o una distribuzione comprendente il core;
|
||||
4. una modalità senza licenza che conserva esattamente il comportamento Community;
|
||||
5. license file offline firmato, senza telemetria obbligatoria né chiamata periodica a TYL;
|
||||
6. test automatici del downgrade e della leggibilità/esportabilità degli artifact.
|
||||
|
||||
Il modello Metabase del codice commerciale visibile nello stesso repository è valido soltanto se si
|
||||
accetta che l'Enterprise sia source-available. Non corrisponde invece all'obiettivo di mantenere quel
|
||||
codice non pubblico. Il modello GitLab della directory dedicata è utile come disciplina di boundary,
|
||||
ma non obbliga ThothII ad adottarne struttura o complessità.
|
||||
|
||||
## Decisioni che questa ricerca non prende
|
||||
|
||||
- MIT contro Apache-2.0: entrambe sono licenze open source permissive e consentono uso commerciale;
|
||||
la scelta finale richiede revisione legale di brevetti, notice, dipendenze e contributi.
|
||||
- prezzo e metrica contrattuale definitivi;
|
||||
- lista iniziale dei connettori certificati;
|
||||
- scelta fra plugin Enterprise, distribuzione unica o immagini separate;
|
||||
- dettagli del meccanismo di licenza offline.
|
||||
|
||||
## Conclusione
|
||||
|
||||
Per ThothII l'open core sostenibile è un **core aperto completo nel risultato** e un'Enterprise
|
||||
completa nella **responsabilità organizzativa**. Il primo deve permettere a un IT department di
|
||||
provare e usare realmente il percorso domanda → SQL governato; la seconda deve rendere quel percorso
|
||||
adottabile in un'organizzazione regolata, multi-ruolo e mission-critical. Questo boundary sostiene
|
||||
insieme reputazione, conversione e fiducia senza dipendere da SaaS, consumo token o degradazione
|
||||
artificiale della qualità.
|
||||
@@ -17,7 +17,6 @@ The installation-level model catalog contained these entries, with `glm-53` as t
|
||||
| Selection ID | LiteLLM route | Authentication |
|
||||
| --- | --- | --- |
|
||||
| `deepseek-v4-pro` | `deepseek/deepseek-v4-pro` | Protected `DEEPSEEK_API_KEY` value already used by core |
|
||||
| `deepseek-v4-flash` | `deepseek/deepseek-v4-flash` | Protected `DEEPSEEK_API_KEY` value already used by core |
|
||||
| `glm-53` | `openai/glm-5.3` on the Z.AI coding endpoint | Protected `ZAI_API_KEY` value already used by core |
|
||||
| `qwen-36` | `openai/qwen3.6-35b-a3b` on the VPN-only AritmoLab endpoint | No operator API key |
|
||||
|
||||
|
||||
@@ -1,428 +0,0 @@
|
||||
# Piano di test: metadati di schema, campi sensibili e descrizioni AI
|
||||
|
||||
- Data: 2026-08-31
|
||||
- Stato: proposto
|
||||
- Baseline analizzata: `f586152` sul branch `test/database-baseline`
|
||||
|
||||
## 1. Obiettivo
|
||||
|
||||
Validare insieme le tre capacità recentemente introdotte in Database Management:
|
||||
|
||||
1. acquisizione autorevole dei metadati fisici di uno schema esterno;
|
||||
2. proposta e revisione umana dei campi sensibili dal punto di vista privacy;
|
||||
3. generazione, revisione e consolidamento delle descrizioni di tabelle e colonne.
|
||||
|
||||
Il piano è risk-based: perdita o corruzione di metadati, lettura o invio di valori protetti e
|
||||
applicazione di una selezione al target sbagliato sono rischi bloccanti. La qualità linguistica dei
|
||||
testi AI è invece valutata separatamente dal contratto tecnico, perché l'output del modello è una
|
||||
proposta soggetta a revisione umana.
|
||||
|
||||
Questo documento non costituisce una certificazione normativa o GDPR: verifica i controlli tecnici
|
||||
e il flusso operativo implementati dal prodotto.
|
||||
|
||||
### 1.1 Assunzione operativa: ambiente completamente sacrificabile
|
||||
|
||||
Per indicazione esplicita del proprietario, l'installazione sul Mac è esclusivamente di sviluppo e
|
||||
test. Non contiene produzione e non esiste alcun requisito di conservazione dello stato locale.
|
||||
|
||||
- `catalog-db`, migrazioni, configurazioni locali, run, eventi, flag, descrizioni curate e generate
|
||||
possono essere cancellati e ricreati tutte le volte necessarie;
|
||||
- il database reale già collegato è la fonte autorevole dalla quale ricostruire il catalogo ed è la
|
||||
sorgente primaria per validare schema, classificazione privacy e generazione delle descrizioni;
|
||||
- reset completi, failure injection, dati incoerenti deliberati e prove distruttive sul catalogo
|
||||
locale sono ammessi senza backup o procedura di rollback dell'ambiente;
|
||||
- compatibilità con stato locale preesistente, sessioni legacy e vecchie revisioni del catalogo non
|
||||
è un gate di questo piano;
|
||||
- le prove di atomicità, idempotenza e preservazione dei campi restano necessarie perché verificano
|
||||
il comportamento del prodotto dentro un ciclo di test, non perché debbano proteggere il Mac.
|
||||
|
||||
Il database sorgente resta normalmente read-only: quando una prova richiede DDL o mutazioni fra
|
||||
scan e conferma si usa uno schema di test esplicitamente scrivibile o la fixture supplementare. La
|
||||
disponibilità a buttare lo stato locale non elimina il rischio di inviare dati personali a un
|
||||
provider esterno: proprio questa non-esfiltrazione è uno degli esiti principali da collaudare.
|
||||
|
||||
## 2. Terminologia e risultato atteso
|
||||
|
||||
I tre campi testuali del catalogo hanno autorità diverse e non devono essere confusi:
|
||||
|
||||
| Campo | Origine e autorità | Comportamento atteso |
|
||||
| --- | --- | --- |
|
||||
| `sourceComment` | Commento fisico acquisito dalla sorgente | Si aggiorna con la sincronizzazione e non è modificabile come contenuto curato. |
|
||||
| `generatedDescription` | Proposta prodotta dall'AI o corretta nel catalogo | È salvata separatamente, può essere rigenerata esplicitamente e non sovrascrive gli altri due campi. |
|
||||
| `description` | Descrizione curata dall'amministratore | Cambia solo tramite modifica esplicita o consolidamento di una proposta selezionata. |
|
||||
|
||||
Nel seguito, “commento generato” indica `generatedDescription`. La generazione non deve mai
|
||||
modificare `sourceComment`; il passaggio a `description` avviene solo con il consolidamento umano.
|
||||
|
||||
## 3. Riferimenti e baseline
|
||||
|
||||
Il comportamento da verificare deriva da:
|
||||
|
||||
- stato corrente del progetto in `PROJECT_STATE.md`;
|
||||
- [contratto Catalog Schema Snapshot](../contracts/catalog-schema-snapshot.md);
|
||||
- [piano del Metadata Catalog](../plans/2026-08-26-metadata-catalog-from-thothai.md);
|
||||
- [specifica della generazione descrizioni](../plans/2026-08-28-ai-catalog-description-generation-spec.md);
|
||||
- [ADR 0007: sincronizzazione autorevole durevole](../adr/0007-durable-authoritative-schema-synchronization.md);
|
||||
- [ADR 0009: un solo run sequenziale di generazione](../adr/0009-use-one-sequential-description-generation-run.md);
|
||||
- [ADR 0010: campioni reali limitati](../adr/0010-allow-bounded-real-source-samples-for-description-generation.md);
|
||||
- [ADR 0011: Sensitive Data Flag](../adr/0011-gate-source-samples-with-a-sensitive-data-flag.md);
|
||||
- [accettazione AI del 2026-08-29](./2026-08-29-ai-catalog-description-generation-acceptance.md).
|
||||
|
||||
L'accettazione del 2026-08-29 è una baseline utile ma non chiude il gate attuale: precede i commit
|
||||
`0736983` e `cb40c09`, inviava campioni reali inventati e documentava che un valore campione era
|
||||
stato ripreso nella descrizione. Deve quindi essere ripetuta sul comportamento privacy corrente.
|
||||
|
||||
## 4. Perimetro
|
||||
|
||||
### Incluso
|
||||
|
||||
- trasporti `postgres_direct`, `rest_api` e `ssh_tunnel`;
|
||||
- RPC REST tipizzato `POST /rpc/schema_snapshot` e fallback singolo read-only su
|
||||
`POST /rpc/run_query`;
|
||||
- metadati di tabelle, colonne, commenti sorgente, tipi, default, nullabilità, posizioni PK e coppie
|
||||
FK ordinate;
|
||||
- scope di sincronizzazione `tables`, `columns`, `relationships` e `all`;
|
||||
- run durevoli, conferma delle differenze distruttive, cancellazione, recovery, eventi SSE e
|
||||
fallback polling;
|
||||
- Sensitive Data Flag, suggerimenti AI strutturali, review draft e storico operativo;
|
||||
- scope di generazione `selected_columns`, `selected_tables`, `all` e `missing`;
|
||||
- campionamento read-only, valori sintetici per colonne protette, batching, retry, stop, Unlock,
|
||||
storico e consolidamento;
|
||||
- uso controllato del database reale già collegato, inclusi schema e valori non sensibili, per il
|
||||
collaudo end-to-end con fake provider e provider configurati;
|
||||
- permessi, minimizzazione dei dati, redazione di log/errori e resistenza a input ostili.
|
||||
|
||||
### Escluso o rinviato
|
||||
|
||||
- applicazione del Sensitive Data Flag allo schema-linking/LSH del core, esplicitamente rinviata;
|
||||
- sostituzione di `annotations.yaml`, pubblicazione Qdrant e cutover del runtime NL→SQL;
|
||||
- alias, sinonimi, concetti, value descriptions e relazioni logiche;
|
||||
- audit delle decisioni umane sul flag: per disegno si conserva solo il booleano corrente;
|
||||
- DDL o scritture sul database sorgente;
|
||||
- conservazione di cataloghi, run, descrizioni o sessioni locali precedenti al reset;
|
||||
- compatibilità all'indietro con formati o migrazioni legacy non appartenenti alla baseline corrente.
|
||||
|
||||
## 5. Priorità e strategia
|
||||
|
||||
| Priorità | Significato | Esempi |
|
||||
| --- | --- | --- |
|
||||
| P0 | Gate bloccante di sicurezza o integrità | Nessuna lettura/invio di valori protetti; applicazione atomica; scope esatto; segreti non esposti. |
|
||||
| P1 | Contratto funzionale necessario al rilascio | Trasporti, stati dei run, recovery, batching, review e consolidamento. |
|
||||
| P2 | Qualità, UX e robustezza non distruttiva | Copy, focus, benchmark del modello, carico e degrado SSE. |
|
||||
|
||||
Le prove sono distribuite su quattro livelli:
|
||||
|
||||
1. **Contratto/unità**: repository in memoria, finti connector e Model Completer; nessuna rete.
|
||||
2. **Integrazione**: catalogo PostgreSQL ricreabile via Docker, database reale come sorgente,
|
||||
fixture supplementare, fake REST/SSH e Model Completer osservabile.
|
||||
3. **UI/E2E locale**: component test con MSW e almeno un flusso Playwright sullo stack locale.
|
||||
4. **Accettazione L2**: provider configurato realmente e database reale dopo review dei flag.
|
||||
|
||||
I test deterministici verificano il contratto. Le prove con un modello reale verificano
|
||||
compatibilità e qualità, ma non sostituiscono i gate P0.
|
||||
|
||||
## 6. Ambiente e dati di test
|
||||
|
||||
### 6.1 Ambiente minimo
|
||||
|
||||
- stack locale con `catalog-db` eliminabile, migrazioni ripetibili e backend/frontend della stessa
|
||||
baseline;
|
||||
- database reale già collegato come sorgente primaria, con utenza capace di `SELECT` ma non di
|
||||
`INSERT`, `UPDATE`, `DELETE`, DDL o cambio di schema;
|
||||
- PostgreSQL effimero supplementare solo per le mutazioni controllate che non devono essere fatte
|
||||
sul database reale;
|
||||
- server REST fake in grado di servire snapshot valido, capability `unavailable`, 404, risposta
|
||||
parziale/malformata e fallback `run_query`;
|
||||
- server OpenSSH effimero con `known_hosts` esatto e varianti host key errata/assente;
|
||||
- Model Completer fake che registra transitoriamente i messaggi e restituisce esiti programmabili;
|
||||
- browser senza segreti in Web Storage e raccolta di log backend/SSE/API per le scansioni canary.
|
||||
|
||||
Le prove PostgreSQL di integrazione non devono risultare `skip`: la disponibilità di Docker è una
|
||||
precondizione del gate. Il catalogo locale può essere azzerato prima di ogni wave senza snapshot o
|
||||
backup del suo stato precedente.
|
||||
|
||||
### 6.2 Database reale e fixture supplementare `catalog_qa`
|
||||
|
||||
La prima baseline viene acquisita dal database reale collegato. Il test registra soltanto inventario
|
||||
strutturale, conteggi e risultati sanitizzati; poi svuota il catalogo locale e dimostra di poterlo
|
||||
ricostruire dalla stessa sorgente. Non è richiesto preservare alcun metadato locale precedente.
|
||||
|
||||
La fixture `catalog_qa` integra il database reale soltanto quando servono casi controllabili o
|
||||
mutazioni che la sorgente reale non contiene. Deve includere almeno:
|
||||
|
||||
- `customers`: UUID PK, nome, email, codice fiscale, telefono, data di nascita, indirizzo e note;
|
||||
- `orders`: FK verso `customers`, importo numerico, stato, timestamp, default e campi nullable;
|
||||
- `order_lines`: PK composta e relazione composta ordinata;
|
||||
- `clinical_events`: campi sanitari evidenti e tabella partizionata;
|
||||
- `products`: SKU pubblico, categoria, prezzo e flag booleano;
|
||||
- `empty_table`: nessuna riga ma struttura valida;
|
||||
- `wide_entity`: almeno 23 colonne e metadati lunghi, per forzare batch `10 + 10 + 3` e il limite
|
||||
dimensionale del messaggio;
|
||||
- identificatori quotati, commenti Unicode/italiani, commenti null e oggetti fuori dallo schema.
|
||||
|
||||
Usare valori canary inventati e univoci, per esempio:
|
||||
|
||||
- `PRIV_EMAIL_CANARY_...`, `PRIV_TAX_CANARY_...`, `PRIV_HEALTH_CANARY_...` nelle colonne protette;
|
||||
- `PUBLIC_SKU_CANARY_...` in una colonna esplicitamente non sensibile;
|
||||
- `SECRET_API_CANARY_...` solo nel secret store del test.
|
||||
|
||||
I canary protetti non devono comparire nei messaggi al modello, nel catalogo, negli eventi, nelle
|
||||
API, nel DOM o nei log. Il canary pubblico può apparire nel messaggio al provider entro i limiti
|
||||
documentati e dopo la disclosure esplicita dell'utente. Sul database reale la stessa proprietà va
|
||||
provata soprattutto osservando la proiezione SQL e il payload transitorio: i valori protetti non
|
||||
devono essere letti, e nessun valore grezzo deve entrare nell'evidenza conservata.
|
||||
|
||||
### 6.3 Mutazioni della sorgente
|
||||
|
||||
Preparare tre revisioni dello schema:
|
||||
|
||||
- **A — iniziale**: struttura completa e commenti sorgente valorizzati;
|
||||
- **B — distruttiva**: rimozione di una tabella, una colonna e una FK, più aggiunta di una nuova
|
||||
colonna sensibile per nome;
|
||||
- **C — race di conferma**: modifica ulteriore fra piano distruttivo e conferma, per provare il
|
||||
re-scan.
|
||||
|
||||
Queste revisioni possono vivere nella fixture o in uno schema reale esplicitamente dichiarato
|
||||
scrivibile. Fra una prova e l'altra è consentito eliminare completamente il catalogo locale,
|
||||
riapplicare le migrazioni e ripartire dal database reale.
|
||||
|
||||
## 7. Casi di test — acquisizione dei metadati
|
||||
|
||||
| ID | P | Livello | Scenario | Risultato atteso |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| MET-01 | P0 | API/Integrazione | Avvio senza binding raggiungibile, con versione database obsoleta o senza `database.manage`. | Il run non parte; risposta sicura e catalogo invariato. I segreti restano write-only. |
|
||||
| MET-02 | P0 | Integrazione | Reset completo del catalogo e `all` sul database reale collegato, senza mock del client `pg_catalog`; ripetizione sulla fixture A per gli edge case assenti. | Il catalogo viene ricostruito da zero con snapshot esatta di tabelle, colonne, `sourceComment`, tipo, default, nullabilità, PK e FK ordinate; stato `succeeded`; nessuna scrittura alla sorgente. |
|
||||
| MET-03 | P1 | Contratto/Integrazione | Stessa fixture via RPC REST tipizzato. | `schemaVersion: 1` e capability sono validate strettamente; risultato normalizzato uguale a MET-02. `unavailable` non è interpretato come collezione vuota. |
|
||||
| MET-04 | P0 | Contratto/Integrazione | `/schema_snapshot` assente, fallback `run_query`; poi fallback assente, parziale, non JSON o con campi extra/mancanti. | Il fallback usa una sola query read-only. Ogni errore o snapshot invalida fallisce senza modifiche parziali; nessun fallback nasconde un errore operativo diverso da capability assente. |
|
||||
| MET-05 | P1 | Integrazione | Accesso `ssh_tunnel` con host key corretta, errata e assente; errore durante apertura/chiusura. | Parità con MET-02 nel caso valido; fail-closed negli altri casi; processi, lease e file-segreto sempre rilasciati. |
|
||||
| MET-06 | P0 | API | Esecuzione separata di `tables`, `columns`, `relationships` e `all`, con e senza selezione tabelle. | Ogni scope è autorevole solo nel proprio confine; nessun record fuori scope cambia o viene eliminato. Selezioni duplicate/inesistenti sono rifiutate atomicamente. |
|
||||
| MET-07 | P0 | API/Integrazione | Ripetizione idempotente della fixture A dopo modifica di `description`, `generatedDescription` e `sensitive`. | Nessun diff fisico spurio; contenuti curati, proposte AI e flag delle entità ancora presenti sono preservati. Una nuova colonna nasce con `sensitive=false`. |
|
||||
| MET-08 | P0 | API/Integrazione | Passaggio A→B, conferma assente/errata/scaduta e passaggio A→B→C prima della conferma. | Stato `awaiting_confirmation`, piano visibile e nessuna applicazione anticipata. La conferma valida provoca re-scan; se il diff cambia viene emesso un nuovo piano/token e il vecchio non applica nulla. Apply atomica oppure zero modifiche. |
|
||||
| MET-09 | P0 | API/Integrazione | Timeout, disconnessione, capability incompleta o eccezione durante scan/apply. | Stato terminale coerente, errore sanitizzato, catalogo precedente intatto e lock rilasciato. Nessun segreto, SQL sensibile o stack trace nelle API/eventi. |
|
||||
| MET-10 | P1 | Worker | Cancel in `queued`, `running`, `awaiting_confirmation` e `applying`; retry, restart con run attivo e lease scaduto. | Cancel è efficace solo prima di apply ed è rifiutato durante apply; retry crea un nuovo run. Recovery non duplica l'apply, marca correttamente i run interrotti, rilascia il lock e rimuove snapshot/diff/token interni non più necessari. |
|
||||
| MET-11 | P0 | API | Due operazioni sullo stesso database: sync, cleanup, connection test, edit o generazione; in parallelo, operazioni su database diversi. | Una sola operazione possiede il database; conflitto 409 sicuro sullo stesso target. Nessun lock cross-database non previsto e nessuna release del token altrui. |
|
||||
| MET-12 | P1 | UI | Avvio dai menu database/tabella, visualizzazione piano, conferma, history drawer, chiusura drawer, perdita SSE e polling. | Scope e selezione inviati sono esatti; azioni non eleggibili restano visibili con motivo; chiudere il drawer non ferma il run; replay/polling deduplicano gli eventi e aggiornano griglie/KPI. |
|
||||
| MET-13 | P1 | E2E | Cleanup manuale di tabelle/colonne/relazioni e successiva sincronizzazione. | Cleanup modifica solo il catalogo; la sorgente resta invariata; un sync autorevole ripristina gli oggetti ancora presenti in sorgente. |
|
||||
| MET-14 | P0 | Integrazione | Cambio binding/versione fra scan e apply ed errore iniettato a metà transazione. | La freshness viene ricontrollata sotto lock; il run fallisce senza righe parziali e conserva integralmente il catalogo precedente. |
|
||||
| MET-15 | P1 | API/Worker | Replay SSE con `Last-Event-ID`/`after`, polling concorrente e retention oltre 30 giorni. | Cursori monotoni e nessun duplicato; gli eventi scaduti vengono potati senza corrompere run e stato finale. |
|
||||
|
||||
## 8. Casi di test — identificazione e protezione dei campi sensibili
|
||||
|
||||
| ID | P | Livello | Scenario | Risultato atteso |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| PRV-01 | P0 | Repository/API | Prima sincronizzazione, re-sync e aggiunta di una colonna. | Il default è `false`; il valore umano delle colonne esistenti è preservato; la nuova colonna è esplicitamente da riesaminare ma non riceve uno stato audit inventato. |
|
||||
| PRV-02 | P0 | API | Suggerimento per un database, tabelle selezionate e colonne selezionate; database multipli, target duplicati o mancanti. | Il provider riceve esattamente le colonne dello scope. Input ambigui sono rifiutati prima della chiamata e nessun flag cambia. |
|
||||
| PRV-03 | P0 | Contratto | Ispezione del messaggio al classifier. | Sono presenti solo database, schema, tabella, colonna, tipo, nullabilità, PK e FK. Non compaiono righe, valori, commenti, descrizioni, flag corrente o segreti. |
|
||||
| PRV-04 | P1 | Contratto | `wide_entity`, identificatori lunghi e limite byte. | Ordine deterministico, batch massimi di dieci colonne e rispetto del limite messaggio; una singola colonna non rappresentabile fallisce prima del provider con errore sicuro. |
|
||||
| PRV-05 | P0 | API | Risposta valida, fenced/prosa, JSON malformato, target mancante/duplicato/ignoto e provider failure. | Ogni colonna richiesta compare una sola volta. Una classificazione invalida viene ritentata una volta; dopo esaurimento si ottiene errore sanitizzato e nessuna modifica. |
|
||||
| PRV-06 | P0 | UI/API | Apertura draft, modifica manuale, chiusura/reload e salvataggio. | La proposta non è persistita prima di Save; reload la scarta. Il reviewer può invertire scelte; si salvano solo colonne cambiate con versione ottimistica; un conflitto richiede reload. |
|
||||
| PRV-07 | P1 | Repository/UI | Tentativi completati, falliti e attivi al restart. | Ogni tentativo ha un run distinto con scope, modello, contatori ed eventi sanitizzati; startup marca `interrupted` i run attivi. Storico newest-first senza target ID, proposte, prompt, output grezzo o diagnostica provider. |
|
||||
| PRV-08 | P0 | Integrazione | Generazione descrizioni su target con canary protetti. | Le colonne protette sono assenti dalla proiezione SQL, non semplicemente filtrate dopo la lettura. Se non rimangono colonne leggibili non viene eseguita una `SELECT`. Nessun canary protetto esce dal processo. |
|
||||
| PRV-09 | P0 | Contratto/Integrazione | Tabella mista con colonne sensibili e pubbliche. | Per le sensibili il prompt contiene valori plausibili, deterministici e limitati derivati dai soli metadati, nello stesso formato dei campioni e senza etichettarli al modello come sintetici. Per le pubbliche: massimo cinque righe e cinque valori rappresentativi, valori troncati e transazione read-only chiusa con rollback. |
|
||||
| PRV-10 | P1 | API | Cambio `false→true→false` dopo una descrizione già generata. | Il testo esistente non viene rigenerato retroattivamente. Solo le generazioni future cambiano fonte del contesto; tornando `false` il campionamento reale torna eleggibile. |
|
||||
| PRV-11 | P0 | API/UI | Utente senza `database.manage`, modello non configurato, catalogo/provider indisponibile e richiesta interrotta. | Controlli nascosti/disabilitati in UI e rifiuto server-side; errori non espongono dettagli. Un tentativo fallito compare nello storico senza trasformarsi in audit della decisione umana. |
|
||||
| PRV-12 | P1 | L2 | Corpus strutturale etichettato con identificatori personali, credenziali/token, salute, finanza, localizzazione e controlli non sensibili/ambigui, in inglese e italiano. | Si misurano precisione, recall e falsi negativi per modello. I campi critici mancati sono sottoposti al product owner; la soglia quantitativa va ratificata prima di diventare gate, perché il classifier è advisory e human-in-the-loop. |
|
||||
| PRV-13 | P0 | UI/E2E | Modifica di un flag nella review senza Save e tentativo immediato di generare descrizioni. | Gate di rilascio da formalizzare: la generazione deve essere bloccata finché il draft non è salvato o scartato. In alternativa la UI deve dichiarare inequivocabilmente che verrà usato il valore persistito; non è accettabile mostrare “protetto” e campionare come non protetto. |
|
||||
|
||||
## 9. Casi di test — generazione e consolidamento dei commenti
|
||||
|
||||
| ID | P | Livello | Scenario | Risultato atteso |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| GEN-01 | P0 | Config/API | Modelli validi, default, endpoint anonimo esplicito, secret ref mancante/errato e nessun modello. | Al browser arrivano solo ID, label e default. Nessuna chiave, provider payload o configurazione privata è esposta; feature disabilitata in modo comprensibile se non configurata. |
|
||||
| GEN-02 | P0 | API | `selected_columns`, `selected_tables`, `all`, `missing`, target duplicati/inesistenti e zero eleggibili. | Scope esatto; `missing` include null/vuoto/whitespace e salta proposte esistenti; `all` sostituisce solo dopo conferma esplicita; input invalido non crea run. |
|
||||
| GEN-03 | P1 | Worker | 23 colonne più tabelle, metadati e campioni lunghi. | Colonne prima delle tabelle per lo scope globale; richieste omogenee, sequenziali, massimo dieci target e sotto i limiti byte; contesto tabella aggiornato dopo le colonne. |
|
||||
| GEN-04 | P0 | Contratto | JSON puro, un solo code fence JSON completo, prosa extra, payload multipli, target mancante/duplicato/ignoto, descrizione vuota o oltre limite. | Sono accettati solo i primi due formati validi. Una mappatura ambigua non applica alcun risultato del batch e conta come errore tecnico. |
|
||||
| GEN-05 | P1 | API | Esito `generated` e `non_generatable` in workspace italiano/inglese. | Testo generato trimmato e salvato; il testo standard non generabile è localizzato dall'applicazione, non copiato dal provider. Gli errori tecnici non scrivono tale testo. |
|
||||
| GEN-06 | P0 | Worker | Errore transiente, errore esaurito isolato, tre batch falliti consecutivi e successo fra due errori. | Helper con al massimo un retry e nessun fallback modello. Errori isolati portano a `completed_with_errors`; tre consecutivi a `failed`; un successo azzera il contatore. |
|
||||
| GEN-07 | P0 | Integrazione | Successi seguiti da cancel, interruption o failure. | Ogni risultato valido è persistito subito e resta disponibile; il target fallito non riceve dati ambigui. “Generate Missing” consente il recupero naturale senza resume automatico. |
|
||||
| GEN-08 | P0 | API | Secondo run durante un run attivo e modifica/sync/cleanup/consolidamento sul database posseduto. | Un solo run di generazione attivo nell'installazione; 409 chiaro al secondo Start. Il database target resta riservato e le operazioni incompatibili non alterano selezione o dati. |
|
||||
| GEN-09 | P0 | Integrazione/Worker/UI | Stop in queued/running con helper attivo e con una `SELECT` sorgente deliberatamente bloccata/lenta. | Helper e query/connessione vengono terminati entro 5 secondi, stato `cancelled`, nessuna chiamata modello dopo Stop, risultati precedenti conservati ed eventi consultabili. Un test con sampler fake non è sufficiente. |
|
||||
| GEN-10 | P0 | Worker/API | Restart con run `queued/running`; Unlock con worker/helper vivo e con run realmente stale; race Unlock/Start. | Startup marca `interrupted` senza replay. Unlock è rifiutato se esiste lavoro locale vivo e non può liberare la reservation di un nuovo run. |
|
||||
| GEN-11 | P1 | Integrazione | Sorgente campioni indisponibile ma metadati validi. | La generazione prosegue metadata-only con un solo warning sicuro; nessun tentativo alternativo espone dettagli di connessione. |
|
||||
| GEN-12 | P1 | API/UI | SSE disconnesso, replay da sequence, polling concorrente e riapertura storico. | Eventi persistiti, ordinati e deduplicati; history newest-first; contatori/stato finali coerenti; nessun prompt, campione, risposta completa o stack trace. |
|
||||
| GEN-13 | P0 | UI/API | Revisione manuale della proposta e consolidamento selettivo di tabelle/colonne, inclusi target vuoti/stale. | Si copia solo `generatedDescription` non vuota dei target risolti; conteggio `copied/skipped` corretto; operazione atomica; `sourceComment` invariato. |
|
||||
| GEN-14 | P0 | UI/Sicurezza | Tutti gli scope di generazione da database, tabelle e colonne selezionate, utente senza permesso, metadata contenente istruzioni ostili. | Prima di ogni Start, inclusi `selected_tables` e `selected_columns`, compare la disclosure “fino a cinque righe e cinque valori”. Il server applica comunque l'autorizzazione. Metadati e valori sono trattati come dati non fidati e l'output resta nel contratto JSON. |
|
||||
| GEN-15 | P1 | L2 | Run reale sul modello di default e almeno un endpoint alternativo supportato, usando il database reale dopo la review dei flag e un role read-only. | Descrizioni nella lingua workspace, coerenti con struttura/commenti e senza fatti inventati critici; almeno una colonna e una tabella consolidate. Nessun valore protetto o segreto compare in request osservabile, eventi, API, persistenza o log. |
|
||||
| GEN-16 | P1 | Integrazione | Fastify→worker→processo Python→LiteLLM→endpoint OpenAI-compatible locale di cattura. | Routing provider/model, header API key, `disableThinking`, singolo retry, timeout, limite output e payload sono corretti; chiave e diagnostica non risalgono a stdout, API o log applicativi. |
|
||||
|
||||
## 10. Percorso E2E prioritario
|
||||
|
||||
Il caso `E2E-01` deve attraversare le tre feature senza sostituire i test di contratto:
|
||||
|
||||
1. azzerare `catalog-db`, riapplicare le migrazioni e verificare che il role del database reale sia
|
||||
realmente read-only;
|
||||
2. eseguire `all` sul database reale e confrontare catalogo e snapshot sorgente; usare la fixture A
|
||||
in una seconda esecuzione per gli edge case mancanti;
|
||||
3. richiedere suggerimenti privacy sull'intero database;
|
||||
4. modificare almeno una proposta e salvare i flag revisionati;
|
||||
5. avviare `missing` dopo la disclosure, usando un Model Completer osservabile;
|
||||
6. dimostrare che i canary protetti non sono letti né inviati e che il canary pubblico rispetta i
|
||||
limiti;
|
||||
7. correggere una `generatedDescription` e consolidare una tabella e una colonna;
|
||||
8. applicare la fixture B, controllare il piano distruttivo e introdurre C prima della conferma;
|
||||
9. confermare dopo il re-scan e verificare atomicità, preservazione di descrizioni/flag delle entità
|
||||
superstiti e `sensitive=false` sulla nuova colonna;
|
||||
10. perdere la connessione SSE, riaprire entrambi gli storici e verificare replay, polling e KPI.
|
||||
|
||||
Il percorso deve essere eseguito con fake provider dopo ogni reset rilevante e, in forma ridotta,
|
||||
con il provider configurato sul database reale dopo che i flag sono stati revisionati e salvati.
|
||||
Non è richiesto ripristinare lo stato locale precedente al test.
|
||||
|
||||
## 11. Valutazione qualitativa dei modelli
|
||||
|
||||
La qualità non deve confondersi con la sicurezza: anche un classifier perfetto non autorizza
|
||||
l'esfiltrazione di un canary protetto e una descrizione elegante non rende valido un payload
|
||||
ambiguo.
|
||||
|
||||
### 11.1 Classificazione privacy
|
||||
|
||||
Per ogni modello registrare matrice di confusione, precisione, recall e falsi negativi per categoria.
|
||||
Eseguire almeno tre iterazioni sul corpus fisso per rilevare instabilità. Fino alla ratifica di una
|
||||
soglia da parte del product owner, il risultato è un gate di review: ogni falso negativo su
|
||||
credenziali/token, identificatori fiscali, dati sanitari o finanziari richiede accettazione esplicita
|
||||
o correzione prima del rilascio operativo.
|
||||
|
||||
### 11.2 Descrizioni generate
|
||||
|
||||
Valutare ogni testo da 0 a 2 su:
|
||||
|
||||
- correttezza rispetto a struttura e commenti sorgente;
|
||||
- specificità e utilità per un revisore;
|
||||
- lingua e chiarezza;
|
||||
- assenza di istruzioni seguite dai dati non fidati o fatti inventati;
|
||||
- assenza di valori protetti e segreti.
|
||||
|
||||
Soglia proposta da ratificare: almeno 8/10, nessun punteggio 0 su correttezza o sicurezza. Un valore
|
||||
esplicitamente non sensibile, reale o inventato, può essere ripreso entro il perimetro dichiarato;
|
||||
un valore protetto non può mai esserlo.
|
||||
|
||||
## 12. Copertura esistente e gap da chiudere
|
||||
|
||||
| Area | Evidenza automatica già presente | Gap principale |
|
||||
| --- | --- | --- |
|
||||
| Snapshot e sincronizzazione | `backend/test/catalog-schema-introspector.test.ts`, `catalog-schema-routes.test.ts`, `catalog-table-introspector.test.ts`, `catalog-repository.integration.test.ts` | Introspezione `pg_catalog` realmente end-to-end, parità live dei tre trasporti e un unico E2E con re-scan distruttivo. |
|
||||
| Privacy | `catalog-description-generation-routes.test.ts`, `catalog-description-generation-worker.test.ts`, `catalog-description-source-sampler.test.ts`, `catalog-synthetic-sample-value.test.ts` | Prova canary integrata query→prompt→API/log e benchmark reale post-ADR 0011. |
|
||||
| Generazione | `catalog-description-generation-routes.test.ts`, `catalog-description-generation-worker.test.ts`, `catalog-description-generation.integration.test.ts` e test del helper | Accettazione reale aggiornata, cancellazione di una query PostgreSQL bloccata e integrazione ermetica fino all'endpoint LiteLLM locale. |
|
||||
| UI | `DatabaseManagementPage.test.tsx`, `DescriptionGenerationDrawer.test.tsx`, `SensitiveDataSuggestionHistoryDrawer.test.tsx` | L'E2E Playwright corrente verifica soprattutto il layout, non il workflow funzionale. |
|
||||
|
||||
Nuovi asset consigliati:
|
||||
|
||||
- `backend/test/catalog-metadata-privacy-workflow.integration.test.ts`;
|
||||
- `backend/test/fixtures/catalog-privacy-schema.sql` e snapshot REST equivalenti;
|
||||
- integrazione a due PostgreSQL per le query reali `pg_catalog` e lo stop del sampler;
|
||||
- endpoint OpenAI-compatible locale di cattura per GEN-16;
|
||||
- `frontend/e2e/database-management-workflow.spec.ts`;
|
||||
- corpus strutturale versionato per PRV-12, senza valori business;
|
||||
- nuovo report di accettazione L2 che sostituisca il gate privacy del 2026-08-29.
|
||||
|
||||
## 13. Ordine di esecuzione
|
||||
|
||||
### Wave 1 — contratto rapido
|
||||
|
||||
- parser snapshot, introspector, scope e diff;
|
||||
- classifier strutturale, batching e validazione output;
|
||||
- sampler, valori sintetici, prompt bounds e parser descrizioni;
|
||||
- autorizzazione, redazione e race del coordinator.
|
||||
|
||||
### Wave 2 — integrazione PostgreSQL
|
||||
|
||||
- reset totale di `catalog-db`, bootstrap delle migrazioni e ricostruzione dal database reale;
|
||||
- migrazioni e vincoli del repository;
|
||||
- atomicità sincronizzazione/consolidamento;
|
||||
- run ed eventi persistiti, restart e cancellation;
|
||||
- `E2E-01` con fake provider e canary.
|
||||
|
||||
### Wave 3 — frontend e stack locale
|
||||
|
||||
- component test MSW;
|
||||
- Playwright funzionale, perdita SSE e polling;
|
||||
- verifica disclosure, review draft, history e consolidamento.
|
||||
|
||||
### Wave 4 — accettazione L2
|
||||
|
||||
- provider reale sul database collegato dopo classificazione e review dei flag;
|
||||
- benchmark PRV-12 e rubric GEN-15;
|
||||
- scansione finale di canary e segreti;
|
||||
- approvazione del product owner.
|
||||
|
||||
Comandi di regressione:
|
||||
|
||||
```bash
|
||||
cd backend && npx vitest run
|
||||
cd backend && npx tsc --noEmit -p .
|
||||
cd backend && npm run build
|
||||
cd frontend && npx vitest run
|
||||
cd frontend && npx tsc -b
|
||||
cd frontend && npm run build
|
||||
cd frontend && npm run e2e
|
||||
./scripts/build-docs.sh
|
||||
```
|
||||
|
||||
Il report deve evidenziare esplicitamente eventuali test Docker/L2 saltati; un `skip` non equivale a
|
||||
PASS del relativo gate.
|
||||
|
||||
## 14. Evidenze da conservare
|
||||
|
||||
Per ogni esecuzione registrare:
|
||||
|
||||
- commit, configurazione pubblica dei modelli, versione fixture e trasporto;
|
||||
- ID e stato finale dei run, contatori ed eventi sanitizzati;
|
||||
- snapshot catalogo prima/dopo e piano distruttivo con metadati e conteggi sanitizzati, senza valori
|
||||
grezzi delle righe sorgente;
|
||||
- report test/JUnit, screenshot dei gate UI e risultato della scansione canary;
|
||||
- matrice di confusione privacy e rubric delle descrizioni per le prove L2;
|
||||
- difetti con ID del caso, severità, riproducibilità e decisione finale.
|
||||
|
||||
Non allegare prompt completi, righe campione, output grezzi del provider, chiavi, digest o frammenti
|
||||
di segreti. Il Model Completer spy deve verificare in memoria le asserzioni e scartare il payload al
|
||||
termine del test.
|
||||
|
||||
Non serve conservare backup del catalogo locale, run precedenti o descrizioni generate durante una
|
||||
wave: l'evidenza è il report sanitizzato e la capacità di ricostruire nuovamente il risultato dalla
|
||||
sorgente reale.
|
||||
|
||||
## 15. Criteri di ingresso e uscita
|
||||
|
||||
### Ingresso
|
||||
|
||||
- baseline unica per backend/frontend e procedura verificata per eliminare e ricreare `catalog-db`;
|
||||
- accesso al database reale collegato e inventario delle tabelle/colonne da includere nella review;
|
||||
- fixture e canary supplementari approvati per i soli edge case controllati;
|
||||
- role sorgente read-only verificato con una scrittura deliberatamente negata;
|
||||
- fake connector/provider disponibili e log collection attiva;
|
||||
- per L2, secret reference configurato senza materializzare il valore nell'evidenza.
|
||||
|
||||
### Uscita
|
||||
|
||||
- 100% dei casi P0 e P1 applicabili superati; nessun difetto Sev-1/Sev-2 aperto;
|
||||
- almeno due ricostruzioni complete e coerenti del catalogo a partire dal database reale dopo reset
|
||||
indipendenti;
|
||||
- zero comparsa dei canary protetti e dei segreti fuori dalla sorgente/secret store del test;
|
||||
- nessuna modifica parziale dopo errori, cancel o conferme stale;
|
||||
- parità normalizzata dei trasporti supportati e nessuna integration PostgreSQL richiesta saltata;
|
||||
- stati, contatori, eventi e history coerenti dopo success, partial failure, stop e restart;
|
||||
- `sourceComment`, `generatedDescription` e `description` mantengono l'autorità prevista;
|
||||
- accettazione L2 sul database collegato approvata dal product owner e
|
||||
build/test/typecheck/documentazione verdi;
|
||||
- ogni deviazione P2 o soglia qualitativa non ancora ratificata è documentata con owner e data.
|
||||
|
||||
## 16. Rischi residui da rendere espliciti
|
||||
|
||||
- `sensitive=false` è il default e il prodotto non conserva uno stato “review completata”: dopo ogni
|
||||
reset, classificazione strutturale e salvataggio umano dei flag devono precedere qualunque run con
|
||||
provider reale. Il catalogo è ricostruibile; un invio errato a un provider non lo è.
|
||||
- Il flag protegge i valori campionati. Nomi, `sourceComment` e descrizioni sono metadati inviabili
|
||||
al modello e possono contenere testo libero: se nel database reale includono PII serve una
|
||||
decisione aggiuntiva di redazione, non una diversa aspettativa di test.
|
||||
- La UI deve risolvere il caso di un flag modificato ma non salvato prima della generazione e deve
|
||||
mostrare la disclosure anche per tabelle/colonne selezionate; il piano considera entrambi P0.
|
||||
- L'AbortSignal corrente va provato contro una query PostgreSQL realmente bloccata: la sola
|
||||
cancellazione del helper non dimostra che la lettura sorgente sia interrompibile.
|
||||
- Lo storico dei Sensitive Data Suggestion Run è operativo, non un audit delle decisioni umane.
|
||||
- La policy privacy non è ancora applicata allo schema-linking/LSH; nessun risultato di questo piano
|
||||
deve essere presentato come copertura di quel percorso.
|
||||
- Un provider reale resta non deterministico: il rilascio deve dipendere dai gate tecnici e dalla
|
||||
review umana, non dalla ripetizione byte-identica delle descrizioni.
|
||||
- L'esclusione fra operazioni e Unlock è in parte locale al processo. Se il deployment ammetterà più
|
||||
repliche backend, servirà un gate aggiuntivo con due istanze contro lo stesso catalogo; non va
|
||||
dedotta sicurezza multi-replica dai test single-process.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,4 +1,4 @@
|
||||
import { expect, test, type Locator, type Page, type Route } from "@playwright/test";
|
||||
import { expect, test, type Page, type Route } from "@playwright/test";
|
||||
import { createAuthenticationStack } from "./fixtures/auth-stack.mjs";
|
||||
|
||||
test.describe.configure({ mode: "serial" });
|
||||
@@ -12,13 +12,6 @@ const database = {
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "ready",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
@@ -44,22 +37,6 @@ const database = {
|
||||
},
|
||||
};
|
||||
|
||||
const unconfiguredDatabase = {
|
||||
...database,
|
||||
id: undefined,
|
||||
workspaceId: "research-lab",
|
||||
workspaceName: "Research laboratory",
|
||||
configured: false,
|
||||
databaseName: "research",
|
||||
schema: "analytics",
|
||||
version: 0,
|
||||
createdAt: "",
|
||||
updatedAt: "",
|
||||
binding: { transport: "postgres_direct", port: 5432 },
|
||||
connectionStatus: "untested",
|
||||
testedVersion: undefined,
|
||||
};
|
||||
|
||||
const table = {
|
||||
id: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
|
||||
databaseId,
|
||||
@@ -72,28 +49,6 @@ const table = {
|
||||
updatedAt: "2026-08-27T09:00:00Z",
|
||||
};
|
||||
|
||||
const descriptionRun = {
|
||||
id: "77777777-7777-4777-8777-777777777777",
|
||||
databaseId,
|
||||
scope: "selected_columns",
|
||||
modelId: "glm-53",
|
||||
language: "it",
|
||||
status: "completed",
|
||||
total: 5,
|
||||
processed: 5,
|
||||
generated: 4,
|
||||
nonGeneratable: 1,
|
||||
failed: 0,
|
||||
inputTokens: 1200,
|
||||
cacheReadTokens: 80,
|
||||
outputTokens: 420,
|
||||
createdAt: "2026-08-31T09:00:00Z",
|
||||
startedAt: "2026-08-31T09:00:01Z",
|
||||
updatedAt: "2026-08-31T09:00:04Z",
|
||||
finishedAt: "2026-08-31T09:00:04Z",
|
||||
errorSummary: null,
|
||||
};
|
||||
|
||||
function metrics(requestedDatabaseId: string | null) {
|
||||
return {
|
||||
scope: requestedDatabaseId ? "database" : "global",
|
||||
@@ -155,8 +110,7 @@ async function expectContextPanelGeometry(page: Page, accessibleName: string) {
|
||||
const managerCenter = managerBox.x + managerBox.width / 2;
|
||||
const panelCenter = panelBox.x + panelBox.width / 2;
|
||||
expect(Math.abs(panelCenter - managerCenter)).toBeLessThanOrEqual(1);
|
||||
const expectedPanelWidth = Math.min(1200, managerBox.width - (managerBox.width <= 768 ? 24 : 32));
|
||||
expect(Math.abs(panelBox.width - expectedPanelWidth)).toBeLessThanOrEqual(1);
|
||||
expect(Math.abs(panelBox.width / managerBox.width - 0.6)).toBeLessThanOrEqual(0.005);
|
||||
expect(panelBox.x).toBeGreaterThanOrEqual(managerBox.x - 1);
|
||||
expect(panelBox.x + panelBox.width).toBeLessThanOrEqual(managerBox.x + managerBox.width + 1);
|
||||
expect(panelBox.x + panelBox.width).toBeLessThanOrEqual(sessionRailBox.x + 1);
|
||||
@@ -164,112 +118,7 @@ async function expectContextPanelGeometry(page: Page, accessibleName: string) {
|
||||
expect(panelBox.y + panelBox.height).toBeLessThanOrEqual(managerBox.y + managerBox.height + 1);
|
||||
}
|
||||
|
||||
async function expectCurrentNavigation(page: Page, accessibleName: string) {
|
||||
const navigation = page.getByRole("complementary", { name: "Session navigation" });
|
||||
const button = navigation.getByRole("button", { name: accessibleName, exact: true });
|
||||
const available = navigation.locator('[data-navigation-state="available"]').first();
|
||||
await expect(navigation.locator('[data-navigation-state="current"]')).toHaveCount(1);
|
||||
await expect(button).toHaveAttribute("data-navigation-state", "current");
|
||||
await expect(button).toHaveAttribute("aria-current", "page");
|
||||
expect(await button.getAttribute("class")).toContain("bg-[oklch(var(--nav-active))]");
|
||||
|
||||
const [style, availableStyle] = await Promise.all([
|
||||
button.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderTopColor,
|
||||
})),
|
||||
available.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderTopColor,
|
||||
})),
|
||||
]);
|
||||
expect(style.backgroundColor).not.toBe(availableStyle.backgroundColor);
|
||||
expect(style.borderColor).not.toBe(availableStyle.borderColor);
|
||||
|
||||
const tokens = await page.evaluate(() => {
|
||||
const root = getComputedStyle(document.documentElement);
|
||||
const read = (name: string) => root.getPropertyValue(name).trim().split(/\s+/).map(Number);
|
||||
return { primary: read("--primary"), active: read("--nav-active") };
|
||||
});
|
||||
expect(tokens.active[2]).toBeCloseTo(tokens.primary[2], 1);
|
||||
expect(tokens.active[1]).toBeLessThan(tokens.primary[1]);
|
||||
expect(tokens.active[0]).toBeGreaterThan(tokens.primary[0]);
|
||||
}
|
||||
|
||||
async function expectSelectedSessionScopeTab(page: Page, accessibleName: string) {
|
||||
const tablist = page.getByRole("tablist", { name: "Session scope" });
|
||||
const selected = tablist.getByRole("tab", { name: accessibleName, exact: true });
|
||||
const inactive = tablist.locator('[role="tab"][aria-selected="false"]');
|
||||
await expect(tablist.locator('[role="tab"][aria-selected="true"]')).toHaveCount(1);
|
||||
await expect(selected).toHaveAttribute("data-tab-state", "active");
|
||||
expect(await selected.getAttribute("class")).toContain("bg-[oklch(var(--nav-active))]");
|
||||
|
||||
const [selectedStyle, inactiveStyle] = await Promise.all([
|
||||
selected.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderBottomColor,
|
||||
})),
|
||||
inactive.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderBottomColor,
|
||||
borderWidth: getComputedStyle(element).borderTopWidth,
|
||||
})),
|
||||
]);
|
||||
expect(selectedStyle.backgroundColor).not.toBe(inactiveStyle.backgroundColor);
|
||||
expect(selectedStyle.borderColor).not.toBe(inactiveStyle.borderColor);
|
||||
expect(inactiveStyle.borderWidth).toBe("1px");
|
||||
expect(inactiveStyle.backgroundColor).not.toMatch(/^(?:transparent|rgba\(0, 0, 0, 0\))$/);
|
||||
expect(inactiveStyle.borderColor).not.toMatch(/^(?:transparent|rgba\(0, 0, 0, 0\))$/);
|
||||
}
|
||||
|
||||
function cssLightness(color: string): number {
|
||||
const oklab = color.match(/^okl(?:ab|ch)\(([\d.]+)(%)?/);
|
||||
if (oklab) {
|
||||
const value = Number(oklab[1]);
|
||||
return oklab[2] ? value / 100 : value;
|
||||
}
|
||||
const channels = color.match(/[\d.]+/g)?.slice(0, 3).map(Number) ?? [];
|
||||
if (channels.length !== 3) return Number.NaN;
|
||||
return color.startsWith("color(srgb") ? Math.max(...channels) : Math.max(...channels) / 255;
|
||||
}
|
||||
|
||||
async function expectFleetRowTooltipBelow(trigger: Locator, expectedContent: string) {
|
||||
await trigger.hover();
|
||||
await expect.poll(() => trigger.evaluate((element) => {
|
||||
const owner = element.closest<HTMLElement>("[data-tooltip]");
|
||||
return owner ? getComputedStyle(owner, "::after").visibility : "missing";
|
||||
})).toBe("visible");
|
||||
await expect.poll(() => trigger.evaluate((element) => {
|
||||
const owner = element.closest<HTMLElement>("[data-tooltip]");
|
||||
return owner ? getComputedStyle(owner, "::after").opacity : "missing";
|
||||
})).toBe("1");
|
||||
|
||||
const state = await trigger.evaluate((element) => {
|
||||
const owner = element.closest<HTMLElement>("[data-tooltip]");
|
||||
if (!owner) throw new Error("Row action tooltip owner is missing");
|
||||
const style = getComputedStyle(owner, "::after");
|
||||
return {
|
||||
content: style.content.replace(/^['\"]|['\"]$/g, ""),
|
||||
ownerHeight: owner.getBoundingClientRect().height,
|
||||
top: Number.parseFloat(style.top),
|
||||
right: Number.parseFloat(style.right),
|
||||
opacity: style.opacity,
|
||||
pointerEvents: style.pointerEvents,
|
||||
backgroundColor: style.backgroundColor,
|
||||
color: style.color,
|
||||
};
|
||||
});
|
||||
|
||||
expect(state.content).toBe(expectedContent);
|
||||
expect(state.top - state.ownerHeight).toBeCloseTo(3, 1);
|
||||
expect(state.right).toBe(0);
|
||||
expect(state.opacity).toBe("1");
|
||||
expect(state.pointerEvents).toBe("none");
|
||||
expect(cssLightness(state.backgroundColor)).toBeLessThan(0.35);
|
||||
expect(cssLightness(state.color)).toBeGreaterThan(0.9);
|
||||
}
|
||||
|
||||
async function expectContextPanelHeaderUsesPrimary(
|
||||
async function expectContextPanelHeaderMatchesNewSession(
|
||||
page: Page,
|
||||
accessibleName: string,
|
||||
) {
|
||||
@@ -278,22 +127,26 @@ async function expectContextPanelHeaderUsesPrimary(
|
||||
.getByRole("main", { name: "Database management" });
|
||||
const panel = manager.getByRole("dialog", { name: accessibleName });
|
||||
const header = panel.locator(':scope > [data-catalog-panel-region="header"]');
|
||||
await expect(header).toHaveCount(1);
|
||||
const newSession = page
|
||||
.getByRole("complementary", { name: "Session navigation" })
|
||||
.getByRole("button", { name: "New session", exact: true });
|
||||
|
||||
const currentNavigation = page.locator('[data-navigation-state="current"]');
|
||||
const [headerStyle, navigationStyle] = await Promise.all([
|
||||
await expect(header).toHaveCount(1);
|
||||
await expect(newSession).toBeVisible();
|
||||
|
||||
const [headerStyle, actionStyle] = await Promise.all([
|
||||
header.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
backgroundImage: getComputedStyle(element).backgroundImage,
|
||||
})),
|
||||
currentNavigation.evaluate((element) => ({
|
||||
newSession.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
})),
|
||||
]);
|
||||
|
||||
expect(headerStyle.backgroundImage).toBe("none");
|
||||
expect(headerStyle.backgroundColor).not.toBe("rgba(0, 0, 0, 0)");
|
||||
expect(headerStyle.backgroundColor).not.toBe(navigationStyle.backgroundColor);
|
||||
expect(headerStyle.backgroundColor).toBe(actionStyle.backgroundColor);
|
||||
}
|
||||
|
||||
async function expectWorkAreaPanelGeometry(page: Page, accessibleName: string) {
|
||||
@@ -320,8 +173,7 @@ async function expectWorkAreaPanelGeometry(page: Page, accessibleName: string) {
|
||||
expect(Math.abs(
|
||||
panelBox.x + panelBox.width / 2 - (workAreaBox.x + workAreaBox.width / 2),
|
||||
)).toBeLessThanOrEqual(1);
|
||||
const expectedPanelWidth = Math.min(1200, workAreaBox.width - (workAreaBox.width <= 768 ? 24 : 32));
|
||||
expect(Math.abs(panelBox.width - expectedPanelWidth)).toBeLessThanOrEqual(1);
|
||||
expect(Math.abs(panelBox.width / workAreaBox.width - 0.6)).toBeLessThanOrEqual(0.005);
|
||||
expect(panelBox.x).toBeGreaterThanOrEqual(workAreaBox.x - 1);
|
||||
expect(panelBox.x + panelBox.width).toBeLessThanOrEqual(workAreaBox.x + workAreaBox.width + 1);
|
||||
expect(panelBox.x + panelBox.width).toBeLessThanOrEqual(sessionRailBox.x + 1);
|
||||
@@ -329,7 +181,7 @@ async function expectWorkAreaPanelGeometry(page: Page, accessibleName: string) {
|
||||
expect(panelBox.y + panelBox.height).toBeLessThanOrEqual(workAreaBox.y + workAreaBox.height + 1);
|
||||
}
|
||||
|
||||
async function expectWorkAreaPanelHeaderUsesPrimary(
|
||||
async function expectWorkAreaPanelHeaderMatchesNewSession(
|
||||
page: Page,
|
||||
accessibleName: string,
|
||||
) {
|
||||
@@ -337,19 +189,21 @@ async function expectWorkAreaPanelHeaderUsesPrimary(
|
||||
.getByTestId("conversation-column")
|
||||
.getByRole("dialog", { name: accessibleName });
|
||||
const header = panel.locator(':scope > [data-work-area-panel-region="header"]');
|
||||
const currentNavigation = page.locator('[data-navigation-state="current"]');
|
||||
const [headerStyle, navigationStyle] = await Promise.all([
|
||||
const newSession = page
|
||||
.getByRole("complementary", { name: "Session navigation" })
|
||||
.getByRole("button", { name: "New session", exact: true });
|
||||
const [headerStyle, actionStyle] = await Promise.all([
|
||||
header.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
backgroundImage: getComputedStyle(element).backgroundImage,
|
||||
})),
|
||||
currentNavigation.evaluate((element) => ({
|
||||
newSession.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
})),
|
||||
]);
|
||||
|
||||
expect(headerStyle.backgroundImage).toBe("none");
|
||||
expect(headerStyle.backgroundColor).not.toBe(navigationStyle.backgroundColor);
|
||||
expect(headerStyle.backgroundColor).toBe(actionStyle.backgroundColor);
|
||||
}
|
||||
|
||||
test.beforeAll(async () => {
|
||||
@@ -364,24 +218,12 @@ test.afterAll(async () => {
|
||||
test("context panels stay inside the manager and the Tables grid sits in a sidebar-colored frame", async ({ page }) => {
|
||||
const unexpectedCatalogRequests: string[] = [];
|
||||
const responses: Record<string, unknown> = {
|
||||
"GET /api/catalog/databases": [database, unconfiguredDatabase],
|
||||
"GET /api/catalog/databases": [database],
|
||||
"GET /api/catalog/metadata-generation/models": {
|
||||
models: [
|
||||
{ id: "deepseek-v4-pro", label: "DeepSeek V4 Pro" },
|
||||
{ id: "deepseek-v4-flash", label: "DeepSeek V4 Flash" },
|
||||
{ id: "glm-53", label: "GLM 5.3" },
|
||||
{ id: "qwen-36", label: "AritmoLab Qwen 3.6 35B A3B" },
|
||||
],
|
||||
default: "glm-53",
|
||||
models: [],
|
||||
default: null,
|
||||
},
|
||||
"GET /api/catalog/description-generation-runs?limit=50": [descriptionRun],
|
||||
[`GET /api/catalog/description-generation-runs/${descriptionRun.id}`]: descriptionRun,
|
||||
[`GET /api/catalog/description-generation-runs/${descriptionRun.id}/events-list?after=0`]: [{
|
||||
sequence: 1,
|
||||
level: "info",
|
||||
message: "Description generation completed.",
|
||||
createdAt: "2026-08-31T09:00:04Z",
|
||||
}],
|
||||
"GET /api/catalog/description-generation-runs?limit=50": [],
|
||||
"GET /api/catalog/sensitive-data-suggestion-runs?limit=50": [],
|
||||
[`GET /api/catalog/databases/${databaseId}/tables`]: [table],
|
||||
};
|
||||
@@ -410,41 +252,10 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
|
||||
await page.goto(stack.publicUrl);
|
||||
await signInAsAdmin(page);
|
||||
await expectCurrentNavigation(page, "New session");
|
||||
await expectSelectedSessionScopeTab(page, "My sessions");
|
||||
await page.getByRole("tab", { name: "All sessions", exact: true }).click();
|
||||
await expectSelectedSessionScopeTab(page, "All sessions");
|
||||
await page.getByRole("tab", { name: "My sessions", exact: true }).click();
|
||||
await expectSelectedSessionScopeTab(page, "My sessions");
|
||||
|
||||
const adminNavigationRail = page.getByRole("complementary", { name: "Session navigation" });
|
||||
const administration = adminNavigationRail.getByRole("button", { name: "Administration", exact: true });
|
||||
await expect(administration).toHaveCSS("font-family", /Manrope/);
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "false");
|
||||
await expect(adminNavigationRail.getByRole("region", { name: "Administration" })).toHaveCount(0);
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "true");
|
||||
const administrationPanel = adminNavigationRail.getByRole("region", { name: "Administration" });
|
||||
await expect(administrationPanel).toBeVisible();
|
||||
expect(await administrationPanel.locator(":scope > *").evaluateAll((elements) => elements.map((element) => (
|
||||
element.getAttribute("role") === "separator" ? "separator" : element.textContent?.trim()
|
||||
)))).toEqual([
|
||||
"Database management",
|
||||
"separator",
|
||||
"Workspace management",
|
||||
"Pi management",
|
||||
]);
|
||||
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "false");
|
||||
await expect(adminNavigationRail.getByRole("button", { name: "Database management", exact: true })).toHaveCount(0);
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "true");
|
||||
|
||||
await adminNavigationRail
|
||||
await page
|
||||
.getByRole("button", { name: "Database management", exact: true })
|
||||
.click();
|
||||
await expectCurrentNavigation(page, "Database management");
|
||||
|
||||
await expect(
|
||||
page.getByRole("complementary", { name: "Session navigation" }),
|
||||
@@ -457,42 +268,27 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Back to workspace", exact: true }),
|
||||
).toHaveCount(0);
|
||||
await expect(page.getByRole("button", { name: /Add database/i })).toHaveCount(0);
|
||||
await expect(page.getByRole("columnheader", { name: /Revision \/ Evidence/ })).toBeVisible();
|
||||
await expect(page.getByRole("columnheader", { name: /NL→SQL runtime/ })).toBeVisible();
|
||||
await expect(page.getByRole("columnheader", { name: /Metadata Catalog/ })).toBeVisible();
|
||||
const metadataModelSelector = page.getByRole("combobox", {
|
||||
name: "Metadata-generation LLM model",
|
||||
});
|
||||
await expect(metadataModelSelector).toHaveCount(1);
|
||||
await expect(metadataModelSelector).toHaveValue("glm-53");
|
||||
await expect(metadataModelSelector.locator("option")).toHaveText([
|
||||
"DeepSeek V4 Pro",
|
||||
"DeepSeek V4 Flash",
|
||||
"GLM 5.3",
|
||||
"AritmoLab Qwen 3.6 35B A3B",
|
||||
]);
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Configure catalog for Research laboratory", exact: true })
|
||||
.click();
|
||||
const configurePanel = page.getByRole("dialog", { name: "Configure catalog" });
|
||||
await expectContextPanelGeometry(page, "Configure catalog");
|
||||
const configuredWorkspace = configurePanel.getByLabel("Workspace", { exact: true });
|
||||
await expect(configuredWorkspace).toHaveValue("Research laboratory");
|
||||
await expect(configuredWorkspace).toHaveAttribute("readonly", "");
|
||||
await configurePanel.getByRole("button", { name: "Close database panel" }).click();
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Edit Policlinico San Donato", exact: true })
|
||||
.click();
|
||||
await expectContextPanelGeometry(page, "Edit database");
|
||||
await expectContextPanelHeaderUsesPrimary(page, "Edit database");
|
||||
await expectContextPanelHeaderMatchesNewSession(page, "Edit database");
|
||||
await page
|
||||
.getByRole("dialog", { name: "Edit database" })
|
||||
.getByRole("button", { name: "Close database panel" })
|
||||
.click();
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Description history", exact: true })
|
||||
.click();
|
||||
await expectContextPanelGeometry(page, "Description generation");
|
||||
await expectContextPanelHeaderMatchesNewSession(page, "Description generation");
|
||||
await page
|
||||
.getByRole("dialog", { name: "Description generation" })
|
||||
.getByRole("button", { name: /close/i })
|
||||
.click();
|
||||
|
||||
await page.setViewportSize({ width: 1280, height: 800 });
|
||||
await page
|
||||
.getByRole("button", { name: "Edit Policlinico San Donato", exact: true })
|
||||
@@ -504,12 +300,12 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
.click();
|
||||
await page.setViewportSize({ width: 1910, height: 911 });
|
||||
|
||||
const databaseTablesAction = page.getByRole("button", {
|
||||
name: "View tables for Policlinico San Donato",
|
||||
exact: true,
|
||||
});
|
||||
await expectFleetRowTooltipBelow(databaseTablesAction, "Tables");
|
||||
await databaseTablesAction.click();
|
||||
await page
|
||||
.getByRole("button", {
|
||||
name: "View tables for Policlinico San Donato",
|
||||
exact: true,
|
||||
})
|
||||
.click();
|
||||
|
||||
await expect(
|
||||
page.getByRole("region", {
|
||||
@@ -517,61 +313,6 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
}),
|
||||
).toBeVisible();
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Description history", exact: true })
|
||||
.click();
|
||||
await expectContextPanelGeometry(page, "Description generation");
|
||||
await expectContextPanelHeaderUsesPrimary(page, "Description generation");
|
||||
const descriptionPanel = page.getByRole("dialog", { name: "Description generation" });
|
||||
const descriptionBody = descriptionPanel.locator(
|
||||
':scope > [data-catalog-panel-region="body"]',
|
||||
);
|
||||
const progressTable = descriptionPanel.getByRole("table", {
|
||||
name: "Description generation progress",
|
||||
});
|
||||
const eventLog = descriptionPanel.getByRole("log", {
|
||||
name: "Description generation events",
|
||||
});
|
||||
const recentRuns = descriptionPanel.getByRole("region", {
|
||||
name: "Description generation history",
|
||||
});
|
||||
await expect(progressTable.getByRole("columnheader")).toHaveCount(5);
|
||||
await expect(eventLog).toBeVisible();
|
||||
await expect(recentRuns).toBeVisible();
|
||||
|
||||
const [bodyBox, eventLogBox, recentRunsBox] = await Promise.all([
|
||||
descriptionBody.boundingBox(),
|
||||
eventLog.boundingBox(),
|
||||
recentRuns.boundingBox(),
|
||||
]);
|
||||
expect(bodyBox).not.toBeNull();
|
||||
expect(eventLogBox).not.toBeNull();
|
||||
expect(recentRunsBox).not.toBeNull();
|
||||
if (bodyBox && eventLogBox && recentRunsBox) {
|
||||
const eventBottomGutter = bodyBox.y + bodyBox.height - eventLogBox.y - eventLogBox.height;
|
||||
const recentBottomGutter = bodyBox.y + bodyBox.height - recentRunsBox.y - recentRunsBox.height;
|
||||
expect(eventBottomGutter).toBeGreaterThanOrEqual(10);
|
||||
expect(recentBottomGutter).toBeGreaterThanOrEqual(10);
|
||||
expect(eventLogBox.x + eventLogBox.width).toBeLessThan(recentRunsBox.x);
|
||||
}
|
||||
|
||||
await page.setViewportSize({ width: 720, height: 800 });
|
||||
const [compactManagerBox, compactDescriptionPanelBox] = await Promise.all([
|
||||
page.getByTestId("conversation-column").getByRole("main", {
|
||||
name: "Database management",
|
||||
}).boundingBox(),
|
||||
descriptionPanel.boundingBox(),
|
||||
]);
|
||||
expect(compactManagerBox).not.toBeNull();
|
||||
expect(compactDescriptionPanelBox).not.toBeNull();
|
||||
if (compactManagerBox && compactDescriptionPanelBox) {
|
||||
expect(compactManagerBox.width - compactDescriptionPanelBox.width).toBeGreaterThanOrEqual(20);
|
||||
}
|
||||
expect(await descriptionBody.evaluate((element) => getComputedStyle(element).display)).not.toBe("grid");
|
||||
|
||||
await page.setViewportSize({ width: 1910, height: 911 });
|
||||
await descriptionPanel.getByRole("button", { name: /close/i }).click();
|
||||
|
||||
const applicationBar = page.locator(
|
||||
'main[aria-label="Database management"] .thot-fleet-ledger__header',
|
||||
);
|
||||
@@ -585,10 +326,6 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
await expect(
|
||||
applicationBar.getByRole("group", { name: "Database management actions" }),
|
||||
).toBeVisible();
|
||||
await expect(
|
||||
applicationBar.getByRole("combobox", { name: "Metadata-generation LLM model" }),
|
||||
).toHaveValue("glm-53");
|
||||
await expect(metadataModelSelector).toHaveCount(1);
|
||||
await expect(applicationBar).not.toContainText("Thoth catalog · Fleet ledger");
|
||||
await expect(applicationBar).not.toContainText("Inspect physical metadata");
|
||||
|
||||
@@ -618,14 +355,11 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
|
||||
await expect(page.getByRole("row", { name: /patients/ })).toBeVisible();
|
||||
|
||||
const editPatientsAction = page.getByRole("button", {
|
||||
name: "Edit description for patients",
|
||||
exact: true,
|
||||
});
|
||||
await expectFleetRowTooltipBelow(editPatientsAction, "Edit metadata");
|
||||
await editPatientsAction.click();
|
||||
await page
|
||||
.getByRole("button", { name: "Edit description for patients", exact: true })
|
||||
.click();
|
||||
await expectContextPanelGeometry(page, "Review table description");
|
||||
await expectContextPanelHeaderUsesPrimary(page, "Review table description");
|
||||
await expectContextPanelHeaderMatchesNewSession(page, "Review table description");
|
||||
const tableEditor = page.getByRole("dialog", { name: "Review table description" });
|
||||
const tableEditorFooter = tableEditor.locator(
|
||||
':scope > [data-catalog-panel-region="footer"]',
|
||||
@@ -766,10 +500,6 @@ test("Workspace and Pi management share the centered work-area panel without cov
|
||||
|
||||
await page.goto(stack.publicUrl);
|
||||
await signInAsAdmin(page);
|
||||
const administration = page.getByRole("button", { name: "Administration", exact: true });
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "false");
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "true");
|
||||
|
||||
for (const viewport of [
|
||||
{ width: 1910, height: 911 },
|
||||
@@ -781,29 +511,25 @@ test("Workspace and Pi management share the centered work-area panel without cov
|
||||
exact: true,
|
||||
});
|
||||
await workspaceTrigger.click();
|
||||
await expectCurrentNavigation(page, "Workspace management");
|
||||
await expectWorkAreaPanelGeometry(page, "Workspace management");
|
||||
await expectWorkAreaPanelHeaderUsesPrimary(page, "Workspace management");
|
||||
await expectWorkAreaPanelHeaderMatchesNewSession(page, "Workspace management");
|
||||
|
||||
const piTrigger = page.getByRole("button", { name: "Pi management", exact: true });
|
||||
await piTrigger.click();
|
||||
await expectCurrentNavigation(page, "Pi management");
|
||||
await expect(
|
||||
page.getByRole("dialog", { name: "Workspace management" }),
|
||||
).toHaveCount(0);
|
||||
await expectWorkAreaPanelGeometry(page, "Pi management");
|
||||
await expectWorkAreaPanelHeaderUsesPrimary(page, "Pi management");
|
||||
await expectWorkAreaPanelHeaderMatchesNewSession(page, "Pi management");
|
||||
await page
|
||||
.getByRole("dialog", { name: "Pi management" })
|
||||
.getByRole("button", { name: "Close Pi management" })
|
||||
.click();
|
||||
await expect(piTrigger).toBeFocused();
|
||||
await expectCurrentNavigation(page, "New session");
|
||||
}
|
||||
|
||||
await page.setViewportSize({ width: 720, height: 800 });
|
||||
await page.getByRole("button", { name: "Workspace management", exact: true }).click();
|
||||
await expectCurrentNavigation(page, "Workspace management");
|
||||
const compactWorkArea = page.getByTestId("conversation-column");
|
||||
const compactPanel = compactWorkArea.getByRole("dialog", { name: "Workspace management" });
|
||||
const [compactWorkAreaBox, compactPanelBox] = await Promise.all([
|
||||
@@ -813,7 +539,7 @@ test("Workspace and Pi management share the centered work-area panel without cov
|
||||
expect(compactWorkAreaBox).not.toBeNull();
|
||||
expect(compactPanelBox).not.toBeNull();
|
||||
if (compactWorkAreaBox && compactPanelBox) {
|
||||
expect(Math.abs(compactPanelBox.width - compactWorkAreaBox.width + 24)).toBeLessThanOrEqual(1);
|
||||
expect(Math.abs(compactPanelBox.width - compactWorkAreaBox.width + 16)).toBeLessThanOrEqual(1);
|
||||
expect(Math.abs(
|
||||
compactPanelBox.x + compactPanelBox.width / 2
|
||||
- (compactWorkAreaBox.x + compactWorkAreaBox.width / 2),
|
||||
|
||||
@@ -269,6 +269,7 @@ export function FleetSelectionActions({ props, direction = "row" }: { props: Var
|
||||
if (state.inventoryLevel === "databases") actions = [
|
||||
{ id: "databases-test", group: "Connection", label: "Test connections", icon: <TestTube2 size={14} />, disabled, onClick: run(`Testing ${count} selected connection${count === 1 ? "" : "s"}`) },
|
||||
{ id: "databases-sync-tables", group: "Synchronization", label: "Synchronize tables", icon: <RefreshCw size={14} />, disabled, onClick: run("Table synchronization started", "running") },
|
||||
{ id: "databases-sync-columns", group: "Synchronization", label: "Synchronize all columns", icon: <Columns3 size={14} />, disabled, onClick: run("Column synchronization started", "running") },
|
||||
{ id: "databases-sync-relationships", group: "Synchronization", label: "Synchronize relationships", icon: <Activity size={14} />, disabled, onClick: run("Relationship synchronization started", "running") },
|
||||
{ id: "databases-sync-all", group: "Synchronization", label: "Synchronize all", icon: <RefreshCw size={14} />, disabled, onClick: run("Full schema synchronization started", "running") },
|
||||
{ id: "databases-generate-all", group: "Descriptions", label: "Generate all descriptions", icon: <Sparkles size={14} />, disabled, onClick: run("Generate all confirmation opened") },
|
||||
@@ -278,16 +279,16 @@ export function FleetSelectionActions({ props, direction = "row" }: { props: Var
|
||||
{ id: "databases-delete-relationships", group: "Cleanup", label: "Delete all relationships", icon: <Trash2 size={14} />, executeLabel: "Review deletion", destructive: true, disabled, onClick: run("Delete relationships confirmation opened", "changes") },
|
||||
];
|
||||
if (state.inventoryLevel === "tables") actions = [
|
||||
{ id: "tables-generate", group: "Descriptions", label: "Generate table descriptions", icon: <Sparkles size={14} />, disabled, onClick: run(`Description generation started for ${count} table${count === 1 ? "" : "s"}`) },
|
||||
{ id: "tables-generate", group: "Descriptions", label: "Generate descriptions", icon: <Sparkles size={14} />, disabled, onClick: run(`Description generation started for ${count} table${count === 1 ? "" : "s"}`) },
|
||||
{ id: "tables-sync-columns", group: "Synchronization", label: "Synchronize columns", icon: <RefreshCw size={14} />, disabled, onClick: run("Selected table column synchronization started", "running") },
|
||||
{ id: "tables-consolidate", group: "Descriptions", label: "Copy generated descriptions to Description", icon: <Save size={14} />, disabled, onClick: run("Generated descriptions copied to curated Description") },
|
||||
{ id: "tables-consolidate", group: "Descriptions", label: "Move generated to Description", icon: <Save size={14} />, disabled, onClick: run("Generated descriptions moved to curated Description") },
|
||||
{ id: "tables-suggest-sensitive", group: "Sensitive data", label: "Suggest sensitive fields", icon: <ShieldCheck size={14} />, executeLabel: "Open review", disabled, onClick: () => onOpenPanel("sensitive-review") },
|
||||
{ id: "tables-delete-columns", group: "Cleanup", label: "Delete all columns", icon: <Trash2 size={14} />, executeLabel: "Review deletion", destructive: true, disabled, onClick: run("Delete table columns confirmation opened", "changes") },
|
||||
{ id: "tables-delete-relationships", group: "Cleanup", label: "Delete all relationships", icon: <Trash2 size={14} />, executeLabel: "Review deletion", destructive: true, disabled, onClick: run("Delete table relationships confirmation opened", "changes") },
|
||||
];
|
||||
if (state.inventoryLevel === "columns") actions = [
|
||||
{ id: "columns-generate", group: "Descriptions", label: "Generate column descriptions", icon: <Sparkles size={14} />, disabled, onClick: run(`Description generation started for ${count} column${count === 1 ? "" : "s"}`) },
|
||||
{ id: "columns-consolidate", group: "Descriptions", label: "Copy generated descriptions to Description", icon: <Save size={14} />, disabled, onClick: run("Generated column descriptions copied") },
|
||||
{ id: "columns-generate", group: "Descriptions", label: "Generate descriptions", icon: <Sparkles size={14} />, disabled, onClick: run(`Description generation started for ${count} column${count === 1 ? "" : "s"}`) },
|
||||
{ id: "columns-consolidate", group: "Descriptions", label: "Move generated to Description", icon: <Save size={14} />, disabled, onClick: run("Generated column descriptions consolidated") },
|
||||
{ id: "columns-suggest-sensitive", group: "Sensitive data", label: "Suggest sensitive fields", icon: <ShieldCheck size={14} />, executeLabel: "Open review", disabled, onClick: () => onOpenPanel("sensitive-review") },
|
||||
{ id: "columns-save-sensitive", group: "Sensitive data", label: `Save sensitive fields (${Object.keys(state.sensitiveDrafts).length})`, icon: <LockKeyhole size={14} />, executeLabel: "Save changes", disabled: Object.keys(state.sensitiveDrafts).length === 0, onClick: onSaveSensitive },
|
||||
];
|
||||
|
||||
@@ -1,15 +1,7 @@
|
||||
import { http, HttpResponse } from "msw";
|
||||
import { expect, test } from "vitest";
|
||||
import { server } from "../test/msw";
|
||||
import {
|
||||
createCatalogRelationship,
|
||||
deleteCatalogRelationship,
|
||||
getCatalogMetrics,
|
||||
rebuildGeneratedRelationships,
|
||||
setCatalogRelationshipStatus,
|
||||
type CatalogMetrics,
|
||||
type CatalogRelationship,
|
||||
} from "./catalog-databases";
|
||||
import { getCatalogMetrics, type CatalogMetrics } from "./catalog-databases";
|
||||
|
||||
const databaseId = "d4baf0f5-b9c3-4dc5-aad2-2f5676a36e64";
|
||||
|
||||
@@ -66,99 +58,3 @@ test("propagates an unsuccessful catalog metrics response", async () => {
|
||||
|
||||
await expect(getCatalogMetrics()).rejects.toMatchObject({ status: 503 });
|
||||
});
|
||||
|
||||
const logicalRelationship: CatalogRelationship = {
|
||||
id: "11111111-1111-4111-8111-111111111111",
|
||||
databaseId,
|
||||
constraintName: null,
|
||||
sourceTableId: "22222222-2222-4222-8222-222222222222",
|
||||
sourceTableName: "orders",
|
||||
targetTableId: "33333333-3333-4333-8333-333333333333",
|
||||
targetTableName: "users",
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId: "44444444-4444-4444-8444-444444444444",
|
||||
sourceColumnName: "user_id",
|
||||
targetColumnId: "55555555-5555-4555-8555-555555555555",
|
||||
targetColumnName: "id",
|
||||
}],
|
||||
origin: "manual",
|
||||
status: "active",
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
|
||||
test("creates a manual relationship from the selected source and target columns", async () => {
|
||||
let requestBody: unknown;
|
||||
server.use(http.post("/api/catalog/databases/:databaseId/relationships", async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json(logicalRelationship, { status: 201 });
|
||||
}));
|
||||
|
||||
await expect(createCatalogRelationship(
|
||||
databaseId,
|
||||
logicalRelationship.columns[0].sourceColumnId,
|
||||
logicalRelationship.columns[0].targetColumnId,
|
||||
)).resolves.toEqual(logicalRelationship);
|
||||
expect(requestBody).toEqual({
|
||||
sourceColumnId: logicalRelationship.columns[0].sourceColumnId,
|
||||
targetColumnId: logicalRelationship.columns[0].targetColumnId,
|
||||
});
|
||||
});
|
||||
|
||||
test("rebuilds generated relationships for one database", async () => {
|
||||
let calls = 0;
|
||||
server.use(http.post(
|
||||
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
|
||||
() => {
|
||||
calls += 1;
|
||||
return HttpResponse.json({ added: 12, alreadyPresent: 7, excluded: 3, ambiguous: 2 });
|
||||
},
|
||||
));
|
||||
|
||||
await expect(rebuildGeneratedRelationships(databaseId)).resolves.toEqual({
|
||||
added: 12,
|
||||
alreadyPresent: 7,
|
||||
excluded: 3,
|
||||
ambiguous: 2,
|
||||
});
|
||||
expect(calls).toBe(1);
|
||||
});
|
||||
|
||||
test("changes whether a logical relationship is active or excluded", async () => {
|
||||
let requestBody: unknown;
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json({ ...logicalRelationship, status: "excluded" });
|
||||
},
|
||||
));
|
||||
|
||||
await expect(setCatalogRelationshipStatus(
|
||||
databaseId,
|
||||
logicalRelationship.id,
|
||||
"excluded",
|
||||
)).resolves.toMatchObject({ status: "excluded" });
|
||||
expect(requestBody).toEqual({ status: "excluded" });
|
||||
});
|
||||
|
||||
test("permanently deletes a logical relationship", async () => {
|
||||
let calls = 0;
|
||||
server.use(http.delete(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
() => {
|
||||
calls += 1;
|
||||
return new HttpResponse(null, { status: 204 });
|
||||
},
|
||||
));
|
||||
|
||||
await expect(deleteCatalogRelationship(databaseId, logicalRelationship.id)).resolves.toBeUndefined();
|
||||
expect(calls).toBe(1);
|
||||
});
|
||||
|
||||
@@ -3,12 +3,6 @@ import { joinBackendPath } from "./runtime-config";
|
||||
|
||||
export type DatabaseTransport = "postgres_direct" | "rest_api" | "ssh_tunnel";
|
||||
export type ConnectionStatus = "untested" | "reachable" | "failed";
|
||||
export type WorkspaceEvidenceState =
|
||||
| "not_declared"
|
||||
| "materialized_current_revision"
|
||||
| "configuration_required"
|
||||
| "configured_unverified"
|
||||
| "workspace_unavailable";
|
||||
export type CatalogSecretName =
|
||||
| "password"
|
||||
| "apiKey"
|
||||
@@ -39,16 +33,6 @@ export interface CatalogDatabase {
|
||||
workspaceName: string;
|
||||
workspaceDescription?: string;
|
||||
workspaceAvailable: boolean;
|
||||
workspaceRevision: { commit: string; blob: string } | null;
|
||||
workspaceEvidence: {
|
||||
sourceType: "filesystem" | "http" | "s3" | null;
|
||||
state: WorkspaceEvidenceState;
|
||||
};
|
||||
runtimeBinding: {
|
||||
transport: DatabaseTransport;
|
||||
configurationState: "ready" | "configuration_required";
|
||||
sessionTransportSupported: boolean;
|
||||
} | null;
|
||||
configured: boolean;
|
||||
engine: "postgres";
|
||||
databaseName: string;
|
||||
@@ -157,9 +141,6 @@ export interface SensitiveDataSuggestionRun {
|
||||
total: number;
|
||||
suggestedSensitive: number;
|
||||
suggestedNonSensitive: number;
|
||||
inputTokens?: number;
|
||||
cacheReadTokens?: number;
|
||||
outputTokens?: number;
|
||||
createdAt: string;
|
||||
startedAt: string | null;
|
||||
updatedAt: string;
|
||||
@@ -183,37 +164,25 @@ export interface CatalogRelationshipColumn {
|
||||
targetColumnName: string;
|
||||
}
|
||||
|
||||
export type CatalogRelationshipOrigin = "physical" | "generated" | "manual";
|
||||
export type CatalogRelationshipStatus = "active" | "excluded";
|
||||
|
||||
export interface CatalogRelationship {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
constraintName: string | null;
|
||||
constraintName: string;
|
||||
sourceTableId: string;
|
||||
sourceTableName: string;
|
||||
targetTableId: string;
|
||||
targetTableName: string;
|
||||
updateRule: string | null;
|
||||
deleteRule: string | null;
|
||||
updateRule: string;
|
||||
deleteRule: string;
|
||||
deferrable: boolean;
|
||||
initiallyDeferred: boolean;
|
||||
columns: CatalogRelationshipColumn[];
|
||||
origin: CatalogRelationshipOrigin;
|
||||
status: CatalogRelationshipStatus;
|
||||
lastSyncedDatabaseVersion: number | null;
|
||||
lastSyncedAt: string | null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
export interface CatalogRelationshipRebuildResult {
|
||||
added: number;
|
||||
alreadyPresent: number;
|
||||
excluded: number;
|
||||
ambiguous: number;
|
||||
}
|
||||
|
||||
export type CatalogDatabaseMetadataDeleteTarget = "tables" | "relationships";
|
||||
export type CatalogTableMetadataDeleteTarget = "columns" | "relationships";
|
||||
|
||||
@@ -300,9 +269,6 @@ export interface DescriptionGenerationRun {
|
||||
generated: number;
|
||||
nonGeneratable: number;
|
||||
failed: number;
|
||||
inputTokens?: number;
|
||||
cacheReadTokens?: number;
|
||||
outputTokens?: number;
|
||||
createdAt: string;
|
||||
startedAt: string | null;
|
||||
updatedAt: string;
|
||||
@@ -486,36 +452,6 @@ export const listSensitiveDataSuggestionEvents = (runId: string, after = 0) =>
|
||||
export const listCatalogRelationships = (databaseId: string) =>
|
||||
apiFetch<CatalogRelationship[]>(`/catalog/databases/${encodeURIComponent(databaseId)}/relationships`);
|
||||
|
||||
export const createCatalogRelationship = (
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
) => apiFetch<CatalogRelationship>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships`,
|
||||
{ method: "POST", body: JSON.stringify({ sourceColumnId, targetColumnId }) },
|
||||
);
|
||||
|
||||
export const rebuildGeneratedRelationships = (databaseId: string) =>
|
||||
apiFetch<CatalogRelationshipRebuildResult>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/rebuild-generated`,
|
||||
{ method: "POST" },
|
||||
);
|
||||
|
||||
export const setCatalogRelationshipStatus = (
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogRelationshipStatus,
|
||||
) => apiFetch<CatalogRelationship>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/${encodeURIComponent(relationshipId)}`,
|
||||
{ method: "PATCH", body: JSON.stringify({ status }) },
|
||||
);
|
||||
|
||||
export const deleteCatalogRelationship = (databaseId: string, relationshipId: string) =>
|
||||
apiFetch<void>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/${encodeURIComponent(relationshipId)}`,
|
||||
{ method: "DELETE" },
|
||||
);
|
||||
|
||||
export const deleteCatalogDatabaseMetadata = (
|
||||
databaseIds: string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
|
||||
@@ -152,15 +152,6 @@ test.each([
|
||||
["sensitive_data_suggestion_history_request_invalid", "Sensitive suggestion history parameters are invalid."],
|
||||
["sensitive_data_suggestion_history_failed", "Sensitive suggestion history could not be loaded."],
|
||||
["sensitive_data_suggestion_run_not_found", "The sensitive suggestion run was not found."],
|
||||
["relationship_not_found", "The relationship no longer exists. Refresh and try again."],
|
||||
["relationship_duplicate", "This relationship already exists."],
|
||||
["relationship_target_not_unique", "The target column must be the only primary-key column of its table."],
|
||||
["relationship_type_incompatible", "Source and target column types are not compatible."],
|
||||
["relationship_read_only", "Physical relationships are read-only."],
|
||||
["relationship_request_invalid", "The relationship request is invalid."],
|
||||
["relationship_operation_failed", "The relationship operation failed."],
|
||||
["relationship_schema_stale", "Synchronize the current database schema before managing logical relationships."],
|
||||
["column_not_found", "The selected catalog column was not found. Refresh and try again."],
|
||||
])("maps the catalog error code %s to safe local copy", async (code, message) => {
|
||||
const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue(
|
||||
new Response(JSON.stringify({ code, message: "provider detail must not be trusted" }), {
|
||||
|
||||
@@ -35,9 +35,6 @@ const safeErrorCodes = new Set([
|
||||
"sensitive_data_suggestion_run_not_found",
|
||||
"schema_sync_conflict", "schema_introspection_failed", "schema_request_invalid",
|
||||
"schema_operation_failed", "sync_run_not_found", "table_stale", "column_stale",
|
||||
"relationship_not_found", "relationship_duplicate", "relationship_target_not_unique",
|
||||
"relationship_type_incompatible", "relationship_read_only", "relationship_request_invalid",
|
||||
"relationship_operation_failed", "relationship_schema_stale", "column_not_found",
|
||||
]);
|
||||
|
||||
type SafeErrorPayload = {
|
||||
@@ -107,15 +104,6 @@ const localCodeMessages: Record<string, string> = {
|
||||
sync_run_not_found: "The synchronization run was not found.",
|
||||
table_stale: "Table metadata changed. Reload and try again.",
|
||||
column_stale: "Column metadata changed. Reload and try again.",
|
||||
relationship_not_found: "The relationship no longer exists. Refresh and try again.",
|
||||
relationship_duplicate: "This relationship already exists.",
|
||||
relationship_target_not_unique: "The target column must be the only primary-key column of its table.",
|
||||
relationship_type_incompatible: "Source and target column types are not compatible.",
|
||||
relationship_read_only: "Physical relationships are read-only.",
|
||||
relationship_request_invalid: "The relationship request is invalid.",
|
||||
relationship_operation_failed: "The relationship operation failed.",
|
||||
relationship_schema_stale: "Synchronize the current database schema before managing logical relationships.",
|
||||
column_not_found: "The selected catalog column was not found. Refresh and try again.",
|
||||
};
|
||||
|
||||
const localStatusMessages: Record<number, string> = {
|
||||
|
||||
@@ -78,7 +78,7 @@ export function LoginPage({ config, onAuthenticated, onRetry }: LoginPageProps)
|
||||
<div className="mx-auto grid min-h-[calc(100vh-4rem)] max-w-5xl items-center gap-12 lg:grid-cols-[minmax(0,1fr)_26rem]">
|
||||
<section className="hidden max-w-xl lg:block">
|
||||
<h1 className="max-w-lg font-heading text-5xl font-semibold leading-[1.03] tracking-tight sm:text-6xl">
|
||||
From intent to SQL, with a human in the loop
|
||||
From intent to SQL, with Human In The Loop
|
||||
</h1>
|
||||
<p className="mt-6 max-w-md text-base leading-7 text-muted-foreground">AI generates, you guide and approve.</p>
|
||||
</section>
|
||||
|
||||
@@ -4,7 +4,7 @@ import { cva, type VariantProps } from "class-variance-authority"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
const buttonVariants = cva(
|
||||
"group/button inline-flex shrink-0 items-center justify-center rounded-md border border-transparent bg-clip-padding font-sans text-sm font-semibold tracking-[0.005em] whitespace-nowrap transition-all outline-none select-none focus-visible:ring-3 focus-visible:ring-ring/25 disabled:pointer-events-none disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
|
||||
"group/button inline-flex shrink-0 items-center justify-center rounded-md border border-transparent bg-clip-padding text-sm font-semibold tracking-[0.005em] whitespace-nowrap transition-all outline-none select-none focus-visible:ring-3 focus-visible:ring-ring/25 disabled:pointer-events-none disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
|
||||
{
|
||||
variants: {
|
||||
variant: {
|
||||
@@ -13,8 +13,6 @@ const buttonVariants = cva(
|
||||
"border-border bg-card shadow-xs hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:border-input dark:bg-input/30 dark:hover:bg-input/50",
|
||||
secondary:
|
||||
"bg-secondary text-secondary-foreground hover:bg-[color-mix(in_oklch,oklch(var(--secondary)),oklch(var(--foreground))_5%)] aria-expanded:bg-secondary aria-expanded:text-secondary-foreground",
|
||||
navigationActive:
|
||||
"border-[oklch(var(--nav-active-border))] bg-[oklch(var(--nav-active))] text-[oklch(var(--nav-active-foreground))] shadow-xs hover:bg-[oklch(var(--nav-active-hover))] hover:text-[oklch(var(--nav-active-foreground))] focus-visible:border-[oklch(var(--nav-active-border))] focus-visible:ring-[oklch(var(--nav-active-border)/0.32)]",
|
||||
ghost:
|
||||
"hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:hover:bg-muted/50",
|
||||
destructive:
|
||||
|
||||
@@ -47,12 +47,6 @@
|
||||
--success: 0.7577 0.1581 165.0;
|
||||
--warning: 0.8523 0.1386 78.9;
|
||||
--info: 0.7035 0.1128 221.3;
|
||||
/* Current navigation shares Instrument Red's hue, with lower chroma and
|
||||
higher lightness so selection reads as a muted location signal. */
|
||||
--nav-active: 0.9250 0.0520 23.2;
|
||||
--nav-active-hover: 0.8950 0.0710 23.2;
|
||||
--nav-active-foreground: 0.3650 0.1100 23.2;
|
||||
--nav-active-border: 0.6000 0.1350 23.2;
|
||||
--radius: 0.5rem;
|
||||
--sidebar: 0.9709 0.0011 17.2;
|
||||
--sidebar-foreground: 0.2678 0.0097 355.6;
|
||||
@@ -87,10 +81,6 @@
|
||||
--border: 0.3715 0 90;
|
||||
--input: 0.3715 0 90;
|
||||
--ring: 0.6897 0.1779 16.9;
|
||||
--nav-active: 0.3350 0.0550 20.3;
|
||||
--nav-active-hover: 0.3750 0.0650 20.3;
|
||||
--nav-active-foreground: 0.8900 0.0700 20.3;
|
||||
--nav-active-border: 0.6800 0.1350 20.3;
|
||||
--sidebar: 0.2350 0 90;
|
||||
--sidebar-foreground: 0.9310 0 90;
|
||||
--sidebar-primary: 0.6023 0.1848 20.3;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { act, render, screen, waitFor, within } from "@testing-library/react";
|
||||
import { act, render, screen, waitFor } from "@testing-library/react";
|
||||
import userEvent from "@testing-library/user-event";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { http, HttpResponse } from "msw";
|
||||
@@ -45,51 +45,24 @@ beforeEach(() => {
|
||||
});
|
||||
|
||||
describe("authenticated shell permissions", () => {
|
||||
test("hides administrative navigation from a session user", async () => {
|
||||
test("shows only read-safe workspace chrome to a session user", async () => {
|
||||
renderShell({ subject: "user-1", isAdmin: false, roles: ["user"], permissions: ["session.use"] });
|
||||
|
||||
expect(await screen.findByRole("button", { name: "New session" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
|
||||
expect(await screen.findByRole("button", { name: "Workspace management" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("shows the ordered administrative accordion only to an admin", async () => {
|
||||
const user = userEvent.setup();
|
||||
test("shows management and all-session chrome only for exact permissions", async () => {
|
||||
renderShell({
|
||||
subject: "admin-1",
|
||||
isAdmin: true,
|
||||
roles: ["admin"],
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"],
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "pi.manage"],
|
||||
});
|
||||
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const trigger = await within(sessionNavigation).findByRole("button", { name: "Administration" });
|
||||
expect(trigger).toHaveClass("font-sans");
|
||||
expect(trigger).toHaveAttribute("aria-expanded", "false");
|
||||
expect(within(sessionNavigation).queryByRole("region", { name: "Administration" })).not.toBeInTheDocument();
|
||||
|
||||
trigger.focus();
|
||||
await user.keyboard("{Enter}");
|
||||
expect(trigger).toHaveAttribute("aria-expanded", "true");
|
||||
|
||||
const panel = within(sessionNavigation).getByRole("region", { name: "Administration" });
|
||||
const database = within(panel).getByRole("button", { name: "Database management" });
|
||||
const separator = within(panel).getByRole("separator");
|
||||
const workspace = within(panel).getByRole("button", { name: "Workspace management" });
|
||||
const pi = within(panel).getByRole("button", { name: "Pi management" });
|
||||
expect(panel.children[0]).toBe(database);
|
||||
expect(panel.children[1]).toBe(separator);
|
||||
expect(panel.children[2]).toBe(workspace);
|
||||
expect(panel.children[3]).toBe(pi);
|
||||
expect(screen.getByRole("tab", { name: "All sessions" })).toBeInTheDocument();
|
||||
|
||||
await user.keyboard(" ");
|
||||
expect(trigger).toHaveAttribute("aria-expanded", "false");
|
||||
expect(within(sessionNavigation).queryByRole("region", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(within(sessionNavigation).queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
|
||||
expect(await screen.findByRole("button", { name: "Pi management" })).toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("keeps identity but hides logout outside local authentication", async () => {
|
||||
@@ -205,14 +178,15 @@ describe("authenticated shell permissions", () => {
|
||||
subject: "admin-1", isAdmin: true, roles: ["admin"],
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "pi.manage"],
|
||||
});
|
||||
expect(await screen.findByRole("button", { name: "Administration" })).toBeInTheDocument();
|
||||
expect(screen.getByRole("tab", { name: "All sessions" })).toBeInTheDocument();
|
||||
expect(await screen.findByRole("button", { name: "Pi management" })).toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toBeInTheDocument();
|
||||
|
||||
act(() => clearAuthState());
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
await userEvent.click(screen.getByRole("button", { name: "Workspace management" }));
|
||||
expect(await screen.findByRole("heading", { name: "Workspace management" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Update workspace repository" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("remounts the real shell so user-A panel and transcript state cannot survive user-B", async () => {
|
||||
|
||||
@@ -17,10 +17,6 @@ function renderShell() {
|
||||
);
|
||||
}
|
||||
|
||||
async function expandAdministration() {
|
||||
await userEvent.click(screen.getByRole("button", { name: "Administration" }));
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
clearAuthState();
|
||||
setAuthState({
|
||||
@@ -96,19 +92,10 @@ test("uses Fleet Ledger by default and gates the legacy fallback to non-producti
|
||||
|
||||
test("replaces the core conversation while keeping the session navigation", async () => {
|
||||
renderShell();
|
||||
await expandAdministration();
|
||||
|
||||
const conversationColumn = screen.getByTestId("conversation-column");
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
|
||||
const databaseManagement = within(sessionNavigation).getByRole("button", { name: "Database management" });
|
||||
|
||||
expect(newSession).toHaveAttribute("aria-current", "page");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(newSession).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(databaseManagement).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(databaseManagement).not.toHaveAttribute("aria-current");
|
||||
|
||||
await userEvent.click(databaseManagement);
|
||||
|
||||
const manager = screen.getByRole("main", { name: "Database management" });
|
||||
@@ -116,13 +103,8 @@ test("replaces the core conversation while keeping the session navigation", asyn
|
||||
expect(conversationColumn).toContainElement(manager);
|
||||
expect(screen.getByRole("complementary", { name: "Session navigation" })).toBe(sessionNavigation);
|
||||
expect(sessionNavigation).toBeVisible();
|
||||
expect(newSession).toBeVisible();
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(newSession).not.toHaveAttribute("aria-current");
|
||||
expect(newSession).not.toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(within(sessionNavigation).getByRole("button", { name: "New session" })).toBeVisible();
|
||||
expect(databaseManagement).toHaveAttribute("aria-current", "page");
|
||||
expect(databaseManagement).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(databaseManagement).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(screen.queryByRole("button", { name: "Back to workspace" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("textbox", { name: /new question/i })).not.toBeInTheDocument();
|
||||
});
|
||||
@@ -153,7 +135,6 @@ test("keeps a live core session connected while returning from database manageme
|
||||
})),
|
||||
);
|
||||
renderShell();
|
||||
await expandAdministration();
|
||||
|
||||
const session = await screen.findByTestId("session-item-s1");
|
||||
await userEvent.click(session);
|
||||
@@ -176,7 +157,7 @@ test("keeps a live core session connected while returning from database manageme
|
||||
expect(FakeEventSource.instances).toHaveLength(1);
|
||||
});
|
||||
|
||||
test("hides the complete administrative navigation from a regular user", () => {
|
||||
test("keeps read-safe workspace access but hides privileged management entries", () => {
|
||||
clearAuthState();
|
||||
setAuthState({
|
||||
issuer: "test",
|
||||
@@ -190,9 +171,7 @@ test("hides the complete administrative navigation from a regular user", () => {
|
||||
|
||||
renderShell();
|
||||
|
||||
expect(screen.getByRole("button", { name: "New session" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "Workspace management" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
|
||||
});
|
||||
@@ -203,13 +182,6 @@ test("does not leave database management without confirming a dirty form", async
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "ready",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
@@ -230,7 +202,6 @@ test("does not leave database management without confirming a dirty form", async
|
||||
}])));
|
||||
const confirm = vi.spyOn(window, "confirm").mockReturnValue(false);
|
||||
renderShell();
|
||||
await expandAdministration();
|
||||
|
||||
await userEvent.click(screen.getByRole("button", { name: "Database management" }));
|
||||
await userEvent.click(await screen.findByRole("button", { name: "Edit Policlinico San Donato" }));
|
||||
|
||||
@@ -156,11 +156,6 @@ test("model selector shows the three Pi-enabled models and stores the selected p
|
||||
});
|
||||
|
||||
test("opens Workspace management from the right sidebar without interrupting the shell", async () => {
|
||||
setAuthState({
|
||||
issuer: "test", subject: "admin", roles: ["admin"],
|
||||
permissions: ["session.use", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"],
|
||||
isAdmin: true, csrfToken: null, session: null,
|
||||
});
|
||||
server.use(
|
||||
http.get("/api/workspace-registry/status", () =>
|
||||
HttpResponse.json({ branch: "main", ahead: 0, behind: 0, degraded: false })),
|
||||
@@ -168,31 +163,15 @@ test("opens Workspace management from the right sidebar without interrupting the
|
||||
);
|
||||
renderShell();
|
||||
|
||||
await userEvent.click(screen.getByRole("button", { name: "Administration" }));
|
||||
await userEvent.click(screen.getByRole("button", { name: "Workspace management" }));
|
||||
|
||||
expect(await screen.findByRole("heading", { name: "Workspace management" })).toBeVisible();
|
||||
const dialog = screen.getByRole("dialog", { name: "Workspace management" });
|
||||
const workArea = screen.getByTestId("conversation-column");
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const workspaceManagement = within(sessionNavigation).getByRole("button", { name: "Workspace management" });
|
||||
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
|
||||
expect(workArea).toContainElement(dialog);
|
||||
expect(sessionNavigation).not.toContainElement(dialog);
|
||||
expect(screen.getByTestId("app-shell")).toHaveAttribute("data-activity-layout", "closed");
|
||||
expect(workspaceManagement).toHaveAttribute("aria-current", "page");
|
||||
expect(workspaceManagement).toHaveAttribute("aria-expanded", "true");
|
||||
expect(workspaceManagement).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(workspaceManagement).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(newSession).not.toHaveAttribute("aria-current");
|
||||
|
||||
await userEvent.click(screen.getByRole("button", { name: "Close workspace management" }));
|
||||
await waitFor(() => expect(screen.queryByRole("dialog", { name: "Workspace management" })).not.toBeInTheDocument());
|
||||
expect(newSession).toHaveAttribute("aria-current", "page");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(workspaceManagement).toHaveAttribute("aria-expanded", "false");
|
||||
expect(workspaceManagement).toHaveAttribute("data-navigation-state", "available");
|
||||
});
|
||||
|
||||
test("does not block the shell when the DWH is unavailable at startup", async () => {
|
||||
@@ -205,8 +184,7 @@ test("does not block the shell when the DWH is unavailable at startup", async ()
|
||||
);
|
||||
renderShell();
|
||||
|
||||
expect(await screen.findByRole("button", { name: "New session" })).toBeVisible();
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(await screen.findByRole("button", { name: "Workspace management" })).toBeVisible();
|
||||
expect(screen.queryByRole("heading", { name: "Connection unavailable" })).not.toBeInTheDocument();
|
||||
expect(healthChecks).toBe(0);
|
||||
});
|
||||
|
||||
@@ -16,7 +16,7 @@ const regularUser: AuthenticatedUser = {
|
||||
};
|
||||
const adminUser: AuthenticatedUser = {
|
||||
...regularUser, subject: "alice-id", roles: ["admin"] as const,
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"] as const, isAdmin: true,
|
||||
permissions: ["session.use", "session.read_all", "pi.manage"] as const, isAdmin: true,
|
||||
};
|
||||
|
||||
function wrap(user: AuthenticatedUser = regularUser) {
|
||||
@@ -95,8 +95,7 @@ test("regular users load only their sessions and never see administrator control
|
||||
wrap();
|
||||
await screen.findByText("Attiva uno");
|
||||
expect(scope).toBe("mine");
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByText(/administrator view/i)).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
@@ -116,25 +115,17 @@ test("Pi management preserves the open session summary and the model activity ti
|
||||
http.post("/api/sessions/:id/resume", () => resumeResult("s1")),
|
||||
http.get("/api/sessions/:id", () => HttpResponse.json({ phase: 4 })),
|
||||
);
|
||||
wrap(adminUser);
|
||||
wrap();
|
||||
|
||||
await user.click(await screen.findByText("Attiva uno"));
|
||||
expect(await screen.findByRole("complementary", { name: "Session summary" })).toHaveTextContent("Domanda originale");
|
||||
await user.click(screen.getByRole("button", { name: "Administration" }));
|
||||
await user.click(screen.getByRole("button", { name: "Pi management" }));
|
||||
expect(await screen.findByRole("heading", { name: "Pi management" })).toBeVisible();
|
||||
const dialog = screen.getByRole("dialog", { name: "Pi management" });
|
||||
const workArea = screen.getByTestId("conversation-column");
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const piManagement = within(sessionNavigation).getByRole("button", { name: "Pi management" });
|
||||
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
|
||||
expect(workArea).toContainElement(dialog);
|
||||
expect(sessionNavigation).not.toContainElement(dialog);
|
||||
expect(piManagement).toHaveAttribute("aria-current", "page");
|
||||
expect(piManagement).toHaveAttribute("aria-expanded", "true");
|
||||
expect(piManagement).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(newSession).not.toHaveAttribute("aria-current");
|
||||
const preservedSummary = screen.getByRole("complementary", { name: "Session summary" });
|
||||
expect(preservedSummary).not.toHaveAttribute("aria-hidden", "true");
|
||||
expect(preservedSummary).toHaveTextContent("Attiva uno");
|
||||
@@ -142,10 +133,6 @@ test("Pi management preserves the open session summary and the model activity ti
|
||||
await waitFor(() => {
|
||||
expect(screen.getByRole("complementary", { name: "Session summary" })).toHaveTextContent("Domanda originale");
|
||||
});
|
||||
expect(newSession).toHaveAttribute("aria-current", "page");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(piManagement).toHaveAttribute("aria-expanded", "false");
|
||||
expect(piManagement).toHaveAttribute("data-navigation-state", "available");
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Resume" }));
|
||||
await screen.findByRole("button", { name: "Show model activity" });
|
||||
@@ -173,66 +160,17 @@ test("administrators can explicitly switch to all sessions and see owners", asyn
|
||||
}),
|
||||
);
|
||||
wrap(adminUser);
|
||||
const tablist = await screen.findByRole("tablist", { name: "Session scope" });
|
||||
const mySessions = within(tablist).getByRole("tab", { name: "My sessions" });
|
||||
const allSessions = within(tablist).getByRole("tab", { name: "All sessions" });
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "true");
|
||||
expect(mySessions).toHaveAttribute("aria-controls", "session-scope-panel");
|
||||
expect(mySessions).toHaveAttribute("tabindex", "0");
|
||||
expect(mySessions).toHaveAttribute("data-tab-state", "active");
|
||||
expect(mySessions).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "false");
|
||||
expect(allSessions).toHaveAttribute("aria-controls", "session-scope-panel");
|
||||
expect(allSessions).toHaveAttribute("tabindex", "-1");
|
||||
expect(allSessions).toHaveAttribute("data-tab-state", "inactive");
|
||||
expect(allSessions).toHaveClass("border-border", "bg-card");
|
||||
expect(allSessions).not.toHaveClass("border-transparent", "bg-transparent");
|
||||
expect(screen.getByRole("tabpanel")).toHaveAttribute("aria-labelledby", "session-scope-mine-tab");
|
||||
await userEvent.click(allSessions);
|
||||
await screen.findByRole("button", { name: "All sessions" });
|
||||
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "true");
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "false");
|
||||
await userEvent.click(screen.getByRole("button", { name: "All sessions" }));
|
||||
await waitFor(() => expect(scope).toBe("all"));
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "false");
|
||||
expect(mySessions).toHaveAttribute("data-tab-state", "inactive");
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "true");
|
||||
expect(allSessions).toHaveAttribute("data-tab-state", "active");
|
||||
expect(allSessions).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(screen.getByRole("tabpanel")).toHaveAttribute("aria-labelledby", "session-scope-all-tab");
|
||||
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "false");
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "true");
|
||||
expect(await screen.findByText("Administrator view: all sessions")).toBeInTheDocument();
|
||||
expect(screen.getByText("Owner: Bob")).toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("session scope tabs support roving keyboard navigation", async () => {
|
||||
let scope = "";
|
||||
server.use(http.get("/api/sessions", ({ request }) => {
|
||||
scope = new URL(request.url).searchParams.get("scope") ?? "";
|
||||
return HttpResponse.json(LIST);
|
||||
}));
|
||||
wrap(adminUser);
|
||||
|
||||
const user = userEvent.setup();
|
||||
const tablist = await screen.findByRole("tablist", { name: "Session scope" });
|
||||
const mySessions = within(tablist).getByRole("tab", { name: "My sessions" });
|
||||
const allSessions = within(tablist).getByRole("tab", { name: "All sessions" });
|
||||
mySessions.focus();
|
||||
|
||||
await user.keyboard("{ArrowRight}");
|
||||
await waitFor(() => expect(scope).toBe("all"));
|
||||
expect(allSessions).toHaveFocus();
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "true");
|
||||
expect(mySessions).toHaveAttribute("tabindex", "-1");
|
||||
|
||||
await user.keyboard("{ArrowLeft}");
|
||||
await waitFor(() => expect(scope).toBe("mine"));
|
||||
expect(mySessions).toHaveFocus();
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "true");
|
||||
|
||||
await user.keyboard("{End}");
|
||||
expect(allSessions).toHaveFocus();
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "true");
|
||||
await user.keyboard("{Home}");
|
||||
expect(mySessions).toHaveFocus();
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "true");
|
||||
});
|
||||
|
||||
test("administrator confirms before deleting a same-named user's session", async () => {
|
||||
let deletes = 0;
|
||||
server.use(
|
||||
@@ -249,7 +187,7 @@ test("administrator confirms before deleting a same-named user's session", async
|
||||
}),
|
||||
);
|
||||
wrap(adminUser);
|
||||
await userEvent.click(await screen.findByRole("tab", { name: "All sessions" }));
|
||||
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
|
||||
await screen.findByText("Owner: Alice");
|
||||
await userEvent.click(screen.getByRole("checkbox", { name: "Select Attiva uno" }));
|
||||
await userEvent.click(screen.getByRole("button", { name: "Delete 1 selected sessions" }));
|
||||
@@ -275,7 +213,7 @@ test("administrator confirms before archiving a same-named user's session", asyn
|
||||
}),
|
||||
);
|
||||
wrap(adminUser);
|
||||
await userEvent.click(await screen.findByRole("tab", { name: "All sessions" }));
|
||||
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
|
||||
await screen.findByText("Owner: Alice");
|
||||
await userEvent.click(screen.getByRole("button", { name: "Session actions" }));
|
||||
await userEvent.click(await screen.findByText("Archive"));
|
||||
|
||||
+94
-219
@@ -17,9 +17,8 @@ import { StopConfirmDialog } from "./StopConfirmDialog";
|
||||
import { SteerInput, ComposerFooter } from "./SteerInput";
|
||||
import { WorkflowBar } from "./WorkflowBar";
|
||||
import { DatabaseManagementPage } from "./DatabaseManagementPage";
|
||||
import { Pencil, ArrowLeft, ArrowRight, ChevronDown, Trash2 } from "lucide-react";
|
||||
import { Accordion } from "@base-ui/react/accordion";
|
||||
import { Button, buttonVariants } from "../components/ui/button";
|
||||
import { Pencil, ArrowLeft, ArrowRight, Trash2 } from "lucide-react";
|
||||
import { Button } from "../components/ui/button";
|
||||
import { Checkbox } from "../components/ui/checkbox";
|
||||
import { Toaster } from "../components/ui/sonner";
|
||||
import { toast } from "sonner";
|
||||
@@ -35,7 +34,7 @@ import type { SessionScope, SessionSummary } from "../api/types";
|
||||
import { useAuthGeneration, useAuthUser } from "../auth/authState";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import type { CSSProperties, KeyboardEvent } from "react";
|
||||
import type { CSSProperties } from "react";
|
||||
import { captureAuthOperation, isAuthOperationCurrent, StaleAuthOperationError, type AuthOperationGuard } from "../auth/authOperation";
|
||||
|
||||
interface AppShellProps {
|
||||
@@ -44,31 +43,6 @@ interface AppShellProps {
|
||||
|
||||
type ActiveSurface = "core" | "database-management";
|
||||
type ActiveManagementPanel = "workspace" | "pi" | null;
|
||||
const SESSION_SCOPES: readonly SessionScope[] = ["mine", "all"];
|
||||
const APP_NAVIGATION_STORAGE_KEY = "thothii:app-navigation";
|
||||
|
||||
function readPersistedNavigation(): { surface: ActiveSurface; panel: ActiveManagementPanel } {
|
||||
if (import.meta.env.MODE === "test") return { surface: "core", panel: null };
|
||||
try {
|
||||
const parsed = JSON.parse(window.localStorage.getItem(APP_NAVIGATION_STORAGE_KEY) ?? "null") as {
|
||||
surface?: unknown;
|
||||
panel?: unknown;
|
||||
} | null;
|
||||
return {
|
||||
surface: parsed?.surface === "database-management" ? "database-management" : "core",
|
||||
panel: parsed?.panel === "workspace" || parsed?.panel === "pi" ? parsed.panel : null,
|
||||
};
|
||||
} catch {
|
||||
return { surface: "core", panel: null };
|
||||
}
|
||||
}
|
||||
|
||||
function sessionScopeTabClass(selected: boolean): string {
|
||||
const base = "relative -mb-px flex h-8 w-full cursor-pointer items-center justify-center rounded-t-md border border-b-2 px-2 text-xs font-semibold tracking-[0.005em] outline-none focus-visible:z-10 focus-visible:ring-3 focus-visible:ring-[oklch(var(--nav-active-border)/0.28)]";
|
||||
return selected
|
||||
? `${base} border-[oklch(var(--nav-active-border))] bg-[oklch(var(--nav-active))] text-[oklch(var(--nav-active-foreground))] hover:bg-[oklch(var(--nav-active-hover))]`
|
||||
: `${base} border-border bg-card text-muted-foreground hover:border-muted-foreground/45 hover:bg-muted/55 hover:text-foreground`;
|
||||
}
|
||||
|
||||
type DatabaseManagementPresentation = "legacy" | "fleet";
|
||||
|
||||
@@ -155,13 +129,8 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
const [creatingSession, setCreatingSession] = useState(false);
|
||||
const [awaitingQuestion, setAwaitingQuestion] = useState(false);
|
||||
const [sessionScope, setSessionScope] = useState<SessionScope>("mine");
|
||||
const sessionScopeTabRefs = useRef<Record<SessionScope, HTMLButtonElement | null>>({
|
||||
mine: null,
|
||||
all: null,
|
||||
});
|
||||
const principal = authenticatedUser;
|
||||
const permissions = authenticatedUser?.permissions ?? [];
|
||||
const isAdmin = authenticatedUser?.isAdmin === true;
|
||||
const canReadAllSessions = permissions.includes("session.read_all");
|
||||
const canManageWorkspace = permissions.includes("workspace.manage");
|
||||
const canManageWorkspaceSecrets = permissions.includes("workspace.secrets.manage");
|
||||
@@ -183,8 +152,7 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
|
||||
const queryClient = useQueryClient();
|
||||
const [showActivity, setShowActivity] = useState(false);
|
||||
const persistedNavigation = readPersistedNavigation();
|
||||
const [activeSurface, setActiveSurface] = useState<ActiveSurface>(persistedNavigation.surface);
|
||||
const [activeSurface, setActiveSurface] = useState<ActiveSurface>("core");
|
||||
const databaseManagementPresentation = resolveDatabaseManagementPresentation({
|
||||
isDevelopment: import.meta.env.DEV,
|
||||
mode: import.meta.env.MODE,
|
||||
@@ -195,8 +163,7 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
const updateDatabaseNavigationState = useCallback((state: { dirty: boolean; busy: boolean }) => {
|
||||
databaseNavigationRef.current = state;
|
||||
}, []);
|
||||
const [activeManagementPanel, setActiveManagementPanel] = useState<ActiveManagementPanel>(persistedNavigation.panel);
|
||||
const [adminNavigationValue, setAdminNavigationValue] = useState<string[]>([]);
|
||||
const [activeManagementPanel, setActiveManagementPanel] = useState<ActiveManagementPanel>(null);
|
||||
const [activeOpen, setActiveOpen] = useState(true);
|
||||
const [archiveOpen, setArchiveOpen] = useState(false);
|
||||
const [renameTarget, setRenameTarget] = useState<SessionSummary | null>(null);
|
||||
@@ -206,10 +173,6 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
const [collapsedGroups, setCollapsedGroups] = useState<Record<string, boolean>>({});
|
||||
const [renameGroupTarget, setRenameGroupTarget] = useState<string | null>(null);
|
||||
const operationEpochRef = useRef(0);
|
||||
useEffect(() => {
|
||||
if (import.meta.env.MODE === "test") return;
|
||||
window.localStorage.setItem(APP_NAVIGATION_STORAGE_KEY, JSON.stringify({ surface: activeSurface, panel: activeManagementPanel }));
|
||||
}, [activeManagementPanel, activeSurface]);
|
||||
useEffect(() => () => { operationEpochRef.current += 1; }, []);
|
||||
|
||||
const groups = useMemo(
|
||||
@@ -273,22 +236,6 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
setSelectedSessionIds(selected ? new Set(sessions.map((session) => session.id)) : new Set());
|
||||
}
|
||||
|
||||
function handleSessionScopeTabKeyDown(
|
||||
event: KeyboardEvent<HTMLButtonElement>,
|
||||
current: SessionScope,
|
||||
) {
|
||||
const currentIndex = SESSION_SCOPES.indexOf(current);
|
||||
let target: SessionScope | undefined;
|
||||
if (event.key === "ArrowRight") target = SESSION_SCOPES[(currentIndex + 1) % SESSION_SCOPES.length];
|
||||
else if (event.key === "ArrowLeft") target = SESSION_SCOPES[(currentIndex - 1 + SESSION_SCOPES.length) % SESSION_SCOPES.length];
|
||||
else if (event.key === "Home") target = SESSION_SCOPES[0];
|
||||
else if (event.key === "End") target = SESSION_SCOPES.at(-1);
|
||||
if (!target) return;
|
||||
event.preventDefault();
|
||||
setSessionScope(target);
|
||||
sessionScopeTabRefs.current[target]?.focus();
|
||||
}
|
||||
|
||||
function canLeaveDatabaseManagement(): boolean {
|
||||
if (activeSurface !== "database-management") return true;
|
||||
if (databaseNavigationRef.current.busy) {
|
||||
@@ -680,12 +627,6 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
await logoutUser();
|
||||
}
|
||||
|
||||
const currentNavigation = activeSurface === "database-management"
|
||||
? "database"
|
||||
: activeManagementPanel ?? "core";
|
||||
const adminNavigationOpen = adminNavigationValue.includes("administration");
|
||||
const managementNavigationCurrent = currentNavigation !== "core";
|
||||
|
||||
return (
|
||||
<div
|
||||
ref={containerRef}
|
||||
@@ -777,7 +718,7 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
<>
|
||||
{activeSession?.question && (
|
||||
<header className="sticky top-0 z-10 -mx-6 -mt-8 border-b border-border/60 bg-background/95 px-6 pb-3 pt-8 backdrop-blur supports-[backdrop-filter]:bg-background/80">
|
||||
<p className="thot-label mb-0.5 text-muted-foreground">Question</p>
|
||||
<p className="thot-label mb-0.5 text-muted-foreground">Domanda</p>
|
||||
<h2 className="font-heading text-[0.95rem] font-semibold leading-snug text-foreground break-words">
|
||||
{activeSession.question}
|
||||
</h2>
|
||||
@@ -849,7 +790,7 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
<p className="thot-label mt-1.5">
|
||||
Datamart Builder with
|
||||
<br />
|
||||
Human-in-the-loop review
|
||||
Human In The Loop
|
||||
</p>
|
||||
{authenticatedUser && (
|
||||
<div className="mt-4 flex items-center justify-between gap-2 border-t border-border/70 pt-3 text-left">
|
||||
@@ -867,180 +808,115 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
|
||||
<div className="flex flex-col gap-2 px-4 pb-3">
|
||||
<Button
|
||||
variant={currentNavigation === "core" ? "navigationActive" : "outline"}
|
||||
variant="default"
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "core" ? "page" : undefined}
|
||||
data-navigation-state={currentNavigation === "core" ? "current" : "available"}
|
||||
onClick={startNewSession}
|
||||
>
|
||||
New session
|
||||
</Button>
|
||||
{isAdmin && (
|
||||
<Accordion.Root
|
||||
value={adminNavigationValue}
|
||||
onValueChange={setAdminNavigationValue}
|
||||
className="rounded-lg border border-border/80 bg-card/45 p-1.5"
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="w-full"
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("workspace");
|
||||
}}
|
||||
>
|
||||
Workspace management
|
||||
</Button>
|
||||
{canManageDatabase && (
|
||||
<Button
|
||||
variant={activeSurface === "database-management" ? "secondary" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={activeSurface === "database-management" ? "page" : undefined}
|
||||
onClick={() => {
|
||||
setActiveManagementPanel(null);
|
||||
setActiveSurface("database-management");
|
||||
}}
|
||||
>
|
||||
<Accordion.Item value="administration">
|
||||
<Accordion.Header className="m-0">
|
||||
<Accordion.Trigger
|
||||
className={buttonVariants({
|
||||
variant: !adminNavigationOpen && managementNavigationCurrent ? "navigationActive" : "outline",
|
||||
size: "sm",
|
||||
className: "w-full justify-between px-2.5",
|
||||
})}
|
||||
data-navigation-state={!adminNavigationOpen && managementNavigationCurrent ? "current" : "available"}
|
||||
>
|
||||
<span>Administration</span>
|
||||
<ChevronDown
|
||||
aria-hidden="true"
|
||||
data-icon="inline-end"
|
||||
className={`transition-transform duration-150 motion-reduce:transition-none ${adminNavigationOpen ? "rotate-180" : ""}`}
|
||||
/>
|
||||
</Accordion.Trigger>
|
||||
</Accordion.Header>
|
||||
<Accordion.Panel className="grid gap-1.5 px-0.5 pt-1.5">
|
||||
<Button
|
||||
variant={currentNavigation === "database" ? "navigationActive" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "database" ? "page" : undefined}
|
||||
data-navigation-state={currentNavigation === "database" ? "current" : canManageDatabase ? "available" : "unavailable"}
|
||||
disabled={!canManageDatabase}
|
||||
title={canManageDatabase ? undefined : "Database management permission is required"}
|
||||
onClick={() => {
|
||||
setActiveManagementPanel(null);
|
||||
setActiveSurface("database-management");
|
||||
}}
|
||||
>
|
||||
Database management
|
||||
</Button>
|
||||
<div role="separator" aria-orientation="horizontal" className="mx-1 h-px bg-border/90" />
|
||||
<Button
|
||||
variant={currentNavigation === "workspace" ? "navigationActive" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "workspace" ? "page" : undefined}
|
||||
aria-expanded={activeManagementPanel === "workspace"}
|
||||
data-navigation-state={currentNavigation === "workspace" ? "current" : "available"}
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("workspace");
|
||||
}}
|
||||
>
|
||||
Workspace management
|
||||
</Button>
|
||||
<Button
|
||||
variant={currentNavigation === "pi" ? "navigationActive" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "pi" ? "page" : undefined}
|
||||
aria-expanded={activeManagementPanel === "pi"}
|
||||
data-navigation-state={currentNavigation === "pi" ? "current" : canManagePi ? "available" : "unavailable"}
|
||||
disabled={!canManagePi}
|
||||
title={canManagePi ? undefined : "Pi management permission is required"}
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("pi");
|
||||
}}
|
||||
>
|
||||
Pi management
|
||||
</Button>
|
||||
</Accordion.Panel>
|
||||
</Accordion.Item>
|
||||
</Accordion.Root>
|
||||
Database management
|
||||
</Button>
|
||||
)}
|
||||
{canManagePi && (
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="w-full"
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("pi");
|
||||
}}
|
||||
>
|
||||
Pi management
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{canReadAllSessions && (
|
||||
<div className="px-4">
|
||||
<div
|
||||
role="tablist"
|
||||
aria-label="Session scope"
|
||||
aria-orientation="horizontal"
|
||||
className="grid grid-cols-2 border-b border-border"
|
||||
>
|
||||
<button
|
||||
ref={(node) => { sessionScopeTabRefs.current.mine = node; }}
|
||||
id="session-scope-mine-tab"
|
||||
type="button"
|
||||
role="tab"
|
||||
aria-selected={sessionScope === "mine"}
|
||||
aria-controls="session-scope-panel"
|
||||
tabIndex={sessionScope === "mine" ? 0 : -1}
|
||||
data-tab-state={sessionScope === "mine" ? "active" : "inactive"}
|
||||
className={sessionScopeTabClass(sessionScope === "mine")}
|
||||
<div className="px-4 pb-3">
|
||||
<div className="grid grid-cols-2 gap-1 rounded-lg bg-muted p-1" aria-label="Session scope">
|
||||
<Button
|
||||
variant={sessionScope === "mine" ? "secondary" : "ghost"}
|
||||
size="xs"
|
||||
aria-pressed={sessionScope === "mine"}
|
||||
onClick={() => setSessionScope("mine")}
|
||||
onKeyDown={(event) => handleSessionScopeTabKeyDown(event, "mine")}
|
||||
>
|
||||
My sessions
|
||||
</button>
|
||||
<button
|
||||
ref={(node) => { sessionScopeTabRefs.current.all = node; }}
|
||||
id="session-scope-all-tab"
|
||||
type="button"
|
||||
role="tab"
|
||||
aria-selected={showingAllSessions}
|
||||
aria-controls="session-scope-panel"
|
||||
tabIndex={showingAllSessions ? 0 : -1}
|
||||
data-tab-state={showingAllSessions ? "active" : "inactive"}
|
||||
className={sessionScopeTabClass(showingAllSessions)}
|
||||
</Button>
|
||||
<Button
|
||||
variant={showingAllSessions ? "secondary" : "ghost"}
|
||||
size="xs"
|
||||
aria-pressed={showingAllSessions}
|
||||
onClick={() => setSessionScope("all")}
|
||||
onKeyDown={(event) => handleSessionScopeTabKeyDown(event, "all")}
|
||||
>
|
||||
All sessions
|
||||
</button>
|
||||
</Button>
|
||||
</div>
|
||||
{showingAllSessions && (
|
||||
<p className="mt-2 text-xs font-medium text-amber-700 dark:text-amber-400">
|
||||
Administrator view: all sessions
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div
|
||||
id={canReadAllSessions ? "session-scope-panel" : undefined}
|
||||
role={canReadAllSessions ? "tabpanel" : undefined}
|
||||
aria-labelledby={canReadAllSessions ? `session-scope-${sessionScope}-tab` : undefined}
|
||||
className="flex min-h-0 flex-1 flex-col"
|
||||
>
|
||||
{canReadAllSessions && showingAllSessions && (
|
||||
<p className="mx-4 mb-2 mt-2 text-xs font-medium text-amber-700 dark:text-amber-400">
|
||||
Administrator view: all sessions
|
||||
</p>
|
||||
{/* L1 — rail title */}
|
||||
<div className="px-4 pb-1.5 pt-1">
|
||||
<span className="thot-label text-[0.8rem] font-bold tracking-[0.18em] text-primary">
|
||||
Sessions
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between px-4 pb-2">
|
||||
<label className="flex cursor-pointer items-center gap-2 text-xs font-medium text-muted-foreground hover:text-foreground">
|
||||
<Checkbox
|
||||
checked={allSessionsSelected}
|
||||
aria-label="Select all sessions"
|
||||
disabled={sessions.length === 0}
|
||||
onCheckedChange={(selected) => toggleAllSessions(selected === true)}
|
||||
/>
|
||||
<span>Select all</span>
|
||||
</label>
|
||||
{selectedSessions.length > 0 && (
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="xs"
|
||||
aria-label={`Delete ${selectedSessions.length} selected sessions`}
|
||||
onClick={() => {
|
||||
requestDelete(selectedSessions);
|
||||
}}
|
||||
>
|
||||
<Trash2 />
|
||||
Delete ({selectedSessions.length})
|
||||
</Button>
|
||||
)}
|
||||
|
||||
{/* L1 — rail title */}
|
||||
<div className="px-4 pb-1.5 pt-1">
|
||||
<span className="thot-label text-[0.8rem] font-bold tracking-[0.18em] text-primary">
|
||||
Sessions
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between px-4 pb-2">
|
||||
<label className="flex cursor-pointer items-center gap-2 text-xs font-medium text-muted-foreground hover:text-foreground">
|
||||
<Checkbox
|
||||
checked={allSessionsSelected}
|
||||
aria-label="Select all sessions"
|
||||
disabled={sessions.length === 0}
|
||||
onCheckedChange={(selected) => toggleAllSessions(selected === true)}
|
||||
/>
|
||||
<span>Select all</span>
|
||||
</label>
|
||||
{selectedSessions.length > 0 && (
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="xs"
|
||||
aria-label={`Delete ${selectedSessions.length} selected sessions`}
|
||||
onClick={() => {
|
||||
requestDelete(selectedSessions);
|
||||
}}
|
||||
>
|
||||
<Trash2 />
|
||||
Delete ({selectedSessions.length})
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex-1 overflow-y-auto px-2 pb-4">
|
||||
</div>
|
||||
<div className="flex-1 overflow-y-auto px-2 pb-4">
|
||||
{/* L2 — section toggle */}
|
||||
<button
|
||||
type="button"
|
||||
@@ -1126,7 +1002,6 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
showOwner={showingAllSessions}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</aside>
|
||||
)}
|
||||
|
||||
@@ -29,13 +29,6 @@ function makeDatabase(overrides: Partial<CatalogDatabase> = {}): CatalogDatabase
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "ready",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
@@ -73,9 +66,6 @@ const orphan = makeDatabase({
|
||||
workspaceId: "retired",
|
||||
workspaceName: "Retired workspace",
|
||||
workspaceAvailable: false,
|
||||
workspaceRevision: null,
|
||||
workspaceEvidence: { sourceType: null, state: "workspace_unavailable" },
|
||||
runtimeBinding: null,
|
||||
});
|
||||
|
||||
function renderPage({
|
||||
@@ -197,11 +187,12 @@ function registerCompletedSyncRun(run: CatalogSyncRun) {
|
||||
|
||||
const synchronizationScopes = [
|
||||
{ scope: "tables", label: "Synchronize tables" },
|
||||
{ scope: "columns", label: "Synchronize all columns" },
|
||||
{ scope: "relationships", label: "Synchronize relationships" },
|
||||
{ scope: "all", label: "Synchronize all" },
|
||||
] as const;
|
||||
|
||||
test("renders Fleet Ledger with real metrics, conceptual row tooltips, and persistent model selection", async () => {
|
||||
test("renders Fleet Ledger with real metrics, conceptual row tooltips, and contextual model selection", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
http.get("/api/catalog/metrics", () => HttpResponse.json({
|
||||
@@ -232,13 +223,10 @@ test("renders Fleet Ledger with real metrics, conceptual row tooltips, and persi
|
||||
expect(screen.queryByText("Thoth catalog · Fleet ledger")).not.toBeInTheDocument();
|
||||
expect(screen.queryByText("Inspect physical metadata, curate descriptions and control catalog operations.")).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Back to workspace" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
|
||||
expect(screen.getByRole("columnheader", { name: /Catalog status/ })).toBeVisible();
|
||||
const summary = screen.getByRole("region", { name: "Fleet summary" });
|
||||
expect(await within(summary).findByText("2,275")).toBeVisible();
|
||||
expect(within(summary).getByText("75%")).toBeVisible();
|
||||
const modelSelector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
expect(modelSelector).toHaveValue("local-qwen");
|
||||
expect(screen.queryByRole("combobox", { name: "Metadata description model" })).not.toBeInTheDocument();
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
const tablesAction = within(databaseRow).getByRole("button", { name: "View tables for Policlinico San Donato" });
|
||||
@@ -258,143 +246,12 @@ test("renders Fleet Ledger with real metrics, conceptual row tooltips, and persi
|
||||
await user.click(within(restoredDatabaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
const actionPicker = screen.getByRole("combobox", { name: "Batch action" });
|
||||
expect(within(actionPicker).getByRole("option", { name: /^Synchronize all(?:,|$)/ })).toBeVisible();
|
||||
expect(within(actionPicker).queryByRole("option", { name: "Synchronize all columns" })).not.toBeInTheDocument();
|
||||
await user.selectOptions(actionPicker, "generate-missing");
|
||||
expect(screen.getAllByRole("combobox", { name: "Metadata-generation LLM model" })).toHaveLength(1);
|
||||
expect(modelSelector).toHaveValue("local-qwen");
|
||||
expect(await screen.findByRole("combobox", { name: "Metadata description model" })).toHaveValue("local-qwen");
|
||||
|
||||
});
|
||||
|
||||
test("shows synchronization, description, and sensitive-data states independently", async () => {
|
||||
const configured = makeDatabase({ connectionStatus: "reachable", testedVersion: 3 });
|
||||
const needsRuntimeConfiguration = makeDatabase({
|
||||
...unconfigured,
|
||||
workspaceEvidence: { sourceType: "http", state: "configuration_required" },
|
||||
runtimeBinding: {
|
||||
transport: "rest_api",
|
||||
configurationState: "configuration_required",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
});
|
||||
renderPage({ rows: [configured, needsRuntimeConfiguration, orphan], presentation: "fleet" });
|
||||
|
||||
const configuredRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
expect(within(configuredRow).getByText("Not synchronized")).toBeVisible();
|
||||
expect(within(configuredRow).getByText(/Descriptions ·/)).toBeVisible();
|
||||
expect(within(configuredRow).getByText(/Sensitive ·/)).toBeVisible();
|
||||
|
||||
const needsConfigurationRow = screen.getByRole("row", { name: /Research laboratory/ });
|
||||
expect(within(needsConfigurationRow).getByText("Not synchronized")).toBeVisible();
|
||||
expect(within(needsConfigurationRow).getByText(/Descriptions ·/)).toBeVisible();
|
||||
|
||||
const orphanRow = screen.getByRole("row", { name: /Retired workspace/ });
|
||||
expect(within(orphanRow).getByText("Not synchronized")).toBeVisible();
|
||||
});
|
||||
|
||||
test("shows the configured access type and endpoint in catalog status", async () => {
|
||||
const direct = makeDatabase();
|
||||
const rest = makeDatabase({
|
||||
id: "33333333-3333-4333-8333-333333333333",
|
||||
workspaceId: "rest-lab",
|
||||
workspaceName: "REST laboratory",
|
||||
binding: {
|
||||
transport: "rest_api",
|
||||
baseUrl: "https://203.0.113.18/dwh/",
|
||||
restPath: "/health",
|
||||
restAuth: "x-api-key",
|
||||
},
|
||||
});
|
||||
const ssh = makeDatabase({
|
||||
id: "44444444-4444-4444-8444-444444444444",
|
||||
workspaceId: "ssh-lab",
|
||||
workspaceName: "SSH laboratory",
|
||||
binding: {
|
||||
transport: "ssh_tunnel",
|
||||
sshHost: "bastion.example.test",
|
||||
sshPort: 2222,
|
||||
sshUsername: "operator",
|
||||
sshTargetHost: "postgres.internal",
|
||||
sshTargetPort: 5432,
|
||||
username: "reader",
|
||||
},
|
||||
});
|
||||
renderPage({ rows: [direct, rest, ssh], presentation: "fleet" });
|
||||
|
||||
const directRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
expect(within(directRow).getByText("Direct PostgreSQL connection")).toBeVisible();
|
||||
expect(within(directRow).getByText("db.internal:5432")).toBeVisible();
|
||||
|
||||
const restRow = screen.getByRole("row", { name: /REST laboratory/ });
|
||||
expect(within(restRow).getByText("REST")).toBeVisible();
|
||||
expect(within(restRow).getByText("203.0.113.18")).toBeVisible();
|
||||
|
||||
const sshRow = screen.getByRole("row", { name: /SSH laboratory/ });
|
||||
expect(within(sshRow).getByText("SSH")).toBeVisible();
|
||||
expect(within(sshRow).getByText("bastion.example.test:2222")).toBeVisible();
|
||||
});
|
||||
|
||||
test("keeps Fleet Ledger visible when a catalog response omits the Evidence projection", async () => {
|
||||
const legacyDatabase = makeDatabase();
|
||||
delete (legacyDatabase as Partial<CatalogDatabase>).workspaceEvidence;
|
||||
|
||||
renderPage({ rows: [legacyDatabase], presentation: "fleet" });
|
||||
|
||||
expect(await screen.findByRole("heading", { name: "Database management" })).toBeVisible();
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
expect(within(databaseRow).getByText("Not synchronized")).toBeVisible();
|
||||
});
|
||||
|
||||
test("opens the relationship map directly from a Fleet database and restores focus on return", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/relationships", () => HttpResponse.json([])),
|
||||
);
|
||||
renderPage({
|
||||
rows: [makeDatabase({ connectionStatus: "reachable", testedVersion: 3 })],
|
||||
presentation: "fleet",
|
||||
});
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
const relationshipsAction = within(databaseRow).getByRole("button", {
|
||||
name: "View relationships for Policlinico San Donato",
|
||||
});
|
||||
expect(relationshipsAction.parentElement).toHaveAttribute("data-tooltip", "Relationships");
|
||||
expect(relationshipsAction).not.toHaveAttribute("title");
|
||||
|
||||
await user.click(relationshipsAction);
|
||||
expect(await screen.findByRole("region", {
|
||||
name: "Relationships for Policlinico San Donato",
|
||||
})).toBeVisible();
|
||||
expect(screen.getByText("Relationship map")).toBeVisible();
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Back to databases" }));
|
||||
await waitFor(() => expect(screen.getByRole("button", {
|
||||
name: "View relationships for Policlinico San Donato",
|
||||
})).toHaveFocus());
|
||||
});
|
||||
|
||||
test("offers catalog configuration directly on an unconfigured Fleet workspace", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderPage({ rows: [unconfigured], presentation: "fleet" });
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Research laboratory/ });
|
||||
expect(within(databaseRow).queryByRole("button", {
|
||||
name: "View relationships for Research laboratory",
|
||||
})).not.toBeInTheDocument();
|
||||
expect(within(databaseRow).queryByRole("button", {
|
||||
name: "View tables for Research laboratory",
|
||||
})).not.toBeInTheDocument();
|
||||
|
||||
await user.click(within(databaseRow).getByRole("button", {
|
||||
name: "Configure catalog for Research laboratory",
|
||||
}));
|
||||
|
||||
const drawer = await screen.findByRole("dialog", { name: "Configure catalog" });
|
||||
expect(within(drawer).getByLabelText("Workspace")).toHaveAttribute("readonly");
|
||||
expect(within(drawer).getByLabelText("Workspace")).toHaveValue("Research laboratory");
|
||||
});
|
||||
|
||||
test("keeps the table back action in the Fleet Ledger breadcrumb while browsing columns", async () => {
|
||||
test("keeps only the local back action while browsing Fleet Ledger columns", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([patientsTable])),
|
||||
@@ -407,17 +264,9 @@ test("keeps the table back action in the Fleet Ledger breadcrumb while browsing
|
||||
name: "View tables for Policlinico San Donato",
|
||||
}));
|
||||
expect(await screen.findByRole("button", { name: "Back to databases" })).toBeVisible();
|
||||
expect(screen.getByRole("combobox", { name: "Table action" })).toBeDisabled();
|
||||
expect(screen.queryByRole("button", { name: "Sync history" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Description history" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Sensitive history" })).not.toBeInTheDocument();
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "View columns for patients" }));
|
||||
const tableBackAction = await screen.findByRole("button", { name: "Back to tables" });
|
||||
expect(tableBackAction).toBeVisible();
|
||||
expect(screen.getByRole("combobox", { name: "Column action" })).toBeDisabled();
|
||||
expect(screen.getByRole("navigation", { name: "Database hierarchy" }).parentElement)
|
||||
.toContainElement(tableBackAction);
|
||||
expect(await screen.findByRole("button", { name: "Back to tables" })).toBeVisible();
|
||||
await waitFor(() => {
|
||||
expect(screen.queryByRole("button", { name: "Back to databases" })).not.toBeInTheDocument();
|
||||
});
|
||||
@@ -427,32 +276,6 @@ test("keeps the table back action in the Fleet Ledger breadcrumb while browsing
|
||||
expect(screen.queryByRole("button", { name: "Back to tables" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("uses the database back-action background treatment for the table back action", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([patientsTable])),
|
||||
http.get("/api/catalog/databases/:databaseId/tables/:tableId/columns", () => HttpResponse.json([])),
|
||||
);
|
||||
renderPage({ rows: [makeDatabase()], presentation: "fleet" });
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("button", {
|
||||
name: "View tables for Policlinico San Donato",
|
||||
}));
|
||||
const databaseBackAction = await screen.findByRole("button", { name: "Back to databases" });
|
||||
expect(databaseBackAction).toHaveClass("thot-fleet-back-action");
|
||||
const backgroundTreatment = (element: HTMLElement) => element.className
|
||||
.split(/\s+/)
|
||||
.filter((className) => /(^|:)bg-/.test(className))
|
||||
.sort();
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "View columns for patients" }));
|
||||
|
||||
const tableBackAction = await screen.findByRole("button", { name: "Back to tables" });
|
||||
expect(tableBackAction).toHaveClass("thot-fleet-back-action");
|
||||
expect(backgroundTreatment(tableBackAction)).toEqual(backgroundTreatment(databaseBackAction));
|
||||
});
|
||||
|
||||
test("reopens database synchronization history when no run is active", async () => {
|
||||
const user = userEvent.setup();
|
||||
const completed: CatalogSyncRun = {
|
||||
@@ -470,7 +293,7 @@ test("reopens database synchronization history when no run is active", async ()
|
||||
renderPage({ rows: [makeDatabase()], presentation: "fleet" });
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByText("Policlinico San Donato"));
|
||||
await user.click(within(databaseRow).getByRole("button", { name: "View synchronization history for Policlinico San Donato" }));
|
||||
|
||||
const drawer = await screen.findByRole("dialog", { name: "Schema synchronization" });
|
||||
expect(drawer).toBeVisible();
|
||||
@@ -489,7 +312,7 @@ test("selects the configured metadata-description model by default", async () =>
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
await waitFor(() => expect(selector).toHaveValue("local-qwen"));
|
||||
});
|
||||
|
||||
@@ -497,15 +320,15 @@ test("keeps both run-history buttons visible beside the metadata-description sel
|
||||
const user = userEvent.setup();
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const toolbar = screen.getByRole("group", { name: "Database management controls" });
|
||||
const metadataControls = screen.getByRole("group", { name: "Metadata description controls" });
|
||||
const actions = screen.getByRole("group", { name: "Database management actions" });
|
||||
const descriptionHistoryButton = within(metadataControls).getByRole("button", {
|
||||
name: "View description generation history",
|
||||
name: "Run descriptions generation history",
|
||||
});
|
||||
const suggestionHistoryButton = within(metadataControls).getByRole("button", {
|
||||
name: "View sensitive suggestion history",
|
||||
name: "Run sensitive suggestions history",
|
||||
});
|
||||
|
||||
expect(toolbar).toHaveClass("sm:items-end", "sm:justify-between");
|
||||
@@ -513,15 +336,15 @@ test("keeps both run-history buttons visible beside the metadata-description sel
|
||||
expect(metadataControls).toContainElement(selector);
|
||||
expect(descriptionHistoryButton).toBeVisible();
|
||||
expect(descriptionHistoryButton).toBeEnabled();
|
||||
expect(descriptionHistoryButton).toHaveTextContent("View description generation history");
|
||||
expect(descriptionHistoryButton).toHaveTextContent("Run descriptions generation history");
|
||||
expect(descriptionHistoryButton).toHaveClass("disabled:opacity-70");
|
||||
expect(suggestionHistoryButton).toBeVisible();
|
||||
expect(suggestionHistoryButton).toBeEnabled();
|
||||
expect(suggestionHistoryButton).toHaveTextContent("View sensitive suggestion history");
|
||||
expect(suggestionHistoryButton).toHaveTextContent("Run sensitive suggestions history");
|
||||
expect(toolbar.lastElementChild).toBe(actions);
|
||||
expect(actions).toHaveClass("sm:justify-end");
|
||||
expect(actions).toContainElement(screen.getByRole("button", { name: "Refresh" }));
|
||||
expect(within(actions).queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
|
||||
expect(actions).toContainElement(screen.getByRole("button", { name: "Add database" }));
|
||||
expect(screen.queryByRole("note", {
|
||||
name: "Metadata generation source data disclosure",
|
||||
})).not.toBeInTheDocument();
|
||||
@@ -548,7 +371,7 @@ test("changes the metadata-description model in page-local state", async () => {
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
await waitFor(() => expect(selector).toHaveValue("local-qwen"));
|
||||
await user.selectOptions(selector, "openai-mini");
|
||||
|
||||
@@ -562,10 +385,10 @@ test("explains application setup when no metadata-description model is configure
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
expect(selector).toBeDisabled();
|
||||
expect(await screen.findByText(
|
||||
"No metadata-generation LLM model is configured for this installation.",
|
||||
"Configure a metadata-generation model in application setup to enable description generation.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
@@ -576,11 +399,11 @@ test("uses a safe disabled setup state when metadata-description models are unav
|
||||
}, { status: 503 })));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
await waitFor(() => expect(selector).toHaveDisplayValue("Model choices unavailable"));
|
||||
expect(selector).toBeDisabled();
|
||||
expect(screen.getByText(
|
||||
"Metadata-generation LLM model choices are unavailable.",
|
||||
"Configure a metadata-generation model in application setup to enable description generation.",
|
||||
)).toBeVisible();
|
||||
expect(screen.queryByText("internal model registry details")).not.toBeInTheDocument();
|
||||
});
|
||||
@@ -594,11 +417,11 @@ test.each([null, "missing-model"])(
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
await waitFor(() => expect(selector).toHaveDisplayValue("No usable model configured"));
|
||||
expect(selector).toBeDisabled();
|
||||
expect(screen.getByText(
|
||||
"No metadata-generation LLM model is configured for this installation.",
|
||||
"Configure a metadata-generation model in application setup to enable description generation.",
|
||||
)).toBeVisible();
|
||||
},
|
||||
);
|
||||
@@ -618,7 +441,7 @@ test("renders only the public metadata-generation model contract", async () => {
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
await waitFor(() => expect(selector).toHaveValue("approved-model"));
|
||||
expect(within(selector).getAllByRole("option").map((option) => option.textContent))
|
||||
.toEqual(["Approved model"]);
|
||||
@@ -635,9 +458,8 @@ test("starts with a full-width list and applies the row action matrix", async ()
|
||||
expect(screen.getByRole("button", { name: "Delete Policlinico San Donato" })).toBeEnabled();
|
||||
|
||||
expect(screen.getByRole("button", { name: "View Research laboratory" })).toBeEnabled();
|
||||
expect(screen.getByRole("button", { name: "Configure catalog for Research laboratory" })).toBeEnabled();
|
||||
expect(screen.queryByRole("button", { name: "Edit Research laboratory" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Delete Research laboratory" })).not.toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "Edit Research laboratory" })).toBeEnabled();
|
||||
expect(screen.getByRole("button", { name: "Delete Research laboratory" })).toBeDisabled();
|
||||
|
||||
expect(screen.getByRole("button", { name: "View Retired workspace" })).toBeEnabled();
|
||||
expect(screen.getByRole("button", { name: "Edit Retired workspace" })).toBeDisabled();
|
||||
@@ -671,11 +493,10 @@ test.each(synchronizationScopes)(
|
||||
await user.click(screen.getByRole("menuitem", { name: label }));
|
||||
|
||||
await waitFor(() => expect(startBody).toEqual({ version: 3, scope, tableIds: [] }));
|
||||
expect(screen.getByText("1 selected")).toBeVisible();
|
||||
},
|
||||
);
|
||||
|
||||
test("discloses source sampling before starting Generate missing descriptions", async () => {
|
||||
test("discloses source sampling before starting Generate Missing", async () => {
|
||||
const user = userEvent.setup();
|
||||
const queuedRun = makeDescriptionGenerationRun({ scope: "missing", total: 3 });
|
||||
let startBody: unknown;
|
||||
@@ -696,7 +517,7 @@ test("discloses source sampling before starting Generate missing descriptions",
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate missing descriptions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate Missing" }));
|
||||
|
||||
expect(screen.getByText("Generate missing descriptions for Policlinico San Donato?")).toBeVisible();
|
||||
expect(screen.getByRole("note", {
|
||||
@@ -706,7 +527,7 @@ test("discloses source sampling before starting Generate missing descriptions",
|
||||
);
|
||||
expect(startBody).toBeUndefined();
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Generate missing descriptions" }));
|
||||
await user.click(screen.getByRole("button", { name: "Generate Missing" }));
|
||||
await waitFor(() => expect(startBody).toEqual({
|
||||
modelId: "local-qwen",
|
||||
scope: "missing",
|
||||
@@ -715,7 +536,7 @@ test("discloses source sampling before starting Generate missing descriptions",
|
||||
expect(await screen.findByRole("dialog", { name: "Description generation" })).toBeVisible();
|
||||
});
|
||||
|
||||
test("keeps Fleet Generate missing descriptions behind the source-data disclosure", async () => {
|
||||
test("keeps Fleet Generate Missing behind the source-data disclosure", async () => {
|
||||
const user = userEvent.setup();
|
||||
let generationStarts = 0;
|
||||
server.use(
|
||||
@@ -737,7 +558,7 @@ test("keeps Fleet Generate missing descriptions behind the source-data disclosur
|
||||
"generate-missing",
|
||||
);
|
||||
await waitFor(() => expect(screen.getByRole("combobox", {
|
||||
name: "Metadata-generation LLM model",
|
||||
name: "Metadata description model",
|
||||
})).toHaveValue("local-qwen"));
|
||||
await user.click(screen.getByRole("button", { name: "Run action" }));
|
||||
|
||||
@@ -753,7 +574,7 @@ test("keeps Fleet Generate missing descriptions behind the source-data disclosur
|
||||
expect(generationStarts).toBe(0);
|
||||
});
|
||||
|
||||
test("confirms generating all descriptions, supports cancel, and sends the database-wide scope", async () => {
|
||||
test("confirms Generate All replacement, supports cancel, and sends the database-wide scope", async () => {
|
||||
const user = userEvent.setup();
|
||||
const queuedRun = makeDescriptionGenerationRun({ scope: "all", total: 6 });
|
||||
let startBody: unknown;
|
||||
@@ -774,7 +595,7 @@ test("confirms generating all descriptions, supports cancel, and sends the datab
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate all descriptions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate All" }));
|
||||
|
||||
expect(screen.getByText("Replace generated descriptions for Policlinico San Donato?")).toBeVisible();
|
||||
expect(screen.getByText(/existing generated descriptions for eligible tables and columns will be replaced/i)).toBeVisible();
|
||||
@@ -788,8 +609,8 @@ test("confirms generating all descriptions, supports cancel, and sends the datab
|
||||
expect(startBody).toBeUndefined();
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate all descriptions" }));
|
||||
await user.click(screen.getByRole("button", { name: "Generate all descriptions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate All" }));
|
||||
await user.click(screen.getByRole("button", { name: "Generate All" }));
|
||||
|
||||
await waitFor(() => expect(startBody).toEqual({
|
||||
modelId: "local-qwen",
|
||||
@@ -815,8 +636,8 @@ test("keeps the database selected and safely explains when no descriptions are e
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate missing descriptions" }));
|
||||
await user.click(screen.getByRole("button", { name: "Generate missing descriptions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate Missing" }));
|
||||
await user.click(screen.getByRole("button", { name: "Generate Missing" }));
|
||||
|
||||
expect(await screen.findByText(
|
||||
"No eligible catalog tables or columns need description generation.",
|
||||
@@ -876,10 +697,10 @@ test.each([
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", {
|
||||
name: scope === "all" ? "Generate all descriptions" : "Generate missing descriptions",
|
||||
name: scope === "all" ? "Generate All" : "Generate Missing",
|
||||
}));
|
||||
await user.click(screen.getByRole("button", {
|
||||
name: scope === "all" ? "Generate all descriptions" : "Generate missing descriptions",
|
||||
name: scope === "all" ? "Generate All" : "Generate Missing",
|
||||
}));
|
||||
|
||||
expect(await screen.findByRole("heading", {
|
||||
@@ -954,9 +775,9 @@ test.each([
|
||||
}
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate all descriptions" }))
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate All" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
expect(screen.getByRole("menuitem", { name: "Generate missing descriptions" }))
|
||||
expect(screen.getByRole("menuitem", { name: "Generate Missing" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
});
|
||||
|
||||
@@ -983,16 +804,16 @@ test("disables database-wide generation while a description generation is active
|
||||
let databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate missing descriptions" }));
|
||||
await user.click(screen.getByRole("button", { name: "Generate missing descriptions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate Missing" }));
|
||||
await user.click(screen.getByRole("button", { name: "Generate Missing" }));
|
||||
expect(await screen.findByRole("dialog", { name: "Description generation" })).toBeVisible();
|
||||
|
||||
databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate all descriptions" }))
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate All" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
expect(screen.getByRole("menuitem", { name: "Generate missing descriptions" }))
|
||||
expect(screen.getByRole("menuitem", { name: "Generate Missing" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
});
|
||||
|
||||
@@ -1045,16 +866,15 @@ test("selected database Actions confirms and deletes catalog tables for the full
|
||||
],
|
||||
target: "tables",
|
||||
}));
|
||||
expect(screen.getByText("2 selected")).toBeVisible();
|
||||
await user.click(screen.getByRole("button", { name: "Clear" }));
|
||||
expect(screen.queryByText("2 selected")).not.toBeInTheDocument();
|
||||
await waitFor(() => expect(screen.queryByText("2 selected")).not.toBeInTheDocument());
|
||||
await waitFor(() => expect(screen.getByRole("textbox", { name: "Search databases" })).toHaveFocus());
|
||||
});
|
||||
|
||||
test("presents completed database synchronization steps as success and skips unneeded confirmation", async () => {
|
||||
test("presents completed synchronization steps as success and skips unneeded confirmation", async () => {
|
||||
const user = userEvent.setup();
|
||||
const run = {
|
||||
...makeSyncRun("all"),
|
||||
counts: { tables: 163, columns: 2_275, relationships: 18 },
|
||||
...makeSyncRun("columns"),
|
||||
counts: { tables: 163, columns: 2_275 },
|
||||
};
|
||||
registerCompletedSyncRun(run);
|
||||
server.use(http.post("/api/catalog/databases/:databaseId/sync-runs", () => (
|
||||
@@ -1067,7 +887,7 @@ test("presents completed database synchronization steps as success and skips unn
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Synchronize all" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Synchronize all columns" }));
|
||||
|
||||
expect(await screen.findByRole("heading", { name: "Succeeded" })).toBeVisible();
|
||||
const steps = screen.getByRole("list", { name: "Synchronization steps" });
|
||||
@@ -1084,7 +904,7 @@ test("presents completed database synchronization steps as success and skips unn
|
||||
expect(steps.querySelectorAll(".text-primary")).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("database Overview exposes the database synchronization scopes", async () => {
|
||||
test("database Overview exposes every synchronization scope", async () => {
|
||||
const user = userEvent.setup();
|
||||
let startBody: unknown;
|
||||
const run = makeSyncRun("relationships");
|
||||
@@ -1131,7 +951,7 @@ test("replaces the list with View and returns focus to the originating action",
|
||||
expect(screen.getByRole("button", { name: "View Policlinico San Donato" })).toBeVisible();
|
||||
});
|
||||
|
||||
test("configures the catalog for the selected workspace and locks that workspace", async () => {
|
||||
test("opens an unconfigured row as Edit while creating its first saved configuration", async () => {
|
||||
const user = userEvent.setup();
|
||||
let postBody: unknown;
|
||||
const saved = makeDatabase({
|
||||
@@ -1151,16 +971,16 @@ test("configures the catalog for the selected workspace and locks that workspace
|
||||
);
|
||||
renderPage({ rows: () => rows });
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "Configure catalog for Research laboratory" }));
|
||||
await user.click(await screen.findByRole("button", { name: "Edit Research laboratory" }));
|
||||
|
||||
expect(screen.getByRole("heading", { name: "Configure catalog" })).toBeVisible();
|
||||
expect(screen.getByLabelText("Workspace")).toHaveAttribute("readonly");
|
||||
expect(screen.getByRole("heading", { name: "Edit database" })).toBeVisible();
|
||||
expect(screen.getByLabelText("Workspace")).toBeDisabled();
|
||||
await user.selectOptions(screen.getByLabelText("Transport"), "rest_api");
|
||||
await user.type(screen.getByLabelText("Base URL"), "https://psd.example/api");
|
||||
expect(screen.getByLabelText("Diagnostic endpoint")).toHaveValue("/health");
|
||||
expect(screen.getByLabelText("Diagnostic endpoint")).toHaveAttribute("readonly");
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Save catalog configuration" }));
|
||||
await user.click(screen.getByRole("button", { name: "Save database" }));
|
||||
|
||||
await waitFor(() => expect(postBody).toMatchObject({
|
||||
workspaceId: "lab",
|
||||
@@ -1169,7 +989,7 @@ test("configures the catalog for the selected workspace and locks that workspace
|
||||
expect(await screen.findByRole("heading", { name: "Edit database" })).toBeVisible();
|
||||
});
|
||||
|
||||
test("has no global database creation path and scopes configuration to the chosen row", async () => {
|
||||
test("global Add offers only unconfigured workspaces and lets the operator choose one", async () => {
|
||||
const user = userEvent.setup();
|
||||
const second = makeDatabase({
|
||||
...unconfigured,
|
||||
@@ -1179,13 +999,14 @@ test("has no global database creation path and scopes configuration to the chose
|
||||
});
|
||||
renderPage({ rows: [makeDatabase(), unconfigured, second] });
|
||||
|
||||
expect(screen.queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
|
||||
await user.click(await screen.findByRole("button", { name: "Configure catalog for Radiology" }));
|
||||
await user.click(await screen.findByRole("button", { name: "Add database" }));
|
||||
|
||||
expect(screen.getByRole("heading", { name: "Configure catalog" })).toBeVisible();
|
||||
const workspace = screen.getByLabelText("Workspace");
|
||||
expect(workspace).toHaveAttribute("readonly");
|
||||
expect(workspace).toHaveValue("Radiology");
|
||||
expect(screen.getByRole("heading", { name: "Add database" })).toBeVisible();
|
||||
expect(screen.getByRole("button", { name: "Add database" })).toBeVisible();
|
||||
const selector = screen.getByLabelText("Workspace");
|
||||
expect(selector).toBeEnabled();
|
||||
expect(screen.queryByRole("option", { name: "Policlinico San Donato" })).not.toBeInTheDocument();
|
||||
await user.selectOptions(selector, "radiology");
|
||||
expect(screen.getByDisplayValue("radiology_dwh")).toBeVisible();
|
||||
});
|
||||
|
||||
@@ -1243,8 +1064,7 @@ test("uses an in-page destructive form and returns the YAML workspace to Not con
|
||||
|
||||
await waitFor(() => expect(deleteCalled).toBe(true));
|
||||
expect(deleteVersion).toBe("3");
|
||||
expect(await screen.findByRole("button", { name: "Configure catalog for Policlinico San Donato" })).toBeEnabled();
|
||||
expect(screen.queryByRole("button", { name: "Delete Policlinico San Donato" })).not.toBeInTheDocument();
|
||||
expect(await screen.findByRole("button", { name: "Delete Policlinico San Donato" })).toBeDisabled();
|
||||
expect(screen.getByText("Not configured")).toBeVisible();
|
||||
});
|
||||
|
||||
@@ -1472,7 +1292,7 @@ test("filters catalog tables as the operator types", async () => {
|
||||
});
|
||||
});
|
||||
|
||||
test("selected table Actions expose table commands and send synchronization ids", async () => {
|
||||
test("selected table Actions exposes cleanup commands and sends synchronization ids", async () => {
|
||||
const user = userEvent.setup();
|
||||
let startBody: unknown;
|
||||
const run = makeSyncRun("columns", [patientsTable.id]);
|
||||
@@ -1494,11 +1314,10 @@ test("selected table Actions expose table commands and send synchronization ids"
|
||||
await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
const synchronizeColumns = await screen.findByRole("menuitem", { name: "Synchronize columns for selected tables" });
|
||||
const synchronizeColumns = await screen.findByRole("menuitem", { name: "Synchronize columns" });
|
||||
expect(synchronizeColumns).toBeEnabled();
|
||||
expect(screen.getByRole("menuitem", { name: "Synchronize database tables" })).toBeEnabled();
|
||||
expect(screen.getByRole("menuitem", { name: "Remove synchronized columns" })).toBeEnabled();
|
||||
expect(screen.queryByRole("menuitem", { name: "Delete all relationships" })).not.toBeInTheDocument();
|
||||
expect(screen.getByRole("menuitem", { name: "Delete all columns" })).toBeEnabled();
|
||||
expect(screen.getByRole("menuitem", { name: "Delete all relationships" })).toBeEnabled();
|
||||
await user.click(synchronizeColumns);
|
||||
|
||||
await waitFor(() => expect(startBody).toEqual({
|
||||
@@ -1547,10 +1366,10 @@ test("starts description generation for multiple selected tables", async () => {
|
||||
await user.click(within(await screen.findByRole("row", { name: /visits/ })).getByRole("checkbox"));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
|
||||
const generate = await screen.findByRole("menuitem", { name: "Generate table description" });
|
||||
const generate = await screen.findByRole("menuitem", { name: "Generate descriptions" });
|
||||
expect(generate).toBeEnabled();
|
||||
expect(screen.getByRole("menuitem", {
|
||||
name: "Copy generated descriptions to Description",
|
||||
name: "Move generated description to Description",
|
||||
})).toBeEnabled();
|
||||
await user.click(generate);
|
||||
|
||||
@@ -1566,37 +1385,6 @@ test("starts description generation for multiple selected tables", async () => {
|
||||
expect(screen.getByText("2 selected")).toBeVisible();
|
||||
});
|
||||
|
||||
test("generates descriptions for every column in selected tables", async () => {
|
||||
const user = userEvent.setup();
|
||||
const queuedRun = makeDescriptionGenerationRun({ scope: "selected_columns", total: 1 });
|
||||
let startBody: unknown;
|
||||
server.use(
|
||||
http.get("/api/catalog/metadata-generation/models", () => HttpResponse.json({ models: [{ id: "local-qwen", label: "Local Qwen" }], default: "local-qwen" })),
|
||||
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([patientsTable])),
|
||||
http.get(`/api/catalog/databases/:databaseId/tables/${patientsTable.id}/columns`, () => HttpResponse.json([patientIdColumn])),
|
||||
http.post("/api/catalog/databases/:databaseId/description-generation-runs", async ({ request }) => {
|
||||
startBody = await request.json();
|
||||
return HttpResponse.json(queuedRun, { status: 202 });
|
||||
}),
|
||||
http.get("/api/catalog/description-generation-runs/:runId", () => HttpResponse.json(queuedRun)),
|
||||
http.get("/api/catalog/description-generation-runs/:runId/events-list", () => HttpResponse.json([])),
|
||||
);
|
||||
renderPage({ rows: [makeDatabase({ connectionStatus: "reachable", testedVersion: 3 })] });
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" }));
|
||||
await user.click(screen.getByRole("tab", { name: "Tables" }));
|
||||
await user.click(within(await screen.findByRole("row", { name: /patients/ })).getByRole("checkbox"));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate descriptions for all columns" }));
|
||||
|
||||
await waitFor(() => expect(startBody).toEqual({
|
||||
modelId: "local-qwen",
|
||||
scope: "selected_columns",
|
||||
targetIds: [patientIdColumn.id],
|
||||
}));
|
||||
expect(await screen.findByText("Column description generation started for 1 column")).toBeVisible();
|
||||
});
|
||||
|
||||
test("observes an active run from another browser and reopens a terminal run from history", async () => {
|
||||
const user = userEvent.setup();
|
||||
const activeRun = makeDescriptionGenerationRun({
|
||||
@@ -1629,9 +1417,9 @@ test("observes an active run from another browser and reopens a terminal run fro
|
||||
renderPage();
|
||||
|
||||
const observe = await screen.findByRole("button", {
|
||||
name: "View description generation history",
|
||||
name: "Run descriptions generation history",
|
||||
});
|
||||
expect(observe).toHaveTextContent("View description generation history");
|
||||
expect(observe).toHaveTextContent("Run descriptions generation history");
|
||||
expect(requestedLimit).toBe("50");
|
||||
await user.click(observe);
|
||||
|
||||
@@ -1655,9 +1443,9 @@ test("keeps the sensitive suggestion history label stable while showing active s
|
||||
renderPage();
|
||||
|
||||
const historyButton = await screen.findByRole("button", {
|
||||
name: "View sensitive suggestion history",
|
||||
name: "Run sensitive suggestions history",
|
||||
});
|
||||
expect(historyButton).toHaveTextContent("View sensitive suggestion history");
|
||||
expect(historyButton).toHaveTextContent("Run sensitive suggestions history");
|
||||
await waitFor(() => expect(historyButton).toHaveAttribute(
|
||||
"title",
|
||||
"Sensitive suggestion generation is active",
|
||||
@@ -1693,7 +1481,7 @@ test.each([
|
||||
client.setQueryData(unrelatedColumnKey, []);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View description generation history",
|
||||
name: "Run descriptions generation history",
|
||||
}));
|
||||
|
||||
await waitFor(() => expect(client.getQueryState(tableKey)?.isInvalidated).toBe(true));
|
||||
@@ -1732,7 +1520,7 @@ test("keeps selected tables when description generation cannot start", async ()
|
||||
await user.click(within(await screen.findByRole("row", { name: /patients/ })).getByRole("checkbox"));
|
||||
await user.click(within(await screen.findByRole("row", { name: /visits/ })).getByRole("checkbox"));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate table description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate descriptions" }));
|
||||
|
||||
expect(await screen.findByText("A description generation run is already active.")).toBeVisible();
|
||||
expect(screen.getByText("2 selected")).toBeVisible();
|
||||
@@ -1802,7 +1590,7 @@ test("refreshes Catalog Tables and Catalog Columns after table generation comple
|
||||
await user.click(within(await screen.findByRole("row", { name: /patients/ })).getByRole("checkbox"));
|
||||
await user.click(within(await screen.findByRole("row", { name: /visits/ })).getByRole("checkbox"));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate table description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate descriptions" }));
|
||||
|
||||
expect(await screen.findByRole("heading", { name: "Completed" })).toBeVisible();
|
||||
await waitFor(() => expect(tableReads).toBe(2));
|
||||
@@ -1858,7 +1646,7 @@ test("moves selected generated table descriptions and reports copied and skipped
|
||||
await user.click(within(visitsRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", {
|
||||
name: "Copy generated descriptions to Description",
|
||||
name: "Move generated description to Description",
|
||||
}));
|
||||
|
||||
await waitFor(() => expect(consolidationBody).toEqual({
|
||||
@@ -1946,9 +1734,8 @@ test("selects columns, moves generated descriptions, and refreshes only the affe
|
||||
await user.click(within(idRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(within(nameRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
expect(await screen.findByRole("menuitem", { name: "Synchronize columns for this table" })).toBeEnabled();
|
||||
await user.click(await screen.findByRole("menuitem", {
|
||||
name: "Copy generated descriptions to Description",
|
||||
name: "Move generated description to Description",
|
||||
}));
|
||||
|
||||
await waitFor(() => expect(consolidationBody).toEqual({
|
||||
@@ -1991,18 +1778,12 @@ test("starts one selected column with the configured default model", async () =>
|
||||
await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" }));
|
||||
await user.click(screen.getByRole("tab", { name: "Tables" }));
|
||||
await user.click(await screen.findByRole("button", { name: "View columns for patients" }));
|
||||
expect(screen.queryByRole("button", { name: "Sync columns" })).not.toBeInTheDocument();
|
||||
expect(screen.getByRole("columnheader", { name: /Select/i })).toBeVisible();
|
||||
expect(await screen.findByRole("checkbox", { name: "Sensitive data for id" })).toHaveAttribute(
|
||||
"title",
|
||||
"Mark this column as sensitive; this does not select it for actions.",
|
||||
);
|
||||
const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ }))
|
||||
.find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }));
|
||||
expect(columnRow).toBeDefined();
|
||||
await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate column description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate description" }));
|
||||
|
||||
await waitFor(() => expect(startBody).toEqual({
|
||||
modelId: "local-qwen",
|
||||
@@ -2356,7 +2137,7 @@ test("polls a description run and renders its events in sequence order", async (
|
||||
.find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate column description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate description" }));
|
||||
|
||||
const drawer = await screen.findByRole("dialog", { name: "Description generation" });
|
||||
expect(within(drawer).getByText(/local-qwen/)).toBeVisible();
|
||||
@@ -2438,7 +2219,7 @@ test("refreshes Catalog Tables and Catalog Columns after column generation compl
|
||||
.find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate column description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate description" }));
|
||||
|
||||
expect(await screen.findByRole("heading", { name: "Completed" })).toBeVisible();
|
||||
await waitFor(() => expect(columnReads).toBe(2));
|
||||
@@ -2471,7 +2252,7 @@ test("keeps the selected column and shows a safe message when generation cannot
|
||||
.find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate column description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate description" }));
|
||||
|
||||
expect(await screen.findByText("A description generation run is already active.")).toBeVisible();
|
||||
expect(screen.getByText("1 selected")).toBeVisible();
|
||||
@@ -2538,7 +2319,7 @@ test("shows a basic failed run without exposing private model or provider fields
|
||||
.find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate column description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate description" }));
|
||||
|
||||
const drawer = await screen.findByRole("dialog", { name: "Description generation" });
|
||||
expect(await within(drawer).findByRole("heading", { name: "Failed" })).toBeVisible();
|
||||
@@ -2592,18 +2373,18 @@ test("offers generation and consolidation for multiple selected columns", async
|
||||
.find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(within(idRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate column description" })).toBeEnabled();
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate description" })).toBeEnabled();
|
||||
expect(screen.getByRole("menuitem", {
|
||||
name: "Copy generated descriptions to Description",
|
||||
name: "Move generated description to Description",
|
||||
})).toBeEnabled();
|
||||
await user.keyboard("{Escape}");
|
||||
|
||||
await user.click(within(nameRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
const generate = await screen.findByRole("menuitem", { name: "Generate column descriptions" });
|
||||
const generate = await screen.findByRole("menuitem", { name: "Generate descriptions" });
|
||||
expect(generate).toBeEnabled();
|
||||
expect(await screen.findByRole("menuitem", {
|
||||
name: "Copy generated descriptions to Description",
|
||||
name: "Move generated description to Description",
|
||||
})).toBeEnabled();
|
||||
await user.click(generate);
|
||||
|
||||
@@ -2639,7 +2420,7 @@ test.each([
|
||||
const tableRow = await screen.findByRole("row", { name: /patients/ });
|
||||
await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate descriptions for all columns" }))
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate description" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
await user.keyboard("{Escape}");
|
||||
await user.click(screen.getByRole("button", { name: "Clear" }));
|
||||
@@ -2649,7 +2430,7 @@ test.each([
|
||||
await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate column description" }))
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate description" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
});
|
||||
|
||||
@@ -2682,11 +2463,11 @@ test("disables another generation start while the selected-column run is active"
|
||||
.find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate column description" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Generate description" }));
|
||||
expect(await screen.findByRole("dialog", { name: "Description generation" })).toBeVisible();
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate column description" }))
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate description" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
expect(screen.getByText("1 selected")).toBeVisible();
|
||||
|
||||
@@ -2694,7 +2475,7 @@ test("disables another generation start while the selected-column run is active"
|
||||
const tableRow = await screen.findByRole("row", { name: /patients/ });
|
||||
await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate descriptions for all columns" }))
|
||||
expect(await screen.findByRole("menuitem", { name: "Generate description" }))
|
||||
.toHaveAttribute("aria-disabled", "true");
|
||||
});
|
||||
|
||||
@@ -2722,7 +2503,7 @@ test("disables selected description consolidation without database.manage", asyn
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
|
||||
expect(await screen.findByRole("menuitem", {
|
||||
name: "Copy generated descriptions to Description",
|
||||
name: "Move generated description to Description",
|
||||
})).toHaveAttribute("aria-disabled", "true");
|
||||
expect(consolidationCalls).toBe(0);
|
||||
});
|
||||
@@ -2750,7 +2531,7 @@ test("shows an operation conflict without clearing selected descriptions or refr
|
||||
await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", {
|
||||
name: "Copy generated descriptions to Description",
|
||||
name: "Move generated description to Description",
|
||||
}));
|
||||
|
||||
expect(await screen.findByText("A database operation is already in progress.")).toBeVisible();
|
||||
@@ -2758,22 +2539,52 @@ test("shows an operation conflict without clearing selected descriptions or refr
|
||||
expect(tableReads).toBe(1);
|
||||
});
|
||||
|
||||
test("selected table Actions do not expose relationship cleanup", async () => {
|
||||
test("selected table Actions confirms and deletes incoming and outgoing relationships", async () => {
|
||||
const user = userEvent.setup();
|
||||
let cleanupBody: unknown;
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([patientsTable])),
|
||||
http.post("/api/catalog/databases/:databaseId/tables/metadata-cleanup", async ({ request }) => {
|
||||
cleanupBody = await request.json();
|
||||
return HttpResponse.json({ tables: 0, columns: 0, relationships: 2 });
|
||||
}),
|
||||
);
|
||||
renderPage({
|
||||
const { client } = renderPage({
|
||||
rows: [makeDatabase({ connectionStatus: "reachable", testedVersion: 3 })],
|
||||
});
|
||||
const unselectedColumnQuery = [
|
||||
"catalog-columns",
|
||||
patientsTable.databaseId,
|
||||
"bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb",
|
||||
] as const;
|
||||
client.setQueryData(unselectedColumnQuery, []);
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" }));
|
||||
await user.click(screen.getByRole("tab", { name: "Tables" }));
|
||||
const tableRow = await screen.findByRole("row", { name: /patients/ });
|
||||
await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
expect(screen.queryByRole("menuitem", { name: "Delete all relationships" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("menuitem", { name: "Clear catalog relationships" })).not.toBeInTheDocument();
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Delete all relationships" }));
|
||||
|
||||
expect(screen.getByText("Delete all relationships for 1 table?")).toBeVisible();
|
||||
expect(screen.getByText(/incoming and outgoing relationships/i)).toBeVisible();
|
||||
expect(cleanupBody).toBeUndefined();
|
||||
|
||||
await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
expect(screen.queryByText("Delete all relationships for 1 table?")).not.toBeInTheDocument();
|
||||
expect(cleanupBody).toBeUndefined();
|
||||
await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Delete all relationships" }));
|
||||
await user.click(screen.getByRole("button", { name: "Delete relationships" }));
|
||||
|
||||
await waitFor(() => expect(cleanupBody).toEqual({
|
||||
tableIds: [patientsTable.id],
|
||||
target: "relationships",
|
||||
}));
|
||||
await waitFor(() => expect(screen.queryByText("1 selected")).not.toBeInTheDocument());
|
||||
expect(client.getQueryState(unselectedColumnQuery)?.isInvalidated).toBe(true);
|
||||
await waitFor(() => expect(screen.getByRole("textbox", { name: "Search tables" })).toHaveFocus());
|
||||
});
|
||||
|
||||
test("navigates purely from a database to its tables and edits review metadata", async () => {
|
||||
@@ -2866,8 +2677,6 @@ test("navigates from a table to columns and from the database to physical relati
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{ position: 1, sourceColumnId: "1", sourceColumnName: "patient_id", targetColumnId: idColumn.id, targetColumnName: "id" }],
|
||||
origin: "physical",
|
||||
status: "active",
|
||||
lastSyncedDatabaseVersion: 3,
|
||||
lastSyncedAt: "2026-08-27T10:00:00Z",
|
||||
createdAt: "2026-08-27T10:00:00Z",
|
||||
@@ -2946,10 +2755,9 @@ test("opens the durable job drawer and confirms its exact destructive plan", asy
|
||||
rows: [makeDatabase({ connectionStatus: "reachable", testedVersion: 3 })],
|
||||
});
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i }));
|
||||
await user.click(screen.getByRole("button", { name: "Actions" }));
|
||||
await user.click(await screen.findByRole("menuitem", { name: "Synchronize tables" }));
|
||||
await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" }));
|
||||
await user.click(screen.getByRole("tab", { name: "Tables" }));
|
||||
await user.click(await screen.findByRole("button", { name: "Sync tables" }));
|
||||
|
||||
const synchronizationDrawer = await screen.findByRole("dialog", {
|
||||
name: "Schema synchronization",
|
||||
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
useState,
|
||||
} from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { History, RefreshCw } from "lucide-react";
|
||||
import { History, Plus, RefreshCw } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import { Button } from "../components/ui/button";
|
||||
import { ApiError, apiErrorMessage } from "../api/client";
|
||||
@@ -80,20 +80,6 @@ const SYNC_STARTED_MESSAGES: Record<CatalogSyncScope, string> = {
|
||||
relationships: "Relationship synchronization started",
|
||||
all: "Full schema synchronization started",
|
||||
};
|
||||
const DATABASE_SCREEN_STORAGE_KEY = "thothii:database-management-screen";
|
||||
|
||||
function readPersistedDatabaseScreen(): DatabaseScreen {
|
||||
if (import.meta.env.MODE === "test") return { kind: "list" };
|
||||
try {
|
||||
const parsed = JSON.parse(window.localStorage.getItem(DATABASE_SCREEN_STORAGE_KEY) ?? "null") as { kind?: unknown; workspaceId?: unknown } | null;
|
||||
if ((parsed?.kind === "tables" || parsed?.kind === "relationships") && typeof parsed.workspaceId === "string") {
|
||||
return { kind: parsed.kind, workspaceId: parsed.workspaceId };
|
||||
}
|
||||
} catch {
|
||||
// Ignore malformed persisted navigation and return to the safe list view.
|
||||
}
|
||||
return { kind: "list" };
|
||||
}
|
||||
|
||||
interface Props {
|
||||
canManage: boolean;
|
||||
@@ -170,13 +156,12 @@ export function DatabaseManagementPage({
|
||||
(run) => run.status === "running",
|
||||
);
|
||||
|
||||
const [screen, setScreen] = useState<DatabaseScreen>(readPersistedDatabaseScreen);
|
||||
const [screen, setScreen] = useState<DatabaseScreen>({ kind: "list" });
|
||||
const [selectedMetadataModel, setSelectedMetadataModel] = useState("");
|
||||
const [draft, setDraft] = useState<DatabaseFormDraft | null>(null);
|
||||
const [baseline, setBaseline] = useState("");
|
||||
const [formSource, setFormSource] = useState<FormSource | null>(null);
|
||||
const [search, setSearch] = useState("");
|
||||
const [selectedDatabaseId, setSelectedDatabaseId] = useState<string | null>(null);
|
||||
const [busyAction, setBusyAction] = useState<DatabaseBusyAction>(null);
|
||||
const [stale, setStale] = useState(false);
|
||||
const [staleBannerOpen, setStaleBannerOpen] = useState(true);
|
||||
@@ -186,11 +171,6 @@ export function DatabaseManagementPage({
|
||||
busy: false,
|
||||
});
|
||||
const [tableNestedNavigationActive, setTableNestedNavigationActive] = useState(false);
|
||||
const [tableNestedNavigationBack, setTableNestedNavigationBack] = useState<(() => void) | null>(null);
|
||||
const handleTableNestedNavigationChange = useCallback((active: boolean, onBack?: () => void) => {
|
||||
setTableNestedNavigationActive(active);
|
||||
setTableNestedNavigationBack(() => (active ? onBack ?? null : null));
|
||||
}, []);
|
||||
const [activeSyncRun, setActiveSyncRun] = useState<CatalogSyncRun | null>(null);
|
||||
const [syncHistoryDatabaseId, setSyncHistoryDatabaseId] = useState<string | null>(null);
|
||||
const [syncDrawerOpen, setSyncDrawerOpen] = useState(false);
|
||||
@@ -204,12 +184,6 @@ export function DatabaseManagementPage({
|
||||
suggestions: SensitiveDataSuggestion[];
|
||||
} | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (import.meta.env.MODE === "test") return;
|
||||
const persisted = screen.kind === "tables" || screen.kind === "relationships" ? screen : { kind: "list" };
|
||||
window.localStorage.setItem(DATABASE_SCREEN_STORAGE_KEY, JSON.stringify(persisted));
|
||||
}, [screen]);
|
||||
|
||||
const originRef = useRef<HTMLElement | null>(null);
|
||||
const searchInputRef = useRef<HTMLInputElement>(null);
|
||||
const formHeadingRef = useRef<HTMLHeadingElement>(null);
|
||||
@@ -235,7 +209,11 @@ export function DatabaseManagementPage({
|
||||
() => queryClient.invalidateQueries({ queryKey: ["catalog-metrics"] }),
|
||||
[queryClient],
|
||||
);
|
||||
const editable = screen.kind === "configure" || screen.kind === "edit";
|
||||
const availableWorkspaces = useMemo(
|
||||
() => rows.filter((row) => row.workspaceAvailable && !row.configured),
|
||||
[rows],
|
||||
);
|
||||
const editable = screen.kind === "add" || screen.kind === "edit";
|
||||
const configurationDirty = Boolean(
|
||||
editable
|
||||
&& draft
|
||||
@@ -243,9 +221,8 @@ export function DatabaseManagementPage({
|
||||
);
|
||||
const dirty = Boolean(editable && draft && (configurationDirty || hasSecretChanges(draft)));
|
||||
const busy = busyAction !== null;
|
||||
const catalogChildVisible = screen.kind === "tables" || screen.kind === "relationships";
|
||||
const navigationDirty = catalogChildVisible ? tablesNavigationState.dirty : dirty;
|
||||
const navigationBusy = catalogChildVisible ? tablesNavigationState.busy : busy;
|
||||
const navigationDirty = screen.kind === "tables" ? tablesNavigationState.dirty : dirty;
|
||||
const navigationBusy = screen.kind === "tables" ? tablesNavigationState.busy : busy;
|
||||
|
||||
useEffect(() => {
|
||||
onNavigationStateChange?.({ dirty: navigationDirty, busy: navigationBusy });
|
||||
@@ -286,14 +263,9 @@ export function DatabaseManagementPage({
|
||||
}, [queryClient]);
|
||||
|
||||
const restoreListFocus = useCallback(() => {
|
||||
const originLabel = originRef.current?.getAttribute("aria-label");
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) {
|
||||
originRef.current.focus();
|
||||
} else if (originLabel) {
|
||||
const matchingAction = [...document.querySelectorAll<HTMLButtonElement>("button")]
|
||||
.find((button) => button.getAttribute("aria-label") === originLabel);
|
||||
(matchingAction ?? searchInputRef.current)?.focus();
|
||||
} else {
|
||||
searchInputRef.current?.focus();
|
||||
}
|
||||
@@ -310,7 +282,6 @@ export function DatabaseManagementPage({
|
||||
setPartialSecretFailure(null);
|
||||
setTablesNavigationState({ dirty: false, busy: false });
|
||||
setTableNestedNavigationActive(false);
|
||||
setTableNestedNavigationBack(null);
|
||||
restoreListFocus();
|
||||
}, [restoreListFocus]);
|
||||
|
||||
@@ -336,6 +307,7 @@ export function DatabaseManagementPage({
|
||||
mode: DatabaseFormMode,
|
||||
row: CatalogDatabase,
|
||||
origin: HTMLElement,
|
||||
workspaceLocked = false,
|
||||
) => {
|
||||
const nextDraft = draftFrom(row);
|
||||
originRef.current = origin;
|
||||
@@ -348,6 +320,7 @@ export function DatabaseManagementPage({
|
||||
setScreen({
|
||||
kind: mode,
|
||||
workspaceId: row.workspaceId,
|
||||
...(mode === "add" ? { workspaceLocked } : {}),
|
||||
});
|
||||
}, []);
|
||||
|
||||
@@ -356,7 +329,7 @@ export function DatabaseManagementPage({
|
||||
}, [openForm]);
|
||||
|
||||
const editRow = useCallback((row: CatalogDatabase, origin: HTMLButtonElement) => {
|
||||
openForm(row.configured ? "edit" : "configure", row, origin);
|
||||
openForm(row.configured ? "edit" : "add", row, origin, !row.configured);
|
||||
}, [openForm]);
|
||||
|
||||
const deleteRow = useCallback((row: CatalogDatabase, origin: HTMLButtonElement) => {
|
||||
@@ -373,21 +346,17 @@ export function DatabaseManagementPage({
|
||||
setPartialSecretFailure(null);
|
||||
setTablesNavigationState({ dirty: false, busy: false });
|
||||
setTableNestedNavigationActive(false);
|
||||
setTableNestedNavigationBack(null);
|
||||
setScreen({ kind: "tables", workspaceId: row.workspaceId });
|
||||
}, []);
|
||||
|
||||
const openRelationships = useCallback((row: CatalogDatabase, origin?: HTMLElement) => {
|
||||
const openRelationships = useCallback((row: CatalogDatabase) => {
|
||||
if (!row.configured || !row.id) return;
|
||||
if (origin) originRef.current = origin;
|
||||
setDraft(null);
|
||||
setBaseline("");
|
||||
setFormSource(null);
|
||||
setStale(false);
|
||||
setPartialSecretFailure(null);
|
||||
setTablesNavigationState({ dirty: false, busy: false });
|
||||
setTableNestedNavigationActive(false);
|
||||
setTableNestedNavigationBack(null);
|
||||
setScreen({ kind: "relationships", workspaceId: row.workspaceId });
|
||||
}, []);
|
||||
|
||||
@@ -403,6 +372,25 @@ export function DatabaseManagementPage({
|
||||
setScreen({ kind: "view", workspaceId: row.workspaceId });
|
||||
}, []);
|
||||
|
||||
const addDatabase = useCallback((origin: HTMLElement) => {
|
||||
const workspace = availableWorkspaces[0];
|
||||
if (!workspace || !canManage) return;
|
||||
openForm("add", workspace, origin, false);
|
||||
}, [availableWorkspaces, canManage, openForm]);
|
||||
|
||||
const changeWorkspace = useCallback((workspaceId: string) => {
|
||||
const workspace = availableWorkspaces.find((row) => row.workspaceId === workspaceId);
|
||||
if (!workspace) return;
|
||||
if (dirty && !window.confirm("Discard changes and choose another workspace?")) return;
|
||||
const nextDraft = draftFrom(workspace);
|
||||
setDraft(nextDraft);
|
||||
setBaseline(configurationFingerprint(nextDraft));
|
||||
setFormSource({ id: workspace.id, version: workspace.version });
|
||||
setStale(false);
|
||||
setPartialSecretFailure(null);
|
||||
setScreen({ kind: "add", workspaceId, workspaceLocked: false });
|
||||
}, [availableWorkspaces, dirty]);
|
||||
|
||||
const changeField = useCallback((field: "databaseName" | "schema", value: string) => {
|
||||
setDraft((current) => current ? { ...current, [field]: value } : current);
|
||||
}, []);
|
||||
@@ -636,7 +624,7 @@ export function DatabaseManagementPage({
|
||||
showList();
|
||||
toast.info("This database configuration has already been deleted");
|
||||
} else if (screen.kind === "edit" && !latest.configured) {
|
||||
setScreen({ kind: "configure", workspaceId: latest.workspaceId });
|
||||
setScreen({ kind: "add", workspaceId: latest.workspaceId, workspaceLocked: true });
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
@@ -681,40 +669,22 @@ export function DatabaseManagementPage({
|
||||
}, [queryClient]);
|
||||
|
||||
const openDescriptionGenerationHistory = useCallback(() => {
|
||||
const scopedRuns = activeRow ? descriptionGenerationRuns.filter((item) => item.databaseId === activeRow.id) : descriptionGenerationRuns;
|
||||
const scopedActiveRun = observedActiveDescriptionGenerationRun
|
||||
&& (!activeRow || observedActiveDescriptionGenerationRun.databaseId === activeRow.id)
|
||||
? observedActiveDescriptionGenerationRun
|
||||
: undefined;
|
||||
const scopedSelectedRun = activeDescriptionGenerationRun
|
||||
&& (!activeRow || activeDescriptionGenerationRun.databaseId === activeRow.id)
|
||||
? activeDescriptionGenerationRun
|
||||
: undefined;
|
||||
const run = scopedActiveRun
|
||||
?? scopedRuns[0]
|
||||
?? scopedSelectedRun;
|
||||
const run = observedActiveDescriptionGenerationRun
|
||||
?? descriptionGenerationRuns[0]
|
||||
?? activeDescriptionGenerationRun;
|
||||
if (run) setActiveDescriptionGenerationRun(run);
|
||||
setSensitiveDataSuggestionHistoryDrawerOpen(false);
|
||||
setDescriptionGenerationDrawerOpen(true);
|
||||
}, [activeDescriptionGenerationRun, activeRow, descriptionGenerationRuns, observedActiveDescriptionGenerationRun]);
|
||||
}, [activeDescriptionGenerationRun, descriptionGenerationRuns, observedActiveDescriptionGenerationRun]);
|
||||
|
||||
const openSensitiveDataSuggestionHistory = useCallback(() => {
|
||||
const scopedRuns = activeRow ? sensitiveDataSuggestionRuns.filter((item) => item.databaseId === activeRow.id) : sensitiveDataSuggestionRuns;
|
||||
const scopedActiveRun = observedActiveSensitiveDataSuggestionRun
|
||||
&& (!activeRow || observedActiveSensitiveDataSuggestionRun.databaseId === activeRow.id)
|
||||
? observedActiveSensitiveDataSuggestionRun
|
||||
: undefined;
|
||||
const scopedSelectedRun = activeSensitiveDataSuggestionRun
|
||||
&& (!activeRow || activeSensitiveDataSuggestionRun.databaseId === activeRow.id)
|
||||
? activeSensitiveDataSuggestionRun
|
||||
: undefined;
|
||||
const run = scopedActiveRun
|
||||
?? scopedRuns[0]
|
||||
?? scopedSelectedRun;
|
||||
const run = observedActiveSensitiveDataSuggestionRun
|
||||
?? sensitiveDataSuggestionRuns[0]
|
||||
?? activeSensitiveDataSuggestionRun;
|
||||
if (run) setActiveSensitiveDataSuggestionRun(run);
|
||||
setDescriptionGenerationDrawerOpen(false);
|
||||
setSensitiveDataSuggestionHistoryDrawerOpen(true);
|
||||
}, [activeRow, activeSensitiveDataSuggestionRun, observedActiveSensitiveDataSuggestionRun, sensitiveDataSuggestionRuns]);
|
||||
}, [activeSensitiveDataSuggestionRun, observedActiveSensitiveDataSuggestionRun, sensitiveDataSuggestionRuns]);
|
||||
|
||||
const descriptionGenerationTerminated = useCallback(async (run: DescriptionGenerationRun) => {
|
||||
await Promise.all([
|
||||
@@ -779,7 +749,7 @@ export function DatabaseManagementPage({
|
||||
scope,
|
||||
);
|
||||
rememberDescriptionGenerationRun(run);
|
||||
toast.success(`${scope === "all" ? "Generate all descriptions" : "Generate missing descriptions"} started for ${database.workspaceName}`);
|
||||
toast.success(`${scope === "all" ? "Generate All" : "Generate Missing"} started for ${database.workspaceName}`);
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
throw error;
|
||||
@@ -886,11 +856,6 @@ export function DatabaseManagementPage({
|
||||
}
|
||||
}, [invalidateCatalogMetrics, queryClient]);
|
||||
|
||||
const clearActiveCatalogTables = useCallback(async () => {
|
||||
if (!activeRow?.id) return;
|
||||
await deleteSelectedMetadata([activeRow], "tables");
|
||||
}, [activeRow, deleteSelectedMetadata]);
|
||||
|
||||
const syncDatabase = useCallback(async (scope: CatalogSyncScope) => {
|
||||
if (!activeRow?.id || !activeRow.configured) return;
|
||||
try {
|
||||
@@ -937,7 +902,6 @@ export function DatabaseManagementPage({
|
||||
isLoading={metadataModelsQuery.isLoading}
|
||||
selectedModel={selectedMetadataModel}
|
||||
onSelectedModelChange={setSelectedMetadataModel}
|
||||
variant="compact"
|
||||
/>
|
||||
);
|
||||
|
||||
@@ -979,7 +943,6 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<DescriptionGenerationDrawer
|
||||
open={descriptionGenerationDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeDescriptionGenerationRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeDescriptionGenerationRun?.modelId,
|
||||
@@ -990,7 +953,6 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<SensitiveDataSuggestionHistoryDrawer
|
||||
open={sensitiveDataSuggestionHistoryDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeSensitiveDataSuggestionRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeSensitiveDataSuggestionRun?.modelId,
|
||||
@@ -1018,6 +980,7 @@ export function DatabaseManagementPage({
|
||||
eyebrow="Catalog configuration"
|
||||
title={databaseFormTitle(
|
||||
fleetFormMode,
|
||||
fleetFormMode === "add" && Boolean(screen.kind === "add" && screen.workspaceLocked),
|
||||
"drawer",
|
||||
)}
|
||||
description={`${activeRow.workspaceName} · ${activeRow.workspaceId}`}
|
||||
@@ -1030,6 +993,8 @@ export function DatabaseManagementPage({
|
||||
presentation="drawer"
|
||||
mode={fleetFormMode}
|
||||
row={activeRow}
|
||||
availableWorkspaces={availableWorkspaces}
|
||||
workspaceLocked={fleetFormMode === "add" && Boolean(screen.kind === "add" && screen.workspaceLocked)}
|
||||
draft={draft}
|
||||
dirty={dirty}
|
||||
canManage={canManage}
|
||||
@@ -1040,6 +1005,7 @@ export function DatabaseManagementPage({
|
||||
partialSecretFailure={partialSecretFailure}
|
||||
headingRef={formHeadingRef}
|
||||
onBack={backToList}
|
||||
onWorkspaceChange={changeWorkspace}
|
||||
onFieldChange={changeField}
|
||||
onTransportChange={changeTransport}
|
||||
onBindingChange={changeBinding}
|
||||
@@ -1054,8 +1020,6 @@ export function DatabaseManagementPage({
|
||||
onOpenRelationships={() => openRelationships(activeRow)}
|
||||
onSync={(scope) => void syncDatabase(scope)}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
activeSyncRun={currentActiveRun}
|
||||
/>
|
||||
</FleetLedgerDrawer>
|
||||
@@ -1073,13 +1037,13 @@ export function DatabaseManagementPage({
|
||||
onOpenOverview={() => openOverview(activeRow)}
|
||||
onOpenRelationships={() => openRelationships(activeRow)}
|
||||
onNavigationStateChange={setTablesNavigationState}
|
||||
onNestedNavigationChange={handleTableNestedNavigationChange}
|
||||
onNestedNavigationChange={setTableNestedNavigationActive}
|
||||
onRunStarted={rememberSyncRun}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onDescriptionGenerationRunStarted={rememberDescriptionGenerationRun}
|
||||
onSuggestSensitive={suggestActiveDatabaseSensitiveFields}
|
||||
onClearCatalogTables={clearActiveCatalogTables}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
metadataModelControl={fleetMetadataModelControl}
|
||||
/>
|
||||
) : screen.kind === "relationships" && relationshipsVisible ? (
|
||||
<DatabaseRelationships
|
||||
@@ -1091,10 +1055,6 @@ export function DatabaseManagementPage({
|
||||
onOpenTables={() => openTables(activeRow)}
|
||||
onRunStarted={rememberSyncRun}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
onNavigationStateChange={setTablesNavigationState}
|
||||
/>
|
||||
) : isError && rows.length === 0 ? (
|
||||
<div className="grid h-full place-items-center p-6">
|
||||
@@ -1118,19 +1078,17 @@ export function DatabaseManagementPage({
|
||||
onSearchChange={setSearch}
|
||||
onView={viewRow}
|
||||
onOpenTables={(row, origin) => openTables(row, origin)}
|
||||
onOpenRelationships={(row, origin) => openRelationships(row, origin)}
|
||||
onEdit={editRow}
|
||||
onDelete={deleteRow}
|
||||
onOpenSync={openSync}
|
||||
onSelectionChange={(selected) => setSelectedDatabaseId(selected.length === 1 ? selected[0].id ?? null : null)}
|
||||
onTestSelected={testSelected}
|
||||
onSyncSelected={syncSelected}
|
||||
selectedMetadataModel={selectedMetadataModelAvailable ? selectedMetadataModel : null}
|
||||
descriptionGenerationActive={descriptionGenerationActive}
|
||||
onGenerateDescriptions={generateDatabaseDescriptions}
|
||||
onSuggestSensitive={suggestDatabaseSensitiveFields}
|
||||
sensitiveDataSuggestionRuns={sensitiveDataSuggestionRuns}
|
||||
onDeleteMetadataSelected={deleteSelectedMetadata}
|
||||
metadataModelControl={fleetMetadataModelControl}
|
||||
onRefresh={refreshList}
|
||||
/>
|
||||
);
|
||||
@@ -1160,46 +1118,28 @@ export function DatabaseManagementPage({
|
||||
)}
|
||||
actions={(
|
||||
<>
|
||||
{fleetMetadataModelControl}
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="whitespace-nowrap"
|
||||
aria-label="View schema synchronization logs"
|
||||
title="View schema synchronization progress and history"
|
||||
disabled={!canManage || (!selectedDatabaseId && !activeSyncRun?.databaseId)}
|
||||
onClick={() => openSync(selectedDatabaseId ? rows.find((row) => row.id === selectedDatabaseId) : undefined)}
|
||||
>
|
||||
<RefreshCw aria-hidden="true" />
|
||||
{currentActiveRun ? "Schema sync active" : "Schema sync logs"}
|
||||
<Button type="button" variant="outline" size="sm" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Description history" data-fleet-action="secondary" onClick={openDescriptionGenerationHistory}>
|
||||
<History aria-hidden="true" /> Description history
|
||||
{observedActiveDescriptionGenerationRun ? <span className="size-1.5 rounded-full bg-warning" aria-hidden="true" /> : null}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="whitespace-nowrap"
|
||||
aria-label="View AI description generation logs"
|
||||
title="View AI description generation progress and history"
|
||||
disabled={!canManage}
|
||||
onClick={openDescriptionGenerationHistory}
|
||||
>
|
||||
<History aria-hidden="true" />
|
||||
{descriptionGenerationActive ? "AI descriptions active" : "AI description logs"}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="whitespace-nowrap"
|
||||
aria-label="View sensitive analysis progress"
|
||||
title="View sensitive analysis progress and history"
|
||||
disabled={!canManage}
|
||||
onClick={openSensitiveDataSuggestionHistory}
|
||||
>
|
||||
<History aria-hidden="true" />
|
||||
{observedActiveSensitiveDataSuggestionRun ? "Sensitive analysis active" : "Sensitive analysis history"}
|
||||
<Button type="button" variant="outline" size="sm" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Sensitive history" data-fleet-action="secondary" onClick={openSensitiveDataSuggestionHistory}>
|
||||
<History aria-hidden="true" /> Sensitive history
|
||||
{observedActiveSensitiveDataSuggestionRun ? <span className="size-1.5 rounded-full bg-warning" aria-hidden="true" /> : null}
|
||||
</Button>
|
||||
{(screen.kind === "list" || formVisible) ? (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
className="thot-fleet-nav-action thot-fleet-nav-action--below"
|
||||
data-tooltip="Add database"
|
||||
data-fleet-action="primary"
|
||||
disabled={!canManage || availableWorkspaces.length === 0}
|
||||
title={availableWorkspaces.length === 0 ? "Every available workspace is already configured" : undefined}
|
||||
onClick={(event) => addDatabase(event.currentTarget)}
|
||||
>
|
||||
<Plus aria-hidden="true" /> Add database
|
||||
</Button>
|
||||
) : null}
|
||||
</>
|
||||
)}
|
||||
/>
|
||||
@@ -1219,17 +1159,9 @@ export function DatabaseManagementPage({
|
||||
navigation={(
|
||||
<FleetLedgerBreadcrumb
|
||||
items={fleetBreadcrumb}
|
||||
back={screen.kind === "relationships"
|
||||
back={screen.kind === "relationships" || (screen.kind === "tables" && !tableNestedNavigationActive)
|
||||
? { label: "Back to databases", onBack: showList, disabled: navigationBusy }
|
||||
: screen.kind === "tables" && tableNestedNavigationActive
|
||||
? {
|
||||
label: "Back to tables",
|
||||
onBack: () => tableNestedNavigationBack?.(),
|
||||
disabled: navigationBusy || !tableNestedNavigationBack,
|
||||
}
|
||||
: screen.kind === "tables"
|
||||
? { label: "Back to databases", onBack: showList, disabled: navigationBusy }
|
||||
: undefined}
|
||||
: undefined}
|
||||
/>
|
||||
)}
|
||||
children={<FleetLedgerContent busy={isLoading || isFetching}>{fleetContent}</FleetLedgerContent>}
|
||||
@@ -1273,16 +1205,40 @@ export function DatabaseManagementPage({
|
||||
selectedModel={selectedMetadataModel}
|
||||
onSelectedModelChange={setSelectedMetadataModel}
|
||||
/>
|
||||
{screen.kind === "list" ? <>
|
||||
<Button type="button" variant="outline" className="whitespace-nowrap disabled:opacity-70" aria-label="View description generation history" disabled={!canManage} title="View description generation history" onClick={openDescriptionGenerationHistory}>
|
||||
<History /> View description generation history
|
||||
</Button>
|
||||
<Button type="button" variant="outline" className="whitespace-nowrap disabled:opacity-70" aria-label="View sensitive suggestion history" disabled={!canManage} title={observedActiveSensitiveDataSuggestionRun ? "Sensitive suggestion generation is active" : "View sensitive suggestion history"} onClick={openSensitiveDataSuggestionHistory}>
|
||||
<History />
|
||||
{observedActiveSensitiveDataSuggestionRun ? <span aria-hidden="true" className="size-2 rounded-full bg-primary" /> : null}
|
||||
View sensitive suggestion history
|
||||
</Button>
|
||||
</> : null}
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
className="whitespace-nowrap disabled:opacity-70"
|
||||
aria-label="Run descriptions generation history"
|
||||
disabled={!canManage}
|
||||
title={observedActiveDescriptionGenerationRun
|
||||
? "Description generation is active"
|
||||
: "View description generation history"}
|
||||
onClick={openDescriptionGenerationHistory}
|
||||
>
|
||||
<History />
|
||||
Run descriptions generation history
|
||||
{observedActiveDescriptionGenerationRun ? (
|
||||
<span aria-hidden="true" className="size-1.5 rounded-full bg-primary" />
|
||||
) : null}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
className="whitespace-nowrap disabled:opacity-70"
|
||||
aria-label="Run sensitive suggestions history"
|
||||
disabled={!canManage}
|
||||
title={observedActiveSensitiveDataSuggestionRun
|
||||
? "Sensitive suggestion generation is active"
|
||||
: "View sensitive suggestion history"}
|
||||
onClick={openSensitiveDataSuggestionHistory}
|
||||
>
|
||||
<History />
|
||||
Run sensitive suggestions history
|
||||
{observedActiveSensitiveDataSuggestionRun ? (
|
||||
<span aria-hidden="true" className="size-1.5 rounded-full bg-primary" />
|
||||
) : null}
|
||||
</Button>
|
||||
</div>
|
||||
{screen.kind === "list" ? (
|
||||
<div
|
||||
@@ -1293,6 +1249,14 @@ export function DatabaseManagementPage({
|
||||
<Button type="button" variant="outline" disabled={isFetching} onClick={() => void refreshList()}>
|
||||
<RefreshCw className={isFetching ? "animate-spin" : ""} /> Refresh
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
disabled={!canManage || availableWorkspaces.length === 0}
|
||||
title={availableWorkspaces.length === 0 ? "Every available workspace is already configured" : undefined}
|
||||
onClick={(event) => addDatabase(event.currentTarget)}
|
||||
>
|
||||
<Plus /> Add database
|
||||
</Button>
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
@@ -1329,12 +1293,10 @@ export function DatabaseManagementPage({
|
||||
onEdit={editRow}
|
||||
onDelete={deleteRow}
|
||||
onOpenSync={openSync}
|
||||
onSelectionChange={(selected) => setSelectedDatabaseId(selected.length === 1 ? selected[0].id ?? null : null)}
|
||||
onTestSelected={testSelected}
|
||||
onSyncSelected={syncSelected}
|
||||
selectedMetadataModel={selectedMetadataModelAvailable ? selectedMetadataModel : null}
|
||||
descriptionGenerationActive={descriptionGenerationActive}
|
||||
sensitiveDataSuggestionRuns={sensitiveDataSuggestionRuns}
|
||||
onGenerateDescriptions={generateDatabaseDescriptions}
|
||||
onSuggestSensitive={suggestDatabaseSensitiveFields}
|
||||
onDeleteMetadataSelected={deleteSelectedMetadata}
|
||||
@@ -1346,6 +1308,8 @@ export function DatabaseManagementPage({
|
||||
<DatabaseForm
|
||||
mode={screen.kind as DatabaseFormMode}
|
||||
row={activeRow}
|
||||
availableWorkspaces={availableWorkspaces}
|
||||
workspaceLocked={screen.kind === "add" && Boolean(screen.workspaceLocked)}
|
||||
draft={draft}
|
||||
dirty={dirty}
|
||||
canManage={canManage}
|
||||
@@ -1356,6 +1320,7 @@ export function DatabaseManagementPage({
|
||||
partialSecretFailure={partialSecretFailure}
|
||||
headingRef={formHeadingRef}
|
||||
onBack={backToList}
|
||||
onWorkspaceChange={changeWorkspace}
|
||||
onFieldChange={changeField}
|
||||
onTransportChange={changeTransport}
|
||||
onBindingChange={changeBinding}
|
||||
@@ -1370,8 +1335,6 @@ export function DatabaseManagementPage({
|
||||
onOpenRelationships={() => openRelationships(activeRow)}
|
||||
onSync={(scope) => void syncDatabase(scope)}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
activeSyncRun={currentActiveRun}
|
||||
/>
|
||||
) : null}
|
||||
@@ -1391,7 +1354,6 @@ export function DatabaseManagementPage({
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onDescriptionGenerationRunStarted={rememberDescriptionGenerationRun}
|
||||
onSuggestSensitive={suggestActiveDatabaseSensitiveFields}
|
||||
onClearCatalogTables={clearActiveCatalogTables}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
/>
|
||||
) : null}
|
||||
@@ -1405,10 +1367,6 @@ export function DatabaseManagementPage({
|
||||
onOpenTables={() => openTables(activeRow)}
|
||||
onRunStarted={rememberSyncRun}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
onNavigationStateChange={setTablesNavigationState}
|
||||
/>
|
||||
) : null}
|
||||
</div>
|
||||
@@ -1423,7 +1381,6 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<DescriptionGenerationDrawer
|
||||
open={descriptionGenerationDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeDescriptionGenerationRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeDescriptionGenerationRun?.modelId,
|
||||
@@ -1434,7 +1391,6 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<SensitiveDataSuggestionHistoryDrawer
|
||||
open={sensitiveDataSuggestionHistoryDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeSensitiveDataSuggestionRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeSensitiveDataSuggestionRun?.modelId,
|
||||
|
||||
@@ -356,7 +356,7 @@ test("shows an explicit recoverable incomplete state when no provider model is a
|
||||
renderManagement();
|
||||
|
||||
const incomplete = await screen.findByRole("alert", { name: "Pi configuration incomplete" });
|
||||
expect(incomplete).toHaveTextContent("No provider or model options are available");
|
||||
expect(incomplete).toHaveTextContent("No enabled provider and model choices are available");
|
||||
expect(incomplete).toHaveTextContent("Check the host-managed Pi model configuration");
|
||||
expect(screen.queryByText("Loading Pi management…")).not.toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "Save defaults" })).toBeDisabled();
|
||||
|
||||
@@ -78,7 +78,7 @@ function ReadinessRail({ ready, configured, credentials, smokeState }: {
|
||||
state={credentials === "present" ? "ready" : "attention"}
|
||||
/>
|
||||
<RailItem
|
||||
label="Configuration test"
|
||||
label="Saved-config test"
|
||||
value={smokeState === "passed" ? "Saved configuration test passed" : smokeState === "failed" ? "Saved configuration test failed" : "Saved configuration test not run"}
|
||||
state={smokeState === "passed" ? "ready" : smokeState === "failed" ? "attention" : "idle"}
|
||||
/>
|
||||
@@ -505,7 +505,7 @@ export function PiManagement({ open, onClose }: { open: boolean; onClose: () =>
|
||||
) : (
|
||||
<div className="mt-4">
|
||||
<div role="alert" aria-label="Pi configuration incomplete" className="rounded-md border border-amber-500/30 bg-amber-500/10 p-4 text-sm text-amber-900 dark:text-amber-200">
|
||||
<p className="font-semibold">No provider or model options are available.</p>
|
||||
<p className="font-semibold">No enabled provider and model choices are available.</p>
|
||||
<p className="mt-1">Check the host-managed Pi model configuration, then retry this panel.</p>
|
||||
</div>
|
||||
<div className="mt-4 flex flex-wrap gap-2">
|
||||
|
||||
@@ -296,7 +296,7 @@ export function ComposerFooter() {
|
||||
</FooterSelect>
|
||||
<span
|
||||
className="whitespace-nowrap font-mono text-[0.7rem] tabular-nums text-muted-foreground"
|
||||
aria-label="Input, cached, and output tokens"
|
||||
aria-label="Input non-cached, cached, and output tokens"
|
||||
>
|
||||
{formatTokenUsage(tokenUsage)}
|
||||
</span>
|
||||
|
||||
@@ -249,7 +249,6 @@ export function CatalogSyncDrawer({
|
||||
key={item.id}
|
||||
type="button"
|
||||
aria-current={item.id === run?.id ? "true" : undefined}
|
||||
data-status={item.state}
|
||||
className="flex w-full items-center justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 aria-[current=true]:bg-primary/8"
|
||||
onClick={() => onRunChange(item)}
|
||||
>
|
||||
@@ -285,12 +284,7 @@ export function CatalogSyncDrawer({
|
||||
open
|
||||
ariaLabel="Schema synchronization"
|
||||
eyebrow="Schema synchronization"
|
||||
title={run ? (
|
||||
<span className="inline-flex items-center gap-2">
|
||||
{!terminal(run) ? <LoaderCircle className="size-5 animate-spin" aria-hidden="true" /> : null}
|
||||
{stateLabel(run)[0].toUpperCase()}{stateLabel(run).slice(1)}
|
||||
</span>
|
||||
) : "Synchronization history"}
|
||||
title={run ? `${stateLabel(run)[0].toUpperCase()}${stateLabel(run).slice(1)}` : "Synchronization history"}
|
||||
description={run ? `${run.scope} · ${elapsed(run, now)}` : undefined}
|
||||
onClose={onClose}
|
||||
closeLabel="Close synchronization drawer"
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useEffect, useMemo, useRef, useState, type ReactNode } from "react";
|
||||
import { Menu } from "@base-ui/react/menu";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
@@ -10,11 +10,9 @@ import { ApiError, apiErrorMessage } from "../../api/client";
|
||||
import {
|
||||
consolidateCatalogDescriptions,
|
||||
listCatalogColumns,
|
||||
startCatalogSync,
|
||||
startDescriptionGenerationRun,
|
||||
updateCatalogColumnMetadata,
|
||||
type CatalogColumn,
|
||||
type CatalogSyncRun,
|
||||
type CatalogTable,
|
||||
type DescriptionGenerationRun,
|
||||
type SensitiveDataSuggestionRequest,
|
||||
@@ -22,21 +20,21 @@ import {
|
||||
import type { DatabaseNavigationState } from "./model";
|
||||
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
|
||||
import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
|
||||
|
||||
interface Props {
|
||||
databaseId: string;
|
||||
table: CatalogTable;
|
||||
databaseVersion: number;
|
||||
canManage: boolean;
|
||||
selectedMetadataModel: string | null;
|
||||
descriptionGenerationActive: boolean;
|
||||
onDescriptionGenerationRunStarted: (run: DescriptionGenerationRun) => void;
|
||||
onCatalogSyncRunStarted?: (run: CatalogSyncRun) => void;
|
||||
onNavigationStateChange: (state: DatabaseNavigationState) => void;
|
||||
onSync: () => void;
|
||||
onSuggestSensitive: (selection: SensitiveDataSuggestionRequest, scopeLabel: string) => Promise<void>;
|
||||
bindingReady?: boolean;
|
||||
catalogOperationActive?: boolean;
|
||||
onCatalogMetricsChanged?: () => void | Promise<void>;
|
||||
metadataModelControl?: ReactNode;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
|
||||
@@ -81,7 +79,6 @@ function SensitiveCell({ data, context }: ICellRendererParams<CatalogColumn, unk
|
||||
checked={data.sensitive}
|
||||
disabled={!context.canManage || context.busy}
|
||||
aria-label={`Sensitive data for ${data.name}`}
|
||||
title="Mark this column as sensitive; this does not select it for actions."
|
||||
onChange={(event) => context.onSensitiveChange(data, event.target.checked)}
|
||||
onClick={(event) => event.stopPropagation()}
|
||||
/>
|
||||
@@ -92,16 +89,17 @@ function SensitiveCell({ data, context }: ICellRendererParams<CatalogColumn, unk
|
||||
export function DatabaseColumns({
|
||||
databaseId,
|
||||
table,
|
||||
databaseVersion,
|
||||
canManage,
|
||||
selectedMetadataModel,
|
||||
descriptionGenerationActive,
|
||||
onDescriptionGenerationRunStarted,
|
||||
onCatalogSyncRunStarted,
|
||||
onNavigationStateChange,
|
||||
onSync,
|
||||
onSuggestSensitive,
|
||||
bindingReady = true,
|
||||
catalogOperationActive = false,
|
||||
onCatalogMetricsChanged,
|
||||
metadataModelControl,
|
||||
presentation = "legacy",
|
||||
}: Props) {
|
||||
const queryClient = useQueryClient();
|
||||
@@ -291,33 +289,16 @@ export function DatabaseColumns({
|
||||
};
|
||||
|
||||
type FleetColumnAction =
|
||||
| "sync-columns"
|
||||
| "generate-descriptions"
|
||||
| "consolidate-descriptions"
|
||||
| "suggest-sensitive"
|
||||
| "sync-columns"
|
||||
| "save-sensitive";
|
||||
|
||||
const fleetActions: readonly FleetActionOption<FleetColumnAction>[] = [
|
||||
{
|
||||
id: "sync-columns",
|
||||
label: "Synchronize columns for this table",
|
||||
group: "Synchronization",
|
||||
scopeLabel: "Selected table",
|
||||
runLabel: "Synchronize",
|
||||
disabled: !canManage || selectedIds.length === 0 || catalogOperationActive || busy,
|
||||
disabledReason: !canManage
|
||||
? "You do not have permission to synchronize the catalog."
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one column."
|
||||
: catalogOperationActive
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy
|
||||
? "Another action is running."
|
||||
: undefined,
|
||||
},
|
||||
{
|
||||
id: "generate-descriptions",
|
||||
label: "Generate column descriptions",
|
||||
label: "Generate descriptions",
|
||||
group: "Descriptions",
|
||||
runLabel: "Generate",
|
||||
disabled: !canManage || selectedIds.length === 0 || !selectedMetadataModel || descriptionGenerationActive || catalogOperationActive || busy,
|
||||
@@ -326,7 +307,7 @@ export function DatabaseColumns({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one column."
|
||||
: !selectedMetadataModel
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
? "Choose a metadata model first."
|
||||
: descriptionGenerationActive || catalogOperationActive
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy
|
||||
@@ -335,7 +316,7 @@ export function DatabaseColumns({
|
||||
},
|
||||
{
|
||||
id: "consolidate-descriptions",
|
||||
label: "Copy generated descriptions to Description",
|
||||
label: "Move generated to Description",
|
||||
group: "Descriptions",
|
||||
runLabel: "Move",
|
||||
disabled: !canManage || selectedIds.length === 0 || catalogOperationActive || busy,
|
||||
@@ -360,23 +341,40 @@ export function DatabaseColumns({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one column."
|
||||
: !selectedMetadataModel
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
? "Choose a metadata model first."
|
||||
: descriptionGenerationActive || catalogOperationActive
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy
|
||||
? "Another action is running."
|
||||
: undefined,
|
||||
},
|
||||
{
|
||||
id: "sync-columns",
|
||||
label: "Synchronize columns",
|
||||
group: "Synchronization",
|
||||
scopeLabel: "All columns in this table",
|
||||
runLabel: "Synchronize",
|
||||
disabled: !canManage || !bindingReady || catalogOperationActive || busy,
|
||||
disabledReason: !canManage
|
||||
? "You do not have permission to synchronize the catalog."
|
||||
: !bindingReady
|
||||
? "Test the current database binding first."
|
||||
: catalogOperationActive
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy
|
||||
? "Another action is running."
|
||||
: undefined,
|
||||
},
|
||||
{
|
||||
id: "save-sensitive",
|
||||
label: "Save sensitive field changes",
|
||||
label: "Save sensitive-field changes",
|
||||
group: "Sensitive data",
|
||||
runLabel: "Save",
|
||||
disabled: !canManage || changedSensitiveColumns.length === 0 || catalogOperationActive || busy,
|
||||
disabledReason: !canManage
|
||||
? "You do not have permission to update sensitive fields."
|
||||
: changedSensitiveColumns.length === 0
|
||||
? "No sensitive field changes to save."
|
||||
? "No sensitive-field changes are waiting to be saved."
|
||||
: catalogOperationActive
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy
|
||||
@@ -386,10 +384,10 @@ export function DatabaseColumns({
|
||||
];
|
||||
|
||||
const runFleetAction = async (action: FleetColumnAction) => {
|
||||
if (action === "sync-columns") await synchronizeColumns();
|
||||
else if (action === "generate-descriptions") await generateDescriptions();
|
||||
if (action === "generate-descriptions") await generateDescriptions();
|
||||
else if (action === "consolidate-descriptions") await consolidateDescriptions();
|
||||
else if (action === "suggest-sensitive") await suggestSensitive();
|
||||
else if (action === "sync-columns") onSync();
|
||||
else await saveSensitive();
|
||||
};
|
||||
|
||||
@@ -398,25 +396,9 @@ export function DatabaseColumns({
|
||||
setSelectedIds([]);
|
||||
};
|
||||
|
||||
const synchronizeColumns = async () => {
|
||||
if (selectedIds.length === 0) return;
|
||||
setBusy(true);
|
||||
try {
|
||||
const run = await startCatalogSync(databaseId, databaseVersion, "columns", [table.id]);
|
||||
onCatalogSyncRunStarted?.(run);
|
||||
gridRef.current?.api.deselectAll();
|
||||
setSelectedIds([]);
|
||||
toast.success("Column synchronization started");
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
const columns = useMemo<ColDef<CatalogColumn>[]>(() => [
|
||||
{ field: "ordinalPosition", headerName: "#", width: 64, maxWidth: 64, filter: "agNumberColumnFilter" },
|
||||
{ field: "sensitive", headerName: "Sensitive", headerTooltip: "Mark this column as sensitive; this does not select it for actions.", minWidth: 110, width: 110, sortable: false, filter: false, resizable: false, cellRenderer: SensitiveCell },
|
||||
{ field: "sensitive", headerName: "Sensitive", minWidth: 110, width: 110, sortable: false, filter: false, resizable: false, cellRenderer: SensitiveCell },
|
||||
{ field: "name", headerName: "Name", minWidth: 190, flex: 1, cellClass: "font-mono text-xs" },
|
||||
{ field: "dataType", headerName: "Type", minWidth: 150, flex: 0.8, cellClass: "font-mono text-xs" },
|
||||
{ headerName: "Keys", minWidth: 125, width: 125, sortable: false, filter: false, cellRenderer: KeyCell },
|
||||
@@ -439,7 +421,7 @@ export function DatabaseColumns({
|
||||
<p className="font-semibold text-destructive">Catalog columns could not be loaded.</p>
|
||||
<p className="mt-1 leading-5 text-muted-foreground">{apiErrorMessage(error)}</p>
|
||||
<Button type="button" variant="outline" className="mt-4" disabled={isFetching} onClick={() => void refetch()}>
|
||||
<RefreshCw className={isFetching ? "animate-spin" : ""} aria-hidden="true" /> Retry loading columns
|
||||
<RefreshCw className={isFetching ? "animate-spin" : ""} aria-hidden="true" /> Retry columns
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
@@ -486,6 +468,7 @@ export function DatabaseColumns({
|
||||
busyLabel={sensitiveAction === "suggest" ? "Suggesting…" : sensitiveAction === "save" ? "Saving…" : "Running…"}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Column action"
|
||||
/>
|
||||
</>
|
||||
@@ -497,9 +480,8 @@ export function DatabaseColumns({
|
||||
<Menu.Portal>
|
||||
<Menu.Positioner side="bottom" align="start" sideOffset={4}>
|
||||
<Menu.Popup className="z-50 min-w-64 rounded-lg bg-popover p-1 text-popover-foreground shadow-md ring-1 ring-foreground/10 outline-none">
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !selectedIds.length || catalogOperationActive || busy} onClick={() => void synchronizeColumns()}>Synchronize columns for this table</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !selectedMetadataModel || descriptionGenerationActive || busy} onClick={() => void generateDescriptions()}>Generate {selectedIds.length === 1 ? "column description" : "column descriptions"}</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || busy} onClick={() => void consolidateDescriptions()}>Copy generated descriptions to Description</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !selectedMetadataModel || descriptionGenerationActive || busy} onClick={() => void generateDescriptions()}>Generate {selectedIds.length === 1 ? "description" : "descriptions"}</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || busy} onClick={() => void consolidateDescriptions()}>Move generated description to Description</Menu.Item>
|
||||
</Menu.Popup>
|
||||
</Menu.Positioner>
|
||||
</Menu.Portal>
|
||||
@@ -519,6 +501,7 @@ export function DatabaseColumns({
|
||||
busyLabel={sensitiveAction === "save" ? "Saving…" : "Running…"}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Column action"
|
||||
/>
|
||||
<input className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search columns" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
|
||||
@@ -532,6 +515,7 @@ export function DatabaseColumns({
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{data.length}</span>
|
||||
{changedSensitiveColumns.length > 0 ? <Button type="button" disabled={!canManage || busy} onClick={() => void saveSensitive()}><Save />{sensitiveAction === "save" ? "Saving…" : "Save sensitive fields"}</Button> : null}
|
||||
<Button type="button" variant="outline" disabled={isFetching || busy} onClick={() => void refetch()}><RefreshCw className={isFetching ? "animate-spin" : ""} />Refresh</Button>
|
||||
<Button type="button" disabled={!canManage || busy} onClick={onSync}><RefreshCw />Sync columns</Button>
|
||||
</>
|
||||
)
|
||||
)}
|
||||
@@ -548,17 +532,6 @@ export function DatabaseColumns({
|
||||
defaultColDef={{ sortable: true, filter: true, resizable: true }}
|
||||
getRowId={({ data: row }) => row.id}
|
||||
rowSelection={{ mode: "multiRow", selectAll: "filtered", enableClickSelection: false }}
|
||||
selectionColumnDef={{
|
||||
headerName: "Select",
|
||||
headerTooltip: "Select columns for actions",
|
||||
width: 72,
|
||||
minWidth: 72,
|
||||
maxWidth: 72,
|
||||
pinned: "left",
|
||||
sortable: false,
|
||||
resizable: false,
|
||||
suppressMovable: true,
|
||||
}}
|
||||
onSelectionChanged={({ api }) => setSelectedIds(api.getSelectedRows().map((column) => column.id))}
|
||||
rowHeight={44}
|
||||
headerHeight={38}
|
||||
|
||||
@@ -14,9 +14,6 @@ const database: CatalogDatabase = {
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: { transport: "postgres_direct", configurationState: "ready", sessionTransportSupported: true },
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
@@ -110,12 +107,12 @@ test("Fleet columns distinguish a failed query from an empty catalog and retry i
|
||||
presentation="fleet"
|
||||
databaseId={database.id!}
|
||||
table={table}
|
||||
databaseVersion={database.version}
|
||||
canManage
|
||||
selectedMetadataModel={null}
|
||||
descriptionGenerationActive={false}
|
||||
onDescriptionGenerationRunStarted={noop}
|
||||
onNavigationStateChange={noop}
|
||||
onSync={noop}
|
||||
onSuggestSensitive={noopAsync}
|
||||
/>,
|
||||
);
|
||||
@@ -125,7 +122,7 @@ test("Fleet columns distinguish a failed query from an empty catalog and retry i
|
||||
expect(alert).toHaveTextContent("The database catalog is unavailable.");
|
||||
expect(screen.queryByText(/No columns synchronized/i)).not.toBeInTheDocument();
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Retry loading columns" }));
|
||||
await user.click(screen.getByRole("button", { name: "Retry columns" }));
|
||||
await waitFor(() => expect(requests).toBe(2));
|
||||
await waitFor(() => expect(screen.queryByText("Catalog columns could not be loaded.")).not.toBeInTheDocument());
|
||||
});
|
||||
|
||||
@@ -12,6 +12,7 @@ import { Button } from "../../components/ui/button";
|
||||
import type {
|
||||
CatalogDatabase,
|
||||
CatalogSecretName,
|
||||
CatalogSyncScope,
|
||||
CatalogSyncRun,
|
||||
DatabaseBinding,
|
||||
DatabaseTransport,
|
||||
@@ -22,7 +23,7 @@ import type {
|
||||
DatabaseFormMode,
|
||||
} from "./model";
|
||||
import { statusLabel } from "./model";
|
||||
import { DatabaseSyncMenu, type DatabaseSyncScope } from "./DatabaseSyncMenu";
|
||||
import { DatabaseSyncMenu } from "./DatabaseSyncMenu";
|
||||
|
||||
const inputClass = "h-9 w-full rounded-md border border-input bg-card px-3 text-sm outline-none transition focus:border-primary/60 focus:ring-3 focus:ring-ring/15 read-only:bg-muted/40 read-only:text-muted-foreground disabled:bg-muted disabled:text-muted-foreground";
|
||||
const labelClass = "grid min-w-0 gap-1.5 text-xs font-medium text-foreground";
|
||||
@@ -112,6 +113,8 @@ function SecretField({
|
||||
interface DatabaseFormProps {
|
||||
mode: DatabaseFormMode;
|
||||
row: CatalogDatabase;
|
||||
availableWorkspaces: CatalogDatabase[];
|
||||
workspaceLocked: boolean;
|
||||
draft: DatabaseFormDraft;
|
||||
dirty: boolean;
|
||||
canManage: boolean;
|
||||
@@ -122,6 +125,7 @@ interface DatabaseFormProps {
|
||||
partialSecretFailure: string | null;
|
||||
headingRef: RefObject<HTMLHeadingElement>;
|
||||
onBack: () => void;
|
||||
onWorkspaceChange: (workspaceId: string) => void;
|
||||
onFieldChange: (field: "databaseName" | "schema", value: string) => void;
|
||||
onTransportChange: (transport: DatabaseTransport) => void;
|
||||
onBindingChange: <K extends keyof DatabaseBinding>(key: K, value: DatabaseBinding[K]) => void;
|
||||
@@ -134,19 +138,18 @@ interface DatabaseFormProps {
|
||||
onRetrySecrets: () => void;
|
||||
onOpenTables: (origin: HTMLButtonElement) => void;
|
||||
onOpenRelationships: () => void;
|
||||
onSync: (scope: DatabaseSyncScope) => void;
|
||||
onSync: (scope: CatalogSyncScope) => void;
|
||||
onOpenSync: () => void;
|
||||
onOpenDescriptionHistory?: () => void;
|
||||
onOpenSensitiveHistory?: () => void;
|
||||
activeSyncRun?: CatalogSyncRun;
|
||||
presentation?: "page" | "drawer";
|
||||
}
|
||||
|
||||
export function databaseFormTitle(
|
||||
mode: DatabaseFormMode,
|
||||
workspaceLocked: boolean,
|
||||
presentation: "page" | "drawer" = "page",
|
||||
): string {
|
||||
if (mode === "configure") return "Configure catalog";
|
||||
if (mode === "add") return workspaceLocked ? "Edit database" : "Add database";
|
||||
if (mode === "edit") return "Edit database";
|
||||
if (mode === "delete") return presentation === "drawer" ? "Remove configuration" : "Delete database";
|
||||
return "Database details";
|
||||
@@ -155,6 +158,8 @@ export function databaseFormTitle(
|
||||
export function DatabaseForm({
|
||||
mode,
|
||||
row,
|
||||
availableWorkspaces,
|
||||
workspaceLocked,
|
||||
draft,
|
||||
dirty,
|
||||
canManage,
|
||||
@@ -165,6 +170,7 @@ export function DatabaseForm({
|
||||
partialSecretFailure,
|
||||
headingRef,
|
||||
onBack,
|
||||
onWorkspaceChange,
|
||||
onFieldChange,
|
||||
onTransportChange,
|
||||
onBindingChange,
|
||||
@@ -179,14 +185,12 @@ export function DatabaseForm({
|
||||
onOpenRelationships,
|
||||
onSync,
|
||||
onOpenSync,
|
||||
onOpenDescriptionHistory,
|
||||
onOpenSensitiveHistory,
|
||||
activeSyncRun,
|
||||
presentation = "page",
|
||||
}: DatabaseFormProps) {
|
||||
const title = databaseFormTitle(mode, presentation);
|
||||
const title = databaseFormTitle(mode, workspaceLocked, presentation);
|
||||
const readOnly = mode === "view" || mode === "delete";
|
||||
const editable = mode === "configure" || mode === "edit";
|
||||
const editable = mode === "add" || mode === "edit";
|
||||
const busy = busyAction !== null;
|
||||
const testDisabled = !canManage
|
||||
|| !row.configured
|
||||
@@ -264,9 +268,27 @@ export function DatabaseForm({
|
||||
) : null}
|
||||
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
<Field label="Workspace">
|
||||
<input className={inputClass} value={row.workspaceName} readOnly />
|
||||
</Field>
|
||||
{mode === "add" ? (
|
||||
<Field label="Workspace">
|
||||
<select
|
||||
className={inputClass}
|
||||
aria-label="Workspace"
|
||||
value={draft.workspaceId}
|
||||
disabled={workspaceLocked || busy}
|
||||
onChange={(event) => onWorkspaceChange(event.target.value)}
|
||||
>
|
||||
{availableWorkspaces.map((workspace) => (
|
||||
<option key={workspace.workspaceId} value={workspace.workspaceId}>
|
||||
{workspace.workspaceName}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
</Field>
|
||||
) : (
|
||||
<Field label="Workspace">
|
||||
<input className={inputClass} value={row.workspaceName} readOnly />
|
||||
</Field>
|
||||
)}
|
||||
<Field label="Workspace ID">
|
||||
<input className={`${inputClass} font-mono text-xs`} value={row.workspaceId} readOnly />
|
||||
</Field>
|
||||
@@ -281,7 +303,7 @@ export function DatabaseForm({
|
||||
disabled={readOnly || busy}
|
||||
onChange={(event) => onTransportChange(event.target.value as DatabaseTransport)}
|
||||
>
|
||||
<option value="postgres_direct">Direct PostgreSQL connection</option>
|
||||
<option value="postgres_direct">Direct PostgreSQL</option>
|
||||
<option value="rest_api">REST API</option>
|
||||
<option value="ssh_tunnel">SSH tunnel</option>
|
||||
</select>
|
||||
@@ -398,13 +420,13 @@ export function DatabaseForm({
|
||||
<TestTube2 /> {busyAction === "test" ? "Testing…" : "Test connection"}
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "configure" || mode === "edit" ? (
|
||||
{mode === "add" || mode === "edit" ? (
|
||||
<Button type="submit" disabled={saveDisabled}>
|
||||
<Save />
|
||||
{busyAction === "save"
|
||||
? "Saving…"
|
||||
: mode === "configure"
|
||||
? "Save catalog configuration"
|
||||
: mode === "add"
|
||||
? workspaceLocked ? "Save database" : "Add database"
|
||||
: "Save changes"}
|
||||
</Button>
|
||||
) : null}
|
||||
@@ -437,16 +459,6 @@ export function DatabaseForm({
|
||||
{activeSyncRun ? <RefreshCw className="animate-spin" /> : <History />} Sync history
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "view" && row.configured && onOpenDescriptionHistory ? (
|
||||
<Button type="button" variant="outline" size="sm" onClick={onOpenDescriptionHistory}>
|
||||
<History /> Description history
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "view" && row.configured && onOpenSensitiveHistory ? (
|
||||
<Button type="button" variant="outline" size="sm" onClick={onOpenSensitiveHistory}>
|
||||
<History /> Sensitive history
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "view" ? (
|
||||
<DatabaseSyncMenu
|
||||
disabled={!canManage || !bindingReady || busy || Boolean(activeSyncRun)}
|
||||
@@ -454,8 +466,6 @@ export function DatabaseForm({
|
||||
onSelect={onSync}
|
||||
/>
|
||||
) : null}
|
||||
{mode === "view" && row.configured && onOpenDescriptionHistory ? <Button type="button" variant="outline" onClick={onOpenDescriptionHistory}><History /> Description history</Button> : null}
|
||||
{mode === "view" && row.configured && onOpenSensitiveHistory ? <Button type="button" variant="outline" onClick={onOpenSensitiveHistory}><History /> Sensitive history</Button> : null}
|
||||
</div>
|
||||
</div>
|
||||
{row.lastTestedAt ? (
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { useEffect, useMemo, useRef, useState, type RefObject } from "react";
|
||||
import { useQueries } from "@tanstack/react-query";
|
||||
import { useEffect, useMemo, useRef, useState, type ReactNode, type RefObject } from "react";
|
||||
import { Menu } from "@base-ui/react/menu";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
import {
|
||||
@@ -10,19 +9,17 @@ import {
|
||||
} from "ag-grid-community";
|
||||
import "ag-grid-community/styles/ag-grid.css";
|
||||
import "ag-grid-community/styles/ag-theme-alpine.css";
|
||||
import { ChevronDown, Eye, History, Info, Link2, Pencil, RefreshCw, Rows3, Sparkles, Trash2, X } from "lucide-react";
|
||||
import { ChevronDown, Eye, History, Info, Pencil, RefreshCw, Rows3, Sparkles, Trash2, X } from "lucide-react";
|
||||
import { Button } from "../../components/ui/button";
|
||||
import { getCatalogMetrics } from "../../api/catalog-databases";
|
||||
import type {
|
||||
CatalogDatabase,
|
||||
CatalogMetrics,
|
||||
SensitiveDataSuggestionRun,
|
||||
CatalogDatabaseMetadataDeleteTarget,
|
||||
CatalogSyncScope,
|
||||
DescriptionGenerationScope,
|
||||
} from "../../api/catalog-databases";
|
||||
import { databaseSyncItemClass, databaseSyncScopes, type DatabaseSyncScope } from "./DatabaseSyncMenu";
|
||||
import { statusLabel } from "./model";
|
||||
import { databaseSyncItemClass, databaseSyncScopes } from "./DatabaseSyncMenu";
|
||||
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
|
||||
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
|
||||
|
||||
ModuleRegistry.registerModules([AllCommunityModule]);
|
||||
|
||||
@@ -36,16 +33,13 @@ interface DatabaseGridProps {
|
||||
onSearchChange: (value: string) => void;
|
||||
onView: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onOpenTables?: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onOpenRelationships?: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onEdit: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onDelete: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onOpenSync: (row: CatalogDatabase) => void;
|
||||
onSelectionChange?: (rows: CatalogDatabase[]) => void;
|
||||
onTestSelected: (rows: CatalogDatabase[]) => Promise<void>;
|
||||
onSyncSelected: (rows: CatalogDatabase[], scope: DatabaseSyncScope) => Promise<void>;
|
||||
onSyncSelected: (rows: CatalogDatabase[], scope: CatalogSyncScope) => Promise<void>;
|
||||
selectedMetadataModel: string | null;
|
||||
descriptionGenerationActive: boolean;
|
||||
sensitiveDataSuggestionRuns?: SensitiveDataSuggestionRun[];
|
||||
onGenerateDescriptions: (
|
||||
rows: CatalogDatabase[],
|
||||
scope: Extract<DescriptionGenerationScope, "all" | "missing">,
|
||||
@@ -55,6 +49,7 @@ interface DatabaseGridProps {
|
||||
rows: CatalogDatabase[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
) => Promise<void>;
|
||||
metadataModelControl?: ReactNode;
|
||||
onRefresh?: () => void | Promise<void>;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
@@ -64,7 +59,6 @@ interface DatabaseGridContext {
|
||||
presentation: "legacy" | "fleet";
|
||||
onView: DatabaseGridProps["onView"];
|
||||
onOpenTables?: DatabaseGridProps["onOpenTables"];
|
||||
onOpenRelationships?: DatabaseGridProps["onOpenRelationships"];
|
||||
onEdit: DatabaseGridProps["onEdit"];
|
||||
onDelete: DatabaseGridProps["onDelete"];
|
||||
onOpenSync: DatabaseGridProps["onOpenSync"];
|
||||
@@ -73,6 +67,7 @@ interface DatabaseGridContext {
|
||||
type DatabaseFleetAction =
|
||||
| "test"
|
||||
| "sync-tables"
|
||||
| "sync-columns"
|
||||
| "sync-relationships"
|
||||
| "sync-all"
|
||||
| "generate-all"
|
||||
@@ -96,96 +91,18 @@ function useCompactViewport(): boolean {
|
||||
return compact;
|
||||
}
|
||||
|
||||
type StateTone = "neutral" | "info" | "success" | "warning" | "danger";
|
||||
function StatusPill({ row }: { row: CatalogDatabase }) {
|
||||
const tone = !row.workspaceAvailable
|
||||
? "border-destructive/30 bg-destructive/10 text-destructive"
|
||||
: !row.configured
|
||||
? "border-border bg-muted text-muted-foreground"
|
||||
: row.connectionStatus === "reachable"
|
||||
? "border-emerald-300/70 bg-emerald-50 text-emerald-800 dark:bg-emerald-950/30 dark:text-emerald-300"
|
||||
: row.connectionStatus === "failed"
|
||||
? "border-destructive/30 bg-destructive/10 text-destructive"
|
||||
: "border-amber-300/70 bg-amber-50 text-amber-900 dark:bg-amber-950/30 dark:text-amber-300";
|
||||
|
||||
const stateToneClass: Record<StateTone, string> = {
|
||||
neutral: "bg-muted-foreground/55",
|
||||
info: "bg-sky-500",
|
||||
success: "bg-emerald-500",
|
||||
warning: "bg-amber-500",
|
||||
danger: "bg-destructive",
|
||||
};
|
||||
|
||||
function StateCell({
|
||||
label,
|
||||
detail,
|
||||
tone,
|
||||
detailClassName = "text-muted-foreground",
|
||||
}: {
|
||||
label: string;
|
||||
detail: string;
|
||||
tone: StateTone;
|
||||
detailClassName?: string;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex h-full min-w-0 flex-col justify-center gap-0.5 py-1 leading-tight">
|
||||
<span className="flex min-w-0 items-center gap-1.5 text-xs font-semibold text-foreground">
|
||||
<span className={`size-1.5 shrink-0 rounded-full ${stateToneClass[tone]}`} aria-hidden="true" />
|
||||
<span className="truncate">{label}</span>
|
||||
</span>
|
||||
<span className={`truncate pl-3 text-[11px] ${detailClassName}`}>{detail}</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function endpoint(host: string | undefined, port: number | undefined): string {
|
||||
if (!host) return "Endpoint not configured";
|
||||
return port ? `${host}:${port}` : host;
|
||||
}
|
||||
|
||||
function accessSummary(row: CatalogDatabase): { label: string; detail: string } {
|
||||
const transport = row.configured
|
||||
? row.binding.transport
|
||||
: row.runtimeBinding?.transport ?? row.binding.transport;
|
||||
if (transport === "rest_api") {
|
||||
if (!row.binding.baseUrl) return { label: "REST", detail: "Endpoint not configured" };
|
||||
try {
|
||||
return { label: "REST", detail: new URL(row.binding.baseUrl).host };
|
||||
} catch {
|
||||
return { label: "REST", detail: row.binding.baseUrl };
|
||||
}
|
||||
}
|
||||
if (transport === "ssh_tunnel") {
|
||||
return { label: "SSH", detail: endpoint(row.binding.sshHost, row.binding.sshPort) };
|
||||
}
|
||||
return {
|
||||
label: "Direct PostgreSQL connection",
|
||||
detail: endpoint(row.binding.host, row.binding.port),
|
||||
};
|
||||
}
|
||||
|
||||
function coverageStatus(total: number, complete: number): { label: string; detail: string; tone: StateTone } {
|
||||
if (total === 0) return { label: "Not started", detail: "No catalog objects", tone: "danger" };
|
||||
if (complete >= total) return { label: "Complete", detail: `${complete}/${total}`, tone: "success" };
|
||||
if (complete > 0) return { label: "Partial", detail: `${complete}/${total}`, tone: "warning" };
|
||||
return { label: "Not started", detail: `0/${total}`, tone: "danger" };
|
||||
}
|
||||
|
||||
function CatalogStatusCells({ row, metrics, sensitiveRun }: { row: CatalogDatabase; metrics?: CatalogMetrics; sensitiveRun?: SensitiveDataSuggestionRun }) {
|
||||
const access = accessSummary(row);
|
||||
const synchronized = row.schemaSyncedVersion === row.version;
|
||||
const syncStatus = !row.schemaSyncedVersion
|
||||
? { label: "Not synchronized", detail: "Schema not synchronized", tone: "danger" as const }
|
||||
: synchronized
|
||||
? { label: "Synchronized", detail: "Current schema", tone: "success" as const }
|
||||
: { label: "Partial", detail: "Schema revision changed", tone: "warning" as const };
|
||||
const descriptions = coverageStatus(metrics?.descriptionTargets ?? 0, metrics?.describedTargets ?? 0);
|
||||
const sensitiveProcessed = sensitiveRun ? sensitiveRun.suggestedSensitive + sensitiveRun.suggestedNonSensitive : 0;
|
||||
const sensitive = !sensitiveRun
|
||||
? { label: "Not started", detail: "No analysis has been run", tone: "danger" as const }
|
||||
: sensitiveRun.status === "completed" && sensitiveProcessed >= sensitiveRun.total
|
||||
? { label: "Complete", detail: `${sensitiveProcessed}/${sensitiveRun.total}`, tone: "success" as const }
|
||||
: ["failed", "interrupted"].includes(sensitiveRun.status)
|
||||
? { label: "Incomplete", detail: `${sensitiveProcessed}/${sensitiveRun.total}`, tone: "danger" as const }
|
||||
: { label: "Partial", detail: `${sensitiveProcessed}/${sensitiveRun.total}`, tone: "warning" as const };
|
||||
return (
|
||||
<div className="grid h-full min-w-[540px] grid-cols-4 gap-3 py-1">
|
||||
<StateCell label={access.label} detail={access.detail} tone="info" />
|
||||
<StateCell label={syncStatus.label} detail={syncStatus.detail} tone={syncStatus.tone} />
|
||||
<StateCell label={descriptions.label} detail={`Descriptions · ${descriptions.detail}`} tone={descriptions.tone} />
|
||||
<StateCell label={sensitive.label} detail={`Sensitive · ${sensitive.detail}`} tone={sensitive.tone} />
|
||||
</div>
|
||||
);
|
||||
return <div className="flex items-center gap-1.5"><span className={`inline-flex max-w-full truncate rounded-full border px-2 py-0.5 text-[11px] font-semibold ${tone}`}>{statusLabel(row)}</span>{row.activeSyncRun ? <span className="inline-flex rounded-full border border-primary/30 bg-primary/8 px-2 py-0.5 text-[11px] font-semibold text-primary">Syncing</span> : null}</div>;
|
||||
}
|
||||
|
||||
function DatabaseActionsCell({
|
||||
@@ -194,51 +111,19 @@ function DatabaseActionsCell({
|
||||
}: ICellRendererParams<CatalogDatabase, unknown, DatabaseGridContext>) {
|
||||
if (!data || !context) return null;
|
||||
|
||||
if (data.workspaceAvailable && !data.configured) {
|
||||
const configureReasonId = `database-configure-reason-${data.workspaceId}`;
|
||||
return (
|
||||
<div className="flex h-full items-center justify-end gap-1.5" onClick={(event) => event.stopPropagation()}>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon-lg"
|
||||
className={context.presentation === "fleet" ? "thot-fleet-icon-action" : undefined}
|
||||
data-tooltip={context.presentation === "fleet" ? "Details" : undefined}
|
||||
title={context.presentation === "legacy" ? `View ${data.workspaceName}` : undefined}
|
||||
aria-label={`View ${data.workspaceName}`}
|
||||
onClick={(event) => context.onView(data, event.currentTarget)}
|
||||
>
|
||||
{context.presentation === "fleet" ? <Info aria-hidden="true" /> : <Eye aria-hidden="true" />}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
className="whitespace-nowrap"
|
||||
aria-label={`Configure catalog for ${data.workspaceName}`}
|
||||
aria-describedby={!context.canManage ? configureReasonId : undefined}
|
||||
disabled={!context.canManage}
|
||||
onClick={(event) => context.onEdit(data, event.currentTarget)}
|
||||
>
|
||||
Configure catalog
|
||||
</Button>
|
||||
{!context.canManage ? (
|
||||
<span id={configureReasonId} className="sr-only">
|
||||
Database management permission is required.
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const editDisabled = !context.canManage || !data.workspaceAvailable;
|
||||
const deleteDisabled = !context.canManage || !data.configured;
|
||||
const editLabel = `Edit ${data.workspaceName}`;
|
||||
const editReasonId = `database-edit-reason-${data.workspaceId}`;
|
||||
const deleteReasonId = `database-delete-reason-${data.workspaceId}`;
|
||||
const relationshipsReasonId = `database-relationships-reason-${data.workspaceId}`;
|
||||
|
||||
return (
|
||||
<div className="flex h-full items-center justify-end gap-0.5" onClick={(event) => event.stopPropagation()}>
|
||||
{data.activeSyncRun || (context.presentation === "fleet" && data.configured && data.id) ? (
|
||||
<Button type="button" variant="ghost" size="icon-lg" className={context.presentation === "fleet" ? "thot-fleet-icon-action" : undefined} data-tooltip={context.presentation === "fleet" ? "Synchronization history" : undefined} title={context.presentation === "legacy" ? `View synchronization for ${data.workspaceName}` : undefined} aria-label={`View synchronization history for ${data.workspaceName}`} onClick={() => context.onOpenSync(data)}>
|
||||
{data.activeSyncRun ? <RefreshCw className="animate-spin" aria-hidden="true" /> : <History aria-hidden="true" />}
|
||||
</Button>
|
||||
) : null}
|
||||
{context.presentation === "fleet" && context.onOpenTables ? (
|
||||
<span className="thot-fleet-icon-action inline-flex" data-tooltip="Tables">
|
||||
<Button
|
||||
@@ -253,26 +138,6 @@ function DatabaseActionsCell({
|
||||
</Button>
|
||||
</span>
|
||||
) : null}
|
||||
{context.presentation === "fleet" && context.onOpenRelationships ? (
|
||||
<span className="thot-fleet-icon-action inline-flex" data-tooltip="Relationships">
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon-lg"
|
||||
aria-label={`View relationships for ${data.workspaceName}`}
|
||||
aria-describedby={!data.configured || !data.id ? relationshipsReasonId : undefined}
|
||||
disabled={!data.configured || !data.id}
|
||||
onClick={(event) => context.onOpenRelationships?.(data, event.currentTarget)}
|
||||
>
|
||||
<Link2 aria-hidden="true" />
|
||||
</Button>
|
||||
{!data.configured || !data.id ? (
|
||||
<span id={relationshipsReasonId} className="sr-only">
|
||||
Save this database configuration before managing relationships.
|
||||
</span>
|
||||
) : null}
|
||||
</span>
|
||||
) : null}
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
@@ -328,41 +193,20 @@ export function DatabaseGrid({
|
||||
onSearchChange,
|
||||
onView,
|
||||
onOpenTables,
|
||||
onOpenRelationships,
|
||||
onEdit,
|
||||
onDelete,
|
||||
onOpenSync,
|
||||
onSelectionChange,
|
||||
onTestSelected,
|
||||
onSyncSelected,
|
||||
selectedMetadataModel,
|
||||
descriptionGenerationActive,
|
||||
sensitiveDataSuggestionRuns = [],
|
||||
onGenerateDescriptions,
|
||||
onSuggestSensitive,
|
||||
onDeleteMetadataSelected,
|
||||
metadataModelControl,
|
||||
onRefresh,
|
||||
presentation = "legacy",
|
||||
}: DatabaseGridProps) {
|
||||
const metricQueries = useQueries({
|
||||
queries: rows.filter((row) => row.id).map((row) => ({
|
||||
queryKey: ["catalog-metrics", row.id],
|
||||
queryFn: () => getCatalogMetrics(row.id!),
|
||||
enabled: canManage,
|
||||
staleTime: 10_000,
|
||||
})),
|
||||
});
|
||||
const metricsByDatabase = useMemo(() => {
|
||||
const result: Record<string, CatalogMetrics> = {};
|
||||
rows.filter((row) => row.id).forEach((row, index) => {
|
||||
const metrics = metricQueries[index]?.data;
|
||||
if (metrics && row.id) result[row.id] = metrics;
|
||||
});
|
||||
return result;
|
||||
}, [metricQueries, rows]);
|
||||
const sensitiveRunsByDatabase = useMemo(() => new Map(
|
||||
rows.filter((row) => row.id).map((row) => [row.id!, sensitiveDataSuggestionRuns.find((run) => run.databaseId === row.id)]),
|
||||
), [rows, sensitiveDataSuggestionRuns]);
|
||||
const compact = useCompactViewport();
|
||||
const gridRef = useRef<AgGridReact<CatalogDatabase>>(null);
|
||||
const actionsTriggerRef = useRef<HTMLButtonElement>(null);
|
||||
@@ -374,8 +218,8 @@ export function DatabaseGrid({
|
||||
Extract<DescriptionGenerationScope, "all" | "missing"> | null
|
||||
>(null);
|
||||
const context = useMemo<DatabaseGridContext>(
|
||||
() => ({ canManage, presentation, onView, onOpenTables, onOpenRelationships, onEdit, onDelete, onOpenSync }),
|
||||
[canManage, presentation, onView, onOpenTables, onOpenRelationships, onEdit, onDelete, onOpenSync],
|
||||
() => ({ canManage, presentation, onView, onOpenTables, onEdit, onDelete, onOpenSync }),
|
||||
[canManage, presentation, onView, onOpenTables, onEdit, onDelete, onOpenSync],
|
||||
);
|
||||
|
||||
const columnDefs = useMemo<ColDef<CatalogDatabase>[]>(() => {
|
||||
@@ -386,18 +230,49 @@ export function DatabaseGrid({
|
||||
flex: compact ? undefined : 1.35,
|
||||
width: compact ? 112 : undefined,
|
||||
};
|
||||
const status: ColDef<CatalogDatabase> = {
|
||||
headerName: "Catalog status",
|
||||
minWidth: 580,
|
||||
flex: 2.5,
|
||||
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => {
|
||||
const databaseId = data?.id;
|
||||
return data
|
||||
? <CatalogStatusCells row={data} metrics={databaseId ? metricsByDatabase[databaseId] : undefined} sensitiveRun={databaseId ? sensitiveRunsByDatabase.get(databaseId) : undefined} />
|
||||
: null;
|
||||
const database: ColDef<CatalogDatabase> = {
|
||||
field: "databaseName", headerName: "Database", minWidth: 145, flex: 1,
|
||||
};
|
||||
const schema: ColDef<CatalogDatabase> = {
|
||||
field: "schema", headerName: "Schema", minWidth: 140, flex: 0.9,
|
||||
};
|
||||
const transport: ColDef<CatalogDatabase> = {
|
||||
field: "binding.transport",
|
||||
headerName: "Transport",
|
||||
minWidth: 135,
|
||||
flex: 0.9,
|
||||
valueFormatter: ({ value }) => String(value ?? "").replaceAll("_", " "),
|
||||
};
|
||||
const endpoint: ColDef<CatalogDatabase> = {
|
||||
headerName: "Endpoint",
|
||||
minWidth: 190,
|
||||
flex: 1.2,
|
||||
valueGetter: ({ data }) => {
|
||||
if (!data) return "";
|
||||
if (data.binding.transport === "rest_api") return data.binding.baseUrl ?? "";
|
||||
if (data.binding.transport === "ssh_tunnel") return data.binding.sshHost ?? "";
|
||||
return data.binding.host ? `${data.binding.host}:${data.binding.port ?? 5432}` : "";
|
||||
},
|
||||
};
|
||||
const actionsWidth = presentation === "fleet" ? 232 : 210;
|
||||
const status: ColDef<CatalogDatabase> = {
|
||||
headerName: "Status",
|
||||
minWidth: compact ? 96 : 130,
|
||||
width: compact ? 96 : undefined,
|
||||
flex: compact ? undefined : 0.8,
|
||||
cellClass: compact ? "thot-database-status-cell" : undefined,
|
||||
valueGetter: ({ data }) => (data ? statusLabel(data) : ""),
|
||||
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => (
|
||||
data ? <StatusPill row={data} /> : null
|
||||
),
|
||||
};
|
||||
const updated: ColDef<CatalogDatabase> = {
|
||||
field: "updatedAt",
|
||||
headerName: "Updated",
|
||||
minWidth: 170,
|
||||
flex: 0.9,
|
||||
valueFormatter: ({ value }) => value ? new Date(String(value)).toLocaleString() : "",
|
||||
};
|
||||
const actionsWidth = presentation === "fleet" ? 196 : compact ? 148 : 164;
|
||||
const actions: ColDef<CatalogDatabase> = {
|
||||
colId: "actions",
|
||||
headerName: "Actions",
|
||||
@@ -415,8 +290,10 @@ export function DatabaseGrid({
|
||||
cellRenderer: DatabaseActionsCell,
|
||||
};
|
||||
|
||||
return [workspace, status, actions];
|
||||
}, [compact, metricsByDatabase, presentation, sensitiveRunsByDatabase]);
|
||||
return compact
|
||||
? [workspace, status, database, schema, transport, endpoint, updated, actions]
|
||||
: [workspace, database, schema, transport, endpoint, status, updated, actions];
|
||||
}, [compact, presentation]);
|
||||
|
||||
const configuredCount = rows.filter((row) => row.configured).length;
|
||||
const hiddenSelected = selectedRows.filter((row) => {
|
||||
@@ -455,14 +332,14 @@ export function DatabaseGrid({
|
||||
: undefined);
|
||||
const generationReason = permissionReason
|
||||
?? (selectedRows.length !== 1 ? "Select exactly one database" : undefined)
|
||||
?? (!selectedMetadataModel ? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE : undefined)
|
||||
?? (!selectedMetadataModel ? "Choose a metadata model in the generation flow" : undefined)
|
||||
?? (descriptionGenerationActive ? "Wait for active description generation" : undefined)
|
||||
?? (selectedRows.some((row) => !row.configured || !row.id || row.activeSyncRun)
|
||||
? "The selected database must be configured and idle"
|
||||
: undefined);
|
||||
const sensitiveReason = permissionReason
|
||||
?? (selectedRows.length !== 1 ? "Select exactly one database" : undefined)
|
||||
?? (!selectedMetadataModel ? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE : undefined)
|
||||
?? (!selectedMetadataModel ? "Choose a metadata model in the suggestion flow" : undefined)
|
||||
?? (descriptionGenerationActive ? "Wait for the active metadata operation" : undefined)
|
||||
?? (selectedRows.some((row) => !row.configured || !row.id || row.activeSyncRun)
|
||||
? "The selected database must be configured and idle"
|
||||
@@ -475,6 +352,7 @@ export function DatabaseGrid({
|
||||
return [
|
||||
{ id: "test", label: "Test connections", group: "Connection", disabled: !canTestSelection, disabledReason: testReason },
|
||||
{ id: "sync-tables", label: "Synchronize tables", group: "Synchronization", disabled: !canSyncSelection, disabledReason: syncReason },
|
||||
{ id: "sync-columns", label: "Synchronize all columns", group: "Synchronization", disabled: !canSyncSelection, disabledReason: syncReason },
|
||||
{ id: "sync-relationships", label: "Synchronize relationships", group: "Synchronization", disabled: !canSyncSelection, disabledReason: syncReason },
|
||||
{ id: "sync-all", label: "Synchronize all", group: "Synchronization", disabled: !canSyncSelection, disabledReason: syncReason },
|
||||
{ id: "generate-missing", label: "Generate missing descriptions", group: "Descriptions", disabled: !canGenerateDescriptions, disabledReason: generationReason },
|
||||
@@ -521,10 +399,15 @@ export function DatabaseGrid({
|
||||
const perform = async (
|
||||
kind: "test" | "sync" | "generate" | "suggest" | "delete",
|
||||
operation: () => Promise<void>,
|
||||
clearSelection = true,
|
||||
) => {
|
||||
setAction(kind);
|
||||
try {
|
||||
await operation();
|
||||
if (clearSelection) {
|
||||
gridRef.current?.api.deselectAll();
|
||||
setSelectedRows([]);
|
||||
}
|
||||
setPendingDelete(null);
|
||||
setPendingGenerationScope(null);
|
||||
if (kind === "delete") window.setTimeout(() => searchInputRef.current?.focus(), 0);
|
||||
@@ -548,11 +431,11 @@ export function DatabaseGrid({
|
||||
return;
|
||||
}
|
||||
if (selectedAction.startsWith("sync-")) {
|
||||
const scope = selectedAction.slice("sync-".length) as DatabaseSyncScope;
|
||||
const scope = selectedAction.slice("sync-".length) as CatalogSyncScope;
|
||||
await perform("sync", () => onSyncSelected(selectedRows, scope));
|
||||
return;
|
||||
}
|
||||
await perform("suggest", () => onSuggestSensitive(selectedRows));
|
||||
await perform("suggest", () => onSuggestSensitive(selectedRows), false);
|
||||
};
|
||||
|
||||
return (
|
||||
@@ -608,7 +491,7 @@ export function DatabaseGrid({
|
||||
() => onGenerateDescriptions(selectedRows, pendingGenerationScope),
|
||||
)}
|
||||
>
|
||||
{pendingGenerationScope === "all" ? "Generate all descriptions" : "Generate missing descriptions"}
|
||||
{pendingGenerationScope === "all" ? "Generate All" : "Generate Missing"}
|
||||
</Button>
|
||||
</div>
|
||||
) : pendingDelete ? (
|
||||
@@ -655,6 +538,12 @@ export function DatabaseGrid({
|
||||
busyLabel={action === "suggest" ? "Suggesting…" : "Running…"}
|
||||
selectRef={actionSelectRef}
|
||||
onRun={runFleetAction}
|
||||
renderContext={(selectedAction) => (
|
||||
metadataModelControl
|
||||
&& ["generate-all", "generate-missing", "suggest-sensitive"].includes(selectedAction)
|
||||
? <div className="thot-fleet-metadata-model">{metadataModelControl}</div>
|
||||
: null
|
||||
)}
|
||||
onClear={() => {
|
||||
gridRef.current?.api.deselectAll();
|
||||
setSelectedRows([]);
|
||||
@@ -688,14 +577,14 @@ export function DatabaseGrid({
|
||||
disabled={!canGenerateDescriptions}
|
||||
onClick={() => setPendingGenerationScope("all")}
|
||||
>
|
||||
Generate all descriptions
|
||||
Generate All
|
||||
</Menu.Item>
|
||||
<Menu.Item
|
||||
className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45"
|
||||
disabled={!canGenerateDescriptions}
|
||||
onClick={() => setPendingGenerationScope("missing")}
|
||||
>
|
||||
Generate missing descriptions
|
||||
Generate Missing
|
||||
</Menu.Item>
|
||||
<Menu.Separator className="my-1 h-px bg-border" />
|
||||
<Menu.Item
|
||||
@@ -721,7 +610,7 @@ export function DatabaseGrid({
|
||||
variant="outline"
|
||||
disabled={!canManage || action !== null || descriptionGenerationActive}
|
||||
title={descriptionGenerationActive ? "Wait for the active description generation to finish" : undefined}
|
||||
onClick={() => void perform("suggest", () => onSuggestSensitive(selectedRows))}
|
||||
onClick={() => void perform("suggest", () => onSuggestSensitive(selectedRows), false)}
|
||||
>
|
||||
<Sparkles />{action === "suggest" ? "Suggesting…" : "Suggest sensitive fields"}
|
||||
</Button>
|
||||
@@ -742,7 +631,7 @@ export function DatabaseGrid({
|
||||
</Button>
|
||||
) : null}</>}
|
||||
<span className="whitespace-nowrap text-xs tabular-nums text-muted-foreground">
|
||||
{configuredCount}/{rows.length} catalogs configured
|
||||
{configuredCount}/{rows.length}
|
||||
</span>
|
||||
{isFetching && !isLoading ? (
|
||||
<span className="text-xs text-muted-foreground" role="status">Refreshing</span>
|
||||
@@ -769,11 +658,9 @@ export function DatabaseGrid({
|
||||
onSelectionChanged={({ api }) => {
|
||||
if (pendingDelete && action === null) setPendingDelete(null);
|
||||
if (pendingGenerationScope && action === null) setPendingGenerationScope(null);
|
||||
const rows = api.getSelectedRows();
|
||||
setSelectedRows(rows);
|
||||
onSelectionChange?.(rows);
|
||||
setSelectedRows(api.getSelectedRows());
|
||||
}}
|
||||
rowHeight={52}
|
||||
rowHeight={44}
|
||||
headerHeight={38}
|
||||
animateRows={false}
|
||||
/>
|
||||
|
||||
@@ -1,379 +0,0 @@
|
||||
import { render, screen, waitFor, within } from "@testing-library/react";
|
||||
import userEvent from "@testing-library/user-event";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { http, HttpResponse } from "msw";
|
||||
import { expect, test, vi } from "vitest";
|
||||
|
||||
import type {
|
||||
CatalogColumn,
|
||||
CatalogDatabase,
|
||||
CatalogRelationship,
|
||||
CatalogTable,
|
||||
} from "../../api/catalog-databases";
|
||||
import { Toaster } from "../../components/ui/sonner";
|
||||
import { server } from "../../test/msw";
|
||||
import { DatabaseRelationships } from "./DatabaseRelationships";
|
||||
|
||||
const databaseId = "11111111-1111-4111-8111-111111111111";
|
||||
|
||||
const database: CatalogDatabase = {
|
||||
id: databaseId,
|
||||
workspaceId: "psd",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: { transport: "postgres_direct", configurationState: "ready", sessionTransportSupported: true },
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "analytics",
|
||||
schema: "public",
|
||||
binding: { transport: "postgres_direct", port: 5432 },
|
||||
connectionStatus: "reachable",
|
||||
testedVersion: 3,
|
||||
version: 3,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
activeSyncRun: undefined,
|
||||
secrets: {
|
||||
password: false,
|
||||
apiKey: false,
|
||||
sshPrivateKey: false,
|
||||
sshPrivateKeyPassphrase: false,
|
||||
sshKnownHosts: false,
|
||||
tlsCa: false,
|
||||
},
|
||||
};
|
||||
|
||||
function relationship(
|
||||
id: string,
|
||||
origin: CatalogRelationship["origin"],
|
||||
status: CatalogRelationship["status"],
|
||||
sourceTableName: string,
|
||||
): CatalogRelationship {
|
||||
const physical = origin === "physical";
|
||||
return {
|
||||
id,
|
||||
databaseId,
|
||||
constraintName: physical ? `${sourceTableName}_user_id_fkey` : null,
|
||||
sourceTableId: `${id}-source-table`,
|
||||
sourceTableName,
|
||||
targetTableId: `${id}-target-table`,
|
||||
targetTableName: "users",
|
||||
updateRule: physical ? "NO ACTION" : null,
|
||||
deleteRule: physical ? "CASCADE" : null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId: `${id}-source-column`,
|
||||
sourceColumnName: "user_id",
|
||||
targetColumnId: `${id}-target-column`,
|
||||
targetColumnName: "id",
|
||||
}],
|
||||
origin,
|
||||
status,
|
||||
lastSyncedDatabaseVersion: physical ? 3 : null,
|
||||
lastSyncedAt: physical ? "2026-08-31T12:00:00.000Z" : null,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
}
|
||||
|
||||
function renderRelationships(rows: CatalogRelationship[], canManage = true) {
|
||||
server.use(http.get(
|
||||
"/api/catalog/databases/:databaseId/relationships",
|
||||
() => HttpResponse.json(rows),
|
||||
));
|
||||
const client = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
||||
const callbacks = {
|
||||
onBackToDatabases: vi.fn(),
|
||||
onOpenOverview: vi.fn(),
|
||||
onOpenTables: vi.fn(),
|
||||
onRunStarted: vi.fn(),
|
||||
onOpenSync: vi.fn(),
|
||||
};
|
||||
const view = render(
|
||||
<QueryClientProvider client={client}>
|
||||
<DatabaseRelationships
|
||||
presentation="fleet"
|
||||
database={database}
|
||||
canManage={canManage}
|
||||
{...callbacks}
|
||||
/>
|
||||
<Toaster duration={Infinity} />
|
||||
</QueryClientProvider>,
|
||||
);
|
||||
return { ...view, client, callbacks };
|
||||
}
|
||||
|
||||
test("shows one relationship map and filters active, excluded, and all relationships", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderRelationships([
|
||||
relationship("physical", "physical", "active", "visits"),
|
||||
relationship("generated", "generated", "active", "orders"),
|
||||
relationship("manual", "manual", "active", "payments"),
|
||||
relationship("excluded", "generated", "excluded", "legacy_orders"),
|
||||
]);
|
||||
|
||||
expect(await screen.findByText("visits")).toBeVisible();
|
||||
expect(screen.getByText("orders")).toBeVisible();
|
||||
expect(screen.getByText("payments")).toBeVisible();
|
||||
expect(screen.queryByText("legacy_orders")).not.toBeInTheDocument();
|
||||
expect(screen.getByText("Physical")).toBeVisible();
|
||||
expect(screen.getByText("Generated")).toBeVisible();
|
||||
expect(screen.getByText("Manual")).toBeVisible();
|
||||
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Relationship status" }), "excluded");
|
||||
expect(await screen.findByText("legacy_orders")).toBeVisible();
|
||||
await waitFor(() => expect(screen.queryByText("visits")).not.toBeInTheDocument());
|
||||
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Relationship status" }), "all");
|
||||
expect(await screen.findByText("visits")).toBeVisible();
|
||||
expect(screen.getByText("legacy_orders")).toBeVisible();
|
||||
});
|
||||
|
||||
test("adds a manual relationship from four catalog selections", async () => {
|
||||
const user = userEvent.setup();
|
||||
const orders: CatalogTable = {
|
||||
id: "22222222-2222-4222-8222-222222222222",
|
||||
databaseId,
|
||||
name: "orders",
|
||||
sourceComment: null,
|
||||
description: null,
|
||||
generatedDescription: null,
|
||||
version: 1,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
const users: CatalogTable = {
|
||||
...orders,
|
||||
id: "33333333-3333-4333-8333-333333333333",
|
||||
name: "users",
|
||||
};
|
||||
const sourceColumn: CatalogColumn = {
|
||||
id: "44444444-4444-4444-8444-444444444444",
|
||||
tableId: orders.id,
|
||||
name: "user_id",
|
||||
ordinalPosition: 1,
|
||||
dataType: "bigint",
|
||||
isNullable: false,
|
||||
defaultExpression: null,
|
||||
primaryKeyPosition: null,
|
||||
isPrimaryKey: false,
|
||||
isForeignKey: false,
|
||||
foreignKeyCount: 0,
|
||||
sourceComment: null,
|
||||
description: null,
|
||||
generatedDescription: null,
|
||||
sensitive: false,
|
||||
lastSyncedDatabaseVersion: 3,
|
||||
lastSyncedAt: "2026-08-31T12:00:00.000Z",
|
||||
version: 1,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
const targetColumn: CatalogColumn = {
|
||||
...sourceColumn,
|
||||
id: "55555555-5555-4555-8555-555555555555",
|
||||
tableId: users.id,
|
||||
name: "id",
|
||||
primaryKeyPosition: 1,
|
||||
isPrimaryKey: true,
|
||||
};
|
||||
let requestBody: unknown;
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([orders, users])),
|
||||
http.get(
|
||||
"/api/catalog/databases/:databaseId/tables/:tableId/columns",
|
||||
({ params }) => HttpResponse.json(params.tableId === orders.id ? [sourceColumn] : [targetColumn]),
|
||||
),
|
||||
http.post("/api/catalog/databases/:databaseId/relationships", async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json(relationship("created", "manual", "active", "orders"), { status: 201 });
|
||||
}),
|
||||
);
|
||||
renderRelationships([]);
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "Add relationship" }));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Add relationship" });
|
||||
expect(drawer).toHaveAttribute("aria-modal", "false");
|
||||
const submit = within(drawer).getByRole("button", { name: "Add relationship" });
|
||||
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Source table" }), orders.id);
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Source column" }), sourceColumn.id);
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Target table" }), users.id);
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Target column" }), targetColumn.id);
|
||||
expect(submit).toBeEnabled();
|
||||
await user.click(submit);
|
||||
|
||||
await waitFor(() => expect(requestBody).toEqual({
|
||||
sourceColumnId: sourceColumn.id,
|
||||
targetColumnId: targetColumn.id,
|
||||
}));
|
||||
expect(await screen.findByText("Relationship added.")).toBeVisible();
|
||||
await waitFor(() => expect(screen.queryByRole("dialog", { name: "Add relationship" })).not.toBeInTheDocument());
|
||||
});
|
||||
|
||||
test("rebuilds generated relationships with progress and a result summary", async () => {
|
||||
const user = userEvent.setup();
|
||||
let finish: (() => void) | undefined;
|
||||
server.use(http.post(
|
||||
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
|
||||
async () => {
|
||||
await new Promise<void>((resolve) => { finish = resolve; });
|
||||
return HttpResponse.json({ added: 12, alreadyPresent: 7, excluded: 3, ambiguous: 2 });
|
||||
},
|
||||
));
|
||||
renderRelationships([]);
|
||||
|
||||
await user.selectOptions(
|
||||
await screen.findByRole("combobox", { name: "Relationship action" }),
|
||||
"rebuild-generated",
|
||||
);
|
||||
await user.click(screen.getByRole("button", { name: "Run action" }));
|
||||
expect(await screen.findByRole("button", { name: "Rebuilding…" })).toBeDisabled();
|
||||
|
||||
finish?.();
|
||||
expect(await screen.findByText(
|
||||
"Rebuild complete: 12 added, 7 already present, 3 excluded, 2 ambiguous.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
test("reports when rebuilding finds no new relationships", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(http.post(
|
||||
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
|
||||
() => HttpResponse.json({ added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 0 }),
|
||||
));
|
||||
renderRelationships([]);
|
||||
|
||||
await user.selectOptions(
|
||||
await screen.findByRole("combobox", { name: "Relationship action" }),
|
||||
"rebuild-generated",
|
||||
);
|
||||
await user.click(screen.getByRole("button", { name: "Run action" }));
|
||||
expect(await screen.findByText("Rebuild complete: no new relationships found.")).toBeVisible();
|
||||
});
|
||||
|
||||
test("excludes a generated relationship after an explanatory drawer confirmation", async () => {
|
||||
const user = userEvent.setup();
|
||||
const generated = relationship("generated", "generated", "active", "orders");
|
||||
let requestBody: unknown;
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json({ ...generated, status: "excluded" });
|
||||
},
|
||||
));
|
||||
renderRelationships([generated]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship orders.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
expect(within(drawer).getByText("Generated")).toBeVisible();
|
||||
await user.click(within(drawer).getByRole("button", { name: "Exclude" }));
|
||||
expect(within(drawer).getByText(
|
||||
"The relationship will move to Excluded. Rebuilds will not recreate it, and you can restore it later.",
|
||||
)).toBeVisible();
|
||||
const confirmExclude = within(drawer).getByRole("button", { name: "Exclude relationship" });
|
||||
await waitFor(() => expect(confirmExclude).toHaveFocus());
|
||||
await user.click(confirmExclude);
|
||||
|
||||
await waitFor(() => expect(requestBody).toEqual({ status: "excluded" }));
|
||||
expect(await screen.findByText(
|
||||
"Relationship excluded. Future rebuilds will leave it excluded.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
test("keeps physical relationships read-only in the details drawer", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderRelationships([relationship("physical", "physical", "active", "visits")]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship visits.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
expect(within(drawer).getByText("This information is synchronized from the source schema and is read-only.")).toBeVisible();
|
||||
expect(within(drawer).queryByRole("button", { name: "Exclude" })).not.toBeInTheDocument();
|
||||
expect(within(drawer).queryByRole("button", { name: "Restore" })).not.toBeInTheDocument();
|
||||
expect(within(drawer).queryByRole("button", { name: "Delete permanently" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("permanently deletes a logical relationship after warning that rebuild may recreate it", async () => {
|
||||
const user = userEvent.setup();
|
||||
const manual = relationship("manual", "manual", "active", "payments");
|
||||
let deletes = 0;
|
||||
server.use(http.delete(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
() => {
|
||||
deletes += 1;
|
||||
return new HttpResponse(null, { status: 204 });
|
||||
},
|
||||
));
|
||||
renderRelationships([manual]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship payments.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
await user.click(within(drawer).getByRole("button", { name: "Delete permanently" }));
|
||||
expect(within(drawer).getByText(
|
||||
"The relationship record will be erased. A future rebuild may infer it again.",
|
||||
)).toBeVisible();
|
||||
await user.click(within(drawer).getByRole("button", { name: "Delete permanently" }));
|
||||
|
||||
await waitFor(() => expect(deletes).toBe(1));
|
||||
expect(await screen.findByText(
|
||||
"Relationship deleted permanently. A future rebuild may infer it again.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
test("restores an excluded logical relationship", async () => {
|
||||
const user = userEvent.setup();
|
||||
const excluded = relationship("excluded", "generated", "excluded", "legacy_orders");
|
||||
let requestBody: unknown;
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json({ ...excluded, status: "active" });
|
||||
},
|
||||
));
|
||||
renderRelationships([excluded]);
|
||||
|
||||
await user.selectOptions(
|
||||
await screen.findByRole("combobox", { name: "Relationship status" }),
|
||||
"excluded",
|
||||
);
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship legacy_orders.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
await user.click(within(drawer).getByRole("button", { name: "Restore" }));
|
||||
|
||||
await waitFor(() => expect(requestBody).toEqual({ status: "active" }));
|
||||
expect(await screen.findByText("Relationship restored.")).toBeVisible();
|
||||
});
|
||||
|
||||
test("keeps relationship context open when a logical mutation fails", async () => {
|
||||
const user = userEvent.setup();
|
||||
const generated = relationship("generated-error", "generated", "active", "orders");
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
() => HttpResponse.json({ code: "relationship_not_found" }, { status: 404 }),
|
||||
));
|
||||
renderRelationships([generated]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship orders.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
await user.click(within(drawer).getByRole("button", { name: "Exclude" }));
|
||||
await user.click(within(drawer).getByRole("button", { name: "Exclude relationship" }));
|
||||
|
||||
expect(await screen.findByText("The relationship no longer exists. Refresh and try again.")).toBeVisible();
|
||||
expect(screen.getByRole("dialog", { name: "Relationship details" })).toBeVisible();
|
||||
});
|
||||
@@ -1,26 +1,19 @@
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { useMemo, useRef, useState, type ReactNode } from "react";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
import type { ColDef, ICellRendererParams } from "ag-grid-community";
|
||||
import { ArrowLeft, History, Info, Play, Plus, RefreshCw } from "lucide-react";
|
||||
import { ArrowLeft, History, Info, Play, RefreshCw } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import { Button } from "../../components/ui/button";
|
||||
import { ApiError, apiErrorMessage } from "../../api/client";
|
||||
import { apiErrorMessage } from "../../api/client";
|
||||
import {
|
||||
createCatalogRelationship,
|
||||
deleteCatalogRelationship,
|
||||
listCatalogRelationships,
|
||||
listCatalogColumns,
|
||||
listCatalogTables,
|
||||
rebuildGeneratedRelationships,
|
||||
setCatalogRelationshipStatus,
|
||||
startCatalogSync,
|
||||
type CatalogDatabase,
|
||||
type CatalogRelationship,
|
||||
type CatalogSyncRun,
|
||||
} from "../../api/catalog-databases";
|
||||
import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
import type { DatabaseNavigationState } from "./model";
|
||||
|
||||
interface Props {
|
||||
database: CatalogDatabase;
|
||||
@@ -30,10 +23,7 @@ interface Props {
|
||||
onOpenTables: () => void;
|
||||
onRunStarted: (run: CatalogSyncRun) => void;
|
||||
onOpenSync: () => void;
|
||||
onOpenDescriptionHistory?: () => void;
|
||||
onOpenSensitiveHistory?: () => void;
|
||||
onCatalogMetricsChanged?: () => void | Promise<void>;
|
||||
onNavigationStateChange?: (state: DatabaseNavigationState) => void;
|
||||
metadataModelControl?: ReactNode;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
|
||||
@@ -42,19 +32,6 @@ interface RelationshipGridContext {
|
||||
onView: (relationship: CatalogRelationship, origin: HTMLButtonElement) => void;
|
||||
}
|
||||
|
||||
type RelationshipStatusFilter = "active" | "excluded" | "all";
|
||||
|
||||
function titleCase(value: string): string {
|
||||
return value.charAt(0).toUpperCase() + value.slice(1);
|
||||
}
|
||||
|
||||
function relationshipLabel(relationship: CatalogRelationship): string {
|
||||
const pair = relationship.columns[0];
|
||||
const source = `${relationship.sourceTableName}${pair ? `.${pair.sourceColumnName}` : ""}`;
|
||||
const target = `${relationship.targetTableName}${pair ? `.${pair.targetColumnName}` : ""}`;
|
||||
return `${source} to ${target}`;
|
||||
}
|
||||
|
||||
function RelationshipActionsCell({ data, context }: ICellRendererParams<CatalogRelationship, unknown, RelationshipGridContext>) {
|
||||
if (!data || !context || context.presentation !== "fleet") return null;
|
||||
return (
|
||||
@@ -65,7 +42,7 @@ function RelationshipActionsCell({ data, context }: ICellRendererParams<CatalogR
|
||||
size="icon-lg"
|
||||
className="thot-fleet-icon-action"
|
||||
data-tooltip="Details"
|
||||
aria-label={`View relationship ${relationshipLabel(data)}`}
|
||||
aria-label={`View relationship ${data.constraintName}`}
|
||||
onClick={(event) => context.onView(data, event.currentTarget)}
|
||||
>
|
||||
<Info aria-hidden="true" />
|
||||
@@ -82,86 +59,32 @@ export function DatabaseRelationships({
|
||||
onOpenTables,
|
||||
onRunStarted,
|
||||
onOpenSync,
|
||||
onOpenDescriptionHistory,
|
||||
onOpenSensitiveHistory,
|
||||
onCatalogMetricsChanged,
|
||||
onNavigationStateChange,
|
||||
presentation = "legacy",
|
||||
}: Props) {
|
||||
const databaseId = database.id!;
|
||||
const queryClient = useQueryClient();
|
||||
const { data = [], error, isError, isLoading, isFetching, refetch } = useQuery({
|
||||
queryKey: ["catalog-relationships", databaseId],
|
||||
queryFn: () => listCatalogRelationships(databaseId),
|
||||
retry: false,
|
||||
});
|
||||
const [search, setSearch] = useState("");
|
||||
const [statusFilter, setStatusFilter] = useState<RelationshipStatusFilter>("active");
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [selectedAction, setSelectedAction] = useState<"" | "rebuild-generated" | "relationships" | "all">("");
|
||||
const [selectedAction, setSelectedAction] = useState<"" | "relationships" | "all">("");
|
||||
const [activeRelationshipId, setActiveRelationshipId] = useState<string | null>(null);
|
||||
const [addOpen, setAddOpen] = useState(false);
|
||||
const [sourceTableId, setSourceTableId] = useState("");
|
||||
const [sourceColumnId, setSourceColumnId] = useState("");
|
||||
const [targetTableId, setTargetTableId] = useState("");
|
||||
const [targetColumnId, setTargetColumnId] = useState("");
|
||||
const [addError, setAddError] = useState<string | null>(null);
|
||||
const [targetColumnError, setTargetColumnError] = useState<string | null>(null);
|
||||
const [pendingMutation, setPendingMutation] = useState<"exclude" | "delete" | null>(null);
|
||||
const originRef = useRef<HTMLButtonElement | null>(null);
|
||||
const statusFilterRef = useRef<HTMLSelectElement | null>(null);
|
||||
const bindingReady = database.connectionStatus === "reachable" && database.testedVersion === database.version;
|
||||
const activeRelationship = activeRelationshipId
|
||||
? data.find((relationship) => relationship.id === activeRelationshipId)
|
||||
: undefined;
|
||||
const visibleRelationships = useMemo(() => statusFilter === "all"
|
||||
? data
|
||||
: data.filter((relationship) => relationship.status === statusFilter), [data, statusFilter]);
|
||||
const tablesQuery = useQuery({
|
||||
queryKey: ["catalog-tables", databaseId],
|
||||
queryFn: () => listCatalogTables(databaseId),
|
||||
enabled: addOpen,
|
||||
retry: false,
|
||||
});
|
||||
const sourceColumnsQuery = useQuery({
|
||||
queryKey: ["catalog-columns", databaseId, sourceTableId],
|
||||
queryFn: () => listCatalogColumns(databaseId, sourceTableId),
|
||||
enabled: addOpen && Boolean(sourceTableId),
|
||||
retry: false,
|
||||
});
|
||||
const targetColumnsQuery = useQuery({
|
||||
queryKey: ["catalog-columns", databaseId, targetTableId],
|
||||
queryFn: () => listCatalogColumns(databaseId, targetTableId),
|
||||
enabled: addOpen && Boolean(targetTableId),
|
||||
retry: false,
|
||||
});
|
||||
const targetColumns = (targetColumnsQuery.data ?? []).filter((column) => column.isPrimaryKey);
|
||||
useEffect(() => {
|
||||
if (!pendingMutation) return;
|
||||
const timer = window.setTimeout(() => document
|
||||
.getElementById("relationship-mutation-confirm-button")?.focus(), 0);
|
||||
return () => window.clearTimeout(timer);
|
||||
}, [pendingMutation]);
|
||||
useEffect(() => {
|
||||
onNavigationStateChange?.({ dirty: false, busy });
|
||||
return () => onNavigationStateChange?.({ dirty: false, busy: false });
|
||||
}, [busy, onNavigationStateChange]);
|
||||
const selectedActionUnavailable = !selectedAction
|
||||
? null
|
||||
: !canManage
|
||||
? "You do not have permission to manage catalog relationships."
|
||||
: database.activeSyncRun
|
||||
? "Wait for the active synchronization to finish."
|
||||
: selectedAction !== "rebuild-generated" && !bindingReady
|
||||
? "Test the current database binding first."
|
||||
? "You do not have permission to synchronize the catalog."
|
||||
: !bindingReady
|
||||
? "Test the current database binding first."
|
||||
: database.activeSyncRun
|
||||
? "Wait for the active synchronization to finish."
|
||||
: null;
|
||||
const addUnavailable = !canManage
|
||||
? "You do not have permission to add catalog relationships."
|
||||
: database.activeSyncRun
|
||||
? "Wait for the active synchronization to finish."
|
||||
: busy
|
||||
? "Wait for the current relationship operation to finish."
|
||||
: null;
|
||||
|
||||
const synchronize = async (scope: "relationships" | "all") => {
|
||||
setBusy(true);
|
||||
@@ -174,146 +97,28 @@ export function DatabaseRelationships({
|
||||
|
||||
const openRelationship = (relationship: CatalogRelationship, origin: HTMLButtonElement) => {
|
||||
originRef.current = origin;
|
||||
setPendingMutation(null);
|
||||
setAddOpen(false);
|
||||
setActiveRelationshipId(relationship.id);
|
||||
};
|
||||
|
||||
const closeRelationship = () => {
|
||||
if (busy) return;
|
||||
setPendingMutation(null);
|
||||
setActiveRelationshipId(null);
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) originRef.current.focus();
|
||||
else statusFilterRef.current?.focus();
|
||||
}, 0);
|
||||
};
|
||||
|
||||
const changeRelationshipStatus = async (status: "active" | "excluded") => {
|
||||
if (!activeRelationship || activeRelationship.origin === "physical") return;
|
||||
setBusy(true);
|
||||
try {
|
||||
await setCatalogRelationshipStatus(databaseId, activeRelationship.id, status);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
setPendingMutation(null);
|
||||
setActiveRelationshipId(null);
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) originRef.current.focus();
|
||||
else statusFilterRef.current?.focus();
|
||||
}, 0);
|
||||
toast.success(status === "excluded"
|
||||
? "Relationship excluded. Future rebuilds will leave it excluded."
|
||||
: "Relationship restored.");
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
const permanentlyDeleteRelationship = async () => {
|
||||
if (!activeRelationship || activeRelationship.origin === "physical") return;
|
||||
setBusy(true);
|
||||
try {
|
||||
await deleteCatalogRelationship(databaseId, activeRelationship.id);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
setPendingMutation(null);
|
||||
setActiveRelationshipId(null);
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) originRef.current.focus();
|
||||
else statusFilterRef.current?.focus();
|
||||
}, 0);
|
||||
toast.success("Relationship deleted permanently. A future rebuild may infer it again.");
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
const resetAddForm = () => {
|
||||
setSourceTableId("");
|
||||
setSourceColumnId("");
|
||||
setTargetTableId("");
|
||||
setTargetColumnId("");
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
};
|
||||
|
||||
const closeAdd = () => {
|
||||
if (busy) return;
|
||||
setAddOpen(false);
|
||||
resetAddForm();
|
||||
};
|
||||
|
||||
const addRelationship = async () => {
|
||||
if (!sourceColumnId || !targetColumnId) return;
|
||||
setBusy(true);
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
try {
|
||||
await createCatalogRelationship(databaseId, sourceColumnId, targetColumnId);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
setAddOpen(false);
|
||||
resetAddForm();
|
||||
toast.success("Relationship added.");
|
||||
} catch (error) {
|
||||
if (error instanceof ApiError && error.code === "relationship_target_not_unique") {
|
||||
setTargetColumnError("The target column must be the only primary-key column of its table.");
|
||||
} else if (error instanceof ApiError && error.code === "relationship_type_incompatible") {
|
||||
setTargetColumnError("Source and target column types are not compatible.");
|
||||
} else if (error instanceof ApiError && error.code === "relationship_duplicate") {
|
||||
setAddError("This relationship already exists.");
|
||||
} else {
|
||||
setAddError(apiErrorMessage(error));
|
||||
}
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
window.setTimeout(() => originRef.current?.focus(), 0);
|
||||
};
|
||||
|
||||
const runSelectedAction = async () => {
|
||||
if (!selectedAction) return;
|
||||
if (selectedAction === "rebuild-generated") {
|
||||
setBusy(true);
|
||||
try {
|
||||
const result = await rebuildGeneratedRelationships(databaseId);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
const noFindings = result.added === 0
|
||||
&& result.alreadyPresent === 0
|
||||
&& result.excluded === 0
|
||||
&& result.ambiguous === 0;
|
||||
toast.success(noFindings
|
||||
? "Rebuild complete: no new relationships found."
|
||||
: `Rebuild complete: ${result.added} added, ${result.alreadyPresent} already present, ${result.excluded} excluded, ${result.ambiguous} ambiguous.`);
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
} else {
|
||||
await synchronize(selectedAction);
|
||||
}
|
||||
await synchronize(selectedAction);
|
||||
setSelectedAction("");
|
||||
};
|
||||
|
||||
const columns = useMemo<ColDef<CatalogRelationship>[]>(() => [
|
||||
...(presentation === "legacy" ? [{ field: "constraintName", headerName: "Constraint", minWidth: 220, flex: 1, cellClass: "font-mono text-xs" } satisfies ColDef<CatalogRelationship>] : []),
|
||||
{ field: "constraintName", headerName: "Constraint", minWidth: 220, flex: 1, cellClass: "font-mono text-xs" },
|
||||
{ field: "sourceTableName", headerName: "Source table", minWidth: 200, flex: 1 },
|
||||
{ headerName: "Source column", minWidth: 180, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.sourceColumnName).join(", ") ?? "" },
|
||||
{ headerName: "Source columns", minWidth: 200, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.sourceColumnName).join(", ") ?? "" },
|
||||
{ field: "targetTableName", headerName: "Target table", minWidth: 200, flex: 1 },
|
||||
{ headerName: "Target column", minWidth: 180, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.targetColumnName).join(", ") ?? "" },
|
||||
{ field: "origin", headerName: "Origin", minWidth: 120, width: 120, valueFormatter: ({ value }) => titleCase(String(value ?? "")) },
|
||||
{ field: "status", headerName: "Status", minWidth: 110, width: 110, valueFormatter: ({ value }) => titleCase(String(value ?? "")) },
|
||||
...(presentation === "legacy" ? [
|
||||
{ field: "updateRule", headerName: "On update", minWidth: 125, width: 125 },
|
||||
{ field: "deleteRule", headerName: "On delete", minWidth: 125, width: 125 },
|
||||
] satisfies ColDef<CatalogRelationship>[] : []),
|
||||
{ headerName: "Target columns", minWidth: 200, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.targetColumnName).join(", ") ?? "" },
|
||||
{ field: "updateRule", headerName: "On update", minWidth: 125, width: 125 },
|
||||
{ field: "deleteRule", headerName: "On delete", minWidth: 125, width: 125 },
|
||||
...(presentation === "fleet" ? [{ colId: "actions", headerName: "Actions", width: 64, minWidth: 64, maxWidth: 64, pinned: "right" as const, lockPinned: true, sortable: false, filter: false, resizable: false, suppressMovable: true, suppressHeaderMenuButton: true, cellRenderer: RelationshipActionsCell }] : []),
|
||||
], [presentation]);
|
||||
const context = useMemo<RelationshipGridContext>(() => ({ presentation, onView: openRelationship }), [data, presentation]);
|
||||
@@ -352,27 +157,14 @@ export function DatabaseRelationships({
|
||||
</div> : null}
|
||||
<div className={presentation === "fleet" ? "thot-fleet-grid-surface flex min-h-0 flex-1 flex-col overflow-hidden bg-card" : "mx-3 mb-4 mt-4 flex min-h-0 flex-1 flex-col overflow-hidden rounded-md border border-border bg-card sm:mx-5"}>
|
||||
<div className={`${presentation === "fleet" ? "thot-fleet-grid-toolbar " : ""}flex min-h-12 flex-wrap items-center gap-3 border-b border-border px-3 py-2`}>
|
||||
<span className="thot-label whitespace-nowrap">Relationship map</span>
|
||||
<label className="sr-only" htmlFor="relationship-status-filter">Relationship status</label>
|
||||
<select
|
||||
ref={statusFilterRef}
|
||||
id="relationship-status-filter"
|
||||
className="h-8 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Relationship status"
|
||||
value={statusFilter}
|
||||
onChange={(event) => setStatusFilter(event.target.value as RelationshipStatusFilter)}
|
||||
>
|
||||
<option value="active">Active</option>
|
||||
<option value="excluded">Excluded</option>
|
||||
<option value="all">All</option>
|
||||
</select>
|
||||
<span className="thot-label whitespace-nowrap">Physical relationships</span>
|
||||
<input className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search relationships" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{visibleRelationships.length}</span>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{data.length}</span>
|
||||
<Button type="button" variant="outline" disabled={isFetching || busy} onClick={() => void refetch()}><RefreshCw className={isFetching ? "animate-spin" : ""} />Refresh</Button>
|
||||
{presentation === "fleet" ? (
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<div className="thot-fleet-action-selector__scope" aria-live="polite">
|
||||
<strong>{selectedAction === "relationships" ? "Physical relationships in this database" : "This database"}</strong>
|
||||
<strong>{selectedAction === "all" ? "This database" : selectedAction === "relationships" ? "All relationships in this database" : "This database"}</strong>
|
||||
<span>Action scope</span>
|
||||
</div>
|
||||
<label className="sr-only" htmlFor="relationship-action">Relationship action</label>
|
||||
@@ -381,212 +173,45 @@ export function DatabaseRelationships({
|
||||
className="thot-fleet-action-selector__select"
|
||||
value={selectedAction}
|
||||
disabled={busy}
|
||||
onChange={(event) => setSelectedAction(event.target.value as "" | "rebuild-generated" | "relationships" | "all")}
|
||||
onChange={(event) => setSelectedAction(event.target.value as "" | "relationships" | "all")}
|
||||
>
|
||||
<option value="">Choose an action…</option>
|
||||
<optgroup label="Logical relationships">
|
||||
<option value="rebuild-generated">Rebuild generated relationships</option>
|
||||
</optgroup>
|
||||
<optgroup label="Synchronization">
|
||||
<option value="relationships">Synchronize physical relationships</option>
|
||||
<option value="all">Synchronize the full database schema</option>
|
||||
<option value="relationships">Synchronize all relationships in this database</option>
|
||||
<option value="all">Synchronize the full schema for this database</option>
|
||||
</optgroup>
|
||||
</select>
|
||||
<Button type="button" size="sm" disabled={!selectedAction || Boolean(selectedActionUnavailable) || busy} aria-describedby={selectedActionUnavailable ? "relationship-action-reason" : undefined} onClick={() => void runSelectedAction()}><Play aria-hidden="true" />{busy ? selectedAction === "rebuild-generated" ? "Rebuilding…" : "Running…" : "Run action"}</Button>
|
||||
<Button type="button" size="sm" disabled={!selectedAction || Boolean(selectedActionUnavailable) || busy} aria-describedby={selectedActionUnavailable ? "relationship-action-reason" : undefined} onClick={() => void runSelectedAction()}><Play aria-hidden="true" />{busy ? "Running…" : "Run action"}</Button>
|
||||
<Button type="button" size="sm" variant="outline" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Synchronization history" onClick={onOpenSync}>
|
||||
{database.activeSyncRun ? <RefreshCw className="animate-spin" aria-hidden="true" /> : <History aria-hidden="true" />} Sync history
|
||||
</Button>
|
||||
{onOpenDescriptionHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenDescriptionHistory}><History aria-hidden="true" /> Description history</Button> : null}
|
||||
{onOpenSensitiveHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenSensitiveHistory}><History aria-hidden="true" /> Sensitive history</Button> : null}
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
disabled={Boolean(addUnavailable)}
|
||||
aria-describedby={addUnavailable ? "relationship-add-reason" : undefined}
|
||||
onClick={() => {
|
||||
setActiveRelationshipId(null);
|
||||
setPendingMutation(null);
|
||||
setAddOpen(true);
|
||||
}}
|
||||
>
|
||||
<Plus aria-hidden="true" /> Add relationship
|
||||
</Button>
|
||||
{addUnavailable ? <span id="relationship-add-reason" className="sr-only">{addUnavailable}</span> : null}
|
||||
{selectedActionUnavailable ? <span id="relationship-action-reason" className="basis-full text-xs text-muted-foreground" role="status">{selectedActionUnavailable}</span> : null}
|
||||
</div>
|
||||
) : <Button type="button" disabled={!canManage || !bindingReady || busy || Boolean(database.activeSyncRun)} title={!bindingReady ? "Test the current database binding before synchronizing relationships" : undefined} onClick={() => void synchronize("relationships")}><RefreshCw />Synchronize relationships</Button>}
|
||||
) : <Button type="button" disabled={!canManage || !bindingReady || busy || Boolean(database.activeSyncRun)} title={!bindingReady ? "Test the current database binding before synchronizing relationships" : undefined} onClick={() => void synchronize("relationships")}><RefreshCw />Sync relationships</Button>}
|
||||
</div>
|
||||
{!bindingReady ? <div className="border-b border-border bg-muted/35 px-4 py-3 text-sm text-muted-foreground">{presentation === "fleet" ? "Test the current database binding in database details before synchronizing relationships." : "Test the current database binding from Overview before synchronizing relationships."}</div> : null}
|
||||
{fleetQueryError ?? <div className="relative min-h-[280px] flex-1">
|
||||
<div className={`thot-database-grid ${presentation === "fleet" ? "thot-fleet-ledger-grid " : ""}ag-theme-alpine absolute inset-0 h-full w-full`}>
|
||||
<AgGridReact<CatalogRelationship> rowData={visibleRelationships} columnDefs={columns} context={context} loading={isLoading} quickFilterText={search} defaultColDef={{ sortable: true, filter: true, resizable: true }} getRowId={({ data: row }) => row.id} rowHeight={44} headerHeight={38} animateRows={false} overlayNoRowsTemplate={statusFilter === "active" ? "No relationships yet. Rebuild from column names or add one manually." : statusFilter === "excluded" ? "No excluded relationships." : "No relationships for this database."} />
|
||||
<AgGridReact<CatalogRelationship> rowData={data} columnDefs={columns} context={context} loading={isLoading} quickFilterText={search} defaultColDef={{ sortable: true, filter: true, resizable: true }} getRowId={({ data: row }) => row.id} rowHeight={44} headerHeight={38} animateRows={false} overlayNoRowsTemplate="No physical relationships synchronized for this database." />
|
||||
</div>
|
||||
</div>}
|
||||
</div>
|
||||
{presentation === "fleet" && addOpen ? (
|
||||
<FleetLedgerDrawer
|
||||
open
|
||||
eyebrow="Logical relationship"
|
||||
title="Add relationship"
|
||||
description="Connect one source column to one primary-key target column."
|
||||
onClose={closeAdd}
|
||||
closeLabel="Close relationship form"
|
||||
busy={busy}
|
||||
footer={(
|
||||
<>
|
||||
<Button type="button" variant="outline" disabled={busy} onClick={closeAdd}>Cancel</Button>
|
||||
<Button
|
||||
type="button"
|
||||
disabled={busy || !sourceTableId || !sourceColumnId || !targetTableId || !targetColumnId}
|
||||
onClick={() => void addRelationship()}
|
||||
>
|
||||
{busy ? "Adding…" : "Add relationship"}
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
>
|
||||
<div className="grid gap-4 text-sm sm:grid-cols-2">
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Source table</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Source table"
|
||||
value={sourceTableId}
|
||||
disabled={busy || tablesQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setSourceTableId(event.target.value);
|
||||
setSourceColumnId("");
|
||||
setAddError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select source table…</option>
|
||||
{(tablesQuery.data ?? []).map((table) => <option key={table.id} value={table.id}>{table.name}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Source column</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Source column"
|
||||
value={sourceColumnId}
|
||||
disabled={busy || !sourceTableId || sourceColumnsQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setSourceColumnId(event.target.value);
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select source column…</option>
|
||||
{(sourceColumnsQuery.data ?? []).map((column) => <option key={column.id} value={column.id}>{column.name}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Target table</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Target table"
|
||||
value={targetTableId}
|
||||
disabled={busy || tablesQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setTargetTableId(event.target.value);
|
||||
setTargetColumnId("");
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select target table…</option>
|
||||
{(tablesQuery.data ?? []).map((table) => <option key={table.id} value={table.id}>{table.name}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Target column</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Target column"
|
||||
aria-invalid={Boolean(targetColumnError)}
|
||||
aria-describedby={targetColumnError ? "target-column-error" : undefined}
|
||||
value={targetColumnId}
|
||||
disabled={busy || !targetTableId || targetColumnsQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setTargetColumnId(event.target.value);
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select target column…</option>
|
||||
{targetColumns.map((column) => <option key={column.id} value={column.id}>{column.name}</option>)}
|
||||
</select>
|
||||
{targetColumnError ? <span id="target-column-error" className="text-xs text-destructive" role="alert">{targetColumnError}</span> : null}
|
||||
</label>
|
||||
{tablesQuery.isError || sourceColumnsQuery.isError || targetColumnsQuery.isError ? (
|
||||
<p className="sm:col-span-2 text-sm text-destructive" role="alert">Catalog tables or columns could not be loaded.</p>
|
||||
) : null}
|
||||
{addError ? <p className="sm:col-span-2 text-sm text-destructive" role="alert">{addError}</p> : null}
|
||||
</div>
|
||||
</FleetLedgerDrawer>
|
||||
) : null}
|
||||
{presentation === "fleet" && activeRelationship ? (
|
||||
<FleetLedgerDrawer
|
||||
open
|
||||
eyebrow={`${titleCase(activeRelationship.origin)} relationship`}
|
||||
eyebrow="Physical relationship"
|
||||
title="Relationship details"
|
||||
description={activeRelationship.origin === "physical"
|
||||
? "This information is synchronized from the source schema and is read-only."
|
||||
: activeRelationship.origin === "generated"
|
||||
? "This relationship was inferred from catalog column names."
|
||||
: "This relationship was added manually."}
|
||||
description="This information is synchronized from the source schema and is read-only."
|
||||
onClose={closeRelationship}
|
||||
closeLabel="Close relationship details"
|
||||
busy={busy}
|
||||
footer={pendingMutation ? (
|
||||
<div
|
||||
className="flex min-w-0 flex-1 flex-wrap items-center gap-2"
|
||||
role="group"
|
||||
aria-labelledby="relationship-mutation-confirmation"
|
||||
onKeyDown={(event) => {
|
||||
if (event.key === "Escape" && !busy) setPendingMutation(null);
|
||||
}}
|
||||
>
|
||||
<p id="relationship-mutation-confirmation" className="mr-auto max-w-xl text-sm text-muted-foreground">
|
||||
{pendingMutation === "exclude"
|
||||
? "The relationship will move to Excluded. Rebuilds will not recreate it, and you can restore it later."
|
||||
: "The relationship record will be erased. A future rebuild may infer it again."}
|
||||
</p>
|
||||
<Button type="button" variant="ghost" disabled={busy} onClick={() => setPendingMutation(null)}>Cancel</Button>
|
||||
<Button
|
||||
id="relationship-mutation-confirm-button"
|
||||
type="button"
|
||||
variant={pendingMutation === "delete" ? "destructive" : "default"}
|
||||
disabled={busy}
|
||||
onClick={() => void (pendingMutation === "exclude"
|
||||
? changeRelationshipStatus("excluded")
|
||||
: permanentlyDeleteRelationship())}
|
||||
>
|
||||
{pendingMutation === "exclude" ? "Exclude relationship" : "Delete permanently"}
|
||||
</Button>
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<Button type="button" variant="outline" disabled={busy} onClick={closeRelationship}>Close</Button>
|
||||
{activeRelationship.origin !== "physical" && activeRelationship.status === "active" ? (
|
||||
<Button type="button" variant="outline" disabled={!canManage || busy} onClick={() => setPendingMutation("exclude")}>Exclude</Button>
|
||||
) : null}
|
||||
{activeRelationship.origin !== "physical" && activeRelationship.status === "excluded" ? (
|
||||
<Button type="button" disabled={!canManage || busy} onClick={() => void changeRelationshipStatus("active")}>Restore</Button>
|
||||
) : null}
|
||||
{activeRelationship.origin !== "physical" ? (
|
||||
<Button type="button" variant="destructive" disabled={!canManage || busy} onClick={() => setPendingMutation("delete")}>Delete permanently</Button>
|
||||
) : null}
|
||||
</>
|
||||
)}
|
||||
footer={<Button type="button" variant="outline" onClick={closeRelationship}>Close</Button>}
|
||||
>
|
||||
<dl className="grid gap-4 text-sm sm:grid-cols-2">
|
||||
<div><dt className="thot-label">Origin</dt><dd className="mt-1">{titleCase(activeRelationship.origin)}</dd></div>
|
||||
<div><dt className="thot-label">Status</dt><dd className="mt-1">{titleCase(activeRelationship.status)}</dd></div>
|
||||
{activeRelationship.constraintName ? <div className="sm:col-span-2"><dt className="thot-label">Constraint</dt><dd className="mt-1 break-words font-mono text-xs">{activeRelationship.constraintName}</dd></div> : null}
|
||||
<div className="sm:col-span-2"><dt className="thot-label">Constraint</dt><dd className="mt-1 break-words font-mono text-xs">{activeRelationship.constraintName}</dd></div>
|
||||
<div><dt className="thot-label">Source table</dt><dd className="mt-1 break-words">{activeRelationship.sourceTableName}</dd></div>
|
||||
<div><dt className="thot-label">Target table</dt><dd className="mt-1 break-words">{activeRelationship.targetTableName}</dd></div>
|
||||
{activeRelationship.origin === "physical" ? <><div><dt className="thot-label">On update</dt><dd className="mt-1">{activeRelationship.updateRule ?? "Not specified"}</dd></div><div><dt className="thot-label">On delete</dt><dd className="mt-1">{activeRelationship.deleteRule ?? "Not specified"}</dd></div></> : null}
|
||||
<div><dt className="thot-label">On update</dt><dd className="mt-1">{activeRelationship.updateRule ?? "Not specified"}</dd></div>
|
||||
<div><dt className="thot-label">On delete</dt><dd className="mt-1">{activeRelationship.deleteRule ?? "Not specified"}</dd></div>
|
||||
<div className="sm:col-span-2">
|
||||
<dt className="thot-label">Column mapping</dt>
|
||||
<dd className="mt-2 overflow-hidden rounded-md border border-border">
|
||||
|
||||
@@ -3,15 +3,14 @@ import { ChevronDown, RefreshCw } from "lucide-react";
|
||||
import { buttonVariants } from "../../components/ui/button";
|
||||
import type { CatalogSyncScope } from "../../api/catalog-databases";
|
||||
|
||||
export type DatabaseSyncScope = Exclude<CatalogSyncScope, "columns">;
|
||||
|
||||
export const databaseSyncItemClass = [
|
||||
"flex cursor-default select-none items-center rounded-md px-3 py-2 text-sm outline-none",
|
||||
"data-[highlighted]:bg-muted data-[disabled]:opacity-45",
|
||||
].join(" ");
|
||||
|
||||
export const databaseSyncScopes: ReadonlyArray<{ scope: DatabaseSyncScope; label: string }> = [
|
||||
export const databaseSyncScopes: ReadonlyArray<{ scope: CatalogSyncScope; label: string }> = [
|
||||
{ scope: "tables", label: "Synchronize tables" },
|
||||
{ scope: "columns", label: "Synchronize all columns" },
|
||||
{ scope: "relationships", label: "Synchronize relationships" },
|
||||
{ scope: "all", label: "Synchronize all" },
|
||||
];
|
||||
@@ -19,7 +18,7 @@ export const databaseSyncScopes: ReadonlyArray<{ scope: DatabaseSyncScope; label
|
||||
interface DatabaseSyncMenuProps {
|
||||
disabled: boolean;
|
||||
disabledReason?: string;
|
||||
onSelect: (scope: DatabaseSyncScope) => void;
|
||||
onSelect: (scope: CatalogSyncScope) => void;
|
||||
}
|
||||
|
||||
export function DatabaseSyncMenu({ disabled, disabledReason, onSelect }: DatabaseSyncMenuProps) {
|
||||
|
||||
@@ -1,16 +1,15 @@
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useEffect, useMemo, useRef, useState, type ReactNode } from "react";
|
||||
import { Menu } from "@base-ui/react/menu";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
import type { ColDef, ICellRendererParams } from "ag-grid-community";
|
||||
import { ArrowLeft, ChevronDown, Columns3, Pencil, RefreshCw, Save, Sparkles, Trash2, X } from "lucide-react";
|
||||
import { ArrowLeft, ChevronDown, Columns3, History, Pencil, RefreshCw, Save, Sparkles, Trash2, X } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import { Button } from "../../components/ui/button";
|
||||
import { ApiError, apiErrorMessage } from "../../api/client";
|
||||
import {
|
||||
consolidateCatalogDescriptions,
|
||||
deleteCatalogTableMetadata,
|
||||
listCatalogColumns,
|
||||
listCatalogTables,
|
||||
startCatalogSync,
|
||||
startDescriptionGenerationRun,
|
||||
@@ -26,7 +25,6 @@ import type { DatabaseNavigationState } from "./model";
|
||||
import { DatabaseColumns } from "./DatabaseColumns";
|
||||
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
|
||||
import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
|
||||
|
||||
interface Props {
|
||||
database: CatalogDatabase;
|
||||
@@ -38,13 +36,13 @@ interface Props {
|
||||
onOpenOverview: () => void;
|
||||
onOpenRelationships: () => void;
|
||||
onNavigationStateChange: (state: DatabaseNavigationState) => void;
|
||||
onNestedNavigationChange?: (active: boolean, onBack?: () => void) => void;
|
||||
onNestedNavigationChange?: (active: boolean) => void;
|
||||
onRunStarted: (run: CatalogSyncRun) => void;
|
||||
onOpenSync: () => void;
|
||||
onClearCatalogTables?: () => Promise<void>;
|
||||
onDescriptionGenerationRunStarted: (run: DescriptionGenerationRun) => void;
|
||||
onSuggestSensitive: (selection: SensitiveDataSuggestionRequest, scopeLabel: string) => Promise<void>;
|
||||
onCatalogMetricsChanged?: () => void | Promise<void>;
|
||||
metadataModelControl?: ReactNode;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
|
||||
@@ -95,10 +93,10 @@ export function DatabaseTables({
|
||||
onNestedNavigationChange,
|
||||
onRunStarted,
|
||||
onOpenSync,
|
||||
onClearCatalogTables,
|
||||
onDescriptionGenerationRunStarted,
|
||||
onSuggestSensitive,
|
||||
onCatalogMetricsChanged,
|
||||
metadataModelControl,
|
||||
presentation = "legacy",
|
||||
}: Props) {
|
||||
const databaseId = database.id!;
|
||||
@@ -133,22 +131,9 @@ export function DatabaseTables({
|
||||
const nestedNavigationActive = presentation === "fleet" && Boolean(activeTable) && tableSection === "columns";
|
||||
const bindingReady = database.connectionStatus === "reachable" && database.testedVersion === database.version;
|
||||
const currentRun = activeRun ?? database.activeSyncRun;
|
||||
const clearAllCatalogTables = async () => {
|
||||
if (!onClearCatalogTables || !window.confirm("Clear all catalog tables, columns, and relationships for this database? The source database will not be changed.")) return;
|
||||
await onClearCatalogTables();
|
||||
};
|
||||
const leaveTable = useCallback(() => {
|
||||
if (navigationBusy) return;
|
||||
if (dirty && !window.confirm("Discard unsaved metadata?")) return;
|
||||
setActiveTableId(null);
|
||||
setTableSection("overview");
|
||||
window.setTimeout(() => originRef.current?.focus(), 0);
|
||||
}, [dirty, navigationBusy]);
|
||||
|
||||
useEffect(() => { onNavigationStateChange({ dirty, busy: navigationBusy }); }, [dirty, navigationBusy, onNavigationStateChange]);
|
||||
useEffect(() => {
|
||||
onNestedNavigationChange?.(nestedNavigationActive, nestedNavigationActive ? leaveTable : undefined);
|
||||
}, [leaveTable, nestedNavigationActive, onNestedNavigationChange]);
|
||||
useEffect(() => { onNestedNavigationChange?.(nestedNavigationActive); }, [nestedNavigationActive, onNestedNavigationChange]);
|
||||
useEffect(() => () => { onNestedNavigationChange?.(false); }, [onNestedNavigationChange]);
|
||||
useEffect(() => {
|
||||
if (!activeTableId || !activeTable || editorVersion === activeTable.version || busy) return;
|
||||
@@ -180,6 +165,13 @@ export function DatabaseTables({
|
||||
setStaleBannerOpen(true);
|
||||
setColumnNavigation({ dirty: false, busy: false });
|
||||
};
|
||||
const leaveTable = () => {
|
||||
if (navigationBusy) return;
|
||||
if (dirty && !window.confirm("Discard unsaved metadata?")) return;
|
||||
setActiveTableId(null);
|
||||
setTableSection("overview");
|
||||
window.setTimeout(() => originRef.current?.focus(), 0);
|
||||
};
|
||||
const navigateDatabase = (target: "databases" | "overview" | "relationships") => {
|
||||
if (navigationBusy) return;
|
||||
if (dirty && !window.confirm("Discard unsaved metadata?")) return;
|
||||
@@ -286,28 +278,6 @@ export function DatabaseTables({
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally { setBusy(null); }
|
||||
};
|
||||
const generateColumnDescriptions = async () => {
|
||||
if (selectedIds.length === 0 || !selectedMetadataModel) return;
|
||||
setBusy("generate");
|
||||
try {
|
||||
const columns = (await Promise.all(selectedIds.map((tableId) => listCatalogColumns(databaseId, tableId)))).flat();
|
||||
const columnIds = columns.map((column) => column.id);
|
||||
if (columnIds.length === 0) {
|
||||
toast.error("The selected tables have no synchronized catalog columns.");
|
||||
return;
|
||||
}
|
||||
const run = await startDescriptionGenerationRun(
|
||||
databaseId,
|
||||
selectedMetadataModel,
|
||||
"selected_columns",
|
||||
columnIds,
|
||||
);
|
||||
onDescriptionGenerationRunStarted(run);
|
||||
toast.success(`Column description generation started for ${columnIds.length} column${columnIds.length === 1 ? "" : "s"}`);
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally { setBusy(null); }
|
||||
};
|
||||
const suggestSensitive = async () => {
|
||||
if (selectedIds.length === 0) return;
|
||||
setBusy("suggest");
|
||||
@@ -325,17 +295,17 @@ export function DatabaseTables({
|
||||
|
||||
type FleetTableAction =
|
||||
| "generate-descriptions"
|
||||
| "generate-column-descriptions"
|
||||
| "sync-tables"
|
||||
| "sync-columns"
|
||||
| "consolidate-descriptions"
|
||||
| "suggest-sensitive"
|
||||
| "clear-columns";
|
||||
| "clear-columns"
|
||||
| "clear-relationships";
|
||||
|
||||
const fleetActions: readonly FleetActionOption<FleetTableAction>[] = [
|
||||
{
|
||||
id: "generate-descriptions",
|
||||
label: "Generate table description",
|
||||
label: "Generate descriptions",
|
||||
group: "Descriptions",
|
||||
runLabel: "Generate",
|
||||
disabled: !canManage || selectedIds.length === 0 || !selectedMetadataModel || descriptionGenerationActive || busy !== null || Boolean(currentRun),
|
||||
@@ -344,25 +314,7 @@ export function DatabaseTables({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one table."
|
||||
: !selectedMetadataModel
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
: descriptionGenerationActive || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy !== null
|
||||
? "Another action is running."
|
||||
: undefined,
|
||||
},
|
||||
{
|
||||
id: "generate-column-descriptions",
|
||||
label: "Generate column descriptions",
|
||||
group: "Descriptions",
|
||||
runLabel: "Generate",
|
||||
disabled: !canManage || selectedIds.length === 0 || !selectedMetadataModel || descriptionGenerationActive || busy !== null || Boolean(currentRun),
|
||||
disabledReason: !canManage
|
||||
? "You do not have permission to generate descriptions."
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one table."
|
||||
: !selectedMetadataModel
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
? "Choose a metadata model first."
|
||||
: descriptionGenerationActive || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy !== null
|
||||
@@ -371,7 +323,7 @@ export function DatabaseTables({
|
||||
},
|
||||
{
|
||||
id: "consolidate-descriptions",
|
||||
label: "Copy generated descriptions to Description",
|
||||
label: "Move generated to Description",
|
||||
group: "Descriptions",
|
||||
runLabel: "Move",
|
||||
disabled: !canManage || selectedIds.length === 0 || busy !== null || Boolean(currentRun),
|
||||
@@ -385,24 +337,22 @@ export function DatabaseTables({
|
||||
},
|
||||
{
|
||||
id: "sync-tables",
|
||||
label: "Synchronize database tables",
|
||||
label: "Synchronize tables",
|
||||
group: "Synchronization",
|
||||
scopeLabel: "Selected database",
|
||||
scopeLabel: "All tables in this database",
|
||||
runLabel: "Synchronize",
|
||||
disabled: !canManage || selectedIds.length === 0 || !bindingReady || busy !== null || Boolean(currentRun),
|
||||
disabled: !canManage || !bindingReady || busy !== null || Boolean(currentRun),
|
||||
disabledReason: !canManage
|
||||
? "You do not have permission to synchronize the catalog."
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one table."
|
||||
: !bindingReady
|
||||
? "Test the current database binding first."
|
||||
: busy !== null || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: undefined,
|
||||
: !bindingReady
|
||||
? "Test the current database binding first."
|
||||
: busy !== null || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: undefined,
|
||||
},
|
||||
{
|
||||
id: "sync-columns",
|
||||
label: "Synchronize columns for selected tables",
|
||||
label: "Synchronize columns",
|
||||
group: "Synchronization",
|
||||
runLabel: "Synchronize",
|
||||
disabled: !canManage || selectedIds.length === 0 || !bindingReady || busy !== null || Boolean(currentRun),
|
||||
@@ -427,7 +377,7 @@ export function DatabaseTables({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one table."
|
||||
: !selectedMetadataModel
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
? "Choose a metadata model first."
|
||||
: descriptionGenerationActive || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy !== null
|
||||
@@ -436,7 +386,22 @@ export function DatabaseTables({
|
||||
},
|
||||
{
|
||||
id: "clear-columns",
|
||||
label: "Remove synchronized columns",
|
||||
label: "Clear catalog columns",
|
||||
group: "Catalog cleanup",
|
||||
tone: "destructive",
|
||||
runLabel: "Review cleanup",
|
||||
disabled: !canManage || selectedIds.length === 0 || busy !== null || Boolean(currentRun),
|
||||
disabledReason: !canManage
|
||||
? "You do not have permission to clear catalog metadata."
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one table."
|
||||
: busy !== null || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: undefined,
|
||||
},
|
||||
{
|
||||
id: "clear-relationships",
|
||||
label: "Clear catalog relationships",
|
||||
group: "Catalog cleanup",
|
||||
tone: "destructive",
|
||||
runLabel: "Review cleanup",
|
||||
@@ -453,12 +418,12 @@ export function DatabaseTables({
|
||||
|
||||
const runFleetAction = async (action: FleetTableAction) => {
|
||||
if (action === "generate-descriptions") await generateDescriptions();
|
||||
else if (action === "generate-column-descriptions") await generateColumnDescriptions();
|
||||
else if (action === "sync-tables") await synchronize("tables");
|
||||
else if (action === "consolidate-descriptions") await consolidateDescriptions();
|
||||
else if (action === "sync-columns") await synchronize("columns", selectedIds);
|
||||
else if (action === "suggest-sensitive") await suggestSensitive();
|
||||
else if (action === "clear-columns") setPendingDelete("columns");
|
||||
else setPendingDelete("relationships");
|
||||
};
|
||||
|
||||
const clearSelection = () => {
|
||||
@@ -496,7 +461,7 @@ export function DatabaseTables({
|
||||
const databaseHeader = (
|
||||
<div className="px-4 pt-5 sm:px-5">
|
||||
<Button type="button" variant="ghost" className="-ml-2 mb-4" disabled={navigationBusy} onClick={() => navigateDatabase("databases")}><ArrowLeft />Back to databases</Button>
|
||||
<div className="flex flex-wrap items-start justify-between gap-4 border-b border-border pb-5">
|
||||
<div className="flex flex-wrap items-start justify-between gap-4 border-b border-border pb-5">
|
||||
<div>
|
||||
<p className="thot-label mb-1">Database management</p>
|
||||
<h2 ref={headingRef} tabIndex={-1} className="font-heading text-2xl font-semibold tracking-tight outline-none">{database.workspaceName}</h2>
|
||||
@@ -504,7 +469,6 @@ export function DatabaseTables({
|
||||
</div>
|
||||
<div className="flex gap-2">
|
||||
{currentRun ? <Button type="button" variant="outline" onClick={onOpenSync}><RefreshCw className="animate-spin" />View sync</Button> : null}
|
||||
{onClearCatalogTables ? <Button type="button" variant="outline" className="text-destructive" disabled={!canManage || busy !== null || Boolean(currentRun)} onClick={() => void clearAllCatalogTables()}>Clear catalog tables</Button> : null}
|
||||
<Button type="button" disabled={!canManage || !bindingReady || busy !== null || Boolean(currentRun)} onClick={() => void synchronize("all")}><RefreshCw />Sync all</Button>
|
||||
</div>
|
||||
</div>
|
||||
@@ -521,6 +485,17 @@ export function DatabaseTables({
|
||||
return (
|
||||
<section aria-label={`Columns for ${activeTable.name}`} className="flex min-h-0 flex-1 flex-col overflow-hidden bg-background">
|
||||
<div className="flex min-h-12 flex-wrap items-center gap-3 border-b border-border bg-muted/25 px-3 py-2">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
className="thot-fleet-icon-action border-primary/35 bg-primary/8 shadow-sm"
|
||||
disabled={navigationBusy}
|
||||
data-tooltip="Tables"
|
||||
aria-label="Back to tables"
|
||||
onClick={leaveTable}
|
||||
>
|
||||
<ArrowLeft aria-hidden="true" /> Back to tables
|
||||
</Button>
|
||||
<div className="min-w-0">
|
||||
<p className="thot-label">Physical table</p>
|
||||
<h3 className="truncate font-heading text-lg font-semibold">{activeTable.name}</h3>
|
||||
@@ -529,16 +504,17 @@ export function DatabaseTables({
|
||||
<DatabaseColumns
|
||||
databaseId={databaseId}
|
||||
table={activeTable}
|
||||
databaseVersion={database.version}
|
||||
canManage={canManage}
|
||||
selectedMetadataModel={selectedMetadataModel}
|
||||
descriptionGenerationActive={descriptionGenerationActive}
|
||||
onDescriptionGenerationRunStarted={onDescriptionGenerationRunStarted}
|
||||
onCatalogSyncRunStarted={onRunStarted}
|
||||
onNavigationStateChange={setColumnNavigation}
|
||||
onSync={() => void synchronize("columns", [activeTable.id])}
|
||||
onSuggestSensitive={onSuggestSensitive}
|
||||
bindingReady={bindingReady}
|
||||
catalogOperationActive={Boolean(currentRun)}
|
||||
onCatalogMetricsChanged={onCatalogMetricsChanged}
|
||||
metadataModelControl={metadataModelControl}
|
||||
presentation="fleet"
|
||||
/>
|
||||
</section>
|
||||
@@ -581,7 +557,7 @@ export function DatabaseTables({
|
||||
rowHeight={44}
|
||||
headerHeight={38}
|
||||
animateRows={false}
|
||||
overlayNoRowsTemplate="No catalog tables. Synchronize the schema from the database view."
|
||||
overlayNoRowsTemplate="No catalog tables. Test the binding, then sync tables."
|
||||
/>
|
||||
</div>
|
||||
</div>}
|
||||
@@ -642,13 +618,12 @@ export function DatabaseTables({
|
||||
<DatabaseColumns
|
||||
databaseId={databaseId}
|
||||
table={activeTable}
|
||||
databaseVersion={database.version}
|
||||
canManage={canManage}
|
||||
selectedMetadataModel={selectedMetadataModel}
|
||||
descriptionGenerationActive={descriptionGenerationActive}
|
||||
onDescriptionGenerationRunStarted={onDescriptionGenerationRunStarted}
|
||||
onCatalogSyncRunStarted={onRunStarted}
|
||||
onNavigationStateChange={setColumnNavigation}
|
||||
onSync={() => void synchronize("columns", [activeTable.id])}
|
||||
onSuggestSensitive={onSuggestSensitive}
|
||||
onCatalogMetricsChanged={onCatalogMetricsChanged}
|
||||
/>
|
||||
@@ -699,11 +674,19 @@ export function DatabaseTables({
|
||||
<div className="mr-auto min-w-56">
|
||||
<p id="table-cleanup-confirmation" className="text-sm font-semibold">
|
||||
{presentation === "fleet"
|
||||
? `Remove synchronized columns for ${selectedIds.length} table${selectedIds.length === 1 ? "" : "s"}?`
|
||||
: `Remove synchronized columns for ${selectedIds.length} table${selectedIds.length === 1 ? "" : "s"}?`}
|
||||
? `Clear catalog ${pendingDelete} for ${selectedIds.length} table${selectedIds.length === 1 ? "" : "s"}?`
|
||||
: pendingDelete === "columns"
|
||||
? `Delete all catalog columns for ${selectedIds.length} table${selectedIds.length === 1 ? "" : "s"}?`
|
||||
: `Delete all relationships for ${selectedIds.length} table${selectedIds.length === 1 ? "" : "s"}?`}
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Only synchronized catalog columns for these tables will be removed. The source database is not modified.
|
||||
{presentation === "fleet"
|
||||
? pendingDelete === "columns"
|
||||
? "Only synchronized catalog columns for these tables will be cleared. The source database is not modified."
|
||||
: "Only synchronized catalog relationships for these tables will be cleared. The source database is not modified."
|
||||
: pendingDelete === "columns"
|
||||
? "Only columns belonging to the selected catalog tables will be removed."
|
||||
: "All incoming and outgoing relationships for the selected catalog tables will be removed."}
|
||||
</p>
|
||||
</div>
|
||||
<Button type="button" variant="ghost" disabled={busy !== null} onClick={closeDeleteConfirmation}>Cancel</Button>
|
||||
@@ -714,7 +697,7 @@ export function DatabaseTables({
|
||||
disabled={busy !== null}
|
||||
onClick={() => void deleteMetadata(pendingDelete)}
|
||||
>
|
||||
<Trash2 />Remove synchronized columns
|
||||
<Trash2 />{presentation === "fleet" ? "Clear catalog" : pendingDelete === "columns" ? "Delete columns" : "Delete relationships"}
|
||||
</Button>
|
||||
</div>
|
||||
) : (
|
||||
@@ -729,6 +712,7 @@ export function DatabaseTables({
|
||||
selectRef={fleetActionSelectRef}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Table action"
|
||||
/>
|
||||
</>
|
||||
@@ -741,13 +725,12 @@ export function DatabaseTables({
|
||||
<Menu.Portal>
|
||||
<Menu.Positioner side="bottom" align="start" sideOffset={4}>
|
||||
<Menu.Popup className="z-50 min-w-64 rounded-lg bg-popover p-1 text-popover-foreground shadow-md ring-1 ring-foreground/10 outline-none">
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !selectedMetadataModel || descriptionGenerationActive || busy !== null || Boolean(currentRun)} onClick={() => void generateDescriptions()}>Generate table description</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !selectedMetadataModel || descriptionGenerationActive || busy !== null || Boolean(currentRun)} onClick={() => void generateColumnDescriptions()}>Generate descriptions for all columns</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !bindingReady || busy !== null || Boolean(currentRun)} onClick={() => void synchronize("tables")}>Synchronize database tables</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !bindingReady || busy !== null || Boolean(currentRun)} onClick={() => void synchronize("columns", selectedIds)}>Synchronize columns for selected tables</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || busy !== null || Boolean(currentRun)} onClick={() => void consolidateDescriptions()}>Copy generated descriptions to Description</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !selectedMetadataModel || descriptionGenerationActive || busy !== null || Boolean(currentRun)} onClick={() => void generateDescriptions()}>Generate {selectedIds.length === 1 ? "description" : "descriptions"}</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || !bindingReady || busy !== null || Boolean(currentRun)} onClick={() => void synchronize("columns", selectedIds)}>Synchronize columns</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm outline-none data-[highlighted]:bg-muted data-[disabled]:opacity-45" disabled={!canManage || busy !== null || Boolean(currentRun)} onClick={() => void consolidateDescriptions()}>Move generated description to Description</Menu.Item>
|
||||
<Menu.Separator className="my-1 h-px bg-border" />
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm text-destructive outline-none data-[highlighted]:bg-destructive/10 data-[disabled]:opacity-45" disabled={!canManage || busy !== null || Boolean(currentRun)} onClick={() => setPendingDelete("columns")}>Remove synchronized columns</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm text-destructive outline-none data-[highlighted]:bg-destructive/10 data-[disabled]:opacity-45" disabled={!canManage || busy !== null || Boolean(currentRun)} onClick={() => setPendingDelete("columns")}>Delete all columns</Menu.Item>
|
||||
<Menu.Item className="rounded-md px-3 py-2 text-sm text-destructive outline-none data-[highlighted]:bg-destructive/10 data-[disabled]:opacity-45" disabled={!canManage || busy !== null || Boolean(currentRun)} onClick={() => setPendingDelete("relationships")}>Delete all relationships</Menu.Item>
|
||||
</Menu.Popup>
|
||||
</Menu.Positioner>
|
||||
</Menu.Portal>
|
||||
@@ -768,6 +751,7 @@ export function DatabaseTables({
|
||||
selectRef={fleetActionSelectRef}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Table action"
|
||||
/>
|
||||
<input ref={searchInputRef} className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search tables" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
|
||||
@@ -780,11 +764,17 @@ export function DatabaseTables({
|
||||
<input ref={searchInputRef} className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search tables" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
|
||||
<span aria-live="polite" className="whitespace-nowrap text-xs tabular-nums text-muted-foreground">{search.trim() ? `${displayedCount} of ${data.length}` : displayedCount}</span>
|
||||
<Button type="button" variant="outline" disabled={isFetching || busy !== null} onClick={() => void refetch()}><RefreshCw className={isFetching ? "animate-spin" : ""} />Refresh</Button>
|
||||
<Button type="button" disabled={!canManage || !bindingReady || busy !== null || Boolean(currentRun)} title={!bindingReady ? "Test the current database binding before synchronizing tables" : undefined} onClick={() => void synchronize("tables")}><RefreshCw className={busy === "sync" ? "animate-spin" : ""} />Sync tables</Button>
|
||||
</>
|
||||
)
|
||||
)}
|
||||
{presentation === "fleet" ? (
|
||||
<Button type="button" size="sm" variant="outline" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Synchronization history" onClick={onOpenSync}>
|
||||
{currentRun ? <RefreshCw className="animate-spin" aria-hidden="true" /> : <History aria-hidden="true" />} Sync history
|
||||
</Button>
|
||||
) : null}
|
||||
</div>
|
||||
{!bindingReady ? <div className="border-b border-border bg-muted/35 px-4 py-3 text-sm text-muted-foreground">Test the current database binding from Overview before synchronizing columns.</div> : null}
|
||||
{!bindingReady ? <div className="border-b border-border bg-muted/35 px-4 py-3 text-sm text-muted-foreground">Test the current database binding from Overview before synchronizing tables.</div> : null}
|
||||
{fleetQueryError ?? <div className="relative min-h-[280px] flex-1">
|
||||
<div className={`thot-database-grid ${presentation === "fleet" ? "thot-fleet-ledger-grid " : ""}ag-theme-alpine absolute inset-0 h-full w-full`}>
|
||||
<AgGridReact<CatalogTable>
|
||||
@@ -805,7 +795,7 @@ export function DatabaseTables({
|
||||
rowHeight={44}
|
||||
headerHeight={38}
|
||||
animateRows={false}
|
||||
overlayNoRowsTemplate="No catalog tables. Synchronize the schema from the database view."
|
||||
overlayNoRowsTemplate="No catalog tables. Test the binding, then sync tables."
|
||||
/>
|
||||
</div>
|
||||
</div>}
|
||||
|
||||
@@ -109,9 +109,6 @@ test("replays ordered SSE events, reconnects from the latest sequence, and keeps
|
||||
renderDrawer();
|
||||
|
||||
const log = await screen.findByRole("log", { name: "Description generation events" });
|
||||
const progress = screen.getByRole("table", { name: "Description generation progress" });
|
||||
expect(within(progress).getAllByRole("columnheader")).toHaveLength(5);
|
||||
expect(log).toHaveClass("thot-catalog-drawer__event-log");
|
||||
expect(await within(log).findByText("Run queued")).toBeVisible();
|
||||
await waitFor(() => expect(FakeEventSource.instances).toHaveLength(1));
|
||||
expect(FakeEventSource.instances[0].url).toBe(
|
||||
@@ -251,33 +248,6 @@ test("reopens a terminal run from newest-first persisted history", async () => {
|
||||
expect(within(drawer).getByText("3", { selector: "[data-count='generated']" })).toBeVisible();
|
||||
});
|
||||
|
||||
test("styles a successfully completed run with the success background", async () => {
|
||||
const completedRun: DescriptionGenerationRun = {
|
||||
...runningRun,
|
||||
status: "completed",
|
||||
total: 1,
|
||||
processed: 1,
|
||||
generated: 1,
|
||||
updatedAt: "2026-08-28T08:02:00Z",
|
||||
finishedAt: "2026-08-28T08:02:00Z",
|
||||
};
|
||||
server.use(
|
||||
http.get("/api/catalog/description-generation-runs", () => HttpResponse.json([completedRun])),
|
||||
http.get("/api/catalog/description-generation-runs/:runId", () => HttpResponse.json(completedRun)),
|
||||
http.get("/api/catalog/description-generation-runs/:runId/events-list", () => HttpResponse.json([])),
|
||||
);
|
||||
renderDrawer({ initialRun: completedRun });
|
||||
|
||||
const history = await screen.findByRole("region", {
|
||||
name: "Description generation history",
|
||||
});
|
||||
const completedHistoryItem = within(history).getByRole("button", {
|
||||
name: /Completed.*missing/i,
|
||||
});
|
||||
expect(completedHistoryItem).toHaveAttribute("data-status", "completed");
|
||||
expect(completedHistoryItem).toHaveClass("bg-[oklch(var(--success)/0.1)]");
|
||||
});
|
||||
|
||||
test("offers Unlock only for an apparently stale active run and explains the interruption", async () => {
|
||||
const user = userEvent.setup();
|
||||
const staleRun: DescriptionGenerationRun = {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { LoaderCircle, LockOpen, Square } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
@@ -18,7 +18,6 @@ import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
|
||||
interface Props {
|
||||
open: boolean;
|
||||
databaseId?: string | null;
|
||||
run: DescriptionGenerationRun | null;
|
||||
modelLabel: string;
|
||||
onClose: () => void;
|
||||
@@ -63,7 +62,6 @@ function timestamp(value: string | null): string {
|
||||
|
||||
export function DescriptionGenerationDrawer({
|
||||
open,
|
||||
databaseId = null,
|
||||
run: initialRun,
|
||||
modelLabel,
|
||||
onClose,
|
||||
@@ -102,15 +100,11 @@ export function DescriptionGenerationDrawer({
|
||||
? 3_000
|
||||
: false,
|
||||
});
|
||||
const visibleHistory = useMemo(
|
||||
() => (historyQuery.data ?? []).filter((item) => !databaseId || item.databaseId === databaseId),
|
||||
[databaseId, historyQuery.data],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open || initialRun || !visibleHistory[0]) return;
|
||||
onRunUpdate(visibleHistory[0]);
|
||||
}, [initialRun, onRunUpdate, open, visibleHistory]);
|
||||
if (!open || initialRun || !historyQuery.data?.[0]) return;
|
||||
onRunUpdate(historyQuery.data[0]);
|
||||
}, [historyQuery.data, initialRun, onRunUpdate, open]);
|
||||
|
||||
const mergeEvents = useCallback((incoming: DescriptionGenerationEvent[]) => {
|
||||
if (incoming.length === 0) return;
|
||||
@@ -300,16 +294,6 @@ export function DescriptionGenerationDrawer({
|
||||
}
|
||||
const apparentlyStale = isApparentlyStale(run);
|
||||
const finalError = safeFinalError(run.errorSummary);
|
||||
const modelDisplay = modelLabel && modelLabel !== run.modelId
|
||||
? `${modelLabel} (${run.modelId})`
|
||||
: run.modelId;
|
||||
const progressCounters = [
|
||||
["total", "Total"],
|
||||
["processed", "Processed"],
|
||||
["generated", "Generated"],
|
||||
["nonGeneratable", "Not generated"],
|
||||
["failed", "Failed"],
|
||||
] as const;
|
||||
const footer = ["queued", "running"].includes(run.status) ? (
|
||||
<div className="w-full">
|
||||
{apparentlyStale ? (
|
||||
@@ -351,80 +335,55 @@ export function DescriptionGenerationDrawer({
|
||||
open
|
||||
ariaLabel="Description generation"
|
||||
eyebrow="Description generation"
|
||||
title={(
|
||||
<span className="inline-flex items-center gap-2">
|
||||
{!isTerminal(run) ? <LoaderCircle className="size-5 animate-spin" aria-hidden="true" /> : null}
|
||||
{statusLabel(run)}
|
||||
</span>
|
||||
)}
|
||||
description={run.scope.replaceAll("_", " ")}
|
||||
title={statusLabel(run)}
|
||||
description={`${modelLabel}${modelLabel !== run.modelId ? ` (${run.modelId})` : ""} · ${run.scope.replaceAll("_", " ")}`}
|
||||
onClose={onClose}
|
||||
closeLabel="Close description generation"
|
||||
busy={action !== null}
|
||||
bodyClassName="thot-catalog-drawer__history-layout thot-catalog-drawer__run-layout"
|
||||
bodyClassName="thot-catalog-drawer__history-layout"
|
||||
footer={footer}
|
||||
>
|
||||
<div className="thot-catalog-drawer__run-main">
|
||||
{runQuery.isError ? (
|
||||
{runQuery.isError ? (
|
||||
<div role="alert" className="rounded-md border border-destructive/35 bg-destructive/5 px-4 py-3 text-sm text-destructive">
|
||||
{apiErrorMessage(runQuery.error)}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
<div className={`grid gap-4 sm:grid-cols-2${runQuery.isError ? " mt-4" : ""}`}>
|
||||
<section aria-label="AI usage">
|
||||
<h3 className="thot-label mb-2">AI usage</h3>
|
||||
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1 text-xs">
|
||||
<dt>Model</dt><dd className="truncate text-right" title={modelDisplay}>{modelDisplay}</dd>
|
||||
<dt>Input tokens</dt><dd className="text-right tabular-nums">{(run.inputTokens ?? 0).toLocaleString("en-US")}</dd>
|
||||
<dt>Cache tokens</dt><dd className="text-right tabular-nums">{(run.cacheReadTokens ?? 0).toLocaleString("en-US")}</dd>
|
||||
<dt>Output tokens</dt><dd className="text-right tabular-nums">{(run.outputTokens ?? 0).toLocaleString("en-US")}</dd>
|
||||
</dl>
|
||||
</section>
|
||||
<section aria-label="Description generation timestamps">
|
||||
<h3 className="thot-label mb-2">Timestamps</h3>
|
||||
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1 text-xs">
|
||||
<dt className="text-muted-foreground">Created</dt><dd className="text-right tabular-nums">{timestamp(run.createdAt)}</dd>
|
||||
<dt className="text-muted-foreground">Started</dt><dd className="text-right tabular-nums">{timestamp(run.startedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Updated</dt><dd className="text-right tabular-nums">{timestamp(run.updatedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Finished</dt><dd className="text-right tabular-nums">{timestamp(run.finishedAt)}</dd>
|
||||
</dl>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section className="mt-4" aria-label="Description generation counters">
|
||||
<section aria-label="Description generation counters">
|
||||
<h3 className="thot-label mb-2">Progress</h3>
|
||||
<div className="overflow-x-auto rounded-md border border-border bg-muted/20">
|
||||
<table className="w-full min-w-[28rem] table-fixed" aria-label="Description generation progress">
|
||||
<thead>
|
||||
<tr className="border-b border-border bg-muted/45">
|
||||
{progressCounters.map(([name, label]) => (
|
||||
<th key={name} scope="col" className="border-l border-border px-2 py-1.5 text-left text-[10px] font-semibold uppercase tracking-wide text-muted-foreground first:border-l-0">
|
||||
{label}
|
||||
</th>
|
||||
))}
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
{progressCounters.map(([name]) => (
|
||||
<td key={name} data-count={name} className="border-l border-border px-2 py-1.5 text-base font-semibold tabular-nums first:border-l-0">
|
||||
{run[name]}
|
||||
</td>
|
||||
))}
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div className="grid grid-cols-2 gap-2">
|
||||
{([
|
||||
["total", "Total"],
|
||||
["processed", "Processed"],
|
||||
["generated", "Generated"],
|
||||
["nonGeneratable", "Not generated"],
|
||||
["failed", "Failed"],
|
||||
] as const).map(([name, label]) => (
|
||||
<div key={name} className="rounded-md border border-border bg-muted/25 px-3 py-2">
|
||||
<p className="text-[11px] font-semibold uppercase tracking-wide text-muted-foreground">{label}</p>
|
||||
<p data-count={name} className="mt-1 text-lg font-semibold tabular-nums">{run[name]}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section className="mt-5" aria-label="Description generation timestamps">
|
||||
<h3 className="thot-label mb-2">Timestamps</h3>
|
||||
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1.5 text-sm">
|
||||
<dt className="text-muted-foreground">Created</dt><dd className="text-right">{timestamp(run.createdAt)}</dd>
|
||||
<dt className="text-muted-foreground">Started</dt><dd className="text-right">{timestamp(run.startedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Updated</dt><dd className="text-right">{timestamp(run.updatedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Finished</dt><dd className="text-right">{timestamp(run.finishedAt)}</dd>
|
||||
</dl>
|
||||
</section>
|
||||
|
||||
{finalError ? (
|
||||
<div className="mt-3 rounded-md border border-destructive/35 bg-destructive/5 px-4 py-3 text-sm text-destructive">
|
||||
<div className="mt-5 rounded-md border border-destructive/35 bg-destructive/5 px-4 py-3 text-sm text-destructive">
|
||||
{finalError}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
<section className="thot-catalog-drawer__events mt-3" aria-label="Description generation log">
|
||||
<section className="mt-5" aria-label="Description generation log">
|
||||
<div className="mb-2 flex items-center justify-between">
|
||||
<h3 className="thot-label">Events</h3>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{events.length}</span>
|
||||
@@ -432,7 +391,7 @@ export function DescriptionGenerationDrawer({
|
||||
<div
|
||||
role="log"
|
||||
aria-label="Description generation events"
|
||||
className="thot-catalog-drawer__event-log overflow-y-auto rounded-md bg-zinc-950 p-3 font-mono text-xs leading-5 text-zinc-200"
|
||||
className="max-h-72 overflow-y-auto rounded-md bg-zinc-950 p-3 font-mono text-xs leading-5 text-zinc-200"
|
||||
>
|
||||
{events.length === 0 ? <p className="text-zinc-500">Waiting for events…</p> : events.map((event) => (
|
||||
<p key={event.sequence} className={event.level === "error" ? "text-red-300" : event.level === "warning" ? "text-amber-300" : undefined}>
|
||||
@@ -442,13 +401,12 @@ export function DescriptionGenerationDrawer({
|
||||
</div>
|
||||
{eventQuery.isError ? <p role="alert" className="mt-2 text-sm text-destructive">{apiErrorMessage(eventQuery.error)}</p> : null}
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section className="thot-catalog-drawer__history" aria-label="Description generation history">
|
||||
<section className="thot-catalog-drawer__history mt-5" aria-label="Description generation history">
|
||||
<div className="mb-2 flex items-center justify-between gap-3">
|
||||
<h3 className="thot-label">Recent runs</h3>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">
|
||||
{visibleHistory.length}
|
||||
{historyQuery.data?.length ?? 0}
|
||||
</span>
|
||||
</div>
|
||||
{historyQuery.isError ? (
|
||||
@@ -457,18 +415,17 @@ export function DescriptionGenerationDrawer({
|
||||
</p>
|
||||
) : historyQuery.isLoading ? (
|
||||
<p className="text-sm text-muted-foreground">Loading run history…</p>
|
||||
) : visibleHistory.length === 0 ? (
|
||||
) : (historyQuery.data ?? []).length === 0 ? (
|
||||
<p className="text-sm text-muted-foreground">No description generation runs yet.</p>
|
||||
) : (
|
||||
<div className="thot-catalog-drawer__history-list divide-y divide-border overflow-y-auto rounded-md border border-border">
|
||||
{visibleHistory.map((item) => (
|
||||
<div className="max-h-56 divide-y divide-border overflow-y-auto rounded-md border border-border">
|
||||
{(historyQuery.data ?? []).map((item) => (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
aria-label={`${statusLabel(item)} · ${item.scope.replaceAll("_", " ")}`}
|
||||
aria-current={item.id === run.id ? "true" : undefined}
|
||||
data-status={item.status}
|
||||
className={`flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25${item.status === "completed" ? " bg-[oklch(var(--success)/0.1)]" : ""}`}
|
||||
className="flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25"
|
||||
onClick={() => onRunUpdate(item)}
|
||||
>
|
||||
<span className="font-medium">{statusLabel(item)}</span>
|
||||
|
||||
@@ -8,26 +8,13 @@ const actions = [
|
||||
{ id: "sync", label: "Synchronize tables", group: "Synchronization", scopeLabel: "All tables in this database" },
|
||||
{
|
||||
id: "generate",
|
||||
label: "Generate column descriptions",
|
||||
label: "Generate descriptions",
|
||||
group: "Descriptions",
|
||||
disabled: true,
|
||||
disabledReason: "Choose a metadata model first",
|
||||
},
|
||||
] as const;
|
||||
|
||||
test("disables the picker when no rows are selected", () => {
|
||||
render(
|
||||
<FleetActionSelector
|
||||
actions={actions}
|
||||
selectedCount={0}
|
||||
onRun={vi.fn()}
|
||||
onClear={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(screen.getByRole("combobox", { name: "Batch action" })).toBeDisabled();
|
||||
});
|
||||
|
||||
test("keeps unavailable commands discoverable and explains why Run is disabled", async () => {
|
||||
const user = userEvent.setup();
|
||||
render(
|
||||
@@ -63,7 +50,6 @@ test("runs an enabled command explicitly and resets the picker", async () => {
|
||||
await user.selectOptions(picker, "sync");
|
||||
expect(screen.getByText("All tables in this database")).toBeVisible();
|
||||
expect(screen.getByText("Action scope")).toBeVisible();
|
||||
expect(screen.getByRole("button", { name: "Run action" })).toHaveClass("bg-destructive");
|
||||
await user.click(screen.getByRole("button", { name: "Run action" }));
|
||||
|
||||
expect(onRun).toHaveBeenCalledWith("sync");
|
||||
|
||||
@@ -44,6 +44,7 @@ export function FleetActionSelector<ActionId extends string>({
|
||||
selectedCount,
|
||||
hiddenSelectedCount = 0,
|
||||
busy = false,
|
||||
busyLabel = "Running…",
|
||||
selectRef,
|
||||
onRun,
|
||||
onClear,
|
||||
@@ -101,7 +102,7 @@ export function FleetActionSelector<ActionId extends string>({
|
||||
className="thot-fleet-action-selector__select"
|
||||
aria-label={label}
|
||||
value={selectedActionId}
|
||||
disabled={busy || selectedCount === 0}
|
||||
disabled={busy}
|
||||
onChange={(event) => setSelectedActionId(event.target.value as ActionId | "")}
|
||||
>
|
||||
<option value="">Choose an action…</option>
|
||||
@@ -122,14 +123,13 @@ export function FleetActionSelector<ActionId extends string>({
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="destructive"
|
||||
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
|
||||
variant={selectedAction?.tone === "destructive" ? "destructive" : "default"}
|
||||
disabled={!selectedAction || selectedAction.disabled || busy}
|
||||
aria-describedby={unavailableReason ? `${hintId}-reason` : undefined}
|
||||
onClick={() => void run()}
|
||||
>
|
||||
<Play aria-hidden="true" />
|
||||
Run action
|
||||
{busy ? busyLabel : selectedAction?.runLabel ?? "Run action"}
|
||||
</Button>
|
||||
<Button type="button" size="sm" variant="ghost" disabled={busy} onClick={onClear}>
|
||||
<X aria-hidden="true" /> Clear
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
import { readFileSync, readdirSync } from "node:fs";
|
||||
|
||||
import { expect, test } from "vitest";
|
||||
|
||||
const sourceDirectory = "src/shell/database-management";
|
||||
const styles = readFileSync(`${sourceDirectory}/FleetLedgerShell.css`, "utf8");
|
||||
|
||||
test("uses operational body text for every available-command list", () => {
|
||||
const availableCommandLists = readdirSync(sourceDirectory)
|
||||
.filter((fileName) => fileName.endsWith(".tsx") && !fileName.includes(".test."))
|
||||
.flatMap((fileName) => {
|
||||
const source = readFileSync(`${sourceDirectory}/${fileName}`, "utf8");
|
||||
const matches = source.match(
|
||||
/<select\b[\s\S]*?<option value="">Choose an action…<\/option>[\s\S]*?<\/select>/g,
|
||||
);
|
||||
return (matches ?? []).map((markup) => ({ fileName, markup }));
|
||||
});
|
||||
|
||||
expect(availableCommandLists.map(({ fileName }) => fileName).sort()).toEqual([
|
||||
"DatabaseRelationships.tsx",
|
||||
"FleetActionSelector.tsx",
|
||||
]);
|
||||
for (const { markup } of availableCommandLists) {
|
||||
expect(markup).toContain('className="thot-fleet-action-selector__select"');
|
||||
}
|
||||
|
||||
expect(styles).toMatch(
|
||||
/\.thot-fleet-action-selector__select\s*\{[^}]*font-size:\s*0\.9375rem;/s,
|
||||
);
|
||||
expect(styles).toMatch(
|
||||
/\.thot-fleet-action-selector\s*\{[^}]*flex:\s*0 0 auto;/s,
|
||||
);
|
||||
expect(styles).toMatch(
|
||||
/\.thot-fleet-action-selector__select\s*\{[^}]*width:\s*max-content;/s,
|
||||
);
|
||||
});
|
||||
@@ -274,7 +274,7 @@
|
||||
background: oklch(var(--card));
|
||||
}
|
||||
|
||||
.thot-fleet-back-action {
|
||||
.thot-fleet-ledger__navigation > button:first-child {
|
||||
min-height: 2.25rem;
|
||||
border: 1px solid oklch(var(--primary) / 0.28);
|
||||
color: oklch(var(--primary));
|
||||
@@ -283,7 +283,7 @@
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.thot-fleet-back-action:not(:disabled):hover {
|
||||
.thot-fleet-ledger__navigation > button:first-child:not(:disabled):hover {
|
||||
border-color: oklch(var(--primary) / 0.48);
|
||||
color: oklch(var(--primary));
|
||||
background: oklch(var(--primary) / 0.14);
|
||||
@@ -381,8 +381,7 @@
|
||||
.thot-fleet-action-selector {
|
||||
display: flex;
|
||||
min-width: 0;
|
||||
max-width: 100%;
|
||||
flex: 0 0 auto;
|
||||
flex: 1;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.5rem;
|
||||
align-items: center;
|
||||
@@ -403,17 +402,15 @@
|
||||
}
|
||||
|
||||
.thot-fleet-action-selector__select {
|
||||
flex: 0 0 auto;
|
||||
width: max-content;
|
||||
min-width: 0;
|
||||
max-width: 100%;
|
||||
width: min(100%, 22rem);
|
||||
min-width: 13rem;
|
||||
height: 2rem;
|
||||
padding: 0 2rem 0 0.625rem;
|
||||
border: 1px solid oklch(var(--input));
|
||||
border-radius: var(--radius);
|
||||
color: oklch(var(--foreground));
|
||||
background: oklch(var(--background));
|
||||
font-size: 0.9375rem;
|
||||
font-size: 0.8125rem;
|
||||
outline: none;
|
||||
}
|
||||
|
||||
@@ -434,6 +431,24 @@
|
||||
line-height: 1.4;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model {
|
||||
min-width: min(100%, 14rem);
|
||||
flex: 0 1 16rem;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model > div {
|
||||
min-width: 0;
|
||||
gap: 0.25rem;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model label {
|
||||
gap: 0.25rem;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model select {
|
||||
height: 2rem;
|
||||
}
|
||||
|
||||
.thot-fleet-ledger__selection-summary {
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
@@ -462,11 +477,11 @@
|
||||
.thot-fleet-ledger__drawer {
|
||||
position: absolute;
|
||||
z-index: 50;
|
||||
inset-block: clamp(0.625rem, 2vh, 1rem);
|
||||
inset-block: 0.75rem;
|
||||
inset-inline-start: 50%;
|
||||
inset-inline-end: auto;
|
||||
display: flex;
|
||||
width: min(75rem, calc(100% - 2rem));
|
||||
width: 60%;
|
||||
min-width: 0;
|
||||
flex-direction: column;
|
||||
overflow: hidden;
|
||||
@@ -482,10 +497,10 @@
|
||||
|
||||
.thot-fleet-ledger .thot-fleet-ledger__context-panel {
|
||||
position: absolute;
|
||||
inset-block: clamp(0.625rem, 2vh, 1rem);
|
||||
inset-block: 0.75rem;
|
||||
inset-inline-start: 50%;
|
||||
inset-inline-end: auto;
|
||||
width: min(75rem, calc(100% - 2rem));
|
||||
width: 60%;
|
||||
max-width: none;
|
||||
min-width: 0;
|
||||
transform: translateX(-50%);
|
||||
@@ -540,20 +555,6 @@
|
||||
box-shadow: inset 0 -1px 0 oklch(var(--primary-foreground) / 0.18);
|
||||
}
|
||||
|
||||
.thot-sensitive-run--success .thot-fleet-ledger__drawer-header,
|
||||
.thot-sensitive-run--success .thot-catalog-drawer__header {
|
||||
color: oklch(var(--primary-foreground));
|
||||
background-color: oklch(var(--success));
|
||||
border-color: oklch(var(--primary-foreground) / 0.18);
|
||||
}
|
||||
|
||||
.thot-sensitive-run--failure .thot-fleet-ledger__drawer-header,
|
||||
.thot-sensitive-run--failure .thot-catalog-drawer__header {
|
||||
color: oklch(var(--destructive-foreground));
|
||||
background-color: oklch(var(--destructive));
|
||||
border-color: oklch(var(--destructive-foreground) / 0.18);
|
||||
}
|
||||
|
||||
.thot-fleet-ledger__drawer-header .thot-fleet-ledger__drawer-title,
|
||||
.thot-fleet-ledger__drawer-header .thot-fleet-ledger__drawer-description,
|
||||
.thot-fleet-ledger__drawer-header .thot-fleet-ledger__eyebrow,
|
||||
@@ -581,30 +582,6 @@
|
||||
box-shadow: inset 3px 0 0 oklch(var(--primary));
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__history button[data-status="completed"],
|
||||
.thot-catalog-drawer__history button[data-status="succeeded"] {
|
||||
background: oklch(var(--success) / 0.1);
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__history button[aria-current="true"][data-status="completed"],
|
||||
.thot-catalog-drawer__history button[aria-current="true"][data-status="succeeded"] {
|
||||
background: oklch(var(--success) / 0.1);
|
||||
box-shadow: inset 0 0 0 1px oklch(var(--success) / 0.5);
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__event-log {
|
||||
min-height: 8rem;
|
||||
max-height: 18rem;
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__history-list {
|
||||
max-height: 14rem;
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__run-layout > .thot-catalog-drawer__history {
|
||||
margin-top: 1.25rem;
|
||||
}
|
||||
|
||||
.thot-fleet-grid-surface {
|
||||
position: relative;
|
||||
}
|
||||
@@ -613,13 +590,6 @@
|
||||
background: oklch(var(--muted) / 0.32);
|
||||
}
|
||||
|
||||
@media (min-width: 48rem) {
|
||||
.thot-fleet-grid-toolbar > .thot-fleet-action-selector {
|
||||
flex: 0 0 auto;
|
||||
min-width: 0;
|
||||
}
|
||||
}
|
||||
|
||||
.thot-fleet-ledger-grid {
|
||||
--ag-header-background-color: oklch(var(--muted) / 0.84);
|
||||
--ag-odd-row-background-color: oklch(var(--card));
|
||||
@@ -642,14 +612,6 @@
|
||||
background: oklch(var(--muted) / 0.12);
|
||||
}
|
||||
|
||||
/* Keep selected rows readable without overpowering the status indicators. */
|
||||
.thot-fleet-ledger-grid .ag-row.ag-row-selected,
|
||||
.thot-fleet-ledger-grid .ag-row.ag-row-selected .ag-cell,
|
||||
.thot-fleet-ledger-grid .ag-row.ag-row-selected .ag-grid-pinned-left-cells,
|
||||
.thot-fleet-ledger-grid .ag-row.ag-row-selected .ag-grid-pinned-right-cells {
|
||||
background-color: oklch(93% 0.002 17.2) !important;
|
||||
}
|
||||
|
||||
.thot-fleet-icon-action {
|
||||
position: relative;
|
||||
}
|
||||
@@ -661,29 +623,24 @@
|
||||
.thot-fleet-icon-action::after {
|
||||
position: absolute;
|
||||
z-index: 80;
|
||||
inset-inline-end: 0;
|
||||
inset-block-start: calc(100% + 3px);
|
||||
inset-inline-end: calc(100% + 5px);
|
||||
inset-block-start: 50%;
|
||||
width: max-content;
|
||||
max-width: 12rem;
|
||||
padding: 5px 8px;
|
||||
border: 1px solid oklch(42% 0.018 258);
|
||||
border-radius: 6px;
|
||||
color: oklch(96% 0.006 258);
|
||||
background: oklch(24% 0.014 258);
|
||||
box-shadow: 0 8px 20px -10px oklch(12% 0.02 258 / 0.72);
|
||||
padding: 0.3rem 0.45rem;
|
||||
border: 1px solid oklch(var(--border));
|
||||
border-radius: calc(var(--radius) - 0.125rem);
|
||||
color: oklch(var(--popover-foreground));
|
||||
background: oklch(var(--popover));
|
||||
box-shadow: var(--shadow-sm);
|
||||
content: attr(data-tooltip);
|
||||
font-size: 0.6875rem;
|
||||
font-weight: 600;
|
||||
line-height: 1.25;
|
||||
text-align: center;
|
||||
white-space: normal;
|
||||
font-weight: 650;
|
||||
line-height: 1.2;
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
transform: translateY(-2px);
|
||||
transition:
|
||||
opacity 90ms linear,
|
||||
transform 120ms cubic-bezier(0.25, 1, 0.5, 1),
|
||||
visibility 0ms linear 120ms;
|
||||
transform: translate(2px, -50%);
|
||||
transition: opacity 60ms cubic-bezier(0.22, 1, 0.36, 1), transform 60ms cubic-bezier(0.22, 1, 0.36, 1);
|
||||
visibility: hidden;
|
||||
}
|
||||
|
||||
@@ -720,8 +677,7 @@
|
||||
.thot-fleet-icon-action:focus-visible::after,
|
||||
.thot-fleet-icon-action:focus-within::after {
|
||||
opacity: 1;
|
||||
transform: translateY(0);
|
||||
transition-delay: 60ms;
|
||||
transform: translate(0, -50%);
|
||||
visibility: visible;
|
||||
}
|
||||
|
||||
@@ -789,10 +745,6 @@
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.thot-fleet-ledger__drawer-body.thot-catalog-drawer__run-layout {
|
||||
padding: clamp(0.75rem, 1.5cqi, 1.25rem);
|
||||
}
|
||||
|
||||
.thot-fleet-ledger__drawer-footer {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
@@ -833,8 +785,8 @@
|
||||
@container fleet-surface (max-width: 47.999rem) {
|
||||
.thot-fleet-ledger__drawer,
|
||||
.thot-fleet-ledger .thot-fleet-ledger__context-panel {
|
||||
inset-block: 0.625rem;
|
||||
width: calc(100% - 1.5rem);
|
||||
inset-block: 0.5rem;
|
||||
width: calc(100% - 1rem);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -865,11 +817,11 @@
|
||||
}
|
||||
}
|
||||
|
||||
@container work-area-panel (min-width: 48rem) {
|
||||
@media (min-width: 40rem) {
|
||||
.thot-catalog-drawer__history-layout {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, 1fr) clamp(12.5rem, 25cqi, 15rem);
|
||||
gap: 0 clamp(0.75rem, 2cqi, 1.25rem);
|
||||
grid-template-columns: minmax(0, 1fr) 13.5rem;
|
||||
gap: 0 1rem;
|
||||
align-content: start;
|
||||
}
|
||||
|
||||
@@ -885,45 +837,6 @@
|
||||
align-self: start;
|
||||
margin-top: 0;
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__run-layout {
|
||||
height: 100%;
|
||||
min-height: 0;
|
||||
grid-template-rows: minmax(0, 1fr);
|
||||
align-content: stretch;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__run-layout > .thot-catalog-drawer__run-main {
|
||||
display: flex;
|
||||
min-height: 0;
|
||||
flex-direction: column;
|
||||
grid-column: 1;
|
||||
grid-row: 1;
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__run-layout > .thot-catalog-drawer__history {
|
||||
position: static;
|
||||
display: flex;
|
||||
min-height: 0;
|
||||
flex-direction: column;
|
||||
align-self: stretch;
|
||||
grid-row: 1;
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__run-layout .thot-catalog-drawer__events {
|
||||
display: flex;
|
||||
min-height: 10rem;
|
||||
flex: 1;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__run-layout .thot-catalog-drawer__event-log,
|
||||
.thot-catalog-drawer__run-layout .thot-catalog-drawer__history-list {
|
||||
min-height: 0;
|
||||
max-height: none;
|
||||
flex: 1;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 47.999rem) {
|
||||
@@ -965,6 +878,9 @@
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.thot-fleet-action-selector__select {
|
||||
width: 100%;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-height: 44rem) and (min-width: 48rem) {
|
||||
|
||||
@@ -233,7 +233,7 @@ export function FleetLedgerBreadcrumb({
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="thot-fleet-nav-action thot-fleet-back-action"
|
||||
className="thot-fleet-nav-action"
|
||||
data-tooltip={back.label.replace(/^Back to\s+/i, "")}
|
||||
onClick={back.onBack}
|
||||
disabled={back.disabled}
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { describe, expect, test } from "vitest";
|
||||
|
||||
const pageSource = readFileSync(resolve(__dirname, "../DatabaseManagementPage.tsx"), "utf8");
|
||||
const componentSource = (name: string) => readFileSync(resolve(__dirname, name), "utf8");
|
||||
|
||||
describe("history action placement", () => {
|
||||
test("workspace header does not own database history actions", () => {
|
||||
const fleetHeader = pageSource.slice(pageSource.indexOf('title="Database management"'), pageSource.indexOf("kpis={"));
|
||||
expect(fleetHeader).not.toContain("Description history");
|
||||
expect(fleetHeader).not.toContain("Sensitive history");
|
||||
});
|
||||
|
||||
test("database tables do not own metadata history actions", () => {
|
||||
for (const file of ["DatabaseTables.tsx"]) {
|
||||
const source = componentSource(file);
|
||||
expect(source, file).not.toContain("Description history");
|
||||
expect(source, file).not.toContain("Sensitive history");
|
||||
}
|
||||
for (const file of ["DatabaseForm.tsx", "DatabaseRelationships.tsx"]) {
|
||||
const source = componentSource(file);
|
||||
expect(source, file).toContain("Description history");
|
||||
expect(source, file).toContain("Sensitive history");
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,9 +1,5 @@
|
||||
import { useId } from "react";
|
||||
import type { MetadataGenerationModels } from "../../api/catalog-databases";
|
||||
|
||||
export const NO_METADATA_GENERATION_LLM_MODEL_MESSAGE =
|
||||
"No metadata-generation LLM model is configured for this installation.";
|
||||
|
||||
interface Props {
|
||||
canManage: boolean;
|
||||
data?: MetadataGenerationModels;
|
||||
@@ -11,7 +7,6 @@ interface Props {
|
||||
isLoading: boolean;
|
||||
selectedModel: string;
|
||||
onSelectedModelChange: (modelId: string) => void;
|
||||
variant?: "default" | "compact";
|
||||
}
|
||||
|
||||
export function MetadataGenerationModelSelector({
|
||||
@@ -21,35 +16,21 @@ export function MetadataGenerationModelSelector({
|
||||
isLoading,
|
||||
selectedModel,
|
||||
onSelectedModelChange,
|
||||
variant = "default",
|
||||
}: Props) {
|
||||
const helpId = useId();
|
||||
const models = data?.models ?? [];
|
||||
const configuredDefault = data?.default ?? "";
|
||||
const hasUsableDefault = configuredDefault !== ""
|
||||
&& models.some((model) => model.id === configuredDefault);
|
||||
const noUsableModel = !isLoading && !isError && Boolean(data) && !hasUsableDefault;
|
||||
const unavailableMessage = !isLoading && isError
|
||||
? "Metadata-generation LLM model choices are unavailable."
|
||||
: noUsableModel
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
: null;
|
||||
const compact = variant === "compact";
|
||||
|
||||
const unavailable = noUsableModel || (!isLoading && isError);
|
||||
return (
|
||||
<div className={compact
|
||||
? "flex min-w-0 flex-wrap items-center gap-x-2 gap-y-1"
|
||||
: "grid min-w-56 max-w-lg gap-1.5"}
|
||||
>
|
||||
<label className={compact
|
||||
? "flex min-w-0 items-center gap-2 text-xs font-medium text-foreground"
|
||||
: "grid gap-1.5 text-xs font-medium text-foreground"}
|
||||
>
|
||||
<span className={compact ? "whitespace-nowrap" : undefined}>Metadata-generation LLM model</span>
|
||||
<div className="grid min-w-56 max-w-lg gap-1.5">
|
||||
<label className="grid gap-1.5 text-xs font-medium text-foreground">
|
||||
<span>Metadata description model</span>
|
||||
<select
|
||||
className={`${compact ? "h-8 min-w-52" : "h-9 w-full"} rounded-md border border-input bg-card px-3 text-sm outline-none transition focus:border-primary/60 focus:ring-3 focus:ring-ring/15 disabled:bg-muted disabled:text-muted-foreground`}
|
||||
aria-label="Metadata-generation LLM model"
|
||||
aria-describedby={unavailableMessage ? helpId : undefined}
|
||||
className="h-9 w-full rounded-md border border-input bg-card px-3 text-sm outline-none transition focus:border-primary/60 focus:ring-3 focus:ring-ring/15 disabled:bg-muted disabled:text-muted-foreground"
|
||||
aria-label="Metadata description model"
|
||||
aria-describedby={unavailable ? "metadata-generation-model-help" : undefined}
|
||||
value={selectedModel}
|
||||
disabled={!canManage || isLoading || isError || !hasUsableDefault}
|
||||
onChange={(event) => onSelectedModelChange(event.target.value)}
|
||||
@@ -64,9 +45,9 @@ export function MetadataGenerationModelSelector({
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
{unavailableMessage ? (
|
||||
<p id={helpId} className="max-w-72 text-xs font-normal leading-4 text-muted-foreground">
|
||||
{unavailableMessage}
|
||||
{unavailable ? (
|
||||
<p id="metadata-generation-model-help" className="max-w-72 text-xs font-normal leading-4 text-muted-foreground">
|
||||
Configure a metadata-generation model in application setup to enable description generation.
|
||||
</p>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { describe, expect, test } from "vitest";
|
||||
|
||||
const sourceRoot = resolve(__dirname);
|
||||
|
||||
describe("Recent runs success styling contract", () => {
|
||||
test("every history renderer exposes its run status to the shared styles", () => {
|
||||
const renderers = [
|
||||
"DescriptionGenerationDrawer.tsx",
|
||||
"SensitiveDataSuggestionHistoryDrawer.tsx",
|
||||
"CatalogSyncDrawer.tsx",
|
||||
];
|
||||
|
||||
for (const renderer of renderers) {
|
||||
const source = readFileSync(resolve(sourceRoot, renderer), "utf8");
|
||||
expect(source, renderer).toMatch(/data-status=\{item\.(?:status|state)\}/);
|
||||
}
|
||||
});
|
||||
|
||||
test("shared history styles cover both successful status vocabularies", () => {
|
||||
const styles = readFileSync(resolve(sourceRoot, "FleetLedgerShell.css"), "utf8");
|
||||
expect(styles).toContain('button[data-status="completed"]');
|
||||
expect(styles).toContain('button[data-status="succeeded"]');
|
||||
expect(styles).toMatch(/oklch\(var\(--success\)/);
|
||||
});
|
||||
|
||||
test("run drawers keep responsive gutters and switch columns from their own width", () => {
|
||||
const styles = readFileSync(resolve(sourceRoot, "FleetLedgerShell.css"), "utf8");
|
||||
|
||||
expect(styles).toContain("width: min(75rem, calc(100% - 2rem));");
|
||||
expect(styles).toContain(
|
||||
".thot-fleet-ledger__drawer-body.thot-catalog-drawer__run-layout",
|
||||
);
|
||||
expect(styles).toContain("padding: clamp(0.75rem, 1.5cqi, 1.25rem);");
|
||||
expect(styles).toContain("@container work-area-panel (min-width: 48rem)");
|
||||
expect(styles).toContain(".thot-catalog-drawer__history-list");
|
||||
expect(styles).toContain(".thot-catalog-drawer__event-log");
|
||||
});
|
||||
});
|
||||
@@ -97,11 +97,6 @@ test("shows sensitive suggestion results, safe events, and lets operators inspec
|
||||
|
||||
const drawer = await screen.findByRole("dialog", { name: "Sensitive suggestion history" });
|
||||
expect(within(drawer).getByRole("heading", { name: "Running" })).toBeVisible();
|
||||
const progress = within(drawer).getByRole("table", { name: "Sensitive suggestion progress" });
|
||||
expect(within(progress).getAllByRole("columnheader")).toHaveLength(3);
|
||||
expect(within(drawer).getByRole("log", { name: "Sensitive suggestion events" })).toHaveClass(
|
||||
"thot-catalog-drawer__event-log",
|
||||
);
|
||||
expect(await within(drawer).findByText("Classifying columns")).toBeVisible();
|
||||
expect(within(drawer).queryByRole("button", { name: "Stop" })).not.toBeInTheDocument();
|
||||
expect(within(drawer).queryByRole("button", { name: "Unlock stale run" })).not.toBeInTheDocument();
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { useEffect, useMemo } from "react";
|
||||
import { LoaderCircle } from "lucide-react";
|
||||
import { useEffect } from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { apiErrorMessage } from "../../api/client";
|
||||
import {
|
||||
@@ -12,7 +11,6 @@ import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
|
||||
interface Props {
|
||||
open: boolean;
|
||||
databaseId?: string | null;
|
||||
run: SensitiveDataSuggestionRun | null;
|
||||
modelLabel: string;
|
||||
onClose: () => void;
|
||||
@@ -31,12 +29,6 @@ function isTerminal(run?: SensitiveDataSuggestionRun): boolean {
|
||||
return Boolean(run && ["completed", "failed", "interrupted"].includes(run.status));
|
||||
}
|
||||
|
||||
function outcomeClass(run: SensitiveDataSuggestionRun): string {
|
||||
if (run.status === "completed") return "thot-sensitive-run--success";
|
||||
if (["failed", "interrupted"].includes(run.status)) return "thot-sensitive-run--failure";
|
||||
return "";
|
||||
}
|
||||
|
||||
function safeFinalError(summary: string | null): string | null {
|
||||
const value = summary?.trim();
|
||||
if (!value) return null;
|
||||
@@ -54,7 +46,6 @@ function timestamp(value: string | null): string {
|
||||
|
||||
export function SensitiveDataSuggestionHistoryDrawer({
|
||||
open,
|
||||
databaseId = null,
|
||||
run: initialRun,
|
||||
modelLabel,
|
||||
onClose,
|
||||
@@ -87,15 +78,11 @@ export function SensitiveDataSuggestionHistoryDrawer({
|
||||
? 3_000
|
||||
: false,
|
||||
});
|
||||
const visibleHistory = useMemo(
|
||||
() => (historyQuery.data ?? []).filter((item) => !databaseId || item.databaseId === databaseId),
|
||||
[databaseId, historyQuery.data],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open || initialRun || !visibleHistory[0]) return;
|
||||
onRunUpdate(visibleHistory[0]);
|
||||
}, [initialRun, onRunUpdate, open, visibleHistory]);
|
||||
if (!open || initialRun || !historyQuery.data?.[0]) return;
|
||||
onRunUpdate(historyQuery.data[0]);
|
||||
}, [historyQuery.data, initialRun, onRunUpdate, open]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!run) return;
|
||||
@@ -140,92 +127,56 @@ export function SensitiveDataSuggestionHistoryDrawer({
|
||||
|
||||
const events = eventQuery.data ?? [];
|
||||
const finalError = safeFinalError(run.errorSummary);
|
||||
const modelDisplay = modelLabel && modelLabel !== run.modelId
|
||||
? `${modelLabel} (${run.modelId})`
|
||||
: run.modelId;
|
||||
const progressCounters = [
|
||||
["total", "Total columns"],
|
||||
["suggestedSensitive", "Sensitive"],
|
||||
["suggestedNonSensitive", "Not sensitive"],
|
||||
] as const;
|
||||
return (
|
||||
<FleetLedgerDrawer
|
||||
open
|
||||
className={outcomeClass(run)}
|
||||
ariaLabel="Sensitive suggestion history"
|
||||
eyebrow="Sensitive data suggestions"
|
||||
title={(
|
||||
<span className="inline-flex items-center gap-2">
|
||||
{!isTerminal(run) ? <LoaderCircle className="size-5 animate-spin" aria-hidden="true" /> : null}
|
||||
{statusLabel(run)}
|
||||
</span>
|
||||
)}
|
||||
description={run.scope.replaceAll("_", " ")}
|
||||
title={statusLabel(run)}
|
||||
description={`${modelLabel}${modelLabel !== run.modelId ? ` (${run.modelId})` : ""} · ${run.scope.replaceAll("_", " ")}`}
|
||||
onClose={onClose}
|
||||
closeLabel="Close sensitive suggestion history"
|
||||
bodyClassName="thot-catalog-drawer__history-layout thot-catalog-drawer__run-layout"
|
||||
bodyClassName="thot-catalog-drawer__history-layout"
|
||||
>
|
||||
<div className="thot-catalog-drawer__run-main">
|
||||
{runQuery.isError ? (
|
||||
{runQuery.isError ? (
|
||||
<div role="alert" className="rounded-md border border-destructive/35 bg-destructive/5 px-4 py-3 text-sm text-destructive">
|
||||
{apiErrorMessage(runQuery.error)}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
<div className={`grid gap-4 sm:grid-cols-2${runQuery.isError ? " mt-4" : ""}`}>
|
||||
<section aria-label="AI usage">
|
||||
<h3 className="thot-label mb-2">AI usage</h3>
|
||||
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1 text-xs">
|
||||
<dt>Model</dt><dd className="truncate text-right" title={modelDisplay}>{modelDisplay}</dd>
|
||||
<dt>Input tokens</dt><dd className="text-right tabular-nums">{(run.inputTokens ?? 0).toLocaleString("en-US")}</dd>
|
||||
<dt>Cache tokens</dt><dd className="text-right tabular-nums">{(run.cacheReadTokens ?? 0).toLocaleString("en-US")}</dd>
|
||||
<dt>Output tokens</dt><dd className="text-right tabular-nums">{(run.outputTokens ?? 0).toLocaleString("en-US")}</dd>
|
||||
</dl>
|
||||
</section>
|
||||
<section aria-label="Sensitive suggestion timestamps">
|
||||
<h3 className="thot-label mb-2">Timestamps</h3>
|
||||
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1 text-xs">
|
||||
<dt className="text-muted-foreground">Created</dt><dd className="text-right tabular-nums">{timestamp(run.createdAt)}</dd>
|
||||
<dt className="text-muted-foreground">Started</dt><dd className="text-right tabular-nums">{timestamp(run.startedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Updated</dt><dd className="text-right tabular-nums">{timestamp(run.updatedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Finished</dt><dd className="text-right tabular-nums">{timestamp(run.finishedAt)}</dd>
|
||||
</dl>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section className="mt-4" aria-label="Sensitive suggestion counters">
|
||||
<section aria-label="Sensitive suggestion counters">
|
||||
<h3 className="thot-label mb-2">Results</h3>
|
||||
<div className="overflow-x-auto rounded-md border border-border bg-muted/20">
|
||||
<table className="w-full min-w-[20rem] table-fixed" aria-label="Sensitive suggestion progress">
|
||||
<thead>
|
||||
<tr className="border-b border-border bg-muted/45">
|
||||
{progressCounters.map(([name, label]) => (
|
||||
<th key={name} scope="col" className="border-l border-border px-2 py-1.5 text-left text-[10px] font-semibold uppercase tracking-wide text-muted-foreground first:border-l-0">
|
||||
{label}
|
||||
</th>
|
||||
))}
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
{progressCounters.map(([name]) => (
|
||||
<td key={name} data-count={name} className="border-l border-border px-2 py-1.5 text-base font-semibold tabular-nums first:border-l-0">
|
||||
{run[name]}
|
||||
</td>
|
||||
))}
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div className="grid grid-cols-2 gap-2">
|
||||
{([
|
||||
["total", "Total columns"],
|
||||
["suggestedSensitive", "Sensitive"],
|
||||
["suggestedNonSensitive", "Not sensitive"],
|
||||
] as const).map(([name, label]) => (
|
||||
<div key={name} className="rounded-md border border-border bg-muted/25 px-3 py-2">
|
||||
<p className="text-[11px] font-semibold uppercase tracking-wide text-muted-foreground">{label}</p>
|
||||
<p data-count={name} className="mt-1 text-lg font-semibold tabular-nums">{run[name]}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section className="mt-5" aria-label="Sensitive suggestion timestamps">
|
||||
<h3 className="thot-label mb-2">Timestamps</h3>
|
||||
<dl className="grid grid-cols-[auto_1fr] gap-x-4 gap-y-1.5 text-sm">
|
||||
<dt className="text-muted-foreground">Created</dt><dd className="text-right">{timestamp(run.createdAt)}</dd>
|
||||
<dt className="text-muted-foreground">Started</dt><dd className="text-right">{timestamp(run.startedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Updated</dt><dd className="text-right">{timestamp(run.updatedAt)}</dd>
|
||||
<dt className="text-muted-foreground">Finished</dt><dd className="text-right">{timestamp(run.finishedAt)}</dd>
|
||||
</dl>
|
||||
</section>
|
||||
|
||||
{finalError ? (
|
||||
<div className="mt-3 rounded-md border border-destructive/35 bg-destructive/5 px-4 py-3 text-sm text-destructive">
|
||||
<div className="mt-5 rounded-md border border-destructive/35 bg-destructive/5 px-4 py-3 text-sm text-destructive">
|
||||
{finalError}
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
<section className="thot-catalog-drawer__events mt-3" aria-label="Sensitive suggestion log">
|
||||
<section className="mt-5" aria-label="Sensitive suggestion log">
|
||||
<div className="mb-2 flex items-center justify-between">
|
||||
<h3 className="thot-label">Events</h3>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{events.length}</span>
|
||||
@@ -233,7 +184,7 @@ export function SensitiveDataSuggestionHistoryDrawer({
|
||||
<div
|
||||
role="log"
|
||||
aria-label="Sensitive suggestion events"
|
||||
className="thot-catalog-drawer__event-log overflow-y-auto rounded-md bg-zinc-950 p-3 font-mono text-xs leading-5 text-zinc-200"
|
||||
className="max-h-72 overflow-y-auto rounded-md bg-zinc-950 p-3 font-mono text-xs leading-5 text-zinc-200"
|
||||
>
|
||||
{events.length === 0 ? <p className="text-zinc-500">Waiting for events…</p> : events.map((event) => (
|
||||
<p key={event.sequence} className={event.level === "error" ? "text-red-300" : event.level === "warning" ? "text-amber-300" : undefined}>
|
||||
@@ -243,30 +194,28 @@ export function SensitiveDataSuggestionHistoryDrawer({
|
||||
</div>
|
||||
{eventQuery.isError ? <p role="alert" className="mt-2 text-sm text-destructive">{apiErrorMessage(eventQuery.error)}</p> : null}
|
||||
</section>
|
||||
</div>
|
||||
|
||||
<section className="thot-catalog-drawer__history" aria-label="Sensitive suggestion run history">
|
||||
<section className="thot-catalog-drawer__history mt-5" aria-label="Sensitive suggestion run history">
|
||||
<div className="mb-2 flex items-center justify-between gap-3">
|
||||
<h3 className="thot-label">Recent runs</h3>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">
|
||||
{visibleHistory.length}
|
||||
{historyQuery.data?.length ?? 0}
|
||||
</span>
|
||||
</div>
|
||||
{historyQuery.isError ? (
|
||||
<p role="alert" className="text-sm text-destructive">{apiErrorMessage(historyQuery.error)}</p>
|
||||
) : historyQuery.isLoading ? (
|
||||
<p className="text-sm text-muted-foreground">Loading suggestion history…</p>
|
||||
) : visibleHistory.length === 0 ? (
|
||||
) : (historyQuery.data ?? []).length === 0 ? (
|
||||
<p className="text-sm text-muted-foreground">No sensitive suggestion runs yet.</p>
|
||||
) : (
|
||||
<div className="thot-catalog-drawer__history-list divide-y divide-border overflow-y-auto rounded-md border border-border">
|
||||
{visibleHistory.map((item) => (
|
||||
<div className="max-h-56 divide-y divide-border overflow-y-auto rounded-md border border-border">
|
||||
{(historyQuery.data ?? []).map((item) => (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
aria-label={`${statusLabel(item)} · ${item.scope.replaceAll("_", " ")}`}
|
||||
aria-current={item.id === run.id ? "true" : undefined}
|
||||
data-status={item.status}
|
||||
className="flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25"
|
||||
onClick={() => onRunUpdate(item)}
|
||||
>
|
||||
|
||||
@@ -4,7 +4,7 @@ import type {
|
||||
DatabaseConfiguration,
|
||||
} from "../../api/catalog-databases";
|
||||
|
||||
export type DatabaseFormMode = "configure" | "view" | "edit" | "delete";
|
||||
export type DatabaseFormMode = "add" | "view" | "edit" | "delete";
|
||||
|
||||
export type DatabaseScreen =
|
||||
| { kind: "list" }
|
||||
@@ -13,6 +13,7 @@ export type DatabaseScreen =
|
||||
| {
|
||||
kind: DatabaseFormMode;
|
||||
workspaceId: string;
|
||||
workspaceLocked?: boolean;
|
||||
};
|
||||
|
||||
export type DatabaseBusyAction = "save" | "test" | "delete" | "retry-secrets" | "reload" | null;
|
||||
|
||||
@@ -176,7 +176,7 @@ test("compacts table, filter, detail, and rationale rows", () => {
|
||||
expect(within(rationale!).getByText("Rationale")).toHaveClass("mb-1");
|
||||
});
|
||||
|
||||
test("shows 'No dependencies' for a CTE with an empty depends_on", () => {
|
||||
test("shows 'no dependencies' for a CTE with an empty depends_on", () => {
|
||||
render(<CtePlanViewer plan={plan} />);
|
||||
expect(screen.getByText(/no dependencies/i)).toBeInTheDocument();
|
||||
});
|
||||
|
||||
@@ -93,7 +93,7 @@ function CteCard({ cte, total }: { cte: CtePlanCte; total: number }) {
|
||||
))}
|
||||
</div>
|
||||
) : (
|
||||
<p className="text-xs text-muted-foreground">No dependencies</p>
|
||||
<p className="text-xs text-muted-foreground">no dependencies</p>
|
||||
)}
|
||||
</section>
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ test("(b) clicking toggle switches to table listing a promoted table name", asyn
|
||||
test("(c) oversized linking (>45) shows cap notice", () => {
|
||||
render(<SchemaLinkingViewer linking={bigLinking} />);
|
||||
expect(
|
||||
screen.getByText(/too many nodes to display in the graph/i)
|
||||
screen.getByText(/too many elements for the graph/i)
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
|
||||
|
||||
@@ -147,7 +147,7 @@ export function SchemaLinkingViewer({ linking }: { linking: SchemaLinking }) {
|
||||
|
||||
{oversized && (
|
||||
<p className="text-sm text-amber-600">
|
||||
Too many nodes to display in the graph. Use the table instead.
|
||||
Too many elements for the graph, use the table
|
||||
</p>
|
||||
)}
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user