feat: consolidate database management work

Add catalog-owned logical relationships and runtime snapshots, extend the database-management UI and validation coverage, and document the updated operational workflow.

Keep active sensitive-generation status in a tooltip and indicator, and update the layout E2E to follow the history action in its new database-scoped location.
This commit is contained in:
Codex
2026-09-01 14:46:55 +02:00
parent f586152636
commit 076c9742c5
73 changed files with 6966 additions and 610 deletions
+34 -2
View File
@@ -296,8 +296,40 @@ Metadata Catalog. Non è creata o modificata manualmente, ma può essere rimossa
Metadata Cleanup.
_Avoid_: denormalized FK, relationship string
**Logical Relationship** — Una relazione semantica curata o inferita che non corrisponde
necessariamente a un vincolo fisico. Ha ownership e lifecycle distinti da Catalog Relationship.
**Logical Relationship** — Una relazione modificabile fra due Catalog Column che non corrisponde
necessariamente a un vincolo fisico. Può essere Generated o Manual e rimane distinta dalla Catalog
Relationship osservata nel database.
**Generated Relationship** — Una Logical Relationship ricavata dai nomi delle colonne, dalle
primary key e dalla compatibilità dei tipi mediante regole deterministiche, senza LLM, embedding o
campionamento dei dati. Una ricostruzione non riattiva una Generated Relationship cancellata
logicamente, ma può ricrearne una cancellata fisicamente.
**Manual Relationship** — Una Logical Relationship aggiunta dall'utente. La ricostruzione delle
Generated Relationship non la modifica.
**Logical Relationship Deletion** — L'esclusione persistente di una Logical Relationship che ne
conserva l'identità per impedirne la ricreazione automatica finché esistono entrambe le Catalog
Column alle quali è collegata.
**Permanent Relationship Deletion** — La rimozione completa di una Logical Relationship. Una
ricostruzione successiva può ricrearla quando soddisfa nuovamente le regole di inferenza. Anche il
cleanup distruttivo di una tabella o colonna endpoint rimuove permanentemente le relative esclusioni.
**Relationship Reconstruction** — L'operazione amministrativa esplicita che scopre e aggiunge le
Generated Relationship mancanti. Conserva le Manual Relationship e le relationship già presenti e
non riattiva quelle cancellate logicamente.
**Relationship Restore** — La riattivazione esplicita di una Logical Relationship cancellata
logicamente.
**Effective Relationship Map** — La vista unificata delle Catalog Relationship fisiche e delle
Logical Relationship, con origine e stato espliciti. È l'interfaccia usata dall'amministrazione e
dalla comprensione dello schema, non un ulteriore modello persistito.
**Effective Relationship Snapshot** — La proiezione runtime immutabile delle relationship attive
contenute nell'Effective Relationship Map. È derivata dal Metadata Catalog per una singola sessione
e viene eliminata insieme alla relativa configurazione runtime.
**Description** — Il testo curato e consolidato che descrive una Catalog Table o Catalog Column
per gli usi downstream.
+40 -4
View File
@@ -12,6 +12,10 @@ colors:
muted-graphite: "oklch(51.33% 0.0088 345.6)"
quiet-border: "oklch(90.93% 0.0035 354.7)"
success-mint: "oklch(75.77% 0.1581 165)"
navigation-active: "oklch(92.5% 0.052 23.2)"
navigation-active-hover: "oklch(89.5% 0.071 23.2)"
navigation-active-foreground: "oklch(36.5% 0.11 23.2)"
navigation-active-border: "oklch(60% 0.135 23.2)"
warning-amber: "oklch(85.23% 0.1386 78.9)"
information-blue: "oklch(70.35% 0.1128 221.3)"
typography:
@@ -152,8 +156,8 @@ frontend uses OKLCH tokens directly.
### Primary
- **Instrument Red** (`instrument-red`): primary actions, current selection, focus identity, and
destructive meaning where the context already makes the action explicit.
- **Instrument Red** (`instrument-red`): primary actions, focus identity, and destructive meaning
where the context already makes the action explicit.
- **Instrument Red Pressed** (`instrument-red-hover`): hover and active emphasis for the primary
action family.
@@ -170,6 +174,9 @@ frontend uses OKLCH tokens directly.
### Semantic
- **Success Mint** (`success-mint`): completed and ready states.
- **Navigation Active** (`navigation-active`): the one application surface currently in the
foreground. It shares Instrument Red's hue but uses a lighter, lower-chroma fill, so location is
visible without carrying the full weight of a primary action.
- **Warning Amber** (`warning-amber`): waiting, attention, and in-progress states.
- **Information Blue** (`information-blue`): informational state when red would imply action.
@@ -272,15 +279,44 @@ default, hover, focus, active, disabled, loading, and error behavior where those
- **Focus:** three-pixel Instrument Red ring with a clear border shift.
- **Error / Disabled:** errors combine destructive color with explanatory text; disabled controls
retain readable contrast and use 50 percent opacity.
- **Metadata catalog model:** Database Management keeps one compact, installation-level
metadata-generation LLM selector in the application header. The selection persists across
database, table, column, and relationship views; when no usable profile is configured, the
disabled control explains: “No metadata-generation LLM model is configured for this installation.”
### Navigation
- **Style:** compact session rows use `8px` corners and restrained vertical padding.
- **Default / Hover / Active:** transparent at rest, Sunken Surface on hover, and the same surface
with stronger text weight when active.
- **Default / Hover / Active:** porcelain at rest, Sunken Surface on hover, and a muted Navigation
Active red with a defined border when current. Exactly one top-level navigation control is current.
- **Administrative controls:** the admin-only Administration accordion groups Database management,
a structural divider, Workspace management, and Pi management in that order. Its trigger exposes
expanded state and starts collapsed by default, while non-admin users do not receive the accordion
or its navigation actions.
- **Responsive:** collapse navigation structurally at the application breakpoint. Do not shrink
labels into illegibility.
### Tabs
- **Shape:** compact label tabs sit on a shared baseline with rounded top corners and a two-pixel
lower edge. Inactive labels retain a complete Quiet Border and Porcelain Card surface, so every
label reads as a tab before interaction; hover feedback reinforces clickability.
- **Current:** the selected tab uses the muted Navigation Active red for its fill, text, and defined border.
It must expose `aria-selected`, participate in a labelled `tablist`/`tabpanel`, and be the only
tab in the roving keyboard tab order.
- **Keyboard:** Left/Right move between adjacent tabs with wrapping; Home/End select the first or
last tab.
### Tooltips
- **Row actions:** icon-action tooltips open three pixels below the trigger and align to its trailing
edge, so they never cover the icon row. They use a dark slate surface, porcelain text, and a
defined border rather than the light popover treatment.
- **Interaction:** tooltip layers never receive pointer events. They appear on hover and keyboard
focus with a short ease-out transition, while the icon button keeps its complete accessible name.
- **Scope:** this treatment is shared by database, table, column, and relationship row actions.
Toolbar and navigation hints may use separate collision-aware placement.
### Curated Evidence Documents
Curated evidence follows a fixed reading order: title, compact type and purpose summary, scope,
+25 -11
View File
@@ -1,6 +1,6 @@
# ThothII — Project State
Last updated: 2026-08-27.
Last updated: 2026-08-31.
This file is the short operational snapshot. Stable commands and the architecture mental model
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
@@ -90,6 +90,18 @@ relationships. Curated and generated descriptions are editable; generated descri
and Database Management can generate or consolidate them for selected tables, selected columns,
all targets, or only targets whose Generated Description is missing.
Relationship Management is now reachable directly from each configured Fleet database. One
Relationship Map shows read-only Physical Relationships together with Generated and Manual Logical
Relationships, with Active, Excluded, and All filters. Administrators can add a single-column
relationship, run deterministic name/PK/type inference, exclude or restore a logical relationship,
or delete it permanently. Exclusion retains a tombstone that a rebuild cannot reactivate; permanent
deletion allows a later rebuild to infer the same endpoints again. Inference uses no LLM, embedding,
or source values. It supports normalized table-qualified names, unique non-generic PK names,
composite-PK source columns, and the `*time_key -> dim_time.<single PK>` warehouse convention while
ignoring bare generic names. Explicit table/column metadata cleanup remains a destructive boundary: it removes
the attached logical relationships and exclusions and requires a full schema synchronization before
inference or runtime publication can continue.
The previous Database Management renderer remains a temporary comparison fallback for development
and staging only: `?db-ui=legacy` is honored in Vite development or when
`VITE_DB_MANAGEMENT_LEGACY=true`; it is not a production presentation. The standalone Fleet Ledger
@@ -115,13 +127,15 @@ SSH is not yet enabled for NL→SQL session runtime.
The catalog runs in the internal `catalog-db` PostgreSQL service. Kysely migrations are an explicit
one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before local startup. Runtime
sessions still consume the existing workspace configuration in this slice: database-management
records do not yet change the NL→SQL handoff. The accepted design is recorded in
sessions now consume the Catalog's active effective relationship map through an immutable JSON
snapshot tied to the runtime-config lease. The harness uses that snapshot as its exclusive
relationship source while retaining Git-pinned annotations for descriptive metadata; legacy
runtimes without a snapshot keep the previous merge behavior. The accepted design is recorded in
`docs/plans/2026-08-26-metadata-catalog-from-thothai.md`, the snapshot contract under
`docs/contracts/`, and ADRs 0001–0011.
`docs/contracts/`, and ADRs 0001–0012.
Semantic aliases, value descriptions, synonyms, concepts, and logical relationships remain
deferred to their dedicated slices.
Semantic aliases, value descriptions, synonyms, and concepts remain deferred to their dedicated
slices.
AI Description Generation uses the catalog's human-owned Sensitive Data Flag. The flag defaults to
`false`, including for newly synchronized columns. An administrator may request an AI proposal based
@@ -158,11 +172,11 @@ credential. The Python client supplies only its fixed non-secret compatibility p
The AritmoLab entry also sets `disableThinking: true`, mapped to the endpoint's chat-template flag,
because its default reasoning prose would violate the worker's exact JSON response contract.
Integration of the completed metadata catalog with core schema-linking is explicitly deferred
until the database, table, column, relationship, and synchronization slices are complete. At that
point the next required design gate is to compare the catalog snapshot with the current DWH
preprocessing/schema-linking contracts and plan the cutover; this follow-up must not be treated as
optional cleanup or silently omitted.
Logical relationship integration with core schema-linking is complete: session creation and resume
materialize the active physical/generated/manual map, retrieval-pack generation and Pi receive the
same runtime config, and snapshot validation fails closed on a declared missing, invalid, or orphaned
endpoint. Broader publication of other Catalog metadata to schema-linking remains a separate future
slice.
**Deferred follow-up — Sensitive Data Policy in schema-linking.** The policy is first delivered
and tested in catalog description generation. Its enforcement for core schema-linking remains
+20
View File
@@ -63,6 +63,9 @@ import {
type DescriptionSourceSampler,
} from "./catalog/description-source-sampler.js";
import { catalogDescriptionGenerationRoutes } from "./routes/catalog-description-generation.js";
import { CatalogLogicalRelationshipService } from "./catalog/logical-relationship-service.js";
import { catalogLogicalRelationshipRoutes } from "./routes/catalog-logical-relationships.js";
import { EffectiveRelationshipSnapshotProvider } from "./catalog/effective-relationship-snapshot.js";
export interface BuildAppDeps {
thtRunner?: ThtRunner;
@@ -79,6 +82,8 @@ export interface BuildAppDeps {
catalogService?: CatalogService;
catalogPostgresAccess?: CatalogPostgresAccess;
catalogTableService?: CatalogTableService;
catalogLogicalRelationshipService?: CatalogLogicalRelationshipService;
effectiveRelationshipSnapshotProvider?: EffectiveRelationshipSnapshotProvider;
catalogSchemaIntrospector?: CatalogSchemaIntrospector;
catalogSyncWorker?: CatalogSyncWorker;
catalogOperationCoordinator?: CatalogOperationCoordinator;
@@ -200,6 +205,16 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
catalogOperationCoordinator,
);
const catalogTableService = deps?.catalogTableService ?? new CatalogTableService(catalogRepository);
const catalogLogicalRelationshipService = deps?.catalogLogicalRelationshipService
?? new CatalogLogicalRelationshipService(catalogRepository);
const effectiveRelationships = deps?.effectiveRelationshipSnapshotProvider
?? (config.catalogDatabase === undefined
? undefined
: new EffectiveRelationshipSnapshotProvider(
catalogRepository,
catalogLogicalRelationshipService,
catalogOperationCoordinator,
));
const catalogSchemaIntrospector = deps?.catalogSchemaIntrospector ?? new ConcreteCatalogSchemaIntrospector(
catalogPostgresAccess,
workspaceSecretStore,
@@ -394,6 +409,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
legacyWorkspaceMode: config.legacyWorkspaceMode,
workspaceRuntimeSupport,
maintenanceBarrier,
effectiveRelationships,
});
app.post("/internal/maintenance/activate", async (req, reply) => {
try {
@@ -442,6 +458,10 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
worker: catalogSyncWorker,
operations: catalogOperationCoordinator,
});
catalogLogicalRelationshipRoutes(app, {
service: catalogLogicalRelationshipService,
operations: catalogOperationCoordinator,
});
catalogDescriptionConsolidationRoutes(app, {
repository: catalogRepository,
operations: catalogOperationCoordinator,
@@ -0,0 +1,93 @@
import type { CatalogRelationship, CatalogRepository } from "./types.js";
export interface EffectiveRelationshipSnapshotReader {
list(databaseId: string): Promise<CatalogRelationship[]>;
}
export interface EffectiveRelationshipSnapshotCoordinator {
run<T>(databaseId: string, operation: () => Promise<T>): Promise<T>;
}
export class EffectiveRelationshipSnapshotStaleError extends Error {}
interface EffectiveRelationship {
sourceTable: string;
sourceColumns: string[];
targetTable: string;
targetColumns: string[];
origin: CatalogRelationship["origin"];
}
interface EffectiveRelationshipSnapshot {
schemaVersion: 1;
workspaceId: string;
relationships: EffectiveRelationship[];
}
const originRank: Record<CatalogRelationship["origin"], number> = {
physical: 0,
manual: 1,
generated: 2,
};
function endpointKey(relationship: EffectiveRelationship): string {
return [
relationship.sourceTable,
relationship.sourceColumns.join("\u0000"),
relationship.targetTable,
relationship.targetColumns.join("\u0000"),
].join("\u0001");
}
function compareRelationships(left: EffectiveRelationship, right: EffectiveRelationship): number {
return endpointKey(left).localeCompare(endpointKey(right))
|| originRank[left.origin] - originRank[right.origin];
}
/**
* Adapter from the mutable Catalog model to the immutable relationship contract consumed by the
* harness. The returned JSON is a deterministic projection, never an authored second store.
*/
export class EffectiveRelationshipSnapshotProvider {
constructor(
private readonly repository: Pick<CatalogRepository, "get" | "getByWorkspace">,
private readonly relationships: EffectiveRelationshipSnapshotReader,
private readonly operations: EffectiveRelationshipSnapshotCoordinator,
) {}
async render(workspaceId: string): Promise<string | undefined> {
const database = await this.repository.getByWorkspace(workspaceId);
if (!database) return undefined;
return await this.operations.run(database.id, async () => {
const current = await this.repository.get(database.id);
if (!current || current.schemaSyncedVersion !== current.version) {
throw new EffectiveRelationshipSnapshotStaleError(
"effective relationship snapshot requires a current full schema synchronization",
);
}
const projected = (await this.relationships.list(database.id))
.filter((relationship) => relationship.status === "active")
.map((relationship): EffectiveRelationship => ({
sourceTable: relationship.sourceTableName,
sourceColumns: relationship.columns.map((column) => column.sourceColumnName),
targetTable: relationship.targetTableName,
targetColumns: relationship.columns.map((column) => column.targetColumnName),
origin: relationship.origin,
}))
.sort(compareRelationships);
const seen = new Set<string>();
const snapshot: EffectiveRelationshipSnapshot = {
schemaVersion: 1,
workspaceId,
relationships: projected.filter((relationship) => {
const key = endpointKey(relationship);
if (seen.has(key)) return false;
seen.add(key);
return true;
}),
};
return `${JSON.stringify(snapshot, null, 2)}\n`;
});
}
}
@@ -0,0 +1,260 @@
import type {
CatalogLogicalRelationship,
CatalogLogicalRelationshipCandidate,
CatalogLogicalRelationshipContext,
CatalogLogicalRelationshipEndpoint,
CatalogRelationship,
CatalogRepository,
} from "./types.js";
export class LogicalRelationshipDatabaseNotFoundError extends Error {}
export class LogicalRelationshipDuplicateError extends Error {}
export class LogicalRelationshipNotFoundError extends Error {}
export class LogicalRelationshipReadOnlyError extends Error {}
export class LogicalRelationshipSchemaStaleError extends Error {}
export class LogicalRelationshipTargetNotUniqueError extends Error {}
export class LogicalRelationshipTypeIncompatibleError extends Error {}
export class LogicalRelationshipColumnNotFoundError extends Error {
constructor(readonly field: "sourceColumnId" | "targetColumnId") {
super(`Catalog column '${field}' was not found`);
}
}
export interface RebuildGeneratedRelationshipsResult {
added: number;
alreadyPresent: number;
excluded: number;
ambiguous: number;
}
function identifierTokens(value: string): string[] {
return value
.replace(/([a-z0-9])([A-Z])/g, "$1_$2")
.toLowerCase()
.split(/[^a-z0-9]+/)
.filter(Boolean);
}
function singularWord(value: string): string {
if (value.length > 4 && value.endsWith("ies")) return `${value.slice(0, -3)}y`;
if (value.length > 4 && /(ches|shes|xes|zes|ses)$/.test(value)) return value.slice(0, -2);
if (value.length > 3 && value.endsWith("s") && !/(ss|us)$/.test(value)) return value.slice(0, -1);
return value;
}
function tableAliases(tableName: string): string[] {
const tokens = identifierTokens(tableName);
if (tokens.length === 0) return [];
const normalized = tokens.join("_");
const singular = [...tokens];
singular[singular.length - 1] = singularWord(singular[singular.length - 1]);
return [...new Set([normalized, singular.join("_")])];
}
const GENERIC_PRIMARY_KEY_NAMES = new Set(["id", "key", "code", "pk"]);
function nameMatches(
source: CatalogLogicalRelationshipEndpoint,
target: CatalogLogicalRelationshipEndpoint,
): boolean {
const sourceName = identifierTokens(source.columnName).join("_");
const targetName = identifierTokens(target.columnName).join("_");
if (!sourceName || !targetName) return false;
const expected = new Set<string>();
if (!GENERIC_PRIMARY_KEY_NAMES.has(targetName)) expected.add(targetName);
for (const alias of tableAliases(target.tableName)) {
expected.add(`${alias}_${targetName}`);
expected.add(`${alias.replaceAll("_", "")}${targetName.replaceAll("_", "")}`);
if (targetName === "id" || targetName === "pk") expected.add(alias);
}
return expected.has(sourceName);
}
function canonicalDataType(value: string): string {
const normalized = value.trim().toLowerCase().replace(/\s+/g, " ");
const arraySuffix = normalized.endsWith("[]") ? "[]" : "";
const base = arraySuffix ? normalized.slice(0, -2) : normalized;
const withoutModifier = base.replace(/\([^)]*\)/g, "").trim();
const aliases: Record<string, string> = {
int2: "smallint",
smallserial: "smallint",
int4: "integer",
int: "integer",
serial: "integer",
int8: "bigint",
bigserial: "bigint",
decimal: "numeric",
varchar: "text",
"character varying": "text",
bool: "boolean",
"timestamp without time zone": "timestamp",
"timestamp with time zone": "timestamptz",
"time without time zone": "time",
"time with time zone": "timetz",
};
return `${aliases[withoutModifier] ?? withoutModifier}${arraySuffix}`;
}
function typesCompatible(left: string, right: string): boolean {
return canonicalDataType(left) === canonicalDataType(right);
}
function pairKey(sourceColumnId: string, targetColumnId: string): string {
return `${sourceColumnId}\u0000${targetColumnId}`;
}
function relationshipPair(relationship: CatalogLogicalRelationship): CatalogLogicalRelationshipCandidate {
return {
sourceColumnId: relationship.columns[0].sourceColumnId,
targetColumnId: relationship.columns[0].targetColumnId,
};
}
function relationshipSortKey(relationship: CatalogRelationship): string {
const sourceColumns = relationship.columns.map((column) => column.sourceColumnName).join(",");
const targetColumns = relationship.columns.map((column) => column.targetColumnName).join(",");
return [
relationship.sourceTableName,
sourceColumns,
relationship.targetTableName,
targetColumns,
relationship.origin,
].join("\u0000");
}
export class CatalogLogicalRelationshipService {
constructor(private readonly repository: CatalogRepository) {}
async list(databaseId: string): Promise<CatalogRelationship[]> {
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
const relationships: CatalogRelationship[] = [
...await this.repository.listRelationships(databaseId),
...await this.repository.listLogicalRelationships(databaseId),
];
return relationships.sort((left, right) => relationshipSortKey(left).localeCompare(relationshipSortKey(right)));
}
async addManual(
databaseId: string,
sourceColumnId: string,
targetColumnId: string,
): Promise<CatalogLogicalRelationship> {
const context = await this.requiredContext(databaseId);
const source = context.endpoints.find((endpoint) => endpoint.columnId === sourceColumnId);
if (!source) throw new LogicalRelationshipColumnNotFoundError("sourceColumnId");
const target = context.endpoints.find((endpoint) => endpoint.columnId === targetColumnId);
if (!target) throw new LogicalRelationshipColumnNotFoundError("targetColumnId");
if (sourceColumnId === targetColumnId
|| target.primaryKeyPosition === null
|| target.tablePrimaryKeyColumnCount !== 1) {
throw new LogicalRelationshipTargetNotUniqueError();
}
if (!typesCompatible(source.dataType, target.dataType)) {
throw new LogicalRelationshipTypeIncompatibleError();
}
const key = pairKey(sourceColumnId, targetColumnId);
if (context.physicalPairs.some((pair) => pairKey(pair.sourceColumnId, pair.targetColumnId) === key)
|| context.logicalRelationships.some((relationship) => {
const pair = relationshipPair(relationship);
return pairKey(pair.sourceColumnId, pair.targetColumnId) === key;
})) throw new LogicalRelationshipDuplicateError();
const created = await this.repository.insertLogicalRelationship(
databaseId,
sourceColumnId,
targetColumnId,
false,
);
if (!created) throw new LogicalRelationshipDuplicateError();
return created;
}
async rebuildGenerated(databaseId: string): Promise<RebuildGeneratedRelationshipsResult> {
const context = await this.requiredContext(databaseId);
const targets = context.endpoints.filter((endpoint) => (
endpoint.primaryKeyPosition !== null && endpoint.tablePrimaryKeyColumnCount === 1
));
const physical = new Set(context.physicalPairs.map((pair) => pairKey(pair.sourceColumnId, pair.targetColumnId)));
const active = new Set<string>();
const excluded = new Set<string>();
for (const relationship of context.logicalRelationships) {
const pair = relationshipPair(relationship);
(relationship.status === "excluded" ? excluded : active)
.add(pairKey(pair.sourceColumnId, pair.targetColumnId));
}
const pending: CatalogLogicalRelationshipCandidate[] = [];
let alreadyPresent = 0;
let excludedCount = 0;
let ambiguous = 0;
const dimTimeTargets = targets.filter((target) => (
identifierTokens(target.tableName).join("_") === "dim_time"
));
const sources = context.endpoints.filter((endpoint) => (
endpoint.primaryKeyPosition === null || endpoint.tablePrimaryKeyColumnCount > 1
));
for (const source of sources) {
const sourceName = identifierTokens(source.columnName).join("_");
const isTimeKey = sourceName.endsWith("time_key")
&& identifierTokens(source.tableName).join("_") !== "dim_time";
const candidates = isTimeKey ? dimTimeTargets : targets;
const matches = candidates.filter((target) => (
target.columnId !== source.columnId
&& typesCompatible(source.dataType, target.dataType)
&& (isTimeKey || nameMatches(source, target))
));
if (matches.length > 1) {
ambiguous += 1;
continue;
}
if (matches.length === 0) continue;
const candidate = { sourceColumnId: source.columnId, targetColumnId: matches[0].columnId };
const key = pairKey(candidate.sourceColumnId, candidate.targetColumnId);
if (physical.has(key) || active.has(key)) {
alreadyPresent += 1;
} else if (excluded.has(key)) {
excludedCount += 1;
} else {
pending.push(candidate);
}
}
const added = await this.repository.insertGeneratedLogicalRelationships(databaseId, pending);
alreadyPresent += pending.length - added;
return { added, alreadyPresent, excluded: excludedCount, ambiguous };
}
async setStatus(
databaseId: string,
relationshipId: string,
status: CatalogLogicalRelationship["status"],
): Promise<CatalogLogicalRelationship> {
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
const updated = await this.repository.setLogicalRelationshipStatus(databaseId, relationshipId, status);
if (updated) return updated;
await this.assertNotPhysical(databaseId, relationshipId);
throw new LogicalRelationshipNotFoundError();
}
async deletePermanently(databaseId: string, relationshipId: string): Promise<void> {
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
if (await this.repository.deleteLogicalRelationship(databaseId, relationshipId)) return;
await this.assertNotPhysical(databaseId, relationshipId);
throw new LogicalRelationshipNotFoundError();
}
private async requiredContext(databaseId: string): Promise<CatalogLogicalRelationshipContext> {
const database = await this.repository.get(databaseId);
if (!database) throw new LogicalRelationshipDatabaseNotFoundError();
if (database.schemaSyncedVersion !== database.version) {
throw new LogicalRelationshipSchemaStaleError();
}
const context = await this.repository.getLogicalRelationshipContext(databaseId);
if (!context) throw new LogicalRelationshipDatabaseNotFoundError();
return context;
}
private async assertNotPhysical(databaseId: string, relationshipId: string): Promise<void> {
if ((await this.repository.listRelationships(databaseId)).some((relationship) => relationship.id === relationshipId)) {
throw new LogicalRelationshipReadOnlyError();
}
}
}
+150 -3
View File
@@ -14,7 +14,10 @@ import {
type CatalogDatabaseMetadataDeleteTarget,
type CatalogMetadataDeleteCounts,
type CatalogMetrics,
type CatalogRelationship,
type CatalogLogicalRelationship,
type CatalogLogicalRelationshipCandidate,
type CatalogLogicalRelationshipContext,
type CatalogPhysicalRelationship,
type CatalogSchemaDiff,
type CatalogSyncCounts,
type CatalogSyncEvent,
@@ -49,7 +52,8 @@ export class MemoryCatalogRepository implements CatalogRepository {
private readonly records = new Map<string, WorkspaceDatabase>();
private readonly tables = new Map<string, CatalogTable>();
private readonly columns = new Map<string, CatalogColumn>();
private readonly relationships = new Map<string, CatalogRelationship>();
private readonly relationships = new Map<string, CatalogPhysicalRelationship>();
private readonly logicalRelationships = new Map<string, CatalogLogicalRelationship>();
private readonly descriptionGenerationRuns = new Map<string, DescriptionGenerationRun>();
private readonly descriptionGenerationEvents = new Map<string, DescriptionGenerationEvent[]>();
private readonly sensitiveDataSuggestionRuns = new Map<string, SensitiveDataSuggestionRun>();
@@ -171,6 +175,9 @@ export class MemoryCatalogRepository implements CatalogRepository {
for (const [relationshipId, relationship] of this.relationships) {
if (relationship.databaseId === id) this.relationships.delete(relationshipId);
}
for (const [relationshipId, relationship] of this.logicalRelationships) {
if (relationship.databaseId === id) this.logicalRelationships.delete(relationshipId);
}
for (const [runId, run] of this.descriptionGenerationRuns) {
if (run.databaseId !== id) continue;
this.descriptionGenerationRuns.delete(runId);
@@ -527,12 +534,138 @@ export class MemoryCatalogRepository implements CatalogRepository {
.map((event) => structuredClone(event));
}
async listRelationships(databaseId: string): Promise<CatalogRelationship[]> {
async listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]> {
return [...this.relationships.values()].filter((relationship) => relationship.databaseId === databaseId)
.sort((a, b) => `${a.sourceTableName}.${a.constraintName}`.localeCompare(`${b.sourceTableName}.${b.constraintName}`))
.map((relationship) => structuredClone(relationship));
}
async listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]> {
return [...this.logicalRelationships.values()]
.filter((relationship) => relationship.databaseId === databaseId)
.sort((a, b) => {
const left = `${a.sourceTableName}.${a.columns[0].sourceColumnName}.${a.targetTableName}.${a.columns[0].targetColumnName}`;
const right = `${b.sourceTableName}.${b.columns[0].sourceColumnName}.${b.targetTableName}.${b.columns[0].targetColumnName}`;
return left.localeCompare(right);
})
.map((relationship) => structuredClone(relationship));
}
async getLogicalRelationshipContext(
databaseId: string,
): Promise<CatalogLogicalRelationshipContext | undefined> {
if (!this.records.has(databaseId)) return undefined;
const tables = [...this.tables.values()].filter((table) => table.databaseId === databaseId);
const tableById = new Map(tables.map((table) => [table.id, table]));
const columns = [...this.columns.values()].filter((column) => tableById.has(column.tableId));
const primaryKeyCounts = new Map<string, number>();
for (const column of columns) {
if (column.primaryKeyPosition !== null) {
primaryKeyCounts.set(column.tableId, (primaryKeyCounts.get(column.tableId) ?? 0) + 1);
}
}
return {
endpoints: columns.map((column) => ({
columnId: column.id,
columnName: column.name,
tableId: column.tableId,
tableName: tableById.get(column.tableId)!.name,
dataType: column.dataType,
primaryKeyPosition: column.primaryKeyPosition,
tablePrimaryKeyColumnCount: primaryKeyCounts.get(column.tableId) ?? 0,
})),
physicalPairs: [...this.relationships.values()]
.filter((relationship) => relationship.databaseId === databaseId)
.flatMap((relationship) => relationship.columns.map((column) => ({
sourceColumnId: column.sourceColumnId,
targetColumnId: column.targetColumnId,
}))),
logicalRelationships: await this.listLogicalRelationships(databaseId),
};
}
async insertLogicalRelationship(
databaseId: string,
sourceColumnId: string,
targetColumnId: string,
generated: boolean,
): Promise<CatalogLogicalRelationship | undefined> {
if ([...this.logicalRelationships.values()].some((relationship) => (
relationship.databaseId === databaseId
&& relationship.columns[0].sourceColumnId === sourceColumnId
&& relationship.columns[0].targetColumnId === targetColumnId
))) return undefined;
const sourceColumn = this.columns.get(sourceColumnId);
const targetColumn = this.columns.get(targetColumnId);
const sourceTable = sourceColumn ? this.tables.get(sourceColumn.tableId) : undefined;
const targetTable = targetColumn ? this.tables.get(targetColumn.tableId) : undefined;
if (!sourceColumn || !targetColumn || !sourceTable || !targetTable
|| sourceTable.databaseId !== databaseId || targetTable.databaseId !== databaseId
|| sourceColumnId === targetColumnId) return undefined;
const now = new Date().toISOString();
const relationship: CatalogLogicalRelationship = {
id: randomUUID(),
databaseId,
constraintName: null,
sourceTableId: sourceTable.id,
sourceTableName: sourceTable.name,
targetTableId: targetTable.id,
targetTableName: targetTable.name,
updateRule: null,
deleteRule: null,
deferrable: false,
initiallyDeferred: false,
columns: [{
position: 1,
sourceColumnId,
sourceColumnName: sourceColumn.name,
targetColumnId,
targetColumnName: targetColumn.name,
}],
lastSyncedDatabaseVersion: null,
lastSyncedAt: null,
createdAt: now,
updatedAt: now,
origin: generated ? "generated" : "manual",
status: "active",
};
this.logicalRelationships.set(relationship.id, relationship);
return structuredClone(relationship);
}
async insertGeneratedLogicalRelationships(
databaseId: string,
candidates: readonly CatalogLogicalRelationshipCandidate[],
): Promise<number> {
let added = 0;
for (const candidate of candidates) {
if (await this.insertLogicalRelationship(
databaseId,
candidate.sourceColumnId,
candidate.targetColumnId,
true,
)) added += 1;
}
return added;
}
async setLogicalRelationshipStatus(
databaseId: string,
relationshipId: string,
status: CatalogLogicalRelationship["status"],
): Promise<CatalogLogicalRelationship | undefined> {
const current = this.logicalRelationships.get(relationshipId);
if (!current || current.databaseId !== databaseId) return undefined;
const updated = { ...current, status, updatedAt: new Date().toISOString() };
this.logicalRelationships.set(relationshipId, updated);
return structuredClone(updated);
}
async deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean> {
const current = this.logicalRelationships.get(relationshipId);
return Boolean(current?.databaseId === databaseId && this.logicalRelationships.delete(relationshipId));
}
async deleteDatabaseMetadata(
databaseIds: readonly string[],
target: CatalogDatabaseMetadataDeleteTarget,
@@ -574,6 +707,12 @@ export class MemoryCatalogRepository implements CatalogRepository {
const columns = [...this.columns.values()].filter((column) => selected.has(column.tableId));
const deletedColumnIds = new Set(columns.map((column) => column.id));
for (const column of columns) this.columns.delete(column.id);
for (const relationship of [...this.logicalRelationships.values()]) {
const pair = relationship.columns[0];
if (deletedColumnIds.has(pair.sourceColumnId) || deletedColumnIds.has(pair.targetColumnId)) {
this.logicalRelationships.delete(relationship.id);
}
}
for (const [relationshipId, relationship] of this.relationships) {
if (relationship.databaseId !== databaseId) continue;
this.relationships.set(relationshipId, {
@@ -857,6 +996,8 @@ export class MemoryCatalogRepository implements CatalogRepository {
lastSyncedAt: now,
createdAt: current?.createdAt ?? now,
updatedAt: comparable === nextComparable ? (current?.updatedAt ?? now) : now,
origin: "physical",
status: "active",
});
if (!current) created += 1;
else if (comparable !== nextComparable) updated += 1;
@@ -1073,6 +1214,12 @@ export class MemoryCatalogRepository implements CatalogRepository {
this.relationships.delete(relationship.id);
}
}
for (const relationship of [...this.logicalRelationships.values()]) {
const pair = relationship.columns[0];
if (pair.sourceColumnId === columnId || pair.targetColumnId === columnId) {
this.logicalRelationships.delete(relationship.id);
}
}
}
private markCatalogIncomplete(databaseIds: readonly string[]): void {
+2
View File
@@ -10,6 +10,7 @@ import * as catalogRuntimeSequencePrivilegesMigration from "./migrations/004_cat
import * as descriptionGenerationRunsMigration from "./migrations/005_description_generation_runs.js";
import * as sensitiveDataFlagMigration from "./migrations/006_sensitive_data_flag.js";
import * as sensitiveDataSuggestionRunsMigration from "./migrations/007_sensitive_data_suggestion_runs.js";
import * as catalogLogicalRelationshipsMigration from "./migrations/008_catalog_logical_relationships.js";
const connectionString = process.env.THT_CATALOG_MIGRATOR_DATABASE_URL;
const host = process.env.THT_CATALOG_DB_HOST;
@@ -42,6 +43,7 @@ const provider: MigrationProvider = {
"005_description_generation_runs": descriptionGenerationRunsMigration,
"006_sensitive_data_flag": sensitiveDataFlagMigration,
"007_sensitive_data_suggestion_runs": sensitiveDataSuggestionRunsMigration,
"008_catalog_logical_relationships": catalogLogicalRelationshipsMigration,
};
},
};
@@ -0,0 +1,35 @@
import { type Kysely, sql } from "kysely";
import type { CatalogDatabase } from "../repository.js";
export async function up(db: Kysely<CatalogDatabase>): Promise<void> {
await db.schema.createTable("catalog_logical_relationships")
.addColumn("id", "uuid", (column) => column.primaryKey())
.addColumn("database_id", "uuid", (column) => column.notNull()
.references("workspace_databases.id").onDelete("cascade"))
.addColumn("source_column_id", "uuid", (column) => column.notNull()
.references("catalog_columns.id").onDelete("cascade"))
.addColumn("target_column_id", "uuid", (column) => column.notNull()
.references("catalog_columns.id").onDelete("cascade"))
.addColumn("generated", "boolean", (column) => column.notNull().defaultTo(false))
.addColumn("deleted_at", "timestamptz")
.addColumn("created_at", "timestamptz", (column) => column.notNull().defaultTo(sql`now()`))
.addColumn("updated_at", "timestamptz", (column) => column.notNull().defaultTo(sql`now()`))
.addUniqueConstraint(
"catalog_logical_relationships_endpoint_key",
["database_id", "source_column_id", "target_column_id"],
)
.addCheckConstraint(
"catalog_logical_relationships_distinct_columns_check",
sql`source_column_id <> target_column_id`,
)
.execute();
await db.schema.createIndex("catalog_logical_relationships_database_deleted_idx")
.on("catalog_logical_relationships")
.columns(["database_id", "deleted_at"])
.execute();
}
export async function down(db: Kysely<CatalogDatabase>): Promise<void> {
await db.schema.dropTable("catalog_logical_relationships").execute();
}
+176 -4
View File
@@ -23,7 +23,10 @@ import {
type CatalogDatabaseMetadataDeleteTarget,
type CatalogMetadataDeleteCounts,
type CatalogMetrics,
type CatalogRelationship,
type CatalogLogicalRelationship,
type CatalogLogicalRelationshipCandidate,
type CatalogLogicalRelationshipContext,
type CatalogPhysicalRelationship,
type CatalogSchemaDiff,
type CatalogSyncCounts,
type CatalogSyncEvent,
@@ -145,6 +148,17 @@ interface CatalogRelationshipColumnTable {
targetColumnId: string;
}
interface CatalogLogicalRelationshipTable {
id: string;
databaseId: string;
sourceColumnId: string;
targetColumnId: string;
generated: Generated<boolean>;
deletedAt: Timestamp | null;
createdAt: Timestamp;
updatedAt: Timestamp;
}
interface DescriptionGenerationRunTable {
id: string;
databaseId: string;
@@ -241,6 +255,7 @@ export interface CatalogDatabase {
catalogColumns: CatalogColumnTable;
catalogRelationships: CatalogRelationshipTable;
catalogRelationshipColumns: CatalogRelationshipColumnTable;
catalogLogicalRelationships: CatalogLogicalRelationshipTable;
descriptionGenerationRuns: DescriptionGenerationRunTable;
descriptionGenerationEvents: DescriptionGenerationEventTable;
sensitiveDataSuggestionRuns: SensitiveDataSuggestionRunTable;
@@ -1022,7 +1037,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
return rows.map(serializeSensitiveDataSuggestionEvent);
}
async listRelationships(databaseId: string): Promise<CatalogRelationship[]> {
async listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]> {
const rows = await this.db.selectFrom("catalogRelationships as relationship")
.innerJoin("catalogTables as sourceTable", "sourceTable.id", "relationship.sourceTableId")
.innerJoin("catalogTables as targetTable", "targetTable.id", "relationship.targetTableId")
@@ -1049,7 +1064,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
.where("pair.relationshipId", "in", rows.map((row) => row.id))
.orderBy("pair.relationshipId").orderBy("pair.position")
.execute();
const byRelationship = new Map<string, CatalogRelationship["columns"]>();
const byRelationship = new Map<string, CatalogPhysicalRelationship["columns"]>();
for (const pair of pairs) {
const items = byRelationship.get(pair.relationshipId) ?? [];
items.push(pair);
@@ -1061,9 +1076,160 @@ export class KyselyCatalogRepository implements CatalogRepository {
lastSyncedAt: row.lastSyncedAt === null ? null : new Date(row.lastSyncedAt).toISOString(),
createdAt: new Date(row.createdAt).toISOString(),
updatedAt: new Date(row.updatedAt).toISOString(),
origin: "physical" as const,
status: "active" as const,
}));
}
async listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]> {
const rows = await this.db.selectFrom("catalogLogicalRelationships as relationship")
.innerJoin("catalogColumns as sourceColumn", "sourceColumn.id", "relationship.sourceColumnId")
.innerJoin("catalogTables as sourceTable", "sourceTable.id", "sourceColumn.tableId")
.innerJoin("catalogColumns as targetColumn", "targetColumn.id", "relationship.targetColumnId")
.innerJoin("catalogTables as targetTable", "targetTable.id", "targetColumn.tableId")
.select([
"relationship.id", "relationship.databaseId", "relationship.generated",
"relationship.deletedAt", "relationship.createdAt", "relationship.updatedAt",
"sourceTable.id as sourceTableId", "sourceTable.name as sourceTableName",
"sourceColumn.id as sourceColumnId", "sourceColumn.name as sourceColumnName",
"targetTable.id as targetTableId", "targetTable.name as targetTableName",
"targetColumn.id as targetColumnId", "targetColumn.name as targetColumnName",
])
.where("relationship.databaseId", "=", databaseId)
.orderBy("sourceTable.name")
.orderBy("sourceColumn.name")
.orderBy("targetTable.name")
.orderBy("targetColumn.name")
.execute();
return rows.map((row) => ({
id: row.id,
databaseId: row.databaseId,
constraintName: null,
sourceTableId: row.sourceTableId,
sourceTableName: row.sourceTableName,
targetTableId: row.targetTableId,
targetTableName: row.targetTableName,
updateRule: null,
deleteRule: null,
deferrable: false,
initiallyDeferred: false,
columns: [{
position: 1,
sourceColumnId: row.sourceColumnId,
sourceColumnName: row.sourceColumnName,
targetColumnId: row.targetColumnId,
targetColumnName: row.targetColumnName,
}],
lastSyncedDatabaseVersion: null,
lastSyncedAt: null,
createdAt: new Date(row.createdAt).toISOString(),
updatedAt: new Date(row.updatedAt).toISOString(),
origin: row.generated ? "generated" : "manual",
status: row.deletedAt === null ? "active" : "excluded",
}));
}
async getLogicalRelationshipContext(
databaseId: string,
): Promise<CatalogLogicalRelationshipContext | undefined> {
if (!(await selectOne(this.db, databaseId))) return undefined;
const columns = await this.db.selectFrom("catalogColumns as column")
.innerJoin("catalogTables as table", "table.id", "column.tableId")
.select([
"column.id as columnId", "column.name as columnName", "column.dataType",
"column.primaryKeyPosition", "table.id as tableId", "table.name as tableName",
])
.where("table.databaseId", "=", databaseId)
.orderBy("table.name")
.orderBy("column.ordinalPosition")
.execute();
const primaryKeyCounts = new Map<string, number>();
for (const column of columns) {
if (column.primaryKeyPosition !== null) {
primaryKeyCounts.set(column.tableId, (primaryKeyCounts.get(column.tableId) ?? 0) + 1);
}
}
const physicalPairs = await this.db.selectFrom("catalogRelationshipColumns as pair")
.innerJoin("catalogRelationships as relationship", "relationship.id", "pair.relationshipId")
.select(["pair.sourceColumnId", "pair.targetColumnId"])
.where("relationship.databaseId", "=", databaseId)
.execute();
return {
endpoints: columns.map((column) => ({
...column,
tablePrimaryKeyColumnCount: primaryKeyCounts.get(column.tableId) ?? 0,
})),
physicalPairs,
logicalRelationships: await this.listLogicalRelationships(databaseId),
};
}
async insertLogicalRelationship(
databaseId: string,
sourceColumnId: string,
targetColumnId: string,
generated: boolean,
): Promise<CatalogLogicalRelationship | undefined> {
const id = randomUUID();
const inserted = await this.db.insertInto("catalogLogicalRelationships").values({
id, databaseId, sourceColumnId, targetColumnId, generated,
}).onConflict((conflict) => conflict
.columns(["databaseId", "sourceColumnId", "targetColumnId"])
.doNothing())
.returning("id")
.executeTakeFirst();
if (!inserted) return undefined;
return (await this.listLogicalRelationships(databaseId)).find((item) => item.id === id);
}
async insertGeneratedLogicalRelationships(
databaseId: string,
candidates: readonly CatalogLogicalRelationshipCandidate[],
): Promise<number> {
if (candidates.length === 0) return 0;
return await this.db.transaction().execute(async (trx) => {
let added = 0;
for (const candidate of candidates) {
const inserted = await trx.insertInto("catalogLogicalRelationships").values({
id: randomUUID(),
databaseId,
sourceColumnId: candidate.sourceColumnId,
targetColumnId: candidate.targetColumnId,
generated: true,
}).onConflict((conflict) => conflict
.columns(["databaseId", "sourceColumnId", "targetColumnId"])
.doNothing())
.returning("id")
.executeTakeFirst();
if (inserted) added += 1;
}
return added;
});
}
async setLogicalRelationshipStatus(
databaseId: string,
relationshipId: string,
status: CatalogLogicalRelationship["status"],
): Promise<CatalogLogicalRelationship | undefined> {
const updated = await this.db.updateTable("catalogLogicalRelationships")
.set({ deletedAt: status === "excluded" ? sql`now()` : null, updatedAt: sql`now()` })
.where("databaseId", "=", databaseId)
.where("id", "=", relationshipId)
.returning("id")
.executeTakeFirst();
if (!updated) return undefined;
return (await this.listLogicalRelationships(databaseId)).find((item) => item.id === relationshipId);
}
async deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean> {
const result = await this.db.deleteFrom("catalogLogicalRelationships")
.where("databaseId", "=", databaseId)
.where("id", "=", relationshipId)
.executeTakeFirst();
return result.numDeletedRows > 0n;
}
async deleteDatabaseMetadata(
databaseIds: readonly string[],
target: CatalogDatabaseMetadataDeleteTarget,
@@ -1621,7 +1787,13 @@ export class UnavailableCatalogRepository implements CatalogRepository {
async updateSensitiveDataSuggestionRun(): Promise<SensitiveDataSuggestionRun | undefined> { return this.fail(); }
async appendSensitiveDataSuggestionEvent(): Promise<SensitiveDataSuggestionEvent> { return this.fail(); }
async listSensitiveDataSuggestionEvents(): Promise<SensitiveDataSuggestionEvent[]> { return this.fail(); }
async listRelationships(): Promise<CatalogRelationship[]> { return this.fail(); }
async listRelationships(): Promise<CatalogPhysicalRelationship[]> { return this.fail(); }
async listLogicalRelationships(): Promise<CatalogLogicalRelationship[]> { return this.fail(); }
async getLogicalRelationshipContext(): Promise<CatalogLogicalRelationshipContext | undefined> { return this.fail(); }
async insertLogicalRelationship(): Promise<CatalogLogicalRelationship | undefined> { return this.fail(); }
async insertGeneratedLogicalRelationships(): Promise<number> { return this.fail(); }
async setLogicalRelationshipStatus(): Promise<CatalogLogicalRelationship | undefined> { return this.fail(); }
async deleteLogicalRelationship(): Promise<boolean> { return this.fail(); }
async deleteDatabaseMetadata(): Promise<CatalogMetadataDeleteCounts | undefined> { return this.fail(); }
async deleteTableMetadata(): Promise<CatalogMetadataDeleteCounts | undefined> { return this.fail(); }
async planSchemaSync(): Promise<CatalogSchemaDiff> { return this.fail(); }
+65 -2
View File
@@ -2,6 +2,7 @@ import { buildInstallationContract } from "../workspaces/contracts.js";
import { resolveBinding } from "../workspaces/bindings.js";
import type { WorkspaceRegistry } from "../workspaces/registry.js";
import type { WorkspaceDescriptor } from "../workspaces/schema.js";
import { discoverWorkspaceSecretRequirements } from "../workspaces/secret-requirements.js";
import type { WorkspaceSecretStore } from "../workspaces/secret-store.js";
import { createConcreteDiagnosticAdapters } from "../workspaces/diagnostics.js";
import { CatalogOperationCoordinator } from "./operation-coordinator.js";
@@ -25,6 +26,21 @@ export interface CatalogListItem extends Omit<WorkspaceDatabase, "id"> {
workspaceName: string;
workspaceDescription?: string;
workspaceAvailable: boolean;
workspaceRevision: { commit: string; blob: string } | null;
workspaceEvidence: {
sourceType: "filesystem" | "http" | "s3" | null;
state:
| "not_declared"
| "materialized_current_revision"
| "configuration_required"
| "configured_unverified"
| "workspace_unavailable";
};
runtimeBinding: {
transport: DatabaseBinding["transport"];
configurationState: "ready" | "configuration_required";
sessionTransportSupported: boolean;
} | null;
configured: boolean;
secrets: Record<CatalogSecretName, boolean>;
}
@@ -68,6 +84,43 @@ function secretState(store: WorkspaceSecretStore, workspaceId: string): Record<C
))) as Record<CatalogSecretName, boolean>;
}
function workspaceRuntimeState(
store: WorkspaceSecretStore,
workspace: WorkspaceDescriptor,
secretRoots: readonly string[],
): NonNullable<CatalogListItem["runtimeBinding"]> {
const requirements = discoverWorkspaceSecretRequirements(workspace, process.env);
const secretVariables = new Set(requirements.map(({ variable }) => variable));
const effective = resolveBinding(workspace, "DWH", process.env, secretRoots);
const configurationRequired = requirements.some(({ id, required }) => (
required && !store.has(workspace.workspace.id, id)
)) || effective.missing.some((variable) => !secretVariables.has(variable));
return {
transport: effective.transport,
configurationState: configurationRequired ? "configuration_required" : "ready",
sessionTransportSupported: effective.transport !== "ssh_tunnel",
};
}
function workspaceEvidenceState(
store: WorkspaceSecretStore,
workspace: WorkspaceDescriptor,
): CatalogListItem["workspaceEvidence"] {
const source = workspace.evidence?.source;
if (!source) return { sourceType: null, state: "not_declared" };
if (source.type === "filesystem") {
return { sourceType: source.type, state: "materialized_current_revision" };
}
const configurationRequired = discoverWorkspaceSecretRequirements(workspace, process.env)
.some(({ connector, id, required }) => (
connector === "evidence" && required && !store.has(workspace.workspace.id, id)
));
return {
sourceType: source.type,
state: configurationRequired ? "configuration_required" : "configured_unverified",
};
}
export class CatalogService {
private readonly adapters = createConcreteDiagnosticAdapters();
@@ -91,7 +144,8 @@ export class CatalogService {
const byWorkspace = new Map(configured.map((database) => [database.workspaceId, database]));
const active = await Promise.all(workspaces.map(async (entry) => {
const database = byWorkspace.get(entry.id);
const { workspace } = await this.registry.read(entry.id);
const { workspace } = await this.registry.readPinned(entry.id, entry.revision.commit);
const runtimeDatabaseBinding = yamlBinding(workspace, this.secretRoots);
const base = database ?? {
workspaceId: entry.id,
engine: "postgres" as const,
@@ -100,7 +154,7 @@ export class CatalogService {
version: 0,
createdAt: "",
updatedAt: "",
binding: yamlBinding(workspace, this.secretRoots),
binding: runtimeDatabaseBinding,
connectionStatus: "untested" as const,
};
return {
@@ -108,6 +162,12 @@ export class CatalogService {
workspaceName: entry.name,
workspaceDescription: entry.description,
workspaceAvailable: true,
workspaceRevision: {
commit: entry.revision.commit,
blob: entry.revision.blob,
},
workspaceEvidence: workspaceEvidenceState(this.secretStore, workspace),
runtimeBinding: workspaceRuntimeState(this.secretStore, workspace, this.secretRoots),
configured: database !== undefined,
secrets: secretState(this.secretStore, entry.id),
};
@@ -120,6 +180,9 @@ export class CatalogService {
workspaceName: database.workspaceId,
workspaceDescription: "Workspace is no longer present in the repository catalog.",
workspaceAvailable: false,
workspaceRevision: null,
workspaceEvidence: { sourceType: null, state: "workspace_unavailable" },
runtimeBinding: null,
configured: true,
secrets: secretState(this.secretStore, database.workspaceId),
}));
+66 -2
View File
@@ -128,7 +128,7 @@ export interface CatalogRelationshipColumn {
targetColumnName: string;
}
export interface CatalogRelationship {
export interface CatalogPhysicalRelationship {
id: string;
databaseId: string;
constraintName: string;
@@ -145,6 +145,52 @@ export interface CatalogRelationship {
lastSyncedAt: string | null;
createdAt: string;
updatedAt: string;
origin: "physical";
status: "active";
}
export interface CatalogLogicalRelationship {
id: string;
databaseId: string;
constraintName: null;
sourceTableId: string;
sourceTableName: string;
targetTableId: string;
targetTableName: string;
updateRule: null;
deleteRule: null;
deferrable: false;
initiallyDeferred: false;
columns: [CatalogRelationshipColumn];
lastSyncedDatabaseVersion: null;
lastSyncedAt: null;
createdAt: string;
updatedAt: string;
origin: "generated" | "manual";
status: "active" | "excluded";
}
export type CatalogRelationship = CatalogPhysicalRelationship | CatalogLogicalRelationship;
export interface CatalogLogicalRelationshipEndpoint {
columnId: string;
columnName: string;
tableId: string;
tableName: string;
dataType: string;
primaryKeyPosition: number | null;
tablePrimaryKeyColumnCount: number;
}
export interface CatalogLogicalRelationshipContext {
endpoints: CatalogLogicalRelationshipEndpoint[];
physicalPairs: Array<{ sourceColumnId: string; targetColumnId: string }>;
logicalRelationships: CatalogLogicalRelationship[];
}
export interface CatalogLogicalRelationshipCandidate {
sourceColumnId: string;
targetColumnId: string;
}
export type CatalogDatabaseMetadataDeleteTarget = "tables" | "relationships";
@@ -456,7 +502,25 @@ export interface CatalogRepository {
runId: string,
afterSequence?: number,
): Promise<SensitiveDataSuggestionEvent[]>;
listRelationships(databaseId: string): Promise<CatalogRelationship[]>;
listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]>;
listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]>;
getLogicalRelationshipContext(databaseId: string): Promise<CatalogLogicalRelationshipContext | undefined>;
insertLogicalRelationship(
databaseId: string,
sourceColumnId: string,
targetColumnId: string,
generated: boolean,
): Promise<CatalogLogicalRelationship | undefined>;
insertGeneratedLogicalRelationships(
databaseId: string,
candidates: readonly CatalogLogicalRelationshipCandidate[],
): Promise<number>;
setLogicalRelationshipStatus(
databaseId: string,
relationshipId: string,
status: CatalogLogicalRelationship["status"],
): Promise<CatalogLogicalRelationship | undefined>;
deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean>;
deleteDatabaseMetadata(
databaseIds: readonly string[],
target: CatalogDatabaseMetadataDeleteTarget,
@@ -0,0 +1,154 @@
import type { FastifyInstance, FastifyReply, FastifyRequest } from "fastify";
import { z } from "zod";
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
import {
CatalogLogicalRelationshipService,
LogicalRelationshipColumnNotFoundError,
LogicalRelationshipDatabaseNotFoundError,
LogicalRelationshipDuplicateError,
LogicalRelationshipNotFoundError,
LogicalRelationshipReadOnlyError,
LogicalRelationshipSchemaStaleError,
LogicalRelationshipTargetNotUniqueError,
LogicalRelationshipTypeIncompatibleError,
} from "../catalog/logical-relationship-service.js";
import type { CatalogOperationCoordinator } from "../catalog/operation-coordinator.js";
import { CatalogOperationInProgressError, CatalogUnavailableError } from "../catalog/types.js";
const idSchema = z.uuid();
const createSchema = z.object({
sourceColumnId: idSchema,
targetColumnId: idSchema,
}).strict();
const statusSchema = z.object({ status: z.enum(["active", "excluded"]) }).strict();
const emptySchema = z.object({}).strict();
function manage(request: FastifyRequest, reply: FastifyReply) {
return isPrincipalContext(requirePermission(request, reply, "database.manage"));
}
function safeError(reply: FastifyReply, error: unknown) {
if (error instanceof CatalogUnavailableError) {
return reply.code(503).send({ code: "catalog_unavailable", message: "Database catalog is unavailable." });
}
if (error instanceof CatalogOperationInProgressError) {
return reply.code(409).send({
code: "database_operation_in_progress",
message: "A database operation is already in progress.",
});
}
if (error instanceof LogicalRelationshipDatabaseNotFoundError) {
return reply.code(404).send({ code: "database_not_found", message: "Database configuration was not found." });
}
if (error instanceof LogicalRelationshipColumnNotFoundError) {
return reply.code(404).send({
code: "column_not_found",
message: "Catalog column was not found.",
field: error.field,
});
}
if (error instanceof LogicalRelationshipNotFoundError) {
return reply.code(404).send({ code: "relationship_not_found", message: "Relationship was not found." });
}
if (error instanceof LogicalRelationshipDuplicateError) {
return reply.code(409).send({ code: "relationship_duplicate", message: "Relationship already exists." });
}
if (error instanceof LogicalRelationshipReadOnlyError) {
return reply.code(409).send({ code: "relationship_read_only", message: "Physical relationships are read-only." });
}
if (error instanceof LogicalRelationshipSchemaStaleError) {
return reply.code(409).send({
code: "relationship_schema_stale",
message: "Synchronize the current database schema before managing logical relationships.",
});
}
if (error instanceof LogicalRelationshipTargetNotUniqueError) {
return reply.code(422).send({
code: "relationship_target_not_unique",
message: "Target column must be the only primary-key column of its table.",
field: "targetColumnId",
});
}
if (error instanceof LogicalRelationshipTypeIncompatibleError) {
return reply.code(422).send({
code: "relationship_type_incompatible",
message: "Source and target column types are incompatible.",
field: "targetColumnId",
});
}
if (error instanceof z.ZodError) {
return reply.code(400).send({
code: "relationship_request_invalid",
message: "Relationship request is invalid.",
});
}
return reply.code(500).send({
code: "relationship_operation_failed",
message: "Relationship operation failed.",
});
}
export function catalogLogicalRelationshipRoutes(
app: FastifyInstance,
deps: {
service: CatalogLogicalRelationshipService;
operations: CatalogOperationCoordinator;
},
): void {
app.get("/catalog/databases/:databaseId/relationships", async (request, reply) => {
if (!manage(request, reply)) return reply;
try {
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
return await deps.service.list(databaseId);
} catch (error) { return safeError(reply, error); }
});
app.post("/catalog/databases/:databaseId/relationships", async (request, reply) => {
if (!manage(request, reply)) return reply;
try {
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
const input = createSchema.parse(request.body);
const relationship = await deps.operations.run(databaseId, async () => (
await deps.service.addManual(databaseId, input.sourceColumnId, input.targetColumnId)
));
return reply.code(201).send(relationship);
} catch (error) { return safeError(reply, error); }
});
app.post("/catalog/databases/:databaseId/relationships/rebuild-generated", async (request, reply) => {
if (!manage(request, reply)) return reply;
try {
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
emptySchema.parse(request.body ?? {});
return await deps.operations.run(databaseId, async () => (
await deps.service.rebuildGenerated(databaseId)
));
} catch (error) { return safeError(reply, error); }
});
app.patch("/catalog/databases/:databaseId/relationships/:relationshipId", async (request, reply) => {
if (!manage(request, reply)) return reply;
try {
const params = request.params as { databaseId?: unknown; relationshipId?: unknown };
const databaseId = idSchema.parse(params.databaseId);
const relationshipId = idSchema.parse(params.relationshipId);
const input = statusSchema.parse(request.body);
return await deps.operations.run(databaseId, async () => (
await deps.service.setStatus(databaseId, relationshipId, input.status)
));
} catch (error) { return safeError(reply, error); }
});
app.delete("/catalog/databases/:databaseId/relationships/:relationshipId", async (request, reply) => {
if (!manage(request, reply)) return reply;
try {
const params = request.params as { databaseId?: unknown; relationshipId?: unknown };
const databaseId = idSchema.parse(params.databaseId);
const relationshipId = idSchema.parse(params.relationshipId);
await deps.operations.run(databaseId, async () => {
await deps.service.deletePermanently(databaseId, relationshipId);
});
return reply.code(204).send();
} catch (error) { return safeError(reply, error); }
});
}
-11
View File
@@ -131,17 +131,6 @@ export function catalogSchemaRoutes(
} catch (error) { return safeError(reply, error); }
});
app.get("/catalog/databases/:databaseId/relationships", async (request, reply) => {
if (!manage(request, reply)) return reply;
try {
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
if (!(await deps.repository.get(databaseId))) {
return reply.code(404).send({ code: "database_not_found", message: "Database configuration was not found." });
}
return await deps.repository.listRelationships(databaseId);
} catch (error) { return safeError(reply, error); }
});
app.post("/catalog/databases/metadata-cleanup", async (request, reply) => {
if (!manage(request, reply)) return reply;
try {
+35 -8
View File
@@ -11,6 +11,7 @@ import type { WorkspaceRegistry } from "../workspaces/registry.js";
import { validateOperationalWorkspace, type WorkspaceDescriptor } from "../workspaces/schema.js";
import type { MaintenanceBarrier } from "../runtime/maintenance-gate.js";
import { hasPermission, isPrincipalContext, requirePermission } from "../auth/authorization.js";
import type { EffectiveRelationshipSnapshotProvider } from "../catalog/effective-relationship-snapshot.js";
const BOOTSTRAP_FAILURE_MESSAGE =
"Session startup failed. Check configuration and connectivity, then Resume the session.";
@@ -40,6 +41,8 @@ export function sessionRoutes(
/** Fail-closed installation/runtime transport capability check. */
workspaceRuntimeSupport: (workspace: WorkspaceDescriptor) => boolean;
maintenanceBarrier: MaintenanceBarrier;
/** Optional only for narrow route-test stubs and installations without a Catalog database. */
effectiveRelationships?: EffectiveRelationshipSnapshotProvider;
},
) {
const lifecycleTails = new Map<string, Promise<void>>();
@@ -84,11 +87,21 @@ export function sessionRoutes(
isAdmin: hasPermission(principal, permission),
});
const optionsWithRuntimeConfig = (runner: any, workspaceConfigPath: string | undefined, options: any) => (
workspaceConfigPath && typeof runner.acquireWorkspaceRuntime === "function"
? { ...options, runtimeConfig: runner.acquireWorkspaceRuntime(workspaceConfigPath) }
: options
);
const optionsWithRuntimeConfig = async (
runner: any,
workspaceConfigPath: string | undefined,
workspaceId: string | undefined,
options: any,
) => {
if (!workspaceConfigPath || typeof runner.acquireWorkspaceRuntime !== "function") return options;
const effectiveRelationships = workspaceId && d.effectiveRelationships
? await d.effectiveRelationships.render(workspaceId)
: undefined;
return {
...options,
runtimeConfig: runner.acquireWorkspaceRuntime(workspaceConfigPath, effectiveRelationships),
};
};
const maintenanceReply = (reply: any) => reply.code(503).send({
code: "maintenance",
@@ -448,7 +461,12 @@ export function sessionRoutes(
let runtimeOptions = options;
let rt: ReturnType<PiProcessManager["createFor"]> | undefined;
try {
runtimeOptions = optionsWithRuntimeConfig(runner, workspaceConfigPath, options);
runtimeOptions = await optionsWithRuntimeConfig(
runner,
workspaceConfigPath,
workspaceId,
options,
);
rt = d.mgr.createFor(id, runtimeOptions);
bindRuntime(id, rt, runner, workspaceConfigPath);
} catch (error) {
@@ -465,7 +483,11 @@ export function sessionRoutes(
info(id, "Session created");
bootstrap(
id, rt, runner, workspaceConfigPath, d.mgr.configure(rt, runtimeOptions),
runner.searchPack(b.question, id, workspaceConfigPath),
runner.searchPack(
b.question,
id,
(runtimeOptions as any).runtimeConfig?.path ?? workspaceConfigPath,
),
() => d.mgr.start(id, rt, runtimeOptions),
);
return { id };
@@ -639,7 +661,12 @@ export function sessionRoutes(
if (boundRuntimes.get(id) === current) boundRuntimes.delete(id);
d.mgr.teardownIfCurrent(id, current);
}
runtimeOptions = optionsWithRuntimeConfig(runner, workspaceConfigPath, options);
runtimeOptions = await optionsWithRuntimeConfig(
runner,
workspaceConfigPath,
saved.workspace_id,
options,
);
rt = d.mgr.createFor(id, runtimeOptions);
bindRuntime(id, rt, runner, workspaceConfigPath);
} catch {
+27 -12
View File
@@ -213,23 +213,37 @@ export class ThtRunner {
}
/** Render one immutable canonical registry revision into a backend-owned harness config. */
acquireWorkspaceRuntime(workspaceConfigPath: string): RuntimeConfigLease {
const rendered = renderWorkspaceRuntimeFromSnapshotPath({
snapshotPath: workspaceConfigPath,
harnessDir: this.cfg.harnessDir,
configPath: this.cfg.configPath,
dataRoot: this.cfg.dataRoot ?? (() => {
throw new Error("registry workspace runtime requires an absolute data root");
})(),
secretRoots: this.cfg.secretRoots ?? [],
semanticRuntime: this.cfg.semanticRuntime ?? DEFAULT_SEMANTIC_RUNTIME,
workspaceSecretStore: this.cfg.workspaceSecretStore,
});
acquireWorkspaceRuntime(
workspaceConfigPath: string,
effectiveRelationships?: string,
): RuntimeConfigLease {
const effectiveRelationshipsPath = effectiveRelationships === undefined
? undefined
: this.createRuntimeSnapshot(effectiveRelationships);
let rendered: ReturnType<typeof renderWorkspaceRuntimeFromSnapshotPath>;
try {
rendered = renderWorkspaceRuntimeFromSnapshotPath({
snapshotPath: workspaceConfigPath,
harnessDir: this.cfg.harnessDir,
configPath: this.cfg.configPath,
dataRoot: this.cfg.dataRoot ?? (() => {
throw new Error("registry workspace runtime requires an absolute data root");
})(),
secretRoots: this.cfg.secretRoots ?? [],
semanticRuntime: this.cfg.semanticRuntime ?? DEFAULT_SEMANTIC_RUNTIME,
workspaceSecretStore: this.cfg.workspaceSecretStore,
effectiveRelationshipsPath,
});
} catch (error) {
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
throw error;
}
let path: string;
try {
path = this.createRuntimeSnapshot(rendered.renderedConfig);
} catch (error) {
rendered.releaseSecrets();
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
throw error;
}
let released = false;
@@ -241,6 +255,7 @@ export class ThtRunner {
if (released) return;
released = true;
this.cleanupRuntimeSnapshot(path);
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
rendered.releaseSecrets();
},
};
+24 -3
View File
@@ -243,7 +243,12 @@ function readSnapshotWorkspace(snapshotPath: string): {
}
}
function runtimePaths(dataRoot: string, workspaceId: string, workspaceRevision?: string): RuntimePaths {
function runtimePaths(
dataRoot: string,
workspaceId: string,
workspaceRevision?: string,
effectiveRelationshipsPath?: string,
): RuntimePaths {
if (!isAbsolute(dataRoot)) throw new Error("registry workspace runtime requires an absolute data root");
const root = join(dataRoot, "sessions", workspaceId);
return {
@@ -254,6 +259,9 @@ function runtimePaths(dataRoot: string, workspaceId: string, workspaceRevision?:
...(workspaceRevision === undefined
? {}
: { annotations_root: join(dataRoot, "sessions", workspaceId, "revisions", workspaceRevision, "artifacts") }),
...(effectiveRelationshipsPath === undefined
? {}
: { effective_relationships: effectiveRelationshipsPath }),
};
}
@@ -301,6 +309,7 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
dataRoot: string;
secretRoots: readonly string[];
semanticRuntime: SemanticRuntimeConfig;
effectiveRelationshipsPath?: string;
workspaceSecretStore?: WorkspaceSecretStore;
}): RenderedWorkspaceRuntime {
const secretLease = options.workspaceSecretStore === undefined
@@ -325,7 +334,12 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
workspaceId: options.workspaceId,
workspaceRevision: options.workspaceRevision,
revisionContentRoot: options.revisionContentRoot,
runtimePaths: runtimePaths(options.dataRoot, options.workspaceId, options.workspaceRevision),
runtimePaths: runtimePaths(
options.dataRoot,
options.workspaceId,
options.workspaceRevision,
options.effectiveRelationshipsPath,
),
installationOverlay: overlay,
bindings,
bindingDigest: stableBindingDigest(bindings),
@@ -334,7 +348,12 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
renderedConfig: renderRuntimeConfig(
options.workspace,
bindings,
runtimePaths(options.dataRoot, options.workspaceId, options.workspaceRevision),
runtimePaths(
options.dataRoot,
options.workspaceId,
options.workspaceRevision,
options.effectiveRelationshipsPath,
),
context,
overlay,
options.semanticRuntime,
@@ -353,6 +372,7 @@ export function renderWorkspaceRuntimeFromSnapshotPath(options: {
dataRoot: string;
secretRoots: readonly string[];
semanticRuntime: SemanticRuntimeConfig;
effectiveRelationshipsPath?: string;
workspaceSecretStore?: WorkspaceSecretStore;
}): RenderedWorkspaceRuntime {
const snapshot = readSnapshotWorkspace(options.snapshotPath);
@@ -366,6 +386,7 @@ export function renderWorkspaceRuntimeFromSnapshotPath(options: {
dataRoot: options.dataRoot,
secretRoots: options.secretRoots,
semanticRuntime: options.semanticRuntime,
effectiveRelationshipsPath: options.effectiveRelationshipsPath,
workspaceSecretStore: options.workspaceSecretStore,
});
}
@@ -12,6 +12,8 @@ export interface RuntimePaths {
memory: string;
/** Revision-qualified root for curated FK annotations (P5); optional for legacy callers. */
annotations_root?: string;
/** Immutable Catalog projection used as the exclusive runtime relationship source. */
effective_relationships?: string;
}
export interface RuntimeIdentity {
+63 -4
View File
@@ -13,7 +13,10 @@ import type { WorkspaceRegistry, WorkspaceRevision } from "../src/workspaces/reg
import type { WorkspaceDescriptor } from "../src/workspaces/schema.js";
const roots: string[] = [];
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
afterEach(() => {
vi.unstubAllEnvs();
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
});
const workspace: WorkspaceDescriptor = {
workspace: { schema_version: 3, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
@@ -36,6 +39,7 @@ function setup(
catalogOperationCoordinator?: CatalogOperationCoordinator;
catalogPostgresAccess?: CatalogPostgresAccess;
} = {},
workspaceDescriptor: WorkspaceDescriptor = workspace,
) {
const secretRoot = mkdtempSync(join(tmpdir(), "catalog-secret-"));
const runtimeRoot = mkdtempSync(join(tmpdir(), "catalog-secret-runtime-"));
@@ -45,7 +49,8 @@ function setup(
const registry = {
list: vi.fn(async () => [revision]),
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
read: vi.fn(async () => ({ workspace, revision })),
read: vi.fn(async () => ({ workspace: workspaceDescriptor, revision })),
readPinned: vi.fn(async () => ({ workspace: workspaceDescriptor, workspaceConfigPath: revision.snapshotPath })),
} as unknown as WorkspaceRegistry;
const app = buildApp(loadConfig({
THT_HARNESS_DIR: "/missing",
@@ -96,10 +101,31 @@ const fleetSnapshot: ObservedSchemaSnapshot = {
};
test("lists every YAML workspace and creates its one database configuration", async () => {
const { app } = setup();
const { app, secretStore } = setup();
const initial = await app.inject({ method: "GET", url: "/catalog/databases" });
expect(initial.statusCode).toBe(200);
expect(initial.json()).toMatchObject([{ workspaceId: "psd-clinical", configured: false, databaseName: "warehouse" }]);
expect(initial.json()).toMatchObject([{
workspaceId: "psd-clinical",
configured: false,
databaseName: "warehouse",
workspaceRevision: { commit: revision.commit, blob: revision.blob },
workspaceEvidence: { sourceType: null, state: "not_declared" },
runtimeBinding: {
transport: "postgres_direct",
configurationState: "configuration_required",
sessionTransportSupported: true,
},
}]);
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_HOST", "runtime-db.internal");
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_PORT", "5432");
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_USER", "runtime-reader");
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_TRANSPORT", "postgres_direct");
secretStore.putMany("psd-clinical", { "dwh.password": "runtime-password" });
const runtimeReady = await app.inject({ method: "GET", url: "/catalog/databases" });
expect(runtimeReady.json()).toMatchObject([{
runtimeBinding: { configurationState: "ready", sessionTransportSupported: true },
}]);
const created = await app.inject({ method: "POST", url: "/catalog/databases", payload: direct });
expect(created.statusCode).toBe(201);
@@ -110,6 +136,39 @@ test("lists every YAML workspace and creates its one database configuration", as
expect(listed.json()).toMatchObject([{ configured: true, binding: { transport: "postgres_direct", host: "db.internal" } }]);
});
test("projects remote Evidence credential state without conflating catalog secrets", async () => {
const evidenceWorkspace: WorkspaceDescriptor = {
...workspace,
evidence: {
schema_version: 2,
source: {
type: "http",
uris: ["https://evidence.example.test/guide.md"],
authentication: "signed_urls_file",
connect_timeout_ms: 5_000,
read_timeout_ms: 30_000,
max_bytes: 10 * 1024 * 1024,
max_redirects: 5,
allow_private_hosts: false,
max_cache_bytes: 64 * 1024 * 1024,
},
policy: { max_chunk_chars: 4_000, retain_published_generations: 3 },
},
};
const { app, secretStore } = setup({}, {}, evidenceWorkspace);
const missing = await app.inject({ method: "GET", url: "/catalog/databases" });
expect(missing.json()).toMatchObject([{
workspaceEvidence: { sourceType: "http", state: "configuration_required" },
}]);
secretStore.putMany("psd-clinical", { "evidence.signed_urls": "https://signed.example.test/evidence" });
const configured = await app.inject({ method: "GET", url: "/catalog/databases" });
expect(configured.json()).toMatchObject([{
workspaceEvidence: { sourceType: "http", state: "configured_unverified" },
}]);
});
test("lists orphaned records and takes the REST diagnostic path from workspace YAML", async () => {
const { app, repository } = setup();
await repository.create({
@@ -0,0 +1,207 @@
import { expect, test } from "vitest";
import {
CatalogLogicalRelationshipService,
LogicalRelationshipDuplicateError,
LogicalRelationshipSchemaStaleError,
LogicalRelationshipTargetNotUniqueError,
LogicalRelationshipTypeIncompatibleError,
} from "../src/catalog/logical-relationship-service.js";
import { MemoryCatalogRepository } from "../src/catalog/memory-repository.js";
import type { ObservedSchemaSnapshot } from "../src/catalog/types.js";
const column = (
tableName: string,
name: string,
ordinalPosition: number,
dataType: string,
primaryKeyPosition: number | null,
) => ({
tableName, name, ordinalPosition, dataType, primaryKeyPosition,
isNullable: false, defaultExpression: null, sourceComment: null,
});
function schema(
tableNames: string[],
columns: ObservedSchemaSnapshot["columns"],
relationships: ObservedSchemaSnapshot["relationships"] = [],
): ObservedSchemaSnapshot {
return {
schemaVersion: 1,
capabilities: { tables: "available", columns: "available", relationships: "available" },
tables: tableNames.map((name) => ({ name, sourceComment: null })),
columns,
relationships,
};
}
async function setup(snapshot: ObservedSchemaSnapshot) {
const repository = new MemoryCatalogRepository();
const database = await repository.create({
workspaceId: "relationships", engine: "postgres", databaseName: "warehouse", schema: "public",
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
});
await repository.applySchemaSync(database.id, database.version, "all", [], snapshot);
const service = new CatalogLogicalRelationshipService(repository);
const context = (await repository.getLogicalRelationshipContext(database.id))!;
const endpoint = (tableName: string, columnName: string) => context.endpoints.find((item) => (
item.tableName === tableName && item.columnName === columnName
))!;
return { repository, database, service, endpoint };
}
test("keeps excluded relationships across rebuild and recreates hard-deleted relationships", async () => {
const { database, service, endpoint } = await setup(schema(
["users", "orders"],
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
column("orders", "user_id", 2, "bigint", null)],
));
const source = endpoint("orders", "user_id");
const target = endpoint("users", "id");
const manual = await service.addManual(database.id, source.columnId, target.columnId);
expect(manual).toMatchObject({ origin: "manual", status: "active" });
expect(await service.setStatus(database.id, manual.id, "excluded"))
.toMatchObject({ status: "excluded" });
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
added: 0, alreadyPresent: 0, excluded: 1, ambiguous: 0,
});
expect((await service.list(database.id)).filter((item) => item.origin !== "physical"))
.toMatchObject([{ id: manual.id, origin: "manual", status: "excluded" }]);
await service.deletePermanently(database.id, manual.id);
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0,
});
expect((await service.list(database.id)).filter((item) => item.origin !== "physical"))
.toMatchObject([{ origin: "generated", status: "active" }]);
});
test("normalizes snake, kebab, and camel names without fuzzy matching", async () => {
const { database, service } = await setup(schema(
["users", "events"],
[column("users", "id", 1, "bigint", 1), column("events", "id", 1, "bigint", 1),
column("events", "user_id", 2, "bigint", null),
column("events", "user-id", 3, "bigint", null),
column("events", "userId", 4, "bigint", null),
column("events", "userid", 5, "bigint", null),
column("events", "unrelated", 6, "bigint", null)],
));
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
added: 4, alreadyPresent: 0, excluded: 0, ambiguous: 0,
});
});
test("skips generic bare primary-key names while retaining table-qualified matches", async () => {
const { database, service } = await setup(schema(
["users", "accounts", "events"],
[column("users", "id", 1, "bigint", 1), column("accounts", "id", 1, "bigint", 1),
column("events", "event_key", 1, "bigint", 1), column("events", "id", 2, "bigint", null),
column("events", "user_id", 3, "bigint", null)],
));
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0,
});
});
test("infers foreign keys from composite-primary-key sources", async () => {
const { database, service } = await setup(schema(
["users", "groups", "memberships"],
[column("users", "id", 1, "bigint", 1), column("groups", "id", 1, "bigint", 1),
column("memberships", "user_id", 1, "bigint", 1),
column("memberships", "group_id", 2, "bigint", 2)],
));
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
added: 2, alreadyPresent: 0, excluded: 0, ambiguous: 0,
});
});
test("maps time-key columns to the single primary key of dim_time", async () => {
const { database, service } = await setup(schema(
["dim_time", "admissions"],
[column("dim_time", "day_key", 1, "integer", 1),
column("admissions", "id", 1, "bigint", 1),
column("admissions", "admission_time_key", 2, "integer", null),
column("admissions", "discharge_time_key", 3, "integer", null)],
));
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
added: 2, alreadyPresent: 0, excluded: 0, ambiguous: 0,
});
});
test("skips ambiguous targets and physical foreign-key pairs", async () => {
const ambiguous = await setup(schema(
["user", "users", "events"],
[column("user", "id", 1, "bigint", 1), column("users", "id", 1, "bigint", 1),
column("events", "id", 1, "bigint", 1), column("events", "user_id", 2, "bigint", null)],
));
await expect(ambiguous.service.rebuildGenerated(ambiguous.database.id)).resolves.toEqual({
added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 1,
});
const physical = await setup(schema(
["users", "orders"],
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
column("orders", "user_id", 2, "bigint", null)],
[{
constraintName: "orders_user_id_fkey", sourceTableName: "orders", targetTableName: "users",
updateRule: "NO ACTION", deleteRule: "NO ACTION", deferrable: false, initiallyDeferred: false,
columns: [{ position: 1, sourceColumnName: "user_id", targetColumnName: "id" }],
}],
));
await expect(physical.service.rebuildGenerated(physical.database.id)).resolves.toEqual({
added: 0, alreadyPresent: 1, excluded: 0, ambiguous: 0,
});
await expect(physical.service.addManual(
physical.database.id,
physical.endpoint("orders", "user_id").columnId,
physical.endpoint("users", "id").columnId,
)).rejects.toBeInstanceOf(LogicalRelationshipDuplicateError);
});
test("validates target uniqueness and canonical type compatibility for manual relationships", async () => {
const { database, service, endpoint } = await setup(schema(
["users", "composite", "events"],
[column("users", "id", 1, "bigint", 1),
column("composite", "left_id", 1, "bigint", 1), column("composite", "right_id", 2, "bigint", 2),
column("events", "id", 1, "bigint", 1), column("events", "user_id", 2, "integer", null),
column("events", "composite_id", 3, "bigint", null)],
));
await expect(service.addManual(
database.id, endpoint("events", "composite_id").columnId, endpoint("composite", "left_id").columnId,
)).rejects.toBeInstanceOf(LogicalRelationshipTargetNotUniqueError);
await expect(service.addManual(
database.id, endpoint("events", "user_id").columnId, endpoint("users", "id").columnId,
)).rejects.toBeInstanceOf(LogicalRelationshipTypeIncompatibleError);
});
test("rebuild is additive when an existing generated relationship stops matching", async () => {
const initial = schema(
["users", "orders"],
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
column("orders", "user_id", 2, "bigint", null)],
);
const { repository, database, service } = await setup(initial);
await service.rebuildGenerated(database.id);
const changed = structuredClone(initial);
changed.columns.find((item) => item.tableName === "orders" && item.name === "user_id")!.dataType = "text";
await repository.applySchemaSync(database.id, database.version, "columns", [], changed);
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 0,
});
expect((await service.list(database.id)).filter((item) => item.origin === "generated")).toHaveLength(1);
});
test("refuses inference until the current database version has a full schema sync", async () => {
const repository = new MemoryCatalogRepository();
const database = await repository.create({
workspaceId: "unsynced-relationships",
engine: "postgres",
databaseName: "warehouse",
schema: "public",
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
});
const service = new CatalogLogicalRelationshipService(repository);
await expect(service.rebuildGenerated(database.id))
.rejects.toBeInstanceOf(LogicalRelationshipSchemaStaleError);
});
@@ -12,6 +12,7 @@ import { up as upRuntimeSequencePrivileges } from "../src/catalog/migrations/004
import { up as upDescriptionGeneration } from "../src/catalog/migrations/005_description_generation_runs.js";
import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js";
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
import { up as upLogicalRelationships } from "../src/catalog/migrations/008_catalog_logical_relationships.js";
const dockerAvailable = spawnSync("docker", ["info"], { stdio: "ignore" }).status === 0;
@@ -26,6 +27,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
await upTables(db);
await upSchemaSync(db);
await upSensitiveDataFlag(db);
await upLogicalRelationships(db);
await sql`CREATE ROLE thothii_catalog_runtime`.execute(db);
await upRuntimeSequencePrivileges(db);
const sequencePrivilege = await sql<{ allowed: boolean }>`
@@ -202,6 +204,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository performs scoped metadata cl
await upTables(db);
await upSchemaSync(db);
await upSensitiveDataFlag(db);
await upLogicalRelationships(db);
const repository = new KyselyCatalogRepository(db);
const database = await repository.create({
workspaceId: "cleanup-test",
@@ -280,6 +283,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository atomically consolidates sel
await upTables(db);
await upSchemaSync(db);
await upSensitiveDataFlag(db);
await upLogicalRelationships(db);
const repository = new KyselyCatalogRepository(db);
const database = await repository.create({
workspaceId: "consolidation-test",
@@ -379,6 +383,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
await upTables(db);
await upSchemaSync(db);
await upSensitiveDataFlag(db);
await upLogicalRelationships(db);
await upDescriptionGeneration(db);
await upSensitiveSuggestionRuns(db);
const repository = new KyselyCatalogRepository(db);
@@ -610,3 +615,63 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
await container.stop();
}
}, 60_000);
test.skipIf(!dockerAvailable)("PostgreSQL repository persists logical relationship lifecycle and tombstones", async () => {
const container = await new PostgreSqlContainer("postgres:17.6-bookworm").start();
const db = new Kysely<CatalogDatabase>({
dialect: new PostgresDialect({ pool: new Pool({ connectionString: container.getConnectionUri() }) }),
plugins: [new CamelCasePlugin()],
});
try {
await upDatabases(db);
await upTables(db);
await upSchemaSync(db);
await upSensitiveDataFlag(db);
await upLogicalRelationships(db);
const repository = new KyselyCatalogRepository(db);
const database = await repository.create({
workspaceId: "logical-relationships",
engine: "postgres",
databaseName: "warehouse",
schema: "public",
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
});
await repository.applySchemaSync(database.id, database.version, "all", [], {
schemaVersion: 1,
capabilities: { tables: "available", columns: "available", relationships: "available" },
tables: [{ name: "users", sourceComment: null }, { name: "orders", sourceComment: null }],
columns: [
{ tableName: "users", name: "id", ordinalPosition: 1, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: 1, sourceComment: null },
{ tableName: "orders", name: "id", ordinalPosition: 1, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: 1, sourceComment: null },
{ tableName: "orders", name: "user_id", ordinalPosition: 2, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: null, sourceComment: null },
],
relationships: [],
});
const context = (await repository.getLogicalRelationshipContext(database.id))!;
const source = context.endpoints.find((item) => item.tableName === "orders" && item.columnName === "user_id")!;
const target = context.endpoints.find((item) => item.tableName === "users" && item.columnName === "id")!;
const created = await repository.insertLogicalRelationship(database.id, source.columnId, target.columnId, false);
expect(created).toMatchObject({ origin: "manual", status: "active" });
await expect(repository.insertLogicalRelationship(database.id, source.columnId, target.columnId, true))
.resolves.toBeUndefined();
expect(await repository.setLogicalRelationshipStatus(database.id, created!.id, "excluded"))
.toMatchObject({ status: "excluded" });
await expect(repository.insertGeneratedLogicalRelationships(database.id, [{
sourceColumnId: source.columnId, targetColumnId: target.columnId,
}])).resolves.toBe(0);
await expect(repository.deleteLogicalRelationship(database.id, created!.id)).resolves.toBe(true);
await expect(repository.insertGeneratedLogicalRelationships(database.id, [{
sourceColumnId: source.columnId, targetColumnId: target.columnId,
}])).resolves.toBe(1);
expect(await repository.listLogicalRelationships(database.id))
.toMatchObject([{ origin: "generated", status: "active" }]);
await db.deleteFrom("catalogColumns").where("id", "=", source.columnId).execute();
expect(await repository.listLogicalRelationships(database.id)).toEqual([]);
} finally {
await db.destroy();
await container.stop();
}
}, 60_000);
+123
View File
@@ -122,6 +122,7 @@ async function setup(env: Record<string, string> = {}) {
list: vi.fn(async () => [revision]),
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
read: vi.fn(async () => ({ workspace, revision })),
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
} as unknown as WorkspaceRegistry;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test", ...env }), {
thtRunner: {} as never,
@@ -605,3 +606,125 @@ test("rejects a missing database without partially cleaning valid selections", a
expect(response.statusCode).toBe(404);
expect(await repository.listTables(database.id)).toHaveLength(2);
});
test("serves one relationship map and supports the manual relationship lifecycle", async () => {
const { app, repository, database } = await setup();
await seedCatalog(repository, database);
const tables = await repository.listTables(database.id);
const patients = tables.find((table) => table.name === "patients")!;
const visits = tables.find((table) => table.name === "visits")!;
const patientId = (await repository.listColumns(database.id, patients.id)).find((item) => item.name === "id")!;
const visitId = (await repository.listColumns(database.id, visits.id)).find((item) => item.name === "id")!;
const created = await app.inject({
method: "POST",
url: `/catalog/databases/${database.id}/relationships`,
payload: { sourceColumnId: visitId.id, targetColumnId: patientId.id },
});
expect(created.statusCode).toBe(201);
expect(created.json()).toMatchObject({ origin: "manual", status: "active", constraintName: null });
const listed = (await app.inject({
method: "GET", url: `/catalog/databases/${database.id}/relationships`,
})).json();
expect(listed.map((item: { origin: string }) => item.origin).sort()).toEqual(["manual", "physical"]);
const relationshipId = created.json().id;
const excluded = await app.inject({
method: "PATCH",
url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
payload: { status: "excluded" },
});
expect(excluded.statusCode).toBe(200);
expect(excluded.json()).toMatchObject({ origin: "manual", status: "excluded" });
const restored = await app.inject({
method: "PATCH",
url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
payload: { status: "active" },
});
expect(restored.json()).toMatchObject({ status: "active" });
expect((await app.inject({
method: "DELETE", url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
})).statusCode).toBe(204);
});
test("rejects generated inference while the Catalog schema is not current", async () => {
const { app, database } = await setup();
const response = await app.inject({
method: "POST",
url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
});
expect(response.statusCode).toBe(409);
expect(response.json()).toEqual({
code: "relationship_schema_stale",
message: "Synchronize the current database schema before managing logical relationships.",
});
});
test("rebuilds generated relationships and returns the exact summary", async () => {
const { app, repository, database } = await setup();
const observed = snapshot();
observed.relationships = [];
await seedCatalog(repository, database, observed);
const first = await app.inject({
method: "POST", url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
});
expect(first.statusCode).toBe(200);
expect(first.json()).toEqual({ added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0 });
const generated = (await repository.listLogicalRelationships(database.id))[0]!;
await app.inject({
method: "PATCH",
url: `/catalog/databases/${database.id}/relationships/${generated.id}`,
payload: { status: "excluded" },
});
const second = await app.inject({
method: "POST", url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
});
expect(second.json()).toEqual({ added: 0, alreadyPresent: 0, excluded: 1, ambiguous: 0 });
});
test("validates relationship requests and keeps physical relationships read-only", async () => {
const { app, repository, database } = await setup();
await seedCatalog(repository, database);
const physical = (await repository.listRelationships(database.id))[0]!;
const invalid = await app.inject({
method: "POST",
url: `/catalog/databases/${database.id}/relationships`,
payload: { sourceColumnId: physical.columns[0].sourceColumnId, targetColumnId: physical.columns[0].targetColumnId, generated: true },
});
expect(invalid.statusCode).toBe(400);
expect(invalid.json()).toMatchObject({ code: "relationship_request_invalid" });
const readOnly = await app.inject({
method: "PATCH",
url: `/catalog/databases/${database.id}/relationships/${physical.id}`,
payload: { status: "excluded" },
});
expect(readOnly.statusCode).toBe(409);
expect(readOnly.json()).toMatchObject({ code: "relationship_read_only" });
});
test("requires database.manage for relationship map mutations", async () => {
const { app, repository, database } = await setup({ AUTH_MODE: "upstream" });
const observed = snapshot();
observed.relationships = [];
await seedCatalog(repository, database, observed);
const response = await app.inject({
method: "POST",
url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
headers: {
"x-thoth-principal-issuer": "portal",
"x-thoth-principal-subject": "catalog-reader",
"x-thoth-is-admin": "0",
},
});
expect(response.statusCode).toBe(403);
expect(response.json()).toEqual({ code: "auth_forbidden", error: "This operation is not permitted" });
});
@@ -54,6 +54,7 @@ async function setup() {
list: vi.fn(async () => [revision]),
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
read: vi.fn(async () => ({ workspace, revision })),
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
} as unknown as WorkspaceRegistry;
const secretStore = new WorkspaceSecretStore({ root: secretRoot, runtimeRoot, installationId: "test" });
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test" }), {
@@ -0,0 +1,196 @@
import { describe, expect, test, vi } from "vitest";
import { EffectiveRelationshipSnapshotProvider } from "../src/catalog/effective-relationship-snapshot.js";
import type { CatalogRelationship, WorkspaceDatabase } from "../src/catalog/types.js";
const timestamp = "2026-08-31T10:00:00.000Z";
function database(): WorkspaceDatabase {
return {
id: "database-1",
workspaceId: "psd",
engine: "postgres",
databaseName: "warehouse",
schema: "public",
version: 1,
createdAt: timestamp,
updatedAt: timestamp,
binding: { transport: "postgres_direct" },
connectionStatus: "reachable",
schemaSyncedVersion: 1,
schemaSyncedAt: timestamp,
};
}
const operations = {
async run<T>(_databaseId: string, operation: () => Promise<T>): Promise<T> {
return await operation();
},
};
function relationship(options: {
id: string;
origin: "physical" | "generated" | "manual";
status?: "active" | "excluded";
sourceTable?: string;
sourceColumns?: string[];
targetTable?: string;
targetColumns?: string[];
}): CatalogRelationship {
const sourceColumns = options.sourceColumns ?? ["user_id"];
const targetColumns = options.targetColumns ?? ["id"];
const columns = sourceColumns.map((sourceColumnName, position) => ({
position: position + 1,
sourceColumnId: `source-${position}`,
sourceColumnName,
targetColumnId: `target-${position}`,
targetColumnName: targetColumns[position],
}));
const common = {
id: options.id,
databaseId: "database-1",
sourceTableId: "source-table",
sourceTableName: options.sourceTable ?? "orders",
targetTableId: "target-table",
targetTableName: options.targetTable ?? "users",
columns,
createdAt: timestamp,
updatedAt: timestamp,
};
if (options.origin === "physical") {
return {
...common,
constraintName: `fk_${options.id}`,
updateRule: "NO ACTION",
deleteRule: "NO ACTION",
deferrable: false,
initiallyDeferred: false,
lastSyncedDatabaseVersion: 1,
lastSyncedAt: timestamp,
origin: "physical",
status: "active",
};
}
return {
...common,
constraintName: null,
updateRule: null,
deleteRule: null,
deferrable: false,
initiallyDeferred: false,
lastSyncedDatabaseVersion: null,
lastSyncedAt: null,
origin: options.origin,
status: options.status ?? "active",
columns: [columns[0]!],
};
}
describe("EffectiveRelationshipSnapshotProvider", () => {
test("returns no projection for a workspace without a Catalog database", async () => {
const list = vi.fn();
const provider = new EffectiveRelationshipSnapshotProvider(
{
get: vi.fn().mockResolvedValue(undefined),
getByWorkspace: vi.fn().mockResolvedValue(undefined),
},
{ list },
operations,
);
await expect(provider.render("legacy")).resolves.toBeUndefined();
expect(list).not.toHaveBeenCalled();
});
test("renders a deterministic active map with physical precedence", async () => {
const duplicateGenerated = relationship({ id: "generated", origin: "generated" });
const physical = relationship({ id: "physical", origin: "physical" });
const compositePhysical = relationship({
id: "physical-composite",
origin: "physical",
sourceTable: "order_lines",
sourceColumns: ["order_id", "tenant_id"],
targetTable: "orders",
targetColumns: ["id", "tenant_id"],
});
const manual = relationship({
id: "manual",
origin: "manual",
sourceTable: "invoices",
sourceColumns: ["customer_id"],
targetTable: "customers",
targetColumns: ["id"],
});
const excluded = relationship({
id: "excluded",
origin: "generated",
status: "excluded",
sourceTable: "invoices",
});
const list = vi.fn().mockResolvedValue([
manual,
duplicateGenerated,
excluded,
physical,
compositePhysical,
]);
const provider = new EffectiveRelationshipSnapshotProvider(
{
get: vi.fn().mockResolvedValue(database()),
getByWorkspace: vi.fn().mockResolvedValue(database()),
},
{ list },
operations,
);
const rendered = await provider.render("psd");
expect(rendered?.endsWith("\n")).toBe(true);
expect(JSON.parse(rendered!)).toEqual({
schemaVersion: 1,
workspaceId: "psd",
relationships: [
{
sourceTable: "invoices",
sourceColumns: ["customer_id"],
targetTable: "customers",
targetColumns: ["id"],
origin: "manual",
},
{
sourceTable: "order_lines",
sourceColumns: ["order_id", "tenant_id"],
targetTable: "orders",
targetColumns: ["id", "tenant_id"],
origin: "physical",
},
{
sourceTable: "orders",
sourceColumns: ["user_id"],
targetTable: "users",
targetColumns: ["id"],
origin: "physical",
},
],
});
expect(list).toHaveBeenCalledWith("database-1");
});
test("fails closed when the Catalog schema is absent or stale", async () => {
const stale = database();
delete stale.schemaSyncedVersion;
delete stale.schemaSyncedAt;
const list = vi.fn();
const provider = new EffectiveRelationshipSnapshotProvider(
{
get: vi.fn().mockResolvedValue(stale),
getByWorkspace: vi.fn().mockResolvedValue(stale),
},
{ list },
operations,
);
await expect(provider.render("psd")).rejects.toThrow(
"effective relationship snapshot requires a current full schema synchronization",
);
expect(list).not.toHaveBeenCalled();
});
});
+57
View File
@@ -517,6 +517,63 @@ test("creates a session from the active immutable workspace revision", async ()
}));
});
test("hands one effective relationship snapshot to both retrieval and Pi", async () => {
const effective = JSON.stringify({
schemaVersion: 1,
workspaceId: "default",
relationships: [],
});
const render = vi.fn(async () => effective);
const acquireWorkspaceRuntime = vi.fn((_workspace: string, relationships?: string) => ({
path: "/runtime/with-relationships.yaml",
workspaceId: "default",
workspaceRevision: "e".repeat(40),
release: vi.fn(),
}));
const searchPack = vi.fn(async () => {});
const createFor = vi.fn(() => ({ bridge: { onClientEvent: () => {} } }));
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
sessionNew: async () => ({ id: "effective-map" }),
acquireWorkspaceRuntime,
searchPack,
} as any,
effectiveRelationshipSnapshotProvider: { render } as any,
readiness: { ensure: async () => ({ ok: true }) } as any,
mgr: {
get: () => undefined,
createFor,
configure: async () => {},
start: () => {},
} as any,
getSettings: () => ({ workspace: "default", thinking: "low" }) as any,
});
const response = await app.inject({
method: "POST",
url: "/sessions",
payload: { question: "Which users placed orders?", workspaceId: "default" },
});
expect(response.statusCode).toBe(200);
expect(render).toHaveBeenCalledWith("default");
expect(acquireWorkspaceRuntime).toHaveBeenCalledWith(
expect.stringContaining("/default.yaml"),
effective,
);
expect(createFor).toHaveBeenCalledWith(
"effective-map",
expect.objectContaining({
runtimeConfig: expect.objectContaining({ path: "/runtime/with-relationships.yaml" }),
}),
);
expect(searchPack).toHaveBeenCalledWith(
"Which users placed orders?",
"effective-map",
"/runtime/with-relationships.yaml",
);
});
test("rejects an SSH-only workspace before persisting or starting a session", async () => {
const sessionNew = vi.fn(async () => ({ id: "must-not-exist" }));
const ensure = vi.fn(async () => ({ ok: true }));
@@ -197,6 +197,25 @@ test("ThtRunner uses a vault secret only for the lifetime of its runtime lease",
expect(existsSync(rendered.database.password_file)).toBe(false);
});
test("ThtRunner binds and cleans the effective relationship snapshot with its runtime lease", async () => {
const f = await fixture();
const runner = runnerFor(f);
const relationships = JSON.stringify({
schemaVersion: 1,
workspaceId: "psd-clinical",
relationships: [],
});
const lease = runner.acquireWorkspaceRuntime(f.revision.snapshotPath, relationships);
const rendered = parse(readFileSync(lease.path, "utf8")) as {
paths: { effective_relationships: string };
};
expect(readFileSync(rendered.paths.effective_relationships, "utf8")).toBe(relationships);
lease.release();
expect(existsSync(rendered.paths.effective_relationships)).toBe(false);
});
test("separate runtime leases hand off byte-identical revision Evidence configs accepted by tht", async () => {
const f = await fixture();
const runner = runnerFor(f);
@@ -0,0 +1,36 @@
# Use the Catalog as the logical relationship authority
ThothII stores database-declared foreign keys and user-managed Logical Relationships in separate
Catalog models, as required by ADR-0006, but exposes them through one effective relationship map.
Physical relationships remain read-only and are refreshed from the database. Logical relationships
are either Generated by deterministic column-name inference or Manual; inference does not call an
AI model and does not inspect source values.
A generated rebuild is additive. It preserves active and Manual relationships, never reactivates a
logically deleted relationship, and may recreate a relationship only after permanent deletion. A
logical deletion is therefore represented by retaining the relationship with an exclusion marker;
a permanent deletion removes it. Inference accepts only unambiguous, type-compatible matches to a
single-column primary key and skips all other candidates. It recognizes normalized table-qualified
names such as `user_id -> users.id`, exact non-generic primary-key names with one owner, and the
warehouse convention `*time_key -> dim_time.<single PK>`. A source column may participate in a
composite primary key; bare generic names such as `id`, `key`, `code`, and `pk` are not evidence by
themselves.
An exclusion is durable while both Catalog Column endpoints exist. Explicit metadata cleanup of an
endpoint table or column is a destructive boundary: it permanently removes every relationship and
exclusion attached to that endpoint, invalidates the synchronized-catalog marker, and requires a
full schema synchronization. The newly imported endpoints may then be inferred again. Preserving an
exclusion across endpoint destruction would require a second denormalized name-based identity, which
this design deliberately avoids.
The installation-local Catalog is the sole writable authority for Logical Relationships. Git-pinned
workspace annotations remain authoritative for descriptive metadata but their embedded foreign keys
are legacy compatibility data. The backend materializes the active effective map as an immutable,
deterministic runtime snapshot when a session starts or resumes. When that snapshot is present, the
harness uses it as the exclusive relationship source and ignores relationships embedded in both the
physical schema artifact and workspace annotations; a missing or invalid declared snapshot fails
closed. Legacy runtimes without a snapshot retain the previous merge behavior.
The snapshot is a projection, not another authored store. Its lifetime is tied to the runtime-config
lease, physical relationships take precedence over duplicate Logical Relationships, composite-key
column order is retained, and one Pi process observes one stable map for its complete lifetime.
+16 -3
View File
@@ -6,7 +6,7 @@ session workflow.
## What the catalog owns
For each YAML workspace, an administrator may create at most one database configuration. It holds
For each YAML workspace, an administrator may configure at most one Metadata Catalog binding. It holds
the database name, schema, connection binding, write-only encrypted secrets, observed physical
schema, optional curated descriptions, generated descriptions, and durable operation history.
@@ -27,6 +27,18 @@ It uses `GET /catalog/metrics` without `databaseId` for installation totals and
the current database. Choose a selection-scoped operation from the action selector and then press
**Run**; unavailable operations remain listed with an explanation. Row-specific actions are the icon
controls in the final column, and each navigation or action icon has an immediate conceptual tooltip.
An unconfigured workspace exposes **Configure catalog** directly on its row; there is no global
database-creation action and the selected workspace cannot be changed in the configuration form.
The master grid keeps three independent states visible:
- **Revision / Evidence** comes from the active immutable workspace revision. Filesystem Evidence is
materialized with that revision; remote Evidence is reported as configured-but-unverified or as
requiring credentials.
- **NL→SQL runtime** is calculated from the workspace DWH/Evidence requirements and runtime secret
store. It also reports transports, such as SSH, that are diagnostic-only and unsupported by sessions.
- **Metadata Catalog** reports whether the installation-local catalog configuration exists, then shows
its separately versioned connection-test or synchronization state.
Configuration, object details, metadata editors, synchronization history, description history,
sensitive-field review, and suggestion-run history open in right-side drawers backed by the
@@ -42,8 +54,9 @@ remains available only until the integrated Fleet Ledger surface passes owner ac
## Configure and test a database
1. Open **Database Management** and choose a workspace.
2. Create its PostgreSQL configuration. Choose `postgres_direct`, `rest_api`, or `ssh_tunnel` and
1. Open **Database Management** and find the repository workspace marked **Not configured**.
2. Choose **Configure catalog** on that row. Configure its PostgreSQL catalog binding with
`postgres_direct`, `rest_api`, or `ssh_tunnel` and
complete the binding fields that the chosen transport requires.
3. Enter secrets only when replacing them. They remain write-only and are never returned by the
application.
+43
View File
@@ -0,0 +1,43 @@
# ThothII Docker refresh
The active local `psd-local` stack uses the operator environment generated for the installation:
`deploy/psd/operator.env`
It is not `deploy/thothii.env` (that file is empty) and `deploy/env/local.env` is only an example
path referenced by the generic launcher. The running stack also uses these Compose overlays:
```text
compose.yaml
deploy/compose.local.yaml
deploy/compose.git-ssh.yaml
deploy/psd/connector-secrets.yaml
```
Refresh the stack from the repository root with:
```bash
compose_psd=(
docker compose
--env-file deploy/psd/operator.env
-p thothii-18998cca7b0a
-f compose.yaml
-f deploy/compose.local.yaml
-f deploy/compose.git-ssh.yaml
-f deploy/psd/connector-secrets.yaml
)
"${compose_psd[@]}" config --quiet
"${compose_psd[@]}" build core frontend
"${compose_psd[@]}" stop core frontend
"${compose_psd[@]}" up -d catalog-db
"${compose_psd[@]}" run --rm catalog-migrate
"${compose_psd[@]}" up -d --remove-orphans
```
Building before the stop keeps the existing application available if an image fails to compile.
Stopping only `core` and `frontend` prevents the old backend from using a newly migrated catalog;
the database, Qdrant, embedding service, named volumes, and installation state remain in place.
The installation secret files referenced by that env file live under `deploy/psd/secrets/` and
must never be committed or printed.
@@ -109,12 +109,13 @@ canonico; i percorsi e i comportamenti descrivono il sorgente disponibile il 202
conserva il valore esistente e la sostituzione è un'azione esplicita. Delete rimuove anche i
segreti associati.
34. La pagina usa AG Grid come master e un form React come detail, con sezioni Database, Connection
e TLS/SSH condizionali. La toolbar offre `Add database`; le righe `unconfigured` offrono
`Configure`. Entrambe selezionano esclusivamente workspace YAML senza un database e creano il
record soltanto al Save; `workspace_id` diventa immutabile dopo la creazione.
35. La grid mostra workspace, database, schema, transport, endpoint, stato connessione e ultimo
aggiornamento. Su schermi piccoli il dettaglio occupa il pannello completo. Il cambio riga con
modifiche non salvate e Delete richiedono conferma, senza conferma testuale tipizzata.
e TLS/SSH condizionali. Non esiste un'azione globale `Add database`: ogni riga `unconfigured`
offre `Configure catalog`, apre il form già vincolato a quello specifico workspace YAML e crea il
record soltanto al Save; `workspace_id` non è selezionabile né modificabile.
35. La grid mostra separatamente revisione/Evidence del workspace, binding runtime NL→SQL e
configurazione del Metadata Catalog, oltre a database, schema, endpoint e ultimo aggiornamento.
Su schermi piccoli il dettaglio occupa il pannello completo. Il cambio riga con modifiche non
salvate e Delete richiedono conferma, senza conferma testuale tipizzata.
36. La Database Binding conserva `connection_status`, `tested_version`, `last_tested_at`, un codice
errore e un messaggio breve sanificato. Non conserva stack trace, DSN, credenziali o output grezzo
del driver.
@@ -0,0 +1,299 @@
# Visualizzatore ERD nel browser: proposta solo open source
Data dell'audit: 31 agosto 2026.
## Decisione
Per ThothII sceglierei **AntV X6 + ELK.js** come fondazione del visualizzatore ERD.
- **AntV X6** è MIT, non ha una distinta edizione Professional e include nel progetto OSS
le funzioni che servono davvero a un diagramma complesso: nodi e porte personalizzabili,
pan/zoom, selezione, minimappa, history, clipboard, tastiera, routing, virtual rendering ed
export SVG/PNG/JPEG.
- **ELK.js** è il motore di layout automatico, EPL-2.0 e senza tier commerciale. Il layout
layered gestisce porte, vincoli sull'ordine delle porte, archi multipli, self-loop e routing
ortogonale: proprietà importanti per foreign key composite e schemi affollati.
- La combinazione non dipende da esempi o componenti a pagamento. Il lavoro applicativo che
resta da fare — semantica ER, livelli di dettaglio, filtri e adattatore X6/ELK — appartiene
davvero al dominio di ThothII e non è una funzione nascosta dietro un piano Pro.
La seconda scelta è **React Flow + ELK.js**. È probabilmente l'integrazione più naturale con
l'attuale frontend React e ha una migliore storia documentata per accessibilità. Il runtime è
MIT e non viene tecnicamente depotenziato, ma diversi pattern avanzati già implementati
(auto-layout, edge routing, undo/redo, copy/paste, expand/collapse) sono pubblicati come esempi
con licenza React Flow Pro. Non è quindi la scelta più lineare se il criterio prioritario è
evitare anche una dipendenza progettuale da materiale Professional.
## Cosa significa «open source senza limitazioni Professional»
L'audit distingue tre casi:
1. **OSS completo**: licenza OSI e nessun tier commerciale che trattenga funzioni essenziali.
2. **OSS con caveat**: il motore è aperto, ma esistono esempi premium, API instabili o limiti
architetturali rilevanti. Può essere usato, purché il limite sia accettato esplicitamente.
3. **Escluso**: licenza proprietaria oppure core aperto con funzioni necessarie al viewer
complesso riservate alla versione commerciale.
La valutazione riguarda sia la licenza sia ciò che si può effettivamente costruire senza
acquistare un prodotto complementare. Le note sulle licenze sono tecniche, non consulenza legale.
## Matrice di selezione
| Soluzione | Licenza OSS | Tier Pro rilevante | Adeguatezza a ERD complessi | Verdetto |
|---|---|---|---|---|
| **AntV X6 + ELK.js** | MIT + EPL-2.0 | Nessuno individuato | Porte per colonna, minimappa, routing, export, virtual rendering e layout avanzato | **Consigliata** |
| **React Flow + ELK.js** | MIT + EPL-2.0 | Esempi/pattern avanzati Pro, non runtime separato | Ottima UX React, nodi HTML, minimappa e accessibilità; export SVG non nativo | **Valida con caveat** |
| **maxGraph** | Apache-2.0 | Nessuno | Molto completo, SVG, porte, folding e layout; API imperativa e integrazione React costosa | **Valida con caveat** |
| **Cytoscape.js + ELK** | MIT | Nessuno | Molto scalabile come grafo, ma meno adatto a tabelle ricche e porte per campo | **Alternativa di nicchia** |
| **Liam ERD/CLI** | Apache-2.0 | Nessuno | Viewer ERD già rifinito e self-hostable; il package embeddable è interno e instabile | **Reference o app separata** |
| **Graphviz + Viz.js** | EPL-2.0 + MIT | Nessuno | Layout ed SVG eccellenti; non offre da solo un explorer applicativo | **Renderer di export** |
| **Mermaid.js + Panzoom** | MIT + MIT/ISC | Mermaid Chart è separato | Facile, ma insufficiente per interazioni e modelli ER ricchi | **Solo preview semplice** |
| **Sprotty/GLSP** | EPL-2.0 | Nessuno | Potente e completo, ma è un framework di modeling più pesante del necessario | **Non prioritario** |
| **JointJS** | Core MPL-2.0 | Molte funzioni utili sono in JointJS+ | Lo split commerciale intercetta proprio le necessità del viewer | **Esclusa** |
| **GoJS** | Proprietaria | Licenza di deployment | Completa tecnicamente, ma non open source | **Esclusa** |
| **yFiles** | Proprietaria | Licenza commerciale | Completa tecnicamente, ma non open source | **Esclusa** |
## 1. Scelta principale: AntV X6 + ELK.js
### Funzioni disponibili nell'open source
X6 offre un canvas diagrammatico basato su SVG con supporto anche a nodi HTML/React. Il core
espone porte, archi personalizzabili, self-loop e multiedge. I plugin distribuiti dal progetto
comprendono Scroller, MiniMap, Selection, History, Clipboard, Keyboard, DnD, Stencil, Snapline,
Transform ed Export. La documentazione copre inoltre router `orth`, `manhattan` ed `er`, zoom e
panning. Non è emersa un'edizione X6 Pro che renda a pagamento queste capacità.
Fonti primarie: [repository e licenza X6](https://github.com/antvis/X6),
[porte](https://x6.antv.antgroup.com/en/tutorial/basic/port),
[router](https://x6.antv.antgroup.com/en/api/registry/router),
[minimappa](https://x6.antv.antgroup.com/en/tutorial/plugins/minimap),
[scroller](https://x6.antv.antgroup.com/en/tutorial/plugins/scroller) ed
[export](https://x6.antv.antgroup.com/en/tutorial/plugins/export).
ELK non è un renderer: calcola la geometria del grafo. L'algoritmo layered supporta porte e
relativi vincoli, archi ortogonali, self-loop e archi multipli. Va eseguito in un Web Worker per
non bloccare l'interfaccia sui modelli grandi. L'adattatore dovrà passare a ELK le porte delle
colonne e usare anche le `edge sections` e i bend point restituiti, non soltanto le coordinate
dei nodi.
Fonti primarie: [ELK.js e licenza](https://github.com/kieler/elkjs) e
[ELK layered](https://eclipse.dev/elk/reference/algorithms/org-eclipse-elk-layered.html).
### Limiti reali
- X6 è più imperativo di React Flow: conviene incapsularlo in un singolo componente React con
un adapter stabile, evitando di spargere istanze e listener nel resto dell'applicazione.
- L'accessibilità per screen reader non è documentata al livello di React Flow; tab order,
focus, descrizioni ARIA e navigazione da tastiera richiedono test e lavoro applicativo.
- Un nodo React/HTML può introdurre `foreignObject` nell'SVG. Per un export portabile e
stampabile è meglio produrre un SVG separato dallo stesso modello e dalla geometria ELK.
Questi sono costi tecnici, non limitazioni commerciali.
## 2. Seconda scelta: React Flow + ELK.js
`@xyflow/react` è MIT. Custom nodes React, handle multipli, pan/zoom, fit view, Controls,
MiniMap, selezione e funzioni di accessibilità sono nel runtime OSS. React Flow Pro vende
supporto, template ed esempi con relativo codice sorgente; non esiste una libreria runtime Pro
che sblocchi il canvas.
Il caveat è comunque concreto: auto-layout, edge routing, undo/redo, copy/paste ed
expand/collapse compaiono nel catalogo degli esempi Pro e quel codice usa la **xyflow Pro
License**, non la MIT del core. Le stesse funzioni possono essere sviluppate sopra le API OSS,
ma non si può considerare tutto il materiale ufficiale liberamente riutilizzabile.
Fonti primarie: [package React Flow](https://github.com/xyflow/xyflow/blob/main/packages/react/package.json),
[funzioni del core](https://reactflow.dev/index),
[catalogo degli esempi Pro](https://reactflow.dev/examples/pro-examples),
[auto-layout](https://reactflow.dev/examples/layout/auto-layout) e
[termini Pro](https://reactflow.dev/pro).
È una buona scelta se si privilegiano velocità d'integrazione React, componenti HTML e
accessibilità. Non è la prima scelta di questo audit perché l'utente ha chiesto espressamente
di minimizzare la distanza fra ciò che è open source e l'offerta Professional.
Altro limite: React Flow combina nodi DOM e archi SVG. L'esempio ufficiale di download usa
`html-to-image`, quindi il diagramma interattivo non si traduce automaticamente in un SVG puro.
## 3. Altre soluzioni interamente open source
### maxGraph
maxGraph, successore TypeScript di mxGraph, è Apache-2.0 e non ha un piano Pro. Offre rendering
SVG, connection constraints/porte, loop e multigraph, layout, routing, outline, grouping e
folding. Può quindi sostenere un editor ERD ricco.
Lo terrei come terza scelta: l'API è imperativa, non esiste un wrapper React ufficiale, la
virtualizzazione non è documentata e gran parte dell'accessibilità va costruita. Il progetto è
ancora pre-1.0. Il costo di integrazione e manutenzione sarebbe maggiore di X6.
Fonti: [licenza e repository](https://github.com/maxGraph/maxGraph),
[guida Vite/TypeScript](https://maxgraph.github.io/maxGraph/docs/getting-started/) e
[gestione della complessità](https://maxgraph.github.io/maxGraph/docs/usage/group-and-complexity-management/).
### Cytoscape.js
Cytoscape.js e molte estensioni sono MIT, senza tier professionale. È ottimo per grandi grafi,
filtri, selezioni e algoritmi di rete. Il rendering Canvas è però meno naturale per schede-tabella
ricche, testo selezionabile e handle collegati alle singole colonne. Inoltre l'adapter
`cytoscape.js-elk` non passa le porte a ELK né usa le route degli archi: non risolve da solo il
problema delle foreign key per colonna.
È appropriato per una vista di dipendenze ad alto livello, non come renderer ERD principale.
L'estensione `cytoscape-svg` è GPL-3.0; in un prodotto che non vuole assorbire quel vincolo è
preferibile una pipeline Graphviz/Viz.js o un generatore SVG proprio.
Fonti: [Cytoscape.js](https://github.com/cytoscape/cytoscape.js),
[adapter ELK](https://github.com/cytoscape/cytoscape.js-elk) e
[cytoscape-svg](https://github.com/kinimesi/cytoscape-svg).
### Liam ERD
Liam ERD è Apache-2.0, self-hostable e già orientato al problema: pan/zoom, ricerca, filtro,
command palette e modalità `TABLE_NAME`, `KEY_ONLY`, `ALL_FIELDS`. La documentazione dichiara
supporto a schemi con oltre cento tabelle. La CLI può generare un'app Vite statica.
Non userei però direttamente `@liam-hq/erd-core` dentro ThothII: il maintainer lo definisce una
dipendenza interna, ne sconsiglia l'uso diretto e avverte che l'API può cambiare; il package è
ancora 0.x. Inoltre occorre verificare la compatibilità con la versione React di ThothII e la
fedeltà delle foreign key composite. Liam è quindi un ottimo benchmark UX, una CLI per una vista
separata o una base da forkare consapevolmente, non l'interfaccia stabile su cui fondare il
prodotto.
Fonti: [repository Liam](https://github.com/liam-hq/liam),
[README di erd-core](https://github.com/liam-hq/liam/blob/main/frontend/packages/erd-core/README.md),
[funzioni UI](https://liambx.com/docs/ui-features) e [CLI](https://liambx.com/docs/cli).
### Sprotty/GLSP
Sprotty e Eclipse GLSP sono EPL-2.0 e offrono un'infrastruttura seria per diagrammi SVG,
layout e protocolli client/server. Sono pensati per strumenti di modeling estensibili, con
dependency injection e un'architettura più ampia di un viewer. Restano una soluzione OSS valida,
ma sproporzionata per questa esigenza salvo che ThothII evolva in un vero editor di modelli.
Fonti: [Sprotty](https://github.com/eclipse-sprotty/sprotty) ed
[Eclipse GLSP](https://eclipse.dev/glsp/documentation/overview/).
## 4. Renderer complementari, non fondazioni del viewer
### Graphviz e Viz.js
Graphviz è EPL-2.0; `@viz-js/viz`, il port WebAssembly utilizzabile nel browser, è MIT. Graphviz
produce SVG di alta qualità, supporta label HTML-like e porte e rimane molto utile per export,
stampa o una vista read-only. Non fornisce però da solo ricerca, filtri, livelli di dettaglio,
editing o gestione dello stato applicativo.
Lo userei come renderer di export alternativo, non come UI primaria. Se il layout a schermo
deve corrispondere esattamente all'export, è preferibile generare l'SVG direttamente dalla
geometria ELK invece di mantenere due motori di layout indipendenti.
Fonti: [licenza Graphviz](https://graphviz.org/license/),
[formati SVG](https://graphviz.org/docs/outputs/svg/) e
[Viz.js](https://github.com/mdaines/viz-js).
### Mermaid.js e librerie di pan/zoom
Mermaid.js è MIT e non ha feature del renderer open source bloccate. **Mermaid Chart** è invece
un servizio commerciale distinto e il suo piano gratuito ha limiti: non va confuso con la
libreria self-hosted.
Si può aggiungere navigazione a un SVG Mermaid con `@panzoom/panzoom` (MIT) o `d3-zoom` (ISC),
entrambi senza tier Pro. Questo migliora l'esperienza corrente, ma non supera i limiti strutturali
del diagramma ER Mermaid: interazioni a livello di tabella, controllo ridotto delle porte e del
routing, assenza di semantic zoom e difficoltà crescente su schemi molto grandi.
Mermaid resta adatto a preview e documentazione, non al viewer finale.
Fonti: [Mermaid.js](https://github.com/mermaid-js/mermaid),
[distinzione da Mermaid Chart](https://mermaid.ai/open-source/ecosystem/mermaid-chart.html),
[Panzoom](https://github.com/timmywil/panzoom) e [d3-zoom](https://github.com/d3/d3-zoom).
### Python
SchemaSpy ed ERAlchemy sono open source e possono generare documentazione o grafi attraverso
Graphviz, ma non sostituiscono il componente interattivo React. Python è utile lato backend per
normalizzare metadati o produrre artefatti batch; pan/zoom, selezione, ricerca e dettagli restano
responsabilità del browser. Aggiungere un servizio Python solo per disegnare l'ERD non porta un
vantaggio architetturale a ThothII.
## 5. Soluzioni escluse
### JointJS / JointJS+
Il core JointJS è MPL-2.0, ma JointJS+ è commerciale e comprende proprio molte funzioni che
servirebbero qui: PaperScroller, Navigator/minimappa, selection, clipboard, keyboard, undo/redo,
toolbar, export e layout aggiuntivi. Sarebbe tecnicamente possibile ricostruirle sul core, ma la
distanza fra OSS e Professional è troppo grande rispetto al criterio richiesto.
Fonti: [licenza](https://www.jointjs.com/license),
[confronto delle funzioni](https://www.jointjs.com/features) e
[prezzi](https://www.jointjs.com/pricing).
### GoJS e yFiles
Sono prodotti completi e maturi, ma non open source. GoJS richiede una licenza per il deployment
e mostra una filigrana senza chiave; yFiles usa licenze proprietarie, con evaluation temporanea e
licenza necessaria per la distribuzione. Non soddisfano il requisito, quindi non entrano nella
shortlist.
Fonti: [licensing GoJS](https://gojs.net/latest/intro/deployment.html) e
[licensing yFiles](https://docs.yworks.com/yfiles-html/dguide/deployment/licensing.html).
## 6. Architettura proposta per ThothII
Il frontend usa React 18 e oggi `SchemaLinkingViewer.tsx` genera Mermaid con un limite di 45
elementi. La resa corrente perde informazione: accorpa più foreign key fra la stessa coppia di
tabelle, omette i self-reference e usa cardinalità generiche. Il catalogo espone già tabelle,
colonne, chiavi primarie e relazioni con coppie ordinate di colonne, quindi può alimentare un
modello più fedele.
Propongo questa separazione:
1. **Modello renderer-neutral**: `TableNode`, `ColumnPort` e `RelationEdge` con nome del
constraint e lista ordinata delle coppie sorgente/destinazione. Non appiattire le FK composite.
2. **Snapshot API**: un endpoint coerente, per esempio
`GET /catalog/databases/:databaseId/schema-diagram`, che restituisca tabelle, colonne,
relazioni e versione del catalogo in una sola lettura, evitando la richiesta colonne N+1.
3. **Layout worker**: ELK.js in Web Worker, con porte per colonna, port constraints, route degli
archi e cache per versione del database e filtri correnti.
4. **Renderer interattivo**: X6 incapsulato in `DatabaseRelationshipDiagram.tsx`, affiancato
alla vista tabellare con un toggle List/Diagram e con riuso del drawer dei dettagli.
5. **Export**: generatore SVG separato basato sullo stesso modello e sulla geometria ELK;
Graphviz/Viz.js può essere un fallback per layout alternativi o documentazione batch.
Il catalogo non registra oggi tutti i vincoli UNIQUE né il tipo MATCH delle FK. Senza questi
dati non sempre è possibile distinguere correttamente 1:1 da 1:N. Il renderer non deve inventare
la cardinalità: deve mostrare una notazione neutra finché il metadato necessario non viene
raccolto.
### Funzioni necessarie per schemi complessi
- semantic zoom: solo nomi tabella, poi PK/FK, infine tutti i campi;
- ricerca e filtri per schema, tabella, colonna e tipo di relazione;
- modalità focus con vicinato a uno o due hop;
- evidenziazione della relazione e delle due colonne al passaggio/selezione;
- minimappa, fit selection, navigazione da tastiera e pannello dettagli;
- distinzione visiva di PK, FK, nullable, composite key, self-loop e relazioni parallele;
- layout automatico ricalcolabile, ma posizioni manuali persistibili;
- rendering progressivo o virtuale e fallback tabellare sempre disponibile;
- export dell'intero schema e dell'area filtrata in SVG/PNG.
## 7. Proof of concept consigliato
Un POC breve dovrebbe usare **X6 + ELK.js** su tre dataset sintetici: circa 30, 150 e 500
tabelle, includendo FK composite, più FK tra la stessa coppia, self-loop, tabelle isolate e hub
ad alto grado.
I criteri di accettazione dovrebbero misurare:
- tempo di layout nel worker e tempo al primo frame interattivo;
- fluidità di pan/zoom e uso memoria sul caso da 500 tabelle;
- correttezza di porte, archi paralleli, self-loop e FK composite;
- leggibilità nelle tre soglie di semantic zoom;
- navigazione completa da tastiera e comportamento con screen reader;
- qualità e portabilità dell'SVG esportato;
- assenza di codice, esempi o componenti soggetti a licenza commerciale.
Se X6 non raggiunge il livello di accessibilità richiesto senza un costo eccessivo, il confronto
finale va fatto con React Flow + lo stesso adapter ELK e lo stesso modello dati. In questo modo
si cambia renderer senza rifare API, semantica delle relazioni o pipeline di export.
@@ -0,0 +1,246 @@
# Gestione delle relationship: da ThothAI a ThothII
**Stato:** analisi e direzione funzionale/UX confermate nel *grill with docs*; non costituisce ancora un piano di implementazione.
**Revisione esaminata:** commit ThothII `f586152636b1fd653b0bc1d40b54be7f89bbd2bb`. I sorgenti legacy di ThothAI citati sotto sono versionati nello stesso repository, sotto `Thoth/ThothAI/`.
**Ambito:** import delle foreign key fisiche, inferenza di relationship logiche, modifica umana, pubblicazione verso il workflow NL→SQL e principali gap tra i due sistemi.
## Sintesi fattuale
1. In ThothAI le foreign key dichiarate nel database venivano importate e una procedura separata proponeva relationship basate sul nome dei campi e su una verifica dei valori. Entrambi i percorsi scrivevano però nello stesso modello `Relationship` (`Thoth/ThothAI/backend/thoth_core/dbmanagement.py:484-640`; `Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:571-826`; `Thoth/ThothAI/backend/thoth_core/models.py:488-530`).
2. Il ricordo di una relationship inferita persistita come `generated` non trova riscontro nel modello esaminato: `Relationship` contiene solo quattro foreign key verso tabelle e colonne, senza provenienza, stato, confidenza o flag `generated`. La procedura di inferenza calcola localmente pattern e tasso di validazione, ma non li salva (`Thoth/ThothAI/backend/thoth_core/models.py:488-530`; `Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-801`).
3. L'amministratore Django di ThothAI permetteva CRUD manuale sul medesimo insieme di relationship e verificava che gli endpoint appartenessero allo stesso database e alle tabelle selezionate; non distingueva visivamente o semanticamente relationship fisiche, inferite e manuali (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:19-313`).
4. ThothII oggi separa già i due concetti: il catalogo backend conserva solo le foreign key fisiche dichiarate, mentre il core/harness possiede annotazioni di foreign key logiche curate e un comando che suggerisce candidate per nome, primary key e SQL osservato (`CONTEXT.md:291-300`; `docs/adr/0006-separate-physical-and-logical-relationships.md:3-8`; `harness/tht/cli/schema_cmd.py:203-299`).
5. I due mondi ThothII non sono ancora integrati: i record di Database Management non modificano il workflow NL→SQL e l'integrazione catalogo→core/schema-linking è ancora un gate di design esplicitamente differito (`PROJECT_STATE.md:116-124`; `PROJECT_STATE.md:161-165`).
## Evidenze ThothAI
### Foreign key ufficiali
Il percorso di import legge le foreign key dal database, limita l'import alle tabelle già presenti nel catalogo, crea le colonne mancanti, crea o riusa un record `Relationship` e aggiorna anche le stringhe denormalizzate `pk_field`/`fk_field` sulle colonne (`Thoth/ThothAI/backend/thoth_core/dbmanagement.py:484-640`, in particolare `:501-513`, `:526-591` e `:606-607`).
**Fatto:** una foreign key fisica diventa quindi un record applicativo, non rimane soltanto un fatto letto al momento dal database.
### Relationship inferite
La routine legacy dichiara sei famiglie di confronto tra il nome della colonna candidata e quello della primary key: corrispondenza esatta, snake case, kebab case, camel case, concatenazione e solo nome tabella (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:571-689`).
Per una candidata, la routine:
- legge fino a 20 valori distinti e non nulli dalla colonna candidata;
- verifica ogni valore contro la primary key bersaglio;
- accetta la candidata quando almeno il 70% dei valori esaminati trova riscontro;
- crea o recupera un normale `Relationship` e aggiorna i campi denormalizzati delle tabelle (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-801`).
**Correzione documentale:** un commento parla di campionamento casuale, ma la query mostrata non contiene un ordinamento casuale; il comportamento verificabile dal codice è “fino a 20 valori distinti e non nulli”, non un campione statisticamente casuale (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-716`).
**Rischio legacy:** nomi di tabelle e colonne sono interpolati direttamente in SQL in questo percorso. Portare la logica letteralmente in ThothII riprodurrebbe un problema di quoting/sicurezza e richiederebbe inoltre una policy esplicita per l'accesso ai valori del DWH (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:709-750`).
### Un solo modello per tre origini
Il modello `Relationship` legacy contiene soltanto `source_table`, `target_table`, `source_column` e `target_column`, più metodi di rappresentazione/aggiornamento. Non contiene campi per origine, algoritmo, evidenza, confidenza, approvazione o disabilitazione, né un vincolo di unicità dichiarato nel modello (`Thoth/ThothAI/backend/thoth_core/models.py:488-530`).
L'admin consente aggiunta, modifica e cancellazione ordinarie e valida la coerenza tra database, tabelle e colonne (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:30-38`, `:125-157`, `:218-245`).
**Conclusione fattuale:** relationship importate, inferite e create manualmente convergono nello stesso tipo persistito. Dal record finale non è possibile ricostruirne con certezza l'origine. Il termine `generated`, se usato nell'interfaccia o nel linguaggio operativo dell'epoca, non era una qualificazione persistita dal modello esaminato.
### Uso nella comprensione dello schema
La generazione M-Schema conserva colonne PK/FK e ricostruisce la sezione `【Foreign keys】` analizzando le stringhe denormalizzate `fk_field` (`Thoth/ThothAI/frontend/sql_generator/helpers/main_helpers/main_generate_mschema.py:25-61`, `:94-101`, `:160-187`).
**Fatto:** le relationship curate nel catalogo legacy influenzavano la rappresentazione dello schema consumata dal generatore SQL, anche se tramite una proiezione denormalizzata.
## Stato attuale di ThothII
### Catalogo backend: relationship fisiche
Il modello di dominio corrente distingue esplicitamente:
- **Physical Relationship:** vincolo dichiarato nel database;
- **Catalog Relationship:** copia persistita di quel fatto fisico, non creabile o modificabile manualmente ma eliminabile per cleanup;
- **Logical Relationship:** relazione semantica curata o inferita, con ownership e lifecycle separati (`CONTEXT.md:291-300`; `docs/adr/0006-separate-physical-and-logical-relationships.md:3-8`).
La migrazione del catalogo crea `catalog_relationships` e le coppie ordinate in `catalog_relationship_columns`. L'identità della relazione fisica è `(source_table_id, constraint_name)`; non esistono campi di origine, stato o confidenza (`backend/src/catalog/migrations/003_catalog_schema_sync.ts:42-74`). Il tipo TypeScript è solo strutturale e le descrizioni generate riguardano esclusivamente tabelle o colonne (`backend/src/catalog/types.ts:123-152`).
L'introspezione PostgreSQL legge soltanto `pg_constraint` con `contype = 'f'` e mantiene l'ordine delle coppie per le chiavi composite (`backend/src/catalog/schema-introspector.ts:110-171`; `backend/src/catalog/schema-introspector.ts:360-416`; `docs/contracts/catalog-schema-snapshot.md:40-53`). La sincronizzazione autoritativa crea, aggiorna o elimina le copie fisiche in base allo snapshot (`backend/src/catalog/repository.ts:1165-1218`; `backend/src/catalog/repository.ts:1316-1393`).
Le API e la UI espongono lettura, sync e cleanup in massa, non CRUD logico per singola relationship (`backend/src/routes/catalog-schema.ts:25-37`; `backend/src/routes/catalog-schema.ts:134-150`; `frontend/src/api/catalog-databases.ts:452-489`; `frontend/src/shell/database-management/DatabaseRelationships.tsx:114-123`; `frontend/src/shell/database-management/DatabaseRelationships.tsx:160-225`).
**Effetto rilevante:** il cleanup è intenzionalmente reversibile tramite una sync successiva e può lasciare temporaneamente un catalogo incompleto (`docs/adr/0008-allow-manual-catalog-metadata-cleanup.md:7-16`). Un test di integrazione ammette anche una relationship rimasta senza coppie di colonne dopo la cancellazione delle colonne catalogate (`backend/test/catalog-repository.integration.test.ts:194-264`, in particolare `:246-252`). Un futuro consumer non può quindi assumere che ogni stato intermedio del catalogo sia pubblicabile così com'è.
### Core/harness: relationship logiche e suggerimenti
Il modello M-Schema del core possiede già `TableAnnotation.foreign_keys`, descritte come foreign key logiche curate e unite alle foreign key fisiche (`harness/tht/mschema/models.py:35-39`; `harness/tht/mschema/models.py:76-84`). Il renderer fonde i due insiemi e li presenta insieme nella sezione `【Foreign keys】` (`harness/tht/mschema/render.py:9-20`; `harness/tht/mschema/render.py:46-80`).
Il comando `schema suggest-fks` costruisce candidate da:
- uguaglianze trovate in SQL, quando esattamente un lato è una primary key;
- convenzione speciale `*time_key → dim_time.<PK singola>`;
- stesso nome tra colonna e primary key a proprietario univoco;
- assunzioni esplicite per disambiguare;
- esclusione di nomi PK generici come `id`, `key` e `code` e dei proprietari ambigui (`harness/tht/cli/schema_cmd.py:203-299`; `harness/tht/mschema/fkmine.py:1-58`).
Il comando può produrre un documento candidato e, con `--write`, aggiungere annotazioni mancanti in modo idempotente; il messaggio stesso chiede revisione manuale (`harness/tht/cli/schema_cmd.py:406-429`; `harness/tht/cli/schema_cmd.py:519-540`; `harness/tests/test_schema_fk_annotations.py:131-146`; `harness/tests/test_schema_fk_annotations.py:454-470`).
**Fatto:** ThothII non parte da zero sull'inferenza. Possiede già un motore più conservativo, basato su PK univoche e SQL osservato, ma non conserva per ogni relazione origine, evidenza, frequenza o confidenza. Il miner conta le occorrenze internamente, ma il modello candidato non promuove quel conteggio a lifecycle persistito (`harness/tht/mschema/fkmine.py:1-58`; `harness/tht/mschema/models.py:35-39`).
**Rischi strutturali già visibili:**
- `ForeignKey` ammette liste di colonne, ma non valida che source e target abbiano la stessa cardinalità; il renderer usa `zip`, quindi una relazione malformata può essere troncata silenziosamente (`harness/tht/mschema/models.py:35-39`; `harness/tht/mschema/render.py:76-78`).
- il merge evita duplicati rispetto alle FK fisiche iniziali, ma non aggiorna l'insieme `seen` dopo aver aggiunto un'annotazione; due annotazioni logiche uguali possono sopravvivere al merge (`harness/tht/mschema/render.py:9-20`).
- il catalogo fisico supporta coppie composite ordinate, mentre le euristiche correnti e legacy sono sostanzialmente unary. La semantica delle candidate composite resta da decidere (`backend/src/catalog/migrations/003_catalog_schema_sync.ts:56-74`; `harness/tht/cli/schema_cmd.py:203-299`).
### Revisione e pubblicazione correnti
Le annotazioni canoniche del workspace sono un blob Git. Il flusso pubblico produce candidate, verifica le annotazioni e richiede un'accettazione umana esplicita dopo commit/push/pull; l'accettazione registra digest del candidato e delle annotazioni, revisione e blob (`docs/contracts/workspace-preprocessing-cli.md:89-106`; `backend/src/workspaces/preprocessing-service.ts:202-297`). Il preprocessing successivo procede solo se il digest accettato coincide con le annotazioni correnti (`backend/src/workspaces/preprocessing-service.ts:335-388`).
**Limite fattuale:** il record di review conserva digest, revisione e blob, ma non attore, motivazione o decisioni per singola candidata (`backend/src/workspaces/preprocessing-state.ts:67-73`).
Il runtime usa le annotazioni quando renderizza lo schema, ma la ricerca vettoriale indicizza record di tabelle e colonne senza contenuto esplicito delle relationship (`harness/tht/vectorstore/records.py:106-138`; `harness/tht/cli/search_cmd.py:233-273`). Inoltre la vista colonne usata in F4 continua a leggere i commenti fisici, non le annotazioni (`harness/tht/cli/schema_cmd.py:592-621`).
La review dei join durante una sessione è distinta dalla curatela globale: il reviewer conferma l'insieme dei join oppure chiede una revisione completa; non modifica la mappa canonica delle relationship (`harness/.pi/skills/tht-sessione/SKILL.md:301-341`; `frontend/src/widgets/JoinReviewWidget.tsx:5-84`). Il modello di sessione registra join come due stringhe e una decisione opzionale, senza ID stabile della relationship o coppie ordinate strutturate (`harness/tht/session/models.py:147-170`).
## Delta e rischi da sottoporre al grill
| Tema | Fatto documentato | Delta/rischio aperto |
|---|---|---|
| Origine | ThothAI perdeva l'origine; ThothII separa fisico e logico a livello concettuale | Decidere quale provenienza debba essere persistita per manuale, euristica, SQL osservato e import fisico |
| `generated` | Non era un flag del modello ThothAI; in ThothII “Generated Description” è già un termine del catalogo (`CONTEXT.md:302-311`) | Usare `generated` anche per relationship potrebbe creare ambiguità terminologica |
| Cancellazione utente | In ThothAI era CRUD sul record unico; in ThothII il cleanup fisico viene ricostruito dalla sync | “Eliminare” può significare cancellare una relazione logica, sopprimere un fatto fisico per il core oppure pulire temporaneamente la copia catalogata: sono operazioni diverse |
| Inferenza | ThothAI usava sei pattern e valori DWH; ThothII usa PK univoche, nomi e SQL osservato | Stabilire se sostituire, integrare o non portare il campionamento dei valori; servono policy di dati sensibili, query read-only, quoting e limiti |
| Approvazione | ThothII dispone di review Git/digest dell'intero artefatto | Mancano decisioni per candidata, motivazione, attore, sticky rejection, versione algoritmo e gestione dello stale |
| Compositi | Il catalogo fisico conserva coppie ordinate; le annotazioni accettano liste | Mancano invarianti forti e una strategia di inferenza/review per join compositi |
| Pubblicazione | Catalogo management e runtime core sono oggi separati | Va stabilito se pubblicare tutto, una selezione esplicita o una revisione immutabile; il catalogo può essere incompleto durante cleanup/sync |
| UI e ownership | UI catalogo fisico read-only; curatela logica via CLI/Git; review join per sessione separata | Va scelto chi cura la mappa e in quale superficie, senza confondere amministrazione globale e correzione della singola sessione |
| Retrieval | Le relationship entrano nel render M-Schema ma non nei record vettoriali | Va deciso se e come influenzano selezione tabelle, ranking e descrizioni, oltre al rendering finale |
| Drift | Sync fisica è autoritativa; annotazioni sono una revisione separata | Servono semantiche per endpoint rinominati/eliminati, candidate stale, orphan e riapprovazione |
Le analisi già presenti nel repository trattano l'integrazione catalogo→core, la selezione pubblicabile e la riparazione degli indici come questioni ancora aperte; le loro proposte non sono decisioni implementate (`docs/research/2026-08-23-thothii-metadata-publication-qdrant-seams.md:20-43`; `docs/research/2026-08-23-thothii-metadata-publication-qdrant-seams.md:239-301`). Anche il piano di migrazione rinvia il lifecycle/admin delle relationship logiche (`docs/plans/2026-08-26-metadata-catalog-from-thothai.md:681-691`).
## Direzione confermata nel grill
Il 31 agosto 2026 è stato concordato il seguente flusso minimo:
1. Le foreign key dichiarate continuano a essere sincronizzate come Catalog Relationship fisiche.
2. Le foreign key ipotetiche sono salvate una sola volta come Logical Relationship fra colonna
sorgente e colonna destinazione; le tabelle sono ricavate dalle colonne e la UI le presenta nel
relativo contesto tabella.
3. Una relationship inferita è marcata `generated`; una relationship aggiunta dall'utente non lo è.
4. La cancellazione logica conserva il record e impedisce a una ricostruzione di riattivarlo.
5. La cancellazione fisica rimuove il record; una ricostruzione può ricrearlo se viene nuovamente
inferito.
6. La ricostruzione è additiva: conserva le relationship attive già presenti, non rimuove quelle
non più inferibili e non modifica le relationship manuali.
7. L'inferenza usa nomi, primary key e compatibilità dei tipi. Non campiona valori del DWH.
8. La relationship è l'unica fonte di verità: non viene duplicata in stringhe `fk_field` sulle
colonne.
9. I nomi vengono confrontati senza distinzione fra maiuscole/minuscole e normalizzando snake case,
kebab case e camel case. La regola riconosce anche casi come `user_id → users.id`, richiede tipi
compatibili e una sola destinazione possibile; riconosce inoltre un nome PK non generico con un
unico proprietario e la convenzione `*time_key → dim_time.<PK singola>`. Le colonne sorgenti
possono appartenere a PK composite, mentre nomi generici isolati come `id`, `key`, `code` e `pk`
non costituiscono evidenza. I casi ambigui vengono ignorati e non si usa fuzzy matching o un LLM.
10. La ricostruzione parte da un'azione amministrativa esplicita `Rebuild generated relationships`,
separata dalla sincronizzazione dello schema.
11. Le relationship cancellate logicamente restano consultabili tramite filtro e possono essere
riattivate con un'azione `Restore`.
Non restano decisioni di dominio aperte per il flusso minimo. La progettazione UX e il seam tecnico
sono descritti nelle sezioni seguenti.
## Lacuna UX emersa nel grill
La vista Fleet `Relationships` esiste, ma nella UI di produzione non ha oggi un punto di ingresso
raggiungibile. La riga del database espone sincronizzazione, tabelle, dettagli, modifica e rimozione,
ma non le relationship; inoltre i tab `Overview / Tables / Relationships` appartengono soltanto alla
presentazione legacy. Il test di navigazione esistente esercita anch'esso la presentazione legacy,
non quella Fleet (`frontend/src/shell/database-management/DatabaseGrid.tsx:122-180`;
`frontend/src/shell/database-management/DatabaseForm.tsx:442-475`;
`frontend/src/shell/database-management/DatabaseForm.tsx:517-546`;
`frontend/src/shell/DatabaseManagementPage.test.tsx:2635-2703`).
Se aperta programmaticamente, la vista mostra soltanto `Physical relationships` in sola lettura. La
toolbar contiene ricerca, conteggio, `Refresh`, un selettore azione con esecuzione esplicita e
`Sync history`; la griglia offre unicamente `Details`, che apre il drawer della relationship fisica.
Sono assenti ingresso visibile, aggiunta manuale, ricostruzione delle generated relationship, origine,
stato attivo/cancellato, cancellazione logica, cancellazione fisica e ripristino
(`frontend/src/shell/database-management/DatabaseRelationships.tsx:114-225`).
I pattern Fleet già consolidati da riutilizzare sono:
- una sola griglia nel livello corrente, con breadcrumb e controllo Back;
- azioni di pagina nel selettore `Choose an action…` con `Run action` e motivo visibile quando
indisponibili;
- azioni della singola riga nella colonna finale fissata a destra;
- form e dettagli in un drawer modeless che restituisce il focus al controllo di origine;
- conferme distruttive inline o nel drawer, non tramite una nuova pagina;
- toast per accettazione o errore e feedback persistente soltanto per le operazioni lunghe;
- card di errore con Retry ed empty state che indica la prossima azione possibile.
ThothAI non offre un modello UX da copiare. L'inferenza è nascosta fra 21 bulk action della lista
database, non mostra avanzamento in tempo reale e restituisce soltanto messaggi a fine richiesta. Il
CRUD manuale vive in un'altra schermata Django Admin, non distingue origine o stato, offre soltanto
la cancellazione fisica e il form di aggiunta ha una validazione server strutturalmente incoerente
con le select popolate dal browser
(`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:252-274`;
`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:58-119`;
`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:218-313`).
## UX confermata
Il 31 agosto 2026 sono state confermate le seguenti scelte:
1. La colonna Actions della riga database espone un accesso diretto `Relationships`, accanto a
`Tables`. La vista conserva breadcrumb e controllo `Back to databases` esistenti.
2. La pagina presenta una sola griglia `Relationship map`, contenente relationship `Physical`,
`Generated` e `Manual`. Le colonne sono Source table, Source column, Target table, Target column,
Origin, Status e Actions. Constraint e regole update/delete rimangono nel drawer delle FK fisiche.
3. Un filtro visibile seleziona `Active`, `Excluded` o `All`; il valore predefinito è `Active`. Le FK
fisiche sono consultabili ma non modificabili da questa vista.
4. `Add relationship` è il pulsante primario visibile nella toolbar. Apre il drawer standard con i
quattro campi Source table, Source column, Target table e Target column e i comandi `Cancel` e
`Add relationship`. Il flusso minimo gestisce una sola coppia di colonne e mostra la validazione
accanto al campo interessato.
5. `Rebuild generated relationships` entra nell'attuale selettore `Choose an action…`, insieme a
`Synchronize physical relationships` e `Synchronize full schema`, con esecuzione esplicita tramite
`Run action`. `Refresh` ricarica la griglia; `Sync history` resta riservato alla sincronizzazione
fisica.
6. Il drawer di dettaglio contiene le azioni sulle relationship logiche. `Exclude` realizza la
cancellazione logica, `Delete permanently` quella fisica e `Restore` riattiva una relationship
esclusa. La conferma avviene nel drawer e spiega rispettivamente che la ricostruzione non
riattiverà un record escluso e potrà invece ricreare un record eliminato definitivamente.
7. La ricostruzione non introduce un nuovo sistema di job o di storico. Durante l'esecuzione mostra
`Rebuilding…`; al termine un toast riporta added, already present, excluded e ambiguous. Add,
Exclude, Delete permanently e Restore producono toast specifici; gli errori mantengono aperto il
contesto corrente. L'empty state propone di ricostruire dai nomi o aggiungere manualmente.
8. L'inferenza non usa AI. È codice deterministico nel backend del catalogo, basato su normalizzazione
dei nomi, primary key/unicità, compatibilità dei tipi e assenza di ambiguità. Non usa LLM,
embedding o campionamento dei dati. L'AI consuma la mappa risultante per comprendere lo schema,
ma non la costruisce.
## Seam tecnico confermato
Il Catalog PostgreSQL è l'unica fonte modificabile delle Logical Relationship. Le relationship
fisiche e logiche restano in modelli distinti, coerentemente con ADR-0006, mentre un servizio profondo
espone a API e UI una sola mappa discriminata per origine e stato.
All'avvio o alla ripresa di una sessione, il backend materializza le sole relationship attive in una
snapshot JSON canonica e immutabile, collegata alla stessa lease della configurazione runtime. La
snapshot comprende anche le FK fisiche e conserva l'ordine delle coppie composite. Se due record hanno
gli stessi endpoint, la FK fisica ha precedenza.
Quando la snapshot è presente, l'harness la usa come fonte esclusiva delle relationship e continua a
leggere dalle annotazioni Git-pinned soltanto descrizioni, sinonimi, concetti e altri metadati. Non
scrive `annotations.yaml` e non interroga direttamente il Catalog. Una snapshot dichiarata ma assente,
invalida o incoerente con lo schema fisico fallisce esplicitamente; un runtime legacy che non dichiara
la snapshot mantiene il precedente comportamento di compatibilità.
Questa proiezione non è un secondo store: non può essere modificata, viene eliminata insieme alla
configurazione runtime e una sessione Pi vede una mappa stabile per tutta la propria vita. La decisione
duratura è registrata in ADR-0012.
La proiezione e la ricostruzione sono ammesse soltanto dopo una sincronizzazione completa della
versione corrente del database e vengono serializzate con le mutazioni del catalogo. Un catalogo mai
sincronizzato, reso stale da una modifica della configurazione o invalidato da metadata cleanup non
può quindi diventare accidentalmente la fonte esclusiva del runtime. Il cleanup esplicito di una
tabella o colonna endpoint è anche il confine distruttivo del tombstone: rimuove definitivamente la
relationship esclusa, perché conservarla richiederebbe una seconda identità testuale denormalizzata.
@@ -0,0 +1,428 @@
# Piano di test: metadati di schema, campi sensibili e descrizioni AI
- Data: 2026-08-31
- Stato: proposto
- Baseline analizzata: `f586152` sul branch `test/database-baseline`
## 1. Obiettivo
Validare insieme le tre capacità recentemente introdotte in Database Management:
1. acquisizione autorevole dei metadati fisici di uno schema esterno;
2. proposta e revisione umana dei campi sensibili dal punto di vista privacy;
3. generazione, revisione e consolidamento delle descrizioni di tabelle e colonne.
Il piano è risk-based: perdita o corruzione di metadati, lettura o invio di valori protetti e
applicazione di una selezione al target sbagliato sono rischi bloccanti. La qualità linguistica dei
testi AI è invece valutata separatamente dal contratto tecnico, perché l'output del modello è una
proposta soggetta a revisione umana.
Questo documento non costituisce una certificazione normativa o GDPR: verifica i controlli tecnici
e il flusso operativo implementati dal prodotto.
### 1.1 Assunzione operativa: ambiente completamente sacrificabile
Per indicazione esplicita del proprietario, l'installazione sul Mac è esclusivamente di sviluppo e
test. Non contiene produzione e non esiste alcun requisito di conservazione dello stato locale.
- `catalog-db`, migrazioni, configurazioni locali, run, eventi, flag, descrizioni curate e generate
possono essere cancellati e ricreati tutte le volte necessarie;
- il database reale già collegato è la fonte autorevole dalla quale ricostruire il catalogo ed è la
sorgente primaria per validare schema, classificazione privacy e generazione delle descrizioni;
- reset completi, failure injection, dati incoerenti deliberati e prove distruttive sul catalogo
locale sono ammessi senza backup o procedura di rollback dell'ambiente;
- compatibilità con stato locale preesistente, sessioni legacy e vecchie revisioni del catalogo non
è un gate di questo piano;
- le prove di atomicità, idempotenza e preservazione dei campi restano necessarie perché verificano
il comportamento del prodotto dentro un ciclo di test, non perché debbano proteggere il Mac.
Il database sorgente resta normalmente read-only: quando una prova richiede DDL o mutazioni fra
scan e conferma si usa uno schema di test esplicitamente scrivibile o la fixture supplementare. La
disponibilità a buttare lo stato locale non elimina il rischio di inviare dati personali a un
provider esterno: proprio questa non-esfiltrazione è uno degli esiti principali da collaudare.
## 2. Terminologia e risultato atteso
I tre campi testuali del catalogo hanno autorità diverse e non devono essere confusi:
| Campo | Origine e autorità | Comportamento atteso |
| --- | --- | --- |
| `sourceComment` | Commento fisico acquisito dalla sorgente | Si aggiorna con la sincronizzazione e non è modificabile come contenuto curato. |
| `generatedDescription` | Proposta prodotta dall'AI o corretta nel catalogo | È salvata separatamente, può essere rigenerata esplicitamente e non sovrascrive gli altri due campi. |
| `description` | Descrizione curata dall'amministratore | Cambia solo tramite modifica esplicita o consolidamento di una proposta selezionata. |
Nel seguito, “commento generato” indica `generatedDescription`. La generazione non deve mai
modificare `sourceComment`; il passaggio a `description` avviene solo con il consolidamento umano.
## 3. Riferimenti e baseline
Il comportamento da verificare deriva da:
- stato corrente del progetto in `PROJECT_STATE.md`;
- [contratto Catalog Schema Snapshot](../contracts/catalog-schema-snapshot.md);
- [piano del Metadata Catalog](../plans/2026-08-26-metadata-catalog-from-thothai.md);
- [specifica della generazione descrizioni](../plans/2026-08-28-ai-catalog-description-generation-spec.md);
- [ADR 0007: sincronizzazione autorevole durevole](../adr/0007-durable-authoritative-schema-synchronization.md);
- [ADR 0009: un solo run sequenziale di generazione](../adr/0009-use-one-sequential-description-generation-run.md);
- [ADR 0010: campioni reali limitati](../adr/0010-allow-bounded-real-source-samples-for-description-generation.md);
- [ADR 0011: Sensitive Data Flag](../adr/0011-gate-source-samples-with-a-sensitive-data-flag.md);
- [accettazione AI del 2026-08-29](./2026-08-29-ai-catalog-description-generation-acceptance.md).
L'accettazione del 2026-08-29 è una baseline utile ma non chiude il gate attuale: precede i commit
`0736983` e `cb40c09`, inviava campioni reali inventati e documentava che un valore campione era
stato ripreso nella descrizione. Deve quindi essere ripetuta sul comportamento privacy corrente.
## 4. Perimetro
### Incluso
- trasporti `postgres_direct`, `rest_api` e `ssh_tunnel`;
- RPC REST tipizzato `POST /rpc/schema_snapshot` e fallback singolo read-only su
`POST /rpc/run_query`;
- metadati di tabelle, colonne, commenti sorgente, tipi, default, nullabilità, posizioni PK e coppie
FK ordinate;
- scope di sincronizzazione `tables`, `columns`, `relationships` e `all`;
- run durevoli, conferma delle differenze distruttive, cancellazione, recovery, eventi SSE e
fallback polling;
- Sensitive Data Flag, suggerimenti AI strutturali, review draft e storico operativo;
- scope di generazione `selected_columns`, `selected_tables`, `all` e `missing`;
- campionamento read-only, valori sintetici per colonne protette, batching, retry, stop, Unlock,
storico e consolidamento;
- uso controllato del database reale già collegato, inclusi schema e valori non sensibili, per il
collaudo end-to-end con fake provider e provider configurati;
- permessi, minimizzazione dei dati, redazione di log/errori e resistenza a input ostili.
### Escluso o rinviato
- applicazione del Sensitive Data Flag allo schema-linking/LSH del core, esplicitamente rinviata;
- sostituzione di `annotations.yaml`, pubblicazione Qdrant e cutover del runtime NL→SQL;
- alias, sinonimi, concetti, value descriptions e relazioni logiche;
- audit delle decisioni umane sul flag: per disegno si conserva solo il booleano corrente;
- DDL o scritture sul database sorgente;
- conservazione di cataloghi, run, descrizioni o sessioni locali precedenti al reset;
- compatibilità all'indietro con formati o migrazioni legacy non appartenenti alla baseline corrente.
## 5. Priorità e strategia
| Priorità | Significato | Esempi |
| --- | --- | --- |
| P0 | Gate bloccante di sicurezza o integrità | Nessuna lettura/invio di valori protetti; applicazione atomica; scope esatto; segreti non esposti. |
| P1 | Contratto funzionale necessario al rilascio | Trasporti, stati dei run, recovery, batching, review e consolidamento. |
| P2 | Qualità, UX e robustezza non distruttiva | Copy, focus, benchmark del modello, carico e degrado SSE. |
Le prove sono distribuite su quattro livelli:
1. **Contratto/unità**: repository in memoria, finti connector e Model Completer; nessuna rete.
2. **Integrazione**: catalogo PostgreSQL ricreabile via Docker, database reale come sorgente,
fixture supplementare, fake REST/SSH e Model Completer osservabile.
3. **UI/E2E locale**: component test con MSW e almeno un flusso Playwright sullo stack locale.
4. **Accettazione L2**: provider configurato realmente e database reale dopo review dei flag.
I test deterministici verificano il contratto. Le prove con un modello reale verificano
compatibilità e qualità, ma non sostituiscono i gate P0.
## 6. Ambiente e dati di test
### 6.1 Ambiente minimo
- stack locale con `catalog-db` eliminabile, migrazioni ripetibili e backend/frontend della stessa
baseline;
- database reale già collegato come sorgente primaria, con utenza capace di `SELECT` ma non di
`INSERT`, `UPDATE`, `DELETE`, DDL o cambio di schema;
- PostgreSQL effimero supplementare solo per le mutazioni controllate che non devono essere fatte
sul database reale;
- server REST fake in grado di servire snapshot valido, capability `unavailable`, 404, risposta
parziale/malformata e fallback `run_query`;
- server OpenSSH effimero con `known_hosts` esatto e varianti host key errata/assente;
- Model Completer fake che registra transitoriamente i messaggi e restituisce esiti programmabili;
- browser senza segreti in Web Storage e raccolta di log backend/SSE/API per le scansioni canary.
Le prove PostgreSQL di integrazione non devono risultare `skip`: la disponibilità di Docker è una
precondizione del gate. Il catalogo locale può essere azzerato prima di ogni wave senza snapshot o
backup del suo stato precedente.
### 6.2 Database reale e fixture supplementare `catalog_qa`
La prima baseline viene acquisita dal database reale collegato. Il test registra soltanto inventario
strutturale, conteggi e risultati sanitizzati; poi svuota il catalogo locale e dimostra di poterlo
ricostruire dalla stessa sorgente. Non è richiesto preservare alcun metadato locale precedente.
La fixture `catalog_qa` integra il database reale soltanto quando servono casi controllabili o
mutazioni che la sorgente reale non contiene. Deve includere almeno:
- `customers`: UUID PK, nome, email, codice fiscale, telefono, data di nascita, indirizzo e note;
- `orders`: FK verso `customers`, importo numerico, stato, timestamp, default e campi nullable;
- `order_lines`: PK composta e relazione composta ordinata;
- `clinical_events`: campi sanitari evidenti e tabella partizionata;
- `products`: SKU pubblico, categoria, prezzo e flag booleano;
- `empty_table`: nessuna riga ma struttura valida;
- `wide_entity`: almeno 23 colonne e metadati lunghi, per forzare batch `10 + 10 + 3` e il limite
dimensionale del messaggio;
- identificatori quotati, commenti Unicode/italiani, commenti null e oggetti fuori dallo schema.
Usare valori canary inventati e univoci, per esempio:
- `PRIV_EMAIL_CANARY_...`, `PRIV_TAX_CANARY_...`, `PRIV_HEALTH_CANARY_...` nelle colonne protette;
- `PUBLIC_SKU_CANARY_...` in una colonna esplicitamente non sensibile;
- `SECRET_API_CANARY_...` solo nel secret store del test.
I canary protetti non devono comparire nei messaggi al modello, nel catalogo, negli eventi, nelle
API, nel DOM o nei log. Il canary pubblico può apparire nel messaggio al provider entro i limiti
documentati e dopo la disclosure esplicita dell'utente. Sul database reale la stessa proprietà va
provata soprattutto osservando la proiezione SQL e il payload transitorio: i valori protetti non
devono essere letti, e nessun valore grezzo deve entrare nell'evidenza conservata.
### 6.3 Mutazioni della sorgente
Preparare tre revisioni dello schema:
- **A — iniziale**: struttura completa e commenti sorgente valorizzati;
- **B — distruttiva**: rimozione di una tabella, una colonna e una FK, più aggiunta di una nuova
colonna sensibile per nome;
- **C — race di conferma**: modifica ulteriore fra piano distruttivo e conferma, per provare il
re-scan.
Queste revisioni possono vivere nella fixture o in uno schema reale esplicitamente dichiarato
scrivibile. Fra una prova e l'altra è consentito eliminare completamente il catalogo locale,
riapplicare le migrazioni e ripartire dal database reale.
## 7. Casi di test — acquisizione dei metadati
| ID | P | Livello | Scenario | Risultato atteso |
| --- | --- | --- | --- | --- |
| MET-01 | P0 | API/Integrazione | Avvio senza binding raggiungibile, con versione database obsoleta o senza `database.manage`. | Il run non parte; risposta sicura e catalogo invariato. I segreti restano write-only. |
| MET-02 | P0 | Integrazione | Reset completo del catalogo e `all` sul database reale collegato, senza mock del client `pg_catalog`; ripetizione sulla fixture A per gli edge case assenti. | Il catalogo viene ricostruito da zero con snapshot esatta di tabelle, colonne, `sourceComment`, tipo, default, nullabilità, PK e FK ordinate; stato `succeeded`; nessuna scrittura alla sorgente. |
| MET-03 | P1 | Contratto/Integrazione | Stessa fixture via RPC REST tipizzato. | `schemaVersion: 1` e capability sono validate strettamente; risultato normalizzato uguale a MET-02. `unavailable` non è interpretato come collezione vuota. |
| MET-04 | P0 | Contratto/Integrazione | `/schema_snapshot` assente, fallback `run_query`; poi fallback assente, parziale, non JSON o con campi extra/mancanti. | Il fallback usa una sola query read-only. Ogni errore o snapshot invalida fallisce senza modifiche parziali; nessun fallback nasconde un errore operativo diverso da capability assente. |
| MET-05 | P1 | Integrazione | Accesso `ssh_tunnel` con host key corretta, errata e assente; errore durante apertura/chiusura. | Parità con MET-02 nel caso valido; fail-closed negli altri casi; processi, lease e file-segreto sempre rilasciati. |
| MET-06 | P0 | API | Esecuzione separata di `tables`, `columns`, `relationships` e `all`, con e senza selezione tabelle. | Ogni scope è autorevole solo nel proprio confine; nessun record fuori scope cambia o viene eliminato. Selezioni duplicate/inesistenti sono rifiutate atomicamente. |
| MET-07 | P0 | API/Integrazione | Ripetizione idempotente della fixture A dopo modifica di `description`, `generatedDescription` e `sensitive`. | Nessun diff fisico spurio; contenuti curati, proposte AI e flag delle entità ancora presenti sono preservati. Una nuova colonna nasce con `sensitive=false`. |
| MET-08 | P0 | API/Integrazione | Passaggio A→B, conferma assente/errata/scaduta e passaggio A→B→C prima della conferma. | Stato `awaiting_confirmation`, piano visibile e nessuna applicazione anticipata. La conferma valida provoca re-scan; se il diff cambia viene emesso un nuovo piano/token e il vecchio non applica nulla. Apply atomica oppure zero modifiche. |
| MET-09 | P0 | API/Integrazione | Timeout, disconnessione, capability incompleta o eccezione durante scan/apply. | Stato terminale coerente, errore sanitizzato, catalogo precedente intatto e lock rilasciato. Nessun segreto, SQL sensibile o stack trace nelle API/eventi. |
| MET-10 | P1 | Worker | Cancel in `queued`, `running`, `awaiting_confirmation` e `applying`; retry, restart con run attivo e lease scaduto. | Cancel è efficace solo prima di apply ed è rifiutato durante apply; retry crea un nuovo run. Recovery non duplica l'apply, marca correttamente i run interrotti, rilascia il lock e rimuove snapshot/diff/token interni non più necessari. |
| MET-11 | P0 | API | Due operazioni sullo stesso database: sync, cleanup, connection test, edit o generazione; in parallelo, operazioni su database diversi. | Una sola operazione possiede il database; conflitto 409 sicuro sullo stesso target. Nessun lock cross-database non previsto e nessuna release del token altrui. |
| MET-12 | P1 | UI | Avvio dai menu database/tabella, visualizzazione piano, conferma, history drawer, chiusura drawer, perdita SSE e polling. | Scope e selezione inviati sono esatti; azioni non eleggibili restano visibili con motivo; chiudere il drawer non ferma il run; replay/polling deduplicano gli eventi e aggiornano griglie/KPI. |
| MET-13 | P1 | E2E | Cleanup manuale di tabelle/colonne/relazioni e successiva sincronizzazione. | Cleanup modifica solo il catalogo; la sorgente resta invariata; un sync autorevole ripristina gli oggetti ancora presenti in sorgente. |
| MET-14 | P0 | Integrazione | Cambio binding/versione fra scan e apply ed errore iniettato a metà transazione. | La freshness viene ricontrollata sotto lock; il run fallisce senza righe parziali e conserva integralmente il catalogo precedente. |
| MET-15 | P1 | API/Worker | Replay SSE con `Last-Event-ID`/`after`, polling concorrente e retention oltre 30 giorni. | Cursori monotoni e nessun duplicato; gli eventi scaduti vengono potati senza corrompere run e stato finale. |
## 8. Casi di test — identificazione e protezione dei campi sensibili
| ID | P | Livello | Scenario | Risultato atteso |
| --- | --- | --- | --- | --- |
| PRV-01 | P0 | Repository/API | Prima sincronizzazione, re-sync e aggiunta di una colonna. | Il default è `false`; il valore umano delle colonne esistenti è preservato; la nuova colonna è esplicitamente da riesaminare ma non riceve uno stato audit inventato. |
| PRV-02 | P0 | API | Suggerimento per un database, tabelle selezionate e colonne selezionate; database multipli, target duplicati o mancanti. | Il provider riceve esattamente le colonne dello scope. Input ambigui sono rifiutati prima della chiamata e nessun flag cambia. |
| PRV-03 | P0 | Contratto | Ispezione del messaggio al classifier. | Sono presenti solo database, schema, tabella, colonna, tipo, nullabilità, PK e FK. Non compaiono righe, valori, commenti, descrizioni, flag corrente o segreti. |
| PRV-04 | P1 | Contratto | `wide_entity`, identificatori lunghi e limite byte. | Ordine deterministico, batch massimi di dieci colonne e rispetto del limite messaggio; una singola colonna non rappresentabile fallisce prima del provider con errore sicuro. |
| PRV-05 | P0 | API | Risposta valida, fenced/prosa, JSON malformato, target mancante/duplicato/ignoto e provider failure. | Ogni colonna richiesta compare una sola volta. Una classificazione invalida viene ritentata una volta; dopo esaurimento si ottiene errore sanitizzato e nessuna modifica. |
| PRV-06 | P0 | UI/API | Apertura draft, modifica manuale, chiusura/reload e salvataggio. | La proposta non è persistita prima di Save; reload la scarta. Il reviewer può invertire scelte; si salvano solo colonne cambiate con versione ottimistica; un conflitto richiede reload. |
| PRV-07 | P1 | Repository/UI | Tentativi completati, falliti e attivi al restart. | Ogni tentativo ha un run distinto con scope, modello, contatori ed eventi sanitizzati; startup marca `interrupted` i run attivi. Storico newest-first senza target ID, proposte, prompt, output grezzo o diagnostica provider. |
| PRV-08 | P0 | Integrazione | Generazione descrizioni su target con canary protetti. | Le colonne protette sono assenti dalla proiezione SQL, non semplicemente filtrate dopo la lettura. Se non rimangono colonne leggibili non viene eseguita una `SELECT`. Nessun canary protetto esce dal processo. |
| PRV-09 | P0 | Contratto/Integrazione | Tabella mista con colonne sensibili e pubbliche. | Per le sensibili il prompt contiene valori plausibili, deterministici e limitati derivati dai soli metadati, nello stesso formato dei campioni e senza etichettarli al modello come sintetici. Per le pubbliche: massimo cinque righe e cinque valori rappresentativi, valori troncati e transazione read-only chiusa con rollback. |
| PRV-10 | P1 | API | Cambio `false→true→false` dopo una descrizione già generata. | Il testo esistente non viene rigenerato retroattivamente. Solo le generazioni future cambiano fonte del contesto; tornando `false` il campionamento reale torna eleggibile. |
| PRV-11 | P0 | API/UI | Utente senza `database.manage`, modello non configurato, catalogo/provider indisponibile e richiesta interrotta. | Controlli nascosti/disabilitati in UI e rifiuto server-side; errori non espongono dettagli. Un tentativo fallito compare nello storico senza trasformarsi in audit della decisione umana. |
| PRV-12 | P1 | L2 | Corpus strutturale etichettato con identificatori personali, credenziali/token, salute, finanza, localizzazione e controlli non sensibili/ambigui, in inglese e italiano. | Si misurano precisione, recall e falsi negativi per modello. I campi critici mancati sono sottoposti al product owner; la soglia quantitativa va ratificata prima di diventare gate, perché il classifier è advisory e human-in-the-loop. |
| PRV-13 | P0 | UI/E2E | Modifica di un flag nella review senza Save e tentativo immediato di generare descrizioni. | Gate di rilascio da formalizzare: la generazione deve essere bloccata finché il draft non è salvato o scartato. In alternativa la UI deve dichiarare inequivocabilmente che verrà usato il valore persistito; non è accettabile mostrare “protetto” e campionare come non protetto. |
## 9. Casi di test — generazione e consolidamento dei commenti
| ID | P | Livello | Scenario | Risultato atteso |
| --- | --- | --- | --- | --- |
| GEN-01 | P0 | Config/API | Modelli validi, default, endpoint anonimo esplicito, secret ref mancante/errato e nessun modello. | Al browser arrivano solo ID, label e default. Nessuna chiave, provider payload o configurazione privata è esposta; feature disabilitata in modo comprensibile se non configurata. |
| GEN-02 | P0 | API | `selected_columns`, `selected_tables`, `all`, `missing`, target duplicati/inesistenti e zero eleggibili. | Scope esatto; `missing` include null/vuoto/whitespace e salta proposte esistenti; `all` sostituisce solo dopo conferma esplicita; input invalido non crea run. |
| GEN-03 | P1 | Worker | 23 colonne più tabelle, metadati e campioni lunghi. | Colonne prima delle tabelle per lo scope globale; richieste omogenee, sequenziali, massimo dieci target e sotto i limiti byte; contesto tabella aggiornato dopo le colonne. |
| GEN-04 | P0 | Contratto | JSON puro, un solo code fence JSON completo, prosa extra, payload multipli, target mancante/duplicato/ignoto, descrizione vuota o oltre limite. | Sono accettati solo i primi due formati validi. Una mappatura ambigua non applica alcun risultato del batch e conta come errore tecnico. |
| GEN-05 | P1 | API | Esito `generated` e `non_generatable` in workspace italiano/inglese. | Testo generato trimmato e salvato; il testo standard non generabile è localizzato dall'applicazione, non copiato dal provider. Gli errori tecnici non scrivono tale testo. |
| GEN-06 | P0 | Worker | Errore transiente, errore esaurito isolato, tre batch falliti consecutivi e successo fra due errori. | Helper con al massimo un retry e nessun fallback modello. Errori isolati portano a `completed_with_errors`; tre consecutivi a `failed`; un successo azzera il contatore. |
| GEN-07 | P0 | Integrazione | Successi seguiti da cancel, interruption o failure. | Ogni risultato valido è persistito subito e resta disponibile; il target fallito non riceve dati ambigui. “Generate Missing” consente il recupero naturale senza resume automatico. |
| GEN-08 | P0 | API | Secondo run durante un run attivo e modifica/sync/cleanup/consolidamento sul database posseduto. | Un solo run di generazione attivo nell'installazione; 409 chiaro al secondo Start. Il database target resta riservato e le operazioni incompatibili non alterano selezione o dati. |
| GEN-09 | P0 | Integrazione/Worker/UI | Stop in queued/running con helper attivo e con una `SELECT` sorgente deliberatamente bloccata/lenta. | Helper e query/connessione vengono terminati entro 5 secondi, stato `cancelled`, nessuna chiamata modello dopo Stop, risultati precedenti conservati ed eventi consultabili. Un test con sampler fake non è sufficiente. |
| GEN-10 | P0 | Worker/API | Restart con run `queued/running`; Unlock con worker/helper vivo e con run realmente stale; race Unlock/Start. | Startup marca `interrupted` senza replay. Unlock è rifiutato se esiste lavoro locale vivo e non può liberare la reservation di un nuovo run. |
| GEN-11 | P1 | Integrazione | Sorgente campioni indisponibile ma metadati validi. | La generazione prosegue metadata-only con un solo warning sicuro; nessun tentativo alternativo espone dettagli di connessione. |
| GEN-12 | P1 | API/UI | SSE disconnesso, replay da sequence, polling concorrente e riapertura storico. | Eventi persistiti, ordinati e deduplicati; history newest-first; contatori/stato finali coerenti; nessun prompt, campione, risposta completa o stack trace. |
| GEN-13 | P0 | UI/API | Revisione manuale della proposta e consolidamento selettivo di tabelle/colonne, inclusi target vuoti/stale. | Si copia solo `generatedDescription` non vuota dei target risolti; conteggio `copied/skipped` corretto; operazione atomica; `sourceComment` invariato. |
| GEN-14 | P0 | UI/Sicurezza | Tutti gli scope di generazione da database, tabelle e colonne selezionate, utente senza permesso, metadata contenente istruzioni ostili. | Prima di ogni Start, inclusi `selected_tables` e `selected_columns`, compare la disclosure “fino a cinque righe e cinque valori”. Il server applica comunque l'autorizzazione. Metadati e valori sono trattati come dati non fidati e l'output resta nel contratto JSON. |
| GEN-15 | P1 | L2 | Run reale sul modello di default e almeno un endpoint alternativo supportato, usando il database reale dopo la review dei flag e un role read-only. | Descrizioni nella lingua workspace, coerenti con struttura/commenti e senza fatti inventati critici; almeno una colonna e una tabella consolidate. Nessun valore protetto o segreto compare in request osservabile, eventi, API, persistenza o log. |
| GEN-16 | P1 | Integrazione | Fastify→worker→processo Python→LiteLLM→endpoint OpenAI-compatible locale di cattura. | Routing provider/model, header API key, `disableThinking`, singolo retry, timeout, limite output e payload sono corretti; chiave e diagnostica non risalgono a stdout, API o log applicativi. |
## 10. Percorso E2E prioritario
Il caso `E2E-01` deve attraversare le tre feature senza sostituire i test di contratto:
1. azzerare `catalog-db`, riapplicare le migrazioni e verificare che il role del database reale sia
realmente read-only;
2. eseguire `all` sul database reale e confrontare catalogo e snapshot sorgente; usare la fixture A
in una seconda esecuzione per gli edge case mancanti;
3. richiedere suggerimenti privacy sull'intero database;
4. modificare almeno una proposta e salvare i flag revisionati;
5. avviare `missing` dopo la disclosure, usando un Model Completer osservabile;
6. dimostrare che i canary protetti non sono letti né inviati e che il canary pubblico rispetta i
limiti;
7. correggere una `generatedDescription` e consolidare una tabella e una colonna;
8. applicare la fixture B, controllare il piano distruttivo e introdurre C prima della conferma;
9. confermare dopo il re-scan e verificare atomicità, preservazione di descrizioni/flag delle entità
superstiti e `sensitive=false` sulla nuova colonna;
10. perdere la connessione SSE, riaprire entrambi gli storici e verificare replay, polling e KPI.
Il percorso deve essere eseguito con fake provider dopo ogni reset rilevante e, in forma ridotta,
con il provider configurato sul database reale dopo che i flag sono stati revisionati e salvati.
Non è richiesto ripristinare lo stato locale precedente al test.
## 11. Valutazione qualitativa dei modelli
La qualità non deve confondersi con la sicurezza: anche un classifier perfetto non autorizza
l'esfiltrazione di un canary protetto e una descrizione elegante non rende valido un payload
ambiguo.
### 11.1 Classificazione privacy
Per ogni modello registrare matrice di confusione, precisione, recall e falsi negativi per categoria.
Eseguire almeno tre iterazioni sul corpus fisso per rilevare instabilità. Fino alla ratifica di una
soglia da parte del product owner, il risultato è un gate di review: ogni falso negativo su
credenziali/token, identificatori fiscali, dati sanitari o finanziari richiede accettazione esplicita
o correzione prima del rilascio operativo.
### 11.2 Descrizioni generate
Valutare ogni testo da 0 a 2 su:
- correttezza rispetto a struttura e commenti sorgente;
- specificità e utilità per un revisore;
- lingua e chiarezza;
- assenza di istruzioni seguite dai dati non fidati o fatti inventati;
- assenza di valori protetti e segreti.
Soglia proposta da ratificare: almeno 8/10, nessun punteggio 0 su correttezza o sicurezza. Un valore
esplicitamente non sensibile, reale o inventato, può essere ripreso entro il perimetro dichiarato;
un valore protetto non può mai esserlo.
## 12. Copertura esistente e gap da chiudere
| Area | Evidenza automatica già presente | Gap principale |
| --- | --- | --- |
| Snapshot e sincronizzazione | `backend/test/catalog-schema-introspector.test.ts`, `catalog-schema-routes.test.ts`, `catalog-table-introspector.test.ts`, `catalog-repository.integration.test.ts` | Introspezione `pg_catalog` realmente end-to-end, parità live dei tre trasporti e un unico E2E con re-scan distruttivo. |
| Privacy | `catalog-description-generation-routes.test.ts`, `catalog-description-generation-worker.test.ts`, `catalog-description-source-sampler.test.ts`, `catalog-synthetic-sample-value.test.ts` | Prova canary integrata query→prompt→API/log e benchmark reale post-ADR 0011. |
| Generazione | `catalog-description-generation-routes.test.ts`, `catalog-description-generation-worker.test.ts`, `catalog-description-generation.integration.test.ts` e test del helper | Accettazione reale aggiornata, cancellazione di una query PostgreSQL bloccata e integrazione ermetica fino all'endpoint LiteLLM locale. |
| UI | `DatabaseManagementPage.test.tsx`, `DescriptionGenerationDrawer.test.tsx`, `SensitiveDataSuggestionHistoryDrawer.test.tsx` | L'E2E Playwright corrente verifica soprattutto il layout, non il workflow funzionale. |
Nuovi asset consigliati:
- `backend/test/catalog-metadata-privacy-workflow.integration.test.ts`;
- `backend/test/fixtures/catalog-privacy-schema.sql` e snapshot REST equivalenti;
- integrazione a due PostgreSQL per le query reali `pg_catalog` e lo stop del sampler;
- endpoint OpenAI-compatible locale di cattura per GEN-16;
- `frontend/e2e/database-management-workflow.spec.ts`;
- corpus strutturale versionato per PRV-12, senza valori business;
- nuovo report di accettazione L2 che sostituisca il gate privacy del 2026-08-29.
## 13. Ordine di esecuzione
### Wave 1 — contratto rapido
- parser snapshot, introspector, scope e diff;
- classifier strutturale, batching e validazione output;
- sampler, valori sintetici, prompt bounds e parser descrizioni;
- autorizzazione, redazione e race del coordinator.
### Wave 2 — integrazione PostgreSQL
- reset totale di `catalog-db`, bootstrap delle migrazioni e ricostruzione dal database reale;
- migrazioni e vincoli del repository;
- atomicità sincronizzazione/consolidamento;
- run ed eventi persistiti, restart e cancellation;
- `E2E-01` con fake provider e canary.
### Wave 3 — frontend e stack locale
- component test MSW;
- Playwright funzionale, perdita SSE e polling;
- verifica disclosure, review draft, history e consolidamento.
### Wave 4 — accettazione L2
- provider reale sul database collegato dopo classificazione e review dei flag;
- benchmark PRV-12 e rubric GEN-15;
- scansione finale di canary e segreti;
- approvazione del product owner.
Comandi di regressione:
```bash
cd backend && npx vitest run
cd backend && npx tsc --noEmit -p .
cd backend && npm run build
cd frontend && npx vitest run
cd frontend && npx tsc -b
cd frontend && npm run build
cd frontend && npm run e2e
./scripts/build-docs.sh
```
Il report deve evidenziare esplicitamente eventuali test Docker/L2 saltati; un `skip` non equivale a
PASS del relativo gate.
## 14. Evidenze da conservare
Per ogni esecuzione registrare:
- commit, configurazione pubblica dei modelli, versione fixture e trasporto;
- ID e stato finale dei run, contatori ed eventi sanitizzati;
- snapshot catalogo prima/dopo e piano distruttivo con metadati e conteggi sanitizzati, senza valori
grezzi delle righe sorgente;
- report test/JUnit, screenshot dei gate UI e risultato della scansione canary;
- matrice di confusione privacy e rubric delle descrizioni per le prove L2;
- difetti con ID del caso, severità, riproducibilità e decisione finale.
Non allegare prompt completi, righe campione, output grezzi del provider, chiavi, digest o frammenti
di segreti. Il Model Completer spy deve verificare in memoria le asserzioni e scartare il payload al
termine del test.
Non serve conservare backup del catalogo locale, run precedenti o descrizioni generate durante una
wave: l'evidenza è il report sanitizzato e la capacità di ricostruire nuovamente il risultato dalla
sorgente reale.
## 15. Criteri di ingresso e uscita
### Ingresso
- baseline unica per backend/frontend e procedura verificata per eliminare e ricreare `catalog-db`;
- accesso al database reale collegato e inventario delle tabelle/colonne da includere nella review;
- fixture e canary supplementari approvati per i soli edge case controllati;
- role sorgente read-only verificato con una scrittura deliberatamente negata;
- fake connector/provider disponibili e log collection attiva;
- per L2, secret reference configurato senza materializzare il valore nell'evidenza.
### Uscita
- 100% dei casi P0 e P1 applicabili superati; nessun difetto Sev-1/Sev-2 aperto;
- almeno due ricostruzioni complete e coerenti del catalogo a partire dal database reale dopo reset
indipendenti;
- zero comparsa dei canary protetti e dei segreti fuori dalla sorgente/secret store del test;
- nessuna modifica parziale dopo errori, cancel o conferme stale;
- parità normalizzata dei trasporti supportati e nessuna integration PostgreSQL richiesta saltata;
- stati, contatori, eventi e history coerenti dopo success, partial failure, stop e restart;
- `sourceComment`, `generatedDescription` e `description` mantengono l'autorità prevista;
- accettazione L2 sul database collegato approvata dal product owner e
build/test/typecheck/documentazione verdi;
- ogni deviazione P2 o soglia qualitativa non ancora ratificata è documentata con owner e data.
## 16. Rischi residui da rendere espliciti
- `sensitive=false` è il default e il prodotto non conserva uno stato “review completata”: dopo ogni
reset, classificazione strutturale e salvataggio umano dei flag devono precedere qualunque run con
provider reale. Il catalogo è ricostruibile; un invio errato a un provider non lo è.
- Il flag protegge i valori campionati. Nomi, `sourceComment` e descrizioni sono metadati inviabili
al modello e possono contenere testo libero: se nel database reale includono PII serve una
decisione aggiuntiva di redazione, non una diversa aspettativa di test.
- La UI deve risolvere il caso di un flag modificato ma non salvato prima della generazione e deve
mostrare la disclosure anche per tabelle/colonne selezionate; il piano considera entrambi P0.
- L'AbortSignal corrente va provato contro una query PostgreSQL realmente bloccata: la sola
cancellazione del helper non dimostra che la lettura sorgente sia interrompibile.
- Lo storico dei Sensitive Data Suggestion Run è operativo, non un audit delle decisioni umane.
- La policy privacy non è ancora applicata allo schema-linking/LSH; nessun risultato di questo piano
deve essere presentato come copertura di quel percorso.
- Un provider reale resta non deterministico: il rilascio deve dipendere dai gate tecnici e dalla
review umana, non dalla ripetizione byte-identica delle descrizioni.
- L'esclusione fra operazioni e Unlock è in parte locale al processo. Se il deployment ammetterà più
repliche backend, servirà un gate aggiuntivo con due istanze contro lo stesso catalogo; non va
dedotta sicurezza multi-replica dai test single-process.
File diff suppressed because it is too large Load Diff
+239 -44
View File
@@ -1,4 +1,4 @@
import { expect, test, type Page, type Route } from "@playwright/test";
import { expect, test, type Locator, type Page, type Route } from "@playwright/test";
import { createAuthenticationStack } from "./fixtures/auth-stack.mjs";
test.describe.configure({ mode: "serial" });
@@ -12,6 +12,13 @@ const database = {
workspaceId: "psd-clinical",
workspaceName: "Policlinico San Donato",
workspaceAvailable: true,
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
runtimeBinding: {
transport: "postgres_direct",
configurationState: "ready",
sessionTransportSupported: true,
},
configured: true,
engine: "postgres",
databaseName: "warehouse",
@@ -37,6 +44,22 @@ const database = {
},
};
const unconfiguredDatabase = {
...database,
id: undefined,
workspaceId: "research-lab",
workspaceName: "Research laboratory",
configured: false,
databaseName: "research",
schema: "analytics",
version: 0,
createdAt: "",
updatedAt: "",
binding: { transport: "postgres_direct", port: 5432 },
connectionStatus: "untested",
testedVersion: undefined,
};
const table = {
id: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
databaseId,
@@ -118,7 +141,112 @@ async function expectContextPanelGeometry(page: Page, accessibleName: string) {
expect(panelBox.y + panelBox.height).toBeLessThanOrEqual(managerBox.y + managerBox.height + 1);
}
async function expectContextPanelHeaderMatchesNewSession(
async function expectCurrentNavigation(page: Page, accessibleName: string) {
const navigation = page.getByRole("complementary", { name: "Session navigation" });
const button = navigation.getByRole("button", { name: accessibleName, exact: true });
const available = navigation.locator('[data-navigation-state="available"]').first();
await expect(navigation.locator('[data-navigation-state="current"]')).toHaveCount(1);
await expect(button).toHaveAttribute("data-navigation-state", "current");
await expect(button).toHaveAttribute("aria-current", "page");
expect(await button.getAttribute("class")).toContain("bg-[oklch(var(--nav-active))]");
const [style, availableStyle] = await Promise.all([
button.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
borderColor: getComputedStyle(element).borderTopColor,
})),
available.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
borderColor: getComputedStyle(element).borderTopColor,
})),
]);
expect(style.backgroundColor).not.toBe(availableStyle.backgroundColor);
expect(style.borderColor).not.toBe(availableStyle.borderColor);
const tokens = await page.evaluate(() => {
const root = getComputedStyle(document.documentElement);
const read = (name: string) => root.getPropertyValue(name).trim().split(/\s+/).map(Number);
return { primary: read("--primary"), active: read("--nav-active") };
});
expect(tokens.active[2]).toBeCloseTo(tokens.primary[2], 1);
expect(tokens.active[1]).toBeLessThan(tokens.primary[1]);
expect(tokens.active[0]).toBeGreaterThan(tokens.primary[0]);
}
async function expectSelectedSessionScopeTab(page: Page, accessibleName: string) {
const tablist = page.getByRole("tablist", { name: "Session scope" });
const selected = tablist.getByRole("tab", { name: accessibleName, exact: true });
const inactive = tablist.locator('[role="tab"][aria-selected="false"]');
await expect(tablist.locator('[role="tab"][aria-selected="true"]')).toHaveCount(1);
await expect(selected).toHaveAttribute("data-tab-state", "active");
expect(await selected.getAttribute("class")).toContain("bg-[oklch(var(--nav-active))]");
const [selectedStyle, inactiveStyle] = await Promise.all([
selected.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
borderColor: getComputedStyle(element).borderBottomColor,
})),
inactive.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
borderColor: getComputedStyle(element).borderBottomColor,
borderWidth: getComputedStyle(element).borderTopWidth,
})),
]);
expect(selectedStyle.backgroundColor).not.toBe(inactiveStyle.backgroundColor);
expect(selectedStyle.borderColor).not.toBe(inactiveStyle.borderColor);
expect(inactiveStyle.borderWidth).toBe("1px");
expect(inactiveStyle.backgroundColor).not.toMatch(/^(?:transparent|rgba\(0, 0, 0, 0\))$/);
expect(inactiveStyle.borderColor).not.toMatch(/^(?:transparent|rgba\(0, 0, 0, 0\))$/);
}
function cssLightness(color: string): number {
const oklab = color.match(/^okl(?:ab|ch)\(([\d.]+)(%)?/);
if (oklab) {
const value = Number(oklab[1]);
return oklab[2] ? value / 100 : value;
}
const channels = color.match(/[\d.]+/g)?.slice(0, 3).map(Number) ?? [];
if (channels.length !== 3) return Number.NaN;
return color.startsWith("color(srgb") ? Math.max(...channels) : Math.max(...channels) / 255;
}
async function expectFleetRowTooltipBelow(trigger: Locator, expectedContent: string) {
await trigger.hover();
await expect.poll(() => trigger.evaluate((element) => {
const owner = element.closest<HTMLElement>("[data-tooltip]");
return owner ? getComputedStyle(owner, "::after").visibility : "missing";
})).toBe("visible");
await expect.poll(() => trigger.evaluate((element) => {
const owner = element.closest<HTMLElement>("[data-tooltip]");
return owner ? getComputedStyle(owner, "::after").opacity : "missing";
})).toBe("1");
const state = await trigger.evaluate((element) => {
const owner = element.closest<HTMLElement>("[data-tooltip]");
if (!owner) throw new Error("Row action tooltip owner is missing");
const style = getComputedStyle(owner, "::after");
return {
content: style.content.replace(/^['\"]|['\"]$/g, ""),
ownerHeight: owner.getBoundingClientRect().height,
top: Number.parseFloat(style.top),
right: Number.parseFloat(style.right),
opacity: style.opacity,
pointerEvents: style.pointerEvents,
backgroundColor: style.backgroundColor,
color: style.color,
};
});
expect(state.content).toBe(expectedContent);
expect(state.top - state.ownerHeight).toBeCloseTo(3, 1);
expect(state.right).toBe(0);
expect(state.opacity).toBe("1");
expect(state.pointerEvents).toBe("none");
expect(cssLightness(state.backgroundColor)).toBeLessThan(0.35);
expect(cssLightness(state.color)).toBeGreaterThan(0.9);
}
async function expectContextPanelHeaderUsesPrimary(
page: Page,
accessibleName: string,
) {
@@ -127,26 +255,22 @@ async function expectContextPanelHeaderMatchesNewSession(
.getByRole("main", { name: "Database management" });
const panel = manager.getByRole("dialog", { name: accessibleName });
const header = panel.locator(':scope > [data-catalog-panel-region="header"]');
const newSession = page
.getByRole("complementary", { name: "Session navigation" })
.getByRole("button", { name: "New session", exact: true });
await expect(header).toHaveCount(1);
await expect(newSession).toBeVisible();
const [headerStyle, actionStyle] = await Promise.all([
const currentNavigation = page.locator('[data-navigation-state="current"]');
const [headerStyle, navigationStyle] = await Promise.all([
header.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
backgroundImage: getComputedStyle(element).backgroundImage,
})),
newSession.evaluate((element) => ({
currentNavigation.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
})),
]);
expect(headerStyle.backgroundImage).toBe("none");
expect(headerStyle.backgroundColor).not.toBe("rgba(0, 0, 0, 0)");
expect(headerStyle.backgroundColor).toBe(actionStyle.backgroundColor);
expect(headerStyle.backgroundColor).not.toBe(navigationStyle.backgroundColor);
}
async function expectWorkAreaPanelGeometry(page: Page, accessibleName: string) {
@@ -181,7 +305,7 @@ async function expectWorkAreaPanelGeometry(page: Page, accessibleName: string) {
expect(panelBox.y + panelBox.height).toBeLessThanOrEqual(workAreaBox.y + workAreaBox.height + 1);
}
async function expectWorkAreaPanelHeaderMatchesNewSession(
async function expectWorkAreaPanelHeaderUsesPrimary(
page: Page,
accessibleName: string,
) {
@@ -189,21 +313,19 @@ async function expectWorkAreaPanelHeaderMatchesNewSession(
.getByTestId("conversation-column")
.getByRole("dialog", { name: accessibleName });
const header = panel.locator(':scope > [data-work-area-panel-region="header"]');
const newSession = page
.getByRole("complementary", { name: "Session navigation" })
.getByRole("button", { name: "New session", exact: true });
const [headerStyle, actionStyle] = await Promise.all([
const currentNavigation = page.locator('[data-navigation-state="current"]');
const [headerStyle, navigationStyle] = await Promise.all([
header.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
backgroundImage: getComputedStyle(element).backgroundImage,
})),
newSession.evaluate((element) => ({
currentNavigation.evaluate((element) => ({
backgroundColor: getComputedStyle(element).backgroundColor,
})),
]);
expect(headerStyle.backgroundImage).toBe("none");
expect(headerStyle.backgroundColor).toBe(actionStyle.backgroundColor);
expect(headerStyle.backgroundColor).not.toBe(navigationStyle.backgroundColor);
}
test.beforeAll(async () => {
@@ -218,10 +340,14 @@ test.afterAll(async () => {
test("context panels stay inside the manager and the Tables grid sits in a sidebar-colored frame", async ({ page }) => {
const unexpectedCatalogRequests: string[] = [];
const responses: Record<string, unknown> = {
"GET /api/catalog/databases": [database],
"GET /api/catalog/databases": [database, unconfiguredDatabase],
"GET /api/catalog/metadata-generation/models": {
models: [],
default: null,
models: [
{ id: "deepseek-v4-pro", label: "DeepSeek V4 Pro" },
{ id: "glm-53", label: "GLM 5.3" },
{ id: "qwen-36", label: "AritmoLab Qwen 3.6 35B A3B" },
],
default: "glm-53",
},
"GET /api/catalog/description-generation-runs?limit=50": [],
"GET /api/catalog/sensitive-data-suggestion-runs?limit=50": [],
@@ -252,10 +378,40 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
await page.goto(stack.publicUrl);
await signInAsAdmin(page);
await expectCurrentNavigation(page, "New session");
await expectSelectedSessionScopeTab(page, "My sessions");
await page.getByRole("tab", { name: "All sessions", exact: true }).click();
await expectSelectedSessionScopeTab(page, "All sessions");
await page.getByRole("tab", { name: "My sessions", exact: true }).click();
await expectSelectedSessionScopeTab(page, "My sessions");
await page
const adminNavigationRail = page.getByRole("complementary", { name: "Session navigation" });
const administration = adminNavigationRail.getByRole("button", { name: "Administration", exact: true });
await expect(administration).toHaveAttribute("aria-expanded", "false");
await expect(adminNavigationRail.getByRole("region", { name: "Administration" })).toHaveCount(0);
await administration.click();
await expect(administration).toHaveAttribute("aria-expanded", "true");
const administrationPanel = adminNavigationRail.getByRole("region", { name: "Administration" });
await expect(administrationPanel).toBeVisible();
expect(await administrationPanel.locator(":scope > *").evaluateAll((elements) => elements.map((element) => (
element.getAttribute("role") === "separator" ? "separator" : element.textContent?.trim()
)))).toEqual([
"Database management",
"separator",
"Workspace management",
"Pi management",
]);
await administration.click();
await expect(administration).toHaveAttribute("aria-expanded", "false");
await expect(adminNavigationRail.getByRole("button", { name: "Database management", exact: true })).toHaveCount(0);
await administration.click();
await expect(administration).toHaveAttribute("aria-expanded", "true");
await adminNavigationRail
.getByRole("button", { name: "Database management", exact: true })
.click();
await expectCurrentNavigation(page, "Database management");
await expect(
page.getByRole("complementary", { name: "Session navigation" }),
@@ -268,27 +424,41 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
await expect(
page.getByRole("button", { name: "Back to workspace", exact: true }),
).toHaveCount(0);
await expect(page.getByRole("button", { name: /Add database/i })).toHaveCount(0);
await expect(page.getByRole("columnheader", { name: /Revision \/ Evidence/ })).toBeVisible();
await expect(page.getByRole("columnheader", { name: /NL→SQL runtime/ })).toBeVisible();
await expect(page.getByRole("columnheader", { name: /Metadata Catalog/ })).toBeVisible();
const metadataModelSelector = page.getByRole("combobox", {
name: "Metadata-generation LLM model",
});
await expect(metadataModelSelector).toHaveCount(1);
await expect(metadataModelSelector).toHaveValue("glm-53");
await expect(metadataModelSelector.locator("option")).toHaveText([
"DeepSeek V4 Pro",
"GLM 5.3",
"AritmoLab Qwen 3.6 35B A3B",
]);
await page
.getByRole("button", { name: "Configure catalog for Research laboratory", exact: true })
.click();
const configurePanel = page.getByRole("dialog", { name: "Configure catalog" });
await expectContextPanelGeometry(page, "Configure catalog");
const configuredWorkspace = configurePanel.getByLabel("Workspace", { exact: true });
await expect(configuredWorkspace).toHaveValue("Research laboratory");
await expect(configuredWorkspace).toHaveAttribute("readonly", "");
await configurePanel.getByRole("button", { name: "Close database panel" }).click();
await page
.getByRole("button", { name: "Edit Policlinico San Donato", exact: true })
.click();
await expectContextPanelGeometry(page, "Edit database");
await expectContextPanelHeaderMatchesNewSession(page, "Edit database");
await expectContextPanelHeaderUsesPrimary(page, "Edit database");
await page
.getByRole("dialog", { name: "Edit database" })
.getByRole("button", { name: "Close database panel" })
.click();
await page
.getByRole("button", { name: "Description history", exact: true })
.click();
await expectContextPanelGeometry(page, "Description generation");
await expectContextPanelHeaderMatchesNewSession(page, "Description generation");
await page
.getByRole("dialog", { name: "Description generation" })
.getByRole("button", { name: /close/i })
.click();
await page.setViewportSize({ width: 1280, height: 800 });
await page
.getByRole("button", { name: "Edit Policlinico San Donato", exact: true })
@@ -300,12 +470,12 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
.click();
await page.setViewportSize({ width: 1910, height: 911 });
await page
.getByRole("button", {
name: "View tables for Policlinico San Donato",
exact: true,
})
.click();
const databaseTablesAction = page.getByRole("button", {
name: "View tables for Policlinico San Donato",
exact: true,
});
await expectFleetRowTooltipBelow(databaseTablesAction, "Tables");
await databaseTablesAction.click();
await expect(
page.getByRole("region", {
@@ -313,6 +483,16 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
}),
).toBeVisible();
await page
.getByRole("button", { name: "Description history", exact: true })
.click();
await expectContextPanelGeometry(page, "Description generation");
await expectContextPanelHeaderUsesPrimary(page, "Description generation");
await page
.getByRole("dialog", { name: "Description generation" })
.getByRole("button", { name: /close/i })
.click();
const applicationBar = page.locator(
'main[aria-label="Database management"] .thot-fleet-ledger__header',
);
@@ -326,6 +506,10 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
await expect(
applicationBar.getByRole("group", { name: "Database management actions" }),
).toBeVisible();
await expect(
applicationBar.getByRole("combobox", { name: "Metadata-generation LLM model" }),
).toHaveValue("glm-53");
await expect(metadataModelSelector).toHaveCount(1);
await expect(applicationBar).not.toContainText("Thoth catalog · Fleet ledger");
await expect(applicationBar).not.toContainText("Inspect physical metadata");
@@ -355,11 +539,14 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
await expect(page.getByRole("row", { name: /patients/ })).toBeVisible();
await page
.getByRole("button", { name: "Edit description for patients", exact: true })
.click();
const editPatientsAction = page.getByRole("button", {
name: "Edit description for patients",
exact: true,
});
await expectFleetRowTooltipBelow(editPatientsAction, "Edit metadata");
await editPatientsAction.click();
await expectContextPanelGeometry(page, "Review table description");
await expectContextPanelHeaderMatchesNewSession(page, "Review table description");
await expectContextPanelHeaderUsesPrimary(page, "Review table description");
const tableEditor = page.getByRole("dialog", { name: "Review table description" });
const tableEditorFooter = tableEditor.locator(
':scope > [data-catalog-panel-region="footer"]',
@@ -500,6 +687,10 @@ test("Workspace and Pi management share the centered work-area panel without cov
await page.goto(stack.publicUrl);
await signInAsAdmin(page);
const administration = page.getByRole("button", { name: "Administration", exact: true });
await expect(administration).toHaveAttribute("aria-expanded", "false");
await administration.click();
await expect(administration).toHaveAttribute("aria-expanded", "true");
for (const viewport of [
{ width: 1910, height: 911 },
@@ -511,25 +702,29 @@ test("Workspace and Pi management share the centered work-area panel without cov
exact: true,
});
await workspaceTrigger.click();
await expectCurrentNavigation(page, "Workspace management");
await expectWorkAreaPanelGeometry(page, "Workspace management");
await expectWorkAreaPanelHeaderMatchesNewSession(page, "Workspace management");
await expectWorkAreaPanelHeaderUsesPrimary(page, "Workspace management");
const piTrigger = page.getByRole("button", { name: "Pi management", exact: true });
await piTrigger.click();
await expectCurrentNavigation(page, "Pi management");
await expect(
page.getByRole("dialog", { name: "Workspace management" }),
).toHaveCount(0);
await expectWorkAreaPanelGeometry(page, "Pi management");
await expectWorkAreaPanelHeaderMatchesNewSession(page, "Pi management");
await expectWorkAreaPanelHeaderUsesPrimary(page, "Pi management");
await page
.getByRole("dialog", { name: "Pi management" })
.getByRole("button", { name: "Close Pi management" })
.click();
await expect(piTrigger).toBeFocused();
await expectCurrentNavigation(page, "New session");
}
await page.setViewportSize({ width: 720, height: 800 });
await page.getByRole("button", { name: "Workspace management", exact: true }).click();
await expectCurrentNavigation(page, "Workspace management");
const compactWorkArea = page.getByTestId("conversation-column");
const compactPanel = compactWorkArea.getByRole("dialog", { name: "Workspace management" });
const [compactWorkAreaBox, compactPanelBox] = await Promise.all([
+105 -1
View File
@@ -1,7 +1,15 @@
import { http, HttpResponse } from "msw";
import { expect, test } from "vitest";
import { server } from "../test/msw";
import { getCatalogMetrics, type CatalogMetrics } from "./catalog-databases";
import {
createCatalogRelationship,
deleteCatalogRelationship,
getCatalogMetrics,
rebuildGeneratedRelationships,
setCatalogRelationshipStatus,
type CatalogMetrics,
type CatalogRelationship,
} from "./catalog-databases";
const databaseId = "d4baf0f5-b9c3-4dc5-aad2-2f5676a36e64";
@@ -58,3 +66,99 @@ test("propagates an unsuccessful catalog metrics response", async () => {
await expect(getCatalogMetrics()).rejects.toMatchObject({ status: 503 });
});
const logicalRelationship: CatalogRelationship = {
id: "11111111-1111-4111-8111-111111111111",
databaseId,
constraintName: null,
sourceTableId: "22222222-2222-4222-8222-222222222222",
sourceTableName: "orders",
targetTableId: "33333333-3333-4333-8333-333333333333",
targetTableName: "users",
updateRule: null,
deleteRule: null,
deferrable: false,
initiallyDeferred: false,
columns: [{
position: 1,
sourceColumnId: "44444444-4444-4444-8444-444444444444",
sourceColumnName: "user_id",
targetColumnId: "55555555-5555-4555-8555-555555555555",
targetColumnName: "id",
}],
origin: "manual",
status: "active",
lastSyncedDatabaseVersion: null,
lastSyncedAt: null,
createdAt: "2026-08-31T12:00:00.000Z",
updatedAt: "2026-08-31T12:00:00.000Z",
};
test("creates a manual relationship from the selected source and target columns", async () => {
let requestBody: unknown;
server.use(http.post("/api/catalog/databases/:databaseId/relationships", async ({ request }) => {
requestBody = await request.json();
return HttpResponse.json(logicalRelationship, { status: 201 });
}));
await expect(createCatalogRelationship(
databaseId,
logicalRelationship.columns[0].sourceColumnId,
logicalRelationship.columns[0].targetColumnId,
)).resolves.toEqual(logicalRelationship);
expect(requestBody).toEqual({
sourceColumnId: logicalRelationship.columns[0].sourceColumnId,
targetColumnId: logicalRelationship.columns[0].targetColumnId,
});
});
test("rebuilds generated relationships for one database", async () => {
let calls = 0;
server.use(http.post(
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
() => {
calls += 1;
return HttpResponse.json({ added: 12, alreadyPresent: 7, excluded: 3, ambiguous: 2 });
},
));
await expect(rebuildGeneratedRelationships(databaseId)).resolves.toEqual({
added: 12,
alreadyPresent: 7,
excluded: 3,
ambiguous: 2,
});
expect(calls).toBe(1);
});
test("changes whether a logical relationship is active or excluded", async () => {
let requestBody: unknown;
server.use(http.patch(
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
async ({ request }) => {
requestBody = await request.json();
return HttpResponse.json({ ...logicalRelationship, status: "excluded" });
},
));
await expect(setCatalogRelationshipStatus(
databaseId,
logicalRelationship.id,
"excluded",
)).resolves.toMatchObject({ status: "excluded" });
expect(requestBody).toEqual({ status: "excluded" });
});
test("permanently deletes a logical relationship", async () => {
let calls = 0;
server.use(http.delete(
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
() => {
calls += 1;
return new HttpResponse(null, { status: 204 });
},
));
await expect(deleteCatalogRelationship(databaseId, logicalRelationship.id)).resolves.toBeUndefined();
expect(calls).toBe(1);
});
+61 -3
View File
@@ -3,6 +3,12 @@ import { joinBackendPath } from "./runtime-config";
export type DatabaseTransport = "postgres_direct" | "rest_api" | "ssh_tunnel";
export type ConnectionStatus = "untested" | "reachable" | "failed";
export type WorkspaceEvidenceState =
| "not_declared"
| "materialized_current_revision"
| "configuration_required"
| "configured_unverified"
| "workspace_unavailable";
export type CatalogSecretName =
| "password"
| "apiKey"
@@ -33,6 +39,16 @@ export interface CatalogDatabase {
workspaceName: string;
workspaceDescription?: string;
workspaceAvailable: boolean;
workspaceRevision: { commit: string; blob: string } | null;
workspaceEvidence: {
sourceType: "filesystem" | "http" | "s3" | null;
state: WorkspaceEvidenceState;
};
runtimeBinding: {
transport: DatabaseTransport;
configurationState: "ready" | "configuration_required";
sessionTransportSupported: boolean;
} | null;
configured: boolean;
engine: "postgres";
databaseName: string;
@@ -164,25 +180,37 @@ export interface CatalogRelationshipColumn {
targetColumnName: string;
}
export type CatalogRelationshipOrigin = "physical" | "generated" | "manual";
export type CatalogRelationshipStatus = "active" | "excluded";
export interface CatalogRelationship {
id: string;
databaseId: string;
constraintName: string;
constraintName: string | null;
sourceTableId: string;
sourceTableName: string;
targetTableId: string;
targetTableName: string;
updateRule: string;
deleteRule: string;
updateRule: string | null;
deleteRule: string | null;
deferrable: boolean;
initiallyDeferred: boolean;
columns: CatalogRelationshipColumn[];
origin: CatalogRelationshipOrigin;
status: CatalogRelationshipStatus;
lastSyncedDatabaseVersion: number | null;
lastSyncedAt: string | null;
createdAt: string;
updatedAt: string;
}
export interface CatalogRelationshipRebuildResult {
added: number;
alreadyPresent: number;
excluded: number;
ambiguous: number;
}
export type CatalogDatabaseMetadataDeleteTarget = "tables" | "relationships";
export type CatalogTableMetadataDeleteTarget = "columns" | "relationships";
@@ -452,6 +480,36 @@ export const listSensitiveDataSuggestionEvents = (runId: string, after = 0) =>
export const listCatalogRelationships = (databaseId: string) =>
apiFetch<CatalogRelationship[]>(`/catalog/databases/${encodeURIComponent(databaseId)}/relationships`);
export const createCatalogRelationship = (
databaseId: string,
sourceColumnId: string,
targetColumnId: string,
) => apiFetch<CatalogRelationship>(
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships`,
{ method: "POST", body: JSON.stringify({ sourceColumnId, targetColumnId }) },
);
export const rebuildGeneratedRelationships = (databaseId: string) =>
apiFetch<CatalogRelationshipRebuildResult>(
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/rebuild-generated`,
{ method: "POST" },
);
export const setCatalogRelationshipStatus = (
databaseId: string,
relationshipId: string,
status: CatalogRelationshipStatus,
) => apiFetch<CatalogRelationship>(
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/${encodeURIComponent(relationshipId)}`,
{ method: "PATCH", body: JSON.stringify({ status }) },
);
export const deleteCatalogRelationship = (databaseId: string, relationshipId: string) =>
apiFetch<void>(
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/${encodeURIComponent(relationshipId)}`,
{ method: "DELETE" },
);
export const deleteCatalogDatabaseMetadata = (
databaseIds: string[],
target: CatalogDatabaseMetadataDeleteTarget,
+9
View File
@@ -152,6 +152,15 @@ test.each([
["sensitive_data_suggestion_history_request_invalid", "Sensitive suggestion history parameters are invalid."],
["sensitive_data_suggestion_history_failed", "Sensitive suggestion history could not be loaded."],
["sensitive_data_suggestion_run_not_found", "The sensitive suggestion run was not found."],
["relationship_not_found", "The relationship no longer exists. Refresh and try again."],
["relationship_duplicate", "This relationship already exists."],
["relationship_target_not_unique", "The target column must be the only primary-key column of its table."],
["relationship_type_incompatible", "Source and target column types are not compatible."],
["relationship_read_only", "Physical relationships are read-only."],
["relationship_request_invalid", "The relationship request is invalid."],
["relationship_operation_failed", "The relationship operation failed."],
["relationship_schema_stale", "Synchronize the current database schema before managing logical relationships."],
["column_not_found", "The selected catalog column was not found. Refresh and try again."],
])("maps the catalog error code %s to safe local copy", async (code, message) => {
const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue(
new Response(JSON.stringify({ code, message: "provider detail must not be trusted" }), {
+12
View File
@@ -35,6 +35,9 @@ const safeErrorCodes = new Set([
"sensitive_data_suggestion_run_not_found",
"schema_sync_conflict", "schema_introspection_failed", "schema_request_invalid",
"schema_operation_failed", "sync_run_not_found", "table_stale", "column_stale",
"relationship_not_found", "relationship_duplicate", "relationship_target_not_unique",
"relationship_type_incompatible", "relationship_read_only", "relationship_request_invalid",
"relationship_operation_failed", "relationship_schema_stale", "column_not_found",
]);
type SafeErrorPayload = {
@@ -104,6 +107,15 @@ const localCodeMessages: Record<string, string> = {
sync_run_not_found: "The synchronization run was not found.",
table_stale: "Table metadata changed. Reload and try again.",
column_stale: "Column metadata changed. Reload and try again.",
relationship_not_found: "The relationship no longer exists. Refresh and try again.",
relationship_duplicate: "This relationship already exists.",
relationship_target_not_unique: "The target column must be the only primary-key column of its table.",
relationship_type_incompatible: "Source and target column types are not compatible.",
relationship_read_only: "Physical relationships are read-only.",
relationship_request_invalid: "The relationship request is invalid.",
relationship_operation_failed: "The relationship operation failed.",
relationship_schema_stale: "Synchronize the current database schema before managing logical relationships.",
column_not_found: "The selected catalog column was not found. Refresh and try again.",
};
const localStatusMessages: Record<number, string> = {
+2
View File
@@ -13,6 +13,8 @@ const buttonVariants = cva(
"border-border bg-card shadow-xs hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:border-input dark:bg-input/30 dark:hover:bg-input/50",
secondary:
"bg-secondary text-secondary-foreground hover:bg-[color-mix(in_oklch,oklch(var(--secondary)),oklch(var(--foreground))_5%)] aria-expanded:bg-secondary aria-expanded:text-secondary-foreground",
navigationActive:
"border-[oklch(var(--nav-active-border))] bg-[oklch(var(--nav-active))] text-[oklch(var(--nav-active-foreground))] shadow-xs hover:bg-[oklch(var(--nav-active-hover))] hover:text-[oklch(var(--nav-active-foreground))] focus-visible:border-[oklch(var(--nav-active-border))] focus-visible:ring-[oklch(var(--nav-active-border)/0.32)]",
ghost:
"hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:hover:bg-muted/50",
destructive:
+10
View File
@@ -47,6 +47,12 @@
--success: 0.7577 0.1581 165.0;
--warning: 0.8523 0.1386 78.9;
--info: 0.7035 0.1128 221.3;
/* Current navigation shares Instrument Red's hue, with lower chroma and
higher lightness so selection reads as a muted location signal. */
--nav-active: 0.9250 0.0520 23.2;
--nav-active-hover: 0.8950 0.0710 23.2;
--nav-active-foreground: 0.3650 0.1100 23.2;
--nav-active-border: 0.6000 0.1350 23.2;
--radius: 0.5rem;
--sidebar: 0.9709 0.0011 17.2;
--sidebar-foreground: 0.2678 0.0097 355.6;
@@ -81,6 +87,10 @@
--border: 0.3715 0 90;
--input: 0.3715 0 90;
--ring: 0.6897 0.1779 16.9;
--nav-active: 0.3350 0.0550 20.3;
--nav-active-hover: 0.3750 0.0650 20.3;
--nav-active-foreground: 0.8900 0.0700 20.3;
--nav-active-border: 0.6800 0.1350 20.3;
--sidebar: 0.2350 0 90;
--sidebar-foreground: 0.9310 0 90;
--sidebar-primary: 0.6023 0.1848 20.3;
+39 -14
View File
@@ -1,4 +1,4 @@
import { act, render, screen, waitFor } from "@testing-library/react";
import { act, render, screen, waitFor, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { http, HttpResponse } from "msw";
@@ -45,24 +45,50 @@ beforeEach(() => {
});
describe("authenticated shell permissions", () => {
test("shows only read-safe workspace chrome to a session user", async () => {
test("hides administrative navigation from a session user", async () => {
renderShell({ subject: "user-1", isAdmin: false, roles: ["user"], permissions: ["session.use"] });
expect(await screen.findByRole("button", { name: "Workspace management" })).toBeInTheDocument();
expect(await screen.findByRole("button", { name: "New session" })).toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
});
test("shows management and all-session chrome only for exact permissions", async () => {
test("shows the ordered administrative accordion only to an admin", async () => {
const user = userEvent.setup();
renderShell({
subject: "admin-1",
isAdmin: true,
roles: ["admin"],
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "pi.manage"],
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"],
});
expect(await screen.findByRole("button", { name: "Pi management" })).toBeInTheDocument();
expect(screen.getByRole("button", { name: "All sessions" })).toBeInTheDocument();
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
const trigger = await within(sessionNavigation).findByRole("button", { name: "Administration" });
expect(trigger).toHaveAttribute("aria-expanded", "false");
expect(within(sessionNavigation).queryByRole("region", { name: "Administration" })).not.toBeInTheDocument();
trigger.focus();
await user.keyboard("{Enter}");
expect(trigger).toHaveAttribute("aria-expanded", "true");
const panel = within(sessionNavigation).getByRole("region", { name: "Administration" });
const database = within(panel).getByRole("button", { name: "Database management" });
const separator = within(panel).getByRole("separator");
const workspace = within(panel).getByRole("button", { name: "Workspace management" });
const pi = within(panel).getByRole("button", { name: "Pi management" });
expect(panel.children[0]).toBe(database);
expect(panel.children[1]).toBe(separator);
expect(panel.children[2]).toBe(workspace);
expect(panel.children[3]).toBe(pi);
expect(screen.getByRole("tab", { name: "All sessions" })).toBeInTheDocument();
await user.keyboard(" ");
expect(trigger).toHaveAttribute("aria-expanded", "false");
expect(within(sessionNavigation).queryByRole("region", { name: "Administration" })).not.toBeInTheDocument();
expect(within(sessionNavigation).queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
});
test("keeps identity but hides logout outside local authentication", async () => {
@@ -178,15 +204,14 @@ describe("authenticated shell permissions", () => {
subject: "admin-1", isAdmin: true, roles: ["admin"],
permissions: ["session.use", "session.read_all", "workspace.manage", "pi.manage"],
});
expect(await screen.findByRole("button", { name: "Pi management" })).toBeInTheDocument();
expect(screen.getByRole("button", { name: "All sessions" })).toBeInTheDocument();
expect(await screen.findByRole("button", { name: "Administration" })).toBeInTheDocument();
expect(screen.getByRole("tab", { name: "All sessions" })).toBeInTheDocument();
act(() => clearAuthState());
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
await userEvent.click(screen.getByRole("button", { name: "Workspace management" }));
expect(await screen.findByRole("heading", { name: "Workspace management" })).toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Update workspace repository" })).not.toBeInTheDocument();
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
});
test("remounts the real shell so user-A panel and transcript state cannot survive user-B", async () => {
@@ -17,6 +17,10 @@ function renderShell() {
);
}
async function expandAdministration() {
await userEvent.click(screen.getByRole("button", { name: "Administration" }));
}
beforeEach(() => {
clearAuthState();
setAuthState({
@@ -92,10 +96,19 @@ test("uses Fleet Ledger by default and gates the legacy fallback to non-producti
test("replaces the core conversation while keeping the session navigation", async () => {
renderShell();
await expandAdministration();
const conversationColumn = screen.getByTestId("conversation-column");
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
const databaseManagement = within(sessionNavigation).getByRole("button", { name: "Database management" });
expect(newSession).toHaveAttribute("aria-current", "page");
expect(newSession).toHaveAttribute("data-navigation-state", "current");
expect(newSession).toHaveClass("bg-[oklch(var(--nav-active))]");
expect(databaseManagement).toHaveAttribute("data-navigation-state", "available");
expect(databaseManagement).not.toHaveAttribute("aria-current");
await userEvent.click(databaseManagement);
const manager = screen.getByRole("main", { name: "Database management" });
@@ -103,8 +116,13 @@ test("replaces the core conversation while keeping the session navigation", asyn
expect(conversationColumn).toContainElement(manager);
expect(screen.getByRole("complementary", { name: "Session navigation" })).toBe(sessionNavigation);
expect(sessionNavigation).toBeVisible();
expect(within(sessionNavigation).getByRole("button", { name: "New session" })).toBeVisible();
expect(newSession).toBeVisible();
expect(newSession).toHaveAttribute("data-navigation-state", "available");
expect(newSession).not.toHaveAttribute("aria-current");
expect(newSession).not.toHaveClass("bg-[oklch(var(--nav-active))]");
expect(databaseManagement).toHaveAttribute("aria-current", "page");
expect(databaseManagement).toHaveAttribute("data-navigation-state", "current");
expect(databaseManagement).toHaveClass("bg-[oklch(var(--nav-active))]");
expect(screen.queryByRole("button", { name: "Back to workspace" })).not.toBeInTheDocument();
expect(screen.queryByRole("textbox", { name: /new question/i })).not.toBeInTheDocument();
});
@@ -135,6 +153,7 @@ test("keeps a live core session connected while returning from database manageme
})),
);
renderShell();
await expandAdministration();
const session = await screen.findByTestId("session-item-s1");
await userEvent.click(session);
@@ -157,7 +176,7 @@ test("keeps a live core session connected while returning from database manageme
expect(FakeEventSource.instances).toHaveLength(1);
});
test("keeps read-safe workspace access but hides privileged management entries", () => {
test("hides the complete administrative navigation from a regular user", () => {
clearAuthState();
setAuthState({
issuer: "test",
@@ -171,7 +190,9 @@ test("keeps read-safe workspace access but hides privileged management entries",
renderShell();
expect(screen.getByRole("button", { name: "Workspace management" })).toBeInTheDocument();
expect(screen.getByRole("button", { name: "New session" })).toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
});
@@ -182,6 +203,13 @@ test("does not leave database management without confirming a dirty form", async
workspaceId: "psd-clinical",
workspaceName: "Policlinico San Donato",
workspaceAvailable: true,
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
runtimeBinding: {
transport: "postgres_direct",
configurationState: "ready",
sessionTransportSupported: true,
},
configured: true,
engine: "postgres",
databaseName: "warehouse",
@@ -202,6 +230,7 @@ test("does not leave database management without confirming a dirty form", async
}])));
const confirm = vi.spyOn(window, "confirm").mockReturnValue(false);
renderShell();
await expandAdministration();
await userEvent.click(screen.getByRole("button", { name: "Database management" }));
await userEvent.click(await screen.findByRole("button", { name: "Edit Policlinico San Donato" }));
@@ -156,6 +156,11 @@ test("model selector shows the three Pi-enabled models and stores the selected p
});
test("opens Workspace management from the right sidebar without interrupting the shell", async () => {
setAuthState({
issuer: "test", subject: "admin", roles: ["admin"],
permissions: ["session.use", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"],
isAdmin: true, csrfToken: null, session: null,
});
server.use(
http.get("/api/workspace-registry/status", () =>
HttpResponse.json({ branch: "main", ahead: 0, behind: 0, degraded: false })),
@@ -163,15 +168,31 @@ test("opens Workspace management from the right sidebar without interrupting the
);
renderShell();
await userEvent.click(screen.getByRole("button", { name: "Administration" }));
await userEvent.click(screen.getByRole("button", { name: "Workspace management" }));
expect(await screen.findByRole("heading", { name: "Workspace management" })).toBeVisible();
const dialog = screen.getByRole("dialog", { name: "Workspace management" });
const workArea = screen.getByTestId("conversation-column");
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
const workspaceManagement = within(sessionNavigation).getByRole("button", { name: "Workspace management" });
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
expect(workArea).toContainElement(dialog);
expect(sessionNavigation).not.toContainElement(dialog);
expect(screen.getByTestId("app-shell")).toHaveAttribute("data-activity-layout", "closed");
expect(workspaceManagement).toHaveAttribute("aria-current", "page");
expect(workspaceManagement).toHaveAttribute("aria-expanded", "true");
expect(workspaceManagement).toHaveAttribute("data-navigation-state", "current");
expect(workspaceManagement).toHaveClass("bg-[oklch(var(--nav-active))]");
expect(newSession).toHaveAttribute("data-navigation-state", "available");
expect(newSession).not.toHaveAttribute("aria-current");
await userEvent.click(screen.getByRole("button", { name: "Close workspace management" }));
await waitFor(() => expect(screen.queryByRole("dialog", { name: "Workspace management" })).not.toBeInTheDocument());
expect(newSession).toHaveAttribute("aria-current", "page");
expect(newSession).toHaveAttribute("data-navigation-state", "current");
expect(workspaceManagement).toHaveAttribute("aria-expanded", "false");
expect(workspaceManagement).toHaveAttribute("data-navigation-state", "available");
});
test("does not block the shell when the DWH is unavailable at startup", async () => {
@@ -184,7 +205,8 @@ test("does not block the shell when the DWH is unavailable at startup", async ()
);
renderShell();
expect(await screen.findByRole("button", { name: "Workspace management" })).toBeVisible();
expect(await screen.findByRole("button", { name: "New session" })).toBeVisible();
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
expect(screen.queryByRole("heading", { name: "Connection unavailable" })).not.toBeInTheDocument();
expect(healthChecks).toBe(0);
});
@@ -16,7 +16,7 @@ const regularUser: AuthenticatedUser = {
};
const adminUser: AuthenticatedUser = {
...regularUser, subject: "alice-id", roles: ["admin"] as const,
permissions: ["session.use", "session.read_all", "pi.manage"] as const, isAdmin: true,
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"] as const, isAdmin: true,
};
function wrap(user: AuthenticatedUser = regularUser) {
@@ -95,7 +95,8 @@ test("regular users load only their sessions and never see administrator control
wrap();
await screen.findByText("Attiva uno");
expect(scope).toBe("mine");
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
expect(screen.queryByText(/administrator view/i)).not.toBeInTheDocument();
});
@@ -115,17 +116,25 @@ test("Pi management preserves the open session summary and the model activity ti
http.post("/api/sessions/:id/resume", () => resumeResult("s1")),
http.get("/api/sessions/:id", () => HttpResponse.json({ phase: 4 })),
);
wrap();
wrap(adminUser);
await user.click(await screen.findByText("Attiva uno"));
expect(await screen.findByRole("complementary", { name: "Session summary" })).toHaveTextContent("Domanda originale");
await user.click(screen.getByRole("button", { name: "Administration" }));
await user.click(screen.getByRole("button", { name: "Pi management" }));
expect(await screen.findByRole("heading", { name: "Pi management" })).toBeVisible();
const dialog = screen.getByRole("dialog", { name: "Pi management" });
const workArea = screen.getByTestId("conversation-column");
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
const piManagement = within(sessionNavigation).getByRole("button", { name: "Pi management" });
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
expect(workArea).toContainElement(dialog);
expect(sessionNavigation).not.toContainElement(dialog);
expect(piManagement).toHaveAttribute("aria-current", "page");
expect(piManagement).toHaveAttribute("aria-expanded", "true");
expect(piManagement).toHaveAttribute("data-navigation-state", "current");
expect(newSession).toHaveAttribute("data-navigation-state", "available");
expect(newSession).not.toHaveAttribute("aria-current");
const preservedSummary = screen.getByRole("complementary", { name: "Session summary" });
expect(preservedSummary).not.toHaveAttribute("aria-hidden", "true");
expect(preservedSummary).toHaveTextContent("Attiva uno");
@@ -133,6 +142,10 @@ test("Pi management preserves the open session summary and the model activity ti
await waitFor(() => {
expect(screen.getByRole("complementary", { name: "Session summary" })).toHaveTextContent("Domanda originale");
});
expect(newSession).toHaveAttribute("aria-current", "page");
expect(newSession).toHaveAttribute("data-navigation-state", "current");
expect(piManagement).toHaveAttribute("aria-expanded", "false");
expect(piManagement).toHaveAttribute("data-navigation-state", "available");
await user.click(screen.getByRole("button", { name: "Resume" }));
await screen.findByRole("button", { name: "Show model activity" });
@@ -160,17 +173,66 @@ test("administrators can explicitly switch to all sessions and see owners", asyn
}),
);
wrap(adminUser);
await screen.findByRole("button", { name: "All sessions" });
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "true");
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "false");
await userEvent.click(screen.getByRole("button", { name: "All sessions" }));
const tablist = await screen.findByRole("tablist", { name: "Session scope" });
const mySessions = within(tablist).getByRole("tab", { name: "My sessions" });
const allSessions = within(tablist).getByRole("tab", { name: "All sessions" });
expect(mySessions).toHaveAttribute("aria-selected", "true");
expect(mySessions).toHaveAttribute("aria-controls", "session-scope-panel");
expect(mySessions).toHaveAttribute("tabindex", "0");
expect(mySessions).toHaveAttribute("data-tab-state", "active");
expect(mySessions).toHaveClass("bg-[oklch(var(--nav-active))]");
expect(allSessions).toHaveAttribute("aria-selected", "false");
expect(allSessions).toHaveAttribute("aria-controls", "session-scope-panel");
expect(allSessions).toHaveAttribute("tabindex", "-1");
expect(allSessions).toHaveAttribute("data-tab-state", "inactive");
expect(allSessions).toHaveClass("border-border", "bg-card");
expect(allSessions).not.toHaveClass("border-transparent", "bg-transparent");
expect(screen.getByRole("tabpanel")).toHaveAttribute("aria-labelledby", "session-scope-mine-tab");
await userEvent.click(allSessions);
await waitFor(() => expect(scope).toBe("all"));
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "false");
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "true");
expect(mySessions).toHaveAttribute("aria-selected", "false");
expect(mySessions).toHaveAttribute("data-tab-state", "inactive");
expect(allSessions).toHaveAttribute("aria-selected", "true");
expect(allSessions).toHaveAttribute("data-tab-state", "active");
expect(allSessions).toHaveClass("bg-[oklch(var(--nav-active))]");
expect(screen.getByRole("tabpanel")).toHaveAttribute("aria-labelledby", "session-scope-all-tab");
expect(await screen.findByText("Administrator view: all sessions")).toBeInTheDocument();
expect(screen.getByText("Owner: Bob")).toBeInTheDocument();
});
test("session scope tabs support roving keyboard navigation", async () => {
let scope = "";
server.use(http.get("/api/sessions", ({ request }) => {
scope = new URL(request.url).searchParams.get("scope") ?? "";
return HttpResponse.json(LIST);
}));
wrap(adminUser);
const user = userEvent.setup();
const tablist = await screen.findByRole("tablist", { name: "Session scope" });
const mySessions = within(tablist).getByRole("tab", { name: "My sessions" });
const allSessions = within(tablist).getByRole("tab", { name: "All sessions" });
mySessions.focus();
await user.keyboard("{ArrowRight}");
await waitFor(() => expect(scope).toBe("all"));
expect(allSessions).toHaveFocus();
expect(allSessions).toHaveAttribute("aria-selected", "true");
expect(mySessions).toHaveAttribute("tabindex", "-1");
await user.keyboard("{ArrowLeft}");
await waitFor(() => expect(scope).toBe("mine"));
expect(mySessions).toHaveFocus();
expect(mySessions).toHaveAttribute("aria-selected", "true");
await user.keyboard("{End}");
expect(allSessions).toHaveFocus();
expect(allSessions).toHaveAttribute("aria-selected", "true");
await user.keyboard("{Home}");
expect(mySessions).toHaveFocus();
expect(mySessions).toHaveAttribute("aria-selected", "true");
});
test("administrator confirms before deleting a same-named user's session", async () => {
let deletes = 0;
server.use(
@@ -187,7 +249,7 @@ test("administrator confirms before deleting a same-named user's session", async
}),
);
wrap(adminUser);
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
await userEvent.click(await screen.findByRole("tab", { name: "All sessions" }));
await screen.findByText("Owner: Alice");
await userEvent.click(screen.getByRole("checkbox", { name: "Select Attiva uno" }));
await userEvent.click(screen.getByRole("button", { name: "Delete 1 selected sessions" }));
@@ -213,7 +275,7 @@ test("administrator confirms before archiving a same-named user's session", asyn
}),
);
wrap(adminUser);
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
await userEvent.click(await screen.findByRole("tab", { name: "All sessions" }));
await screen.findByText("Owner: Alice");
await userEvent.click(screen.getByRole("button", { name: "Session actions" }));
await userEvent.click(await screen.findByText("Archive"));
+193 -90
View File
@@ -17,8 +17,9 @@ import { StopConfirmDialog } from "./StopConfirmDialog";
import { SteerInput, ComposerFooter } from "./SteerInput";
import { WorkflowBar } from "./WorkflowBar";
import { DatabaseManagementPage } from "./DatabaseManagementPage";
import { Pencil, ArrowLeft, ArrowRight, Trash2 } from "lucide-react";
import { Button } from "../components/ui/button";
import { Pencil, ArrowLeft, ArrowRight, ChevronDown, Trash2 } from "lucide-react";
import { Accordion } from "@base-ui/react/accordion";
import { Button, buttonVariants } from "../components/ui/button";
import { Checkbox } from "../components/ui/checkbox";
import { Toaster } from "../components/ui/sonner";
import { toast } from "sonner";
@@ -34,7 +35,7 @@ import type { SessionScope, SessionSummary } from "../api/types";
import { useAuthGeneration, useAuthUser } from "../auth/authState";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import type { CSSProperties } from "react";
import type { CSSProperties, KeyboardEvent } from "react";
import { captureAuthOperation, isAuthOperationCurrent, StaleAuthOperationError, type AuthOperationGuard } from "../auth/authOperation";
interface AppShellProps {
@@ -43,6 +44,14 @@ interface AppShellProps {
type ActiveSurface = "core" | "database-management";
type ActiveManagementPanel = "workspace" | "pi" | null;
const SESSION_SCOPES: readonly SessionScope[] = ["mine", "all"];
function sessionScopeTabClass(selected: boolean): string {
const base = "relative -mb-px flex h-8 w-full cursor-pointer items-center justify-center rounded-t-md border border-b-2 px-2 text-xs font-semibold tracking-[0.005em] outline-none focus-visible:z-10 focus-visible:ring-3 focus-visible:ring-[oklch(var(--nav-active-border)/0.28)]";
return selected
? `${base} border-[oklch(var(--nav-active-border))] bg-[oklch(var(--nav-active))] text-[oklch(var(--nav-active-foreground))] hover:bg-[oklch(var(--nav-active-hover))]`
: `${base} border-border bg-card text-muted-foreground hover:border-muted-foreground/45 hover:bg-muted/55 hover:text-foreground`;
}
type DatabaseManagementPresentation = "legacy" | "fleet";
@@ -129,8 +138,13 @@ export function AppShell({ canLogout }: AppShellProps) {
const [creatingSession, setCreatingSession] = useState(false);
const [awaitingQuestion, setAwaitingQuestion] = useState(false);
const [sessionScope, setSessionScope] = useState<SessionScope>("mine");
const sessionScopeTabRefs = useRef<Record<SessionScope, HTMLButtonElement | null>>({
mine: null,
all: null,
});
const principal = authenticatedUser;
const permissions = authenticatedUser?.permissions ?? [];
const isAdmin = authenticatedUser?.isAdmin === true;
const canReadAllSessions = permissions.includes("session.read_all");
const canManageWorkspace = permissions.includes("workspace.manage");
const canManageWorkspaceSecrets = permissions.includes("workspace.secrets.manage");
@@ -164,6 +178,7 @@ export function AppShell({ canLogout }: AppShellProps) {
databaseNavigationRef.current = state;
}, []);
const [activeManagementPanel, setActiveManagementPanel] = useState<ActiveManagementPanel>(null);
const [adminNavigationValue, setAdminNavigationValue] = useState<string[]>([]);
const [activeOpen, setActiveOpen] = useState(true);
const [archiveOpen, setArchiveOpen] = useState(false);
const [renameTarget, setRenameTarget] = useState<SessionSummary | null>(null);
@@ -236,6 +251,22 @@ export function AppShell({ canLogout }: AppShellProps) {
setSelectedSessionIds(selected ? new Set(sessions.map((session) => session.id)) : new Set());
}
function handleSessionScopeTabKeyDown(
event: KeyboardEvent<HTMLButtonElement>,
current: SessionScope,
) {
const currentIndex = SESSION_SCOPES.indexOf(current);
let target: SessionScope | undefined;
if (event.key === "ArrowRight") target = SESSION_SCOPES[(currentIndex + 1) % SESSION_SCOPES.length];
else if (event.key === "ArrowLeft") target = SESSION_SCOPES[(currentIndex - 1 + SESSION_SCOPES.length) % SESSION_SCOPES.length];
else if (event.key === "Home") target = SESSION_SCOPES[0];
else if (event.key === "End") target = SESSION_SCOPES.at(-1);
if (!target) return;
event.preventDefault();
setSessionScope(target);
sessionScopeTabRefs.current[target]?.focus();
}
function canLeaveDatabaseManagement(): boolean {
if (activeSurface !== "database-management") return true;
if (databaseNavigationRef.current.busy) {
@@ -627,6 +658,12 @@ export function AppShell({ canLogout }: AppShellProps) {
await logoutUser();
}
const currentNavigation = activeSurface === "database-management"
? "database"
: activeManagementPanel ?? "core";
const adminNavigationOpen = adminNavigationValue.includes("administration");
const managementNavigationCurrent = currentNavigation !== "core";
return (
<div
ref={containerRef}
@@ -808,115 +845,180 @@ export function AppShell({ canLogout }: AppShellProps) {
<div className="flex flex-col gap-2 px-4 pb-3">
<Button
variant="default"
variant={currentNavigation === "core" ? "navigationActive" : "outline"}
size="sm"
className="w-full"
aria-current={currentNavigation === "core" ? "page" : undefined}
data-navigation-state={currentNavigation === "core" ? "current" : "available"}
onClick={startNewSession}
>
New session
</Button>
<Button
variant="outline"
size="sm"
className="w-full"
onClick={() => {
if (!canLeaveDatabaseManagement()) return;
setActiveSurface("core");
setActiveManagementPanel("workspace");
}}
>
Workspace management
</Button>
{canManageDatabase && (
<Button
variant={activeSurface === "database-management" ? "secondary" : "outline"}
size="sm"
className="w-full"
aria-current={activeSurface === "database-management" ? "page" : undefined}
onClick={() => {
setActiveManagementPanel(null);
setActiveSurface("database-management");
}}
{isAdmin && (
<Accordion.Root
value={adminNavigationValue}
onValueChange={setAdminNavigationValue}
className="rounded-lg border border-border/80 bg-card/45 p-1.5"
>
Database management
</Button>
)}
{canManagePi && (
<Button
variant="outline"
size="sm"
className="w-full"
onClick={() => {
if (!canLeaveDatabaseManagement()) return;
setActiveSurface("core");
setActiveManagementPanel("pi");
}}
>
Pi management
</Button>
<Accordion.Item value="administration">
<Accordion.Header className="m-0">
<Accordion.Trigger
className={buttonVariants({
variant: !adminNavigationOpen && managementNavigationCurrent ? "navigationActive" : "outline",
size: "sm",
className: "w-full justify-between px-2.5",
})}
data-navigation-state={!adminNavigationOpen && managementNavigationCurrent ? "current" : "available"}
>
<span>Administration</span>
<ChevronDown
aria-hidden="true"
data-icon="inline-end"
className={`transition-transform duration-150 motion-reduce:transition-none ${adminNavigationOpen ? "rotate-180" : ""}`}
/>
</Accordion.Trigger>
</Accordion.Header>
<Accordion.Panel className="grid gap-1.5 px-0.5 pt-1.5">
<Button
variant={currentNavigation === "database" ? "navigationActive" : "outline"}
size="sm"
className="w-full"
aria-current={currentNavigation === "database" ? "page" : undefined}
data-navigation-state={currentNavigation === "database" ? "current" : canManageDatabase ? "available" : "unavailable"}
disabled={!canManageDatabase}
title={canManageDatabase ? undefined : "Database management permission is required"}
onClick={() => {
setActiveManagementPanel(null);
setActiveSurface("database-management");
}}
>
Database management
</Button>
<div role="separator" aria-orientation="horizontal" className="mx-1 h-px bg-border/90" />
<Button
variant={currentNavigation === "workspace" ? "navigationActive" : "outline"}
size="sm"
className="w-full"
aria-current={currentNavigation === "workspace" ? "page" : undefined}
aria-expanded={activeManagementPanel === "workspace"}
data-navigation-state={currentNavigation === "workspace" ? "current" : "available"}
onClick={() => {
if (!canLeaveDatabaseManagement()) return;
setActiveSurface("core");
setActiveManagementPanel("workspace");
}}
>
Workspace management
</Button>
<Button
variant={currentNavigation === "pi" ? "navigationActive" : "outline"}
size="sm"
className="w-full"
aria-current={currentNavigation === "pi" ? "page" : undefined}
aria-expanded={activeManagementPanel === "pi"}
data-navigation-state={currentNavigation === "pi" ? "current" : canManagePi ? "available" : "unavailable"}
disabled={!canManagePi}
title={canManagePi ? undefined : "Pi management permission is required"}
onClick={() => {
if (!canLeaveDatabaseManagement()) return;
setActiveSurface("core");
setActiveManagementPanel("pi");
}}
>
Pi management
</Button>
</Accordion.Panel>
</Accordion.Item>
</Accordion.Root>
)}
</div>
{canReadAllSessions && (
<div className="px-4 pb-3">
<div className="grid grid-cols-2 gap-1 rounded-lg bg-muted p-1" aria-label="Session scope">
<Button
variant={sessionScope === "mine" ? "secondary" : "ghost"}
size="xs"
aria-pressed={sessionScope === "mine"}
<div className="px-4">
<div
role="tablist"
aria-label="Session scope"
aria-orientation="horizontal"
className="grid grid-cols-2 border-b border-border"
>
<button
ref={(node) => { sessionScopeTabRefs.current.mine = node; }}
id="session-scope-mine-tab"
type="button"
role="tab"
aria-selected={sessionScope === "mine"}
aria-controls="session-scope-panel"
tabIndex={sessionScope === "mine" ? 0 : -1}
data-tab-state={sessionScope === "mine" ? "active" : "inactive"}
className={sessionScopeTabClass(sessionScope === "mine")}
onClick={() => setSessionScope("mine")}
onKeyDown={(event) => handleSessionScopeTabKeyDown(event, "mine")}
>
My sessions
</Button>
<Button
variant={showingAllSessions ? "secondary" : "ghost"}
size="xs"
aria-pressed={showingAllSessions}
</button>
<button
ref={(node) => { sessionScopeTabRefs.current.all = node; }}
id="session-scope-all-tab"
type="button"
role="tab"
aria-selected={showingAllSessions}
aria-controls="session-scope-panel"
tabIndex={showingAllSessions ? 0 : -1}
data-tab-state={showingAllSessions ? "active" : "inactive"}
className={sessionScopeTabClass(showingAllSessions)}
onClick={() => setSessionScope("all")}
onKeyDown={(event) => handleSessionScopeTabKeyDown(event, "all")}
>
All sessions
</Button>
</button>
</div>
{showingAllSessions && (
<p className="mt-2 text-xs font-medium text-amber-700 dark:text-amber-400">
Administrator view: all sessions
</p>
)}
</div>
)}
{/* L1 — rail title */}
<div className="px-4 pb-1.5 pt-1">
<span className="thot-label text-[0.8rem] font-bold tracking-[0.18em] text-primary">
Sessions
</span>
</div>
<div className="flex items-center justify-between px-4 pb-2">
<label className="flex cursor-pointer items-center gap-2 text-xs font-medium text-muted-foreground hover:text-foreground">
<Checkbox
checked={allSessionsSelected}
aria-label="Select all sessions"
disabled={sessions.length === 0}
onCheckedChange={(selected) => toggleAllSessions(selected === true)}
/>
<span>Select all</span>
</label>
{selectedSessions.length > 0 && (
<Button
variant="destructive"
size="xs"
aria-label={`Delete ${selectedSessions.length} selected sessions`}
onClick={() => {
requestDelete(selectedSessions);
}}
>
<Trash2 />
Delete ({selectedSessions.length})
</Button>
<div
id={canReadAllSessions ? "session-scope-panel" : undefined}
role={canReadAllSessions ? "tabpanel" : undefined}
aria-labelledby={canReadAllSessions ? `session-scope-${sessionScope}-tab` : undefined}
className="flex min-h-0 flex-1 flex-col"
>
{canReadAllSessions && showingAllSessions && (
<p className="mx-4 mb-2 mt-2 text-xs font-medium text-amber-700 dark:text-amber-400">
Administrator view: all sessions
</p>
)}
</div>
<div className="flex-1 overflow-y-auto px-2 pb-4">
{/* L1 — rail title */}
<div className="px-4 pb-1.5 pt-1">
<span className="thot-label text-[0.8rem] font-bold tracking-[0.18em] text-primary">
Sessions
</span>
</div>
<div className="flex items-center justify-between px-4 pb-2">
<label className="flex cursor-pointer items-center gap-2 text-xs font-medium text-muted-foreground hover:text-foreground">
<Checkbox
checked={allSessionsSelected}
aria-label="Select all sessions"
disabled={sessions.length === 0}
onCheckedChange={(selected) => toggleAllSessions(selected === true)}
/>
<span>Select all</span>
</label>
{selectedSessions.length > 0 && (
<Button
variant="destructive"
size="xs"
aria-label={`Delete ${selectedSessions.length} selected sessions`}
onClick={() => {
requestDelete(selectedSessions);
}}
>
<Trash2 />
Delete ({selectedSessions.length})
</Button>
)}
</div>
<div className="flex-1 overflow-y-auto px-2 pb-4">
{/* L2 — section toggle */}
<button
type="button"
@@ -1002,6 +1104,7 @@ export function AppShell({ canLogout }: AppShellProps) {
showOwner={showingAllSessions}
/>
)}
</div>
</div>
</aside>
)}
@@ -29,6 +29,13 @@ function makeDatabase(overrides: Partial<CatalogDatabase> = {}): CatalogDatabase
workspaceId: "psd-clinical",
workspaceName: "Policlinico San Donato",
workspaceAvailable: true,
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
runtimeBinding: {
transport: "postgres_direct",
configurationState: "ready",
sessionTransportSupported: true,
},
configured: true,
engine: "postgres",
databaseName: "warehouse",
@@ -66,6 +73,9 @@ const orphan = makeDatabase({
workspaceId: "retired",
workspaceName: "Retired workspace",
workspaceAvailable: false,
workspaceRevision: null,
workspaceEvidence: { sourceType: null, state: "workspace_unavailable" },
runtimeBinding: null,
});
function renderPage({
@@ -192,7 +202,7 @@ const synchronizationScopes = [
{ scope: "all", label: "Synchronize all" },
] as const;
test("renders Fleet Ledger with real metrics, conceptual row tooltips, and contextual model selection", async () => {
test("renders Fleet Ledger with real metrics, conceptual row tooltips, and persistent model selection", async () => {
const user = userEvent.setup();
server.use(
http.get("/api/catalog/metrics", () => HttpResponse.json({
@@ -223,10 +233,15 @@ test("renders Fleet Ledger with real metrics, conceptual row tooltips, and conte
expect(screen.queryByText("Thoth catalog · Fleet ledger")).not.toBeInTheDocument();
expect(screen.queryByText("Inspect physical metadata, curate descriptions and control catalog operations.")).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Back to workspace" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
expect(screen.getByRole("columnheader", { name: /Revision \/ Evidence/ })).toBeVisible();
expect(screen.getByRole("columnheader", { name: /NL→SQL runtime/ })).toBeVisible();
expect(screen.getByRole("columnheader", { name: /Metadata Catalog/ })).toBeVisible();
const summary = screen.getByRole("region", { name: "Fleet summary" });
expect(await within(summary).findByText("2,275")).toBeVisible();
expect(within(summary).getByText("75%")).toBeVisible();
expect(screen.queryByRole("combobox", { name: "Metadata description model" })).not.toBeInTheDocument();
const modelSelector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
expect(modelSelector).toHaveValue("local-qwen");
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
const tablesAction = within(databaseRow).getByRole("button", { name: "View tables for Policlinico San Donato" });
@@ -247,10 +262,103 @@ test("renders Fleet Ledger with real metrics, conceptual row tooltips, and conte
const actionPicker = screen.getByRole("combobox", { name: "Batch action" });
expect(within(actionPicker).getByRole("option", { name: /^Synchronize all(?:,|$)/ })).toBeVisible();
await user.selectOptions(actionPicker, "generate-missing");
expect(await screen.findByRole("combobox", { name: "Metadata description model" })).toHaveValue("local-qwen");
expect(screen.getAllByRole("combobox", { name: "Metadata-generation LLM model" })).toHaveLength(1);
expect(modelSelector).toHaveValue("local-qwen");
});
test("shows repository, NL→SQL runtime, and Metadata Catalog states independently", async () => {
const configured = makeDatabase({ connectionStatus: "reachable", testedVersion: 3 });
const needsRuntimeConfiguration = makeDatabase({
...unconfigured,
workspaceEvidence: { sourceType: "http", state: "configuration_required" },
runtimeBinding: {
transport: "rest_api",
configurationState: "configuration_required",
sessionTransportSupported: true,
},
});
renderPage({ rows: [configured, needsRuntimeConfiguration, orphan], presentation: "fleet" });
const configuredRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
expect(within(configuredRow).getByText("Active revision aaaaaaa")).toBeVisible();
expect(within(configuredRow).getByText("Evidence materialized · filesystem")).toBeVisible();
expect(within(configuredRow).getByText("Ready")).toBeVisible();
expect(within(configuredRow).getByText("Configured")).toBeVisible();
expect(within(configuredRow).getByText("Connection reachable")).toBeVisible();
const needsConfigurationRow = screen.getByRole("row", { name: /Research laboratory/ });
expect(within(needsConfigurationRow).getByText("Evidence credentials required · http")).toBeVisible();
expect(within(needsConfigurationRow).getByText("Configuration required")).toBeVisible();
expect(within(needsConfigurationRow).getByText("Not configured")).toBeVisible();
const orphanRow = screen.getByRole("row", { name: /Retired workspace/ });
expect(within(orphanRow).getByText("Workspace missing")).toBeVisible();
expect(within(orphanRow).getByText("Unavailable")).toBeVisible();
expect(within(orphanRow).getByText("Orphaned configuration")).toBeVisible();
});
test("keeps Fleet Ledger visible when a catalog response omits the Evidence projection", async () => {
const legacyDatabase = makeDatabase();
delete (legacyDatabase as Partial<CatalogDatabase>).workspaceEvidence;
renderPage({ rows: [legacyDatabase], presentation: "fleet" });
expect(await screen.findByRole("heading", { name: "Database management" })).toBeVisible();
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
expect(within(databaseRow).getByText("Evidence state unavailable")).toBeVisible();
});
test("opens the relationship map directly from a Fleet database and restores focus on return", async () => {
const user = userEvent.setup();
server.use(
http.get("/api/catalog/databases/:databaseId/relationships", () => HttpResponse.json([])),
);
renderPage({
rows: [makeDatabase({ connectionStatus: "reachable", testedVersion: 3 })],
presentation: "fleet",
});
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
const relationshipsAction = within(databaseRow).getByRole("button", {
name: "View relationships for Policlinico San Donato",
});
expect(relationshipsAction.parentElement).toHaveAttribute("data-tooltip", "Relationships");
expect(relationshipsAction).not.toHaveAttribute("title");
await user.click(relationshipsAction);
expect(await screen.findByRole("region", {
name: "Relationships for Policlinico San Donato",
})).toBeVisible();
expect(screen.getByText("Relationship map")).toBeVisible();
await user.click(screen.getByRole("button", { name: "Back to databases" }));
await waitFor(() => expect(screen.getByRole("button", {
name: "View relationships for Policlinico San Donato",
})).toHaveFocus());
});
test("offers catalog configuration directly on an unconfigured Fleet workspace", async () => {
const user = userEvent.setup();
renderPage({ rows: [unconfigured], presentation: "fleet" });
const databaseRow = await screen.findByRole("row", { name: /Research laboratory/ });
expect(within(databaseRow).queryByRole("button", {
name: "View relationships for Research laboratory",
})).not.toBeInTheDocument();
expect(within(databaseRow).queryByRole("button", {
name: "View tables for Research laboratory",
})).not.toBeInTheDocument();
await user.click(within(databaseRow).getByRole("button", {
name: "Configure catalog for Research laboratory",
}));
const drawer = await screen.findByRole("dialog", { name: "Configure catalog" });
expect(within(drawer).getByLabelText("Workspace")).toHaveAttribute("readonly");
expect(within(drawer).getByLabelText("Workspace")).toHaveValue("Research laboratory");
});
test("keeps only the local back action while browsing Fleet Ledger columns", async () => {
const user = userEvent.setup();
server.use(
@@ -276,6 +384,32 @@ test("keeps only the local back action while browsing Fleet Ledger columns", asy
expect(screen.queryByRole("button", { name: "Back to tables" })).not.toBeInTheDocument();
});
test("uses the database back-action background treatment for the table back action", async () => {
const user = userEvent.setup();
server.use(
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([patientsTable])),
http.get("/api/catalog/databases/:databaseId/tables/:tableId/columns", () => HttpResponse.json([])),
);
renderPage({ rows: [makeDatabase()], presentation: "fleet" });
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
await user.click(within(databaseRow).getByRole("button", {
name: "View tables for Policlinico San Donato",
}));
const databaseBackAction = await screen.findByRole("button", { name: "Back to databases" });
expect(databaseBackAction).toHaveClass("thot-fleet-back-action");
const backgroundTreatment = (element: HTMLElement) => element.className
.split(/\s+/)
.filter((className) => /(^|:)bg-/.test(className))
.sort();
await user.click(await screen.findByRole("button", { name: "View columns for patients" }));
const tableBackAction = await screen.findByRole("button", { name: "Back to tables" });
expect(tableBackAction).toHaveClass("thot-fleet-back-action");
expect(backgroundTreatment(tableBackAction)).toEqual(backgroundTreatment(databaseBackAction));
});
test("reopens database synchronization history when no run is active", async () => {
const user = userEvent.setup();
const completed: CatalogSyncRun = {
@@ -312,7 +446,7 @@ test("selects the configured metadata-description model by default", async () =>
})));
renderPage();
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
await waitFor(() => expect(selector).toHaveValue("local-qwen"));
});
@@ -320,7 +454,7 @@ test("keeps both run-history buttons visible beside the metadata-description sel
const user = userEvent.setup();
renderPage();
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
const toolbar = screen.getByRole("group", { name: "Database management controls" });
const metadataControls = screen.getByRole("group", { name: "Metadata description controls" });
const actions = screen.getByRole("group", { name: "Database management actions" });
@@ -344,7 +478,7 @@ test("keeps both run-history buttons visible beside the metadata-description sel
expect(toolbar.lastElementChild).toBe(actions);
expect(actions).toHaveClass("sm:justify-end");
expect(actions).toContainElement(screen.getByRole("button", { name: "Refresh" }));
expect(actions).toContainElement(screen.getByRole("button", { name: "Add database" }));
expect(within(actions).queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
expect(screen.queryByRole("note", {
name: "Metadata generation source data disclosure",
})).not.toBeInTheDocument();
@@ -371,7 +505,7 @@ test("changes the metadata-description model in page-local state", async () => {
})));
renderPage();
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
await waitFor(() => expect(selector).toHaveValue("local-qwen"));
await user.selectOptions(selector, "openai-mini");
@@ -385,10 +519,10 @@ test("explains application setup when no metadata-description model is configure
})));
renderPage();
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
expect(selector).toBeDisabled();
expect(await screen.findByText(
"Configure a metadata-generation model in application setup to enable description generation.",
"No metadata-generation LLM model is configured for this installation.",
)).toBeVisible();
});
@@ -399,11 +533,11 @@ test("uses a safe disabled setup state when metadata-description models are unav
}, { status: 503 })));
renderPage();
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
await waitFor(() => expect(selector).toHaveDisplayValue("Model choices unavailable"));
expect(selector).toBeDisabled();
expect(screen.getByText(
"Configure a metadata-generation model in application setup to enable description generation.",
"Metadata-generation LLM model choices are unavailable.",
)).toBeVisible();
expect(screen.queryByText("internal model registry details")).not.toBeInTheDocument();
});
@@ -417,11 +551,11 @@ test.each([null, "missing-model"])(
})));
renderPage();
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
await waitFor(() => expect(selector).toHaveDisplayValue("No usable model configured"));
expect(selector).toBeDisabled();
expect(screen.getByText(
"Configure a metadata-generation model in application setup to enable description generation.",
"No metadata-generation LLM model is configured for this installation.",
)).toBeVisible();
},
);
@@ -441,7 +575,7 @@ test("renders only the public metadata-generation model contract", async () => {
})));
renderPage();
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
await waitFor(() => expect(selector).toHaveValue("approved-model"));
expect(within(selector).getAllByRole("option").map((option) => option.textContent))
.toEqual(["Approved model"]);
@@ -458,8 +592,9 @@ test("starts with a full-width list and applies the row action matrix", async ()
expect(screen.getByRole("button", { name: "Delete Policlinico San Donato" })).toBeEnabled();
expect(screen.getByRole("button", { name: "View Research laboratory" })).toBeEnabled();
expect(screen.getByRole("button", { name: "Edit Research laboratory" })).toBeEnabled();
expect(screen.getByRole("button", { name: "Delete Research laboratory" })).toBeDisabled();
expect(screen.getByRole("button", { name: "Configure catalog for Research laboratory" })).toBeEnabled();
expect(screen.queryByRole("button", { name: "Edit Research laboratory" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: "Delete Research laboratory" })).not.toBeInTheDocument();
expect(screen.getByRole("button", { name: "View Retired workspace" })).toBeEnabled();
expect(screen.getByRole("button", { name: "Edit Retired workspace" })).toBeDisabled();
@@ -558,7 +693,7 @@ test("keeps Fleet Generate Missing behind the source-data disclosure", async ()
"generate-missing",
);
await waitFor(() => expect(screen.getByRole("combobox", {
name: "Metadata description model",
name: "Metadata-generation LLM model",
})).toHaveValue("local-qwen"));
await user.click(screen.getByRole("button", { name: "Run action" }));
@@ -951,7 +1086,7 @@ test("replaces the list with View and returns focus to the originating action",
expect(screen.getByRole("button", { name: "View Policlinico San Donato" })).toBeVisible();
});
test("opens an unconfigured row as Edit while creating its first saved configuration", async () => {
test("configures the catalog for the selected workspace and locks that workspace", async () => {
const user = userEvent.setup();
let postBody: unknown;
const saved = makeDatabase({
@@ -971,16 +1106,16 @@ test("opens an unconfigured row as Edit while creating its first saved configura
);
renderPage({ rows: () => rows });
await user.click(await screen.findByRole("button", { name: "Edit Research laboratory" }));
await user.click(await screen.findByRole("button", { name: "Configure catalog for Research laboratory" }));
expect(screen.getByRole("heading", { name: "Edit database" })).toBeVisible();
expect(screen.getByLabelText("Workspace")).toBeDisabled();
expect(screen.getByRole("heading", { name: "Configure catalog" })).toBeVisible();
expect(screen.getByLabelText("Workspace")).toHaveAttribute("readonly");
await user.selectOptions(screen.getByLabelText("Transport"), "rest_api");
await user.type(screen.getByLabelText("Base URL"), "https://psd.example/api");
expect(screen.getByLabelText("Diagnostic endpoint")).toHaveValue("/health");
expect(screen.getByLabelText("Diagnostic endpoint")).toHaveAttribute("readonly");
await user.click(screen.getByRole("button", { name: "Save database" }));
await user.click(screen.getByRole("button", { name: "Save catalog configuration" }));
await waitFor(() => expect(postBody).toMatchObject({
workspaceId: "lab",
@@ -989,7 +1124,7 @@ test("opens an unconfigured row as Edit while creating its first saved configura
expect(await screen.findByRole("heading", { name: "Edit database" })).toBeVisible();
});
test("global Add offers only unconfigured workspaces and lets the operator choose one", async () => {
test("has no global database creation path and scopes configuration to the chosen row", async () => {
const user = userEvent.setup();
const second = makeDatabase({
...unconfigured,
@@ -999,14 +1134,13 @@ test("global Add offers only unconfigured workspaces and lets the operator choos
});
renderPage({ rows: [makeDatabase(), unconfigured, second] });
await user.click(await screen.findByRole("button", { name: "Add database" }));
expect(screen.queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
await user.click(await screen.findByRole("button", { name: "Configure catalog for Radiology" }));
expect(screen.getByRole("heading", { name: "Add database" })).toBeVisible();
expect(screen.getByRole("button", { name: "Add database" })).toBeVisible();
const selector = screen.getByLabelText("Workspace");
expect(selector).toBeEnabled();
expect(screen.queryByRole("option", { name: "Policlinico San Donato" })).not.toBeInTheDocument();
await user.selectOptions(selector, "radiology");
expect(screen.getByRole("heading", { name: "Configure catalog" })).toBeVisible();
const workspace = screen.getByLabelText("Workspace");
expect(workspace).toHaveAttribute("readonly");
expect(workspace).toHaveValue("Radiology");
expect(screen.getByDisplayValue("radiology_dwh")).toBeVisible();
});
@@ -1064,7 +1198,8 @@ test("uses an in-page destructive form and returns the YAML workspace to Not con
await waitFor(() => expect(deleteCalled).toBe(true));
expect(deleteVersion).toBe("3");
expect(await screen.findByRole("button", { name: "Delete Policlinico San Donato" })).toBeDisabled();
expect(await screen.findByRole("button", { name: "Configure catalog for Policlinico San Donato" })).toBeEnabled();
expect(screen.queryByRole("button", { name: "Delete Policlinico San Donato" })).not.toBeInTheDocument();
expect(screen.getByText("Not configured")).toBeVisible();
});
@@ -2677,6 +2812,8 @@ test("navigates from a table to columns and from the database to physical relati
deferrable: false,
initiallyDeferred: false,
columns: [{ position: 1, sourceColumnId: "1", sourceColumnName: "patient_id", targetColumnId: idColumn.id, targetColumnName: "id" }],
origin: "physical",
status: "active",
lastSyncedDatabaseVersion: 3,
lastSyncedAt: "2026-08-27T10:00:00Z",
createdAt: "2026-08-27T10:00:00Z",
+73 -113
View File
@@ -6,7 +6,7 @@ import {
useState,
} from "react";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { History, Plus, RefreshCw } from "lucide-react";
import { History, RefreshCw } from "lucide-react";
import { toast } from "sonner";
import { Button } from "../components/ui/button";
import { ApiError, apiErrorMessage } from "../api/client";
@@ -209,11 +209,7 @@ export function DatabaseManagementPage({
() => queryClient.invalidateQueries({ queryKey: ["catalog-metrics"] }),
[queryClient],
);
const availableWorkspaces = useMemo(
() => rows.filter((row) => row.workspaceAvailable && !row.configured),
[rows],
);
const editable = screen.kind === "add" || screen.kind === "edit";
const editable = screen.kind === "configure" || screen.kind === "edit";
const configurationDirty = Boolean(
editable
&& draft
@@ -221,8 +217,9 @@ export function DatabaseManagementPage({
);
const dirty = Boolean(editable && draft && (configurationDirty || hasSecretChanges(draft)));
const busy = busyAction !== null;
const navigationDirty = screen.kind === "tables" ? tablesNavigationState.dirty : dirty;
const navigationBusy = screen.kind === "tables" ? tablesNavigationState.busy : busy;
const catalogChildVisible = screen.kind === "tables" || screen.kind === "relationships";
const navigationDirty = catalogChildVisible ? tablesNavigationState.dirty : dirty;
const navigationBusy = catalogChildVisible ? tablesNavigationState.busy : busy;
useEffect(() => {
onNavigationStateChange?.({ dirty: navigationDirty, busy: navigationBusy });
@@ -263,9 +260,14 @@ export function DatabaseManagementPage({
}, [queryClient]);
const restoreListFocus = useCallback(() => {
const originLabel = originRef.current?.getAttribute("aria-label");
window.setTimeout(() => {
if (originRef.current?.isConnected) {
originRef.current.focus();
} else if (originLabel) {
const matchingAction = [...document.querySelectorAll<HTMLButtonElement>("button")]
.find((button) => button.getAttribute("aria-label") === originLabel);
(matchingAction ?? searchInputRef.current)?.focus();
} else {
searchInputRef.current?.focus();
}
@@ -307,7 +309,6 @@ export function DatabaseManagementPage({
mode: DatabaseFormMode,
row: CatalogDatabase,
origin: HTMLElement,
workspaceLocked = false,
) => {
const nextDraft = draftFrom(row);
originRef.current = origin;
@@ -320,7 +321,6 @@ export function DatabaseManagementPage({
setScreen({
kind: mode,
workspaceId: row.workspaceId,
...(mode === "add" ? { workspaceLocked } : {}),
});
}, []);
@@ -329,7 +329,7 @@ export function DatabaseManagementPage({
}, [openForm]);
const editRow = useCallback((row: CatalogDatabase, origin: HTMLButtonElement) => {
openForm(row.configured ? "edit" : "add", row, origin, !row.configured);
openForm(row.configured ? "edit" : "configure", row, origin);
}, [openForm]);
const deleteRow = useCallback((row: CatalogDatabase, origin: HTMLButtonElement) => {
@@ -349,8 +349,9 @@ export function DatabaseManagementPage({
setScreen({ kind: "tables", workspaceId: row.workspaceId });
}, []);
const openRelationships = useCallback((row: CatalogDatabase) => {
const openRelationships = useCallback((row: CatalogDatabase, origin?: HTMLElement) => {
if (!row.configured || !row.id) return;
if (origin) originRef.current = origin;
setDraft(null);
setBaseline("");
setFormSource(null);
@@ -372,25 +373,6 @@ export function DatabaseManagementPage({
setScreen({ kind: "view", workspaceId: row.workspaceId });
}, []);
const addDatabase = useCallback((origin: HTMLElement) => {
const workspace = availableWorkspaces[0];
if (!workspace || !canManage) return;
openForm("add", workspace, origin, false);
}, [availableWorkspaces, canManage, openForm]);
const changeWorkspace = useCallback((workspaceId: string) => {
const workspace = availableWorkspaces.find((row) => row.workspaceId === workspaceId);
if (!workspace) return;
if (dirty && !window.confirm("Discard changes and choose another workspace?")) return;
const nextDraft = draftFrom(workspace);
setDraft(nextDraft);
setBaseline(configurationFingerprint(nextDraft));
setFormSource({ id: workspace.id, version: workspace.version });
setStale(false);
setPartialSecretFailure(null);
setScreen({ kind: "add", workspaceId, workspaceLocked: false });
}, [availableWorkspaces, dirty]);
const changeField = useCallback((field: "databaseName" | "schema", value: string) => {
setDraft((current) => current ? { ...current, [field]: value } : current);
}, []);
@@ -624,7 +606,7 @@ export function DatabaseManagementPage({
showList();
toast.info("This database configuration has already been deleted");
} else if (screen.kind === "edit" && !latest.configured) {
setScreen({ kind: "add", workspaceId: latest.workspaceId, workspaceLocked: true });
setScreen({ kind: "configure", workspaceId: latest.workspaceId });
}
} catch (error) {
toast.error(apiErrorMessage(error));
@@ -669,22 +651,40 @@ export function DatabaseManagementPage({
}, [queryClient]);
const openDescriptionGenerationHistory = useCallback(() => {
const run = observedActiveDescriptionGenerationRun
?? descriptionGenerationRuns[0]
?? activeDescriptionGenerationRun;
const scopedRuns = activeRow ? descriptionGenerationRuns.filter((item) => item.databaseId === activeRow.id) : descriptionGenerationRuns;
const scopedActiveRun = observedActiveDescriptionGenerationRun
&& (!activeRow || observedActiveDescriptionGenerationRun.databaseId === activeRow.id)
? observedActiveDescriptionGenerationRun
: undefined;
const scopedSelectedRun = activeDescriptionGenerationRun
&& (!activeRow || activeDescriptionGenerationRun.databaseId === activeRow.id)
? activeDescriptionGenerationRun
: undefined;
const run = scopedActiveRun
?? scopedRuns[0]
?? scopedSelectedRun;
if (run) setActiveDescriptionGenerationRun(run);
setSensitiveDataSuggestionHistoryDrawerOpen(false);
setDescriptionGenerationDrawerOpen(true);
}, [activeDescriptionGenerationRun, descriptionGenerationRuns, observedActiveDescriptionGenerationRun]);
}, [activeDescriptionGenerationRun, activeRow, descriptionGenerationRuns, observedActiveDescriptionGenerationRun]);
const openSensitiveDataSuggestionHistory = useCallback(() => {
const run = observedActiveSensitiveDataSuggestionRun
?? sensitiveDataSuggestionRuns[0]
?? activeSensitiveDataSuggestionRun;
const scopedRuns = activeRow ? sensitiveDataSuggestionRuns.filter((item) => item.databaseId === activeRow.id) : sensitiveDataSuggestionRuns;
const scopedActiveRun = observedActiveSensitiveDataSuggestionRun
&& (!activeRow || observedActiveSensitiveDataSuggestionRun.databaseId === activeRow.id)
? observedActiveSensitiveDataSuggestionRun
: undefined;
const scopedSelectedRun = activeSensitiveDataSuggestionRun
&& (!activeRow || activeSensitiveDataSuggestionRun.databaseId === activeRow.id)
? activeSensitiveDataSuggestionRun
: undefined;
const run = scopedActiveRun
?? scopedRuns[0]
?? scopedSelectedRun;
if (run) setActiveSensitiveDataSuggestionRun(run);
setDescriptionGenerationDrawerOpen(false);
setSensitiveDataSuggestionHistoryDrawerOpen(true);
}, [activeSensitiveDataSuggestionRun, observedActiveSensitiveDataSuggestionRun, sensitiveDataSuggestionRuns]);
}, [activeRow, activeSensitiveDataSuggestionRun, observedActiveSensitiveDataSuggestionRun, sensitiveDataSuggestionRuns]);
const descriptionGenerationTerminated = useCallback(async (run: DescriptionGenerationRun) => {
await Promise.all([
@@ -902,6 +902,7 @@ export function DatabaseManagementPage({
isLoading={metadataModelsQuery.isLoading}
selectedModel={selectedMetadataModel}
onSelectedModelChange={setSelectedMetadataModel}
variant="compact"
/>
);
@@ -943,6 +944,7 @@ export function DatabaseManagementPage({
/>
<DescriptionGenerationDrawer
open={descriptionGenerationDrawerOpen}
databaseId={activeRow?.id ?? null}
run={activeDescriptionGenerationRun}
modelLabel={metadataModels.find(
(model) => model.id === activeDescriptionGenerationRun?.modelId,
@@ -953,6 +955,7 @@ export function DatabaseManagementPage({
/>
<SensitiveDataSuggestionHistoryDrawer
open={sensitiveDataSuggestionHistoryDrawerOpen}
databaseId={activeRow?.id ?? null}
run={activeSensitiveDataSuggestionRun}
modelLabel={metadataModels.find(
(model) => model.id === activeSensitiveDataSuggestionRun?.modelId,
@@ -980,7 +983,6 @@ export function DatabaseManagementPage({
eyebrow="Catalog configuration"
title={databaseFormTitle(
fleetFormMode,
fleetFormMode === "add" && Boolean(screen.kind === "add" && screen.workspaceLocked),
"drawer",
)}
description={`${activeRow.workspaceName} · ${activeRow.workspaceId}`}
@@ -993,8 +995,6 @@ export function DatabaseManagementPage({
presentation="drawer"
mode={fleetFormMode}
row={activeRow}
availableWorkspaces={availableWorkspaces}
workspaceLocked={fleetFormMode === "add" && Boolean(screen.kind === "add" && screen.workspaceLocked)}
draft={draft}
dirty={dirty}
canManage={canManage}
@@ -1005,7 +1005,6 @@ export function DatabaseManagementPage({
partialSecretFailure={partialSecretFailure}
headingRef={formHeadingRef}
onBack={backToList}
onWorkspaceChange={changeWorkspace}
onFieldChange={changeField}
onTransportChange={changeTransport}
onBindingChange={changeBinding}
@@ -1020,6 +1019,8 @@ export function DatabaseManagementPage({
onOpenRelationships={() => openRelationships(activeRow)}
onSync={(scope) => void syncDatabase(scope)}
onOpenSync={() => openSync(activeRow)}
onOpenDescriptionHistory={openDescriptionGenerationHistory}
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
activeSyncRun={currentActiveRun}
/>
</FleetLedgerDrawer>
@@ -1040,10 +1041,11 @@ export function DatabaseManagementPage({
onNestedNavigationChange={setTableNestedNavigationActive}
onRunStarted={rememberSyncRun}
onOpenSync={() => openSync(activeRow)}
onOpenDescriptionHistory={openDescriptionGenerationHistory}
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
onDescriptionGenerationRunStarted={rememberDescriptionGenerationRun}
onSuggestSensitive={suggestActiveDatabaseSensitiveFields}
onCatalogMetricsChanged={invalidateCatalogMetrics}
metadataModelControl={fleetMetadataModelControl}
/>
) : screen.kind === "relationships" && relationshipsVisible ? (
<DatabaseRelationships
@@ -1055,6 +1057,10 @@ export function DatabaseManagementPage({
onOpenTables={() => openTables(activeRow)}
onRunStarted={rememberSyncRun}
onOpenSync={() => openSync(activeRow)}
onOpenDescriptionHistory={openDescriptionGenerationHistory}
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
onCatalogMetricsChanged={invalidateCatalogMetrics}
onNavigationStateChange={setTablesNavigationState}
/>
) : isError && rows.length === 0 ? (
<div className="grid h-full place-items-center p-6">
@@ -1078,6 +1084,7 @@ export function DatabaseManagementPage({
onSearchChange={setSearch}
onView={viewRow}
onOpenTables={(row, origin) => openTables(row, origin)}
onOpenRelationships={(row, origin) => openRelationships(row, origin)}
onEdit={editRow}
onDelete={deleteRow}
onOpenSync={openSync}
@@ -1088,7 +1095,6 @@ export function DatabaseManagementPage({
onGenerateDescriptions={generateDatabaseDescriptions}
onSuggestSensitive={suggestDatabaseSensitiveFields}
onDeleteMetadataSelected={deleteSelectedMetadata}
metadataModelControl={fleetMetadataModelControl}
onRefresh={refreshList}
/>
);
@@ -1118,28 +1124,7 @@ export function DatabaseManagementPage({
)}
actions={(
<>
<Button type="button" variant="outline" size="sm" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Description history" data-fleet-action="secondary" onClick={openDescriptionGenerationHistory}>
<History aria-hidden="true" /> Description history
{observedActiveDescriptionGenerationRun ? <span className="size-1.5 rounded-full bg-warning" aria-hidden="true" /> : null}
</Button>
<Button type="button" variant="outline" size="sm" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Sensitive history" data-fleet-action="secondary" onClick={openSensitiveDataSuggestionHistory}>
<History aria-hidden="true" /> Sensitive history
{observedActiveSensitiveDataSuggestionRun ? <span className="size-1.5 rounded-full bg-warning" aria-hidden="true" /> : null}
</Button>
{(screen.kind === "list" || formVisible) ? (
<Button
type="button"
size="sm"
className="thot-fleet-nav-action thot-fleet-nav-action--below"
data-tooltip="Add database"
data-fleet-action="primary"
disabled={!canManage || availableWorkspaces.length === 0}
title={availableWorkspaces.length === 0 ? "Every available workspace is already configured" : undefined}
onClick={(event) => addDatabase(event.currentTarget)}
>
<Plus aria-hidden="true" /> Add database
</Button>
) : null}
{fleetMetadataModelControl}
</>
)}
/>
@@ -1205,40 +1190,16 @@ export function DatabaseManagementPage({
selectedModel={selectedMetadataModel}
onSelectedModelChange={setSelectedMetadataModel}
/>
<Button
type="button"
variant="outline"
className="whitespace-nowrap disabled:opacity-70"
aria-label="Run descriptions generation history"
disabled={!canManage}
title={observedActiveDescriptionGenerationRun
? "Description generation is active"
: "View description generation history"}
onClick={openDescriptionGenerationHistory}
>
<History />
Run descriptions generation history
{observedActiveDescriptionGenerationRun ? (
<span aria-hidden="true" className="size-1.5 rounded-full bg-primary" />
) : null}
</Button>
<Button
type="button"
variant="outline"
className="whitespace-nowrap disabled:opacity-70"
aria-label="Run sensitive suggestions history"
disabled={!canManage}
title={observedActiveSensitiveDataSuggestionRun
? "Sensitive suggestion generation is active"
: "View sensitive suggestion history"}
onClick={openSensitiveDataSuggestionHistory}
>
<History />
Run sensitive suggestions history
{observedActiveSensitiveDataSuggestionRun ? (
<span aria-hidden="true" className="size-1.5 rounded-full bg-primary" />
) : null}
</Button>
{screen.kind === "list" ? <>
<Button type="button" variant="outline" className="whitespace-nowrap disabled:opacity-70" aria-label="Run descriptions generation history" disabled={!canManage} title="View description generation history" onClick={openDescriptionGenerationHistory}>
<History /> Run descriptions generation history
</Button>
<Button type="button" variant="outline" className="whitespace-nowrap disabled:opacity-70" aria-label="Run sensitive suggestions history" disabled={!canManage} title={observedActiveSensitiveDataSuggestionRun ? "Sensitive suggestion generation is active" : "View sensitive suggestion history"} onClick={openSensitiveDataSuggestionHistory}>
<History />
{observedActiveSensitiveDataSuggestionRun ? <span aria-hidden="true" className="size-2 rounded-full bg-primary" /> : null}
Run sensitive suggestions history
</Button>
</> : null}
</div>
{screen.kind === "list" ? (
<div
@@ -1249,14 +1210,6 @@ export function DatabaseManagementPage({
<Button type="button" variant="outline" disabled={isFetching} onClick={() => void refreshList()}>
<RefreshCw className={isFetching ? "animate-spin" : ""} /> Refresh
</Button>
<Button
type="button"
disabled={!canManage || availableWorkspaces.length === 0}
title={availableWorkspaces.length === 0 ? "Every available workspace is already configured" : undefined}
onClick={(event) => addDatabase(event.currentTarget)}
>
<Plus /> Add database
</Button>
</div>
) : null}
</div>
@@ -1308,8 +1261,6 @@ export function DatabaseManagementPage({
<DatabaseForm
mode={screen.kind as DatabaseFormMode}
row={activeRow}
availableWorkspaces={availableWorkspaces}
workspaceLocked={screen.kind === "add" && Boolean(screen.workspaceLocked)}
draft={draft}
dirty={dirty}
canManage={canManage}
@@ -1320,7 +1271,6 @@ export function DatabaseManagementPage({
partialSecretFailure={partialSecretFailure}
headingRef={formHeadingRef}
onBack={backToList}
onWorkspaceChange={changeWorkspace}
onFieldChange={changeField}
onTransportChange={changeTransport}
onBindingChange={changeBinding}
@@ -1335,6 +1285,8 @@ export function DatabaseManagementPage({
onOpenRelationships={() => openRelationships(activeRow)}
onSync={(scope) => void syncDatabase(scope)}
onOpenSync={() => openSync(activeRow)}
onOpenDescriptionHistory={openDescriptionGenerationHistory}
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
activeSyncRun={currentActiveRun}
/>
) : null}
@@ -1352,6 +1304,8 @@ export function DatabaseManagementPage({
onNavigationStateChange={setTablesNavigationState}
onRunStarted={rememberSyncRun}
onOpenSync={() => openSync(activeRow)}
onOpenDescriptionHistory={openDescriptionGenerationHistory}
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
onDescriptionGenerationRunStarted={rememberDescriptionGenerationRun}
onSuggestSensitive={suggestActiveDatabaseSensitiveFields}
onCatalogMetricsChanged={invalidateCatalogMetrics}
@@ -1367,6 +1321,10 @@ export function DatabaseManagementPage({
onOpenTables={() => openTables(activeRow)}
onRunStarted={rememberSyncRun}
onOpenSync={() => openSync(activeRow)}
onOpenDescriptionHistory={openDescriptionGenerationHistory}
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
onCatalogMetricsChanged={invalidateCatalogMetrics}
onNavigationStateChange={setTablesNavigationState}
/>
) : null}
</div>
@@ -1381,6 +1339,7 @@ export function DatabaseManagementPage({
/>
<DescriptionGenerationDrawer
open={descriptionGenerationDrawerOpen}
databaseId={activeRow?.id ?? null}
run={activeDescriptionGenerationRun}
modelLabel={metadataModels.find(
(model) => model.id === activeDescriptionGenerationRun?.modelId,
@@ -1391,6 +1350,7 @@ export function DatabaseManagementPage({
/>
<SensitiveDataSuggestionHistoryDrawer
open={sensitiveDataSuggestionHistoryDrawerOpen}
databaseId={activeRow?.id ?? null}
run={activeSensitiveDataSuggestionRun}
modelLabel={metadataModels.find(
(model) => model.id === activeSensitiveDataSuggestionRun?.modelId,
@@ -249,6 +249,7 @@ export function CatalogSyncDrawer({
key={item.id}
type="button"
aria-current={item.id === run?.id ? "true" : undefined}
data-status={item.state}
className="flex w-full items-center justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 aria-[current=true]:bg-primary/8"
onClick={() => onRunChange(item)}
>
@@ -1,4 +1,4 @@
import { useEffect, useMemo, useRef, useState, type ReactNode } from "react";
import { useEffect, useMemo, useRef, useState } from "react";
import { Menu } from "@base-ui/react/menu";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { AgGridReact } from "ag-grid-react";
@@ -20,6 +20,7 @@ import {
import type { DatabaseNavigationState } from "./model";
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
import { FleetLedgerDrawer } from "./FleetLedgerShell";
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
interface Props {
databaseId: string;
@@ -34,7 +35,6 @@ interface Props {
bindingReady?: boolean;
catalogOperationActive?: boolean;
onCatalogMetricsChanged?: () => void | Promise<void>;
metadataModelControl?: ReactNode;
presentation?: "legacy" | "fleet";
}
@@ -99,7 +99,6 @@ export function DatabaseColumns({
bindingReady = true,
catalogOperationActive = false,
onCatalogMetricsChanged,
metadataModelControl,
presentation = "legacy",
}: Props) {
const queryClient = useQueryClient();
@@ -307,7 +306,7 @@ export function DatabaseColumns({
: selectedIds.length === 0
? "Select at least one column."
: !selectedMetadataModel
? "Choose a metadata model first."
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
: descriptionGenerationActive || catalogOperationActive
? "Wait for the active catalog operation to finish."
: busy
@@ -341,7 +340,7 @@ export function DatabaseColumns({
: selectedIds.length === 0
? "Select at least one column."
: !selectedMetadataModel
? "Choose a metadata model first."
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
: descriptionGenerationActive || catalogOperationActive
? "Wait for the active catalog operation to finish."
: busy
@@ -468,7 +467,6 @@ export function DatabaseColumns({
busyLabel={sensitiveAction === "suggest" ? "Suggesting…" : sensitiveAction === "save" ? "Saving…" : "Running…"}
onRun={runFleetAction}
onClear={clearSelection}
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
label="Column action"
/>
</>
@@ -501,7 +499,6 @@ export function DatabaseColumns({
busyLabel={sensitiveAction === "save" ? "Saving…" : "Running…"}
onRun={runFleetAction}
onClear={clearSelection}
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
label="Column action"
/>
<input className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search columns" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
@@ -14,6 +14,9 @@ const database: CatalogDatabase = {
workspaceId: "psd-clinical",
workspaceName: "Policlinico San Donato",
workspaceAvailable: true,
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
runtimeBinding: { transport: "postgres_direct", configurationState: "ready", sessionTransportSupported: true },
configured: true,
engine: "postgres",
databaseName: "warehouse",
@@ -113,8 +113,6 @@ function SecretField({
interface DatabaseFormProps {
mode: DatabaseFormMode;
row: CatalogDatabase;
availableWorkspaces: CatalogDatabase[];
workspaceLocked: boolean;
draft: DatabaseFormDraft;
dirty: boolean;
canManage: boolean;
@@ -125,7 +123,6 @@ interface DatabaseFormProps {
partialSecretFailure: string | null;
headingRef: RefObject<HTMLHeadingElement>;
onBack: () => void;
onWorkspaceChange: (workspaceId: string) => void;
onFieldChange: (field: "databaseName" | "schema", value: string) => void;
onTransportChange: (transport: DatabaseTransport) => void;
onBindingChange: <K extends keyof DatabaseBinding>(key: K, value: DatabaseBinding[K]) => void;
@@ -140,16 +137,17 @@ interface DatabaseFormProps {
onOpenRelationships: () => void;
onSync: (scope: CatalogSyncScope) => void;
onOpenSync: () => void;
onOpenDescriptionHistory?: () => void;
onOpenSensitiveHistory?: () => void;
activeSyncRun?: CatalogSyncRun;
presentation?: "page" | "drawer";
}
export function databaseFormTitle(
mode: DatabaseFormMode,
workspaceLocked: boolean,
presentation: "page" | "drawer" = "page",
): string {
if (mode === "add") return workspaceLocked ? "Edit database" : "Add database";
if (mode === "configure") return "Configure catalog";
if (mode === "edit") return "Edit database";
if (mode === "delete") return presentation === "drawer" ? "Remove configuration" : "Delete database";
return "Database details";
@@ -158,8 +156,6 @@ export function databaseFormTitle(
export function DatabaseForm({
mode,
row,
availableWorkspaces,
workspaceLocked,
draft,
dirty,
canManage,
@@ -170,7 +166,6 @@ export function DatabaseForm({
partialSecretFailure,
headingRef,
onBack,
onWorkspaceChange,
onFieldChange,
onTransportChange,
onBindingChange,
@@ -185,12 +180,14 @@ export function DatabaseForm({
onOpenRelationships,
onSync,
onOpenSync,
onOpenDescriptionHistory,
onOpenSensitiveHistory,
activeSyncRun,
presentation = "page",
}: DatabaseFormProps) {
const title = databaseFormTitle(mode, workspaceLocked, presentation);
const title = databaseFormTitle(mode, presentation);
const readOnly = mode === "view" || mode === "delete";
const editable = mode === "add" || mode === "edit";
const editable = mode === "configure" || mode === "edit";
const busy = busyAction !== null;
const testDisabled = !canManage
|| !row.configured
@@ -268,27 +265,9 @@ export function DatabaseForm({
) : null}
<div className="grid gap-4 md:grid-cols-2">
{mode === "add" ? (
<Field label="Workspace">
<select
className={inputClass}
aria-label="Workspace"
value={draft.workspaceId}
disabled={workspaceLocked || busy}
onChange={(event) => onWorkspaceChange(event.target.value)}
>
{availableWorkspaces.map((workspace) => (
<option key={workspace.workspaceId} value={workspace.workspaceId}>
{workspace.workspaceName}
</option>
))}
</select>
</Field>
) : (
<Field label="Workspace">
<input className={inputClass} value={row.workspaceName} readOnly />
</Field>
)}
<Field label="Workspace">
<input className={inputClass} value={row.workspaceName} readOnly />
</Field>
<Field label="Workspace ID">
<input className={`${inputClass} font-mono text-xs`} value={row.workspaceId} readOnly />
</Field>
@@ -420,13 +399,13 @@ export function DatabaseForm({
<TestTube2 /> {busyAction === "test" ? "Testing…" : "Test connection"}
</Button>
) : null}
{mode === "add" || mode === "edit" ? (
{mode === "configure" || mode === "edit" ? (
<Button type="submit" disabled={saveDisabled}>
<Save />
{busyAction === "save"
? "Saving…"
: mode === "add"
? workspaceLocked ? "Save database" : "Add database"
: mode === "configure"
? "Save catalog configuration"
: "Save changes"}
</Button>
) : null}
@@ -459,6 +438,16 @@ export function DatabaseForm({
{activeSyncRun ? <RefreshCw className="animate-spin" /> : <History />} Sync history
</Button>
) : null}
{mode === "view" && row.configured && onOpenDescriptionHistory ? (
<Button type="button" variant="outline" size="sm" onClick={onOpenDescriptionHistory}>
<History /> Description history
</Button>
) : null}
{mode === "view" && row.configured && onOpenSensitiveHistory ? (
<Button type="button" variant="outline" size="sm" onClick={onOpenSensitiveHistory}>
<History /> Sensitive history
</Button>
) : null}
{mode === "view" ? (
<DatabaseSyncMenu
disabled={!canManage || !bindingReady || busy || Boolean(activeSyncRun)}
@@ -466,6 +455,8 @@ export function DatabaseForm({
onSelect={onSync}
/>
) : null}
{mode === "view" && row.configured && onOpenDescriptionHistory ? <Button type="button" variant="outline" onClick={onOpenDescriptionHistory}><History /> Description history</Button> : null}
{mode === "view" && row.configured && onOpenSensitiveHistory ? <Button type="button" variant="outline" onClick={onOpenSensitiveHistory}><History /> Sensitive history</Button> : null}
</div>
</div>
{row.lastTestedAt ? (
@@ -1,4 +1,4 @@
import { useEffect, useMemo, useRef, useState, type ReactNode, type RefObject } from "react";
import { useEffect, useMemo, useRef, useState, type RefObject } from "react";
import { Menu } from "@base-ui/react/menu";
import { AgGridReact } from "ag-grid-react";
import {
@@ -9,7 +9,7 @@ import {
} from "ag-grid-community";
import "ag-grid-community/styles/ag-grid.css";
import "ag-grid-community/styles/ag-theme-alpine.css";
import { ChevronDown, Eye, History, Info, Pencil, RefreshCw, Rows3, Sparkles, Trash2, X } from "lucide-react";
import { ChevronDown, Eye, History, Info, Link2, Pencil, RefreshCw, Rows3, Sparkles, Trash2, X } from "lucide-react";
import { Button } from "../../components/ui/button";
import type {
CatalogDatabase,
@@ -17,9 +17,9 @@ import type {
CatalogSyncScope,
DescriptionGenerationScope,
} from "../../api/catalog-databases";
import { statusLabel } from "./model";
import { databaseSyncItemClass, databaseSyncScopes } from "./DatabaseSyncMenu";
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
ModuleRegistry.registerModules([AllCommunityModule]);
@@ -33,6 +33,7 @@ interface DatabaseGridProps {
onSearchChange: (value: string) => void;
onView: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
onOpenTables?: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
onOpenRelationships?: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
onEdit: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
onDelete: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
onOpenSync: (row: CatalogDatabase) => void;
@@ -49,7 +50,6 @@ interface DatabaseGridProps {
rows: CatalogDatabase[],
target: CatalogDatabaseMetadataDeleteTarget,
) => Promise<void>;
metadataModelControl?: ReactNode;
onRefresh?: () => void | Promise<void>;
presentation?: "legacy" | "fleet";
}
@@ -59,6 +59,7 @@ interface DatabaseGridContext {
presentation: "legacy" | "fleet";
onView: DatabaseGridProps["onView"];
onOpenTables?: DatabaseGridProps["onOpenTables"];
onOpenRelationships?: DatabaseGridProps["onOpenRelationships"];
onEdit: DatabaseGridProps["onEdit"];
onDelete: DatabaseGridProps["onDelete"];
onOpenSync: DatabaseGridProps["onOpenSync"];
@@ -91,18 +92,120 @@ function useCompactViewport(): boolean {
return compact;
}
function StatusPill({ row }: { row: CatalogDatabase }) {
const tone = !row.workspaceAvailable
? "border-destructive/30 bg-destructive/10 text-destructive"
: !row.configured
? "border-border bg-muted text-muted-foreground"
: row.connectionStatus === "reachable"
? "border-emerald-300/70 bg-emerald-50 text-emerald-800 dark:bg-emerald-950/30 dark:text-emerald-300"
: row.connectionStatus === "failed"
? "border-destructive/30 bg-destructive/10 text-destructive"
: "border-amber-300/70 bg-amber-50 text-amber-900 dark:bg-amber-950/30 dark:text-amber-300";
type StateTone = "neutral" | "info" | "success" | "warning" | "danger";
return <div className="flex items-center gap-1.5"><span className={`inline-flex max-w-full truncate rounded-full border px-2 py-0.5 text-[11px] font-semibold ${tone}`}>{statusLabel(row)}</span>{row.activeSyncRun ? <span className="inline-flex rounded-full border border-primary/30 bg-primary/8 px-2 py-0.5 text-[11px] font-semibold text-primary">Syncing</span> : null}</div>;
const stateToneClass: Record<StateTone, string> = {
neutral: "bg-muted-foreground/55",
info: "bg-sky-500",
success: "bg-emerald-500",
warning: "bg-amber-500",
danger: "bg-destructive",
};
function StateCell({
label,
detail,
tone,
detailClassName = "text-muted-foreground",
}: {
label: string;
detail: string;
tone: StateTone;
detailClassName?: string;
}) {
return (
<div className="flex h-full min-w-0 flex-col justify-center gap-0.5 py-1 leading-tight">
<span className="flex min-w-0 items-center gap-1.5 text-xs font-semibold text-foreground">
<span className={`size-1.5 shrink-0 rounded-full ${stateToneClass[tone]}`} aria-hidden="true" />
<span className="truncate">{label}</span>
</span>
<span className={`truncate pl-3 text-[11px] ${detailClassName}`}>{detail}</span>
</div>
);
}
function evidenceStatus(row: CatalogDatabase): { label: string; tone: StateTone } {
if (!row.workspaceEvidence) {
return { label: "Evidence state unavailable", tone: "warning" };
}
switch (row.workspaceEvidence.state) {
case "materialized_current_revision":
return { label: "Evidence materialized", tone: "success" };
case "configuration_required":
return { label: "Evidence credentials required", tone: "warning" };
case "configured_unverified":
return { label: "Evidence configured, unverified", tone: "info" };
case "workspace_unavailable":
return { label: "Evidence unavailable", tone: "danger" };
default:
return { label: "No Evidence declared", tone: "neutral" };
}
}
function RevisionEvidenceCell({ row }: { row: CatalogDatabase }) {
if (!row.workspaceRevision) {
return <StateCell label="Workspace missing" detail="Revision and Evidence unavailable" tone="danger" />;
}
const evidence = evidenceStatus(row);
const source = row.workspaceEvidence?.sourceType ? ` · ${row.workspaceEvidence.sourceType}` : "";
return (
<StateCell
label={`Active revision ${row.workspaceRevision.commit.slice(0, 7)}`}
detail={`${evidence.label}${source}`}
tone={evidence.tone}
/>
);
}
function transportLabel(row: CatalogDatabase): string {
const transport = row.runtimeBinding?.transport;
if (transport === "postgres_direct") return "Direct PostgreSQL";
if (transport === "rest_api") return "REST API";
if (transport === "ssh_tunnel") return "SSH tunnel";
return "No runtime binding";
}
function RuntimeBindingCell({ row }: { row: CatalogDatabase }) {
if (!row.runtimeBinding) {
return <StateCell label="Unavailable" detail="Workspace is not in the active revision" tone="danger" />;
}
if (!row.runtimeBinding.sessionTransportSupported) {
return <StateCell label="Unsupported for NL→SQL" detail={transportLabel(row)} tone="danger" />;
}
if (row.runtimeBinding.configurationState === "configuration_required") {
return <StateCell label="Configuration required" detail={transportLabel(row)} tone="warning" />;
}
return <StateCell label="Ready" detail={transportLabel(row)} tone="success" />;
}
function catalogConnectionDetail(row: CatalogDatabase): { label: string; className?: string } {
if (row.activeSyncRun) return { label: "Metadata sync in progress", className: "text-primary" };
if (row.testedVersion !== undefined && row.testedVersion !== row.version) {
return { label: "Connection retest required", className: "text-amber-700 dark:text-amber-300" };
}
if (row.connectionStatus === "reachable") return { label: "Connection reachable" };
if (row.connectionStatus === "failed") {
return { label: "Connection test failed", className: "text-destructive" };
}
return { label: "Connection not tested" };
}
function MetadataCatalogCell({ row }: { row: CatalogDatabase }) {
if (!row.workspaceAvailable) {
return <StateCell label="Orphaned configuration" detail="Workspace missing from repository" tone="danger" />;
}
if (!row.configured) {
return <StateCell label="Not configured" detail="Configure this workspace" tone="neutral" />;
}
const connection = catalogConnectionDetail(row);
return (
<StateCell
label="Configured"
detail={connection.label}
detailClassName={connection.className}
tone="success"
/>
);
}
function DatabaseActionsCell({
@@ -111,11 +214,48 @@ function DatabaseActionsCell({
}: ICellRendererParams<CatalogDatabase, unknown, DatabaseGridContext>) {
if (!data || !context) return null;
if (data.workspaceAvailable && !data.configured) {
const configureReasonId = `database-configure-reason-${data.workspaceId}`;
return (
<div className="flex h-full items-center justify-end gap-1.5" onClick={(event) => event.stopPropagation()}>
<Button
type="button"
variant="ghost"
size="icon-lg"
className={context.presentation === "fleet" ? "thot-fleet-icon-action" : undefined}
data-tooltip={context.presentation === "fleet" ? "Details" : undefined}
title={context.presentation === "legacy" ? `View ${data.workspaceName}` : undefined}
aria-label={`View ${data.workspaceName}`}
onClick={(event) => context.onView(data, event.currentTarget)}
>
{context.presentation === "fleet" ? <Info aria-hidden="true" /> : <Eye aria-hidden="true" />}
</Button>
<Button
type="button"
size="sm"
className="whitespace-nowrap"
aria-label={`Configure catalog for ${data.workspaceName}`}
aria-describedby={!context.canManage ? configureReasonId : undefined}
disabled={!context.canManage}
onClick={(event) => context.onEdit(data, event.currentTarget)}
>
Configure catalog
</Button>
{!context.canManage ? (
<span id={configureReasonId} className="sr-only">
Database management permission is required.
</span>
) : null}
</div>
);
}
const editDisabled = !context.canManage || !data.workspaceAvailable;
const deleteDisabled = !context.canManage || !data.configured;
const editLabel = `Edit ${data.workspaceName}`;
const editReasonId = `database-edit-reason-${data.workspaceId}`;
const deleteReasonId = `database-delete-reason-${data.workspaceId}`;
const relationshipsReasonId = `database-relationships-reason-${data.workspaceId}`;
return (
<div className="flex h-full items-center justify-end gap-0.5" onClick={(event) => event.stopPropagation()}>
@@ -138,6 +278,26 @@ function DatabaseActionsCell({
</Button>
</span>
) : null}
{context.presentation === "fleet" && context.onOpenRelationships ? (
<span className="thot-fleet-icon-action inline-flex" data-tooltip="Relationships">
<Button
type="button"
variant="ghost"
size="icon-lg"
aria-label={`View relationships for ${data.workspaceName}`}
aria-describedby={!data.configured || !data.id ? relationshipsReasonId : undefined}
disabled={!data.configured || !data.id}
onClick={(event) => context.onOpenRelationships?.(data, event.currentTarget)}
>
<Link2 aria-hidden="true" />
</Button>
{!data.configured || !data.id ? (
<span id={relationshipsReasonId} className="sr-only">
Save this database configuration before managing relationships.
</span>
) : null}
</span>
) : null}
<Button
type="button"
variant="ghost"
@@ -193,6 +353,7 @@ export function DatabaseGrid({
onSearchChange,
onView,
onOpenTables,
onOpenRelationships,
onEdit,
onDelete,
onOpenSync,
@@ -203,7 +364,6 @@ export function DatabaseGrid({
onGenerateDescriptions,
onSuggestSensitive,
onDeleteMetadataSelected,
metadataModelControl,
onRefresh,
presentation = "legacy",
}: DatabaseGridProps) {
@@ -218,8 +378,8 @@ export function DatabaseGrid({
Extract<DescriptionGenerationScope, "all" | "missing"> | null
>(null);
const context = useMemo<DatabaseGridContext>(
() => ({ canManage, presentation, onView, onOpenTables, onEdit, onDelete, onOpenSync }),
[canManage, presentation, onView, onOpenTables, onEdit, onDelete, onOpenSync],
() => ({ canManage, presentation, onView, onOpenTables, onOpenRelationships, onEdit, onDelete, onOpenSync }),
[canManage, presentation, onView, onOpenTables, onOpenRelationships, onEdit, onDelete, onOpenSync],
);
const columnDefs = useMemo<ColDef<CatalogDatabase>[]>(() => {
@@ -230,19 +390,38 @@ export function DatabaseGrid({
flex: compact ? undefined : 1.35,
width: compact ? 112 : undefined,
};
const revisionEvidence: ColDef<CatalogDatabase> = {
headerName: "Revision / Evidence",
minWidth: 215,
flex: 1.2,
valueGetter: ({ data }) => {
if (!data) return "";
const revision = data.workspaceRevision?.commit.slice(0, 7) ?? "workspace missing";
return `${revision} ${evidenceStatus(data).label}`;
},
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => (
data ? <RevisionEvidenceCell row={data} /> : null
),
};
const runtimeBinding: ColDef<CatalogDatabase> = {
headerName: "NL→SQL runtime",
minWidth: 175,
flex: 1,
valueGetter: ({ data }) => {
if (!data?.runtimeBinding) return "Unavailable";
if (!data.runtimeBinding.sessionTransportSupported) return "Unsupported for NL→SQL";
return data.runtimeBinding.configurationState === "ready" ? "Ready" : "Configuration required";
},
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => (
data ? <RuntimeBindingCell row={data} /> : null
),
};
const database: ColDef<CatalogDatabase> = {
field: "databaseName", headerName: "Database", minWidth: 145, flex: 1,
};
const schema: ColDef<CatalogDatabase> = {
field: "schema", headerName: "Schema", minWidth: 140, flex: 0.9,
};
const transport: ColDef<CatalogDatabase> = {
field: "binding.transport",
headerName: "Transport",
minWidth: 135,
flex: 0.9,
valueFormatter: ({ value }) => String(value ?? "").replaceAll("_", " "),
};
const endpoint: ColDef<CatalogDatabase> = {
headerName: "Endpoint",
minWidth: 190,
@@ -254,15 +433,17 @@ export function DatabaseGrid({
return data.binding.host ? `${data.binding.host}:${data.binding.port ?? 5432}` : "";
},
};
const status: ColDef<CatalogDatabase> = {
headerName: "Status",
minWidth: compact ? 96 : 130,
width: compact ? 96 : undefined,
flex: compact ? undefined : 0.8,
cellClass: compact ? "thot-database-status-cell" : undefined,
valueGetter: ({ data }) => (data ? statusLabel(data) : ""),
const metadataCatalog: ColDef<CatalogDatabase> = {
headerName: "Metadata Catalog",
minWidth: 185,
flex: 1.05,
valueGetter: ({ data }) => {
if (!data) return "";
if (!data.workspaceAvailable) return "Orphaned configuration";
return data.configured ? `Configured ${catalogConnectionDetail(data).label}` : "Not configured";
},
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => (
data ? <StatusPill row={data} /> : null
data ? <MetadataCatalogCell row={data} /> : null
),
};
const updated: ColDef<CatalogDatabase> = {
@@ -272,7 +453,7 @@ export function DatabaseGrid({
flex: 0.9,
valueFormatter: ({ value }) => value ? new Date(String(value)).toLocaleString() : "",
};
const actionsWidth = presentation === "fleet" ? 196 : compact ? 148 : 164;
const actionsWidth = presentation === "fleet" ? 232 : 210;
const actions: ColDef<CatalogDatabase> = {
colId: "actions",
headerName: "Actions",
@@ -291,8 +472,8 @@ export function DatabaseGrid({
};
return compact
? [workspace, status, database, schema, transport, endpoint, updated, actions]
: [workspace, database, schema, transport, endpoint, status, updated, actions];
? [workspace, revisionEvidence, runtimeBinding, metadataCatalog, database, schema, endpoint, updated, actions]
: [workspace, revisionEvidence, runtimeBinding, database, schema, endpoint, metadataCatalog, updated, actions];
}, [compact, presentation]);
const configuredCount = rows.filter((row) => row.configured).length;
@@ -332,14 +513,14 @@ export function DatabaseGrid({
: undefined);
const generationReason = permissionReason
?? (selectedRows.length !== 1 ? "Select exactly one database" : undefined)
?? (!selectedMetadataModel ? "Choose a metadata model in the generation flow" : undefined)
?? (!selectedMetadataModel ? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE : undefined)
?? (descriptionGenerationActive ? "Wait for active description generation" : undefined)
?? (selectedRows.some((row) => !row.configured || !row.id || row.activeSyncRun)
? "The selected database must be configured and idle"
: undefined);
const sensitiveReason = permissionReason
?? (selectedRows.length !== 1 ? "Select exactly one database" : undefined)
?? (!selectedMetadataModel ? "Choose a metadata model in the suggestion flow" : undefined)
?? (!selectedMetadataModel ? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE : undefined)
?? (descriptionGenerationActive ? "Wait for the active metadata operation" : undefined)
?? (selectedRows.some((row) => !row.configured || !row.id || row.activeSyncRun)
? "The selected database must be configured and idle"
@@ -538,12 +719,6 @@ export function DatabaseGrid({
busyLabel={action === "suggest" ? "Suggesting…" : "Running…"}
selectRef={actionSelectRef}
onRun={runFleetAction}
renderContext={(selectedAction) => (
metadataModelControl
&& ["generate-all", "generate-missing", "suggest-sensitive"].includes(selectedAction)
? <div className="thot-fleet-metadata-model">{metadataModelControl}</div>
: null
)}
onClear={() => {
gridRef.current?.api.deselectAll();
setSelectedRows([]);
@@ -631,7 +806,7 @@ export function DatabaseGrid({
</Button>
) : null}</>}
<span className="whitespace-nowrap text-xs tabular-nums text-muted-foreground">
{configuredCount}/{rows.length}
{configuredCount}/{rows.length} catalogs configured
</span>
{isFetching && !isLoading ? (
<span className="text-xs text-muted-foreground" role="status">Refreshing</span>
@@ -660,7 +835,7 @@ export function DatabaseGrid({
if (pendingGenerationScope && action === null) setPendingGenerationScope(null);
setSelectedRows(api.getSelectedRows());
}}
rowHeight={44}
rowHeight={52}
headerHeight={38}
animateRows={false}
/>
@@ -0,0 +1,379 @@
import { render, screen, waitFor, within } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { http, HttpResponse } from "msw";
import { expect, test, vi } from "vitest";
import type {
CatalogColumn,
CatalogDatabase,
CatalogRelationship,
CatalogTable,
} from "../../api/catalog-databases";
import { Toaster } from "../../components/ui/sonner";
import { server } from "../../test/msw";
import { DatabaseRelationships } from "./DatabaseRelationships";
const databaseId = "11111111-1111-4111-8111-111111111111";
const database: CatalogDatabase = {
id: databaseId,
workspaceId: "psd",
workspaceName: "Policlinico San Donato",
workspaceAvailable: true,
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
runtimeBinding: { transport: "postgres_direct", configurationState: "ready", sessionTransportSupported: true },
configured: true,
engine: "postgres",
databaseName: "analytics",
schema: "public",
binding: { transport: "postgres_direct", port: 5432 },
connectionStatus: "reachable",
testedVersion: 3,
version: 3,
createdAt: "2026-08-31T12:00:00.000Z",
updatedAt: "2026-08-31T12:00:00.000Z",
activeSyncRun: undefined,
secrets: {
password: false,
apiKey: false,
sshPrivateKey: false,
sshPrivateKeyPassphrase: false,
sshKnownHosts: false,
tlsCa: false,
},
};
function relationship(
id: string,
origin: CatalogRelationship["origin"],
status: CatalogRelationship["status"],
sourceTableName: string,
): CatalogRelationship {
const physical = origin === "physical";
return {
id,
databaseId,
constraintName: physical ? `${sourceTableName}_user_id_fkey` : null,
sourceTableId: `${id}-source-table`,
sourceTableName,
targetTableId: `${id}-target-table`,
targetTableName: "users",
updateRule: physical ? "NO ACTION" : null,
deleteRule: physical ? "CASCADE" : null,
deferrable: false,
initiallyDeferred: false,
columns: [{
position: 1,
sourceColumnId: `${id}-source-column`,
sourceColumnName: "user_id",
targetColumnId: `${id}-target-column`,
targetColumnName: "id",
}],
origin,
status,
lastSyncedDatabaseVersion: physical ? 3 : null,
lastSyncedAt: physical ? "2026-08-31T12:00:00.000Z" : null,
createdAt: "2026-08-31T12:00:00.000Z",
updatedAt: "2026-08-31T12:00:00.000Z",
};
}
function renderRelationships(rows: CatalogRelationship[], canManage = true) {
server.use(http.get(
"/api/catalog/databases/:databaseId/relationships",
() => HttpResponse.json(rows),
));
const client = new QueryClient({ defaultOptions: { queries: { retry: false } } });
const callbacks = {
onBackToDatabases: vi.fn(),
onOpenOverview: vi.fn(),
onOpenTables: vi.fn(),
onRunStarted: vi.fn(),
onOpenSync: vi.fn(),
};
const view = render(
<QueryClientProvider client={client}>
<DatabaseRelationships
presentation="fleet"
database={database}
canManage={canManage}
{...callbacks}
/>
<Toaster duration={Infinity} />
</QueryClientProvider>,
);
return { ...view, client, callbacks };
}
test("shows one relationship map and filters active, excluded, and all relationships", async () => {
const user = userEvent.setup();
renderRelationships([
relationship("physical", "physical", "active", "visits"),
relationship("generated", "generated", "active", "orders"),
relationship("manual", "manual", "active", "payments"),
relationship("excluded", "generated", "excluded", "legacy_orders"),
]);
expect(await screen.findByText("visits")).toBeVisible();
expect(screen.getByText("orders")).toBeVisible();
expect(screen.getByText("payments")).toBeVisible();
expect(screen.queryByText("legacy_orders")).not.toBeInTheDocument();
expect(screen.getByText("Physical")).toBeVisible();
expect(screen.getByText("Generated")).toBeVisible();
expect(screen.getByText("Manual")).toBeVisible();
await user.selectOptions(screen.getByRole("combobox", { name: "Relationship status" }), "excluded");
expect(await screen.findByText("legacy_orders")).toBeVisible();
await waitFor(() => expect(screen.queryByText("visits")).not.toBeInTheDocument());
await user.selectOptions(screen.getByRole("combobox", { name: "Relationship status" }), "all");
expect(await screen.findByText("visits")).toBeVisible();
expect(screen.getByText("legacy_orders")).toBeVisible();
});
test("adds a manual relationship from four catalog selections", async () => {
const user = userEvent.setup();
const orders: CatalogTable = {
id: "22222222-2222-4222-8222-222222222222",
databaseId,
name: "orders",
sourceComment: null,
description: null,
generatedDescription: null,
version: 1,
createdAt: "2026-08-31T12:00:00.000Z",
updatedAt: "2026-08-31T12:00:00.000Z",
};
const users: CatalogTable = {
...orders,
id: "33333333-3333-4333-8333-333333333333",
name: "users",
};
const sourceColumn: CatalogColumn = {
id: "44444444-4444-4444-8444-444444444444",
tableId: orders.id,
name: "user_id",
ordinalPosition: 1,
dataType: "bigint",
isNullable: false,
defaultExpression: null,
primaryKeyPosition: null,
isPrimaryKey: false,
isForeignKey: false,
foreignKeyCount: 0,
sourceComment: null,
description: null,
generatedDescription: null,
sensitive: false,
lastSyncedDatabaseVersion: 3,
lastSyncedAt: "2026-08-31T12:00:00.000Z",
version: 1,
createdAt: "2026-08-31T12:00:00.000Z",
updatedAt: "2026-08-31T12:00:00.000Z",
};
const targetColumn: CatalogColumn = {
...sourceColumn,
id: "55555555-5555-4555-8555-555555555555",
tableId: users.id,
name: "id",
primaryKeyPosition: 1,
isPrimaryKey: true,
};
let requestBody: unknown;
server.use(
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([orders, users])),
http.get(
"/api/catalog/databases/:databaseId/tables/:tableId/columns",
({ params }) => HttpResponse.json(params.tableId === orders.id ? [sourceColumn] : [targetColumn]),
),
http.post("/api/catalog/databases/:databaseId/relationships", async ({ request }) => {
requestBody = await request.json();
return HttpResponse.json(relationship("created", "manual", "active", "orders"), { status: 201 });
}),
);
renderRelationships([]);
await user.click(await screen.findByRole("button", { name: "Add relationship" }));
const drawer = await screen.findByRole("dialog", { name: "Add relationship" });
expect(drawer).toHaveAttribute("aria-modal", "false");
const submit = within(drawer).getByRole("button", { name: "Add relationship" });
await user.selectOptions(screen.getByRole("combobox", { name: "Source table" }), orders.id);
await user.selectOptions(screen.getByRole("combobox", { name: "Source column" }), sourceColumn.id);
await user.selectOptions(screen.getByRole("combobox", { name: "Target table" }), users.id);
await user.selectOptions(screen.getByRole("combobox", { name: "Target column" }), targetColumn.id);
expect(submit).toBeEnabled();
await user.click(submit);
await waitFor(() => expect(requestBody).toEqual({
sourceColumnId: sourceColumn.id,
targetColumnId: targetColumn.id,
}));
expect(await screen.findByText("Relationship added.")).toBeVisible();
await waitFor(() => expect(screen.queryByRole("dialog", { name: "Add relationship" })).not.toBeInTheDocument());
});
test("rebuilds generated relationships with progress and a result summary", async () => {
const user = userEvent.setup();
let finish: (() => void) | undefined;
server.use(http.post(
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
async () => {
await new Promise<void>((resolve) => { finish = resolve; });
return HttpResponse.json({ added: 12, alreadyPresent: 7, excluded: 3, ambiguous: 2 });
},
));
renderRelationships([]);
await user.selectOptions(
await screen.findByRole("combobox", { name: "Relationship action" }),
"rebuild-generated",
);
await user.click(screen.getByRole("button", { name: "Run action" }));
expect(await screen.findByRole("button", { name: "Rebuilding…" })).toBeDisabled();
finish?.();
expect(await screen.findByText(
"Rebuild complete: 12 added, 7 already present, 3 excluded, 2 ambiguous.",
)).toBeVisible();
});
test("reports when rebuilding finds no new relationships", async () => {
const user = userEvent.setup();
server.use(http.post(
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
() => HttpResponse.json({ added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 0 }),
));
renderRelationships([]);
await user.selectOptions(
await screen.findByRole("combobox", { name: "Relationship action" }),
"rebuild-generated",
);
await user.click(screen.getByRole("button", { name: "Run action" }));
expect(await screen.findByText("Rebuild complete: no new relationships found.")).toBeVisible();
});
test("excludes a generated relationship after an explanatory drawer confirmation", async () => {
const user = userEvent.setup();
const generated = relationship("generated", "generated", "active", "orders");
let requestBody: unknown;
server.use(http.patch(
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
async ({ request }) => {
requestBody = await request.json();
return HttpResponse.json({ ...generated, status: "excluded" });
},
));
renderRelationships([generated]);
await user.click(await screen.findByRole("button", {
name: "View relationship orders.user_id to users.id",
}));
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
expect(within(drawer).getByText("Generated")).toBeVisible();
await user.click(within(drawer).getByRole("button", { name: "Exclude" }));
expect(within(drawer).getByText(
"The relationship will move to Excluded. Rebuilds will not recreate it, and you can restore it later.",
)).toBeVisible();
const confirmExclude = within(drawer).getByRole("button", { name: "Exclude relationship" });
await waitFor(() => expect(confirmExclude).toHaveFocus());
await user.click(confirmExclude);
await waitFor(() => expect(requestBody).toEqual({ status: "excluded" }));
expect(await screen.findByText(
"Relationship excluded. Future rebuilds will leave it excluded.",
)).toBeVisible();
});
test("keeps physical relationships read-only in the details drawer", async () => {
const user = userEvent.setup();
renderRelationships([relationship("physical", "physical", "active", "visits")]);
await user.click(await screen.findByRole("button", {
name: "View relationship visits.user_id to users.id",
}));
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
expect(within(drawer).getByText("This information is synchronized from the source schema and is read-only.")).toBeVisible();
expect(within(drawer).queryByRole("button", { name: "Exclude" })).not.toBeInTheDocument();
expect(within(drawer).queryByRole("button", { name: "Restore" })).not.toBeInTheDocument();
expect(within(drawer).queryByRole("button", { name: "Delete permanently" })).not.toBeInTheDocument();
});
test("permanently deletes a logical relationship after warning that rebuild may recreate it", async () => {
const user = userEvent.setup();
const manual = relationship("manual", "manual", "active", "payments");
let deletes = 0;
server.use(http.delete(
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
() => {
deletes += 1;
return new HttpResponse(null, { status: 204 });
},
));
renderRelationships([manual]);
await user.click(await screen.findByRole("button", {
name: "View relationship payments.user_id to users.id",
}));
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
await user.click(within(drawer).getByRole("button", { name: "Delete permanently" }));
expect(within(drawer).getByText(
"The relationship record will be erased. A future rebuild may infer it again.",
)).toBeVisible();
await user.click(within(drawer).getByRole("button", { name: "Delete permanently" }));
await waitFor(() => expect(deletes).toBe(1));
expect(await screen.findByText(
"Relationship deleted permanently. A future rebuild may infer it again.",
)).toBeVisible();
});
test("restores an excluded logical relationship", async () => {
const user = userEvent.setup();
const excluded = relationship("excluded", "generated", "excluded", "legacy_orders");
let requestBody: unknown;
server.use(http.patch(
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
async ({ request }) => {
requestBody = await request.json();
return HttpResponse.json({ ...excluded, status: "active" });
},
));
renderRelationships([excluded]);
await user.selectOptions(
await screen.findByRole("combobox", { name: "Relationship status" }),
"excluded",
);
await user.click(await screen.findByRole("button", {
name: "View relationship legacy_orders.user_id to users.id",
}));
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
await user.click(within(drawer).getByRole("button", { name: "Restore" }));
await waitFor(() => expect(requestBody).toEqual({ status: "active" }));
expect(await screen.findByText("Relationship restored.")).toBeVisible();
});
test("keeps relationship context open when a logical mutation fails", async () => {
const user = userEvent.setup();
const generated = relationship("generated-error", "generated", "active", "orders");
server.use(http.patch(
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
() => HttpResponse.json({ code: "relationship_not_found" }, { status: 404 }),
));
renderRelationships([generated]);
await user.click(await screen.findByRole("button", {
name: "View relationship orders.user_id to users.id",
}));
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
await user.click(within(drawer).getByRole("button", { name: "Exclude" }));
await user.click(within(drawer).getByRole("button", { name: "Exclude relationship" }));
expect(await screen.findByText("The relationship no longer exists. Refresh and try again.")).toBeVisible();
expect(screen.getByRole("dialog", { name: "Relationship details" })).toBeVisible();
});
@@ -1,19 +1,26 @@
import { useMemo, useRef, useState, type ReactNode } from "react";
import { useQuery } from "@tanstack/react-query";
import { useEffect, useMemo, useRef, useState } from "react";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { AgGridReact } from "ag-grid-react";
import type { ColDef, ICellRendererParams } from "ag-grid-community";
import { ArrowLeft, History, Info, Play, RefreshCw } from "lucide-react";
import { ArrowLeft, History, Info, Play, Plus, RefreshCw } from "lucide-react";
import { toast } from "sonner";
import { Button } from "../../components/ui/button";
import { apiErrorMessage } from "../../api/client";
import { ApiError, apiErrorMessage } from "../../api/client";
import {
createCatalogRelationship,
deleteCatalogRelationship,
listCatalogRelationships,
listCatalogColumns,
listCatalogTables,
rebuildGeneratedRelationships,
setCatalogRelationshipStatus,
startCatalogSync,
type CatalogDatabase,
type CatalogRelationship,
type CatalogSyncRun,
} from "../../api/catalog-databases";
import { FleetLedgerDrawer } from "./FleetLedgerShell";
import type { DatabaseNavigationState } from "./model";
interface Props {
database: CatalogDatabase;
@@ -23,7 +30,10 @@ interface Props {
onOpenTables: () => void;
onRunStarted: (run: CatalogSyncRun) => void;
onOpenSync: () => void;
metadataModelControl?: ReactNode;
onOpenDescriptionHistory?: () => void;
onOpenSensitiveHistory?: () => void;
onCatalogMetricsChanged?: () => void | Promise<void>;
onNavigationStateChange?: (state: DatabaseNavigationState) => void;
presentation?: "legacy" | "fleet";
}
@@ -32,6 +42,19 @@ interface RelationshipGridContext {
onView: (relationship: CatalogRelationship, origin: HTMLButtonElement) => void;
}
type RelationshipStatusFilter = "active" | "excluded" | "all";
function titleCase(value: string): string {
return value.charAt(0).toUpperCase() + value.slice(1);
}
function relationshipLabel(relationship: CatalogRelationship): string {
const pair = relationship.columns[0];
const source = `${relationship.sourceTableName}${pair ? `.${pair.sourceColumnName}` : ""}`;
const target = `${relationship.targetTableName}${pair ? `.${pair.targetColumnName}` : ""}`;
return `${source} to ${target}`;
}
function RelationshipActionsCell({ data, context }: ICellRendererParams<CatalogRelationship, unknown, RelationshipGridContext>) {
if (!data || !context || context.presentation !== "fleet") return null;
return (
@@ -42,7 +65,7 @@ function RelationshipActionsCell({ data, context }: ICellRendererParams<CatalogR
size="icon-lg"
className="thot-fleet-icon-action"
data-tooltip="Details"
aria-label={`View relationship ${data.constraintName}`}
aria-label={`View relationship ${relationshipLabel(data)}`}
onClick={(event) => context.onView(data, event.currentTarget)}
>
<Info aria-hidden="true" />
@@ -59,32 +82,86 @@ export function DatabaseRelationships({
onOpenTables,
onRunStarted,
onOpenSync,
onOpenDescriptionHistory,
onOpenSensitiveHistory,
onCatalogMetricsChanged,
onNavigationStateChange,
presentation = "legacy",
}: Props) {
const databaseId = database.id!;
const queryClient = useQueryClient();
const { data = [], error, isError, isLoading, isFetching, refetch } = useQuery({
queryKey: ["catalog-relationships", databaseId],
queryFn: () => listCatalogRelationships(databaseId),
retry: false,
});
const [search, setSearch] = useState("");
const [statusFilter, setStatusFilter] = useState<RelationshipStatusFilter>("active");
const [busy, setBusy] = useState(false);
const [selectedAction, setSelectedAction] = useState<"" | "relationships" | "all">("");
const [selectedAction, setSelectedAction] = useState<"" | "rebuild-generated" | "relationships" | "all">("");
const [activeRelationshipId, setActiveRelationshipId] = useState<string | null>(null);
const [addOpen, setAddOpen] = useState(false);
const [sourceTableId, setSourceTableId] = useState("");
const [sourceColumnId, setSourceColumnId] = useState("");
const [targetTableId, setTargetTableId] = useState("");
const [targetColumnId, setTargetColumnId] = useState("");
const [addError, setAddError] = useState<string | null>(null);
const [targetColumnError, setTargetColumnError] = useState<string | null>(null);
const [pendingMutation, setPendingMutation] = useState<"exclude" | "delete" | null>(null);
const originRef = useRef<HTMLButtonElement | null>(null);
const statusFilterRef = useRef<HTMLSelectElement | null>(null);
const bindingReady = database.connectionStatus === "reachable" && database.testedVersion === database.version;
const activeRelationship = activeRelationshipId
? data.find((relationship) => relationship.id === activeRelationshipId)
: undefined;
const visibleRelationships = useMemo(() => statusFilter === "all"
? data
: data.filter((relationship) => relationship.status === statusFilter), [data, statusFilter]);
const tablesQuery = useQuery({
queryKey: ["catalog-tables", databaseId],
queryFn: () => listCatalogTables(databaseId),
enabled: addOpen,
retry: false,
});
const sourceColumnsQuery = useQuery({
queryKey: ["catalog-columns", databaseId, sourceTableId],
queryFn: () => listCatalogColumns(databaseId, sourceTableId),
enabled: addOpen && Boolean(sourceTableId),
retry: false,
});
const targetColumnsQuery = useQuery({
queryKey: ["catalog-columns", databaseId, targetTableId],
queryFn: () => listCatalogColumns(databaseId, targetTableId),
enabled: addOpen && Boolean(targetTableId),
retry: false,
});
const targetColumns = (targetColumnsQuery.data ?? []).filter((column) => column.isPrimaryKey);
useEffect(() => {
if (!pendingMutation) return;
const timer = window.setTimeout(() => document
.getElementById("relationship-mutation-confirm-button")?.focus(), 0);
return () => window.clearTimeout(timer);
}, [pendingMutation]);
useEffect(() => {
onNavigationStateChange?.({ dirty: false, busy });
return () => onNavigationStateChange?.({ dirty: false, busy: false });
}, [busy, onNavigationStateChange]);
const selectedActionUnavailable = !selectedAction
? null
: !canManage
? "You do not have permission to synchronize the catalog."
: !bindingReady
? "Test the current database binding first."
: database.activeSyncRun
? "Wait for the active synchronization to finish."
? "You do not have permission to manage catalog relationships."
: database.activeSyncRun
? "Wait for the active synchronization to finish."
: selectedAction !== "rebuild-generated" && !bindingReady
? "Test the current database binding first."
: null;
const addUnavailable = !canManage
? "You do not have permission to add catalog relationships."
: database.activeSyncRun
? "Wait for the active synchronization to finish."
: busy
? "Wait for the current relationship operation to finish."
: null;
const synchronize = async (scope: "relationships" | "all") => {
setBusy(true);
@@ -97,28 +174,146 @@ export function DatabaseRelationships({
const openRelationship = (relationship: CatalogRelationship, origin: HTMLButtonElement) => {
originRef.current = origin;
setPendingMutation(null);
setAddOpen(false);
setActiveRelationshipId(relationship.id);
};
const closeRelationship = () => {
if (busy) return;
setPendingMutation(null);
setActiveRelationshipId(null);
window.setTimeout(() => originRef.current?.focus(), 0);
window.setTimeout(() => {
if (originRef.current?.isConnected) originRef.current.focus();
else statusFilterRef.current?.focus();
}, 0);
};
const changeRelationshipStatus = async (status: "active" | "excluded") => {
if (!activeRelationship || activeRelationship.origin === "physical") return;
setBusy(true);
try {
await setCatalogRelationshipStatus(databaseId, activeRelationship.id, status);
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
await onCatalogMetricsChanged?.();
setPendingMutation(null);
setActiveRelationshipId(null);
window.setTimeout(() => {
if (originRef.current?.isConnected) originRef.current.focus();
else statusFilterRef.current?.focus();
}, 0);
toast.success(status === "excluded"
? "Relationship excluded. Future rebuilds will leave it excluded."
: "Relationship restored.");
} catch (error) {
toast.error(apiErrorMessage(error));
} finally {
setBusy(false);
}
};
const permanentlyDeleteRelationship = async () => {
if (!activeRelationship || activeRelationship.origin === "physical") return;
setBusy(true);
try {
await deleteCatalogRelationship(databaseId, activeRelationship.id);
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
await onCatalogMetricsChanged?.();
setPendingMutation(null);
setActiveRelationshipId(null);
window.setTimeout(() => {
if (originRef.current?.isConnected) originRef.current.focus();
else statusFilterRef.current?.focus();
}, 0);
toast.success("Relationship deleted permanently. A future rebuild may infer it again.");
} catch (error) {
toast.error(apiErrorMessage(error));
} finally {
setBusy(false);
}
};
const resetAddForm = () => {
setSourceTableId("");
setSourceColumnId("");
setTargetTableId("");
setTargetColumnId("");
setAddError(null);
setTargetColumnError(null);
};
const closeAdd = () => {
if (busy) return;
setAddOpen(false);
resetAddForm();
};
const addRelationship = async () => {
if (!sourceColumnId || !targetColumnId) return;
setBusy(true);
setAddError(null);
setTargetColumnError(null);
try {
await createCatalogRelationship(databaseId, sourceColumnId, targetColumnId);
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
await onCatalogMetricsChanged?.();
setAddOpen(false);
resetAddForm();
toast.success("Relationship added.");
} catch (error) {
if (error instanceof ApiError && error.code === "relationship_target_not_unique") {
setTargetColumnError("The target column must be the only primary-key column of its table.");
} else if (error instanceof ApiError && error.code === "relationship_type_incompatible") {
setTargetColumnError("Source and target column types are not compatible.");
} else if (error instanceof ApiError && error.code === "relationship_duplicate") {
setAddError("This relationship already exists.");
} else {
setAddError(apiErrorMessage(error));
}
toast.error(apiErrorMessage(error));
} finally {
setBusy(false);
}
};
const runSelectedAction = async () => {
if (!selectedAction) return;
await synchronize(selectedAction);
if (selectedAction === "rebuild-generated") {
setBusy(true);
try {
const result = await rebuildGeneratedRelationships(databaseId);
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
await onCatalogMetricsChanged?.();
const noFindings = result.added === 0
&& result.alreadyPresent === 0
&& result.excluded === 0
&& result.ambiguous === 0;
toast.success(noFindings
? "Rebuild complete: no new relationships found."
: `Rebuild complete: ${result.added} added, ${result.alreadyPresent} already present, ${result.excluded} excluded, ${result.ambiguous} ambiguous.`);
} catch (error) {
toast.error(apiErrorMessage(error));
} finally {
setBusy(false);
}
} else {
await synchronize(selectedAction);
}
setSelectedAction("");
};
const columns = useMemo<ColDef<CatalogRelationship>[]>(() => [
{ field: "constraintName", headerName: "Constraint", minWidth: 220, flex: 1, cellClass: "font-mono text-xs" },
...(presentation === "legacy" ? [{ field: "constraintName", headerName: "Constraint", minWidth: 220, flex: 1, cellClass: "font-mono text-xs" } satisfies ColDef<CatalogRelationship>] : []),
{ field: "sourceTableName", headerName: "Source table", minWidth: 200, flex: 1 },
{ headerName: "Source columns", minWidth: 200, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.sourceColumnName).join(", ") ?? "" },
{ headerName: "Source column", minWidth: 180, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.sourceColumnName).join(", ") ?? "" },
{ field: "targetTableName", headerName: "Target table", minWidth: 200, flex: 1 },
{ headerName: "Target columns", minWidth: 200, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.targetColumnName).join(", ") ?? "" },
{ field: "updateRule", headerName: "On update", minWidth: 125, width: 125 },
{ field: "deleteRule", headerName: "On delete", minWidth: 125, width: 125 },
{ headerName: "Target column", minWidth: 180, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.targetColumnName).join(", ") ?? "" },
{ field: "origin", headerName: "Origin", minWidth: 120, width: 120, valueFormatter: ({ value }) => titleCase(String(value ?? "")) },
{ field: "status", headerName: "Status", minWidth: 110, width: 110, valueFormatter: ({ value }) => titleCase(String(value ?? "")) },
...(presentation === "legacy" ? [
{ field: "updateRule", headerName: "On update", minWidth: 125, width: 125 },
{ field: "deleteRule", headerName: "On delete", minWidth: 125, width: 125 },
] satisfies ColDef<CatalogRelationship>[] : []),
...(presentation === "fleet" ? [{ colId: "actions", headerName: "Actions", width: 64, minWidth: 64, maxWidth: 64, pinned: "right" as const, lockPinned: true, sortable: false, filter: false, resizable: false, suppressMovable: true, suppressHeaderMenuButton: true, cellRenderer: RelationshipActionsCell }] : []),
], [presentation]);
const context = useMemo<RelationshipGridContext>(() => ({ presentation, onView: openRelationship }), [data, presentation]);
@@ -157,14 +352,27 @@ export function DatabaseRelationships({
</div> : null}
<div className={presentation === "fleet" ? "thot-fleet-grid-surface flex min-h-0 flex-1 flex-col overflow-hidden bg-card" : "mx-3 mb-4 mt-4 flex min-h-0 flex-1 flex-col overflow-hidden rounded-md border border-border bg-card sm:mx-5"}>
<div className={`${presentation === "fleet" ? "thot-fleet-grid-toolbar " : ""}flex min-h-12 flex-wrap items-center gap-3 border-b border-border px-3 py-2`}>
<span className="thot-label whitespace-nowrap">Physical relationships</span>
<span className="thot-label whitespace-nowrap">Relationship map</span>
<label className="sr-only" htmlFor="relationship-status-filter">Relationship status</label>
<select
ref={statusFilterRef}
id="relationship-status-filter"
className="h-8 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
aria-label="Relationship status"
value={statusFilter}
onChange={(event) => setStatusFilter(event.target.value as RelationshipStatusFilter)}
>
<option value="active">Active</option>
<option value="excluded">Excluded</option>
<option value="all">All</option>
</select>
<input className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search relationships" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
<span className="text-xs tabular-nums text-muted-foreground">{data.length}</span>
<span className="text-xs tabular-nums text-muted-foreground">{visibleRelationships.length}</span>
<Button type="button" variant="outline" disabled={isFetching || busy} onClick={() => void refetch()}><RefreshCw className={isFetching ? "animate-spin" : ""} />Refresh</Button>
{presentation === "fleet" ? (
<div className="flex flex-wrap items-center gap-2">
<div className="thot-fleet-action-selector__scope" aria-live="polite">
<strong>{selectedAction === "all" ? "This database" : selectedAction === "relationships" ? "All relationships in this database" : "This database"}</strong>
<strong>{selectedAction === "relationships" ? "Physical relationships in this database" : "This database"}</strong>
<span>Action scope</span>
</div>
<label className="sr-only" htmlFor="relationship-action">Relationship action</label>
@@ -173,18 +381,37 @@ export function DatabaseRelationships({
className="thot-fleet-action-selector__select"
value={selectedAction}
disabled={busy}
onChange={(event) => setSelectedAction(event.target.value as "" | "relationships" | "all")}
onChange={(event) => setSelectedAction(event.target.value as "" | "rebuild-generated" | "relationships" | "all")}
>
<option value="">Choose an action…</option>
<optgroup label="Logical relationships">
<option value="rebuild-generated">Rebuild generated relationships</option>
</optgroup>
<optgroup label="Synchronization">
<option value="relationships">Synchronize all relationships in this database</option>
<option value="relationships">Synchronize physical relationships</option>
<option value="all">Synchronize the full schema for this database</option>
</optgroup>
</select>
<Button type="button" size="sm" disabled={!selectedAction || Boolean(selectedActionUnavailable) || busy} aria-describedby={selectedActionUnavailable ? "relationship-action-reason" : undefined} onClick={() => void runSelectedAction()}><Play aria-hidden="true" />{busy ? "Running…" : "Run action"}</Button>
<Button type="button" size="sm" disabled={!selectedAction || Boolean(selectedActionUnavailable) || busy} aria-describedby={selectedActionUnavailable ? "relationship-action-reason" : undefined} onClick={() => void runSelectedAction()}><Play aria-hidden="true" />{busy ? selectedAction === "rebuild-generated" ? "Rebuilding…" : "Running…" : "Run action"}</Button>
<Button type="button" size="sm" variant="outline" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Synchronization history" onClick={onOpenSync}>
{database.activeSyncRun ? <RefreshCw className="animate-spin" aria-hidden="true" /> : <History aria-hidden="true" />} Sync history
</Button>
{onOpenDescriptionHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenDescriptionHistory}><History aria-hidden="true" /> Description history</Button> : null}
{onOpenSensitiveHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenSensitiveHistory}><History aria-hidden="true" /> Sensitive history</Button> : null}
<Button
type="button"
size="sm"
disabled={Boolean(addUnavailable)}
aria-describedby={addUnavailable ? "relationship-add-reason" : undefined}
onClick={() => {
setActiveRelationshipId(null);
setPendingMutation(null);
setAddOpen(true);
}}
>
<Plus aria-hidden="true" /> Add relationship
</Button>
{addUnavailable ? <span id="relationship-add-reason" className="sr-only">{addUnavailable}</span> : null}
{selectedActionUnavailable ? <span id="relationship-action-reason" className="basis-full text-xs text-muted-foreground" role="status">{selectedActionUnavailable}</span> : null}
</div>
) : <Button type="button" disabled={!canManage || !bindingReady || busy || Boolean(database.activeSyncRun)} title={!bindingReady ? "Test the current database binding before synchronizing relationships" : undefined} onClick={() => void synchronize("relationships")}><RefreshCw />Sync relationships</Button>}
@@ -192,26 +419,174 @@ export function DatabaseRelationships({
{!bindingReady ? <div className="border-b border-border bg-muted/35 px-4 py-3 text-sm text-muted-foreground">{presentation === "fleet" ? "Test the current database binding in database details before synchronizing relationships." : "Test the current database binding from Overview before synchronizing relationships."}</div> : null}
{fleetQueryError ?? <div className="relative min-h-[280px] flex-1">
<div className={`thot-database-grid ${presentation === "fleet" ? "thot-fleet-ledger-grid " : ""}ag-theme-alpine absolute inset-0 h-full w-full`}>
<AgGridReact<CatalogRelationship> rowData={data} columnDefs={columns} context={context} loading={isLoading} quickFilterText={search} defaultColDef={{ sortable: true, filter: true, resizable: true }} getRowId={({ data: row }) => row.id} rowHeight={44} headerHeight={38} animateRows={false} overlayNoRowsTemplate="No physical relationships synchronized for this database." />
<AgGridReact<CatalogRelationship> rowData={visibleRelationships} columnDefs={columns} context={context} loading={isLoading} quickFilterText={search} defaultColDef={{ sortable: true, filter: true, resizable: true }} getRowId={({ data: row }) => row.id} rowHeight={44} headerHeight={38} animateRows={false} overlayNoRowsTemplate={statusFilter === "active" ? "No relationships yet. Rebuild from column names or add one manually." : statusFilter === "excluded" ? "No excluded relationships." : "No relationships for this database."} />
</div>
</div>}
</div>
{presentation === "fleet" && addOpen ? (
<FleetLedgerDrawer
open
eyebrow="Logical relationship"
title="Add relationship"
description="Connect one source column to one primary-key target column."
onClose={closeAdd}
closeLabel="Close add relationship"
busy={busy}
footer={(
<>
<Button type="button" variant="outline" disabled={busy} onClick={closeAdd}>Cancel</Button>
<Button
type="button"
disabled={busy || !sourceTableId || !sourceColumnId || !targetTableId || !targetColumnId}
onClick={() => void addRelationship()}
>
{busy ? "Adding…" : "Add relationship"}
</Button>
</>
)}
>
<div className="grid gap-4 text-sm sm:grid-cols-2">
<label className="grid gap-1.5">
<span className="font-semibold">Source table</span>
<select
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
aria-label="Source table"
value={sourceTableId}
disabled={busy || tablesQuery.isLoading}
onChange={(event) => {
setSourceTableId(event.target.value);
setSourceColumnId("");
setAddError(null);
}}
>
<option value="">Select source table…</option>
{(tablesQuery.data ?? []).map((table) => <option key={table.id} value={table.id}>{table.name}</option>)}
</select>
</label>
<label className="grid gap-1.5">
<span className="font-semibold">Source column</span>
<select
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
aria-label="Source column"
value={sourceColumnId}
disabled={busy || !sourceTableId || sourceColumnsQuery.isLoading}
onChange={(event) => {
setSourceColumnId(event.target.value);
setAddError(null);
setTargetColumnError(null);
}}
>
<option value="">Select source column…</option>
{(sourceColumnsQuery.data ?? []).map((column) => <option key={column.id} value={column.id}>{column.name}</option>)}
</select>
</label>
<label className="grid gap-1.5">
<span className="font-semibold">Target table</span>
<select
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
aria-label="Target table"
value={targetTableId}
disabled={busy || tablesQuery.isLoading}
onChange={(event) => {
setTargetTableId(event.target.value);
setTargetColumnId("");
setAddError(null);
setTargetColumnError(null);
}}
>
<option value="">Select target table…</option>
{(tablesQuery.data ?? []).map((table) => <option key={table.id} value={table.id}>{table.name}</option>)}
</select>
</label>
<label className="grid gap-1.5">
<span className="font-semibold">Target column</span>
<select
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
aria-label="Target column"
aria-invalid={Boolean(targetColumnError)}
aria-describedby={targetColumnError ? "target-column-error" : undefined}
value={targetColumnId}
disabled={busy || !targetTableId || targetColumnsQuery.isLoading}
onChange={(event) => {
setTargetColumnId(event.target.value);
setAddError(null);
setTargetColumnError(null);
}}
>
<option value="">Select target column…</option>
{targetColumns.map((column) => <option key={column.id} value={column.id}>{column.name}</option>)}
</select>
{targetColumnError ? <span id="target-column-error" className="text-xs text-destructive" role="alert">{targetColumnError}</span> : null}
</label>
{tablesQuery.isError || sourceColumnsQuery.isError || targetColumnsQuery.isError ? (
<p className="sm:col-span-2 text-sm text-destructive" role="alert">Catalog tables or columns could not be loaded.</p>
) : null}
{addError ? <p className="sm:col-span-2 text-sm text-destructive" role="alert">{addError}</p> : null}
</div>
</FleetLedgerDrawer>
) : null}
{presentation === "fleet" && activeRelationship ? (
<FleetLedgerDrawer
open
eyebrow="Physical relationship"
eyebrow={`${titleCase(activeRelationship.origin)} relationship`}
title="Relationship details"
description="This information is synchronized from the source schema and is read-only."
description={activeRelationship.origin === "physical"
? "This information is synchronized from the source schema and is read-only."
: activeRelationship.origin === "generated"
? "This relationship was inferred from catalog column names."
: "This relationship was added manually."}
onClose={closeRelationship}
closeLabel="Close relationship details"
footer={<Button type="button" variant="outline" onClick={closeRelationship}>Close</Button>}
busy={busy}
footer={pendingMutation ? (
<div
className="flex min-w-0 flex-1 flex-wrap items-center gap-2"
role="group"
aria-labelledby="relationship-mutation-confirmation"
onKeyDown={(event) => {
if (event.key === "Escape" && !busy) setPendingMutation(null);
}}
>
<p id="relationship-mutation-confirmation" className="mr-auto max-w-xl text-sm text-muted-foreground">
{pendingMutation === "exclude"
? "The relationship will move to Excluded. Rebuilds will not recreate it, and you can restore it later."
: "The relationship record will be erased. A future rebuild may infer it again."}
</p>
<Button type="button" variant="ghost" disabled={busy} onClick={() => setPendingMutation(null)}>Cancel</Button>
<Button
id="relationship-mutation-confirm-button"
type="button"
variant={pendingMutation === "delete" ? "destructive" : "default"}
disabled={busy}
onClick={() => void (pendingMutation === "exclude"
? changeRelationshipStatus("excluded")
: permanentlyDeleteRelationship())}
>
{pendingMutation === "exclude" ? "Exclude relationship" : "Delete permanently"}
</Button>
</div>
) : (
<>
<Button type="button" variant="outline" disabled={busy} onClick={closeRelationship}>Close</Button>
{activeRelationship.origin !== "physical" && activeRelationship.status === "active" ? (
<Button type="button" variant="outline" disabled={!canManage || busy} onClick={() => setPendingMutation("exclude")}>Exclude</Button>
) : null}
{activeRelationship.origin !== "physical" && activeRelationship.status === "excluded" ? (
<Button type="button" disabled={!canManage || busy} onClick={() => void changeRelationshipStatus("active")}>Restore</Button>
) : null}
{activeRelationship.origin !== "physical" ? (
<Button type="button" variant="destructive" disabled={!canManage || busy} onClick={() => setPendingMutation("delete")}>Delete permanently</Button>
) : null}
</>
)}
>
<dl className="grid gap-4 text-sm sm:grid-cols-2">
<div className="sm:col-span-2"><dt className="thot-label">Constraint</dt><dd className="mt-1 break-words font-mono text-xs">{activeRelationship.constraintName}</dd></div>
<div><dt className="thot-label">Origin</dt><dd className="mt-1">{titleCase(activeRelationship.origin)}</dd></div>
<div><dt className="thot-label">Status</dt><dd className="mt-1">{titleCase(activeRelationship.status)}</dd></div>
{activeRelationship.constraintName ? <div className="sm:col-span-2"><dt className="thot-label">Constraint</dt><dd className="mt-1 break-words font-mono text-xs">{activeRelationship.constraintName}</dd></div> : null}
<div><dt className="thot-label">Source table</dt><dd className="mt-1 break-words">{activeRelationship.sourceTableName}</dd></div>
<div><dt className="thot-label">Target table</dt><dd className="mt-1 break-words">{activeRelationship.targetTableName}</dd></div>
<div><dt className="thot-label">On update</dt><dd className="mt-1">{activeRelationship.updateRule ?? "Not specified"}</dd></div>
<div><dt className="thot-label">On delete</dt><dd className="mt-1">{activeRelationship.deleteRule ?? "Not specified"}</dd></div>
{activeRelationship.origin === "physical" ? <><div><dt className="thot-label">On update</dt><dd className="mt-1">{activeRelationship.updateRule ?? "Not specified"}</dd></div><div><dt className="thot-label">On delete</dt><dd className="mt-1">{activeRelationship.deleteRule ?? "Not specified"}</dd></div></> : null}
<div className="sm:col-span-2">
<dt className="thot-label">Column mapping</dt>
<dd className="mt-2 overflow-hidden rounded-md border border-border">
@@ -1,4 +1,4 @@
import { useEffect, useMemo, useRef, useState, type ReactNode } from "react";
import { useEffect, useMemo, useRef, useState } from "react";
import { Menu } from "@base-ui/react/menu";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { AgGridReact } from "ag-grid-react";
@@ -25,6 +25,7 @@ import type { DatabaseNavigationState } from "./model";
import { DatabaseColumns } from "./DatabaseColumns";
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
import { FleetLedgerDrawer } from "./FleetLedgerShell";
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
interface Props {
database: CatalogDatabase;
@@ -39,10 +40,11 @@ interface Props {
onNestedNavigationChange?: (active: boolean) => void;
onRunStarted: (run: CatalogSyncRun) => void;
onOpenSync: () => void;
onOpenDescriptionHistory?: () => void;
onOpenSensitiveHistory?: () => void;
onDescriptionGenerationRunStarted: (run: DescriptionGenerationRun) => void;
onSuggestSensitive: (selection: SensitiveDataSuggestionRequest, scopeLabel: string) => Promise<void>;
onCatalogMetricsChanged?: () => void | Promise<void>;
metadataModelControl?: ReactNode;
presentation?: "legacy" | "fleet";
}
@@ -93,10 +95,11 @@ export function DatabaseTables({
onNestedNavigationChange,
onRunStarted,
onOpenSync,
onOpenDescriptionHistory,
onOpenSensitiveHistory,
onDescriptionGenerationRunStarted,
onSuggestSensitive,
onCatalogMetricsChanged,
metadataModelControl,
presentation = "legacy",
}: Props) {
const databaseId = database.id!;
@@ -314,7 +317,7 @@ export function DatabaseTables({
: selectedIds.length === 0
? "Select at least one table."
: !selectedMetadataModel
? "Choose a metadata model first."
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
: descriptionGenerationActive || Boolean(currentRun)
? "Wait for the active catalog operation to finish."
: busy !== null
@@ -377,7 +380,7 @@ export function DatabaseTables({
: selectedIds.length === 0
? "Select at least one table."
: !selectedMetadataModel
? "Choose a metadata model first."
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
: descriptionGenerationActive || Boolean(currentRun)
? "Wait for the active catalog operation to finish."
: busy !== null
@@ -487,8 +490,9 @@ export function DatabaseTables({
<div className="flex min-h-12 flex-wrap items-center gap-3 border-b border-border bg-muted/25 px-3 py-2">
<Button
type="button"
variant="outline"
className="thot-fleet-icon-action border-primary/35 bg-primary/8 shadow-sm"
variant="ghost"
size="sm"
className="thot-fleet-icon-action thot-fleet-back-action"
disabled={navigationBusy}
data-tooltip="Tables"
aria-label="Back to tables"
@@ -514,7 +518,6 @@ export function DatabaseTables({
bindingReady={bindingReady}
catalogOperationActive={Boolean(currentRun)}
onCatalogMetricsChanged={onCatalogMetricsChanged}
metadataModelControl={metadataModelControl}
presentation="fleet"
/>
</section>
@@ -712,7 +715,6 @@ export function DatabaseTables({
selectRef={fleetActionSelectRef}
onRun={runFleetAction}
onClear={clearSelection}
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
label="Table action"
/>
</>
@@ -751,7 +753,6 @@ export function DatabaseTables({
selectRef={fleetActionSelectRef}
onRun={runFleetAction}
onClear={clearSelection}
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
label="Table action"
/>
<input ref={searchInputRef} className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search tables" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
@@ -773,6 +774,8 @@ export function DatabaseTables({
{currentRun ? <RefreshCw className="animate-spin" aria-hidden="true" /> : <History aria-hidden="true" />} Sync history
</Button>
) : null}
{onOpenDescriptionHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenDescriptionHistory}><History aria-hidden="true" /> Description history</Button> : null}
{onOpenSensitiveHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenSensitiveHistory}><History aria-hidden="true" /> Sensitive history</Button> : null}
</div>
{!bindingReady ? <div className="border-b border-border bg-muted/35 px-4 py-3 text-sm text-muted-foreground">Test the current database binding from Overview before synchronizing tables.</div> : null}
{fleetQueryError ?? <div className="relative min-h-[280px] flex-1">
@@ -248,6 +248,33 @@ test("reopens a terminal run from newest-first persisted history", async () => {
expect(within(drawer).getByText("3", { selector: "[data-count='generated']" })).toBeVisible();
});
test("styles a successfully completed run with the success background", async () => {
const completedRun: DescriptionGenerationRun = {
...runningRun,
status: "completed",
total: 1,
processed: 1,
generated: 1,
updatedAt: "2026-08-28T08:02:00Z",
finishedAt: "2026-08-28T08:02:00Z",
};
server.use(
http.get("/api/catalog/description-generation-runs", () => HttpResponse.json([completedRun])),
http.get("/api/catalog/description-generation-runs/:runId", () => HttpResponse.json(completedRun)),
http.get("/api/catalog/description-generation-runs/:runId/events-list", () => HttpResponse.json([])),
);
renderDrawer({ initialRun: completedRun });
const history = await screen.findByRole("region", {
name: "Description generation history",
});
const completedHistoryItem = within(history).getByRole("button", {
name: /Completed.*missing/i,
});
expect(completedHistoryItem).toHaveAttribute("data-status", "completed");
expect(completedHistoryItem).toHaveClass("bg-[oklch(var(--success)/0.1)]");
});
test("offers Unlock only for an apparently stale active run and explains the interruption", async () => {
const user = userEvent.setup();
const staleRun: DescriptionGenerationRun = {
@@ -1,4 +1,4 @@
import { useCallback, useEffect, useRef, useState } from "react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { LoaderCircle, LockOpen, Square } from "lucide-react";
import { toast } from "sonner";
@@ -18,6 +18,7 @@ import { FleetLedgerDrawer } from "./FleetLedgerShell";
interface Props {
open: boolean;
databaseId?: string | null;
run: DescriptionGenerationRun | null;
modelLabel: string;
onClose: () => void;
@@ -62,6 +63,7 @@ function timestamp(value: string | null): string {
export function DescriptionGenerationDrawer({
open,
databaseId = null,
run: initialRun,
modelLabel,
onClose,
@@ -100,11 +102,15 @@ export function DescriptionGenerationDrawer({
? 3_000
: false,
});
const visibleHistory = useMemo(
() => (historyQuery.data ?? []).filter((item) => !databaseId || item.databaseId === databaseId),
[databaseId, historyQuery.data],
);
useEffect(() => {
if (!open || initialRun || !historyQuery.data?.[0]) return;
onRunUpdate(historyQuery.data[0]);
}, [historyQuery.data, initialRun, onRunUpdate, open]);
if (!open || initialRun || !visibleHistory[0]) return;
onRunUpdate(visibleHistory[0]);
}, [initialRun, onRunUpdate, open, visibleHistory]);
const mergeEvents = useCallback((incoming: DescriptionGenerationEvent[]) => {
if (incoming.length === 0) return;
@@ -406,7 +412,7 @@ export function DescriptionGenerationDrawer({
<div className="mb-2 flex items-center justify-between gap-3">
<h3 className="thot-label">Recent runs</h3>
<span className="text-xs tabular-nums text-muted-foreground">
{historyQuery.data?.length ?? 0}
{visibleHistory.length}
</span>
</div>
{historyQuery.isError ? (
@@ -415,17 +421,18 @@ export function DescriptionGenerationDrawer({
</p>
) : historyQuery.isLoading ? (
<p className="text-sm text-muted-foreground">Loading run history…</p>
) : (historyQuery.data ?? []).length === 0 ? (
) : visibleHistory.length === 0 ? (
<p className="text-sm text-muted-foreground">No description generation runs yet.</p>
) : (
<div className="max-h-56 divide-y divide-border overflow-y-auto rounded-md border border-border">
{(historyQuery.data ?? []).map((item) => (
{visibleHistory.map((item) => (
<button
key={item.id}
type="button"
aria-label={`${statusLabel(item)} · ${item.scope.replaceAll("_", " ")}`}
aria-current={item.id === run.id ? "true" : undefined}
className="flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25"
data-status={item.status}
className={`flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25${item.status === "completed" ? " bg-[oklch(var(--success)/0.1)]" : ""}`}
onClick={() => onRunUpdate(item)}
>
<span className="font-medium">{statusLabel(item)}</span>
@@ -0,0 +1,30 @@
import { readFileSync, readdirSync } from "node:fs";
import { expect, test } from "vitest";
const sourceDirectory = "src/shell/database-management";
const styles = readFileSync(`${sourceDirectory}/FleetLedgerShell.css`, "utf8");
test("uses operational body text for every available-command list", () => {
const availableCommandLists = readdirSync(sourceDirectory)
.filter((fileName) => fileName.endsWith(".tsx") && !fileName.includes(".test."))
.flatMap((fileName) => {
const source = readFileSync(`${sourceDirectory}/${fileName}`, "utf8");
const matches = source.match(
/<select\b[\s\S]*?<option value="">Choose an action…<\/option>[\s\S]*?<\/select>/g,
);
return (matches ?? []).map((markup) => ({ fileName, markup }));
});
expect(availableCommandLists.map(({ fileName }) => fileName).sort()).toEqual([
"DatabaseRelationships.tsx",
"FleetActionSelector.tsx",
]);
for (const { markup } of availableCommandLists) {
expect(markup).toContain('className="thot-fleet-action-selector__select"');
}
expect(styles).toMatch(
/\.thot-fleet-action-selector__select\s*\{[^}]*font-size:\s*0\.9375rem;/s,
);
});
@@ -274,7 +274,7 @@
background: oklch(var(--card));
}
.thot-fleet-ledger__navigation > button:first-child {
.thot-fleet-back-action {
min-height: 2.25rem;
border: 1px solid oklch(var(--primary) / 0.28);
color: oklch(var(--primary));
@@ -283,7 +283,7 @@
font-weight: 700;
}
.thot-fleet-ledger__navigation > button:first-child:not(:disabled):hover {
.thot-fleet-back-action:not(:disabled):hover {
border-color: oklch(var(--primary) / 0.48);
color: oklch(var(--primary));
background: oklch(var(--primary) / 0.14);
@@ -410,7 +410,7 @@
border-radius: var(--radius);
color: oklch(var(--foreground));
background: oklch(var(--background));
font-size: 0.8125rem;
font-size: 0.9375rem;
outline: none;
}
@@ -431,24 +431,6 @@
line-height: 1.4;
}
.thot-fleet-metadata-model {
min-width: min(100%, 14rem);
flex: 0 1 16rem;
}
.thot-fleet-metadata-model > div {
min-width: 0;
gap: 0.25rem;
}
.thot-fleet-metadata-model label {
gap: 0.25rem;
}
.thot-fleet-metadata-model select {
height: 2rem;
}
.thot-fleet-ledger__selection-summary {
min-width: 0;
overflow: hidden;
@@ -582,6 +564,17 @@
box-shadow: inset 3px 0 0 oklch(var(--primary));
}
.thot-catalog-drawer__history button[data-status="completed"],
.thot-catalog-drawer__history button[data-status="succeeded"] {
background: oklch(var(--success) / 0.1);
}
.thot-catalog-drawer__history button[aria-current="true"][data-status="completed"],
.thot-catalog-drawer__history button[aria-current="true"][data-status="succeeded"] {
background: oklch(var(--success) / 0.1);
box-shadow: inset 0 0 0 1px oklch(var(--success) / 0.5);
}
.thot-fleet-grid-surface {
position: relative;
}
@@ -623,24 +616,29 @@
.thot-fleet-icon-action::after {
position: absolute;
z-index: 80;
inset-inline-end: calc(100% + 5px);
inset-block-start: 50%;
inset-inline-end: 0;
inset-block-start: calc(100% + 3px);
width: max-content;
max-width: 12rem;
padding: 0.3rem 0.45rem;
border: 1px solid oklch(var(--border));
border-radius: calc(var(--radius) - 0.125rem);
color: oklch(var(--popover-foreground));
background: oklch(var(--popover));
box-shadow: var(--shadow-sm);
padding: 5px 8px;
border: 1px solid oklch(42% 0.018 258);
border-radius: 6px;
color: oklch(96% 0.006 258);
background: oklch(24% 0.014 258);
box-shadow: 0 8px 20px -10px oklch(12% 0.02 258 / 0.72);
content: attr(data-tooltip);
font-size: 0.6875rem;
font-weight: 650;
line-height: 1.2;
font-weight: 600;
line-height: 1.25;
text-align: center;
white-space: normal;
opacity: 0;
pointer-events: none;
transform: translate(2px, -50%);
transition: opacity 60ms cubic-bezier(0.22, 1, 0.36, 1), transform 60ms cubic-bezier(0.22, 1, 0.36, 1);
transform: translateY(-2px);
transition:
opacity 90ms linear,
transform 120ms cubic-bezier(0.25, 1, 0.5, 1),
visibility 0ms linear 120ms;
visibility: hidden;
}
@@ -677,7 +675,8 @@
.thot-fleet-icon-action:focus-visible::after,
.thot-fleet-icon-action:focus-within::after {
opacity: 1;
transform: translate(0, -50%);
transform: translateY(0);
transition-delay: 60ms;
visibility: visible;
}
@@ -233,7 +233,7 @@ export function FleetLedgerBreadcrumb({
type="button"
variant="ghost"
size="sm"
className="thot-fleet-nav-action"
className="thot-fleet-nav-action thot-fleet-back-action"
data-tooltip={back.label.replace(/^Back to\s+/i, "")}
onClick={back.onBack}
disabled={back.disabled}
@@ -0,0 +1,22 @@
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
import { describe, expect, test } from "vitest";
const pageSource = readFileSync(resolve(__dirname, "../DatabaseManagementPage.tsx"), "utf8");
const componentSource = (name: string) => readFileSync(resolve(__dirname, name), "utf8");
describe("history action placement", () => {
test("workspace header does not own database history actions", () => {
const fleetHeader = pageSource.slice(pageSource.indexOf('title="Database management"'), pageSource.indexOf("kpis={"));
expect(fleetHeader).not.toContain("Description history");
expect(fleetHeader).not.toContain("Sensitive history");
});
test("database views expose metadata histories beside database commands", () => {
for (const file of ["DatabaseForm.tsx", "DatabaseTables.tsx", "DatabaseRelationships.tsx"]) {
const source = componentSource(file);
expect(source, file).toContain("Description history");
expect(source, file).toContain("Sensitive history");
}
});
});
@@ -1,5 +1,9 @@
import { useId } from "react";
import type { MetadataGenerationModels } from "../../api/catalog-databases";
export const NO_METADATA_GENERATION_LLM_MODEL_MESSAGE =
"No metadata-generation LLM model is configured for this installation.";
interface Props {
canManage: boolean;
data?: MetadataGenerationModels;
@@ -7,6 +11,7 @@ interface Props {
isLoading: boolean;
selectedModel: string;
onSelectedModelChange: (modelId: string) => void;
variant?: "default" | "compact";
}
export function MetadataGenerationModelSelector({
@@ -16,21 +21,35 @@ export function MetadataGenerationModelSelector({
isLoading,
selectedModel,
onSelectedModelChange,
variant = "default",
}: Props) {
const helpId = useId();
const models = data?.models ?? [];
const configuredDefault = data?.default ?? "";
const hasUsableDefault = configuredDefault !== ""
&& models.some((model) => model.id === configuredDefault);
const noUsableModel = !isLoading && !isError && Boolean(data) && !hasUsableDefault;
const unavailable = noUsableModel || (!isLoading && isError);
const unavailableMessage = !isLoading && isError
? "Metadata-generation LLM model choices are unavailable."
: noUsableModel
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
: null;
const compact = variant === "compact";
return (
<div className="grid min-w-56 max-w-lg gap-1.5">
<label className="grid gap-1.5 text-xs font-medium text-foreground">
<span>Metadata description model</span>
<div className={compact
? "flex min-w-0 flex-wrap items-center gap-x-2 gap-y-1"
: "grid min-w-56 max-w-lg gap-1.5"}
>
<label className={compact
? "flex min-w-0 items-center gap-2 text-xs font-medium text-foreground"
: "grid gap-1.5 text-xs font-medium text-foreground"}
>
<span className={compact ? "whitespace-nowrap" : undefined}>Metadata-generation LLM model</span>
<select
className="h-9 w-full rounded-md border border-input bg-card px-3 text-sm outline-none transition focus:border-primary/60 focus:ring-3 focus:ring-ring/15 disabled:bg-muted disabled:text-muted-foreground"
aria-label="Metadata description model"
aria-describedby={unavailable ? "metadata-generation-model-help" : undefined}
className={`${compact ? "h-8 min-w-52" : "h-9 w-full"} rounded-md border border-input bg-card px-3 text-sm outline-none transition focus:border-primary/60 focus:ring-3 focus:ring-ring/15 disabled:bg-muted disabled:text-muted-foreground`}
aria-label="Metadata-generation LLM model"
aria-describedby={unavailableMessage ? helpId : undefined}
value={selectedModel}
disabled={!canManage || isLoading || isError || !hasUsableDefault}
onChange={(event) => onSelectedModelChange(event.target.value)}
@@ -45,9 +64,9 @@ export function MetadataGenerationModelSelector({
))}
</select>
</label>
{unavailable ? (
<p id="metadata-generation-model-help" className="max-w-72 text-xs font-normal leading-4 text-muted-foreground">
Configure a metadata-generation model in application setup to enable description generation.
{unavailableMessage ? (
<p id={helpId} className="max-w-72 text-xs font-normal leading-4 text-muted-foreground">
{unavailableMessage}
</p>
) : null}
</div>
@@ -0,0 +1,27 @@
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
import { describe, expect, test } from "vitest";
const sourceRoot = resolve(__dirname);
describe("Recent runs success styling contract", () => {
test("every history renderer exposes its run status to the shared styles", () => {
const renderers = [
"DescriptionGenerationDrawer.tsx",
"SensitiveDataSuggestionHistoryDrawer.tsx",
"CatalogSyncDrawer.tsx",
];
for (const renderer of renderers) {
const source = readFileSync(resolve(sourceRoot, renderer), "utf8");
expect(source, renderer).toMatch(/data-status=\{item\.(?:status|state)\}/);
}
});
test("shared history styles cover both successful status vocabularies", () => {
const styles = readFileSync(resolve(sourceRoot, "FleetLedgerShell.css"), "utf8");
expect(styles).toContain('button[data-status="completed"]');
expect(styles).toContain('button[data-status="succeeded"]');
expect(styles).toMatch(/oklch\(var\(--success\)/);
});
});
@@ -1,4 +1,4 @@
import { useEffect } from "react";
import { useEffect, useMemo } from "react";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { apiErrorMessage } from "../../api/client";
import {
@@ -11,6 +11,7 @@ import { FleetLedgerDrawer } from "./FleetLedgerShell";
interface Props {
open: boolean;
databaseId?: string | null;
run: SensitiveDataSuggestionRun | null;
modelLabel: string;
onClose: () => void;
@@ -46,6 +47,7 @@ function timestamp(value: string | null): string {
export function SensitiveDataSuggestionHistoryDrawer({
open,
databaseId = null,
run: initialRun,
modelLabel,
onClose,
@@ -78,11 +80,15 @@ export function SensitiveDataSuggestionHistoryDrawer({
? 3_000
: false,
});
const visibleHistory = useMemo(
() => (historyQuery.data ?? []).filter((item) => !databaseId || item.databaseId === databaseId),
[databaseId, historyQuery.data],
);
useEffect(() => {
if (!open || initialRun || !historyQuery.data?.[0]) return;
onRunUpdate(historyQuery.data[0]);
}, [historyQuery.data, initialRun, onRunUpdate, open]);
if (!open || initialRun || !visibleHistory[0]) return;
onRunUpdate(visibleHistory[0]);
}, [initialRun, onRunUpdate, open, visibleHistory]);
useEffect(() => {
if (!run) return;
@@ -199,23 +205,24 @@ export function SensitiveDataSuggestionHistoryDrawer({
<div className="mb-2 flex items-center justify-between gap-3">
<h3 className="thot-label">Recent runs</h3>
<span className="text-xs tabular-nums text-muted-foreground">
{historyQuery.data?.length ?? 0}
{visibleHistory.length}
</span>
</div>
{historyQuery.isError ? (
<p role="alert" className="text-sm text-destructive">{apiErrorMessage(historyQuery.error)}</p>
) : historyQuery.isLoading ? (
<p className="text-sm text-muted-foreground">Loading suggestion history…</p>
) : (historyQuery.data ?? []).length === 0 ? (
) : visibleHistory.length === 0 ? (
<p className="text-sm text-muted-foreground">No sensitive suggestion runs yet.</p>
) : (
<div className="max-h-56 divide-y divide-border overflow-y-auto rounded-md border border-border">
{(historyQuery.data ?? []).map((item) => (
{visibleHistory.map((item) => (
<button
key={item.id}
type="button"
aria-label={`${statusLabel(item)} · ${item.scope.replaceAll("_", " ")}`}
aria-current={item.id === run.id ? "true" : undefined}
data-status={item.status}
className="flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25"
onClick={() => onRunUpdate(item)}
>
@@ -4,7 +4,7 @@ import type {
DatabaseConfiguration,
} from "../../api/catalog-databases";
export type DatabaseFormMode = "add" | "view" | "edit" | "delete";
export type DatabaseFormMode = "configure" | "view" | "edit" | "delete";
export type DatabaseScreen =
| { kind: "list" }
@@ -13,7 +13,6 @@ export type DatabaseScreen =
| {
kind: DatabaseFormMode;
workspaceId: string;
workspaceLocked?: boolean;
};
export type DatabaseBusyAction = "save" | "test" | "delete" | "retry-secrets" | "reload" | null;
@@ -0,0 +1,403 @@
import json
from datetime import UTC, datetime
from types import SimpleNamespace
import pytest
from typer.testing import CliRunner
from tht.cli import app
from tht.config import load_config
from tht.mschema.context import SchemaContextError, load_schema_context
from tht.mschema.models import (
Annotations,
ColumnPhysical,
ForeignKey,
PhysicalSchema,
TableAnnotation,
TablePhysical,
)
from tht.mschema.render import to_mschema_text
REVISION = "a" * 40
RUNNER = CliRunner()
def _workspace(tmp_path, snapshot: dict) -> object:
artifacts = tmp_path / "artifacts"
relationships = tmp_path / "effective-relationships.json"
relationships.write_text(json.dumps(snapshot))
physical = PhysicalSchema(
database="warehouse",
schema="analytics",
introspected_at=datetime(2026, 1, 1, tzinfo=UTC),
tables={
"users": TablePhysical(
columns={"id": ColumnPhysical(type="bigint", pk=True)},
),
"orders": TablePhysical(
columns={
"physical_user_id": ColumnPhysical(type="bigint"),
"annotated_user_id": ColumnPhysical(type="bigint"),
"generated_user_id": ColumnPhysical(type="bigint"),
},
foreign_keys=[
ForeignKey(
columns=["physical_user_id"],
ref_table="users",
ref_columns=["id"],
)
],
),
},
)
physical.to_yaml(artifacts / "mschema" / "physical.yaml")
Annotations(
tables={
"orders": TableAnnotation(
description="Customer orders",
foreign_keys=[
ForeignKey(
columns=["annotated_user_id"],
ref_table="users",
ref_columns=["id"],
)
],
)
}
).to_yaml(artifacts / "mschema" / "annotations.yaml")
config = tmp_path / "runtime.yaml"
config.write_text(
f"""
runtime_identity:
workspace_id: demo
workspace_revision: {REVISION}
source_identity: workspace://demo
database: {{database: warehouse, schema: analytics, user: reader, password: secret, transport: direct}}
vector_db: {{database: vectors, schema: public, user: reader, password: secret}}
embeddings: {{base_url: 'http://localhost:11434', model: 'qwen3-embedding:0.6b', dim: 1024}}
paths:
artifacts: {artifacts}
indexes: {tmp_path / 'indexes'}
sessions: {tmp_path / 'sessions'}
effective_relationships: {relationships}
"""
)
return load_config(config)
def test_effective_snapshot_is_the_exclusive_relationship_source(tmp_path):
cfg = _workspace(
tmp_path,
{
"schemaVersion": 1,
"workspaceId": "demo",
"relationships": [
{
"sourceTable": "orders",
"sourceColumns": ["generated_user_id"],
"targetTable": "users",
"targetColumns": ["id"],
"origin": "generated",
}
],
},
)
context = load_schema_context(cfg)
rendered = to_mschema_text(
context.physical,
context.annotations,
effective_relationships=context.effective_relationships,
)
assert "-- Customer orders" in rendered
assert "orders.generated_user_id=users.id" in rendered
assert "orders.physical_user_id=users.id" not in rendered
assert "orders.annotated_user_id=users.id" not in rendered
def test_declared_effective_snapshot_must_exist(tmp_path):
cfg = _workspace(
tmp_path,
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
)
cfg.paths.effective_relationships.unlink()
with pytest.raises(SchemaContextError, match="snapshot is missing"):
load_schema_context(cfg)
def test_data_root_override_preserves_the_runtime_relationship_snapshot_path(
tmp_path, monkeypatch
):
monkeypatch.setenv("THT_DATA_ROOT", str(tmp_path / "mounted-data"))
monkeypatch.delenv("THT_HOME", raising=False)
cfg = _workspace(
tmp_path,
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
)
assert cfg.paths.effective_relationships == tmp_path / "effective-relationships.json"
@pytest.mark.parametrize(
("snapshot", "message"),
[
(
{"schemaVersion": 1, "workspaceId": "another", "relationships": []},
"workspace does not match",
),
(
{
"schemaVersion": 1,
"workspaceId": "demo",
"relationships": [
{
"sourceTable": "missing_orders",
"sourceColumns": ["user_id"],
"targetTable": "users",
"targetColumns": ["id"],
"origin": "generated",
}
],
},
"endpoint table is absent",
),
(
{
"schemaVersion": 1,
"workspaceId": "demo",
"relationships": [
{
"sourceTable": "orders",
"sourceColumns": ["missing_user_id"],
"targetTable": "users",
"targetColumns": ["id"],
"origin": "manual",
}
],
},
"endpoint column is absent",
),
(
{
"schemaVersion": 1,
"workspaceId": "demo",
"relationships": [
{
"sourceTable": "orders",
"sourceColumns": ["generated_user_id", "annotated_user_id"],
"targetTable": "users",
"targetColumns": ["id"],
"origin": "physical",
}
],
},
"snapshot is invalid",
),
],
)
def test_effective_snapshot_rejects_wrong_workspace_and_orphan_endpoints(
tmp_path, snapshot, message
):
cfg = _workspace(tmp_path, snapshot)
with pytest.raises(SchemaContextError, match=message):
load_schema_context(cfg)
def test_legacy_relationship_merge_remains_available_and_deduplicated(tmp_path):
cfg = _workspace(
tmp_path,
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
)
cfg.paths.effective_relationships = None
annotations = Annotations.from_yaml(
cfg.paths.artifacts / "mschema" / "annotations.yaml"
)
annotations.tables["orders"].foreign_keys.append(
ForeignKey(
columns=["annotated_user_id"],
ref_table="users",
ref_columns=["id"],
)
)
annotations.to_yaml(cfg.paths.artifacts / "mschema" / "annotations.yaml")
context = load_schema_context(cfg)
rendered = to_mschema_text(context.physical, context.annotations)
assert context.effective_relationships is None
assert rendered.count("orders.physical_user_id=users.id") == 1
assert rendered.count("orders.annotated_user_id=users.id") == 1
def test_schema_render_uses_the_effective_snapshot(tmp_path):
_workspace(
tmp_path,
{
"schemaVersion": 1,
"workspaceId": "demo",
"relationships": [
{
"sourceTable": "orders",
"sourceColumns": ["generated_user_id"],
"targetTable": "users",
"targetColumns": ["id"],
"origin": "manual",
}
],
},
)
result = RUNNER.invoke(
app,
["schema", "render", "--format", "mschema-text", "-c", str(tmp_path / "runtime.yaml")],
)
assert result.exit_code == 0, result.output
assert "orders.generated_user_id=users.id" in result.stdout
assert "orders.physical_user_id=users.id" not in result.stdout
assert "orders.annotated_user_id=users.id" not in result.stdout
def test_schema_search_uses_the_effective_snapshot(tmp_path, monkeypatch):
cfg = _workspace(
tmp_path,
{
"schemaVersion": 1,
"workspaceId": "demo",
"relationships": [
{
"sourceTable": "orders",
"sourceColumns": ["generated_user_id"],
"targetTable": "users",
"targetColumns": ["id"],
"origin": "generated",
}
],
},
)
physical = cfg.paths.artifacts / "mschema" / "physical.yaml"
class Searcher:
def search(self, _vector, **_kwargs):
return [
SimpleNamespace(
id="orders",
kind="schema_table",
ref="orders",
title="orders",
similarity=1.0,
content="orders",
metadata={},
)
]
class Embedder:
def embed_query(self, _text):
return [1.0, 0.0, 0.0]
import tht.cli.search_cmd as search_module
import tht.cli.vector_cmd as vector_module
import tht.evidence as evidence_module
monkeypatch.setattr(
search_module,
"_leased_dwh_snapshot",
lambda _cfg, _ctx: SimpleNamespace(physical=physical, lsh_dir=tmp_path / "no-lsh"),
)
monkeypatch.setattr(vector_module, "require_vector_cfg", lambda _cfg: None)
monkeypatch.setattr(vector_module, "open_searcher", lambda _cfg: Searcher())
monkeypatch.setattr(vector_module, "make_embedder", lambda _cfg: Embedder())
monkeypatch.setattr(evidence_module, "validate_corpus_workspace", lambda *_args: None)
monkeypatch.setattr(evidence_module, "active_searcher", lambda _cfg, searcher, **_kwargs: searcher)
result = RUNNER.invoke(
app,
[
"search",
"find",
"orders",
"--kind",
"schema",
"--json",
"-c",
str(tmp_path / "runtime.yaml"),
],
)
assert result.exit_code == 0, result.output
payload = json.loads(result.stdout)
assert "orders.generated_user_id=users.id" in payload["mschema"]
assert "orders.physical_user_id=users.id" not in payload["mschema"]
assert "orders.annotated_user_id=users.id" not in payload["mschema"]
def test_schema_search_fails_closed_before_returning_no_candidates(tmp_path, monkeypatch):
cfg = _workspace(
tmp_path,
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
)
cfg.paths.effective_relationships.unlink()
physical = cfg.paths.artifacts / "mschema" / "physical.yaml"
import tht.cli.search_cmd as search_module
import tht.cli.vector_cmd as vector_module
import tht.evidence as evidence_module
import tht.search as search_core
monkeypatch.setattr(
search_module,
"_leased_dwh_snapshot",
lambda _cfg, _ctx: SimpleNamespace(physical=physical, lsh_dir=tmp_path / "no-lsh"),
)
monkeypatch.setattr(vector_module, "require_vector_cfg", lambda _cfg: None)
monkeypatch.setattr(vector_module, "open_searcher", lambda _cfg: object())
monkeypatch.setattr(vector_module, "make_embedder", lambda _cfg: object())
monkeypatch.setattr(evidence_module, "validate_corpus_workspace", lambda *_args: None)
monkeypatch.setattr(evidence_module, "active_searcher", lambda _cfg, searcher, **_kwargs: searcher)
monkeypatch.setattr(search_core, "combined_search", lambda **_kwargs: [])
result = RUNNER.invoke(
app,
[
"search",
"find",
"orders",
"--kind",
"schema",
"--json",
"-c",
str(tmp_path / "runtime.yaml"),
],
)
assert result.exit_code == 1
assert "effective relationship snapshot is missing" in result.stderr
def test_suggest_fks_cannot_write_when_relationships_are_catalog_managed(tmp_path):
_workspace(
tmp_path,
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
)
annotations = tmp_path / "artifacts" / "mschema" / "annotations.yaml"
before = annotations.read_text()
result = RUNNER.invoke(
app,
[
"schema",
"suggest-fks",
"--write",
"-c",
str(tmp_path / "runtime.yaml"),
],
)
assert result.exit_code == 1
assert "managed by the catalog" in result.stderr
assert annotations.read_text() == before
+48 -21
View File
@@ -10,6 +10,12 @@ from tht.adapters.factory import build_dwh
from tht.cli.config_cmd import CONFIG_OPT
from tht.config import ConfigError, load_config
from tht.db.sampling import is_text_type
from tht.mschema.context import (
SchemaContextError,
annotations_path,
load_schema_context,
physical_path,
)
from tht.mschema.eligibility import classify_all
schema_app = typer.Typer(help="Gestione mschema (rappresentazione canonica dello schema)")
@@ -40,20 +46,6 @@ def _load_config_or_exit(config: Path):
raise typer.Exit(code=1)
def physical_path(cfg) -> Path:
from tht.jobs.dwh_pipeline import resolve_dwh_snapshot
if not (cfg.paths.artifacts.parent / ".tht-dwh").exists():
return cfg.paths.artifacts / "mschema" / "physical.yaml"
return resolve_dwh_snapshot(cfg).physical
def annotations_path(cfg) -> Path:
if cfg.paths.annotations_root is not None:
return cfg.paths.annotations_root / "mschema" / "annotations.yaml"
return cfg.paths.artifacts / "mschema" / "annotations.yaml"
def refresh_catalog(cfg, *, dwh=None, output_path: Path | None = None):
"""Run the existing catalog algorithm and persist its canonical output."""
target = dwh if dwh is not None else build_dwh(cfg)
@@ -432,6 +424,21 @@ def suggest_fks_cmd(
from tht.mschema.models import Annotations, PhysicalSchema, TableAnnotation
cfg = _load_config_or_exit(config)
if write and cfg.paths.effective_relationships is not None:
error = "effective relationships are managed by the catalog; --write is disabled"
if json_output:
_emit_json({
"code": "catalog_relationships_managed",
"error": error,
"operation": "schema_suggest_fks",
"schemaVersion": 1,
"status": "failed",
"workspaceId": cfg._workspace_id,
"workspaceRevision": cfg._workspace_revision,
})
else:
typer.secho(f"ERRORE: {error}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
phys_file = physical_path(cfg)
if not phys_file.exists():
if json_output:
@@ -556,7 +563,6 @@ def render_cmd(
"""Serializza mschema (physical + annotations) nel formato richiesto."""
import json
from tht.mschema.models import Annotations, PhysicalSchema
from tht.mschema.render import to_markdown, to_mschema_text, to_schema_dict
cfg = _load_config_or_exit(config)
@@ -564,19 +570,40 @@ def render_cmd(
if not phys_file.exists():
typer.secho(
f"ERRORE: {phys_file} non trovato. Esegui prima `tht schema introspect`.",
fg=typer.colors.RED, err=True,
fg=typer.colors.RED,
err=True,
)
raise typer.Exit(code=1)
physical = PhysicalSchema.from_yaml(phys_file)
annotations = Annotations.from_yaml(annotations_path(cfg))
try:
context = load_schema_context(cfg, physical_file=phys_file)
except SchemaContextError as exc:
typer.secho(f"ERRORE: {exc}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1) from None
table_filter = list(tables) if tables else None
if format == "markdown":
out = to_markdown(physical, annotations)
out = to_markdown(
context.physical,
context.annotations,
effective_relationships=context.effective_relationships,
)
elif format == "mschema-text":
out = to_mschema_text(physical, annotations, tables=table_filter)
out = to_mschema_text(
context.physical,
context.annotations,
tables=table_filter,
effective_relationships=context.effective_relationships,
)
elif format == "schema-dict":
out = json.dumps(to_schema_dict(physical, annotations), ensure_ascii=False, indent=2)
out = json.dumps(
to_schema_dict(
context.physical,
context.annotations,
effective_relationships=context.effective_relationships,
),
ensure_ascii=False,
indent=2,
)
else:
typer.secho(f"ERRORE: formato sconosciuto: {format}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
+12 -5
View File
@@ -231,8 +231,7 @@ def search_cmd(
)
if kind == "schema":
from tht.cli.schema_cmd import annotations_path
from tht.mschema.models import Annotations, PhysicalSchema
from tht.mschema.context import SchemaContextError, load_schema_context
from tht.mschema.render import to_mschema_text
from tht.search import schema_tables
@@ -243,6 +242,11 @@ def search_cmd(
fg=typer.colors.RED, err=True,
)
raise typer.Exit(code=1)
try:
schema_context = load_schema_context(cfg, physical_file=phys_file)
except SchemaContextError as exc:
typer.secho(f"ERRORE: {exc}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1) from None
candidates = combined_search(
keyword=keyword, lsh_hits=lsh_hits,
@@ -258,10 +262,13 @@ def search_cmd(
typer.secho(f"Nessuna tabella candidata per '{keyword}'.", fg=typer.colors.YELLOW)
return
physical = PhysicalSchema.from_yaml(phys_file)
annotations = Annotations.from_yaml(annotations_path(cfg))
selected = [t for t, _ in ranked]
mschema = to_mschema_text(physical, annotations, tables=selected)
mschema = to_mschema_text(
schema_context.physical,
schema_context.annotations,
tables=selected,
effective_relationships=schema_context.effective_relationships,
)
if json_out:
typer.echo(json.dumps(
+4
View File
@@ -327,6 +327,9 @@ class PathsConfig(BaseModel):
# Revision-qualified curated FK annotations root (P5). When absent, legacy
# `artifacts/mschema/annotations.yaml` remains the annotations source.
annotations_root: Path | None = None
# Runtime-only, backend-derived effective relationship snapshot. When present,
# it is the exclusive FK source; it is not an authored workspace artifact.
effective_relationships: Path | None = None
class RuntimeIdentityConfig(BaseModel):
@@ -676,6 +679,7 @@ def load_config(path: Path) -> Config:
indexes=resolved.indexes,
memory=cfg.paths.memory,
annotations_root=cfg.paths.annotations_root,
effective_relationships=cfg.paths.effective_relationships,
)
}
)
+133
View File
@@ -0,0 +1,133 @@
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Annotated, Literal
from pydantic import BaseModel, Field, ValidationError, model_validator
from tht.mschema.models import Annotations, ForeignKey, PhysicalSchema
class SchemaContextError(ValueError):
"""The effective schema inputs cannot be used safely."""
RelationshipName = Annotated[str, Field(min_length=1)]
class EffectiveRelationship(BaseModel):
source_table: RelationshipName = Field(alias="sourceTable")
source_columns: list[RelationshipName] = Field(alias="sourceColumns", min_length=1)
target_table: RelationshipName = Field(alias="targetTable")
target_columns: list[RelationshipName] = Field(alias="targetColumns", min_length=1)
origin: Literal["physical", "generated", "manual"]
model_config = {"extra": "forbid", "populate_by_name": True}
@model_validator(mode="after")
def columns_are_paired(self):
if len(self.source_columns) != len(self.target_columns):
raise ValueError("sourceColumns and targetColumns must have the same length")
return self
class EffectiveRelationshipSnapshot(BaseModel):
schema_version: Literal[1] = Field(alias="schemaVersion")
workspace_id: RelationshipName = Field(alias="workspaceId")
relationships: list[EffectiveRelationship]
model_config = {"extra": "forbid", "populate_by_name": True}
@dataclass(frozen=True)
class SchemaContext:
physical: PhysicalSchema
annotations: Annotations
# None means legacy physical + annotation FK merging. A dict, including an
# empty one, means the catalog snapshot is the exclusive relationship source.
effective_relationships: dict[str, list[ForeignKey]] | None
def physical_path(cfg) -> Path:
from tht.jobs.dwh_pipeline import resolve_dwh_snapshot
if not (cfg.paths.artifacts.parent / ".tht-dwh").exists():
return cfg.paths.artifacts / "mschema" / "physical.yaml"
return resolve_dwh_snapshot(cfg).physical
def annotations_path(cfg) -> Path:
if cfg.paths.annotations_root is not None:
return cfg.paths.annotations_root / "mschema" / "annotations.yaml"
return cfg.paths.artifacts / "mschema" / "annotations.yaml"
def _load_effective_relationships(cfg, physical: PhysicalSchema) -> dict[str, list[ForeignKey]] | None:
path = cfg.paths.effective_relationships
if path is None:
return None
if not path.is_file():
raise SchemaContextError(f"effective relationship snapshot is missing: {path}")
try:
raw = json.loads(path.read_text())
snapshot = EffectiveRelationshipSnapshot.model_validate(raw)
except (OSError, json.JSONDecodeError, ValidationError) as exc:
raise SchemaContextError(f"effective relationship snapshot is invalid: {path}") from exc
if snapshot.workspace_id != cfg._workspace_id:
raise SchemaContextError(
"effective relationship snapshot workspace does not match runtime workspace"
)
by_table: dict[str, list[ForeignKey]] = {}
seen: set[tuple[str, tuple[str, ...], str, tuple[str, ...]]] = set()
for relationship in snapshot.relationships:
source = physical.tables.get(relationship.source_table)
target = physical.tables.get(relationship.target_table)
if source is None or target is None:
raise SchemaContextError(
"effective relationship endpoint table is absent from physical schema: "
f"{relationship.source_table}->{relationship.target_table}"
)
missing_source = [name for name in relationship.source_columns if name not in source.columns]
missing_target = [name for name in relationship.target_columns if name not in target.columns]
if missing_source or missing_target:
missing = ", ".join(
[f"{relationship.source_table}.{name}" for name in missing_source]
+ [f"{relationship.target_table}.{name}" for name in missing_target]
)
raise SchemaContextError(
f"effective relationship endpoint column is absent from physical schema: {missing}"
)
key = (
relationship.source_table,
tuple(relationship.source_columns),
relationship.target_table,
tuple(relationship.target_columns),
)
if key in seen:
continue
seen.add(key)
by_table.setdefault(relationship.source_table, []).append(
ForeignKey(
columns=relationship.source_columns,
ref_table=relationship.target_table,
ref_columns=relationship.target_columns,
)
)
return by_table
def load_schema_context(cfg, *, physical_file: Path | None = None) -> SchemaContext:
physical_file = physical_file or physical_path(cfg)
if not physical_file.is_file():
raise SchemaContextError(f"physical schema is missing: {physical_file}")
try:
physical = PhysicalSchema.from_yaml(physical_file)
annotations = Annotations.from_yaml(annotations_path(cfg))
except (OSError, ValidationError, ValueError) as exc:
raise SchemaContextError("physical schema or annotations are invalid") from exc
return SchemaContext(
physical=physical,
annotations=annotations,
effective_relationships=_load_effective_relationships(cfg, physical),
)
+30 -7
View File
@@ -7,16 +7,23 @@ MAX_EXAMPLES_IN_PROMPT = 5
def table_foreign_keys(
physical: PhysicalSchema, annotations: Annotations, table: str
physical: PhysicalSchema,
annotations: Annotations,
table: str,
effective_relationships: dict[str, list[ForeignKey]] | None = None,
) -> list[ForeignKey]:
"""FK fisiche + FK logiche dalle annotations (dedup su columns/ref)."""
if effective_relationships is not None:
return list(effective_relationships.get(table, []))
fks = list(physical.tables[table].foreign_keys)
ann = annotations.tables.get(table)
if ann:
seen = {(tuple(f.columns), f.ref_table, tuple(f.ref_columns)) for f in fks}
for fk in ann.foreign_keys:
if (tuple(fk.columns), fk.ref_table, tuple(fk.ref_columns)) not in seen:
key = (tuple(fk.columns), fk.ref_table, tuple(fk.ref_columns))
if key not in seen:
fks.append(fk)
seen.add(key)
return fks
@@ -47,6 +54,8 @@ def to_mschema_text(
physical: PhysicalSchema,
annotations: Annotations | None = None,
tables: list[str] | None = None,
*,
effective_relationships: dict[str, list[ForeignKey]] | None = None,
) -> str:
"""Serializzazione testuale in stile ThothAI (【Schema】/【Foreign keys】)."""
annotations = annotations or Annotations()
@@ -73,7 +82,9 @@ def to_mschema_text(
shown = ", ".join(column.examples[:MAX_EXAMPLES_IN_PROMPT])
lines.append(f" -- Examples: {shown}")
lines.append(");")
for fk in table_foreign_keys(physical, annotations, table_name):
for fk in table_foreign_keys(
physical, annotations, table_name, effective_relationships
):
for src, dst in zip(fk.columns, fk.ref_columns):
fk_lines.append(f"{table_name}.{src}={fk.ref_table}.{dst}")
lines.extend(["", "【Foreign keys】", *fk_lines])
@@ -81,7 +92,10 @@ def to_mschema_text(
def to_schema_dict(
physical: PhysicalSchema, annotations: Annotations | None = None
physical: PhysicalSchema,
annotations: Annotations | None = None,
*,
effective_relationships: dict[str, list[ForeignKey]] | None = None,
) -> dict[str, Any]:
"""Vista compatibile con le logiche AV-SQL (schema_dict)."""
annotations = annotations or Annotations()
@@ -103,13 +117,20 @@ def to_schema_dict(
"primary_keys": [c for c in cols if table.columns[c].pk],
"foreign_keys": [
{"columns": fk.columns, "ref_table": fk.ref_table, "ref_columns": fk.ref_columns}
for fk in table_foreign_keys(physical, annotations, table_name)
for fk in table_foreign_keys(
physical, annotations, table_name, effective_relationships
)
],
}
return out
def to_markdown(physical: PhysicalSchema, annotations: Annotations | None = None) -> str:
def to_markdown(
physical: PhysicalSchema,
annotations: Annotations | None = None,
*,
effective_relationships: dict[str, list[ForeignKey]] | None = None,
) -> str:
"""Report leggibile per il reviewer."""
annotations = annotations or Annotations()
lines = [
@@ -146,7 +167,9 @@ def to_markdown(physical: PhysicalSchema, annotations: Annotations | None = None
f"| {column_name} | {column.type} | {'sì' if column.nullable else 'no'} "
f"| {'sì' if column.pk else ''} | {cdesc} | {examples} |"
)
fks = table_foreign_keys(physical, annotations, table_name)
fks = table_foreign_keys(
physical, annotations, table_name, effective_relationships
)
if fks:
lines += ["", "Foreign keys:"]
for fk in fks:
+1
View File
@@ -85,6 +85,7 @@ nav:
- 0009 Sequential description generation: adr/0009-use-one-sequential-description-generation-run.md
- 0010 Bounded source samples: adr/0010-allow-bounded-real-source-samples-for-description-generation.md
- 0011 Sensitive data flag: adr/0011-gate-source-samples-with-a-sensitive-data-flag.md
- 0012 Effective relationship authority: adr/0012-use-the-catalog-as-the-logical-relationship-authority.md
- AI catalog description acceptance: testing/2026-08-29-ai-catalog-description-generation-acceptance.md
- Design records:
- Metadata catalog design: plans/2026-08-26-metadata-catalog-from-thothai.md