feat: consolidate database management work
Add catalog-owned logical relationships and runtime snapshots, extend the database-management UI and validation coverage, and document the updated operational workflow. Keep active sensitive-generation status in a tooltip and indicator, and update the layout E2E to follow the history action in its new database-scoped location.
This commit is contained in:
+34
-2
@@ -296,8 +296,40 @@ Metadata Catalog. Non è creata o modificata manualmente, ma può essere rimossa
|
||||
Metadata Cleanup.
|
||||
_Avoid_: denormalized FK, relationship string
|
||||
|
||||
**Logical Relationship** — Una relazione semantica curata o inferita che non corrisponde
|
||||
necessariamente a un vincolo fisico. Ha ownership e lifecycle distinti da Catalog Relationship.
|
||||
**Logical Relationship** — Una relazione modificabile fra due Catalog Column che non corrisponde
|
||||
necessariamente a un vincolo fisico. Può essere Generated o Manual e rimane distinta dalla Catalog
|
||||
Relationship osservata nel database.
|
||||
|
||||
**Generated Relationship** — Una Logical Relationship ricavata dai nomi delle colonne, dalle
|
||||
primary key e dalla compatibilità dei tipi mediante regole deterministiche, senza LLM, embedding o
|
||||
campionamento dei dati. Una ricostruzione non riattiva una Generated Relationship cancellata
|
||||
logicamente, ma può ricrearne una cancellata fisicamente.
|
||||
|
||||
**Manual Relationship** — Una Logical Relationship aggiunta dall'utente. La ricostruzione delle
|
||||
Generated Relationship non la modifica.
|
||||
|
||||
**Logical Relationship Deletion** — L'esclusione persistente di una Logical Relationship che ne
|
||||
conserva l'identità per impedirne la ricreazione automatica finché esistono entrambe le Catalog
|
||||
Column alle quali è collegata.
|
||||
|
||||
**Permanent Relationship Deletion** — La rimozione completa di una Logical Relationship. Una
|
||||
ricostruzione successiva può ricrearla quando soddisfa nuovamente le regole di inferenza. Anche il
|
||||
cleanup distruttivo di una tabella o colonna endpoint rimuove permanentemente le relative esclusioni.
|
||||
|
||||
**Relationship Reconstruction** — L'operazione amministrativa esplicita che scopre e aggiunge le
|
||||
Generated Relationship mancanti. Conserva le Manual Relationship e le relationship già presenti e
|
||||
non riattiva quelle cancellate logicamente.
|
||||
|
||||
**Relationship Restore** — La riattivazione esplicita di una Logical Relationship cancellata
|
||||
logicamente.
|
||||
|
||||
**Effective Relationship Map** — La vista unificata delle Catalog Relationship fisiche e delle
|
||||
Logical Relationship, con origine e stato espliciti. È l'interfaccia usata dall'amministrazione e
|
||||
dalla comprensione dello schema, non un ulteriore modello persistito.
|
||||
|
||||
**Effective Relationship Snapshot** — La proiezione runtime immutabile delle relationship attive
|
||||
contenute nell'Effective Relationship Map. È derivata dal Metadata Catalog per una singola sessione
|
||||
e viene eliminata insieme alla relativa configurazione runtime.
|
||||
|
||||
**Description** — Il testo curato e consolidato che descrive una Catalog Table o Catalog Column
|
||||
per gli usi downstream.
|
||||
|
||||
@@ -12,6 +12,10 @@ colors:
|
||||
muted-graphite: "oklch(51.33% 0.0088 345.6)"
|
||||
quiet-border: "oklch(90.93% 0.0035 354.7)"
|
||||
success-mint: "oklch(75.77% 0.1581 165)"
|
||||
navigation-active: "oklch(92.5% 0.052 23.2)"
|
||||
navigation-active-hover: "oklch(89.5% 0.071 23.2)"
|
||||
navigation-active-foreground: "oklch(36.5% 0.11 23.2)"
|
||||
navigation-active-border: "oklch(60% 0.135 23.2)"
|
||||
warning-amber: "oklch(85.23% 0.1386 78.9)"
|
||||
information-blue: "oklch(70.35% 0.1128 221.3)"
|
||||
typography:
|
||||
@@ -152,8 +156,8 @@ frontend uses OKLCH tokens directly.
|
||||
|
||||
### Primary
|
||||
|
||||
- **Instrument Red** (`instrument-red`): primary actions, current selection, focus identity, and
|
||||
destructive meaning where the context already makes the action explicit.
|
||||
- **Instrument Red** (`instrument-red`): primary actions, focus identity, and destructive meaning
|
||||
where the context already makes the action explicit.
|
||||
- **Instrument Red Pressed** (`instrument-red-hover`): hover and active emphasis for the primary
|
||||
action family.
|
||||
|
||||
@@ -170,6 +174,9 @@ frontend uses OKLCH tokens directly.
|
||||
### Semantic
|
||||
|
||||
- **Success Mint** (`success-mint`): completed and ready states.
|
||||
- **Navigation Active** (`navigation-active`): the one application surface currently in the
|
||||
foreground. It shares Instrument Red's hue but uses a lighter, lower-chroma fill, so location is
|
||||
visible without carrying the full weight of a primary action.
|
||||
- **Warning Amber** (`warning-amber`): waiting, attention, and in-progress states.
|
||||
- **Information Blue** (`information-blue`): informational state when red would imply action.
|
||||
|
||||
@@ -272,15 +279,44 @@ default, hover, focus, active, disabled, loading, and error behavior where those
|
||||
- **Focus:** three-pixel Instrument Red ring with a clear border shift.
|
||||
- **Error / Disabled:** errors combine destructive color with explanatory text; disabled controls
|
||||
retain readable contrast and use 50 percent opacity.
|
||||
- **Metadata catalog model:** Database Management keeps one compact, installation-level
|
||||
metadata-generation LLM selector in the application header. The selection persists across
|
||||
database, table, column, and relationship views; when no usable profile is configured, the
|
||||
disabled control explains: “No metadata-generation LLM model is configured for this installation.”
|
||||
|
||||
### Navigation
|
||||
|
||||
- **Style:** compact session rows use `8px` corners and restrained vertical padding.
|
||||
- **Default / Hover / Active:** transparent at rest, Sunken Surface on hover, and the same surface
|
||||
with stronger text weight when active.
|
||||
- **Default / Hover / Active:** porcelain at rest, Sunken Surface on hover, and a muted Navigation
|
||||
Active red with a defined border when current. Exactly one top-level navigation control is current.
|
||||
- **Administrative controls:** the admin-only Administration accordion groups Database management,
|
||||
a structural divider, Workspace management, and Pi management in that order. Its trigger exposes
|
||||
expanded state and starts collapsed by default, while non-admin users do not receive the accordion
|
||||
or its navigation actions.
|
||||
- **Responsive:** collapse navigation structurally at the application breakpoint. Do not shrink
|
||||
labels into illegibility.
|
||||
|
||||
### Tabs
|
||||
|
||||
- **Shape:** compact label tabs sit on a shared baseline with rounded top corners and a two-pixel
|
||||
lower edge. Inactive labels retain a complete Quiet Border and Porcelain Card surface, so every
|
||||
label reads as a tab before interaction; hover feedback reinforces clickability.
|
||||
- **Current:** the selected tab uses the muted Navigation Active red for its fill, text, and defined border.
|
||||
It must expose `aria-selected`, participate in a labelled `tablist`/`tabpanel`, and be the only
|
||||
tab in the roving keyboard tab order.
|
||||
- **Keyboard:** Left/Right move between adjacent tabs with wrapping; Home/End select the first or
|
||||
last tab.
|
||||
|
||||
### Tooltips
|
||||
|
||||
- **Row actions:** icon-action tooltips open three pixels below the trigger and align to its trailing
|
||||
edge, so they never cover the icon row. They use a dark slate surface, porcelain text, and a
|
||||
defined border rather than the light popover treatment.
|
||||
- **Interaction:** tooltip layers never receive pointer events. They appear on hover and keyboard
|
||||
focus with a short ease-out transition, while the icon button keeps its complete accessible name.
|
||||
- **Scope:** this treatment is shared by database, table, column, and relationship row actions.
|
||||
Toolbar and navigation hints may use separate collision-aware placement.
|
||||
|
||||
### Curated Evidence Documents
|
||||
|
||||
Curated evidence follows a fixed reading order: title, compact type and purpose summary, scope,
|
||||
|
||||
+25
-11
@@ -1,6 +1,6 @@
|
||||
# ThothII — Project State
|
||||
|
||||
Last updated: 2026-08-27.
|
||||
Last updated: 2026-08-31.
|
||||
|
||||
This file is the short operational snapshot. Stable commands and the architecture mental model
|
||||
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
|
||||
@@ -90,6 +90,18 @@ relationships. Curated and generated descriptions are editable; generated descri
|
||||
and Database Management can generate or consolidate them for selected tables, selected columns,
|
||||
all targets, or only targets whose Generated Description is missing.
|
||||
|
||||
Relationship Management is now reachable directly from each configured Fleet database. One
|
||||
Relationship Map shows read-only Physical Relationships together with Generated and Manual Logical
|
||||
Relationships, with Active, Excluded, and All filters. Administrators can add a single-column
|
||||
relationship, run deterministic name/PK/type inference, exclude or restore a logical relationship,
|
||||
or delete it permanently. Exclusion retains a tombstone that a rebuild cannot reactivate; permanent
|
||||
deletion allows a later rebuild to infer the same endpoints again. Inference uses no LLM, embedding,
|
||||
or source values. It supports normalized table-qualified names, unique non-generic PK names,
|
||||
composite-PK source columns, and the `*time_key -> dim_time.<single PK>` warehouse convention while
|
||||
ignoring bare generic names. Explicit table/column metadata cleanup remains a destructive boundary: it removes
|
||||
the attached logical relationships and exclusions and requires a full schema synchronization before
|
||||
inference or runtime publication can continue.
|
||||
|
||||
The previous Database Management renderer remains a temporary comparison fallback for development
|
||||
and staging only: `?db-ui=legacy` is honored in Vite development or when
|
||||
`VITE_DB_MANAGEMENT_LEGACY=true`; it is not a production presentation. The standalone Fleet Ledger
|
||||
@@ -115,13 +127,15 @@ SSH is not yet enabled for NL→SQL session runtime.
|
||||
|
||||
The catalog runs in the internal `catalog-db` PostgreSQL service. Kysely migrations are an explicit
|
||||
one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before local startup. Runtime
|
||||
sessions still consume the existing workspace configuration in this slice: database-management
|
||||
records do not yet change the NL→SQL handoff. The accepted design is recorded in
|
||||
sessions now consume the Catalog's active effective relationship map through an immutable JSON
|
||||
snapshot tied to the runtime-config lease. The harness uses that snapshot as its exclusive
|
||||
relationship source while retaining Git-pinned annotations for descriptive metadata; legacy
|
||||
runtimes without a snapshot keep the previous merge behavior. The accepted design is recorded in
|
||||
`docs/plans/2026-08-26-metadata-catalog-from-thothai.md`, the snapshot contract under
|
||||
`docs/contracts/`, and ADRs 0001–0011.
|
||||
`docs/contracts/`, and ADRs 0001–0012.
|
||||
|
||||
Semantic aliases, value descriptions, synonyms, concepts, and logical relationships remain
|
||||
deferred to their dedicated slices.
|
||||
Semantic aliases, value descriptions, synonyms, and concepts remain deferred to their dedicated
|
||||
slices.
|
||||
|
||||
AI Description Generation uses the catalog's human-owned Sensitive Data Flag. The flag defaults to
|
||||
`false`, including for newly synchronized columns. An administrator may request an AI proposal based
|
||||
@@ -158,11 +172,11 @@ credential. The Python client supplies only its fixed non-secret compatibility p
|
||||
The AritmoLab entry also sets `disableThinking: true`, mapped to the endpoint's chat-template flag,
|
||||
because its default reasoning prose would violate the worker's exact JSON response contract.
|
||||
|
||||
Integration of the completed metadata catalog with core schema-linking is explicitly deferred
|
||||
until the database, table, column, relationship, and synchronization slices are complete. At that
|
||||
point the next required design gate is to compare the catalog snapshot with the current DWH
|
||||
preprocessing/schema-linking contracts and plan the cutover; this follow-up must not be treated as
|
||||
optional cleanup or silently omitted.
|
||||
Logical relationship integration with core schema-linking is complete: session creation and resume
|
||||
materialize the active physical/generated/manual map, retrieval-pack generation and Pi receive the
|
||||
same runtime config, and snapshot validation fails closed on a declared missing, invalid, or orphaned
|
||||
endpoint. Broader publication of other Catalog metadata to schema-linking remains a separate future
|
||||
slice.
|
||||
|
||||
**Deferred follow-up — Sensitive Data Policy in schema-linking.** The policy is first delivered
|
||||
and tested in catalog description generation. Its enforcement for core schema-linking remains
|
||||
|
||||
@@ -63,6 +63,9 @@ import {
|
||||
type DescriptionSourceSampler,
|
||||
} from "./catalog/description-source-sampler.js";
|
||||
import { catalogDescriptionGenerationRoutes } from "./routes/catalog-description-generation.js";
|
||||
import { CatalogLogicalRelationshipService } from "./catalog/logical-relationship-service.js";
|
||||
import { catalogLogicalRelationshipRoutes } from "./routes/catalog-logical-relationships.js";
|
||||
import { EffectiveRelationshipSnapshotProvider } from "./catalog/effective-relationship-snapshot.js";
|
||||
|
||||
export interface BuildAppDeps {
|
||||
thtRunner?: ThtRunner;
|
||||
@@ -79,6 +82,8 @@ export interface BuildAppDeps {
|
||||
catalogService?: CatalogService;
|
||||
catalogPostgresAccess?: CatalogPostgresAccess;
|
||||
catalogTableService?: CatalogTableService;
|
||||
catalogLogicalRelationshipService?: CatalogLogicalRelationshipService;
|
||||
effectiveRelationshipSnapshotProvider?: EffectiveRelationshipSnapshotProvider;
|
||||
catalogSchemaIntrospector?: CatalogSchemaIntrospector;
|
||||
catalogSyncWorker?: CatalogSyncWorker;
|
||||
catalogOperationCoordinator?: CatalogOperationCoordinator;
|
||||
@@ -200,6 +205,16 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
catalogOperationCoordinator,
|
||||
);
|
||||
const catalogTableService = deps?.catalogTableService ?? new CatalogTableService(catalogRepository);
|
||||
const catalogLogicalRelationshipService = deps?.catalogLogicalRelationshipService
|
||||
?? new CatalogLogicalRelationshipService(catalogRepository);
|
||||
const effectiveRelationships = deps?.effectiveRelationshipSnapshotProvider
|
||||
?? (config.catalogDatabase === undefined
|
||||
? undefined
|
||||
: new EffectiveRelationshipSnapshotProvider(
|
||||
catalogRepository,
|
||||
catalogLogicalRelationshipService,
|
||||
catalogOperationCoordinator,
|
||||
));
|
||||
const catalogSchemaIntrospector = deps?.catalogSchemaIntrospector ?? new ConcreteCatalogSchemaIntrospector(
|
||||
catalogPostgresAccess,
|
||||
workspaceSecretStore,
|
||||
@@ -394,6 +409,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
legacyWorkspaceMode: config.legacyWorkspaceMode,
|
||||
workspaceRuntimeSupport,
|
||||
maintenanceBarrier,
|
||||
effectiveRelationships,
|
||||
});
|
||||
app.post("/internal/maintenance/activate", async (req, reply) => {
|
||||
try {
|
||||
@@ -442,6 +458,10 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
||||
worker: catalogSyncWorker,
|
||||
operations: catalogOperationCoordinator,
|
||||
});
|
||||
catalogLogicalRelationshipRoutes(app, {
|
||||
service: catalogLogicalRelationshipService,
|
||||
operations: catalogOperationCoordinator,
|
||||
});
|
||||
catalogDescriptionConsolidationRoutes(app, {
|
||||
repository: catalogRepository,
|
||||
operations: catalogOperationCoordinator,
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
import type { CatalogRelationship, CatalogRepository } from "./types.js";
|
||||
|
||||
export interface EffectiveRelationshipSnapshotReader {
|
||||
list(databaseId: string): Promise<CatalogRelationship[]>;
|
||||
}
|
||||
|
||||
export interface EffectiveRelationshipSnapshotCoordinator {
|
||||
run<T>(databaseId: string, operation: () => Promise<T>): Promise<T>;
|
||||
}
|
||||
|
||||
export class EffectiveRelationshipSnapshotStaleError extends Error {}
|
||||
|
||||
interface EffectiveRelationship {
|
||||
sourceTable: string;
|
||||
sourceColumns: string[];
|
||||
targetTable: string;
|
||||
targetColumns: string[];
|
||||
origin: CatalogRelationship["origin"];
|
||||
}
|
||||
|
||||
interface EffectiveRelationshipSnapshot {
|
||||
schemaVersion: 1;
|
||||
workspaceId: string;
|
||||
relationships: EffectiveRelationship[];
|
||||
}
|
||||
|
||||
const originRank: Record<CatalogRelationship["origin"], number> = {
|
||||
physical: 0,
|
||||
manual: 1,
|
||||
generated: 2,
|
||||
};
|
||||
|
||||
function endpointKey(relationship: EffectiveRelationship): string {
|
||||
return [
|
||||
relationship.sourceTable,
|
||||
relationship.sourceColumns.join("\u0000"),
|
||||
relationship.targetTable,
|
||||
relationship.targetColumns.join("\u0000"),
|
||||
].join("\u0001");
|
||||
}
|
||||
|
||||
function compareRelationships(left: EffectiveRelationship, right: EffectiveRelationship): number {
|
||||
return endpointKey(left).localeCompare(endpointKey(right))
|
||||
|| originRank[left.origin] - originRank[right.origin];
|
||||
}
|
||||
|
||||
/**
|
||||
* Adapter from the mutable Catalog model to the immutable relationship contract consumed by the
|
||||
* harness. The returned JSON is a deterministic projection, never an authored second store.
|
||||
*/
|
||||
export class EffectiveRelationshipSnapshotProvider {
|
||||
constructor(
|
||||
private readonly repository: Pick<CatalogRepository, "get" | "getByWorkspace">,
|
||||
private readonly relationships: EffectiveRelationshipSnapshotReader,
|
||||
private readonly operations: EffectiveRelationshipSnapshotCoordinator,
|
||||
) {}
|
||||
|
||||
async render(workspaceId: string): Promise<string | undefined> {
|
||||
const database = await this.repository.getByWorkspace(workspaceId);
|
||||
if (!database) return undefined;
|
||||
return await this.operations.run(database.id, async () => {
|
||||
const current = await this.repository.get(database.id);
|
||||
if (!current || current.schemaSyncedVersion !== current.version) {
|
||||
throw new EffectiveRelationshipSnapshotStaleError(
|
||||
"effective relationship snapshot requires a current full schema synchronization",
|
||||
);
|
||||
}
|
||||
const projected = (await this.relationships.list(database.id))
|
||||
.filter((relationship) => relationship.status === "active")
|
||||
.map((relationship): EffectiveRelationship => ({
|
||||
sourceTable: relationship.sourceTableName,
|
||||
sourceColumns: relationship.columns.map((column) => column.sourceColumnName),
|
||||
targetTable: relationship.targetTableName,
|
||||
targetColumns: relationship.columns.map((column) => column.targetColumnName),
|
||||
origin: relationship.origin,
|
||||
}))
|
||||
.sort(compareRelationships);
|
||||
|
||||
const seen = new Set<string>();
|
||||
const snapshot: EffectiveRelationshipSnapshot = {
|
||||
schemaVersion: 1,
|
||||
workspaceId,
|
||||
relationships: projected.filter((relationship) => {
|
||||
const key = endpointKey(relationship);
|
||||
if (seen.has(key)) return false;
|
||||
seen.add(key);
|
||||
return true;
|
||||
}),
|
||||
};
|
||||
return `${JSON.stringify(snapshot, null, 2)}\n`;
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,260 @@
|
||||
import type {
|
||||
CatalogLogicalRelationship,
|
||||
CatalogLogicalRelationshipCandidate,
|
||||
CatalogLogicalRelationshipContext,
|
||||
CatalogLogicalRelationshipEndpoint,
|
||||
CatalogRelationship,
|
||||
CatalogRepository,
|
||||
} from "./types.js";
|
||||
|
||||
export class LogicalRelationshipDatabaseNotFoundError extends Error {}
|
||||
export class LogicalRelationshipDuplicateError extends Error {}
|
||||
export class LogicalRelationshipNotFoundError extends Error {}
|
||||
export class LogicalRelationshipReadOnlyError extends Error {}
|
||||
export class LogicalRelationshipSchemaStaleError extends Error {}
|
||||
export class LogicalRelationshipTargetNotUniqueError extends Error {}
|
||||
export class LogicalRelationshipTypeIncompatibleError extends Error {}
|
||||
export class LogicalRelationshipColumnNotFoundError extends Error {
|
||||
constructor(readonly field: "sourceColumnId" | "targetColumnId") {
|
||||
super(`Catalog column '${field}' was not found`);
|
||||
}
|
||||
}
|
||||
|
||||
export interface RebuildGeneratedRelationshipsResult {
|
||||
added: number;
|
||||
alreadyPresent: number;
|
||||
excluded: number;
|
||||
ambiguous: number;
|
||||
}
|
||||
|
||||
function identifierTokens(value: string): string[] {
|
||||
return value
|
||||
.replace(/([a-z0-9])([A-Z])/g, "$1_$2")
|
||||
.toLowerCase()
|
||||
.split(/[^a-z0-9]+/)
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
function singularWord(value: string): string {
|
||||
if (value.length > 4 && value.endsWith("ies")) return `${value.slice(0, -3)}y`;
|
||||
if (value.length > 4 && /(ches|shes|xes|zes|ses)$/.test(value)) return value.slice(0, -2);
|
||||
if (value.length > 3 && value.endsWith("s") && !/(ss|us)$/.test(value)) return value.slice(0, -1);
|
||||
return value;
|
||||
}
|
||||
|
||||
function tableAliases(tableName: string): string[] {
|
||||
const tokens = identifierTokens(tableName);
|
||||
if (tokens.length === 0) return [];
|
||||
const normalized = tokens.join("_");
|
||||
const singular = [...tokens];
|
||||
singular[singular.length - 1] = singularWord(singular[singular.length - 1]);
|
||||
return [...new Set([normalized, singular.join("_")])];
|
||||
}
|
||||
|
||||
const GENERIC_PRIMARY_KEY_NAMES = new Set(["id", "key", "code", "pk"]);
|
||||
|
||||
function nameMatches(
|
||||
source: CatalogLogicalRelationshipEndpoint,
|
||||
target: CatalogLogicalRelationshipEndpoint,
|
||||
): boolean {
|
||||
const sourceName = identifierTokens(source.columnName).join("_");
|
||||
const targetName = identifierTokens(target.columnName).join("_");
|
||||
if (!sourceName || !targetName) return false;
|
||||
const expected = new Set<string>();
|
||||
if (!GENERIC_PRIMARY_KEY_NAMES.has(targetName)) expected.add(targetName);
|
||||
for (const alias of tableAliases(target.tableName)) {
|
||||
expected.add(`${alias}_${targetName}`);
|
||||
expected.add(`${alias.replaceAll("_", "")}${targetName.replaceAll("_", "")}`);
|
||||
if (targetName === "id" || targetName === "pk") expected.add(alias);
|
||||
}
|
||||
return expected.has(sourceName);
|
||||
}
|
||||
|
||||
function canonicalDataType(value: string): string {
|
||||
const normalized = value.trim().toLowerCase().replace(/\s+/g, " ");
|
||||
const arraySuffix = normalized.endsWith("[]") ? "[]" : "";
|
||||
const base = arraySuffix ? normalized.slice(0, -2) : normalized;
|
||||
const withoutModifier = base.replace(/\([^)]*\)/g, "").trim();
|
||||
const aliases: Record<string, string> = {
|
||||
int2: "smallint",
|
||||
smallserial: "smallint",
|
||||
int4: "integer",
|
||||
int: "integer",
|
||||
serial: "integer",
|
||||
int8: "bigint",
|
||||
bigserial: "bigint",
|
||||
decimal: "numeric",
|
||||
varchar: "text",
|
||||
"character varying": "text",
|
||||
bool: "boolean",
|
||||
"timestamp without time zone": "timestamp",
|
||||
"timestamp with time zone": "timestamptz",
|
||||
"time without time zone": "time",
|
||||
"time with time zone": "timetz",
|
||||
};
|
||||
return `${aliases[withoutModifier] ?? withoutModifier}${arraySuffix}`;
|
||||
}
|
||||
|
||||
function typesCompatible(left: string, right: string): boolean {
|
||||
return canonicalDataType(left) === canonicalDataType(right);
|
||||
}
|
||||
|
||||
function pairKey(sourceColumnId: string, targetColumnId: string): string {
|
||||
return `${sourceColumnId}\u0000${targetColumnId}`;
|
||||
}
|
||||
|
||||
function relationshipPair(relationship: CatalogLogicalRelationship): CatalogLogicalRelationshipCandidate {
|
||||
return {
|
||||
sourceColumnId: relationship.columns[0].sourceColumnId,
|
||||
targetColumnId: relationship.columns[0].targetColumnId,
|
||||
};
|
||||
}
|
||||
|
||||
function relationshipSortKey(relationship: CatalogRelationship): string {
|
||||
const sourceColumns = relationship.columns.map((column) => column.sourceColumnName).join(",");
|
||||
const targetColumns = relationship.columns.map((column) => column.targetColumnName).join(",");
|
||||
return [
|
||||
relationship.sourceTableName,
|
||||
sourceColumns,
|
||||
relationship.targetTableName,
|
||||
targetColumns,
|
||||
relationship.origin,
|
||||
].join("\u0000");
|
||||
}
|
||||
|
||||
export class CatalogLogicalRelationshipService {
|
||||
constructor(private readonly repository: CatalogRepository) {}
|
||||
|
||||
async list(databaseId: string): Promise<CatalogRelationship[]> {
|
||||
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
const relationships: CatalogRelationship[] = [
|
||||
...await this.repository.listRelationships(databaseId),
|
||||
...await this.repository.listLogicalRelationships(databaseId),
|
||||
];
|
||||
return relationships.sort((left, right) => relationshipSortKey(left).localeCompare(relationshipSortKey(right)));
|
||||
}
|
||||
|
||||
async addManual(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
): Promise<CatalogLogicalRelationship> {
|
||||
const context = await this.requiredContext(databaseId);
|
||||
const source = context.endpoints.find((endpoint) => endpoint.columnId === sourceColumnId);
|
||||
if (!source) throw new LogicalRelationshipColumnNotFoundError("sourceColumnId");
|
||||
const target = context.endpoints.find((endpoint) => endpoint.columnId === targetColumnId);
|
||||
if (!target) throw new LogicalRelationshipColumnNotFoundError("targetColumnId");
|
||||
if (sourceColumnId === targetColumnId
|
||||
|| target.primaryKeyPosition === null
|
||||
|| target.tablePrimaryKeyColumnCount !== 1) {
|
||||
throw new LogicalRelationshipTargetNotUniqueError();
|
||||
}
|
||||
if (!typesCompatible(source.dataType, target.dataType)) {
|
||||
throw new LogicalRelationshipTypeIncompatibleError();
|
||||
}
|
||||
const key = pairKey(sourceColumnId, targetColumnId);
|
||||
if (context.physicalPairs.some((pair) => pairKey(pair.sourceColumnId, pair.targetColumnId) === key)
|
||||
|| context.logicalRelationships.some((relationship) => {
|
||||
const pair = relationshipPair(relationship);
|
||||
return pairKey(pair.sourceColumnId, pair.targetColumnId) === key;
|
||||
})) throw new LogicalRelationshipDuplicateError();
|
||||
const created = await this.repository.insertLogicalRelationship(
|
||||
databaseId,
|
||||
sourceColumnId,
|
||||
targetColumnId,
|
||||
false,
|
||||
);
|
||||
if (!created) throw new LogicalRelationshipDuplicateError();
|
||||
return created;
|
||||
}
|
||||
|
||||
async rebuildGenerated(databaseId: string): Promise<RebuildGeneratedRelationshipsResult> {
|
||||
const context = await this.requiredContext(databaseId);
|
||||
const targets = context.endpoints.filter((endpoint) => (
|
||||
endpoint.primaryKeyPosition !== null && endpoint.tablePrimaryKeyColumnCount === 1
|
||||
));
|
||||
const physical = new Set(context.physicalPairs.map((pair) => pairKey(pair.sourceColumnId, pair.targetColumnId)));
|
||||
const active = new Set<string>();
|
||||
const excluded = new Set<string>();
|
||||
for (const relationship of context.logicalRelationships) {
|
||||
const pair = relationshipPair(relationship);
|
||||
(relationship.status === "excluded" ? excluded : active)
|
||||
.add(pairKey(pair.sourceColumnId, pair.targetColumnId));
|
||||
}
|
||||
|
||||
const pending: CatalogLogicalRelationshipCandidate[] = [];
|
||||
let alreadyPresent = 0;
|
||||
let excludedCount = 0;
|
||||
let ambiguous = 0;
|
||||
const dimTimeTargets = targets.filter((target) => (
|
||||
identifierTokens(target.tableName).join("_") === "dim_time"
|
||||
));
|
||||
const sources = context.endpoints.filter((endpoint) => (
|
||||
endpoint.primaryKeyPosition === null || endpoint.tablePrimaryKeyColumnCount > 1
|
||||
));
|
||||
for (const source of sources) {
|
||||
const sourceName = identifierTokens(source.columnName).join("_");
|
||||
const isTimeKey = sourceName.endsWith("time_key")
|
||||
&& identifierTokens(source.tableName).join("_") !== "dim_time";
|
||||
const candidates = isTimeKey ? dimTimeTargets : targets;
|
||||
const matches = candidates.filter((target) => (
|
||||
target.columnId !== source.columnId
|
||||
&& typesCompatible(source.dataType, target.dataType)
|
||||
&& (isTimeKey || nameMatches(source, target))
|
||||
));
|
||||
if (matches.length > 1) {
|
||||
ambiguous += 1;
|
||||
continue;
|
||||
}
|
||||
if (matches.length === 0) continue;
|
||||
const candidate = { sourceColumnId: source.columnId, targetColumnId: matches[0].columnId };
|
||||
const key = pairKey(candidate.sourceColumnId, candidate.targetColumnId);
|
||||
if (physical.has(key) || active.has(key)) {
|
||||
alreadyPresent += 1;
|
||||
} else if (excluded.has(key)) {
|
||||
excludedCount += 1;
|
||||
} else {
|
||||
pending.push(candidate);
|
||||
}
|
||||
}
|
||||
const added = await this.repository.insertGeneratedLogicalRelationships(databaseId, pending);
|
||||
alreadyPresent += pending.length - added;
|
||||
return { added, alreadyPresent, excluded: excludedCount, ambiguous };
|
||||
}
|
||||
|
||||
async setStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship> {
|
||||
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
const updated = await this.repository.setLogicalRelationshipStatus(databaseId, relationshipId, status);
|
||||
if (updated) return updated;
|
||||
await this.assertNotPhysical(databaseId, relationshipId);
|
||||
throw new LogicalRelationshipNotFoundError();
|
||||
}
|
||||
|
||||
async deletePermanently(databaseId: string, relationshipId: string): Promise<void> {
|
||||
if (!(await this.repository.get(databaseId))) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
if (await this.repository.deleteLogicalRelationship(databaseId, relationshipId)) return;
|
||||
await this.assertNotPhysical(databaseId, relationshipId);
|
||||
throw new LogicalRelationshipNotFoundError();
|
||||
}
|
||||
|
||||
private async requiredContext(databaseId: string): Promise<CatalogLogicalRelationshipContext> {
|
||||
const database = await this.repository.get(databaseId);
|
||||
if (!database) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
if (database.schemaSyncedVersion !== database.version) {
|
||||
throw new LogicalRelationshipSchemaStaleError();
|
||||
}
|
||||
const context = await this.repository.getLogicalRelationshipContext(databaseId);
|
||||
if (!context) throw new LogicalRelationshipDatabaseNotFoundError();
|
||||
return context;
|
||||
}
|
||||
|
||||
private async assertNotPhysical(databaseId: string, relationshipId: string): Promise<void> {
|
||||
if ((await this.repository.listRelationships(databaseId)).some((relationship) => relationship.id === relationshipId)) {
|
||||
throw new LogicalRelationshipReadOnlyError();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -14,7 +14,10 @@ import {
|
||||
type CatalogDatabaseMetadataDeleteTarget,
|
||||
type CatalogMetadataDeleteCounts,
|
||||
type CatalogMetrics,
|
||||
type CatalogRelationship,
|
||||
type CatalogLogicalRelationship,
|
||||
type CatalogLogicalRelationshipCandidate,
|
||||
type CatalogLogicalRelationshipContext,
|
||||
type CatalogPhysicalRelationship,
|
||||
type CatalogSchemaDiff,
|
||||
type CatalogSyncCounts,
|
||||
type CatalogSyncEvent,
|
||||
@@ -49,7 +52,8 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
private readonly records = new Map<string, WorkspaceDatabase>();
|
||||
private readonly tables = new Map<string, CatalogTable>();
|
||||
private readonly columns = new Map<string, CatalogColumn>();
|
||||
private readonly relationships = new Map<string, CatalogRelationship>();
|
||||
private readonly relationships = new Map<string, CatalogPhysicalRelationship>();
|
||||
private readonly logicalRelationships = new Map<string, CatalogLogicalRelationship>();
|
||||
private readonly descriptionGenerationRuns = new Map<string, DescriptionGenerationRun>();
|
||||
private readonly descriptionGenerationEvents = new Map<string, DescriptionGenerationEvent[]>();
|
||||
private readonly sensitiveDataSuggestionRuns = new Map<string, SensitiveDataSuggestionRun>();
|
||||
@@ -171,6 +175,9 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
for (const [relationshipId, relationship] of this.relationships) {
|
||||
if (relationship.databaseId === id) this.relationships.delete(relationshipId);
|
||||
}
|
||||
for (const [relationshipId, relationship] of this.logicalRelationships) {
|
||||
if (relationship.databaseId === id) this.logicalRelationships.delete(relationshipId);
|
||||
}
|
||||
for (const [runId, run] of this.descriptionGenerationRuns) {
|
||||
if (run.databaseId !== id) continue;
|
||||
this.descriptionGenerationRuns.delete(runId);
|
||||
@@ -527,12 +534,138 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
.map((event) => structuredClone(event));
|
||||
}
|
||||
|
||||
async listRelationships(databaseId: string): Promise<CatalogRelationship[]> {
|
||||
async listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]> {
|
||||
return [...this.relationships.values()].filter((relationship) => relationship.databaseId === databaseId)
|
||||
.sort((a, b) => `${a.sourceTableName}.${a.constraintName}`.localeCompare(`${b.sourceTableName}.${b.constraintName}`))
|
||||
.map((relationship) => structuredClone(relationship));
|
||||
}
|
||||
|
||||
async listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]> {
|
||||
return [...this.logicalRelationships.values()]
|
||||
.filter((relationship) => relationship.databaseId === databaseId)
|
||||
.sort((a, b) => {
|
||||
const left = `${a.sourceTableName}.${a.columns[0].sourceColumnName}.${a.targetTableName}.${a.columns[0].targetColumnName}`;
|
||||
const right = `${b.sourceTableName}.${b.columns[0].sourceColumnName}.${b.targetTableName}.${b.columns[0].targetColumnName}`;
|
||||
return left.localeCompare(right);
|
||||
})
|
||||
.map((relationship) => structuredClone(relationship));
|
||||
}
|
||||
|
||||
async getLogicalRelationshipContext(
|
||||
databaseId: string,
|
||||
): Promise<CatalogLogicalRelationshipContext | undefined> {
|
||||
if (!this.records.has(databaseId)) return undefined;
|
||||
const tables = [...this.tables.values()].filter((table) => table.databaseId === databaseId);
|
||||
const tableById = new Map(tables.map((table) => [table.id, table]));
|
||||
const columns = [...this.columns.values()].filter((column) => tableById.has(column.tableId));
|
||||
const primaryKeyCounts = new Map<string, number>();
|
||||
for (const column of columns) {
|
||||
if (column.primaryKeyPosition !== null) {
|
||||
primaryKeyCounts.set(column.tableId, (primaryKeyCounts.get(column.tableId) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
return {
|
||||
endpoints: columns.map((column) => ({
|
||||
columnId: column.id,
|
||||
columnName: column.name,
|
||||
tableId: column.tableId,
|
||||
tableName: tableById.get(column.tableId)!.name,
|
||||
dataType: column.dataType,
|
||||
primaryKeyPosition: column.primaryKeyPosition,
|
||||
tablePrimaryKeyColumnCount: primaryKeyCounts.get(column.tableId) ?? 0,
|
||||
})),
|
||||
physicalPairs: [...this.relationships.values()]
|
||||
.filter((relationship) => relationship.databaseId === databaseId)
|
||||
.flatMap((relationship) => relationship.columns.map((column) => ({
|
||||
sourceColumnId: column.sourceColumnId,
|
||||
targetColumnId: column.targetColumnId,
|
||||
}))),
|
||||
logicalRelationships: await this.listLogicalRelationships(databaseId),
|
||||
};
|
||||
}
|
||||
|
||||
async insertLogicalRelationship(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
generated: boolean,
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
if ([...this.logicalRelationships.values()].some((relationship) => (
|
||||
relationship.databaseId === databaseId
|
||||
&& relationship.columns[0].sourceColumnId === sourceColumnId
|
||||
&& relationship.columns[0].targetColumnId === targetColumnId
|
||||
))) return undefined;
|
||||
const sourceColumn = this.columns.get(sourceColumnId);
|
||||
const targetColumn = this.columns.get(targetColumnId);
|
||||
const sourceTable = sourceColumn ? this.tables.get(sourceColumn.tableId) : undefined;
|
||||
const targetTable = targetColumn ? this.tables.get(targetColumn.tableId) : undefined;
|
||||
if (!sourceColumn || !targetColumn || !sourceTable || !targetTable
|
||||
|| sourceTable.databaseId !== databaseId || targetTable.databaseId !== databaseId
|
||||
|| sourceColumnId === targetColumnId) return undefined;
|
||||
const now = new Date().toISOString();
|
||||
const relationship: CatalogLogicalRelationship = {
|
||||
id: randomUUID(),
|
||||
databaseId,
|
||||
constraintName: null,
|
||||
sourceTableId: sourceTable.id,
|
||||
sourceTableName: sourceTable.name,
|
||||
targetTableId: targetTable.id,
|
||||
targetTableName: targetTable.name,
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId,
|
||||
sourceColumnName: sourceColumn.name,
|
||||
targetColumnId,
|
||||
targetColumnName: targetColumn.name,
|
||||
}],
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
origin: generated ? "generated" : "manual",
|
||||
status: "active",
|
||||
};
|
||||
this.logicalRelationships.set(relationship.id, relationship);
|
||||
return structuredClone(relationship);
|
||||
}
|
||||
|
||||
async insertGeneratedLogicalRelationships(
|
||||
databaseId: string,
|
||||
candidates: readonly CatalogLogicalRelationshipCandidate[],
|
||||
): Promise<number> {
|
||||
let added = 0;
|
||||
for (const candidate of candidates) {
|
||||
if (await this.insertLogicalRelationship(
|
||||
databaseId,
|
||||
candidate.sourceColumnId,
|
||||
candidate.targetColumnId,
|
||||
true,
|
||||
)) added += 1;
|
||||
}
|
||||
return added;
|
||||
}
|
||||
|
||||
async setLogicalRelationshipStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
const current = this.logicalRelationships.get(relationshipId);
|
||||
if (!current || current.databaseId !== databaseId) return undefined;
|
||||
const updated = { ...current, status, updatedAt: new Date().toISOString() };
|
||||
this.logicalRelationships.set(relationshipId, updated);
|
||||
return structuredClone(updated);
|
||||
}
|
||||
|
||||
async deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean> {
|
||||
const current = this.logicalRelationships.get(relationshipId);
|
||||
return Boolean(current?.databaseId === databaseId && this.logicalRelationships.delete(relationshipId));
|
||||
}
|
||||
|
||||
async deleteDatabaseMetadata(
|
||||
databaseIds: readonly string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
@@ -574,6 +707,12 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
const columns = [...this.columns.values()].filter((column) => selected.has(column.tableId));
|
||||
const deletedColumnIds = new Set(columns.map((column) => column.id));
|
||||
for (const column of columns) this.columns.delete(column.id);
|
||||
for (const relationship of [...this.logicalRelationships.values()]) {
|
||||
const pair = relationship.columns[0];
|
||||
if (deletedColumnIds.has(pair.sourceColumnId) || deletedColumnIds.has(pair.targetColumnId)) {
|
||||
this.logicalRelationships.delete(relationship.id);
|
||||
}
|
||||
}
|
||||
for (const [relationshipId, relationship] of this.relationships) {
|
||||
if (relationship.databaseId !== databaseId) continue;
|
||||
this.relationships.set(relationshipId, {
|
||||
@@ -857,6 +996,8 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
lastSyncedAt: now,
|
||||
createdAt: current?.createdAt ?? now,
|
||||
updatedAt: comparable === nextComparable ? (current?.updatedAt ?? now) : now,
|
||||
origin: "physical",
|
||||
status: "active",
|
||||
});
|
||||
if (!current) created += 1;
|
||||
else if (comparable !== nextComparable) updated += 1;
|
||||
@@ -1073,6 +1214,12 @@ export class MemoryCatalogRepository implements CatalogRepository {
|
||||
this.relationships.delete(relationship.id);
|
||||
}
|
||||
}
|
||||
for (const relationship of [...this.logicalRelationships.values()]) {
|
||||
const pair = relationship.columns[0];
|
||||
if (pair.sourceColumnId === columnId || pair.targetColumnId === columnId) {
|
||||
this.logicalRelationships.delete(relationship.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private markCatalogIncomplete(databaseIds: readonly string[]): void {
|
||||
|
||||
@@ -10,6 +10,7 @@ import * as catalogRuntimeSequencePrivilegesMigration from "./migrations/004_cat
|
||||
import * as descriptionGenerationRunsMigration from "./migrations/005_description_generation_runs.js";
|
||||
import * as sensitiveDataFlagMigration from "./migrations/006_sensitive_data_flag.js";
|
||||
import * as sensitiveDataSuggestionRunsMigration from "./migrations/007_sensitive_data_suggestion_runs.js";
|
||||
import * as catalogLogicalRelationshipsMigration from "./migrations/008_catalog_logical_relationships.js";
|
||||
|
||||
const connectionString = process.env.THT_CATALOG_MIGRATOR_DATABASE_URL;
|
||||
const host = process.env.THT_CATALOG_DB_HOST;
|
||||
@@ -42,6 +43,7 @@ const provider: MigrationProvider = {
|
||||
"005_description_generation_runs": descriptionGenerationRunsMigration,
|
||||
"006_sensitive_data_flag": sensitiveDataFlagMigration,
|
||||
"007_sensitive_data_suggestion_runs": sensitiveDataSuggestionRunsMigration,
|
||||
"008_catalog_logical_relationships": catalogLogicalRelationshipsMigration,
|
||||
};
|
||||
},
|
||||
};
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
import { type Kysely, sql } from "kysely";
|
||||
import type { CatalogDatabase } from "../repository.js";
|
||||
|
||||
export async function up(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||
await db.schema.createTable("catalog_logical_relationships")
|
||||
.addColumn("id", "uuid", (column) => column.primaryKey())
|
||||
.addColumn("database_id", "uuid", (column) => column.notNull()
|
||||
.references("workspace_databases.id").onDelete("cascade"))
|
||||
.addColumn("source_column_id", "uuid", (column) => column.notNull()
|
||||
.references("catalog_columns.id").onDelete("cascade"))
|
||||
.addColumn("target_column_id", "uuid", (column) => column.notNull()
|
||||
.references("catalog_columns.id").onDelete("cascade"))
|
||||
.addColumn("generated", "boolean", (column) => column.notNull().defaultTo(false))
|
||||
.addColumn("deleted_at", "timestamptz")
|
||||
.addColumn("created_at", "timestamptz", (column) => column.notNull().defaultTo(sql`now()`))
|
||||
.addColumn("updated_at", "timestamptz", (column) => column.notNull().defaultTo(sql`now()`))
|
||||
.addUniqueConstraint(
|
||||
"catalog_logical_relationships_endpoint_key",
|
||||
["database_id", "source_column_id", "target_column_id"],
|
||||
)
|
||||
.addCheckConstraint(
|
||||
"catalog_logical_relationships_distinct_columns_check",
|
||||
sql`source_column_id <> target_column_id`,
|
||||
)
|
||||
.execute();
|
||||
|
||||
await db.schema.createIndex("catalog_logical_relationships_database_deleted_idx")
|
||||
.on("catalog_logical_relationships")
|
||||
.columns(["database_id", "deleted_at"])
|
||||
.execute();
|
||||
}
|
||||
|
||||
export async function down(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||
await db.schema.dropTable("catalog_logical_relationships").execute();
|
||||
}
|
||||
@@ -23,7 +23,10 @@ import {
|
||||
type CatalogDatabaseMetadataDeleteTarget,
|
||||
type CatalogMetadataDeleteCounts,
|
||||
type CatalogMetrics,
|
||||
type CatalogRelationship,
|
||||
type CatalogLogicalRelationship,
|
||||
type CatalogLogicalRelationshipCandidate,
|
||||
type CatalogLogicalRelationshipContext,
|
||||
type CatalogPhysicalRelationship,
|
||||
type CatalogSchemaDiff,
|
||||
type CatalogSyncCounts,
|
||||
type CatalogSyncEvent,
|
||||
@@ -145,6 +148,17 @@ interface CatalogRelationshipColumnTable {
|
||||
targetColumnId: string;
|
||||
}
|
||||
|
||||
interface CatalogLogicalRelationshipTable {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
sourceColumnId: string;
|
||||
targetColumnId: string;
|
||||
generated: Generated<boolean>;
|
||||
deletedAt: Timestamp | null;
|
||||
createdAt: Timestamp;
|
||||
updatedAt: Timestamp;
|
||||
}
|
||||
|
||||
interface DescriptionGenerationRunTable {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
@@ -241,6 +255,7 @@ export interface CatalogDatabase {
|
||||
catalogColumns: CatalogColumnTable;
|
||||
catalogRelationships: CatalogRelationshipTable;
|
||||
catalogRelationshipColumns: CatalogRelationshipColumnTable;
|
||||
catalogLogicalRelationships: CatalogLogicalRelationshipTable;
|
||||
descriptionGenerationRuns: DescriptionGenerationRunTable;
|
||||
descriptionGenerationEvents: DescriptionGenerationEventTable;
|
||||
sensitiveDataSuggestionRuns: SensitiveDataSuggestionRunTable;
|
||||
@@ -1022,7 +1037,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
return rows.map(serializeSensitiveDataSuggestionEvent);
|
||||
}
|
||||
|
||||
async listRelationships(databaseId: string): Promise<CatalogRelationship[]> {
|
||||
async listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]> {
|
||||
const rows = await this.db.selectFrom("catalogRelationships as relationship")
|
||||
.innerJoin("catalogTables as sourceTable", "sourceTable.id", "relationship.sourceTableId")
|
||||
.innerJoin("catalogTables as targetTable", "targetTable.id", "relationship.targetTableId")
|
||||
@@ -1049,7 +1064,7 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
.where("pair.relationshipId", "in", rows.map((row) => row.id))
|
||||
.orderBy("pair.relationshipId").orderBy("pair.position")
|
||||
.execute();
|
||||
const byRelationship = new Map<string, CatalogRelationship["columns"]>();
|
||||
const byRelationship = new Map<string, CatalogPhysicalRelationship["columns"]>();
|
||||
for (const pair of pairs) {
|
||||
const items = byRelationship.get(pair.relationshipId) ?? [];
|
||||
items.push(pair);
|
||||
@@ -1061,9 +1076,160 @@ export class KyselyCatalogRepository implements CatalogRepository {
|
||||
lastSyncedAt: row.lastSyncedAt === null ? null : new Date(row.lastSyncedAt).toISOString(),
|
||||
createdAt: new Date(row.createdAt).toISOString(),
|
||||
updatedAt: new Date(row.updatedAt).toISOString(),
|
||||
origin: "physical" as const,
|
||||
status: "active" as const,
|
||||
}));
|
||||
}
|
||||
|
||||
async listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]> {
|
||||
const rows = await this.db.selectFrom("catalogLogicalRelationships as relationship")
|
||||
.innerJoin("catalogColumns as sourceColumn", "sourceColumn.id", "relationship.sourceColumnId")
|
||||
.innerJoin("catalogTables as sourceTable", "sourceTable.id", "sourceColumn.tableId")
|
||||
.innerJoin("catalogColumns as targetColumn", "targetColumn.id", "relationship.targetColumnId")
|
||||
.innerJoin("catalogTables as targetTable", "targetTable.id", "targetColumn.tableId")
|
||||
.select([
|
||||
"relationship.id", "relationship.databaseId", "relationship.generated",
|
||||
"relationship.deletedAt", "relationship.createdAt", "relationship.updatedAt",
|
||||
"sourceTable.id as sourceTableId", "sourceTable.name as sourceTableName",
|
||||
"sourceColumn.id as sourceColumnId", "sourceColumn.name as sourceColumnName",
|
||||
"targetTable.id as targetTableId", "targetTable.name as targetTableName",
|
||||
"targetColumn.id as targetColumnId", "targetColumn.name as targetColumnName",
|
||||
])
|
||||
.where("relationship.databaseId", "=", databaseId)
|
||||
.orderBy("sourceTable.name")
|
||||
.orderBy("sourceColumn.name")
|
||||
.orderBy("targetTable.name")
|
||||
.orderBy("targetColumn.name")
|
||||
.execute();
|
||||
return rows.map((row) => ({
|
||||
id: row.id,
|
||||
databaseId: row.databaseId,
|
||||
constraintName: null,
|
||||
sourceTableId: row.sourceTableId,
|
||||
sourceTableName: row.sourceTableName,
|
||||
targetTableId: row.targetTableId,
|
||||
targetTableName: row.targetTableName,
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId: row.sourceColumnId,
|
||||
sourceColumnName: row.sourceColumnName,
|
||||
targetColumnId: row.targetColumnId,
|
||||
targetColumnName: row.targetColumnName,
|
||||
}],
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
createdAt: new Date(row.createdAt).toISOString(),
|
||||
updatedAt: new Date(row.updatedAt).toISOString(),
|
||||
origin: row.generated ? "generated" : "manual",
|
||||
status: row.deletedAt === null ? "active" : "excluded",
|
||||
}));
|
||||
}
|
||||
|
||||
async getLogicalRelationshipContext(
|
||||
databaseId: string,
|
||||
): Promise<CatalogLogicalRelationshipContext | undefined> {
|
||||
if (!(await selectOne(this.db, databaseId))) return undefined;
|
||||
const columns = await this.db.selectFrom("catalogColumns as column")
|
||||
.innerJoin("catalogTables as table", "table.id", "column.tableId")
|
||||
.select([
|
||||
"column.id as columnId", "column.name as columnName", "column.dataType",
|
||||
"column.primaryKeyPosition", "table.id as tableId", "table.name as tableName",
|
||||
])
|
||||
.where("table.databaseId", "=", databaseId)
|
||||
.orderBy("table.name")
|
||||
.orderBy("column.ordinalPosition")
|
||||
.execute();
|
||||
const primaryKeyCounts = new Map<string, number>();
|
||||
for (const column of columns) {
|
||||
if (column.primaryKeyPosition !== null) {
|
||||
primaryKeyCounts.set(column.tableId, (primaryKeyCounts.get(column.tableId) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
const physicalPairs = await this.db.selectFrom("catalogRelationshipColumns as pair")
|
||||
.innerJoin("catalogRelationships as relationship", "relationship.id", "pair.relationshipId")
|
||||
.select(["pair.sourceColumnId", "pair.targetColumnId"])
|
||||
.where("relationship.databaseId", "=", databaseId)
|
||||
.execute();
|
||||
return {
|
||||
endpoints: columns.map((column) => ({
|
||||
...column,
|
||||
tablePrimaryKeyColumnCount: primaryKeyCounts.get(column.tableId) ?? 0,
|
||||
})),
|
||||
physicalPairs,
|
||||
logicalRelationships: await this.listLogicalRelationships(databaseId),
|
||||
};
|
||||
}
|
||||
|
||||
async insertLogicalRelationship(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
generated: boolean,
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
const id = randomUUID();
|
||||
const inserted = await this.db.insertInto("catalogLogicalRelationships").values({
|
||||
id, databaseId, sourceColumnId, targetColumnId, generated,
|
||||
}).onConflict((conflict) => conflict
|
||||
.columns(["databaseId", "sourceColumnId", "targetColumnId"])
|
||||
.doNothing())
|
||||
.returning("id")
|
||||
.executeTakeFirst();
|
||||
if (!inserted) return undefined;
|
||||
return (await this.listLogicalRelationships(databaseId)).find((item) => item.id === id);
|
||||
}
|
||||
|
||||
async insertGeneratedLogicalRelationships(
|
||||
databaseId: string,
|
||||
candidates: readonly CatalogLogicalRelationshipCandidate[],
|
||||
): Promise<number> {
|
||||
if (candidates.length === 0) return 0;
|
||||
return await this.db.transaction().execute(async (trx) => {
|
||||
let added = 0;
|
||||
for (const candidate of candidates) {
|
||||
const inserted = await trx.insertInto("catalogLogicalRelationships").values({
|
||||
id: randomUUID(),
|
||||
databaseId,
|
||||
sourceColumnId: candidate.sourceColumnId,
|
||||
targetColumnId: candidate.targetColumnId,
|
||||
generated: true,
|
||||
}).onConflict((conflict) => conflict
|
||||
.columns(["databaseId", "sourceColumnId", "targetColumnId"])
|
||||
.doNothing())
|
||||
.returning("id")
|
||||
.executeTakeFirst();
|
||||
if (inserted) added += 1;
|
||||
}
|
||||
return added;
|
||||
});
|
||||
}
|
||||
|
||||
async setLogicalRelationshipStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship | undefined> {
|
||||
const updated = await this.db.updateTable("catalogLogicalRelationships")
|
||||
.set({ deletedAt: status === "excluded" ? sql`now()` : null, updatedAt: sql`now()` })
|
||||
.where("databaseId", "=", databaseId)
|
||||
.where("id", "=", relationshipId)
|
||||
.returning("id")
|
||||
.executeTakeFirst();
|
||||
if (!updated) return undefined;
|
||||
return (await this.listLogicalRelationships(databaseId)).find((item) => item.id === relationshipId);
|
||||
}
|
||||
|
||||
async deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean> {
|
||||
const result = await this.db.deleteFrom("catalogLogicalRelationships")
|
||||
.where("databaseId", "=", databaseId)
|
||||
.where("id", "=", relationshipId)
|
||||
.executeTakeFirst();
|
||||
return result.numDeletedRows > 0n;
|
||||
}
|
||||
|
||||
async deleteDatabaseMetadata(
|
||||
databaseIds: readonly string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
@@ -1621,7 +1787,13 @@ export class UnavailableCatalogRepository implements CatalogRepository {
|
||||
async updateSensitiveDataSuggestionRun(): Promise<SensitiveDataSuggestionRun | undefined> { return this.fail(); }
|
||||
async appendSensitiveDataSuggestionEvent(): Promise<SensitiveDataSuggestionEvent> { return this.fail(); }
|
||||
async listSensitiveDataSuggestionEvents(): Promise<SensitiveDataSuggestionEvent[]> { return this.fail(); }
|
||||
async listRelationships(): Promise<CatalogRelationship[]> { return this.fail(); }
|
||||
async listRelationships(): Promise<CatalogPhysicalRelationship[]> { return this.fail(); }
|
||||
async listLogicalRelationships(): Promise<CatalogLogicalRelationship[]> { return this.fail(); }
|
||||
async getLogicalRelationshipContext(): Promise<CatalogLogicalRelationshipContext | undefined> { return this.fail(); }
|
||||
async insertLogicalRelationship(): Promise<CatalogLogicalRelationship | undefined> { return this.fail(); }
|
||||
async insertGeneratedLogicalRelationships(): Promise<number> { return this.fail(); }
|
||||
async setLogicalRelationshipStatus(): Promise<CatalogLogicalRelationship | undefined> { return this.fail(); }
|
||||
async deleteLogicalRelationship(): Promise<boolean> { return this.fail(); }
|
||||
async deleteDatabaseMetadata(): Promise<CatalogMetadataDeleteCounts | undefined> { return this.fail(); }
|
||||
async deleteTableMetadata(): Promise<CatalogMetadataDeleteCounts | undefined> { return this.fail(); }
|
||||
async planSchemaSync(): Promise<CatalogSchemaDiff> { return this.fail(); }
|
||||
|
||||
@@ -2,6 +2,7 @@ import { buildInstallationContract } from "../workspaces/contracts.js";
|
||||
import { resolveBinding } from "../workspaces/bindings.js";
|
||||
import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||
import type { WorkspaceDescriptor } from "../workspaces/schema.js";
|
||||
import { discoverWorkspaceSecretRequirements } from "../workspaces/secret-requirements.js";
|
||||
import type { WorkspaceSecretStore } from "../workspaces/secret-store.js";
|
||||
import { createConcreteDiagnosticAdapters } from "../workspaces/diagnostics.js";
|
||||
import { CatalogOperationCoordinator } from "./operation-coordinator.js";
|
||||
@@ -25,6 +26,21 @@ export interface CatalogListItem extends Omit<WorkspaceDatabase, "id"> {
|
||||
workspaceName: string;
|
||||
workspaceDescription?: string;
|
||||
workspaceAvailable: boolean;
|
||||
workspaceRevision: { commit: string; blob: string } | null;
|
||||
workspaceEvidence: {
|
||||
sourceType: "filesystem" | "http" | "s3" | null;
|
||||
state:
|
||||
| "not_declared"
|
||||
| "materialized_current_revision"
|
||||
| "configuration_required"
|
||||
| "configured_unverified"
|
||||
| "workspace_unavailable";
|
||||
};
|
||||
runtimeBinding: {
|
||||
transport: DatabaseBinding["transport"];
|
||||
configurationState: "ready" | "configuration_required";
|
||||
sessionTransportSupported: boolean;
|
||||
} | null;
|
||||
configured: boolean;
|
||||
secrets: Record<CatalogSecretName, boolean>;
|
||||
}
|
||||
@@ -68,6 +84,43 @@ function secretState(store: WorkspaceSecretStore, workspaceId: string): Record<C
|
||||
))) as Record<CatalogSecretName, boolean>;
|
||||
}
|
||||
|
||||
function workspaceRuntimeState(
|
||||
store: WorkspaceSecretStore,
|
||||
workspace: WorkspaceDescriptor,
|
||||
secretRoots: readonly string[],
|
||||
): NonNullable<CatalogListItem["runtimeBinding"]> {
|
||||
const requirements = discoverWorkspaceSecretRequirements(workspace, process.env);
|
||||
const secretVariables = new Set(requirements.map(({ variable }) => variable));
|
||||
const effective = resolveBinding(workspace, "DWH", process.env, secretRoots);
|
||||
const configurationRequired = requirements.some(({ id, required }) => (
|
||||
required && !store.has(workspace.workspace.id, id)
|
||||
)) || effective.missing.some((variable) => !secretVariables.has(variable));
|
||||
return {
|
||||
transport: effective.transport,
|
||||
configurationState: configurationRequired ? "configuration_required" : "ready",
|
||||
sessionTransportSupported: effective.transport !== "ssh_tunnel",
|
||||
};
|
||||
}
|
||||
|
||||
function workspaceEvidenceState(
|
||||
store: WorkspaceSecretStore,
|
||||
workspace: WorkspaceDescriptor,
|
||||
): CatalogListItem["workspaceEvidence"] {
|
||||
const source = workspace.evidence?.source;
|
||||
if (!source) return { sourceType: null, state: "not_declared" };
|
||||
if (source.type === "filesystem") {
|
||||
return { sourceType: source.type, state: "materialized_current_revision" };
|
||||
}
|
||||
const configurationRequired = discoverWorkspaceSecretRequirements(workspace, process.env)
|
||||
.some(({ connector, id, required }) => (
|
||||
connector === "evidence" && required && !store.has(workspace.workspace.id, id)
|
||||
));
|
||||
return {
|
||||
sourceType: source.type,
|
||||
state: configurationRequired ? "configuration_required" : "configured_unverified",
|
||||
};
|
||||
}
|
||||
|
||||
export class CatalogService {
|
||||
private readonly adapters = createConcreteDiagnosticAdapters();
|
||||
|
||||
@@ -91,7 +144,8 @@ export class CatalogService {
|
||||
const byWorkspace = new Map(configured.map((database) => [database.workspaceId, database]));
|
||||
const active = await Promise.all(workspaces.map(async (entry) => {
|
||||
const database = byWorkspace.get(entry.id);
|
||||
const { workspace } = await this.registry.read(entry.id);
|
||||
const { workspace } = await this.registry.readPinned(entry.id, entry.revision.commit);
|
||||
const runtimeDatabaseBinding = yamlBinding(workspace, this.secretRoots);
|
||||
const base = database ?? {
|
||||
workspaceId: entry.id,
|
||||
engine: "postgres" as const,
|
||||
@@ -100,7 +154,7 @@ export class CatalogService {
|
||||
version: 0,
|
||||
createdAt: "",
|
||||
updatedAt: "",
|
||||
binding: yamlBinding(workspace, this.secretRoots),
|
||||
binding: runtimeDatabaseBinding,
|
||||
connectionStatus: "untested" as const,
|
||||
};
|
||||
return {
|
||||
@@ -108,6 +162,12 @@ export class CatalogService {
|
||||
workspaceName: entry.name,
|
||||
workspaceDescription: entry.description,
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: {
|
||||
commit: entry.revision.commit,
|
||||
blob: entry.revision.blob,
|
||||
},
|
||||
workspaceEvidence: workspaceEvidenceState(this.secretStore, workspace),
|
||||
runtimeBinding: workspaceRuntimeState(this.secretStore, workspace, this.secretRoots),
|
||||
configured: database !== undefined,
|
||||
secrets: secretState(this.secretStore, entry.id),
|
||||
};
|
||||
@@ -120,6 +180,9 @@ export class CatalogService {
|
||||
workspaceName: database.workspaceId,
|
||||
workspaceDescription: "Workspace is no longer present in the repository catalog.",
|
||||
workspaceAvailable: false,
|
||||
workspaceRevision: null,
|
||||
workspaceEvidence: { sourceType: null, state: "workspace_unavailable" },
|
||||
runtimeBinding: null,
|
||||
configured: true,
|
||||
secrets: secretState(this.secretStore, database.workspaceId),
|
||||
}));
|
||||
|
||||
@@ -128,7 +128,7 @@ export interface CatalogRelationshipColumn {
|
||||
targetColumnName: string;
|
||||
}
|
||||
|
||||
export interface CatalogRelationship {
|
||||
export interface CatalogPhysicalRelationship {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
constraintName: string;
|
||||
@@ -145,6 +145,52 @@ export interface CatalogRelationship {
|
||||
lastSyncedAt: string | null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
origin: "physical";
|
||||
status: "active";
|
||||
}
|
||||
|
||||
export interface CatalogLogicalRelationship {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
constraintName: null;
|
||||
sourceTableId: string;
|
||||
sourceTableName: string;
|
||||
targetTableId: string;
|
||||
targetTableName: string;
|
||||
updateRule: null;
|
||||
deleteRule: null;
|
||||
deferrable: false;
|
||||
initiallyDeferred: false;
|
||||
columns: [CatalogRelationshipColumn];
|
||||
lastSyncedDatabaseVersion: null;
|
||||
lastSyncedAt: null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
origin: "generated" | "manual";
|
||||
status: "active" | "excluded";
|
||||
}
|
||||
|
||||
export type CatalogRelationship = CatalogPhysicalRelationship | CatalogLogicalRelationship;
|
||||
|
||||
export interface CatalogLogicalRelationshipEndpoint {
|
||||
columnId: string;
|
||||
columnName: string;
|
||||
tableId: string;
|
||||
tableName: string;
|
||||
dataType: string;
|
||||
primaryKeyPosition: number | null;
|
||||
tablePrimaryKeyColumnCount: number;
|
||||
}
|
||||
|
||||
export interface CatalogLogicalRelationshipContext {
|
||||
endpoints: CatalogLogicalRelationshipEndpoint[];
|
||||
physicalPairs: Array<{ sourceColumnId: string; targetColumnId: string }>;
|
||||
logicalRelationships: CatalogLogicalRelationship[];
|
||||
}
|
||||
|
||||
export interface CatalogLogicalRelationshipCandidate {
|
||||
sourceColumnId: string;
|
||||
targetColumnId: string;
|
||||
}
|
||||
|
||||
export type CatalogDatabaseMetadataDeleteTarget = "tables" | "relationships";
|
||||
@@ -456,7 +502,25 @@ export interface CatalogRepository {
|
||||
runId: string,
|
||||
afterSequence?: number,
|
||||
): Promise<SensitiveDataSuggestionEvent[]>;
|
||||
listRelationships(databaseId: string): Promise<CatalogRelationship[]>;
|
||||
listRelationships(databaseId: string): Promise<CatalogPhysicalRelationship[]>;
|
||||
listLogicalRelationships(databaseId: string): Promise<CatalogLogicalRelationship[]>;
|
||||
getLogicalRelationshipContext(databaseId: string): Promise<CatalogLogicalRelationshipContext | undefined>;
|
||||
insertLogicalRelationship(
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
generated: boolean,
|
||||
): Promise<CatalogLogicalRelationship | undefined>;
|
||||
insertGeneratedLogicalRelationships(
|
||||
databaseId: string,
|
||||
candidates: readonly CatalogLogicalRelationshipCandidate[],
|
||||
): Promise<number>;
|
||||
setLogicalRelationshipStatus(
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogLogicalRelationship["status"],
|
||||
): Promise<CatalogLogicalRelationship | undefined>;
|
||||
deleteLogicalRelationship(databaseId: string, relationshipId: string): Promise<boolean>;
|
||||
deleteDatabaseMetadata(
|
||||
databaseIds: readonly string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
import type { FastifyInstance, FastifyReply, FastifyRequest } from "fastify";
|
||||
import { z } from "zod";
|
||||
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||
import {
|
||||
CatalogLogicalRelationshipService,
|
||||
LogicalRelationshipColumnNotFoundError,
|
||||
LogicalRelationshipDatabaseNotFoundError,
|
||||
LogicalRelationshipDuplicateError,
|
||||
LogicalRelationshipNotFoundError,
|
||||
LogicalRelationshipReadOnlyError,
|
||||
LogicalRelationshipSchemaStaleError,
|
||||
LogicalRelationshipTargetNotUniqueError,
|
||||
LogicalRelationshipTypeIncompatibleError,
|
||||
} from "../catalog/logical-relationship-service.js";
|
||||
import type { CatalogOperationCoordinator } from "../catalog/operation-coordinator.js";
|
||||
import { CatalogOperationInProgressError, CatalogUnavailableError } from "../catalog/types.js";
|
||||
|
||||
const idSchema = z.uuid();
|
||||
const createSchema = z.object({
|
||||
sourceColumnId: idSchema,
|
||||
targetColumnId: idSchema,
|
||||
}).strict();
|
||||
const statusSchema = z.object({ status: z.enum(["active", "excluded"]) }).strict();
|
||||
const emptySchema = z.object({}).strict();
|
||||
|
||||
function manage(request: FastifyRequest, reply: FastifyReply) {
|
||||
return isPrincipalContext(requirePermission(request, reply, "database.manage"));
|
||||
}
|
||||
|
||||
function safeError(reply: FastifyReply, error: unknown) {
|
||||
if (error instanceof CatalogUnavailableError) {
|
||||
return reply.code(503).send({ code: "catalog_unavailable", message: "Database catalog is unavailable." });
|
||||
}
|
||||
if (error instanceof CatalogOperationInProgressError) {
|
||||
return reply.code(409).send({
|
||||
code: "database_operation_in_progress",
|
||||
message: "A database operation is already in progress.",
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipDatabaseNotFoundError) {
|
||||
return reply.code(404).send({ code: "database_not_found", message: "Database configuration was not found." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipColumnNotFoundError) {
|
||||
return reply.code(404).send({
|
||||
code: "column_not_found",
|
||||
message: "Catalog column was not found.",
|
||||
field: error.field,
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipNotFoundError) {
|
||||
return reply.code(404).send({ code: "relationship_not_found", message: "Relationship was not found." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipDuplicateError) {
|
||||
return reply.code(409).send({ code: "relationship_duplicate", message: "Relationship already exists." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipReadOnlyError) {
|
||||
return reply.code(409).send({ code: "relationship_read_only", message: "Physical relationships are read-only." });
|
||||
}
|
||||
if (error instanceof LogicalRelationshipSchemaStaleError) {
|
||||
return reply.code(409).send({
|
||||
code: "relationship_schema_stale",
|
||||
message: "Synchronize the current database schema before managing logical relationships.",
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipTargetNotUniqueError) {
|
||||
return reply.code(422).send({
|
||||
code: "relationship_target_not_unique",
|
||||
message: "Target column must be the only primary-key column of its table.",
|
||||
field: "targetColumnId",
|
||||
});
|
||||
}
|
||||
if (error instanceof LogicalRelationshipTypeIncompatibleError) {
|
||||
return reply.code(422).send({
|
||||
code: "relationship_type_incompatible",
|
||||
message: "Source and target column types are incompatible.",
|
||||
field: "targetColumnId",
|
||||
});
|
||||
}
|
||||
if (error instanceof z.ZodError) {
|
||||
return reply.code(400).send({
|
||||
code: "relationship_request_invalid",
|
||||
message: "Relationship request is invalid.",
|
||||
});
|
||||
}
|
||||
return reply.code(500).send({
|
||||
code: "relationship_operation_failed",
|
||||
message: "Relationship operation failed.",
|
||||
});
|
||||
}
|
||||
|
||||
export function catalogLogicalRelationshipRoutes(
|
||||
app: FastifyInstance,
|
||||
deps: {
|
||||
service: CatalogLogicalRelationshipService;
|
||||
operations: CatalogOperationCoordinator;
|
||||
},
|
||||
): void {
|
||||
app.get("/catalog/databases/:databaseId/relationships", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
return await deps.service.list(databaseId);
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.post("/catalog/databases/:databaseId/relationships", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
const input = createSchema.parse(request.body);
|
||||
const relationship = await deps.operations.run(databaseId, async () => (
|
||||
await deps.service.addManual(databaseId, input.sourceColumnId, input.targetColumnId)
|
||||
));
|
||||
return reply.code(201).send(relationship);
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.post("/catalog/databases/:databaseId/relationships/rebuild-generated", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
emptySchema.parse(request.body ?? {});
|
||||
return await deps.operations.run(databaseId, async () => (
|
||||
await deps.service.rebuildGenerated(databaseId)
|
||||
));
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.patch("/catalog/databases/:databaseId/relationships/:relationshipId", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const params = request.params as { databaseId?: unknown; relationshipId?: unknown };
|
||||
const databaseId = idSchema.parse(params.databaseId);
|
||||
const relationshipId = idSchema.parse(params.relationshipId);
|
||||
const input = statusSchema.parse(request.body);
|
||||
return await deps.operations.run(databaseId, async () => (
|
||||
await deps.service.setStatus(databaseId, relationshipId, input.status)
|
||||
));
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.delete("/catalog/databases/:databaseId/relationships/:relationshipId", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const params = request.params as { databaseId?: unknown; relationshipId?: unknown };
|
||||
const databaseId = idSchema.parse(params.databaseId);
|
||||
const relationshipId = idSchema.parse(params.relationshipId);
|
||||
await deps.operations.run(databaseId, async () => {
|
||||
await deps.service.deletePermanently(databaseId, relationshipId);
|
||||
});
|
||||
return reply.code(204).send();
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
}
|
||||
@@ -131,17 +131,6 @@ export function catalogSchemaRoutes(
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.get("/catalog/databases/:databaseId/relationships", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId);
|
||||
if (!(await deps.repository.get(databaseId))) {
|
||||
return reply.code(404).send({ code: "database_not_found", message: "Database configuration was not found." });
|
||||
}
|
||||
return await deps.repository.listRelationships(databaseId);
|
||||
} catch (error) { return safeError(reply, error); }
|
||||
});
|
||||
|
||||
app.post("/catalog/databases/metadata-cleanup", async (request, reply) => {
|
||||
if (!manage(request, reply)) return reply;
|
||||
try {
|
||||
|
||||
@@ -11,6 +11,7 @@ import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||
import { validateOperationalWorkspace, type WorkspaceDescriptor } from "../workspaces/schema.js";
|
||||
import type { MaintenanceBarrier } from "../runtime/maintenance-gate.js";
|
||||
import { hasPermission, isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||
import type { EffectiveRelationshipSnapshotProvider } from "../catalog/effective-relationship-snapshot.js";
|
||||
|
||||
const BOOTSTRAP_FAILURE_MESSAGE =
|
||||
"Session startup failed. Check configuration and connectivity, then Resume the session.";
|
||||
@@ -40,6 +41,8 @@ export function sessionRoutes(
|
||||
/** Fail-closed installation/runtime transport capability check. */
|
||||
workspaceRuntimeSupport: (workspace: WorkspaceDescriptor) => boolean;
|
||||
maintenanceBarrier: MaintenanceBarrier;
|
||||
/** Optional only for narrow route-test stubs and installations without a Catalog database. */
|
||||
effectiveRelationships?: EffectiveRelationshipSnapshotProvider;
|
||||
},
|
||||
) {
|
||||
const lifecycleTails = new Map<string, Promise<void>>();
|
||||
@@ -84,11 +87,21 @@ export function sessionRoutes(
|
||||
isAdmin: hasPermission(principal, permission),
|
||||
});
|
||||
|
||||
const optionsWithRuntimeConfig = (runner: any, workspaceConfigPath: string | undefined, options: any) => (
|
||||
workspaceConfigPath && typeof runner.acquireWorkspaceRuntime === "function"
|
||||
? { ...options, runtimeConfig: runner.acquireWorkspaceRuntime(workspaceConfigPath) }
|
||||
: options
|
||||
);
|
||||
const optionsWithRuntimeConfig = async (
|
||||
runner: any,
|
||||
workspaceConfigPath: string | undefined,
|
||||
workspaceId: string | undefined,
|
||||
options: any,
|
||||
) => {
|
||||
if (!workspaceConfigPath || typeof runner.acquireWorkspaceRuntime !== "function") return options;
|
||||
const effectiveRelationships = workspaceId && d.effectiveRelationships
|
||||
? await d.effectiveRelationships.render(workspaceId)
|
||||
: undefined;
|
||||
return {
|
||||
...options,
|
||||
runtimeConfig: runner.acquireWorkspaceRuntime(workspaceConfigPath, effectiveRelationships),
|
||||
};
|
||||
};
|
||||
|
||||
const maintenanceReply = (reply: any) => reply.code(503).send({
|
||||
code: "maintenance",
|
||||
@@ -448,7 +461,12 @@ export function sessionRoutes(
|
||||
let runtimeOptions = options;
|
||||
let rt: ReturnType<PiProcessManager["createFor"]> | undefined;
|
||||
try {
|
||||
runtimeOptions = optionsWithRuntimeConfig(runner, workspaceConfigPath, options);
|
||||
runtimeOptions = await optionsWithRuntimeConfig(
|
||||
runner,
|
||||
workspaceConfigPath,
|
||||
workspaceId,
|
||||
options,
|
||||
);
|
||||
rt = d.mgr.createFor(id, runtimeOptions);
|
||||
bindRuntime(id, rt, runner, workspaceConfigPath);
|
||||
} catch (error) {
|
||||
@@ -465,7 +483,11 @@ export function sessionRoutes(
|
||||
info(id, "Session created");
|
||||
bootstrap(
|
||||
id, rt, runner, workspaceConfigPath, d.mgr.configure(rt, runtimeOptions),
|
||||
runner.searchPack(b.question, id, workspaceConfigPath),
|
||||
runner.searchPack(
|
||||
b.question,
|
||||
id,
|
||||
(runtimeOptions as any).runtimeConfig?.path ?? workspaceConfigPath,
|
||||
),
|
||||
() => d.mgr.start(id, rt, runtimeOptions),
|
||||
);
|
||||
return { id };
|
||||
@@ -639,7 +661,12 @@ export function sessionRoutes(
|
||||
if (boundRuntimes.get(id) === current) boundRuntimes.delete(id);
|
||||
d.mgr.teardownIfCurrent(id, current);
|
||||
}
|
||||
runtimeOptions = optionsWithRuntimeConfig(runner, workspaceConfigPath, options);
|
||||
runtimeOptions = await optionsWithRuntimeConfig(
|
||||
runner,
|
||||
workspaceConfigPath,
|
||||
saved.workspace_id,
|
||||
options,
|
||||
);
|
||||
rt = d.mgr.createFor(id, runtimeOptions);
|
||||
bindRuntime(id, rt, runner, workspaceConfigPath);
|
||||
} catch {
|
||||
|
||||
@@ -213,23 +213,37 @@ export class ThtRunner {
|
||||
}
|
||||
|
||||
/** Render one immutable canonical registry revision into a backend-owned harness config. */
|
||||
acquireWorkspaceRuntime(workspaceConfigPath: string): RuntimeConfigLease {
|
||||
const rendered = renderWorkspaceRuntimeFromSnapshotPath({
|
||||
snapshotPath: workspaceConfigPath,
|
||||
harnessDir: this.cfg.harnessDir,
|
||||
configPath: this.cfg.configPath,
|
||||
dataRoot: this.cfg.dataRoot ?? (() => {
|
||||
throw new Error("registry workspace runtime requires an absolute data root");
|
||||
})(),
|
||||
secretRoots: this.cfg.secretRoots ?? [],
|
||||
semanticRuntime: this.cfg.semanticRuntime ?? DEFAULT_SEMANTIC_RUNTIME,
|
||||
workspaceSecretStore: this.cfg.workspaceSecretStore,
|
||||
});
|
||||
acquireWorkspaceRuntime(
|
||||
workspaceConfigPath: string,
|
||||
effectiveRelationships?: string,
|
||||
): RuntimeConfigLease {
|
||||
const effectiveRelationshipsPath = effectiveRelationships === undefined
|
||||
? undefined
|
||||
: this.createRuntimeSnapshot(effectiveRelationships);
|
||||
let rendered: ReturnType<typeof renderWorkspaceRuntimeFromSnapshotPath>;
|
||||
try {
|
||||
rendered = renderWorkspaceRuntimeFromSnapshotPath({
|
||||
snapshotPath: workspaceConfigPath,
|
||||
harnessDir: this.cfg.harnessDir,
|
||||
configPath: this.cfg.configPath,
|
||||
dataRoot: this.cfg.dataRoot ?? (() => {
|
||||
throw new Error("registry workspace runtime requires an absolute data root");
|
||||
})(),
|
||||
secretRoots: this.cfg.secretRoots ?? [],
|
||||
semanticRuntime: this.cfg.semanticRuntime ?? DEFAULT_SEMANTIC_RUNTIME,
|
||||
workspaceSecretStore: this.cfg.workspaceSecretStore,
|
||||
effectiveRelationshipsPath,
|
||||
});
|
||||
} catch (error) {
|
||||
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
|
||||
throw error;
|
||||
}
|
||||
let path: string;
|
||||
try {
|
||||
path = this.createRuntimeSnapshot(rendered.renderedConfig);
|
||||
} catch (error) {
|
||||
rendered.releaseSecrets();
|
||||
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
|
||||
throw error;
|
||||
}
|
||||
let released = false;
|
||||
@@ -241,6 +255,7 @@ export class ThtRunner {
|
||||
if (released) return;
|
||||
released = true;
|
||||
this.cleanupRuntimeSnapshot(path);
|
||||
if (effectiveRelationshipsPath) this.cleanupRuntimeSnapshot(effectiveRelationshipsPath);
|
||||
rendered.releaseSecrets();
|
||||
},
|
||||
};
|
||||
|
||||
@@ -243,7 +243,12 @@ function readSnapshotWorkspace(snapshotPath: string): {
|
||||
}
|
||||
}
|
||||
|
||||
function runtimePaths(dataRoot: string, workspaceId: string, workspaceRevision?: string): RuntimePaths {
|
||||
function runtimePaths(
|
||||
dataRoot: string,
|
||||
workspaceId: string,
|
||||
workspaceRevision?: string,
|
||||
effectiveRelationshipsPath?: string,
|
||||
): RuntimePaths {
|
||||
if (!isAbsolute(dataRoot)) throw new Error("registry workspace runtime requires an absolute data root");
|
||||
const root = join(dataRoot, "sessions", workspaceId);
|
||||
return {
|
||||
@@ -254,6 +259,9 @@ function runtimePaths(dataRoot: string, workspaceId: string, workspaceRevision?:
|
||||
...(workspaceRevision === undefined
|
||||
? {}
|
||||
: { annotations_root: join(dataRoot, "sessions", workspaceId, "revisions", workspaceRevision, "artifacts") }),
|
||||
...(effectiveRelationshipsPath === undefined
|
||||
? {}
|
||||
: { effective_relationships: effectiveRelationshipsPath }),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -301,6 +309,7 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
|
||||
dataRoot: string;
|
||||
secretRoots: readonly string[];
|
||||
semanticRuntime: SemanticRuntimeConfig;
|
||||
effectiveRelationshipsPath?: string;
|
||||
workspaceSecretStore?: WorkspaceSecretStore;
|
||||
}): RenderedWorkspaceRuntime {
|
||||
const secretLease = options.workspaceSecretStore === undefined
|
||||
@@ -325,7 +334,12 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
|
||||
workspaceId: options.workspaceId,
|
||||
workspaceRevision: options.workspaceRevision,
|
||||
revisionContentRoot: options.revisionContentRoot,
|
||||
runtimePaths: runtimePaths(options.dataRoot, options.workspaceId, options.workspaceRevision),
|
||||
runtimePaths: runtimePaths(
|
||||
options.dataRoot,
|
||||
options.workspaceId,
|
||||
options.workspaceRevision,
|
||||
options.effectiveRelationshipsPath,
|
||||
),
|
||||
installationOverlay: overlay,
|
||||
bindings,
|
||||
bindingDigest: stableBindingDigest(bindings),
|
||||
@@ -334,7 +348,12 @@ function renderWorkspaceRuntimeFromWorkspace(options: {
|
||||
renderedConfig: renderRuntimeConfig(
|
||||
options.workspace,
|
||||
bindings,
|
||||
runtimePaths(options.dataRoot, options.workspaceId, options.workspaceRevision),
|
||||
runtimePaths(
|
||||
options.dataRoot,
|
||||
options.workspaceId,
|
||||
options.workspaceRevision,
|
||||
options.effectiveRelationshipsPath,
|
||||
),
|
||||
context,
|
||||
overlay,
|
||||
options.semanticRuntime,
|
||||
@@ -353,6 +372,7 @@ export function renderWorkspaceRuntimeFromSnapshotPath(options: {
|
||||
dataRoot: string;
|
||||
secretRoots: readonly string[];
|
||||
semanticRuntime: SemanticRuntimeConfig;
|
||||
effectiveRelationshipsPath?: string;
|
||||
workspaceSecretStore?: WorkspaceSecretStore;
|
||||
}): RenderedWorkspaceRuntime {
|
||||
const snapshot = readSnapshotWorkspace(options.snapshotPath);
|
||||
@@ -366,6 +386,7 @@ export function renderWorkspaceRuntimeFromSnapshotPath(options: {
|
||||
dataRoot: options.dataRoot,
|
||||
secretRoots: options.secretRoots,
|
||||
semanticRuntime: options.semanticRuntime,
|
||||
effectiveRelationshipsPath: options.effectiveRelationshipsPath,
|
||||
workspaceSecretStore: options.workspaceSecretStore,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -12,6 +12,8 @@ export interface RuntimePaths {
|
||||
memory: string;
|
||||
/** Revision-qualified root for curated FK annotations (P5); optional for legacy callers. */
|
||||
annotations_root?: string;
|
||||
/** Immutable Catalog projection used as the exclusive runtime relationship source. */
|
||||
effective_relationships?: string;
|
||||
}
|
||||
|
||||
export interface RuntimeIdentity {
|
||||
|
||||
@@ -13,7 +13,10 @@ import type { WorkspaceRegistry, WorkspaceRevision } from "../src/workspaces/reg
|
||||
import type { WorkspaceDescriptor } from "../src/workspaces/schema.js";
|
||||
|
||||
const roots: string[] = [];
|
||||
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
const workspace: WorkspaceDescriptor = {
|
||||
workspace: { schema_version: 3, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
||||
@@ -36,6 +39,7 @@ function setup(
|
||||
catalogOperationCoordinator?: CatalogOperationCoordinator;
|
||||
catalogPostgresAccess?: CatalogPostgresAccess;
|
||||
} = {},
|
||||
workspaceDescriptor: WorkspaceDescriptor = workspace,
|
||||
) {
|
||||
const secretRoot = mkdtempSync(join(tmpdir(), "catalog-secret-"));
|
||||
const runtimeRoot = mkdtempSync(join(tmpdir(), "catalog-secret-runtime-"));
|
||||
@@ -45,7 +49,8 @@ function setup(
|
||||
const registry = {
|
||||
list: vi.fn(async () => [revision]),
|
||||
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
|
||||
read: vi.fn(async () => ({ workspace, revision })),
|
||||
read: vi.fn(async () => ({ workspace: workspaceDescriptor, revision })),
|
||||
readPinned: vi.fn(async () => ({ workspace: workspaceDescriptor, workspaceConfigPath: revision.snapshotPath })),
|
||||
} as unknown as WorkspaceRegistry;
|
||||
const app = buildApp(loadConfig({
|
||||
THT_HARNESS_DIR: "/missing",
|
||||
@@ -96,10 +101,31 @@ const fleetSnapshot: ObservedSchemaSnapshot = {
|
||||
};
|
||||
|
||||
test("lists every YAML workspace and creates its one database configuration", async () => {
|
||||
const { app } = setup();
|
||||
const { app, secretStore } = setup();
|
||||
const initial = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(initial.statusCode).toBe(200);
|
||||
expect(initial.json()).toMatchObject([{ workspaceId: "psd-clinical", configured: false, databaseName: "warehouse" }]);
|
||||
expect(initial.json()).toMatchObject([{
|
||||
workspaceId: "psd-clinical",
|
||||
configured: false,
|
||||
databaseName: "warehouse",
|
||||
workspaceRevision: { commit: revision.commit, blob: revision.blob },
|
||||
workspaceEvidence: { sourceType: null, state: "not_declared" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "configuration_required",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
}]);
|
||||
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_HOST", "runtime-db.internal");
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_PORT", "5432");
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_USER", "runtime-reader");
|
||||
vi.stubEnv("THT_WS_PSD_CLINICAL_DWH_TRANSPORT", "postgres_direct");
|
||||
secretStore.putMany("psd-clinical", { "dwh.password": "runtime-password" });
|
||||
const runtimeReady = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(runtimeReady.json()).toMatchObject([{
|
||||
runtimeBinding: { configurationState: "ready", sessionTransportSupported: true },
|
||||
}]);
|
||||
|
||||
const created = await app.inject({ method: "POST", url: "/catalog/databases", payload: direct });
|
||||
expect(created.statusCode).toBe(201);
|
||||
@@ -110,6 +136,39 @@ test("lists every YAML workspace and creates its one database configuration", as
|
||||
expect(listed.json()).toMatchObject([{ configured: true, binding: { transport: "postgres_direct", host: "db.internal" } }]);
|
||||
});
|
||||
|
||||
test("projects remote Evidence credential state without conflating catalog secrets", async () => {
|
||||
const evidenceWorkspace: WorkspaceDescriptor = {
|
||||
...workspace,
|
||||
evidence: {
|
||||
schema_version: 2,
|
||||
source: {
|
||||
type: "http",
|
||||
uris: ["https://evidence.example.test/guide.md"],
|
||||
authentication: "signed_urls_file",
|
||||
connect_timeout_ms: 5_000,
|
||||
read_timeout_ms: 30_000,
|
||||
max_bytes: 10 * 1024 * 1024,
|
||||
max_redirects: 5,
|
||||
allow_private_hosts: false,
|
||||
max_cache_bytes: 64 * 1024 * 1024,
|
||||
},
|
||||
policy: { max_chunk_chars: 4_000, retain_published_generations: 3 },
|
||||
},
|
||||
};
|
||||
const { app, secretStore } = setup({}, {}, evidenceWorkspace);
|
||||
|
||||
const missing = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(missing.json()).toMatchObject([{
|
||||
workspaceEvidence: { sourceType: "http", state: "configuration_required" },
|
||||
}]);
|
||||
|
||||
secretStore.putMany("psd-clinical", { "evidence.signed_urls": "https://signed.example.test/evidence" });
|
||||
const configured = await app.inject({ method: "GET", url: "/catalog/databases" });
|
||||
expect(configured.json()).toMatchObject([{
|
||||
workspaceEvidence: { sourceType: "http", state: "configured_unverified" },
|
||||
}]);
|
||||
});
|
||||
|
||||
test("lists orphaned records and takes the REST diagnostic path from workspace YAML", async () => {
|
||||
const { app, repository } = setup();
|
||||
await repository.create({
|
||||
|
||||
@@ -0,0 +1,207 @@
|
||||
import { expect, test } from "vitest";
|
||||
import {
|
||||
CatalogLogicalRelationshipService,
|
||||
LogicalRelationshipDuplicateError,
|
||||
LogicalRelationshipSchemaStaleError,
|
||||
LogicalRelationshipTargetNotUniqueError,
|
||||
LogicalRelationshipTypeIncompatibleError,
|
||||
} from "../src/catalog/logical-relationship-service.js";
|
||||
import { MemoryCatalogRepository } from "../src/catalog/memory-repository.js";
|
||||
import type { ObservedSchemaSnapshot } from "../src/catalog/types.js";
|
||||
|
||||
const column = (
|
||||
tableName: string,
|
||||
name: string,
|
||||
ordinalPosition: number,
|
||||
dataType: string,
|
||||
primaryKeyPosition: number | null,
|
||||
) => ({
|
||||
tableName, name, ordinalPosition, dataType, primaryKeyPosition,
|
||||
isNullable: false, defaultExpression: null, sourceComment: null,
|
||||
});
|
||||
|
||||
function schema(
|
||||
tableNames: string[],
|
||||
columns: ObservedSchemaSnapshot["columns"],
|
||||
relationships: ObservedSchemaSnapshot["relationships"] = [],
|
||||
): ObservedSchemaSnapshot {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
capabilities: { tables: "available", columns: "available", relationships: "available" },
|
||||
tables: tableNames.map((name) => ({ name, sourceComment: null })),
|
||||
columns,
|
||||
relationships,
|
||||
};
|
||||
}
|
||||
|
||||
async function setup(snapshot: ObservedSchemaSnapshot) {
|
||||
const repository = new MemoryCatalogRepository();
|
||||
const database = await repository.create({
|
||||
workspaceId: "relationships", engine: "postgres", databaseName: "warehouse", schema: "public",
|
||||
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
|
||||
});
|
||||
await repository.applySchemaSync(database.id, database.version, "all", [], snapshot);
|
||||
const service = new CatalogLogicalRelationshipService(repository);
|
||||
const context = (await repository.getLogicalRelationshipContext(database.id))!;
|
||||
const endpoint = (tableName: string, columnName: string) => context.endpoints.find((item) => (
|
||||
item.tableName === tableName && item.columnName === columnName
|
||||
))!;
|
||||
return { repository, database, service, endpoint };
|
||||
}
|
||||
|
||||
test("keeps excluded relationships across rebuild and recreates hard-deleted relationships", async () => {
|
||||
const { database, service, endpoint } = await setup(schema(
|
||||
["users", "orders"],
|
||||
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
|
||||
column("orders", "user_id", 2, "bigint", null)],
|
||||
));
|
||||
const source = endpoint("orders", "user_id");
|
||||
const target = endpoint("users", "id");
|
||||
const manual = await service.addManual(database.id, source.columnId, target.columnId);
|
||||
expect(manual).toMatchObject({ origin: "manual", status: "active" });
|
||||
|
||||
expect(await service.setStatus(database.id, manual.id, "excluded"))
|
||||
.toMatchObject({ status: "excluded" });
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 0, excluded: 1, ambiguous: 0,
|
||||
});
|
||||
expect((await service.list(database.id)).filter((item) => item.origin !== "physical"))
|
||||
.toMatchObject([{ id: manual.id, origin: "manual", status: "excluded" }]);
|
||||
|
||||
await service.deletePermanently(database.id, manual.id);
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
expect((await service.list(database.id)).filter((item) => item.origin !== "physical"))
|
||||
.toMatchObject([{ origin: "generated", status: "active" }]);
|
||||
});
|
||||
|
||||
test("normalizes snake, kebab, and camel names without fuzzy matching", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["users", "events"],
|
||||
[column("users", "id", 1, "bigint", 1), column("events", "id", 1, "bigint", 1),
|
||||
column("events", "user_id", 2, "bigint", null),
|
||||
column("events", "user-id", 3, "bigint", null),
|
||||
column("events", "userId", 4, "bigint", null),
|
||||
column("events", "userid", 5, "bigint", null),
|
||||
column("events", "unrelated", 6, "bigint", null)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 4, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("skips generic bare primary-key names while retaining table-qualified matches", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["users", "accounts", "events"],
|
||||
[column("users", "id", 1, "bigint", 1), column("accounts", "id", 1, "bigint", 1),
|
||||
column("events", "event_key", 1, "bigint", 1), column("events", "id", 2, "bigint", null),
|
||||
column("events", "user_id", 3, "bigint", null)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("infers foreign keys from composite-primary-key sources", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["users", "groups", "memberships"],
|
||||
[column("users", "id", 1, "bigint", 1), column("groups", "id", 1, "bigint", 1),
|
||||
column("memberships", "user_id", 1, "bigint", 1),
|
||||
column("memberships", "group_id", 2, "bigint", 2)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 2, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("maps time-key columns to the single primary key of dim_time", async () => {
|
||||
const { database, service } = await setup(schema(
|
||||
["dim_time", "admissions"],
|
||||
[column("dim_time", "day_key", 1, "integer", 1),
|
||||
column("admissions", "id", 1, "bigint", 1),
|
||||
column("admissions", "admission_time_key", 2, "integer", null),
|
||||
column("admissions", "discharge_time_key", 3, "integer", null)],
|
||||
));
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 2, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
});
|
||||
|
||||
test("skips ambiguous targets and physical foreign-key pairs", async () => {
|
||||
const ambiguous = await setup(schema(
|
||||
["user", "users", "events"],
|
||||
[column("user", "id", 1, "bigint", 1), column("users", "id", 1, "bigint", 1),
|
||||
column("events", "id", 1, "bigint", 1), column("events", "user_id", 2, "bigint", null)],
|
||||
));
|
||||
await expect(ambiguous.service.rebuildGenerated(ambiguous.database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 1,
|
||||
});
|
||||
|
||||
const physical = await setup(schema(
|
||||
["users", "orders"],
|
||||
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
|
||||
column("orders", "user_id", 2, "bigint", null)],
|
||||
[{
|
||||
constraintName: "orders_user_id_fkey", sourceTableName: "orders", targetTableName: "users",
|
||||
updateRule: "NO ACTION", deleteRule: "NO ACTION", deferrable: false, initiallyDeferred: false,
|
||||
columns: [{ position: 1, sourceColumnName: "user_id", targetColumnName: "id" }],
|
||||
}],
|
||||
));
|
||||
await expect(physical.service.rebuildGenerated(physical.database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 1, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
await expect(physical.service.addManual(
|
||||
physical.database.id,
|
||||
physical.endpoint("orders", "user_id").columnId,
|
||||
physical.endpoint("users", "id").columnId,
|
||||
)).rejects.toBeInstanceOf(LogicalRelationshipDuplicateError);
|
||||
});
|
||||
|
||||
test("validates target uniqueness and canonical type compatibility for manual relationships", async () => {
|
||||
const { database, service, endpoint } = await setup(schema(
|
||||
["users", "composite", "events"],
|
||||
[column("users", "id", 1, "bigint", 1),
|
||||
column("composite", "left_id", 1, "bigint", 1), column("composite", "right_id", 2, "bigint", 2),
|
||||
column("events", "id", 1, "bigint", 1), column("events", "user_id", 2, "integer", null),
|
||||
column("events", "composite_id", 3, "bigint", null)],
|
||||
));
|
||||
await expect(service.addManual(
|
||||
database.id, endpoint("events", "composite_id").columnId, endpoint("composite", "left_id").columnId,
|
||||
)).rejects.toBeInstanceOf(LogicalRelationshipTargetNotUniqueError);
|
||||
await expect(service.addManual(
|
||||
database.id, endpoint("events", "user_id").columnId, endpoint("users", "id").columnId,
|
||||
)).rejects.toBeInstanceOf(LogicalRelationshipTypeIncompatibleError);
|
||||
});
|
||||
|
||||
test("rebuild is additive when an existing generated relationship stops matching", async () => {
|
||||
const initial = schema(
|
||||
["users", "orders"],
|
||||
[column("users", "id", 1, "bigint", 1), column("orders", "id", 1, "bigint", 1),
|
||||
column("orders", "user_id", 2, "bigint", null)],
|
||||
);
|
||||
const { repository, database, service } = await setup(initial);
|
||||
await service.rebuildGenerated(database.id);
|
||||
const changed = structuredClone(initial);
|
||||
changed.columns.find((item) => item.tableName === "orders" && item.name === "user_id")!.dataType = "text";
|
||||
await repository.applySchemaSync(database.id, database.version, "columns", [], changed);
|
||||
await expect(service.rebuildGenerated(database.id)).resolves.toEqual({
|
||||
added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 0,
|
||||
});
|
||||
expect((await service.list(database.id)).filter((item) => item.origin === "generated")).toHaveLength(1);
|
||||
});
|
||||
|
||||
test("refuses inference until the current database version has a full schema sync", async () => {
|
||||
const repository = new MemoryCatalogRepository();
|
||||
const database = await repository.create({
|
||||
workspaceId: "unsynced-relationships",
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
schema: "public",
|
||||
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
|
||||
});
|
||||
const service = new CatalogLogicalRelationshipService(repository);
|
||||
|
||||
await expect(service.rebuildGenerated(database.id))
|
||||
.rejects.toBeInstanceOf(LogicalRelationshipSchemaStaleError);
|
||||
});
|
||||
@@ -12,6 +12,7 @@ import { up as upRuntimeSequencePrivileges } from "../src/catalog/migrations/004
|
||||
import { up as upDescriptionGeneration } from "../src/catalog/migrations/005_description_generation_runs.js";
|
||||
import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js";
|
||||
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
|
||||
import { up as upLogicalRelationships } from "../src/catalog/migrations/008_catalog_logical_relationships.js";
|
||||
|
||||
const dockerAvailable = spawnSync("docker", ["info"], { stdio: "ignore" }).status === 0;
|
||||
|
||||
@@ -26,6 +27,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
await sql`CREATE ROLE thothii_catalog_runtime`.execute(db);
|
||||
await upRuntimeSequencePrivileges(db);
|
||||
const sequencePrivilege = await sql<{ allowed: boolean }>`
|
||||
@@ -202,6 +204,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository performs scoped metadata cl
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const database = await repository.create({
|
||||
workspaceId: "cleanup-test",
|
||||
@@ -280,6 +283,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository atomically consolidates sel
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const database = await repository.create({
|
||||
workspaceId: "consolidation-test",
|
||||
@@ -379,6 +383,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
await upDescriptionGeneration(db);
|
||||
await upSensitiveSuggestionRuns(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
@@ -610,3 +615,63 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
||||
await container.stop();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
test.skipIf(!dockerAvailable)("PostgreSQL repository persists logical relationship lifecycle and tombstones", async () => {
|
||||
const container = await new PostgreSqlContainer("postgres:17.6-bookworm").start();
|
||||
const db = new Kysely<CatalogDatabase>({
|
||||
dialect: new PostgresDialect({ pool: new Pool({ connectionString: container.getConnectionUri() }) }),
|
||||
plugins: [new CamelCasePlugin()],
|
||||
});
|
||||
try {
|
||||
await upDatabases(db);
|
||||
await upTables(db);
|
||||
await upSchemaSync(db);
|
||||
await upSensitiveDataFlag(db);
|
||||
await upLogicalRelationships(db);
|
||||
const repository = new KyselyCatalogRepository(db);
|
||||
const database = await repository.create({
|
||||
workspaceId: "logical-relationships",
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
schema: "public",
|
||||
binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" },
|
||||
});
|
||||
await repository.applySchemaSync(database.id, database.version, "all", [], {
|
||||
schemaVersion: 1,
|
||||
capabilities: { tables: "available", columns: "available", relationships: "available" },
|
||||
tables: [{ name: "users", sourceComment: null }, { name: "orders", sourceComment: null }],
|
||||
columns: [
|
||||
{ tableName: "users", name: "id", ordinalPosition: 1, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: 1, sourceComment: null },
|
||||
{ tableName: "orders", name: "id", ordinalPosition: 1, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: 1, sourceComment: null },
|
||||
{ tableName: "orders", name: "user_id", ordinalPosition: 2, dataType: "bigint", isNullable: false, defaultExpression: null, primaryKeyPosition: null, sourceComment: null },
|
||||
],
|
||||
relationships: [],
|
||||
});
|
||||
const context = (await repository.getLogicalRelationshipContext(database.id))!;
|
||||
const source = context.endpoints.find((item) => item.tableName === "orders" && item.columnName === "user_id")!;
|
||||
const target = context.endpoints.find((item) => item.tableName === "users" && item.columnName === "id")!;
|
||||
const created = await repository.insertLogicalRelationship(database.id, source.columnId, target.columnId, false);
|
||||
expect(created).toMatchObject({ origin: "manual", status: "active" });
|
||||
await expect(repository.insertLogicalRelationship(database.id, source.columnId, target.columnId, true))
|
||||
.resolves.toBeUndefined();
|
||||
|
||||
expect(await repository.setLogicalRelationshipStatus(database.id, created!.id, "excluded"))
|
||||
.toMatchObject({ status: "excluded" });
|
||||
await expect(repository.insertGeneratedLogicalRelationships(database.id, [{
|
||||
sourceColumnId: source.columnId, targetColumnId: target.columnId,
|
||||
}])).resolves.toBe(0);
|
||||
|
||||
await expect(repository.deleteLogicalRelationship(database.id, created!.id)).resolves.toBe(true);
|
||||
await expect(repository.insertGeneratedLogicalRelationships(database.id, [{
|
||||
sourceColumnId: source.columnId, targetColumnId: target.columnId,
|
||||
}])).resolves.toBe(1);
|
||||
expect(await repository.listLogicalRelationships(database.id))
|
||||
.toMatchObject([{ origin: "generated", status: "active" }]);
|
||||
|
||||
await db.deleteFrom("catalogColumns").where("id", "=", source.columnId).execute();
|
||||
expect(await repository.listLogicalRelationships(database.id)).toEqual([]);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
await container.stop();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
@@ -122,6 +122,7 @@ async function setup(env: Record<string, string> = {}) {
|
||||
list: vi.fn(async () => [revision]),
|
||||
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
|
||||
read: vi.fn(async () => ({ workspace, revision })),
|
||||
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
|
||||
} as unknown as WorkspaceRegistry;
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test", ...env }), {
|
||||
thtRunner: {} as never,
|
||||
@@ -605,3 +606,125 @@ test("rejects a missing database without partially cleaning valid selections", a
|
||||
expect(response.statusCode).toBe(404);
|
||||
expect(await repository.listTables(database.id)).toHaveLength(2);
|
||||
});
|
||||
|
||||
test("serves one relationship map and supports the manual relationship lifecycle", async () => {
|
||||
const { app, repository, database } = await setup();
|
||||
await seedCatalog(repository, database);
|
||||
const tables = await repository.listTables(database.id);
|
||||
const patients = tables.find((table) => table.name === "patients")!;
|
||||
const visits = tables.find((table) => table.name === "visits")!;
|
||||
const patientId = (await repository.listColumns(database.id, patients.id)).find((item) => item.name === "id")!;
|
||||
const visitId = (await repository.listColumns(database.id, visits.id)).find((item) => item.name === "id")!;
|
||||
|
||||
const created = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships`,
|
||||
payload: { sourceColumnId: visitId.id, targetColumnId: patientId.id },
|
||||
});
|
||||
expect(created.statusCode).toBe(201);
|
||||
expect(created.json()).toMatchObject({ origin: "manual", status: "active", constraintName: null });
|
||||
|
||||
const listed = (await app.inject({
|
||||
method: "GET", url: `/catalog/databases/${database.id}/relationships`,
|
||||
})).json();
|
||||
expect(listed.map((item: { origin: string }) => item.origin).sort()).toEqual(["manual", "physical"]);
|
||||
|
||||
const relationshipId = created.json().id;
|
||||
const excluded = await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
|
||||
payload: { status: "excluded" },
|
||||
});
|
||||
expect(excluded.statusCode).toBe(200);
|
||||
expect(excluded.json()).toMatchObject({ origin: "manual", status: "excluded" });
|
||||
|
||||
const restored = await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
|
||||
payload: { status: "active" },
|
||||
});
|
||||
expect(restored.json()).toMatchObject({ status: "active" });
|
||||
|
||||
expect((await app.inject({
|
||||
method: "DELETE", url: `/catalog/databases/${database.id}/relationships/${relationshipId}`,
|
||||
})).statusCode).toBe(204);
|
||||
});
|
||||
|
||||
test("rejects generated inference while the Catalog schema is not current", async () => {
|
||||
const { app, database } = await setup();
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(409);
|
||||
expect(response.json()).toEqual({
|
||||
code: "relationship_schema_stale",
|
||||
message: "Synchronize the current database schema before managing logical relationships.",
|
||||
});
|
||||
});
|
||||
|
||||
test("rebuilds generated relationships and returns the exact summary", async () => {
|
||||
const { app, repository, database } = await setup();
|
||||
const observed = snapshot();
|
||||
observed.relationships = [];
|
||||
await seedCatalog(repository, database, observed);
|
||||
|
||||
const first = await app.inject({
|
||||
method: "POST", url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
});
|
||||
expect(first.statusCode).toBe(200);
|
||||
expect(first.json()).toEqual({ added: 1, alreadyPresent: 0, excluded: 0, ambiguous: 0 });
|
||||
const generated = (await repository.listLogicalRelationships(database.id))[0]!;
|
||||
|
||||
await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${generated.id}`,
|
||||
payload: { status: "excluded" },
|
||||
});
|
||||
const second = await app.inject({
|
||||
method: "POST", url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
});
|
||||
expect(second.json()).toEqual({ added: 0, alreadyPresent: 0, excluded: 1, ambiguous: 0 });
|
||||
});
|
||||
|
||||
test("validates relationship requests and keeps physical relationships read-only", async () => {
|
||||
const { app, repository, database } = await setup();
|
||||
await seedCatalog(repository, database);
|
||||
const physical = (await repository.listRelationships(database.id))[0]!;
|
||||
|
||||
const invalid = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships`,
|
||||
payload: { sourceColumnId: physical.columns[0].sourceColumnId, targetColumnId: physical.columns[0].targetColumnId, generated: true },
|
||||
});
|
||||
expect(invalid.statusCode).toBe(400);
|
||||
expect(invalid.json()).toMatchObject({ code: "relationship_request_invalid" });
|
||||
|
||||
const readOnly = await app.inject({
|
||||
method: "PATCH",
|
||||
url: `/catalog/databases/${database.id}/relationships/${physical.id}`,
|
||||
payload: { status: "excluded" },
|
||||
});
|
||||
expect(readOnly.statusCode).toBe(409);
|
||||
expect(readOnly.json()).toMatchObject({ code: "relationship_read_only" });
|
||||
});
|
||||
|
||||
test("requires database.manage for relationship map mutations", async () => {
|
||||
const { app, repository, database } = await setup({ AUTH_MODE: "upstream" });
|
||||
const observed = snapshot();
|
||||
observed.relationships = [];
|
||||
await seedCatalog(repository, database, observed);
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: `/catalog/databases/${database.id}/relationships/rebuild-generated`,
|
||||
headers: {
|
||||
"x-thoth-principal-issuer": "portal",
|
||||
"x-thoth-principal-subject": "catalog-reader",
|
||||
"x-thoth-is-admin": "0",
|
||||
},
|
||||
});
|
||||
expect(response.statusCode).toBe(403);
|
||||
expect(response.json()).toEqual({ code: "auth_forbidden", error: "This operation is not permitted" });
|
||||
});
|
||||
|
||||
@@ -54,6 +54,7 @@ async function setup() {
|
||||
list: vi.fn(async () => [revision]),
|
||||
listCatalog: vi.fn(async () => [{ id: "psd-clinical", name: "Policlinico San Donato", configurationState: "ready", revision }]),
|
||||
read: vi.fn(async () => ({ workspace, revision })),
|
||||
readPinned: vi.fn(async () => ({ workspace, workspaceConfigPath: revision.snapshotPath })),
|
||||
} as unknown as WorkspaceRegistry;
|
||||
const secretStore = new WorkspaceSecretStore({ root: secretRoot, runtimeRoot, installationId: "test" });
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/missing", NODE_ENV: "test" }), {
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
import { describe, expect, test, vi } from "vitest";
|
||||
import { EffectiveRelationshipSnapshotProvider } from "../src/catalog/effective-relationship-snapshot.js";
|
||||
import type { CatalogRelationship, WorkspaceDatabase } from "../src/catalog/types.js";
|
||||
|
||||
const timestamp = "2026-08-31T10:00:00.000Z";
|
||||
|
||||
function database(): WorkspaceDatabase {
|
||||
return {
|
||||
id: "database-1",
|
||||
workspaceId: "psd",
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
schema: "public",
|
||||
version: 1,
|
||||
createdAt: timestamp,
|
||||
updatedAt: timestamp,
|
||||
binding: { transport: "postgres_direct" },
|
||||
connectionStatus: "reachable",
|
||||
schemaSyncedVersion: 1,
|
||||
schemaSyncedAt: timestamp,
|
||||
};
|
||||
}
|
||||
|
||||
const operations = {
|
||||
async run<T>(_databaseId: string, operation: () => Promise<T>): Promise<T> {
|
||||
return await operation();
|
||||
},
|
||||
};
|
||||
|
||||
function relationship(options: {
|
||||
id: string;
|
||||
origin: "physical" | "generated" | "manual";
|
||||
status?: "active" | "excluded";
|
||||
sourceTable?: string;
|
||||
sourceColumns?: string[];
|
||||
targetTable?: string;
|
||||
targetColumns?: string[];
|
||||
}): CatalogRelationship {
|
||||
const sourceColumns = options.sourceColumns ?? ["user_id"];
|
||||
const targetColumns = options.targetColumns ?? ["id"];
|
||||
const columns = sourceColumns.map((sourceColumnName, position) => ({
|
||||
position: position + 1,
|
||||
sourceColumnId: `source-${position}`,
|
||||
sourceColumnName,
|
||||
targetColumnId: `target-${position}`,
|
||||
targetColumnName: targetColumns[position],
|
||||
}));
|
||||
const common = {
|
||||
id: options.id,
|
||||
databaseId: "database-1",
|
||||
sourceTableId: "source-table",
|
||||
sourceTableName: options.sourceTable ?? "orders",
|
||||
targetTableId: "target-table",
|
||||
targetTableName: options.targetTable ?? "users",
|
||||
columns,
|
||||
createdAt: timestamp,
|
||||
updatedAt: timestamp,
|
||||
};
|
||||
if (options.origin === "physical") {
|
||||
return {
|
||||
...common,
|
||||
constraintName: `fk_${options.id}`,
|
||||
updateRule: "NO ACTION",
|
||||
deleteRule: "NO ACTION",
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
lastSyncedDatabaseVersion: 1,
|
||||
lastSyncedAt: timestamp,
|
||||
origin: "physical",
|
||||
status: "active",
|
||||
};
|
||||
}
|
||||
return {
|
||||
...common,
|
||||
constraintName: null,
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
origin: options.origin,
|
||||
status: options.status ?? "active",
|
||||
columns: [columns[0]!],
|
||||
};
|
||||
}
|
||||
|
||||
describe("EffectiveRelationshipSnapshotProvider", () => {
|
||||
test("returns no projection for a workspace without a Catalog database", async () => {
|
||||
const list = vi.fn();
|
||||
const provider = new EffectiveRelationshipSnapshotProvider(
|
||||
{
|
||||
get: vi.fn().mockResolvedValue(undefined),
|
||||
getByWorkspace: vi.fn().mockResolvedValue(undefined),
|
||||
},
|
||||
{ list },
|
||||
operations,
|
||||
);
|
||||
|
||||
await expect(provider.render("legacy")).resolves.toBeUndefined();
|
||||
expect(list).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("renders a deterministic active map with physical precedence", async () => {
|
||||
const duplicateGenerated = relationship({ id: "generated", origin: "generated" });
|
||||
const physical = relationship({ id: "physical", origin: "physical" });
|
||||
const compositePhysical = relationship({
|
||||
id: "physical-composite",
|
||||
origin: "physical",
|
||||
sourceTable: "order_lines",
|
||||
sourceColumns: ["order_id", "tenant_id"],
|
||||
targetTable: "orders",
|
||||
targetColumns: ["id", "tenant_id"],
|
||||
});
|
||||
const manual = relationship({
|
||||
id: "manual",
|
||||
origin: "manual",
|
||||
sourceTable: "invoices",
|
||||
sourceColumns: ["customer_id"],
|
||||
targetTable: "customers",
|
||||
targetColumns: ["id"],
|
||||
});
|
||||
const excluded = relationship({
|
||||
id: "excluded",
|
||||
origin: "generated",
|
||||
status: "excluded",
|
||||
sourceTable: "invoices",
|
||||
});
|
||||
const list = vi.fn().mockResolvedValue([
|
||||
manual,
|
||||
duplicateGenerated,
|
||||
excluded,
|
||||
physical,
|
||||
compositePhysical,
|
||||
]);
|
||||
const provider = new EffectiveRelationshipSnapshotProvider(
|
||||
{
|
||||
get: vi.fn().mockResolvedValue(database()),
|
||||
getByWorkspace: vi.fn().mockResolvedValue(database()),
|
||||
},
|
||||
{ list },
|
||||
operations,
|
||||
);
|
||||
|
||||
const rendered = await provider.render("psd");
|
||||
expect(rendered?.endsWith("\n")).toBe(true);
|
||||
expect(JSON.parse(rendered!)).toEqual({
|
||||
schemaVersion: 1,
|
||||
workspaceId: "psd",
|
||||
relationships: [
|
||||
{
|
||||
sourceTable: "invoices",
|
||||
sourceColumns: ["customer_id"],
|
||||
targetTable: "customers",
|
||||
targetColumns: ["id"],
|
||||
origin: "manual",
|
||||
},
|
||||
{
|
||||
sourceTable: "order_lines",
|
||||
sourceColumns: ["order_id", "tenant_id"],
|
||||
targetTable: "orders",
|
||||
targetColumns: ["id", "tenant_id"],
|
||||
origin: "physical",
|
||||
},
|
||||
{
|
||||
sourceTable: "orders",
|
||||
sourceColumns: ["user_id"],
|
||||
targetTable: "users",
|
||||
targetColumns: ["id"],
|
||||
origin: "physical",
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(list).toHaveBeenCalledWith("database-1");
|
||||
});
|
||||
|
||||
test("fails closed when the Catalog schema is absent or stale", async () => {
|
||||
const stale = database();
|
||||
delete stale.schemaSyncedVersion;
|
||||
delete stale.schemaSyncedAt;
|
||||
const list = vi.fn();
|
||||
const provider = new EffectiveRelationshipSnapshotProvider(
|
||||
{
|
||||
get: vi.fn().mockResolvedValue(stale),
|
||||
getByWorkspace: vi.fn().mockResolvedValue(stale),
|
||||
},
|
||||
{ list },
|
||||
operations,
|
||||
);
|
||||
|
||||
await expect(provider.render("psd")).rejects.toThrow(
|
||||
"effective relationship snapshot requires a current full schema synchronization",
|
||||
);
|
||||
expect(list).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -517,6 +517,63 @@ test("creates a session from the active immutable workspace revision", async ()
|
||||
}));
|
||||
});
|
||||
|
||||
test("hands one effective relationship snapshot to both retrieval and Pi", async () => {
|
||||
const effective = JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
workspaceId: "default",
|
||||
relationships: [],
|
||||
});
|
||||
const render = vi.fn(async () => effective);
|
||||
const acquireWorkspaceRuntime = vi.fn((_workspace: string, relationships?: string) => ({
|
||||
path: "/runtime/with-relationships.yaml",
|
||||
workspaceId: "default",
|
||||
workspaceRevision: "e".repeat(40),
|
||||
release: vi.fn(),
|
||||
}));
|
||||
const searchPack = vi.fn(async () => {});
|
||||
const createFor = vi.fn(() => ({ bridge: { onClientEvent: () => {} } }));
|
||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||
thtRunner: {
|
||||
sessionNew: async () => ({ id: "effective-map" }),
|
||||
acquireWorkspaceRuntime,
|
||||
searchPack,
|
||||
} as any,
|
||||
effectiveRelationshipSnapshotProvider: { render } as any,
|
||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||
mgr: {
|
||||
get: () => undefined,
|
||||
createFor,
|
||||
configure: async () => {},
|
||||
start: () => {},
|
||||
} as any,
|
||||
getSettings: () => ({ workspace: "default", thinking: "low" }) as any,
|
||||
});
|
||||
|
||||
const response = await app.inject({
|
||||
method: "POST",
|
||||
url: "/sessions",
|
||||
payload: { question: "Which users placed orders?", workspaceId: "default" },
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(200);
|
||||
expect(render).toHaveBeenCalledWith("default");
|
||||
expect(acquireWorkspaceRuntime).toHaveBeenCalledWith(
|
||||
expect.stringContaining("/default.yaml"),
|
||||
effective,
|
||||
);
|
||||
expect(createFor).toHaveBeenCalledWith(
|
||||
"effective-map",
|
||||
expect.objectContaining({
|
||||
runtimeConfig: expect.objectContaining({ path: "/runtime/with-relationships.yaml" }),
|
||||
}),
|
||||
);
|
||||
expect(searchPack).toHaveBeenCalledWith(
|
||||
"Which users placed orders?",
|
||||
"effective-map",
|
||||
"/runtime/with-relationships.yaml",
|
||||
);
|
||||
});
|
||||
|
||||
test("rejects an SSH-only workspace before persisting or starting a session", async () => {
|
||||
const sessionNew = vi.fn(async () => ({ id: "must-not-exist" }));
|
||||
const ensure = vi.fn(async () => ({ ok: true }));
|
||||
|
||||
@@ -197,6 +197,25 @@ test("ThtRunner uses a vault secret only for the lifetime of its runtime lease",
|
||||
expect(existsSync(rendered.database.password_file)).toBe(false);
|
||||
});
|
||||
|
||||
test("ThtRunner binds and cleans the effective relationship snapshot with its runtime lease", async () => {
|
||||
const f = await fixture();
|
||||
const runner = runnerFor(f);
|
||||
const relationships = JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
workspaceId: "psd-clinical",
|
||||
relationships: [],
|
||||
});
|
||||
|
||||
const lease = runner.acquireWorkspaceRuntime(f.revision.snapshotPath, relationships);
|
||||
const rendered = parse(readFileSync(lease.path, "utf8")) as {
|
||||
paths: { effective_relationships: string };
|
||||
};
|
||||
|
||||
expect(readFileSync(rendered.paths.effective_relationships, "utf8")).toBe(relationships);
|
||||
lease.release();
|
||||
expect(existsSync(rendered.paths.effective_relationships)).toBe(false);
|
||||
});
|
||||
|
||||
test("separate runtime leases hand off byte-identical revision Evidence configs accepted by tht", async () => {
|
||||
const f = await fixture();
|
||||
const runner = runnerFor(f);
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Use the Catalog as the logical relationship authority
|
||||
|
||||
ThothII stores database-declared foreign keys and user-managed Logical Relationships in separate
|
||||
Catalog models, as required by ADR-0006, but exposes them through one effective relationship map.
|
||||
Physical relationships remain read-only and are refreshed from the database. Logical relationships
|
||||
are either Generated by deterministic column-name inference or Manual; inference does not call an
|
||||
AI model and does not inspect source values.
|
||||
|
||||
A generated rebuild is additive. It preserves active and Manual relationships, never reactivates a
|
||||
logically deleted relationship, and may recreate a relationship only after permanent deletion. A
|
||||
logical deletion is therefore represented by retaining the relationship with an exclusion marker;
|
||||
a permanent deletion removes it. Inference accepts only unambiguous, type-compatible matches to a
|
||||
single-column primary key and skips all other candidates. It recognizes normalized table-qualified
|
||||
names such as `user_id -> users.id`, exact non-generic primary-key names with one owner, and the
|
||||
warehouse convention `*time_key -> dim_time.<single PK>`. A source column may participate in a
|
||||
composite primary key; bare generic names such as `id`, `key`, `code`, and `pk` are not evidence by
|
||||
themselves.
|
||||
|
||||
An exclusion is durable while both Catalog Column endpoints exist. Explicit metadata cleanup of an
|
||||
endpoint table or column is a destructive boundary: it permanently removes every relationship and
|
||||
exclusion attached to that endpoint, invalidates the synchronized-catalog marker, and requires a
|
||||
full schema synchronization. The newly imported endpoints may then be inferred again. Preserving an
|
||||
exclusion across endpoint destruction would require a second denormalized name-based identity, which
|
||||
this design deliberately avoids.
|
||||
|
||||
The installation-local Catalog is the sole writable authority for Logical Relationships. Git-pinned
|
||||
workspace annotations remain authoritative for descriptive metadata but their embedded foreign keys
|
||||
are legacy compatibility data. The backend materializes the active effective map as an immutable,
|
||||
deterministic runtime snapshot when a session starts or resumes. When that snapshot is present, the
|
||||
harness uses it as the exclusive relationship source and ignores relationships embedded in both the
|
||||
physical schema artifact and workspace annotations; a missing or invalid declared snapshot fails
|
||||
closed. Legacy runtimes without a snapshot retain the previous merge behavior.
|
||||
|
||||
The snapshot is a projection, not another authored store. Its lifetime is tied to the runtime-config
|
||||
lease, physical relationships take precedence over duplicate Logical Relationships, composite-key
|
||||
column order is retained, and one Pi process observes one stable map for its complete lifetime.
|
||||
@@ -6,7 +6,7 @@ session workflow.
|
||||
|
||||
## What the catalog owns
|
||||
|
||||
For each YAML workspace, an administrator may create at most one database configuration. It holds
|
||||
For each YAML workspace, an administrator may configure at most one Metadata Catalog binding. It holds
|
||||
the database name, schema, connection binding, write-only encrypted secrets, observed physical
|
||||
schema, optional curated descriptions, generated descriptions, and durable operation history.
|
||||
|
||||
@@ -27,6 +27,18 @@ It uses `GET /catalog/metrics` without `databaseId` for installation totals and
|
||||
the current database. Choose a selection-scoped operation from the action selector and then press
|
||||
**Run**; unavailable operations remain listed with an explanation. Row-specific actions are the icon
|
||||
controls in the final column, and each navigation or action icon has an immediate conceptual tooltip.
|
||||
An unconfigured workspace exposes **Configure catalog** directly on its row; there is no global
|
||||
database-creation action and the selected workspace cannot be changed in the configuration form.
|
||||
|
||||
The master grid keeps three independent states visible:
|
||||
|
||||
- **Revision / Evidence** comes from the active immutable workspace revision. Filesystem Evidence is
|
||||
materialized with that revision; remote Evidence is reported as configured-but-unverified or as
|
||||
requiring credentials.
|
||||
- **NL→SQL runtime** is calculated from the workspace DWH/Evidence requirements and runtime secret
|
||||
store. It also reports transports, such as SSH, that are diagnostic-only and unsupported by sessions.
|
||||
- **Metadata Catalog** reports whether the installation-local catalog configuration exists, then shows
|
||||
its separately versioned connection-test or synchronization state.
|
||||
|
||||
Configuration, object details, metadata editors, synchronization history, description history,
|
||||
sensitive-field review, and suggestion-run history open in right-side drawers backed by the
|
||||
@@ -42,8 +54,9 @@ remains available only until the integrated Fleet Ledger surface passes owner ac
|
||||
|
||||
## Configure and test a database
|
||||
|
||||
1. Open **Database Management** and choose a workspace.
|
||||
2. Create its PostgreSQL configuration. Choose `postgres_direct`, `rest_api`, or `ssh_tunnel` and
|
||||
1. Open **Database Management** and find the repository workspace marked **Not configured**.
|
||||
2. Choose **Configure catalog** on that row. Configure its PostgreSQL catalog binding with
|
||||
`postgres_direct`, `rest_api`, or `ssh_tunnel` and
|
||||
complete the binding fields that the chosen transport requires.
|
||||
3. Enter secrets only when replacing them. They remain write-only and are never returned by the
|
||||
application.
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# ThothII Docker refresh
|
||||
|
||||
The active local `psd-local` stack uses the operator environment generated for the installation:
|
||||
|
||||
`deploy/psd/operator.env`
|
||||
|
||||
It is not `deploy/thothii.env` (that file is empty) and `deploy/env/local.env` is only an example
|
||||
path referenced by the generic launcher. The running stack also uses these Compose overlays:
|
||||
|
||||
```text
|
||||
compose.yaml
|
||||
deploy/compose.local.yaml
|
||||
deploy/compose.git-ssh.yaml
|
||||
deploy/psd/connector-secrets.yaml
|
||||
```
|
||||
|
||||
Refresh the stack from the repository root with:
|
||||
|
||||
```bash
|
||||
compose_psd=(
|
||||
docker compose
|
||||
--env-file deploy/psd/operator.env
|
||||
-p thothii-18998cca7b0a
|
||||
-f compose.yaml
|
||||
-f deploy/compose.local.yaml
|
||||
-f deploy/compose.git-ssh.yaml
|
||||
-f deploy/psd/connector-secrets.yaml
|
||||
)
|
||||
|
||||
"${compose_psd[@]}" config --quiet
|
||||
"${compose_psd[@]}" build core frontend
|
||||
"${compose_psd[@]}" stop core frontend
|
||||
"${compose_psd[@]}" up -d catalog-db
|
||||
"${compose_psd[@]}" run --rm catalog-migrate
|
||||
"${compose_psd[@]}" up -d --remove-orphans
|
||||
```
|
||||
|
||||
Building before the stop keeps the existing application available if an image fails to compile.
|
||||
Stopping only `core` and `frontend` prevents the old backend from using a newly migrated catalog;
|
||||
the database, Qdrant, embedding service, named volumes, and installation state remain in place.
|
||||
|
||||
The installation secret files referenced by that env file live under `deploy/psd/secrets/` and
|
||||
must never be committed or printed.
|
||||
@@ -109,12 +109,13 @@ canonico; i percorsi e i comportamenti descrivono il sorgente disponibile il 202
|
||||
conserva il valore esistente e la sostituzione è un'azione esplicita. Delete rimuove anche i
|
||||
segreti associati.
|
||||
34. La pagina usa AG Grid come master e un form React come detail, con sezioni Database, Connection
|
||||
e TLS/SSH condizionali. La toolbar offre `Add database`; le righe `unconfigured` offrono
|
||||
`Configure`. Entrambe selezionano esclusivamente workspace YAML senza un database e creano il
|
||||
record soltanto al Save; `workspace_id` diventa immutabile dopo la creazione.
|
||||
35. La grid mostra workspace, database, schema, transport, endpoint, stato connessione e ultimo
|
||||
aggiornamento. Su schermi piccoli il dettaglio occupa il pannello completo. Il cambio riga con
|
||||
modifiche non salvate e Delete richiedono conferma, senza conferma testuale tipizzata.
|
||||
e TLS/SSH condizionali. Non esiste un'azione globale `Add database`: ogni riga `unconfigured`
|
||||
offre `Configure catalog`, apre il form già vincolato a quello specifico workspace YAML e crea il
|
||||
record soltanto al Save; `workspace_id` non è selezionabile né modificabile.
|
||||
35. La grid mostra separatamente revisione/Evidence del workspace, binding runtime NL→SQL e
|
||||
configurazione del Metadata Catalog, oltre a database, schema, endpoint e ultimo aggiornamento.
|
||||
Su schermi piccoli il dettaglio occupa il pannello completo. Il cambio riga con modifiche non
|
||||
salvate e Delete richiedono conferma, senza conferma testuale tipizzata.
|
||||
36. La Database Binding conserva `connection_status`, `tested_version`, `last_tested_at`, un codice
|
||||
errore e un messaggio breve sanificato. Non conserva stack trace, DSN, credenziali o output grezzo
|
||||
del driver.
|
||||
|
||||
@@ -0,0 +1,299 @@
|
||||
# Visualizzatore ERD nel browser: proposta solo open source
|
||||
|
||||
Data dell'audit: 31 agosto 2026.
|
||||
|
||||
## Decisione
|
||||
|
||||
Per ThothII sceglierei **AntV X6 + ELK.js** come fondazione del visualizzatore ERD.
|
||||
|
||||
- **AntV X6** è MIT, non ha una distinta edizione Professional e include nel progetto OSS
|
||||
le funzioni che servono davvero a un diagramma complesso: nodi e porte personalizzabili,
|
||||
pan/zoom, selezione, minimappa, history, clipboard, tastiera, routing, virtual rendering ed
|
||||
export SVG/PNG/JPEG.
|
||||
- **ELK.js** è il motore di layout automatico, EPL-2.0 e senza tier commerciale. Il layout
|
||||
layered gestisce porte, vincoli sull'ordine delle porte, archi multipli, self-loop e routing
|
||||
ortogonale: proprietà importanti per foreign key composite e schemi affollati.
|
||||
- La combinazione non dipende da esempi o componenti a pagamento. Il lavoro applicativo che
|
||||
resta da fare — semantica ER, livelli di dettaglio, filtri e adattatore X6/ELK — appartiene
|
||||
davvero al dominio di ThothII e non è una funzione nascosta dietro un piano Pro.
|
||||
|
||||
La seconda scelta è **React Flow + ELK.js**. È probabilmente l'integrazione più naturale con
|
||||
l'attuale frontend React e ha una migliore storia documentata per accessibilità. Il runtime è
|
||||
MIT e non viene tecnicamente depotenziato, ma diversi pattern avanzati già implementati
|
||||
(auto-layout, edge routing, undo/redo, copy/paste, expand/collapse) sono pubblicati come esempi
|
||||
con licenza React Flow Pro. Non è quindi la scelta più lineare se il criterio prioritario è
|
||||
evitare anche una dipendenza progettuale da materiale Professional.
|
||||
|
||||
## Cosa significa «open source senza limitazioni Professional»
|
||||
|
||||
L'audit distingue tre casi:
|
||||
|
||||
1. **OSS completo**: licenza OSI e nessun tier commerciale che trattenga funzioni essenziali.
|
||||
2. **OSS con caveat**: il motore è aperto, ma esistono esempi premium, API instabili o limiti
|
||||
architetturali rilevanti. Può essere usato, purché il limite sia accettato esplicitamente.
|
||||
3. **Escluso**: licenza proprietaria oppure core aperto con funzioni necessarie al viewer
|
||||
complesso riservate alla versione commerciale.
|
||||
|
||||
La valutazione riguarda sia la licenza sia ciò che si può effettivamente costruire senza
|
||||
acquistare un prodotto complementare. Le note sulle licenze sono tecniche, non consulenza legale.
|
||||
|
||||
## Matrice di selezione
|
||||
|
||||
| Soluzione | Licenza OSS | Tier Pro rilevante | Adeguatezza a ERD complessi | Verdetto |
|
||||
|---|---|---|---|---|
|
||||
| **AntV X6 + ELK.js** | MIT + EPL-2.0 | Nessuno individuato | Porte per colonna, minimappa, routing, export, virtual rendering e layout avanzato | **Consigliata** |
|
||||
| **React Flow + ELK.js** | MIT + EPL-2.0 | Esempi/pattern avanzati Pro, non runtime separato | Ottima UX React, nodi HTML, minimappa e accessibilità; export SVG non nativo | **Valida con caveat** |
|
||||
| **maxGraph** | Apache-2.0 | Nessuno | Molto completo, SVG, porte, folding e layout; API imperativa e integrazione React costosa | **Valida con caveat** |
|
||||
| **Cytoscape.js + ELK** | MIT | Nessuno | Molto scalabile come grafo, ma meno adatto a tabelle ricche e porte per campo | **Alternativa di nicchia** |
|
||||
| **Liam ERD/CLI** | Apache-2.0 | Nessuno | Viewer ERD già rifinito e self-hostable; il package embeddable è interno e instabile | **Reference o app separata** |
|
||||
| **Graphviz + Viz.js** | EPL-2.0 + MIT | Nessuno | Layout ed SVG eccellenti; non offre da solo un explorer applicativo | **Renderer di export** |
|
||||
| **Mermaid.js + Panzoom** | MIT + MIT/ISC | Mermaid Chart è separato | Facile, ma insufficiente per interazioni e modelli ER ricchi | **Solo preview semplice** |
|
||||
| **Sprotty/GLSP** | EPL-2.0 | Nessuno | Potente e completo, ma è un framework di modeling più pesante del necessario | **Non prioritario** |
|
||||
| **JointJS** | Core MPL-2.0 | Molte funzioni utili sono in JointJS+ | Lo split commerciale intercetta proprio le necessità del viewer | **Esclusa** |
|
||||
| **GoJS** | Proprietaria | Licenza di deployment | Completa tecnicamente, ma non open source | **Esclusa** |
|
||||
| **yFiles** | Proprietaria | Licenza commerciale | Completa tecnicamente, ma non open source | **Esclusa** |
|
||||
|
||||
## 1. Scelta principale: AntV X6 + ELK.js
|
||||
|
||||
### Funzioni disponibili nell'open source
|
||||
|
||||
X6 offre un canvas diagrammatico basato su SVG con supporto anche a nodi HTML/React. Il core
|
||||
espone porte, archi personalizzabili, self-loop e multiedge. I plugin distribuiti dal progetto
|
||||
comprendono Scroller, MiniMap, Selection, History, Clipboard, Keyboard, DnD, Stencil, Snapline,
|
||||
Transform ed Export. La documentazione copre inoltre router `orth`, `manhattan` ed `er`, zoom e
|
||||
panning. Non è emersa un'edizione X6 Pro che renda a pagamento queste capacità.
|
||||
|
||||
Fonti primarie: [repository e licenza X6](https://github.com/antvis/X6),
|
||||
[porte](https://x6.antv.antgroup.com/en/tutorial/basic/port),
|
||||
[router](https://x6.antv.antgroup.com/en/api/registry/router),
|
||||
[minimappa](https://x6.antv.antgroup.com/en/tutorial/plugins/minimap),
|
||||
[scroller](https://x6.antv.antgroup.com/en/tutorial/plugins/scroller) ed
|
||||
[export](https://x6.antv.antgroup.com/en/tutorial/plugins/export).
|
||||
|
||||
ELK non è un renderer: calcola la geometria del grafo. L'algoritmo layered supporta porte e
|
||||
relativi vincoli, archi ortogonali, self-loop e archi multipli. Va eseguito in un Web Worker per
|
||||
non bloccare l'interfaccia sui modelli grandi. L'adattatore dovrà passare a ELK le porte delle
|
||||
colonne e usare anche le `edge sections` e i bend point restituiti, non soltanto le coordinate
|
||||
dei nodi.
|
||||
|
||||
Fonti primarie: [ELK.js e licenza](https://github.com/kieler/elkjs) e
|
||||
[ELK layered](https://eclipse.dev/elk/reference/algorithms/org-eclipse-elk-layered.html).
|
||||
|
||||
### Limiti reali
|
||||
|
||||
- X6 è più imperativo di React Flow: conviene incapsularlo in un singolo componente React con
|
||||
un adapter stabile, evitando di spargere istanze e listener nel resto dell'applicazione.
|
||||
- L'accessibilità per screen reader non è documentata al livello di React Flow; tab order,
|
||||
focus, descrizioni ARIA e navigazione da tastiera richiedono test e lavoro applicativo.
|
||||
- Un nodo React/HTML può introdurre `foreignObject` nell'SVG. Per un export portabile e
|
||||
stampabile è meglio produrre un SVG separato dallo stesso modello e dalla geometria ELK.
|
||||
|
||||
Questi sono costi tecnici, non limitazioni commerciali.
|
||||
|
||||
## 2. Seconda scelta: React Flow + ELK.js
|
||||
|
||||
`@xyflow/react` è MIT. Custom nodes React, handle multipli, pan/zoom, fit view, Controls,
|
||||
MiniMap, selezione e funzioni di accessibilità sono nel runtime OSS. React Flow Pro vende
|
||||
supporto, template ed esempi con relativo codice sorgente; non esiste una libreria runtime Pro
|
||||
che sblocchi il canvas.
|
||||
|
||||
Il caveat è comunque concreto: auto-layout, edge routing, undo/redo, copy/paste ed
|
||||
expand/collapse compaiono nel catalogo degli esempi Pro e quel codice usa la **xyflow Pro
|
||||
License**, non la MIT del core. Le stesse funzioni possono essere sviluppate sopra le API OSS,
|
||||
ma non si può considerare tutto il materiale ufficiale liberamente riutilizzabile.
|
||||
|
||||
Fonti primarie: [package React Flow](https://github.com/xyflow/xyflow/blob/main/packages/react/package.json),
|
||||
[funzioni del core](https://reactflow.dev/index),
|
||||
[catalogo degli esempi Pro](https://reactflow.dev/examples/pro-examples),
|
||||
[auto-layout](https://reactflow.dev/examples/layout/auto-layout) e
|
||||
[termini Pro](https://reactflow.dev/pro).
|
||||
|
||||
È una buona scelta se si privilegiano velocità d'integrazione React, componenti HTML e
|
||||
accessibilità. Non è la prima scelta di questo audit perché l'utente ha chiesto espressamente
|
||||
di minimizzare la distanza fra ciò che è open source e l'offerta Professional.
|
||||
|
||||
Altro limite: React Flow combina nodi DOM e archi SVG. L'esempio ufficiale di download usa
|
||||
`html-to-image`, quindi il diagramma interattivo non si traduce automaticamente in un SVG puro.
|
||||
|
||||
## 3. Altre soluzioni interamente open source
|
||||
|
||||
### maxGraph
|
||||
|
||||
maxGraph, successore TypeScript di mxGraph, è Apache-2.0 e non ha un piano Pro. Offre rendering
|
||||
SVG, connection constraints/porte, loop e multigraph, layout, routing, outline, grouping e
|
||||
folding. Può quindi sostenere un editor ERD ricco.
|
||||
|
||||
Lo terrei come terza scelta: l'API è imperativa, non esiste un wrapper React ufficiale, la
|
||||
virtualizzazione non è documentata e gran parte dell'accessibilità va costruita. Il progetto è
|
||||
ancora pre-1.0. Il costo di integrazione e manutenzione sarebbe maggiore di X6.
|
||||
|
||||
Fonti: [licenza e repository](https://github.com/maxGraph/maxGraph),
|
||||
[guida Vite/TypeScript](https://maxgraph.github.io/maxGraph/docs/getting-started/) e
|
||||
[gestione della complessità](https://maxgraph.github.io/maxGraph/docs/usage/group-and-complexity-management/).
|
||||
|
||||
### Cytoscape.js
|
||||
|
||||
Cytoscape.js e molte estensioni sono MIT, senza tier professionale. È ottimo per grandi grafi,
|
||||
filtri, selezioni e algoritmi di rete. Il rendering Canvas è però meno naturale per schede-tabella
|
||||
ricche, testo selezionabile e handle collegati alle singole colonne. Inoltre l'adapter
|
||||
`cytoscape.js-elk` non passa le porte a ELK né usa le route degli archi: non risolve da solo il
|
||||
problema delle foreign key per colonna.
|
||||
|
||||
È appropriato per una vista di dipendenze ad alto livello, non come renderer ERD principale.
|
||||
L'estensione `cytoscape-svg` è GPL-3.0; in un prodotto che non vuole assorbire quel vincolo è
|
||||
preferibile una pipeline Graphviz/Viz.js o un generatore SVG proprio.
|
||||
|
||||
Fonti: [Cytoscape.js](https://github.com/cytoscape/cytoscape.js),
|
||||
[adapter ELK](https://github.com/cytoscape/cytoscape.js-elk) e
|
||||
[cytoscape-svg](https://github.com/kinimesi/cytoscape-svg).
|
||||
|
||||
### Liam ERD
|
||||
|
||||
Liam ERD è Apache-2.0, self-hostable e già orientato al problema: pan/zoom, ricerca, filtro,
|
||||
command palette e modalità `TABLE_NAME`, `KEY_ONLY`, `ALL_FIELDS`. La documentazione dichiara
|
||||
supporto a schemi con oltre cento tabelle. La CLI può generare un'app Vite statica.
|
||||
|
||||
Non userei però direttamente `@liam-hq/erd-core` dentro ThothII: il maintainer lo definisce una
|
||||
dipendenza interna, ne sconsiglia l'uso diretto e avverte che l'API può cambiare; il package è
|
||||
ancora 0.x. Inoltre occorre verificare la compatibilità con la versione React di ThothII e la
|
||||
fedeltà delle foreign key composite. Liam è quindi un ottimo benchmark UX, una CLI per una vista
|
||||
separata o una base da forkare consapevolmente, non l'interfaccia stabile su cui fondare il
|
||||
prodotto.
|
||||
|
||||
Fonti: [repository Liam](https://github.com/liam-hq/liam),
|
||||
[README di erd-core](https://github.com/liam-hq/liam/blob/main/frontend/packages/erd-core/README.md),
|
||||
[funzioni UI](https://liambx.com/docs/ui-features) e [CLI](https://liambx.com/docs/cli).
|
||||
|
||||
### Sprotty/GLSP
|
||||
|
||||
Sprotty e Eclipse GLSP sono EPL-2.0 e offrono un'infrastruttura seria per diagrammi SVG,
|
||||
layout e protocolli client/server. Sono pensati per strumenti di modeling estensibili, con
|
||||
dependency injection e un'architettura più ampia di un viewer. Restano una soluzione OSS valida,
|
||||
ma sproporzionata per questa esigenza salvo che ThothII evolva in un vero editor di modelli.
|
||||
|
||||
Fonti: [Sprotty](https://github.com/eclipse-sprotty/sprotty) ed
|
||||
[Eclipse GLSP](https://eclipse.dev/glsp/documentation/overview/).
|
||||
|
||||
## 4. Renderer complementari, non fondazioni del viewer
|
||||
|
||||
### Graphviz e Viz.js
|
||||
|
||||
Graphviz è EPL-2.0; `@viz-js/viz`, il port WebAssembly utilizzabile nel browser, è MIT. Graphviz
|
||||
produce SVG di alta qualità, supporta label HTML-like e porte e rimane molto utile per export,
|
||||
stampa o una vista read-only. Non fornisce però da solo ricerca, filtri, livelli di dettaglio,
|
||||
editing o gestione dello stato applicativo.
|
||||
|
||||
Lo userei come renderer di export alternativo, non come UI primaria. Se il layout a schermo
|
||||
deve corrispondere esattamente all'export, è preferibile generare l'SVG direttamente dalla
|
||||
geometria ELK invece di mantenere due motori di layout indipendenti.
|
||||
|
||||
Fonti: [licenza Graphviz](https://graphviz.org/license/),
|
||||
[formati SVG](https://graphviz.org/docs/outputs/svg/) e
|
||||
[Viz.js](https://github.com/mdaines/viz-js).
|
||||
|
||||
### Mermaid.js e librerie di pan/zoom
|
||||
|
||||
Mermaid.js è MIT e non ha feature del renderer open source bloccate. **Mermaid Chart** è invece
|
||||
un servizio commerciale distinto e il suo piano gratuito ha limiti: non va confuso con la
|
||||
libreria self-hosted.
|
||||
|
||||
Si può aggiungere navigazione a un SVG Mermaid con `@panzoom/panzoom` (MIT) o `d3-zoom` (ISC),
|
||||
entrambi senza tier Pro. Questo migliora l'esperienza corrente, ma non supera i limiti strutturali
|
||||
del diagramma ER Mermaid: interazioni a livello di tabella, controllo ridotto delle porte e del
|
||||
routing, assenza di semantic zoom e difficoltà crescente su schemi molto grandi.
|
||||
|
||||
Mermaid resta adatto a preview e documentazione, non al viewer finale.
|
||||
|
||||
Fonti: [Mermaid.js](https://github.com/mermaid-js/mermaid),
|
||||
[distinzione da Mermaid Chart](https://mermaid.ai/open-source/ecosystem/mermaid-chart.html),
|
||||
[Panzoom](https://github.com/timmywil/panzoom) e [d3-zoom](https://github.com/d3/d3-zoom).
|
||||
|
||||
### Python
|
||||
|
||||
SchemaSpy ed ERAlchemy sono open source e possono generare documentazione o grafi attraverso
|
||||
Graphviz, ma non sostituiscono il componente interattivo React. Python è utile lato backend per
|
||||
normalizzare metadati o produrre artefatti batch; pan/zoom, selezione, ricerca e dettagli restano
|
||||
responsabilità del browser. Aggiungere un servizio Python solo per disegnare l'ERD non porta un
|
||||
vantaggio architetturale a ThothII.
|
||||
|
||||
## 5. Soluzioni escluse
|
||||
|
||||
### JointJS / JointJS+
|
||||
|
||||
Il core JointJS è MPL-2.0, ma JointJS+ è commerciale e comprende proprio molte funzioni che
|
||||
servirebbero qui: PaperScroller, Navigator/minimappa, selection, clipboard, keyboard, undo/redo,
|
||||
toolbar, export e layout aggiuntivi. Sarebbe tecnicamente possibile ricostruirle sul core, ma la
|
||||
distanza fra OSS e Professional è troppo grande rispetto al criterio richiesto.
|
||||
|
||||
Fonti: [licenza](https://www.jointjs.com/license),
|
||||
[confronto delle funzioni](https://www.jointjs.com/features) e
|
||||
[prezzi](https://www.jointjs.com/pricing).
|
||||
|
||||
### GoJS e yFiles
|
||||
|
||||
Sono prodotti completi e maturi, ma non open source. GoJS richiede una licenza per il deployment
|
||||
e mostra una filigrana senza chiave; yFiles usa licenze proprietarie, con evaluation temporanea e
|
||||
licenza necessaria per la distribuzione. Non soddisfano il requisito, quindi non entrano nella
|
||||
shortlist.
|
||||
|
||||
Fonti: [licensing GoJS](https://gojs.net/latest/intro/deployment.html) e
|
||||
[licensing yFiles](https://docs.yworks.com/yfiles-html/dguide/deployment/licensing.html).
|
||||
|
||||
## 6. Architettura proposta per ThothII
|
||||
|
||||
Il frontend usa React 18 e oggi `SchemaLinkingViewer.tsx` genera Mermaid con un limite di 45
|
||||
elementi. La resa corrente perde informazione: accorpa più foreign key fra la stessa coppia di
|
||||
tabelle, omette i self-reference e usa cardinalità generiche. Il catalogo espone già tabelle,
|
||||
colonne, chiavi primarie e relazioni con coppie ordinate di colonne, quindi può alimentare un
|
||||
modello più fedele.
|
||||
|
||||
Propongo questa separazione:
|
||||
|
||||
1. **Modello renderer-neutral**: `TableNode`, `ColumnPort` e `RelationEdge` con nome del
|
||||
constraint e lista ordinata delle coppie sorgente/destinazione. Non appiattire le FK composite.
|
||||
2. **Snapshot API**: un endpoint coerente, per esempio
|
||||
`GET /catalog/databases/:databaseId/schema-diagram`, che restituisca tabelle, colonne,
|
||||
relazioni e versione del catalogo in una sola lettura, evitando la richiesta colonne N+1.
|
||||
3. **Layout worker**: ELK.js in Web Worker, con porte per colonna, port constraints, route degli
|
||||
archi e cache per versione del database e filtri correnti.
|
||||
4. **Renderer interattivo**: X6 incapsulato in `DatabaseRelationshipDiagram.tsx`, affiancato
|
||||
alla vista tabellare con un toggle List/Diagram e con riuso del drawer dei dettagli.
|
||||
5. **Export**: generatore SVG separato basato sullo stesso modello e sulla geometria ELK;
|
||||
Graphviz/Viz.js può essere un fallback per layout alternativi o documentazione batch.
|
||||
|
||||
Il catalogo non registra oggi tutti i vincoli UNIQUE né il tipo MATCH delle FK. Senza questi
|
||||
dati non sempre è possibile distinguere correttamente 1:1 da 1:N. Il renderer non deve inventare
|
||||
la cardinalità: deve mostrare una notazione neutra finché il metadato necessario non viene
|
||||
raccolto.
|
||||
|
||||
### Funzioni necessarie per schemi complessi
|
||||
|
||||
- semantic zoom: solo nomi tabella, poi PK/FK, infine tutti i campi;
|
||||
- ricerca e filtri per schema, tabella, colonna e tipo di relazione;
|
||||
- modalità focus con vicinato a uno o due hop;
|
||||
- evidenziazione della relazione e delle due colonne al passaggio/selezione;
|
||||
- minimappa, fit selection, navigazione da tastiera e pannello dettagli;
|
||||
- distinzione visiva di PK, FK, nullable, composite key, self-loop e relazioni parallele;
|
||||
- layout automatico ricalcolabile, ma posizioni manuali persistibili;
|
||||
- rendering progressivo o virtuale e fallback tabellare sempre disponibile;
|
||||
- export dell'intero schema e dell'area filtrata in SVG/PNG.
|
||||
|
||||
## 7. Proof of concept consigliato
|
||||
|
||||
Un POC breve dovrebbe usare **X6 + ELK.js** su tre dataset sintetici: circa 30, 150 e 500
|
||||
tabelle, includendo FK composite, più FK tra la stessa coppia, self-loop, tabelle isolate e hub
|
||||
ad alto grado.
|
||||
|
||||
I criteri di accettazione dovrebbero misurare:
|
||||
|
||||
- tempo di layout nel worker e tempo al primo frame interattivo;
|
||||
- fluidità di pan/zoom e uso memoria sul caso da 500 tabelle;
|
||||
- correttezza di porte, archi paralleli, self-loop e FK composite;
|
||||
- leggibilità nelle tre soglie di semantic zoom;
|
||||
- navigazione completa da tastiera e comportamento con screen reader;
|
||||
- qualità e portabilità dell'SVG esportato;
|
||||
- assenza di codice, esempi o componenti soggetti a licenza commerciale.
|
||||
|
||||
Se X6 non raggiunge il livello di accessibilità richiesto senza un costo eccessivo, il confronto
|
||||
finale va fatto con React Flow + lo stesso adapter ELK e lo stesso modello dati. In questo modo
|
||||
si cambia renderer senza rifare API, semantica delle relazioni o pipeline di export.
|
||||
@@ -0,0 +1,246 @@
|
||||
# Gestione delle relationship: da ThothAI a ThothII
|
||||
|
||||
**Stato:** analisi e direzione funzionale/UX confermate nel *grill with docs*; non costituisce ancora un piano di implementazione.
|
||||
**Revisione esaminata:** commit ThothII `f586152636b1fd653b0bc1d40b54be7f89bbd2bb`. I sorgenti legacy di ThothAI citati sotto sono versionati nello stesso repository, sotto `Thoth/ThothAI/`.
|
||||
**Ambito:** import delle foreign key fisiche, inferenza di relationship logiche, modifica umana, pubblicazione verso il workflow NL→SQL e principali gap tra i due sistemi.
|
||||
|
||||
## Sintesi fattuale
|
||||
|
||||
1. In ThothAI le foreign key dichiarate nel database venivano importate e una procedura separata proponeva relationship basate sul nome dei campi e su una verifica dei valori. Entrambi i percorsi scrivevano però nello stesso modello `Relationship` (`Thoth/ThothAI/backend/thoth_core/dbmanagement.py:484-640`; `Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:571-826`; `Thoth/ThothAI/backend/thoth_core/models.py:488-530`).
|
||||
2. Il ricordo di una relationship inferita persistita come `generated` non trova riscontro nel modello esaminato: `Relationship` contiene solo quattro foreign key verso tabelle e colonne, senza provenienza, stato, confidenza o flag `generated`. La procedura di inferenza calcola localmente pattern e tasso di validazione, ma non li salva (`Thoth/ThothAI/backend/thoth_core/models.py:488-530`; `Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-801`).
|
||||
3. L'amministratore Django di ThothAI permetteva CRUD manuale sul medesimo insieme di relationship e verificava che gli endpoint appartenessero allo stesso database e alle tabelle selezionate; non distingueva visivamente o semanticamente relationship fisiche, inferite e manuali (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:19-313`).
|
||||
4. ThothII oggi separa già i due concetti: il catalogo backend conserva solo le foreign key fisiche dichiarate, mentre il core/harness possiede annotazioni di foreign key logiche curate e un comando che suggerisce candidate per nome, primary key e SQL osservato (`CONTEXT.md:291-300`; `docs/adr/0006-separate-physical-and-logical-relationships.md:3-8`; `harness/tht/cli/schema_cmd.py:203-299`).
|
||||
5. I due mondi ThothII non sono ancora integrati: i record di Database Management non modificano il workflow NL→SQL e l'integrazione catalogo→core/schema-linking è ancora un gate di design esplicitamente differito (`PROJECT_STATE.md:116-124`; `PROJECT_STATE.md:161-165`).
|
||||
|
||||
## Evidenze ThothAI
|
||||
|
||||
### Foreign key ufficiali
|
||||
|
||||
Il percorso di import legge le foreign key dal database, limita l'import alle tabelle già presenti nel catalogo, crea le colonne mancanti, crea o riusa un record `Relationship` e aggiorna anche le stringhe denormalizzate `pk_field`/`fk_field` sulle colonne (`Thoth/ThothAI/backend/thoth_core/dbmanagement.py:484-640`, in particolare `:501-513`, `:526-591` e `:606-607`).
|
||||
|
||||
**Fatto:** una foreign key fisica diventa quindi un record applicativo, non rimane soltanto un fatto letto al momento dal database.
|
||||
|
||||
### Relationship inferite
|
||||
|
||||
La routine legacy dichiara sei famiglie di confronto tra il nome della colonna candidata e quello della primary key: corrispondenza esatta, snake case, kebab case, camel case, concatenazione e solo nome tabella (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:571-689`).
|
||||
|
||||
Per una candidata, la routine:
|
||||
|
||||
- legge fino a 20 valori distinti e non nulli dalla colonna candidata;
|
||||
- verifica ogni valore contro la primary key bersaglio;
|
||||
- accetta la candidata quando almeno il 70% dei valori esaminati trova riscontro;
|
||||
- crea o recupera un normale `Relationship` e aggiorna i campi denormalizzati delle tabelle (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-801`).
|
||||
|
||||
**Correzione documentale:** un commento parla di campionamento casuale, ma la query mostrata non contiene un ordinamento casuale; il comportamento verificabile dal codice è “fino a 20 valori distinti e non nulli”, non un campione statisticamente casuale (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:697-716`).
|
||||
|
||||
**Rischio legacy:** nomi di tabelle e colonne sono interpolati direttamente in SQL in questo percorso. Portare la logica letteralmente in ThothII riprodurrebbe un problema di quoting/sicurezza e richiederebbe inoltre una policy esplicita per l'accesso ai valori del DWH (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:709-750`).
|
||||
|
||||
### Un solo modello per tre origini
|
||||
|
||||
Il modello `Relationship` legacy contiene soltanto `source_table`, `target_table`, `source_column` e `target_column`, più metodi di rappresentazione/aggiornamento. Non contiene campi per origine, algoritmo, evidenza, confidenza, approvazione o disabilitazione, né un vincolo di unicità dichiarato nel modello (`Thoth/ThothAI/backend/thoth_core/models.py:488-530`).
|
||||
|
||||
L'admin consente aggiunta, modifica e cancellazione ordinarie e valida la coerenza tra database, tabelle e colonne (`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:30-38`, `:125-157`, `:218-245`).
|
||||
|
||||
**Conclusione fattuale:** relationship importate, inferite e create manualmente convergono nello stesso tipo persistito. Dal record finale non è possibile ricostruirne con certezza l'origine. Il termine `generated`, se usato nell'interfaccia o nel linguaggio operativo dell'epoca, non era una qualificazione persistita dal modello esaminato.
|
||||
|
||||
### Uso nella comprensione dello schema
|
||||
|
||||
La generazione M-Schema conserva colonne PK/FK e ricostruisce la sezione `【Foreign keys】` analizzando le stringhe denormalizzate `fk_field` (`Thoth/ThothAI/frontend/sql_generator/helpers/main_helpers/main_generate_mschema.py:25-61`, `:94-101`, `:160-187`).
|
||||
|
||||
**Fatto:** le relationship curate nel catalogo legacy influenzavano la rappresentazione dello schema consumata dal generatore SQL, anche se tramite una proiezione denormalizzata.
|
||||
|
||||
## Stato attuale di ThothII
|
||||
|
||||
### Catalogo backend: relationship fisiche
|
||||
|
||||
Il modello di dominio corrente distingue esplicitamente:
|
||||
|
||||
- **Physical Relationship:** vincolo dichiarato nel database;
|
||||
- **Catalog Relationship:** copia persistita di quel fatto fisico, non creabile o modificabile manualmente ma eliminabile per cleanup;
|
||||
- **Logical Relationship:** relazione semantica curata o inferita, con ownership e lifecycle separati (`CONTEXT.md:291-300`; `docs/adr/0006-separate-physical-and-logical-relationships.md:3-8`).
|
||||
|
||||
La migrazione del catalogo crea `catalog_relationships` e le coppie ordinate in `catalog_relationship_columns`. L'identità della relazione fisica è `(source_table_id, constraint_name)`; non esistono campi di origine, stato o confidenza (`backend/src/catalog/migrations/003_catalog_schema_sync.ts:42-74`). Il tipo TypeScript è solo strutturale e le descrizioni generate riguardano esclusivamente tabelle o colonne (`backend/src/catalog/types.ts:123-152`).
|
||||
|
||||
L'introspezione PostgreSQL legge soltanto `pg_constraint` con `contype = 'f'` e mantiene l'ordine delle coppie per le chiavi composite (`backend/src/catalog/schema-introspector.ts:110-171`; `backend/src/catalog/schema-introspector.ts:360-416`; `docs/contracts/catalog-schema-snapshot.md:40-53`). La sincronizzazione autoritativa crea, aggiorna o elimina le copie fisiche in base allo snapshot (`backend/src/catalog/repository.ts:1165-1218`; `backend/src/catalog/repository.ts:1316-1393`).
|
||||
|
||||
Le API e la UI espongono lettura, sync e cleanup in massa, non CRUD logico per singola relationship (`backend/src/routes/catalog-schema.ts:25-37`; `backend/src/routes/catalog-schema.ts:134-150`; `frontend/src/api/catalog-databases.ts:452-489`; `frontend/src/shell/database-management/DatabaseRelationships.tsx:114-123`; `frontend/src/shell/database-management/DatabaseRelationships.tsx:160-225`).
|
||||
|
||||
**Effetto rilevante:** il cleanup è intenzionalmente reversibile tramite una sync successiva e può lasciare temporaneamente un catalogo incompleto (`docs/adr/0008-allow-manual-catalog-metadata-cleanup.md:7-16`). Un test di integrazione ammette anche una relationship rimasta senza coppie di colonne dopo la cancellazione delle colonne catalogate (`backend/test/catalog-repository.integration.test.ts:194-264`, in particolare `:246-252`). Un futuro consumer non può quindi assumere che ogni stato intermedio del catalogo sia pubblicabile così com'è.
|
||||
|
||||
### Core/harness: relationship logiche e suggerimenti
|
||||
|
||||
Il modello M-Schema del core possiede già `TableAnnotation.foreign_keys`, descritte come foreign key logiche curate e unite alle foreign key fisiche (`harness/tht/mschema/models.py:35-39`; `harness/tht/mschema/models.py:76-84`). Il renderer fonde i due insiemi e li presenta insieme nella sezione `【Foreign keys】` (`harness/tht/mschema/render.py:9-20`; `harness/tht/mschema/render.py:46-80`).
|
||||
|
||||
Il comando `schema suggest-fks` costruisce candidate da:
|
||||
|
||||
- uguaglianze trovate in SQL, quando esattamente un lato è una primary key;
|
||||
- convenzione speciale `*time_key → dim_time.<PK singola>`;
|
||||
- stesso nome tra colonna e primary key a proprietario univoco;
|
||||
- assunzioni esplicite per disambiguare;
|
||||
- esclusione di nomi PK generici come `id`, `key` e `code` e dei proprietari ambigui (`harness/tht/cli/schema_cmd.py:203-299`; `harness/tht/mschema/fkmine.py:1-58`).
|
||||
|
||||
Il comando può produrre un documento candidato e, con `--write`, aggiungere annotazioni mancanti in modo idempotente; il messaggio stesso chiede revisione manuale (`harness/tht/cli/schema_cmd.py:406-429`; `harness/tht/cli/schema_cmd.py:519-540`; `harness/tests/test_schema_fk_annotations.py:131-146`; `harness/tests/test_schema_fk_annotations.py:454-470`).
|
||||
|
||||
**Fatto:** ThothII non parte da zero sull'inferenza. Possiede già un motore più conservativo, basato su PK univoche e SQL osservato, ma non conserva per ogni relazione origine, evidenza, frequenza o confidenza. Il miner conta le occorrenze internamente, ma il modello candidato non promuove quel conteggio a lifecycle persistito (`harness/tht/mschema/fkmine.py:1-58`; `harness/tht/mschema/models.py:35-39`).
|
||||
|
||||
**Rischi strutturali già visibili:**
|
||||
|
||||
- `ForeignKey` ammette liste di colonne, ma non valida che source e target abbiano la stessa cardinalità; il renderer usa `zip`, quindi una relazione malformata può essere troncata silenziosamente (`harness/tht/mschema/models.py:35-39`; `harness/tht/mschema/render.py:76-78`).
|
||||
- il merge evita duplicati rispetto alle FK fisiche iniziali, ma non aggiorna l'insieme `seen` dopo aver aggiunto un'annotazione; due annotazioni logiche uguali possono sopravvivere al merge (`harness/tht/mschema/render.py:9-20`).
|
||||
- il catalogo fisico supporta coppie composite ordinate, mentre le euristiche correnti e legacy sono sostanzialmente unary. La semantica delle candidate composite resta da decidere (`backend/src/catalog/migrations/003_catalog_schema_sync.ts:56-74`; `harness/tht/cli/schema_cmd.py:203-299`).
|
||||
|
||||
### Revisione e pubblicazione correnti
|
||||
|
||||
Le annotazioni canoniche del workspace sono un blob Git. Il flusso pubblico produce candidate, verifica le annotazioni e richiede un'accettazione umana esplicita dopo commit/push/pull; l'accettazione registra digest del candidato e delle annotazioni, revisione e blob (`docs/contracts/workspace-preprocessing-cli.md:89-106`; `backend/src/workspaces/preprocessing-service.ts:202-297`). Il preprocessing successivo procede solo se il digest accettato coincide con le annotazioni correnti (`backend/src/workspaces/preprocessing-service.ts:335-388`).
|
||||
|
||||
**Limite fattuale:** il record di review conserva digest, revisione e blob, ma non attore, motivazione o decisioni per singola candidata (`backend/src/workspaces/preprocessing-state.ts:67-73`).
|
||||
|
||||
Il runtime usa le annotazioni quando renderizza lo schema, ma la ricerca vettoriale indicizza record di tabelle e colonne senza contenuto esplicito delle relationship (`harness/tht/vectorstore/records.py:106-138`; `harness/tht/cli/search_cmd.py:233-273`). Inoltre la vista colonne usata in F4 continua a leggere i commenti fisici, non le annotazioni (`harness/tht/cli/schema_cmd.py:592-621`).
|
||||
|
||||
La review dei join durante una sessione è distinta dalla curatela globale: il reviewer conferma l'insieme dei join oppure chiede una revisione completa; non modifica la mappa canonica delle relationship (`harness/.pi/skills/tht-sessione/SKILL.md:301-341`; `frontend/src/widgets/JoinReviewWidget.tsx:5-84`). Il modello di sessione registra join come due stringhe e una decisione opzionale, senza ID stabile della relationship o coppie ordinate strutturate (`harness/tht/session/models.py:147-170`).
|
||||
|
||||
## Delta e rischi da sottoporre al grill
|
||||
|
||||
| Tema | Fatto documentato | Delta/rischio aperto |
|
||||
|---|---|---|
|
||||
| Origine | ThothAI perdeva l'origine; ThothII separa fisico e logico a livello concettuale | Decidere quale provenienza debba essere persistita per manuale, euristica, SQL osservato e import fisico |
|
||||
| `generated` | Non era un flag del modello ThothAI; in ThothII “Generated Description” è già un termine del catalogo (`CONTEXT.md:302-311`) | Usare `generated` anche per relationship potrebbe creare ambiguità terminologica |
|
||||
| Cancellazione utente | In ThothAI era CRUD sul record unico; in ThothII il cleanup fisico viene ricostruito dalla sync | “Eliminare” può significare cancellare una relazione logica, sopprimere un fatto fisico per il core oppure pulire temporaneamente la copia catalogata: sono operazioni diverse |
|
||||
| Inferenza | ThothAI usava sei pattern e valori DWH; ThothII usa PK univoche, nomi e SQL osservato | Stabilire se sostituire, integrare o non portare il campionamento dei valori; servono policy di dati sensibili, query read-only, quoting e limiti |
|
||||
| Approvazione | ThothII dispone di review Git/digest dell'intero artefatto | Mancano decisioni per candidata, motivazione, attore, sticky rejection, versione algoritmo e gestione dello stale |
|
||||
| Compositi | Il catalogo fisico conserva coppie ordinate; le annotazioni accettano liste | Mancano invarianti forti e una strategia di inferenza/review per join compositi |
|
||||
| Pubblicazione | Catalogo management e runtime core sono oggi separati | Va stabilito se pubblicare tutto, una selezione esplicita o una revisione immutabile; il catalogo può essere incompleto durante cleanup/sync |
|
||||
| UI e ownership | UI catalogo fisico read-only; curatela logica via CLI/Git; review join per sessione separata | Va scelto chi cura la mappa e in quale superficie, senza confondere amministrazione globale e correzione della singola sessione |
|
||||
| Retrieval | Le relationship entrano nel render M-Schema ma non nei record vettoriali | Va deciso se e come influenzano selezione tabelle, ranking e descrizioni, oltre al rendering finale |
|
||||
| Drift | Sync fisica è autoritativa; annotazioni sono una revisione separata | Servono semantiche per endpoint rinominati/eliminati, candidate stale, orphan e riapprovazione |
|
||||
|
||||
Le analisi già presenti nel repository trattano l'integrazione catalogo→core, la selezione pubblicabile e la riparazione degli indici come questioni ancora aperte; le loro proposte non sono decisioni implementate (`docs/research/2026-08-23-thothii-metadata-publication-qdrant-seams.md:20-43`; `docs/research/2026-08-23-thothii-metadata-publication-qdrant-seams.md:239-301`). Anche il piano di migrazione rinvia il lifecycle/admin delle relationship logiche (`docs/plans/2026-08-26-metadata-catalog-from-thothai.md:681-691`).
|
||||
|
||||
## Direzione confermata nel grill
|
||||
|
||||
Il 31 agosto 2026 è stato concordato il seguente flusso minimo:
|
||||
|
||||
1. Le foreign key dichiarate continuano a essere sincronizzate come Catalog Relationship fisiche.
|
||||
2. Le foreign key ipotetiche sono salvate una sola volta come Logical Relationship fra colonna
|
||||
sorgente e colonna destinazione; le tabelle sono ricavate dalle colonne e la UI le presenta nel
|
||||
relativo contesto tabella.
|
||||
3. Una relationship inferita è marcata `generated`; una relationship aggiunta dall'utente non lo è.
|
||||
4. La cancellazione logica conserva il record e impedisce a una ricostruzione di riattivarlo.
|
||||
5. La cancellazione fisica rimuove il record; una ricostruzione può ricrearlo se viene nuovamente
|
||||
inferito.
|
||||
6. La ricostruzione è additiva: conserva le relationship attive già presenti, non rimuove quelle
|
||||
non più inferibili e non modifica le relationship manuali.
|
||||
7. L'inferenza usa nomi, primary key e compatibilità dei tipi. Non campiona valori del DWH.
|
||||
8. La relationship è l'unica fonte di verità: non viene duplicata in stringhe `fk_field` sulle
|
||||
colonne.
|
||||
9. I nomi vengono confrontati senza distinzione fra maiuscole/minuscole e normalizzando snake case,
|
||||
kebab case e camel case. La regola riconosce anche casi come `user_id → users.id`, richiede tipi
|
||||
compatibili e una sola destinazione possibile; riconosce inoltre un nome PK non generico con un
|
||||
unico proprietario e la convenzione `*time_key → dim_time.<PK singola>`. Le colonne sorgenti
|
||||
possono appartenere a PK composite, mentre nomi generici isolati come `id`, `key`, `code` e `pk`
|
||||
non costituiscono evidenza. I casi ambigui vengono ignorati e non si usa fuzzy matching o un LLM.
|
||||
10. La ricostruzione parte da un'azione amministrativa esplicita `Rebuild generated relationships`,
|
||||
separata dalla sincronizzazione dello schema.
|
||||
11. Le relationship cancellate logicamente restano consultabili tramite filtro e possono essere
|
||||
riattivate con un'azione `Restore`.
|
||||
|
||||
Non restano decisioni di dominio aperte per il flusso minimo. La progettazione UX e il seam tecnico
|
||||
sono descritti nelle sezioni seguenti.
|
||||
|
||||
## Lacuna UX emersa nel grill
|
||||
|
||||
La vista Fleet `Relationships` esiste, ma nella UI di produzione non ha oggi un punto di ingresso
|
||||
raggiungibile. La riga del database espone sincronizzazione, tabelle, dettagli, modifica e rimozione,
|
||||
ma non le relationship; inoltre i tab `Overview / Tables / Relationships` appartengono soltanto alla
|
||||
presentazione legacy. Il test di navigazione esistente esercita anch'esso la presentazione legacy,
|
||||
non quella Fleet (`frontend/src/shell/database-management/DatabaseGrid.tsx:122-180`;
|
||||
`frontend/src/shell/database-management/DatabaseForm.tsx:442-475`;
|
||||
`frontend/src/shell/database-management/DatabaseForm.tsx:517-546`;
|
||||
`frontend/src/shell/DatabaseManagementPage.test.tsx:2635-2703`).
|
||||
|
||||
Se aperta programmaticamente, la vista mostra soltanto `Physical relationships` in sola lettura. La
|
||||
toolbar contiene ricerca, conteggio, `Refresh`, un selettore azione con esecuzione esplicita e
|
||||
`Sync history`; la griglia offre unicamente `Details`, che apre il drawer della relationship fisica.
|
||||
Sono assenti ingresso visibile, aggiunta manuale, ricostruzione delle generated relationship, origine,
|
||||
stato attivo/cancellato, cancellazione logica, cancellazione fisica e ripristino
|
||||
(`frontend/src/shell/database-management/DatabaseRelationships.tsx:114-225`).
|
||||
|
||||
I pattern Fleet già consolidati da riutilizzare sono:
|
||||
|
||||
- una sola griglia nel livello corrente, con breadcrumb e controllo Back;
|
||||
- azioni di pagina nel selettore `Choose an action…` con `Run action` e motivo visibile quando
|
||||
indisponibili;
|
||||
- azioni della singola riga nella colonna finale fissata a destra;
|
||||
- form e dettagli in un drawer modeless che restituisce il focus al controllo di origine;
|
||||
- conferme distruttive inline o nel drawer, non tramite una nuova pagina;
|
||||
- toast per accettazione o errore e feedback persistente soltanto per le operazioni lunghe;
|
||||
- card di errore con Retry ed empty state che indica la prossima azione possibile.
|
||||
|
||||
ThothAI non offre un modello UX da copiare. L'inferenza è nascosta fra 21 bulk action della lista
|
||||
database, non mostra avanzamento in tempo reale e restituisce soltanto messaggi a fine richiesta. Il
|
||||
CRUD manuale vive in un'altra schermata Django Admin, non distingue origine o stato, offre soltanto
|
||||
la cancellazione fisica e il form di aggiunta ha una validazione server strutturalmente incoerente
|
||||
con le select popolate dal browser
|
||||
(`Thoth/ThothAI/backend/thoth_core/admin_models/admin_sqldb.py:252-274`;
|
||||
`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:58-119`;
|
||||
`Thoth/ThothAI/backend/thoth_core/admin_models/admin_relationship.py:218-313`).
|
||||
|
||||
## UX confermata
|
||||
|
||||
Il 31 agosto 2026 sono state confermate le seguenti scelte:
|
||||
|
||||
1. La colonna Actions della riga database espone un accesso diretto `Relationships`, accanto a
|
||||
`Tables`. La vista conserva breadcrumb e controllo `Back to databases` esistenti.
|
||||
2. La pagina presenta una sola griglia `Relationship map`, contenente relationship `Physical`,
|
||||
`Generated` e `Manual`. Le colonne sono Source table, Source column, Target table, Target column,
|
||||
Origin, Status e Actions. Constraint e regole update/delete rimangono nel drawer delle FK fisiche.
|
||||
3. Un filtro visibile seleziona `Active`, `Excluded` o `All`; il valore predefinito è `Active`. Le FK
|
||||
fisiche sono consultabili ma non modificabili da questa vista.
|
||||
4. `Add relationship` è il pulsante primario visibile nella toolbar. Apre il drawer standard con i
|
||||
quattro campi Source table, Source column, Target table e Target column e i comandi `Cancel` e
|
||||
`Add relationship`. Il flusso minimo gestisce una sola coppia di colonne e mostra la validazione
|
||||
accanto al campo interessato.
|
||||
5. `Rebuild generated relationships` entra nell'attuale selettore `Choose an action…`, insieme a
|
||||
`Synchronize physical relationships` e `Synchronize full schema`, con esecuzione esplicita tramite
|
||||
`Run action`. `Refresh` ricarica la griglia; `Sync history` resta riservato alla sincronizzazione
|
||||
fisica.
|
||||
6. Il drawer di dettaglio contiene le azioni sulle relationship logiche. `Exclude` realizza la
|
||||
cancellazione logica, `Delete permanently` quella fisica e `Restore` riattiva una relationship
|
||||
esclusa. La conferma avviene nel drawer e spiega rispettivamente che la ricostruzione non
|
||||
riattiverà un record escluso e potrà invece ricreare un record eliminato definitivamente.
|
||||
7. La ricostruzione non introduce un nuovo sistema di job o di storico. Durante l'esecuzione mostra
|
||||
`Rebuilding…`; al termine un toast riporta added, already present, excluded e ambiguous. Add,
|
||||
Exclude, Delete permanently e Restore producono toast specifici; gli errori mantengono aperto il
|
||||
contesto corrente. L'empty state propone di ricostruire dai nomi o aggiungere manualmente.
|
||||
8. L'inferenza non usa AI. È codice deterministico nel backend del catalogo, basato su normalizzazione
|
||||
dei nomi, primary key/unicità, compatibilità dei tipi e assenza di ambiguità. Non usa LLM,
|
||||
embedding o campionamento dei dati. L'AI consuma la mappa risultante per comprendere lo schema,
|
||||
ma non la costruisce.
|
||||
|
||||
## Seam tecnico confermato
|
||||
|
||||
Il Catalog PostgreSQL è l'unica fonte modificabile delle Logical Relationship. Le relationship
|
||||
fisiche e logiche restano in modelli distinti, coerentemente con ADR-0006, mentre un servizio profondo
|
||||
espone a API e UI una sola mappa discriminata per origine e stato.
|
||||
|
||||
All'avvio o alla ripresa di una sessione, il backend materializza le sole relationship attive in una
|
||||
snapshot JSON canonica e immutabile, collegata alla stessa lease della configurazione runtime. La
|
||||
snapshot comprende anche le FK fisiche e conserva l'ordine delle coppie composite. Se due record hanno
|
||||
gli stessi endpoint, la FK fisica ha precedenza.
|
||||
|
||||
Quando la snapshot è presente, l'harness la usa come fonte esclusiva delle relationship e continua a
|
||||
leggere dalle annotazioni Git-pinned soltanto descrizioni, sinonimi, concetti e altri metadati. Non
|
||||
scrive `annotations.yaml` e non interroga direttamente il Catalog. Una snapshot dichiarata ma assente,
|
||||
invalida o incoerente con lo schema fisico fallisce esplicitamente; un runtime legacy che non dichiara
|
||||
la snapshot mantiene il precedente comportamento di compatibilità.
|
||||
|
||||
Questa proiezione non è un secondo store: non può essere modificata, viene eliminata insieme alla
|
||||
configurazione runtime e una sessione Pi vede una mappa stabile per tutta la propria vita. La decisione
|
||||
duratura è registrata in ADR-0012.
|
||||
|
||||
La proiezione e la ricostruzione sono ammesse soltanto dopo una sincronizzazione completa della
|
||||
versione corrente del database e vengono serializzate con le mutazioni del catalogo. Un catalogo mai
|
||||
sincronizzato, reso stale da una modifica della configurazione o invalidato da metadata cleanup non
|
||||
può quindi diventare accidentalmente la fonte esclusiva del runtime. Il cleanup esplicito di una
|
||||
tabella o colonna endpoint è anche il confine distruttivo del tombstone: rimuove definitivamente la
|
||||
relationship esclusa, perché conservarla richiederebbe una seconda identità testuale denormalizzata.
|
||||
@@ -0,0 +1,428 @@
|
||||
# Piano di test: metadati di schema, campi sensibili e descrizioni AI
|
||||
|
||||
- Data: 2026-08-31
|
||||
- Stato: proposto
|
||||
- Baseline analizzata: `f586152` sul branch `test/database-baseline`
|
||||
|
||||
## 1. Obiettivo
|
||||
|
||||
Validare insieme le tre capacità recentemente introdotte in Database Management:
|
||||
|
||||
1. acquisizione autorevole dei metadati fisici di uno schema esterno;
|
||||
2. proposta e revisione umana dei campi sensibili dal punto di vista privacy;
|
||||
3. generazione, revisione e consolidamento delle descrizioni di tabelle e colonne.
|
||||
|
||||
Il piano è risk-based: perdita o corruzione di metadati, lettura o invio di valori protetti e
|
||||
applicazione di una selezione al target sbagliato sono rischi bloccanti. La qualità linguistica dei
|
||||
testi AI è invece valutata separatamente dal contratto tecnico, perché l'output del modello è una
|
||||
proposta soggetta a revisione umana.
|
||||
|
||||
Questo documento non costituisce una certificazione normativa o GDPR: verifica i controlli tecnici
|
||||
e il flusso operativo implementati dal prodotto.
|
||||
|
||||
### 1.1 Assunzione operativa: ambiente completamente sacrificabile
|
||||
|
||||
Per indicazione esplicita del proprietario, l'installazione sul Mac è esclusivamente di sviluppo e
|
||||
test. Non contiene produzione e non esiste alcun requisito di conservazione dello stato locale.
|
||||
|
||||
- `catalog-db`, migrazioni, configurazioni locali, run, eventi, flag, descrizioni curate e generate
|
||||
possono essere cancellati e ricreati tutte le volte necessarie;
|
||||
- il database reale già collegato è la fonte autorevole dalla quale ricostruire il catalogo ed è la
|
||||
sorgente primaria per validare schema, classificazione privacy e generazione delle descrizioni;
|
||||
- reset completi, failure injection, dati incoerenti deliberati e prove distruttive sul catalogo
|
||||
locale sono ammessi senza backup o procedura di rollback dell'ambiente;
|
||||
- compatibilità con stato locale preesistente, sessioni legacy e vecchie revisioni del catalogo non
|
||||
è un gate di questo piano;
|
||||
- le prove di atomicità, idempotenza e preservazione dei campi restano necessarie perché verificano
|
||||
il comportamento del prodotto dentro un ciclo di test, non perché debbano proteggere il Mac.
|
||||
|
||||
Il database sorgente resta normalmente read-only: quando una prova richiede DDL o mutazioni fra
|
||||
scan e conferma si usa uno schema di test esplicitamente scrivibile o la fixture supplementare. La
|
||||
disponibilità a buttare lo stato locale non elimina il rischio di inviare dati personali a un
|
||||
provider esterno: proprio questa non-esfiltrazione è uno degli esiti principali da collaudare.
|
||||
|
||||
## 2. Terminologia e risultato atteso
|
||||
|
||||
I tre campi testuali del catalogo hanno autorità diverse e non devono essere confusi:
|
||||
|
||||
| Campo | Origine e autorità | Comportamento atteso |
|
||||
| --- | --- | --- |
|
||||
| `sourceComment` | Commento fisico acquisito dalla sorgente | Si aggiorna con la sincronizzazione e non è modificabile come contenuto curato. |
|
||||
| `generatedDescription` | Proposta prodotta dall'AI o corretta nel catalogo | È salvata separatamente, può essere rigenerata esplicitamente e non sovrascrive gli altri due campi. |
|
||||
| `description` | Descrizione curata dall'amministratore | Cambia solo tramite modifica esplicita o consolidamento di una proposta selezionata. |
|
||||
|
||||
Nel seguito, “commento generato” indica `generatedDescription`. La generazione non deve mai
|
||||
modificare `sourceComment`; il passaggio a `description` avviene solo con il consolidamento umano.
|
||||
|
||||
## 3. Riferimenti e baseline
|
||||
|
||||
Il comportamento da verificare deriva da:
|
||||
|
||||
- stato corrente del progetto in `PROJECT_STATE.md`;
|
||||
- [contratto Catalog Schema Snapshot](../contracts/catalog-schema-snapshot.md);
|
||||
- [piano del Metadata Catalog](../plans/2026-08-26-metadata-catalog-from-thothai.md);
|
||||
- [specifica della generazione descrizioni](../plans/2026-08-28-ai-catalog-description-generation-spec.md);
|
||||
- [ADR 0007: sincronizzazione autorevole durevole](../adr/0007-durable-authoritative-schema-synchronization.md);
|
||||
- [ADR 0009: un solo run sequenziale di generazione](../adr/0009-use-one-sequential-description-generation-run.md);
|
||||
- [ADR 0010: campioni reali limitati](../adr/0010-allow-bounded-real-source-samples-for-description-generation.md);
|
||||
- [ADR 0011: Sensitive Data Flag](../adr/0011-gate-source-samples-with-a-sensitive-data-flag.md);
|
||||
- [accettazione AI del 2026-08-29](./2026-08-29-ai-catalog-description-generation-acceptance.md).
|
||||
|
||||
L'accettazione del 2026-08-29 è una baseline utile ma non chiude il gate attuale: precede i commit
|
||||
`0736983` e `cb40c09`, inviava campioni reali inventati e documentava che un valore campione era
|
||||
stato ripreso nella descrizione. Deve quindi essere ripetuta sul comportamento privacy corrente.
|
||||
|
||||
## 4. Perimetro
|
||||
|
||||
### Incluso
|
||||
|
||||
- trasporti `postgres_direct`, `rest_api` e `ssh_tunnel`;
|
||||
- RPC REST tipizzato `POST /rpc/schema_snapshot` e fallback singolo read-only su
|
||||
`POST /rpc/run_query`;
|
||||
- metadati di tabelle, colonne, commenti sorgente, tipi, default, nullabilità, posizioni PK e coppie
|
||||
FK ordinate;
|
||||
- scope di sincronizzazione `tables`, `columns`, `relationships` e `all`;
|
||||
- run durevoli, conferma delle differenze distruttive, cancellazione, recovery, eventi SSE e
|
||||
fallback polling;
|
||||
- Sensitive Data Flag, suggerimenti AI strutturali, review draft e storico operativo;
|
||||
- scope di generazione `selected_columns`, `selected_tables`, `all` e `missing`;
|
||||
- campionamento read-only, valori sintetici per colonne protette, batching, retry, stop, Unlock,
|
||||
storico e consolidamento;
|
||||
- uso controllato del database reale già collegato, inclusi schema e valori non sensibili, per il
|
||||
collaudo end-to-end con fake provider e provider configurati;
|
||||
- permessi, minimizzazione dei dati, redazione di log/errori e resistenza a input ostili.
|
||||
|
||||
### Escluso o rinviato
|
||||
|
||||
- applicazione del Sensitive Data Flag allo schema-linking/LSH del core, esplicitamente rinviata;
|
||||
- sostituzione di `annotations.yaml`, pubblicazione Qdrant e cutover del runtime NL→SQL;
|
||||
- alias, sinonimi, concetti, value descriptions e relazioni logiche;
|
||||
- audit delle decisioni umane sul flag: per disegno si conserva solo il booleano corrente;
|
||||
- DDL o scritture sul database sorgente;
|
||||
- conservazione di cataloghi, run, descrizioni o sessioni locali precedenti al reset;
|
||||
- compatibilità all'indietro con formati o migrazioni legacy non appartenenti alla baseline corrente.
|
||||
|
||||
## 5. Priorità e strategia
|
||||
|
||||
| Priorità | Significato | Esempi |
|
||||
| --- | --- | --- |
|
||||
| P0 | Gate bloccante di sicurezza o integrità | Nessuna lettura/invio di valori protetti; applicazione atomica; scope esatto; segreti non esposti. |
|
||||
| P1 | Contratto funzionale necessario al rilascio | Trasporti, stati dei run, recovery, batching, review e consolidamento. |
|
||||
| P2 | Qualità, UX e robustezza non distruttiva | Copy, focus, benchmark del modello, carico e degrado SSE. |
|
||||
|
||||
Le prove sono distribuite su quattro livelli:
|
||||
|
||||
1. **Contratto/unità**: repository in memoria, finti connector e Model Completer; nessuna rete.
|
||||
2. **Integrazione**: catalogo PostgreSQL ricreabile via Docker, database reale come sorgente,
|
||||
fixture supplementare, fake REST/SSH e Model Completer osservabile.
|
||||
3. **UI/E2E locale**: component test con MSW e almeno un flusso Playwright sullo stack locale.
|
||||
4. **Accettazione L2**: provider configurato realmente e database reale dopo review dei flag.
|
||||
|
||||
I test deterministici verificano il contratto. Le prove con un modello reale verificano
|
||||
compatibilità e qualità, ma non sostituiscono i gate P0.
|
||||
|
||||
## 6. Ambiente e dati di test
|
||||
|
||||
### 6.1 Ambiente minimo
|
||||
|
||||
- stack locale con `catalog-db` eliminabile, migrazioni ripetibili e backend/frontend della stessa
|
||||
baseline;
|
||||
- database reale già collegato come sorgente primaria, con utenza capace di `SELECT` ma non di
|
||||
`INSERT`, `UPDATE`, `DELETE`, DDL o cambio di schema;
|
||||
- PostgreSQL effimero supplementare solo per le mutazioni controllate che non devono essere fatte
|
||||
sul database reale;
|
||||
- server REST fake in grado di servire snapshot valido, capability `unavailable`, 404, risposta
|
||||
parziale/malformata e fallback `run_query`;
|
||||
- server OpenSSH effimero con `known_hosts` esatto e varianti host key errata/assente;
|
||||
- Model Completer fake che registra transitoriamente i messaggi e restituisce esiti programmabili;
|
||||
- browser senza segreti in Web Storage e raccolta di log backend/SSE/API per le scansioni canary.
|
||||
|
||||
Le prove PostgreSQL di integrazione non devono risultare `skip`: la disponibilità di Docker è una
|
||||
precondizione del gate. Il catalogo locale può essere azzerato prima di ogni wave senza snapshot o
|
||||
backup del suo stato precedente.
|
||||
|
||||
### 6.2 Database reale e fixture supplementare `catalog_qa`
|
||||
|
||||
La prima baseline viene acquisita dal database reale collegato. Il test registra soltanto inventario
|
||||
strutturale, conteggi e risultati sanitizzati; poi svuota il catalogo locale e dimostra di poterlo
|
||||
ricostruire dalla stessa sorgente. Non è richiesto preservare alcun metadato locale precedente.
|
||||
|
||||
La fixture `catalog_qa` integra il database reale soltanto quando servono casi controllabili o
|
||||
mutazioni che la sorgente reale non contiene. Deve includere almeno:
|
||||
|
||||
- `customers`: UUID PK, nome, email, codice fiscale, telefono, data di nascita, indirizzo e note;
|
||||
- `orders`: FK verso `customers`, importo numerico, stato, timestamp, default e campi nullable;
|
||||
- `order_lines`: PK composta e relazione composta ordinata;
|
||||
- `clinical_events`: campi sanitari evidenti e tabella partizionata;
|
||||
- `products`: SKU pubblico, categoria, prezzo e flag booleano;
|
||||
- `empty_table`: nessuna riga ma struttura valida;
|
||||
- `wide_entity`: almeno 23 colonne e metadati lunghi, per forzare batch `10 + 10 + 3` e il limite
|
||||
dimensionale del messaggio;
|
||||
- identificatori quotati, commenti Unicode/italiani, commenti null e oggetti fuori dallo schema.
|
||||
|
||||
Usare valori canary inventati e univoci, per esempio:
|
||||
|
||||
- `PRIV_EMAIL_CANARY_...`, `PRIV_TAX_CANARY_...`, `PRIV_HEALTH_CANARY_...` nelle colonne protette;
|
||||
- `PUBLIC_SKU_CANARY_...` in una colonna esplicitamente non sensibile;
|
||||
- `SECRET_API_CANARY_...` solo nel secret store del test.
|
||||
|
||||
I canary protetti non devono comparire nei messaggi al modello, nel catalogo, negli eventi, nelle
|
||||
API, nel DOM o nei log. Il canary pubblico può apparire nel messaggio al provider entro i limiti
|
||||
documentati e dopo la disclosure esplicita dell'utente. Sul database reale la stessa proprietà va
|
||||
provata soprattutto osservando la proiezione SQL e il payload transitorio: i valori protetti non
|
||||
devono essere letti, e nessun valore grezzo deve entrare nell'evidenza conservata.
|
||||
|
||||
### 6.3 Mutazioni della sorgente
|
||||
|
||||
Preparare tre revisioni dello schema:
|
||||
|
||||
- **A — iniziale**: struttura completa e commenti sorgente valorizzati;
|
||||
- **B — distruttiva**: rimozione di una tabella, una colonna e una FK, più aggiunta di una nuova
|
||||
colonna sensibile per nome;
|
||||
- **C — race di conferma**: modifica ulteriore fra piano distruttivo e conferma, per provare il
|
||||
re-scan.
|
||||
|
||||
Queste revisioni possono vivere nella fixture o in uno schema reale esplicitamente dichiarato
|
||||
scrivibile. Fra una prova e l'altra è consentito eliminare completamente il catalogo locale,
|
||||
riapplicare le migrazioni e ripartire dal database reale.
|
||||
|
||||
## 7. Casi di test — acquisizione dei metadati
|
||||
|
||||
| ID | P | Livello | Scenario | Risultato atteso |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| MET-01 | P0 | API/Integrazione | Avvio senza binding raggiungibile, con versione database obsoleta o senza `database.manage`. | Il run non parte; risposta sicura e catalogo invariato. I segreti restano write-only. |
|
||||
| MET-02 | P0 | Integrazione | Reset completo del catalogo e `all` sul database reale collegato, senza mock del client `pg_catalog`; ripetizione sulla fixture A per gli edge case assenti. | Il catalogo viene ricostruito da zero con snapshot esatta di tabelle, colonne, `sourceComment`, tipo, default, nullabilità, PK e FK ordinate; stato `succeeded`; nessuna scrittura alla sorgente. |
|
||||
| MET-03 | P1 | Contratto/Integrazione | Stessa fixture via RPC REST tipizzato. | `schemaVersion: 1` e capability sono validate strettamente; risultato normalizzato uguale a MET-02. `unavailable` non è interpretato come collezione vuota. |
|
||||
| MET-04 | P0 | Contratto/Integrazione | `/schema_snapshot` assente, fallback `run_query`; poi fallback assente, parziale, non JSON o con campi extra/mancanti. | Il fallback usa una sola query read-only. Ogni errore o snapshot invalida fallisce senza modifiche parziali; nessun fallback nasconde un errore operativo diverso da capability assente. |
|
||||
| MET-05 | P1 | Integrazione | Accesso `ssh_tunnel` con host key corretta, errata e assente; errore durante apertura/chiusura. | Parità con MET-02 nel caso valido; fail-closed negli altri casi; processi, lease e file-segreto sempre rilasciati. |
|
||||
| MET-06 | P0 | API | Esecuzione separata di `tables`, `columns`, `relationships` e `all`, con e senza selezione tabelle. | Ogni scope è autorevole solo nel proprio confine; nessun record fuori scope cambia o viene eliminato. Selezioni duplicate/inesistenti sono rifiutate atomicamente. |
|
||||
| MET-07 | P0 | API/Integrazione | Ripetizione idempotente della fixture A dopo modifica di `description`, `generatedDescription` e `sensitive`. | Nessun diff fisico spurio; contenuti curati, proposte AI e flag delle entità ancora presenti sono preservati. Una nuova colonna nasce con `sensitive=false`. |
|
||||
| MET-08 | P0 | API/Integrazione | Passaggio A→B, conferma assente/errata/scaduta e passaggio A→B→C prima della conferma. | Stato `awaiting_confirmation`, piano visibile e nessuna applicazione anticipata. La conferma valida provoca re-scan; se il diff cambia viene emesso un nuovo piano/token e il vecchio non applica nulla. Apply atomica oppure zero modifiche. |
|
||||
| MET-09 | P0 | API/Integrazione | Timeout, disconnessione, capability incompleta o eccezione durante scan/apply. | Stato terminale coerente, errore sanitizzato, catalogo precedente intatto e lock rilasciato. Nessun segreto, SQL sensibile o stack trace nelle API/eventi. |
|
||||
| MET-10 | P1 | Worker | Cancel in `queued`, `running`, `awaiting_confirmation` e `applying`; retry, restart con run attivo e lease scaduto. | Cancel è efficace solo prima di apply ed è rifiutato durante apply; retry crea un nuovo run. Recovery non duplica l'apply, marca correttamente i run interrotti, rilascia il lock e rimuove snapshot/diff/token interni non più necessari. |
|
||||
| MET-11 | P0 | API | Due operazioni sullo stesso database: sync, cleanup, connection test, edit o generazione; in parallelo, operazioni su database diversi. | Una sola operazione possiede il database; conflitto 409 sicuro sullo stesso target. Nessun lock cross-database non previsto e nessuna release del token altrui. |
|
||||
| MET-12 | P1 | UI | Avvio dai menu database/tabella, visualizzazione piano, conferma, history drawer, chiusura drawer, perdita SSE e polling. | Scope e selezione inviati sono esatti; azioni non eleggibili restano visibili con motivo; chiudere il drawer non ferma il run; replay/polling deduplicano gli eventi e aggiornano griglie/KPI. |
|
||||
| MET-13 | P1 | E2E | Cleanup manuale di tabelle/colonne/relazioni e successiva sincronizzazione. | Cleanup modifica solo il catalogo; la sorgente resta invariata; un sync autorevole ripristina gli oggetti ancora presenti in sorgente. |
|
||||
| MET-14 | P0 | Integrazione | Cambio binding/versione fra scan e apply ed errore iniettato a metà transazione. | La freshness viene ricontrollata sotto lock; il run fallisce senza righe parziali e conserva integralmente il catalogo precedente. |
|
||||
| MET-15 | P1 | API/Worker | Replay SSE con `Last-Event-ID`/`after`, polling concorrente e retention oltre 30 giorni. | Cursori monotoni e nessun duplicato; gli eventi scaduti vengono potati senza corrompere run e stato finale. |
|
||||
|
||||
## 8. Casi di test — identificazione e protezione dei campi sensibili
|
||||
|
||||
| ID | P | Livello | Scenario | Risultato atteso |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| PRV-01 | P0 | Repository/API | Prima sincronizzazione, re-sync e aggiunta di una colonna. | Il default è `false`; il valore umano delle colonne esistenti è preservato; la nuova colonna è esplicitamente da riesaminare ma non riceve uno stato audit inventato. |
|
||||
| PRV-02 | P0 | API | Suggerimento per un database, tabelle selezionate e colonne selezionate; database multipli, target duplicati o mancanti. | Il provider riceve esattamente le colonne dello scope. Input ambigui sono rifiutati prima della chiamata e nessun flag cambia. |
|
||||
| PRV-03 | P0 | Contratto | Ispezione del messaggio al classifier. | Sono presenti solo database, schema, tabella, colonna, tipo, nullabilità, PK e FK. Non compaiono righe, valori, commenti, descrizioni, flag corrente o segreti. |
|
||||
| PRV-04 | P1 | Contratto | `wide_entity`, identificatori lunghi e limite byte. | Ordine deterministico, batch massimi di dieci colonne e rispetto del limite messaggio; una singola colonna non rappresentabile fallisce prima del provider con errore sicuro. |
|
||||
| PRV-05 | P0 | API | Risposta valida, fenced/prosa, JSON malformato, target mancante/duplicato/ignoto e provider failure. | Ogni colonna richiesta compare una sola volta. Una classificazione invalida viene ritentata una volta; dopo esaurimento si ottiene errore sanitizzato e nessuna modifica. |
|
||||
| PRV-06 | P0 | UI/API | Apertura draft, modifica manuale, chiusura/reload e salvataggio. | La proposta non è persistita prima di Save; reload la scarta. Il reviewer può invertire scelte; si salvano solo colonne cambiate con versione ottimistica; un conflitto richiede reload. |
|
||||
| PRV-07 | P1 | Repository/UI | Tentativi completati, falliti e attivi al restart. | Ogni tentativo ha un run distinto con scope, modello, contatori ed eventi sanitizzati; startup marca `interrupted` i run attivi. Storico newest-first senza target ID, proposte, prompt, output grezzo o diagnostica provider. |
|
||||
| PRV-08 | P0 | Integrazione | Generazione descrizioni su target con canary protetti. | Le colonne protette sono assenti dalla proiezione SQL, non semplicemente filtrate dopo la lettura. Se non rimangono colonne leggibili non viene eseguita una `SELECT`. Nessun canary protetto esce dal processo. |
|
||||
| PRV-09 | P0 | Contratto/Integrazione | Tabella mista con colonne sensibili e pubbliche. | Per le sensibili il prompt contiene valori plausibili, deterministici e limitati derivati dai soli metadati, nello stesso formato dei campioni e senza etichettarli al modello come sintetici. Per le pubbliche: massimo cinque righe e cinque valori rappresentativi, valori troncati e transazione read-only chiusa con rollback. |
|
||||
| PRV-10 | P1 | API | Cambio `false→true→false` dopo una descrizione già generata. | Il testo esistente non viene rigenerato retroattivamente. Solo le generazioni future cambiano fonte del contesto; tornando `false` il campionamento reale torna eleggibile. |
|
||||
| PRV-11 | P0 | API/UI | Utente senza `database.manage`, modello non configurato, catalogo/provider indisponibile e richiesta interrotta. | Controlli nascosti/disabilitati in UI e rifiuto server-side; errori non espongono dettagli. Un tentativo fallito compare nello storico senza trasformarsi in audit della decisione umana. |
|
||||
| PRV-12 | P1 | L2 | Corpus strutturale etichettato con identificatori personali, credenziali/token, salute, finanza, localizzazione e controlli non sensibili/ambigui, in inglese e italiano. | Si misurano precisione, recall e falsi negativi per modello. I campi critici mancati sono sottoposti al product owner; la soglia quantitativa va ratificata prima di diventare gate, perché il classifier è advisory e human-in-the-loop. |
|
||||
| PRV-13 | P0 | UI/E2E | Modifica di un flag nella review senza Save e tentativo immediato di generare descrizioni. | Gate di rilascio da formalizzare: la generazione deve essere bloccata finché il draft non è salvato o scartato. In alternativa la UI deve dichiarare inequivocabilmente che verrà usato il valore persistito; non è accettabile mostrare “protetto” e campionare come non protetto. |
|
||||
|
||||
## 9. Casi di test — generazione e consolidamento dei commenti
|
||||
|
||||
| ID | P | Livello | Scenario | Risultato atteso |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| GEN-01 | P0 | Config/API | Modelli validi, default, endpoint anonimo esplicito, secret ref mancante/errato e nessun modello. | Al browser arrivano solo ID, label e default. Nessuna chiave, provider payload o configurazione privata è esposta; feature disabilitata in modo comprensibile se non configurata. |
|
||||
| GEN-02 | P0 | API | `selected_columns`, `selected_tables`, `all`, `missing`, target duplicati/inesistenti e zero eleggibili. | Scope esatto; `missing` include null/vuoto/whitespace e salta proposte esistenti; `all` sostituisce solo dopo conferma esplicita; input invalido non crea run. |
|
||||
| GEN-03 | P1 | Worker | 23 colonne più tabelle, metadati e campioni lunghi. | Colonne prima delle tabelle per lo scope globale; richieste omogenee, sequenziali, massimo dieci target e sotto i limiti byte; contesto tabella aggiornato dopo le colonne. |
|
||||
| GEN-04 | P0 | Contratto | JSON puro, un solo code fence JSON completo, prosa extra, payload multipli, target mancante/duplicato/ignoto, descrizione vuota o oltre limite. | Sono accettati solo i primi due formati validi. Una mappatura ambigua non applica alcun risultato del batch e conta come errore tecnico. |
|
||||
| GEN-05 | P1 | API | Esito `generated` e `non_generatable` in workspace italiano/inglese. | Testo generato trimmato e salvato; il testo standard non generabile è localizzato dall'applicazione, non copiato dal provider. Gli errori tecnici non scrivono tale testo. |
|
||||
| GEN-06 | P0 | Worker | Errore transiente, errore esaurito isolato, tre batch falliti consecutivi e successo fra due errori. | Helper con al massimo un retry e nessun fallback modello. Errori isolati portano a `completed_with_errors`; tre consecutivi a `failed`; un successo azzera il contatore. |
|
||||
| GEN-07 | P0 | Integrazione | Successi seguiti da cancel, interruption o failure. | Ogni risultato valido è persistito subito e resta disponibile; il target fallito non riceve dati ambigui. “Generate Missing” consente il recupero naturale senza resume automatico. |
|
||||
| GEN-08 | P0 | API | Secondo run durante un run attivo e modifica/sync/cleanup/consolidamento sul database posseduto. | Un solo run di generazione attivo nell'installazione; 409 chiaro al secondo Start. Il database target resta riservato e le operazioni incompatibili non alterano selezione o dati. |
|
||||
| GEN-09 | P0 | Integrazione/Worker/UI | Stop in queued/running con helper attivo e con una `SELECT` sorgente deliberatamente bloccata/lenta. | Helper e query/connessione vengono terminati entro 5 secondi, stato `cancelled`, nessuna chiamata modello dopo Stop, risultati precedenti conservati ed eventi consultabili. Un test con sampler fake non è sufficiente. |
|
||||
| GEN-10 | P0 | Worker/API | Restart con run `queued/running`; Unlock con worker/helper vivo e con run realmente stale; race Unlock/Start. | Startup marca `interrupted` senza replay. Unlock è rifiutato se esiste lavoro locale vivo e non può liberare la reservation di un nuovo run. |
|
||||
| GEN-11 | P1 | Integrazione | Sorgente campioni indisponibile ma metadati validi. | La generazione prosegue metadata-only con un solo warning sicuro; nessun tentativo alternativo espone dettagli di connessione. |
|
||||
| GEN-12 | P1 | API/UI | SSE disconnesso, replay da sequence, polling concorrente e riapertura storico. | Eventi persistiti, ordinati e deduplicati; history newest-first; contatori/stato finali coerenti; nessun prompt, campione, risposta completa o stack trace. |
|
||||
| GEN-13 | P0 | UI/API | Revisione manuale della proposta e consolidamento selettivo di tabelle/colonne, inclusi target vuoti/stale. | Si copia solo `generatedDescription` non vuota dei target risolti; conteggio `copied/skipped` corretto; operazione atomica; `sourceComment` invariato. |
|
||||
| GEN-14 | P0 | UI/Sicurezza | Tutti gli scope di generazione da database, tabelle e colonne selezionate, utente senza permesso, metadata contenente istruzioni ostili. | Prima di ogni Start, inclusi `selected_tables` e `selected_columns`, compare la disclosure “fino a cinque righe e cinque valori”. Il server applica comunque l'autorizzazione. Metadati e valori sono trattati come dati non fidati e l'output resta nel contratto JSON. |
|
||||
| GEN-15 | P1 | L2 | Run reale sul modello di default e almeno un endpoint alternativo supportato, usando il database reale dopo la review dei flag e un role read-only. | Descrizioni nella lingua workspace, coerenti con struttura/commenti e senza fatti inventati critici; almeno una colonna e una tabella consolidate. Nessun valore protetto o segreto compare in request osservabile, eventi, API, persistenza o log. |
|
||||
| GEN-16 | P1 | Integrazione | Fastify→worker→processo Python→LiteLLM→endpoint OpenAI-compatible locale di cattura. | Routing provider/model, header API key, `disableThinking`, singolo retry, timeout, limite output e payload sono corretti; chiave e diagnostica non risalgono a stdout, API o log applicativi. |
|
||||
|
||||
## 10. Percorso E2E prioritario
|
||||
|
||||
Il caso `E2E-01` deve attraversare le tre feature senza sostituire i test di contratto:
|
||||
|
||||
1. azzerare `catalog-db`, riapplicare le migrazioni e verificare che il role del database reale sia
|
||||
realmente read-only;
|
||||
2. eseguire `all` sul database reale e confrontare catalogo e snapshot sorgente; usare la fixture A
|
||||
in una seconda esecuzione per gli edge case mancanti;
|
||||
3. richiedere suggerimenti privacy sull'intero database;
|
||||
4. modificare almeno una proposta e salvare i flag revisionati;
|
||||
5. avviare `missing` dopo la disclosure, usando un Model Completer osservabile;
|
||||
6. dimostrare che i canary protetti non sono letti né inviati e che il canary pubblico rispetta i
|
||||
limiti;
|
||||
7. correggere una `generatedDescription` e consolidare una tabella e una colonna;
|
||||
8. applicare la fixture B, controllare il piano distruttivo e introdurre C prima della conferma;
|
||||
9. confermare dopo il re-scan e verificare atomicità, preservazione di descrizioni/flag delle entità
|
||||
superstiti e `sensitive=false` sulla nuova colonna;
|
||||
10. perdere la connessione SSE, riaprire entrambi gli storici e verificare replay, polling e KPI.
|
||||
|
||||
Il percorso deve essere eseguito con fake provider dopo ogni reset rilevante e, in forma ridotta,
|
||||
con il provider configurato sul database reale dopo che i flag sono stati revisionati e salvati.
|
||||
Non è richiesto ripristinare lo stato locale precedente al test.
|
||||
|
||||
## 11. Valutazione qualitativa dei modelli
|
||||
|
||||
La qualità non deve confondersi con la sicurezza: anche un classifier perfetto non autorizza
|
||||
l'esfiltrazione di un canary protetto e una descrizione elegante non rende valido un payload
|
||||
ambiguo.
|
||||
|
||||
### 11.1 Classificazione privacy
|
||||
|
||||
Per ogni modello registrare matrice di confusione, precisione, recall e falsi negativi per categoria.
|
||||
Eseguire almeno tre iterazioni sul corpus fisso per rilevare instabilità. Fino alla ratifica di una
|
||||
soglia da parte del product owner, il risultato è un gate di review: ogni falso negativo su
|
||||
credenziali/token, identificatori fiscali, dati sanitari o finanziari richiede accettazione esplicita
|
||||
o correzione prima del rilascio operativo.
|
||||
|
||||
### 11.2 Descrizioni generate
|
||||
|
||||
Valutare ogni testo da 0 a 2 su:
|
||||
|
||||
- correttezza rispetto a struttura e commenti sorgente;
|
||||
- specificità e utilità per un revisore;
|
||||
- lingua e chiarezza;
|
||||
- assenza di istruzioni seguite dai dati non fidati o fatti inventati;
|
||||
- assenza di valori protetti e segreti.
|
||||
|
||||
Soglia proposta da ratificare: almeno 8/10, nessun punteggio 0 su correttezza o sicurezza. Un valore
|
||||
esplicitamente non sensibile, reale o inventato, può essere ripreso entro il perimetro dichiarato;
|
||||
un valore protetto non può mai esserlo.
|
||||
|
||||
## 12. Copertura esistente e gap da chiudere
|
||||
|
||||
| Area | Evidenza automatica già presente | Gap principale |
|
||||
| --- | --- | --- |
|
||||
| Snapshot e sincronizzazione | `backend/test/catalog-schema-introspector.test.ts`, `catalog-schema-routes.test.ts`, `catalog-table-introspector.test.ts`, `catalog-repository.integration.test.ts` | Introspezione `pg_catalog` realmente end-to-end, parità live dei tre trasporti e un unico E2E con re-scan distruttivo. |
|
||||
| Privacy | `catalog-description-generation-routes.test.ts`, `catalog-description-generation-worker.test.ts`, `catalog-description-source-sampler.test.ts`, `catalog-synthetic-sample-value.test.ts` | Prova canary integrata query→prompt→API/log e benchmark reale post-ADR 0011. |
|
||||
| Generazione | `catalog-description-generation-routes.test.ts`, `catalog-description-generation-worker.test.ts`, `catalog-description-generation.integration.test.ts` e test del helper | Accettazione reale aggiornata, cancellazione di una query PostgreSQL bloccata e integrazione ermetica fino all'endpoint LiteLLM locale. |
|
||||
| UI | `DatabaseManagementPage.test.tsx`, `DescriptionGenerationDrawer.test.tsx`, `SensitiveDataSuggestionHistoryDrawer.test.tsx` | L'E2E Playwright corrente verifica soprattutto il layout, non il workflow funzionale. |
|
||||
|
||||
Nuovi asset consigliati:
|
||||
|
||||
- `backend/test/catalog-metadata-privacy-workflow.integration.test.ts`;
|
||||
- `backend/test/fixtures/catalog-privacy-schema.sql` e snapshot REST equivalenti;
|
||||
- integrazione a due PostgreSQL per le query reali `pg_catalog` e lo stop del sampler;
|
||||
- endpoint OpenAI-compatible locale di cattura per GEN-16;
|
||||
- `frontend/e2e/database-management-workflow.spec.ts`;
|
||||
- corpus strutturale versionato per PRV-12, senza valori business;
|
||||
- nuovo report di accettazione L2 che sostituisca il gate privacy del 2026-08-29.
|
||||
|
||||
## 13. Ordine di esecuzione
|
||||
|
||||
### Wave 1 — contratto rapido
|
||||
|
||||
- parser snapshot, introspector, scope e diff;
|
||||
- classifier strutturale, batching e validazione output;
|
||||
- sampler, valori sintetici, prompt bounds e parser descrizioni;
|
||||
- autorizzazione, redazione e race del coordinator.
|
||||
|
||||
### Wave 2 — integrazione PostgreSQL
|
||||
|
||||
- reset totale di `catalog-db`, bootstrap delle migrazioni e ricostruzione dal database reale;
|
||||
- migrazioni e vincoli del repository;
|
||||
- atomicità sincronizzazione/consolidamento;
|
||||
- run ed eventi persistiti, restart e cancellation;
|
||||
- `E2E-01` con fake provider e canary.
|
||||
|
||||
### Wave 3 — frontend e stack locale
|
||||
|
||||
- component test MSW;
|
||||
- Playwright funzionale, perdita SSE e polling;
|
||||
- verifica disclosure, review draft, history e consolidamento.
|
||||
|
||||
### Wave 4 — accettazione L2
|
||||
|
||||
- provider reale sul database collegato dopo classificazione e review dei flag;
|
||||
- benchmark PRV-12 e rubric GEN-15;
|
||||
- scansione finale di canary e segreti;
|
||||
- approvazione del product owner.
|
||||
|
||||
Comandi di regressione:
|
||||
|
||||
```bash
|
||||
cd backend && npx vitest run
|
||||
cd backend && npx tsc --noEmit -p .
|
||||
cd backend && npm run build
|
||||
cd frontend && npx vitest run
|
||||
cd frontend && npx tsc -b
|
||||
cd frontend && npm run build
|
||||
cd frontend && npm run e2e
|
||||
./scripts/build-docs.sh
|
||||
```
|
||||
|
||||
Il report deve evidenziare esplicitamente eventuali test Docker/L2 saltati; un `skip` non equivale a
|
||||
PASS del relativo gate.
|
||||
|
||||
## 14. Evidenze da conservare
|
||||
|
||||
Per ogni esecuzione registrare:
|
||||
|
||||
- commit, configurazione pubblica dei modelli, versione fixture e trasporto;
|
||||
- ID e stato finale dei run, contatori ed eventi sanitizzati;
|
||||
- snapshot catalogo prima/dopo e piano distruttivo con metadati e conteggi sanitizzati, senza valori
|
||||
grezzi delle righe sorgente;
|
||||
- report test/JUnit, screenshot dei gate UI e risultato della scansione canary;
|
||||
- matrice di confusione privacy e rubric delle descrizioni per le prove L2;
|
||||
- difetti con ID del caso, severità, riproducibilità e decisione finale.
|
||||
|
||||
Non allegare prompt completi, righe campione, output grezzi del provider, chiavi, digest o frammenti
|
||||
di segreti. Il Model Completer spy deve verificare in memoria le asserzioni e scartare il payload al
|
||||
termine del test.
|
||||
|
||||
Non serve conservare backup del catalogo locale, run precedenti o descrizioni generate durante una
|
||||
wave: l'evidenza è il report sanitizzato e la capacità di ricostruire nuovamente il risultato dalla
|
||||
sorgente reale.
|
||||
|
||||
## 15. Criteri di ingresso e uscita
|
||||
|
||||
### Ingresso
|
||||
|
||||
- baseline unica per backend/frontend e procedura verificata per eliminare e ricreare `catalog-db`;
|
||||
- accesso al database reale collegato e inventario delle tabelle/colonne da includere nella review;
|
||||
- fixture e canary supplementari approvati per i soli edge case controllati;
|
||||
- role sorgente read-only verificato con una scrittura deliberatamente negata;
|
||||
- fake connector/provider disponibili e log collection attiva;
|
||||
- per L2, secret reference configurato senza materializzare il valore nell'evidenza.
|
||||
|
||||
### Uscita
|
||||
|
||||
- 100% dei casi P0 e P1 applicabili superati; nessun difetto Sev-1/Sev-2 aperto;
|
||||
- almeno due ricostruzioni complete e coerenti del catalogo a partire dal database reale dopo reset
|
||||
indipendenti;
|
||||
- zero comparsa dei canary protetti e dei segreti fuori dalla sorgente/secret store del test;
|
||||
- nessuna modifica parziale dopo errori, cancel o conferme stale;
|
||||
- parità normalizzata dei trasporti supportati e nessuna integration PostgreSQL richiesta saltata;
|
||||
- stati, contatori, eventi e history coerenti dopo success, partial failure, stop e restart;
|
||||
- `sourceComment`, `generatedDescription` e `description` mantengono l'autorità prevista;
|
||||
- accettazione L2 sul database collegato approvata dal product owner e
|
||||
build/test/typecheck/documentazione verdi;
|
||||
- ogni deviazione P2 o soglia qualitativa non ancora ratificata è documentata con owner e data.
|
||||
|
||||
## 16. Rischi residui da rendere espliciti
|
||||
|
||||
- `sensitive=false` è il default e il prodotto non conserva uno stato “review completata”: dopo ogni
|
||||
reset, classificazione strutturale e salvataggio umano dei flag devono precedere qualunque run con
|
||||
provider reale. Il catalogo è ricostruibile; un invio errato a un provider non lo è.
|
||||
- Il flag protegge i valori campionati. Nomi, `sourceComment` e descrizioni sono metadati inviabili
|
||||
al modello e possono contenere testo libero: se nel database reale includono PII serve una
|
||||
decisione aggiuntiva di redazione, non una diversa aspettativa di test.
|
||||
- La UI deve risolvere il caso di un flag modificato ma non salvato prima della generazione e deve
|
||||
mostrare la disclosure anche per tabelle/colonne selezionate; il piano considera entrambi P0.
|
||||
- L'AbortSignal corrente va provato contro una query PostgreSQL realmente bloccata: la sola
|
||||
cancellazione del helper non dimostra che la lettura sorgente sia interrompibile.
|
||||
- Lo storico dei Sensitive Data Suggestion Run è operativo, non un audit delle decisioni umane.
|
||||
- La policy privacy non è ancora applicata allo schema-linking/LSH; nessun risultato di questo piano
|
||||
deve essere presentato come copertura di quel percorso.
|
||||
- Un provider reale resta non deterministico: il rilascio deve dipendere dai gate tecnici e dalla
|
||||
review umana, non dalla ripetizione byte-identica delle descrizioni.
|
||||
- L'esclusione fra operazioni e Unlock è in parte locale al processo. Se il deployment ammetterà più
|
||||
repliche backend, servirà un gate aggiuntivo con due istanze contro lo stesso catalogo; non va
|
||||
dedotta sicurezza multi-replica dai test single-process.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,4 +1,4 @@
|
||||
import { expect, test, type Page, type Route } from "@playwright/test";
|
||||
import { expect, test, type Locator, type Page, type Route } from "@playwright/test";
|
||||
import { createAuthenticationStack } from "./fixtures/auth-stack.mjs";
|
||||
|
||||
test.describe.configure({ mode: "serial" });
|
||||
@@ -12,6 +12,13 @@ const database = {
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "ready",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
@@ -37,6 +44,22 @@ const database = {
|
||||
},
|
||||
};
|
||||
|
||||
const unconfiguredDatabase = {
|
||||
...database,
|
||||
id: undefined,
|
||||
workspaceId: "research-lab",
|
||||
workspaceName: "Research laboratory",
|
||||
configured: false,
|
||||
databaseName: "research",
|
||||
schema: "analytics",
|
||||
version: 0,
|
||||
createdAt: "",
|
||||
updatedAt: "",
|
||||
binding: { transport: "postgres_direct", port: 5432 },
|
||||
connectionStatus: "untested",
|
||||
testedVersion: undefined,
|
||||
};
|
||||
|
||||
const table = {
|
||||
id: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
|
||||
databaseId,
|
||||
@@ -118,7 +141,112 @@ async function expectContextPanelGeometry(page: Page, accessibleName: string) {
|
||||
expect(panelBox.y + panelBox.height).toBeLessThanOrEqual(managerBox.y + managerBox.height + 1);
|
||||
}
|
||||
|
||||
async function expectContextPanelHeaderMatchesNewSession(
|
||||
async function expectCurrentNavigation(page: Page, accessibleName: string) {
|
||||
const navigation = page.getByRole("complementary", { name: "Session navigation" });
|
||||
const button = navigation.getByRole("button", { name: accessibleName, exact: true });
|
||||
const available = navigation.locator('[data-navigation-state="available"]').first();
|
||||
await expect(navigation.locator('[data-navigation-state="current"]')).toHaveCount(1);
|
||||
await expect(button).toHaveAttribute("data-navigation-state", "current");
|
||||
await expect(button).toHaveAttribute("aria-current", "page");
|
||||
expect(await button.getAttribute("class")).toContain("bg-[oklch(var(--nav-active))]");
|
||||
|
||||
const [style, availableStyle] = await Promise.all([
|
||||
button.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderTopColor,
|
||||
})),
|
||||
available.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderTopColor,
|
||||
})),
|
||||
]);
|
||||
expect(style.backgroundColor).not.toBe(availableStyle.backgroundColor);
|
||||
expect(style.borderColor).not.toBe(availableStyle.borderColor);
|
||||
|
||||
const tokens = await page.evaluate(() => {
|
||||
const root = getComputedStyle(document.documentElement);
|
||||
const read = (name: string) => root.getPropertyValue(name).trim().split(/\s+/).map(Number);
|
||||
return { primary: read("--primary"), active: read("--nav-active") };
|
||||
});
|
||||
expect(tokens.active[2]).toBeCloseTo(tokens.primary[2], 1);
|
||||
expect(tokens.active[1]).toBeLessThan(tokens.primary[1]);
|
||||
expect(tokens.active[0]).toBeGreaterThan(tokens.primary[0]);
|
||||
}
|
||||
|
||||
async function expectSelectedSessionScopeTab(page: Page, accessibleName: string) {
|
||||
const tablist = page.getByRole("tablist", { name: "Session scope" });
|
||||
const selected = tablist.getByRole("tab", { name: accessibleName, exact: true });
|
||||
const inactive = tablist.locator('[role="tab"][aria-selected="false"]');
|
||||
await expect(tablist.locator('[role="tab"][aria-selected="true"]')).toHaveCount(1);
|
||||
await expect(selected).toHaveAttribute("data-tab-state", "active");
|
||||
expect(await selected.getAttribute("class")).toContain("bg-[oklch(var(--nav-active))]");
|
||||
|
||||
const [selectedStyle, inactiveStyle] = await Promise.all([
|
||||
selected.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderBottomColor,
|
||||
})),
|
||||
inactive.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
borderColor: getComputedStyle(element).borderBottomColor,
|
||||
borderWidth: getComputedStyle(element).borderTopWidth,
|
||||
})),
|
||||
]);
|
||||
expect(selectedStyle.backgroundColor).not.toBe(inactiveStyle.backgroundColor);
|
||||
expect(selectedStyle.borderColor).not.toBe(inactiveStyle.borderColor);
|
||||
expect(inactiveStyle.borderWidth).toBe("1px");
|
||||
expect(inactiveStyle.backgroundColor).not.toMatch(/^(?:transparent|rgba\(0, 0, 0, 0\))$/);
|
||||
expect(inactiveStyle.borderColor).not.toMatch(/^(?:transparent|rgba\(0, 0, 0, 0\))$/);
|
||||
}
|
||||
|
||||
function cssLightness(color: string): number {
|
||||
const oklab = color.match(/^okl(?:ab|ch)\(([\d.]+)(%)?/);
|
||||
if (oklab) {
|
||||
const value = Number(oklab[1]);
|
||||
return oklab[2] ? value / 100 : value;
|
||||
}
|
||||
const channels = color.match(/[\d.]+/g)?.slice(0, 3).map(Number) ?? [];
|
||||
if (channels.length !== 3) return Number.NaN;
|
||||
return color.startsWith("color(srgb") ? Math.max(...channels) : Math.max(...channels) / 255;
|
||||
}
|
||||
|
||||
async function expectFleetRowTooltipBelow(trigger: Locator, expectedContent: string) {
|
||||
await trigger.hover();
|
||||
await expect.poll(() => trigger.evaluate((element) => {
|
||||
const owner = element.closest<HTMLElement>("[data-tooltip]");
|
||||
return owner ? getComputedStyle(owner, "::after").visibility : "missing";
|
||||
})).toBe("visible");
|
||||
await expect.poll(() => trigger.evaluate((element) => {
|
||||
const owner = element.closest<HTMLElement>("[data-tooltip]");
|
||||
return owner ? getComputedStyle(owner, "::after").opacity : "missing";
|
||||
})).toBe("1");
|
||||
|
||||
const state = await trigger.evaluate((element) => {
|
||||
const owner = element.closest<HTMLElement>("[data-tooltip]");
|
||||
if (!owner) throw new Error("Row action tooltip owner is missing");
|
||||
const style = getComputedStyle(owner, "::after");
|
||||
return {
|
||||
content: style.content.replace(/^['\"]|['\"]$/g, ""),
|
||||
ownerHeight: owner.getBoundingClientRect().height,
|
||||
top: Number.parseFloat(style.top),
|
||||
right: Number.parseFloat(style.right),
|
||||
opacity: style.opacity,
|
||||
pointerEvents: style.pointerEvents,
|
||||
backgroundColor: style.backgroundColor,
|
||||
color: style.color,
|
||||
};
|
||||
});
|
||||
|
||||
expect(state.content).toBe(expectedContent);
|
||||
expect(state.top - state.ownerHeight).toBeCloseTo(3, 1);
|
||||
expect(state.right).toBe(0);
|
||||
expect(state.opacity).toBe("1");
|
||||
expect(state.pointerEvents).toBe("none");
|
||||
expect(cssLightness(state.backgroundColor)).toBeLessThan(0.35);
|
||||
expect(cssLightness(state.color)).toBeGreaterThan(0.9);
|
||||
}
|
||||
|
||||
async function expectContextPanelHeaderUsesPrimary(
|
||||
page: Page,
|
||||
accessibleName: string,
|
||||
) {
|
||||
@@ -127,26 +255,22 @@ async function expectContextPanelHeaderMatchesNewSession(
|
||||
.getByRole("main", { name: "Database management" });
|
||||
const panel = manager.getByRole("dialog", { name: accessibleName });
|
||||
const header = panel.locator(':scope > [data-catalog-panel-region="header"]');
|
||||
const newSession = page
|
||||
.getByRole("complementary", { name: "Session navigation" })
|
||||
.getByRole("button", { name: "New session", exact: true });
|
||||
|
||||
await expect(header).toHaveCount(1);
|
||||
await expect(newSession).toBeVisible();
|
||||
|
||||
const [headerStyle, actionStyle] = await Promise.all([
|
||||
const currentNavigation = page.locator('[data-navigation-state="current"]');
|
||||
const [headerStyle, navigationStyle] = await Promise.all([
|
||||
header.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
backgroundImage: getComputedStyle(element).backgroundImage,
|
||||
})),
|
||||
newSession.evaluate((element) => ({
|
||||
currentNavigation.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
})),
|
||||
]);
|
||||
|
||||
expect(headerStyle.backgroundImage).toBe("none");
|
||||
expect(headerStyle.backgroundColor).not.toBe("rgba(0, 0, 0, 0)");
|
||||
expect(headerStyle.backgroundColor).toBe(actionStyle.backgroundColor);
|
||||
expect(headerStyle.backgroundColor).not.toBe(navigationStyle.backgroundColor);
|
||||
}
|
||||
|
||||
async function expectWorkAreaPanelGeometry(page: Page, accessibleName: string) {
|
||||
@@ -181,7 +305,7 @@ async function expectWorkAreaPanelGeometry(page: Page, accessibleName: string) {
|
||||
expect(panelBox.y + panelBox.height).toBeLessThanOrEqual(workAreaBox.y + workAreaBox.height + 1);
|
||||
}
|
||||
|
||||
async function expectWorkAreaPanelHeaderMatchesNewSession(
|
||||
async function expectWorkAreaPanelHeaderUsesPrimary(
|
||||
page: Page,
|
||||
accessibleName: string,
|
||||
) {
|
||||
@@ -189,21 +313,19 @@ async function expectWorkAreaPanelHeaderMatchesNewSession(
|
||||
.getByTestId("conversation-column")
|
||||
.getByRole("dialog", { name: accessibleName });
|
||||
const header = panel.locator(':scope > [data-work-area-panel-region="header"]');
|
||||
const newSession = page
|
||||
.getByRole("complementary", { name: "Session navigation" })
|
||||
.getByRole("button", { name: "New session", exact: true });
|
||||
const [headerStyle, actionStyle] = await Promise.all([
|
||||
const currentNavigation = page.locator('[data-navigation-state="current"]');
|
||||
const [headerStyle, navigationStyle] = await Promise.all([
|
||||
header.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
backgroundImage: getComputedStyle(element).backgroundImage,
|
||||
})),
|
||||
newSession.evaluate((element) => ({
|
||||
currentNavigation.evaluate((element) => ({
|
||||
backgroundColor: getComputedStyle(element).backgroundColor,
|
||||
})),
|
||||
]);
|
||||
|
||||
expect(headerStyle.backgroundImage).toBe("none");
|
||||
expect(headerStyle.backgroundColor).toBe(actionStyle.backgroundColor);
|
||||
expect(headerStyle.backgroundColor).not.toBe(navigationStyle.backgroundColor);
|
||||
}
|
||||
|
||||
test.beforeAll(async () => {
|
||||
@@ -218,10 +340,14 @@ test.afterAll(async () => {
|
||||
test("context panels stay inside the manager and the Tables grid sits in a sidebar-colored frame", async ({ page }) => {
|
||||
const unexpectedCatalogRequests: string[] = [];
|
||||
const responses: Record<string, unknown> = {
|
||||
"GET /api/catalog/databases": [database],
|
||||
"GET /api/catalog/databases": [database, unconfiguredDatabase],
|
||||
"GET /api/catalog/metadata-generation/models": {
|
||||
models: [],
|
||||
default: null,
|
||||
models: [
|
||||
{ id: "deepseek-v4-pro", label: "DeepSeek V4 Pro" },
|
||||
{ id: "glm-53", label: "GLM 5.3" },
|
||||
{ id: "qwen-36", label: "AritmoLab Qwen 3.6 35B A3B" },
|
||||
],
|
||||
default: "glm-53",
|
||||
},
|
||||
"GET /api/catalog/description-generation-runs?limit=50": [],
|
||||
"GET /api/catalog/sensitive-data-suggestion-runs?limit=50": [],
|
||||
@@ -252,10 +378,40 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
|
||||
await page.goto(stack.publicUrl);
|
||||
await signInAsAdmin(page);
|
||||
await expectCurrentNavigation(page, "New session");
|
||||
await expectSelectedSessionScopeTab(page, "My sessions");
|
||||
await page.getByRole("tab", { name: "All sessions", exact: true }).click();
|
||||
await expectSelectedSessionScopeTab(page, "All sessions");
|
||||
await page.getByRole("tab", { name: "My sessions", exact: true }).click();
|
||||
await expectSelectedSessionScopeTab(page, "My sessions");
|
||||
|
||||
await page
|
||||
const adminNavigationRail = page.getByRole("complementary", { name: "Session navigation" });
|
||||
const administration = adminNavigationRail.getByRole("button", { name: "Administration", exact: true });
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "false");
|
||||
await expect(adminNavigationRail.getByRole("region", { name: "Administration" })).toHaveCount(0);
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "true");
|
||||
const administrationPanel = adminNavigationRail.getByRole("region", { name: "Administration" });
|
||||
await expect(administrationPanel).toBeVisible();
|
||||
expect(await administrationPanel.locator(":scope > *").evaluateAll((elements) => elements.map((element) => (
|
||||
element.getAttribute("role") === "separator" ? "separator" : element.textContent?.trim()
|
||||
)))).toEqual([
|
||||
"Database management",
|
||||
"separator",
|
||||
"Workspace management",
|
||||
"Pi management",
|
||||
]);
|
||||
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "false");
|
||||
await expect(adminNavigationRail.getByRole("button", { name: "Database management", exact: true })).toHaveCount(0);
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "true");
|
||||
|
||||
await adminNavigationRail
|
||||
.getByRole("button", { name: "Database management", exact: true })
|
||||
.click();
|
||||
await expectCurrentNavigation(page, "Database management");
|
||||
|
||||
await expect(
|
||||
page.getByRole("complementary", { name: "Session navigation" }),
|
||||
@@ -268,27 +424,41 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
await expect(
|
||||
page.getByRole("button", { name: "Back to workspace", exact: true }),
|
||||
).toHaveCount(0);
|
||||
await expect(page.getByRole("button", { name: /Add database/i })).toHaveCount(0);
|
||||
await expect(page.getByRole("columnheader", { name: /Revision \/ Evidence/ })).toBeVisible();
|
||||
await expect(page.getByRole("columnheader", { name: /NL→SQL runtime/ })).toBeVisible();
|
||||
await expect(page.getByRole("columnheader", { name: /Metadata Catalog/ })).toBeVisible();
|
||||
const metadataModelSelector = page.getByRole("combobox", {
|
||||
name: "Metadata-generation LLM model",
|
||||
});
|
||||
await expect(metadataModelSelector).toHaveCount(1);
|
||||
await expect(metadataModelSelector).toHaveValue("glm-53");
|
||||
await expect(metadataModelSelector.locator("option")).toHaveText([
|
||||
"DeepSeek V4 Pro",
|
||||
"GLM 5.3",
|
||||
"AritmoLab Qwen 3.6 35B A3B",
|
||||
]);
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Configure catalog for Research laboratory", exact: true })
|
||||
.click();
|
||||
const configurePanel = page.getByRole("dialog", { name: "Configure catalog" });
|
||||
await expectContextPanelGeometry(page, "Configure catalog");
|
||||
const configuredWorkspace = configurePanel.getByLabel("Workspace", { exact: true });
|
||||
await expect(configuredWorkspace).toHaveValue("Research laboratory");
|
||||
await expect(configuredWorkspace).toHaveAttribute("readonly", "");
|
||||
await configurePanel.getByRole("button", { name: "Close database panel" }).click();
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Edit Policlinico San Donato", exact: true })
|
||||
.click();
|
||||
await expectContextPanelGeometry(page, "Edit database");
|
||||
await expectContextPanelHeaderMatchesNewSession(page, "Edit database");
|
||||
await expectContextPanelHeaderUsesPrimary(page, "Edit database");
|
||||
await page
|
||||
.getByRole("dialog", { name: "Edit database" })
|
||||
.getByRole("button", { name: "Close database panel" })
|
||||
.click();
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Description history", exact: true })
|
||||
.click();
|
||||
await expectContextPanelGeometry(page, "Description generation");
|
||||
await expectContextPanelHeaderMatchesNewSession(page, "Description generation");
|
||||
await page
|
||||
.getByRole("dialog", { name: "Description generation" })
|
||||
.getByRole("button", { name: /close/i })
|
||||
.click();
|
||||
|
||||
await page.setViewportSize({ width: 1280, height: 800 });
|
||||
await page
|
||||
.getByRole("button", { name: "Edit Policlinico San Donato", exact: true })
|
||||
@@ -300,12 +470,12 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
.click();
|
||||
await page.setViewportSize({ width: 1910, height: 911 });
|
||||
|
||||
await page
|
||||
.getByRole("button", {
|
||||
name: "View tables for Policlinico San Donato",
|
||||
exact: true,
|
||||
})
|
||||
.click();
|
||||
const databaseTablesAction = page.getByRole("button", {
|
||||
name: "View tables for Policlinico San Donato",
|
||||
exact: true,
|
||||
});
|
||||
await expectFleetRowTooltipBelow(databaseTablesAction, "Tables");
|
||||
await databaseTablesAction.click();
|
||||
|
||||
await expect(
|
||||
page.getByRole("region", {
|
||||
@@ -313,6 +483,16 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
}),
|
||||
).toBeVisible();
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Description history", exact: true })
|
||||
.click();
|
||||
await expectContextPanelGeometry(page, "Description generation");
|
||||
await expectContextPanelHeaderUsesPrimary(page, "Description generation");
|
||||
await page
|
||||
.getByRole("dialog", { name: "Description generation" })
|
||||
.getByRole("button", { name: /close/i })
|
||||
.click();
|
||||
|
||||
const applicationBar = page.locator(
|
||||
'main[aria-label="Database management"] .thot-fleet-ledger__header',
|
||||
);
|
||||
@@ -326,6 +506,10 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
await expect(
|
||||
applicationBar.getByRole("group", { name: "Database management actions" }),
|
||||
).toBeVisible();
|
||||
await expect(
|
||||
applicationBar.getByRole("combobox", { name: "Metadata-generation LLM model" }),
|
||||
).toHaveValue("glm-53");
|
||||
await expect(metadataModelSelector).toHaveCount(1);
|
||||
await expect(applicationBar).not.toContainText("Thoth catalog · Fleet ledger");
|
||||
await expect(applicationBar).not.toContainText("Inspect physical metadata");
|
||||
|
||||
@@ -355,11 +539,14 @@ test("context panels stay inside the manager and the Tables grid sits in a sideb
|
||||
|
||||
await expect(page.getByRole("row", { name: /patients/ })).toBeVisible();
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Edit description for patients", exact: true })
|
||||
.click();
|
||||
const editPatientsAction = page.getByRole("button", {
|
||||
name: "Edit description for patients",
|
||||
exact: true,
|
||||
});
|
||||
await expectFleetRowTooltipBelow(editPatientsAction, "Edit metadata");
|
||||
await editPatientsAction.click();
|
||||
await expectContextPanelGeometry(page, "Review table description");
|
||||
await expectContextPanelHeaderMatchesNewSession(page, "Review table description");
|
||||
await expectContextPanelHeaderUsesPrimary(page, "Review table description");
|
||||
const tableEditor = page.getByRole("dialog", { name: "Review table description" });
|
||||
const tableEditorFooter = tableEditor.locator(
|
||||
':scope > [data-catalog-panel-region="footer"]',
|
||||
@@ -500,6 +687,10 @@ test("Workspace and Pi management share the centered work-area panel without cov
|
||||
|
||||
await page.goto(stack.publicUrl);
|
||||
await signInAsAdmin(page);
|
||||
const administration = page.getByRole("button", { name: "Administration", exact: true });
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "false");
|
||||
await administration.click();
|
||||
await expect(administration).toHaveAttribute("aria-expanded", "true");
|
||||
|
||||
for (const viewport of [
|
||||
{ width: 1910, height: 911 },
|
||||
@@ -511,25 +702,29 @@ test("Workspace and Pi management share the centered work-area panel without cov
|
||||
exact: true,
|
||||
});
|
||||
await workspaceTrigger.click();
|
||||
await expectCurrentNavigation(page, "Workspace management");
|
||||
await expectWorkAreaPanelGeometry(page, "Workspace management");
|
||||
await expectWorkAreaPanelHeaderMatchesNewSession(page, "Workspace management");
|
||||
await expectWorkAreaPanelHeaderUsesPrimary(page, "Workspace management");
|
||||
|
||||
const piTrigger = page.getByRole("button", { name: "Pi management", exact: true });
|
||||
await piTrigger.click();
|
||||
await expectCurrentNavigation(page, "Pi management");
|
||||
await expect(
|
||||
page.getByRole("dialog", { name: "Workspace management" }),
|
||||
).toHaveCount(0);
|
||||
await expectWorkAreaPanelGeometry(page, "Pi management");
|
||||
await expectWorkAreaPanelHeaderMatchesNewSession(page, "Pi management");
|
||||
await expectWorkAreaPanelHeaderUsesPrimary(page, "Pi management");
|
||||
await page
|
||||
.getByRole("dialog", { name: "Pi management" })
|
||||
.getByRole("button", { name: "Close Pi management" })
|
||||
.click();
|
||||
await expect(piTrigger).toBeFocused();
|
||||
await expectCurrentNavigation(page, "New session");
|
||||
}
|
||||
|
||||
await page.setViewportSize({ width: 720, height: 800 });
|
||||
await page.getByRole("button", { name: "Workspace management", exact: true }).click();
|
||||
await expectCurrentNavigation(page, "Workspace management");
|
||||
const compactWorkArea = page.getByTestId("conversation-column");
|
||||
const compactPanel = compactWorkArea.getByRole("dialog", { name: "Workspace management" });
|
||||
const [compactWorkAreaBox, compactPanelBox] = await Promise.all([
|
||||
|
||||
@@ -1,7 +1,15 @@
|
||||
import { http, HttpResponse } from "msw";
|
||||
import { expect, test } from "vitest";
|
||||
import { server } from "../test/msw";
|
||||
import { getCatalogMetrics, type CatalogMetrics } from "./catalog-databases";
|
||||
import {
|
||||
createCatalogRelationship,
|
||||
deleteCatalogRelationship,
|
||||
getCatalogMetrics,
|
||||
rebuildGeneratedRelationships,
|
||||
setCatalogRelationshipStatus,
|
||||
type CatalogMetrics,
|
||||
type CatalogRelationship,
|
||||
} from "./catalog-databases";
|
||||
|
||||
const databaseId = "d4baf0f5-b9c3-4dc5-aad2-2f5676a36e64";
|
||||
|
||||
@@ -58,3 +66,99 @@ test("propagates an unsuccessful catalog metrics response", async () => {
|
||||
|
||||
await expect(getCatalogMetrics()).rejects.toMatchObject({ status: 503 });
|
||||
});
|
||||
|
||||
const logicalRelationship: CatalogRelationship = {
|
||||
id: "11111111-1111-4111-8111-111111111111",
|
||||
databaseId,
|
||||
constraintName: null,
|
||||
sourceTableId: "22222222-2222-4222-8222-222222222222",
|
||||
sourceTableName: "orders",
|
||||
targetTableId: "33333333-3333-4333-8333-333333333333",
|
||||
targetTableName: "users",
|
||||
updateRule: null,
|
||||
deleteRule: null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId: "44444444-4444-4444-8444-444444444444",
|
||||
sourceColumnName: "user_id",
|
||||
targetColumnId: "55555555-5555-4555-8555-555555555555",
|
||||
targetColumnName: "id",
|
||||
}],
|
||||
origin: "manual",
|
||||
status: "active",
|
||||
lastSyncedDatabaseVersion: null,
|
||||
lastSyncedAt: null,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
|
||||
test("creates a manual relationship from the selected source and target columns", async () => {
|
||||
let requestBody: unknown;
|
||||
server.use(http.post("/api/catalog/databases/:databaseId/relationships", async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json(logicalRelationship, { status: 201 });
|
||||
}));
|
||||
|
||||
await expect(createCatalogRelationship(
|
||||
databaseId,
|
||||
logicalRelationship.columns[0].sourceColumnId,
|
||||
logicalRelationship.columns[0].targetColumnId,
|
||||
)).resolves.toEqual(logicalRelationship);
|
||||
expect(requestBody).toEqual({
|
||||
sourceColumnId: logicalRelationship.columns[0].sourceColumnId,
|
||||
targetColumnId: logicalRelationship.columns[0].targetColumnId,
|
||||
});
|
||||
});
|
||||
|
||||
test("rebuilds generated relationships for one database", async () => {
|
||||
let calls = 0;
|
||||
server.use(http.post(
|
||||
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
|
||||
() => {
|
||||
calls += 1;
|
||||
return HttpResponse.json({ added: 12, alreadyPresent: 7, excluded: 3, ambiguous: 2 });
|
||||
},
|
||||
));
|
||||
|
||||
await expect(rebuildGeneratedRelationships(databaseId)).resolves.toEqual({
|
||||
added: 12,
|
||||
alreadyPresent: 7,
|
||||
excluded: 3,
|
||||
ambiguous: 2,
|
||||
});
|
||||
expect(calls).toBe(1);
|
||||
});
|
||||
|
||||
test("changes whether a logical relationship is active or excluded", async () => {
|
||||
let requestBody: unknown;
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json({ ...logicalRelationship, status: "excluded" });
|
||||
},
|
||||
));
|
||||
|
||||
await expect(setCatalogRelationshipStatus(
|
||||
databaseId,
|
||||
logicalRelationship.id,
|
||||
"excluded",
|
||||
)).resolves.toMatchObject({ status: "excluded" });
|
||||
expect(requestBody).toEqual({ status: "excluded" });
|
||||
});
|
||||
|
||||
test("permanently deletes a logical relationship", async () => {
|
||||
let calls = 0;
|
||||
server.use(http.delete(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
() => {
|
||||
calls += 1;
|
||||
return new HttpResponse(null, { status: 204 });
|
||||
},
|
||||
));
|
||||
|
||||
await expect(deleteCatalogRelationship(databaseId, logicalRelationship.id)).resolves.toBeUndefined();
|
||||
expect(calls).toBe(1);
|
||||
});
|
||||
|
||||
@@ -3,6 +3,12 @@ import { joinBackendPath } from "./runtime-config";
|
||||
|
||||
export type DatabaseTransport = "postgres_direct" | "rest_api" | "ssh_tunnel";
|
||||
export type ConnectionStatus = "untested" | "reachable" | "failed";
|
||||
export type WorkspaceEvidenceState =
|
||||
| "not_declared"
|
||||
| "materialized_current_revision"
|
||||
| "configuration_required"
|
||||
| "configured_unverified"
|
||||
| "workspace_unavailable";
|
||||
export type CatalogSecretName =
|
||||
| "password"
|
||||
| "apiKey"
|
||||
@@ -33,6 +39,16 @@ export interface CatalogDatabase {
|
||||
workspaceName: string;
|
||||
workspaceDescription?: string;
|
||||
workspaceAvailable: boolean;
|
||||
workspaceRevision: { commit: string; blob: string } | null;
|
||||
workspaceEvidence: {
|
||||
sourceType: "filesystem" | "http" | "s3" | null;
|
||||
state: WorkspaceEvidenceState;
|
||||
};
|
||||
runtimeBinding: {
|
||||
transport: DatabaseTransport;
|
||||
configurationState: "ready" | "configuration_required";
|
||||
sessionTransportSupported: boolean;
|
||||
} | null;
|
||||
configured: boolean;
|
||||
engine: "postgres";
|
||||
databaseName: string;
|
||||
@@ -164,25 +180,37 @@ export interface CatalogRelationshipColumn {
|
||||
targetColumnName: string;
|
||||
}
|
||||
|
||||
export type CatalogRelationshipOrigin = "physical" | "generated" | "manual";
|
||||
export type CatalogRelationshipStatus = "active" | "excluded";
|
||||
|
||||
export interface CatalogRelationship {
|
||||
id: string;
|
||||
databaseId: string;
|
||||
constraintName: string;
|
||||
constraintName: string | null;
|
||||
sourceTableId: string;
|
||||
sourceTableName: string;
|
||||
targetTableId: string;
|
||||
targetTableName: string;
|
||||
updateRule: string;
|
||||
deleteRule: string;
|
||||
updateRule: string | null;
|
||||
deleteRule: string | null;
|
||||
deferrable: boolean;
|
||||
initiallyDeferred: boolean;
|
||||
columns: CatalogRelationshipColumn[];
|
||||
origin: CatalogRelationshipOrigin;
|
||||
status: CatalogRelationshipStatus;
|
||||
lastSyncedDatabaseVersion: number | null;
|
||||
lastSyncedAt: string | null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
export interface CatalogRelationshipRebuildResult {
|
||||
added: number;
|
||||
alreadyPresent: number;
|
||||
excluded: number;
|
||||
ambiguous: number;
|
||||
}
|
||||
|
||||
export type CatalogDatabaseMetadataDeleteTarget = "tables" | "relationships";
|
||||
export type CatalogTableMetadataDeleteTarget = "columns" | "relationships";
|
||||
|
||||
@@ -452,6 +480,36 @@ export const listSensitiveDataSuggestionEvents = (runId: string, after = 0) =>
|
||||
export const listCatalogRelationships = (databaseId: string) =>
|
||||
apiFetch<CatalogRelationship[]>(`/catalog/databases/${encodeURIComponent(databaseId)}/relationships`);
|
||||
|
||||
export const createCatalogRelationship = (
|
||||
databaseId: string,
|
||||
sourceColumnId: string,
|
||||
targetColumnId: string,
|
||||
) => apiFetch<CatalogRelationship>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships`,
|
||||
{ method: "POST", body: JSON.stringify({ sourceColumnId, targetColumnId }) },
|
||||
);
|
||||
|
||||
export const rebuildGeneratedRelationships = (databaseId: string) =>
|
||||
apiFetch<CatalogRelationshipRebuildResult>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/rebuild-generated`,
|
||||
{ method: "POST" },
|
||||
);
|
||||
|
||||
export const setCatalogRelationshipStatus = (
|
||||
databaseId: string,
|
||||
relationshipId: string,
|
||||
status: CatalogRelationshipStatus,
|
||||
) => apiFetch<CatalogRelationship>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/${encodeURIComponent(relationshipId)}`,
|
||||
{ method: "PATCH", body: JSON.stringify({ status }) },
|
||||
);
|
||||
|
||||
export const deleteCatalogRelationship = (databaseId: string, relationshipId: string) =>
|
||||
apiFetch<void>(
|
||||
`/catalog/databases/${encodeURIComponent(databaseId)}/relationships/${encodeURIComponent(relationshipId)}`,
|
||||
{ method: "DELETE" },
|
||||
);
|
||||
|
||||
export const deleteCatalogDatabaseMetadata = (
|
||||
databaseIds: string[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
|
||||
@@ -152,6 +152,15 @@ test.each([
|
||||
["sensitive_data_suggestion_history_request_invalid", "Sensitive suggestion history parameters are invalid."],
|
||||
["sensitive_data_suggestion_history_failed", "Sensitive suggestion history could not be loaded."],
|
||||
["sensitive_data_suggestion_run_not_found", "The sensitive suggestion run was not found."],
|
||||
["relationship_not_found", "The relationship no longer exists. Refresh and try again."],
|
||||
["relationship_duplicate", "This relationship already exists."],
|
||||
["relationship_target_not_unique", "The target column must be the only primary-key column of its table."],
|
||||
["relationship_type_incompatible", "Source and target column types are not compatible."],
|
||||
["relationship_read_only", "Physical relationships are read-only."],
|
||||
["relationship_request_invalid", "The relationship request is invalid."],
|
||||
["relationship_operation_failed", "The relationship operation failed."],
|
||||
["relationship_schema_stale", "Synchronize the current database schema before managing logical relationships."],
|
||||
["column_not_found", "The selected catalog column was not found. Refresh and try again."],
|
||||
])("maps the catalog error code %s to safe local copy", async (code, message) => {
|
||||
const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue(
|
||||
new Response(JSON.stringify({ code, message: "provider detail must not be trusted" }), {
|
||||
|
||||
@@ -35,6 +35,9 @@ const safeErrorCodes = new Set([
|
||||
"sensitive_data_suggestion_run_not_found",
|
||||
"schema_sync_conflict", "schema_introspection_failed", "schema_request_invalid",
|
||||
"schema_operation_failed", "sync_run_not_found", "table_stale", "column_stale",
|
||||
"relationship_not_found", "relationship_duplicate", "relationship_target_not_unique",
|
||||
"relationship_type_incompatible", "relationship_read_only", "relationship_request_invalid",
|
||||
"relationship_operation_failed", "relationship_schema_stale", "column_not_found",
|
||||
]);
|
||||
|
||||
type SafeErrorPayload = {
|
||||
@@ -104,6 +107,15 @@ const localCodeMessages: Record<string, string> = {
|
||||
sync_run_not_found: "The synchronization run was not found.",
|
||||
table_stale: "Table metadata changed. Reload and try again.",
|
||||
column_stale: "Column metadata changed. Reload and try again.",
|
||||
relationship_not_found: "The relationship no longer exists. Refresh and try again.",
|
||||
relationship_duplicate: "This relationship already exists.",
|
||||
relationship_target_not_unique: "The target column must be the only primary-key column of its table.",
|
||||
relationship_type_incompatible: "Source and target column types are not compatible.",
|
||||
relationship_read_only: "Physical relationships are read-only.",
|
||||
relationship_request_invalid: "The relationship request is invalid.",
|
||||
relationship_operation_failed: "The relationship operation failed.",
|
||||
relationship_schema_stale: "Synchronize the current database schema before managing logical relationships.",
|
||||
column_not_found: "The selected catalog column was not found. Refresh and try again.",
|
||||
};
|
||||
|
||||
const localStatusMessages: Record<number, string> = {
|
||||
|
||||
@@ -13,6 +13,8 @@ const buttonVariants = cva(
|
||||
"border-border bg-card shadow-xs hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:border-input dark:bg-input/30 dark:hover:bg-input/50",
|
||||
secondary:
|
||||
"bg-secondary text-secondary-foreground hover:bg-[color-mix(in_oklch,oklch(var(--secondary)),oklch(var(--foreground))_5%)] aria-expanded:bg-secondary aria-expanded:text-secondary-foreground",
|
||||
navigationActive:
|
||||
"border-[oklch(var(--nav-active-border))] bg-[oklch(var(--nav-active))] text-[oklch(var(--nav-active-foreground))] shadow-xs hover:bg-[oklch(var(--nav-active-hover))] hover:text-[oklch(var(--nav-active-foreground))] focus-visible:border-[oklch(var(--nav-active-border))] focus-visible:ring-[oklch(var(--nav-active-border)/0.32)]",
|
||||
ghost:
|
||||
"hover:bg-muted hover:text-foreground aria-expanded:bg-muted aria-expanded:text-foreground dark:hover:bg-muted/50",
|
||||
destructive:
|
||||
|
||||
@@ -47,6 +47,12 @@
|
||||
--success: 0.7577 0.1581 165.0;
|
||||
--warning: 0.8523 0.1386 78.9;
|
||||
--info: 0.7035 0.1128 221.3;
|
||||
/* Current navigation shares Instrument Red's hue, with lower chroma and
|
||||
higher lightness so selection reads as a muted location signal. */
|
||||
--nav-active: 0.9250 0.0520 23.2;
|
||||
--nav-active-hover: 0.8950 0.0710 23.2;
|
||||
--nav-active-foreground: 0.3650 0.1100 23.2;
|
||||
--nav-active-border: 0.6000 0.1350 23.2;
|
||||
--radius: 0.5rem;
|
||||
--sidebar: 0.9709 0.0011 17.2;
|
||||
--sidebar-foreground: 0.2678 0.0097 355.6;
|
||||
@@ -81,6 +87,10 @@
|
||||
--border: 0.3715 0 90;
|
||||
--input: 0.3715 0 90;
|
||||
--ring: 0.6897 0.1779 16.9;
|
||||
--nav-active: 0.3350 0.0550 20.3;
|
||||
--nav-active-hover: 0.3750 0.0650 20.3;
|
||||
--nav-active-foreground: 0.8900 0.0700 20.3;
|
||||
--nav-active-border: 0.6800 0.1350 20.3;
|
||||
--sidebar: 0.2350 0 90;
|
||||
--sidebar-foreground: 0.9310 0 90;
|
||||
--sidebar-primary: 0.6023 0.1848 20.3;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { act, render, screen, waitFor } from "@testing-library/react";
|
||||
import { act, render, screen, waitFor, within } from "@testing-library/react";
|
||||
import userEvent from "@testing-library/user-event";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { http, HttpResponse } from "msw";
|
||||
@@ -45,24 +45,50 @@ beforeEach(() => {
|
||||
});
|
||||
|
||||
describe("authenticated shell permissions", () => {
|
||||
test("shows only read-safe workspace chrome to a session user", async () => {
|
||||
test("hides administrative navigation from a session user", async () => {
|
||||
renderShell({ subject: "user-1", isAdmin: false, roles: ["user"], permissions: ["session.use"] });
|
||||
|
||||
expect(await screen.findByRole("button", { name: "Workspace management" })).toBeInTheDocument();
|
||||
expect(await screen.findByRole("button", { name: "New session" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("shows management and all-session chrome only for exact permissions", async () => {
|
||||
test("shows the ordered administrative accordion only to an admin", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderShell({
|
||||
subject: "admin-1",
|
||||
isAdmin: true,
|
||||
roles: ["admin"],
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "pi.manage"],
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"],
|
||||
});
|
||||
|
||||
expect(await screen.findByRole("button", { name: "Pi management" })).toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toBeInTheDocument();
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const trigger = await within(sessionNavigation).findByRole("button", { name: "Administration" });
|
||||
expect(trigger).toHaveAttribute("aria-expanded", "false");
|
||||
expect(within(sessionNavigation).queryByRole("region", { name: "Administration" })).not.toBeInTheDocument();
|
||||
|
||||
trigger.focus();
|
||||
await user.keyboard("{Enter}");
|
||||
expect(trigger).toHaveAttribute("aria-expanded", "true");
|
||||
|
||||
const panel = within(sessionNavigation).getByRole("region", { name: "Administration" });
|
||||
const database = within(panel).getByRole("button", { name: "Database management" });
|
||||
const separator = within(panel).getByRole("separator");
|
||||
const workspace = within(panel).getByRole("button", { name: "Workspace management" });
|
||||
const pi = within(panel).getByRole("button", { name: "Pi management" });
|
||||
expect(panel.children[0]).toBe(database);
|
||||
expect(panel.children[1]).toBe(separator);
|
||||
expect(panel.children[2]).toBe(workspace);
|
||||
expect(panel.children[3]).toBe(pi);
|
||||
expect(screen.getByRole("tab", { name: "All sessions" })).toBeInTheDocument();
|
||||
|
||||
await user.keyboard(" ");
|
||||
expect(trigger).toHaveAttribute("aria-expanded", "false");
|
||||
expect(within(sessionNavigation).queryByRole("region", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(within(sessionNavigation).queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("keeps identity but hides logout outside local authentication", async () => {
|
||||
@@ -178,15 +204,14 @@ describe("authenticated shell permissions", () => {
|
||||
subject: "admin-1", isAdmin: true, roles: ["admin"],
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "pi.manage"],
|
||||
});
|
||||
expect(await screen.findByRole("button", { name: "Pi management" })).toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toBeInTheDocument();
|
||||
expect(await screen.findByRole("button", { name: "Administration" })).toBeInTheDocument();
|
||||
expect(screen.getByRole("tab", { name: "All sessions" })).toBeInTheDocument();
|
||||
|
||||
act(() => clearAuthState());
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
await userEvent.click(screen.getByRole("button", { name: "Workspace management" }));
|
||||
expect(await screen.findByRole("heading", { name: "Workspace management" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Update workspace repository" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("remounts the real shell so user-A panel and transcript state cannot survive user-B", async () => {
|
||||
|
||||
@@ -17,6 +17,10 @@ function renderShell() {
|
||||
);
|
||||
}
|
||||
|
||||
async function expandAdministration() {
|
||||
await userEvent.click(screen.getByRole("button", { name: "Administration" }));
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
clearAuthState();
|
||||
setAuthState({
|
||||
@@ -92,10 +96,19 @@ test("uses Fleet Ledger by default and gates the legacy fallback to non-producti
|
||||
|
||||
test("replaces the core conversation while keeping the session navigation", async () => {
|
||||
renderShell();
|
||||
await expandAdministration();
|
||||
|
||||
const conversationColumn = screen.getByTestId("conversation-column");
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
|
||||
const databaseManagement = within(sessionNavigation).getByRole("button", { name: "Database management" });
|
||||
|
||||
expect(newSession).toHaveAttribute("aria-current", "page");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(newSession).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(databaseManagement).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(databaseManagement).not.toHaveAttribute("aria-current");
|
||||
|
||||
await userEvent.click(databaseManagement);
|
||||
|
||||
const manager = screen.getByRole("main", { name: "Database management" });
|
||||
@@ -103,8 +116,13 @@ test("replaces the core conversation while keeping the session navigation", asyn
|
||||
expect(conversationColumn).toContainElement(manager);
|
||||
expect(screen.getByRole("complementary", { name: "Session navigation" })).toBe(sessionNavigation);
|
||||
expect(sessionNavigation).toBeVisible();
|
||||
expect(within(sessionNavigation).getByRole("button", { name: "New session" })).toBeVisible();
|
||||
expect(newSession).toBeVisible();
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(newSession).not.toHaveAttribute("aria-current");
|
||||
expect(newSession).not.toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(databaseManagement).toHaveAttribute("aria-current", "page");
|
||||
expect(databaseManagement).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(databaseManagement).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(screen.queryByRole("button", { name: "Back to workspace" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("textbox", { name: /new question/i })).not.toBeInTheDocument();
|
||||
});
|
||||
@@ -135,6 +153,7 @@ test("keeps a live core session connected while returning from database manageme
|
||||
})),
|
||||
);
|
||||
renderShell();
|
||||
await expandAdministration();
|
||||
|
||||
const session = await screen.findByTestId("session-item-s1");
|
||||
await userEvent.click(session);
|
||||
@@ -157,7 +176,7 @@ test("keeps a live core session connected while returning from database manageme
|
||||
expect(FakeEventSource.instances).toHaveLength(1);
|
||||
});
|
||||
|
||||
test("keeps read-safe workspace access but hides privileged management entries", () => {
|
||||
test("hides the complete administrative navigation from a regular user", () => {
|
||||
clearAuthState();
|
||||
setAuthState({
|
||||
issuer: "test",
|
||||
@@ -171,7 +190,9 @@ test("keeps read-safe workspace access but hides privileged management entries",
|
||||
|
||||
renderShell();
|
||||
|
||||
expect(screen.getByRole("button", { name: "Workspace management" })).toBeInTheDocument();
|
||||
expect(screen.getByRole("button", { name: "New session" })).toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Workspace management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Database management" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Pi management" })).not.toBeInTheDocument();
|
||||
});
|
||||
@@ -182,6 +203,13 @@ test("does not leave database management without confirming a dirty form", async
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "ready",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
@@ -202,6 +230,7 @@ test("does not leave database management without confirming a dirty form", async
|
||||
}])));
|
||||
const confirm = vi.spyOn(window, "confirm").mockReturnValue(false);
|
||||
renderShell();
|
||||
await expandAdministration();
|
||||
|
||||
await userEvent.click(screen.getByRole("button", { name: "Database management" }));
|
||||
await userEvent.click(await screen.findByRole("button", { name: "Edit Policlinico San Donato" }));
|
||||
|
||||
@@ -156,6 +156,11 @@ test("model selector shows the three Pi-enabled models and stores the selected p
|
||||
});
|
||||
|
||||
test("opens Workspace management from the right sidebar without interrupting the shell", async () => {
|
||||
setAuthState({
|
||||
issuer: "test", subject: "admin", roles: ["admin"],
|
||||
permissions: ["session.use", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"],
|
||||
isAdmin: true, csrfToken: null, session: null,
|
||||
});
|
||||
server.use(
|
||||
http.get("/api/workspace-registry/status", () =>
|
||||
HttpResponse.json({ branch: "main", ahead: 0, behind: 0, degraded: false })),
|
||||
@@ -163,15 +168,31 @@ test("opens Workspace management from the right sidebar without interrupting the
|
||||
);
|
||||
renderShell();
|
||||
|
||||
await userEvent.click(screen.getByRole("button", { name: "Administration" }));
|
||||
await userEvent.click(screen.getByRole("button", { name: "Workspace management" }));
|
||||
|
||||
expect(await screen.findByRole("heading", { name: "Workspace management" })).toBeVisible();
|
||||
const dialog = screen.getByRole("dialog", { name: "Workspace management" });
|
||||
const workArea = screen.getByTestId("conversation-column");
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const workspaceManagement = within(sessionNavigation).getByRole("button", { name: "Workspace management" });
|
||||
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
|
||||
expect(workArea).toContainElement(dialog);
|
||||
expect(sessionNavigation).not.toContainElement(dialog);
|
||||
expect(screen.getByTestId("app-shell")).toHaveAttribute("data-activity-layout", "closed");
|
||||
expect(workspaceManagement).toHaveAttribute("aria-current", "page");
|
||||
expect(workspaceManagement).toHaveAttribute("aria-expanded", "true");
|
||||
expect(workspaceManagement).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(workspaceManagement).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(newSession).not.toHaveAttribute("aria-current");
|
||||
|
||||
await userEvent.click(screen.getByRole("button", { name: "Close workspace management" }));
|
||||
await waitFor(() => expect(screen.queryByRole("dialog", { name: "Workspace management" })).not.toBeInTheDocument());
|
||||
expect(newSession).toHaveAttribute("aria-current", "page");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(workspaceManagement).toHaveAttribute("aria-expanded", "false");
|
||||
expect(workspaceManagement).toHaveAttribute("data-navigation-state", "available");
|
||||
});
|
||||
|
||||
test("does not block the shell when the DWH is unavailable at startup", async () => {
|
||||
@@ -184,7 +205,8 @@ test("does not block the shell when the DWH is unavailable at startup", async ()
|
||||
);
|
||||
renderShell();
|
||||
|
||||
expect(await screen.findByRole("button", { name: "Workspace management" })).toBeVisible();
|
||||
expect(await screen.findByRole("button", { name: "New session" })).toBeVisible();
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("heading", { name: "Connection unavailable" })).not.toBeInTheDocument();
|
||||
expect(healthChecks).toBe(0);
|
||||
});
|
||||
|
||||
@@ -16,7 +16,7 @@ const regularUser: AuthenticatedUser = {
|
||||
};
|
||||
const adminUser: AuthenticatedUser = {
|
||||
...regularUser, subject: "alice-id", roles: ["admin"] as const,
|
||||
permissions: ["session.use", "session.read_all", "pi.manage"] as const, isAdmin: true,
|
||||
permissions: ["session.use", "session.read_all", "workspace.manage", "workspace.secrets.manage", "database.manage", "pi.manage"] as const, isAdmin: true,
|
||||
};
|
||||
|
||||
function wrap(user: AuthenticatedUser = regularUser) {
|
||||
@@ -95,7 +95,8 @@ test("regular users load only their sessions and never see administrator control
|
||||
wrap();
|
||||
await screen.findByText("Attiva uno");
|
||||
expect(scope).toBe("mine");
|
||||
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Administration" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("tab", { name: "All sessions" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByText(/administrator view/i)).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
@@ -115,17 +116,25 @@ test("Pi management preserves the open session summary and the model activity ti
|
||||
http.post("/api/sessions/:id/resume", () => resumeResult("s1")),
|
||||
http.get("/api/sessions/:id", () => HttpResponse.json({ phase: 4 })),
|
||||
);
|
||||
wrap();
|
||||
wrap(adminUser);
|
||||
|
||||
await user.click(await screen.findByText("Attiva uno"));
|
||||
expect(await screen.findByRole("complementary", { name: "Session summary" })).toHaveTextContent("Domanda originale");
|
||||
await user.click(screen.getByRole("button", { name: "Administration" }));
|
||||
await user.click(screen.getByRole("button", { name: "Pi management" }));
|
||||
expect(await screen.findByRole("heading", { name: "Pi management" })).toBeVisible();
|
||||
const dialog = screen.getByRole("dialog", { name: "Pi management" });
|
||||
const workArea = screen.getByTestId("conversation-column");
|
||||
const sessionNavigation = screen.getByRole("complementary", { name: "Session navigation" });
|
||||
const piManagement = within(sessionNavigation).getByRole("button", { name: "Pi management" });
|
||||
const newSession = within(sessionNavigation).getByRole("button", { name: "New session" });
|
||||
expect(workArea).toContainElement(dialog);
|
||||
expect(sessionNavigation).not.toContainElement(dialog);
|
||||
expect(piManagement).toHaveAttribute("aria-current", "page");
|
||||
expect(piManagement).toHaveAttribute("aria-expanded", "true");
|
||||
expect(piManagement).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "available");
|
||||
expect(newSession).not.toHaveAttribute("aria-current");
|
||||
const preservedSummary = screen.getByRole("complementary", { name: "Session summary" });
|
||||
expect(preservedSummary).not.toHaveAttribute("aria-hidden", "true");
|
||||
expect(preservedSummary).toHaveTextContent("Attiva uno");
|
||||
@@ -133,6 +142,10 @@ test("Pi management preserves the open session summary and the model activity ti
|
||||
await waitFor(() => {
|
||||
expect(screen.getByRole("complementary", { name: "Session summary" })).toHaveTextContent("Domanda originale");
|
||||
});
|
||||
expect(newSession).toHaveAttribute("aria-current", "page");
|
||||
expect(newSession).toHaveAttribute("data-navigation-state", "current");
|
||||
expect(piManagement).toHaveAttribute("aria-expanded", "false");
|
||||
expect(piManagement).toHaveAttribute("data-navigation-state", "available");
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Resume" }));
|
||||
await screen.findByRole("button", { name: "Show model activity" });
|
||||
@@ -160,17 +173,66 @@ test("administrators can explicitly switch to all sessions and see owners", asyn
|
||||
}),
|
||||
);
|
||||
wrap(adminUser);
|
||||
await screen.findByRole("button", { name: "All sessions" });
|
||||
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "true");
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "false");
|
||||
await userEvent.click(screen.getByRole("button", { name: "All sessions" }));
|
||||
const tablist = await screen.findByRole("tablist", { name: "Session scope" });
|
||||
const mySessions = within(tablist).getByRole("tab", { name: "My sessions" });
|
||||
const allSessions = within(tablist).getByRole("tab", { name: "All sessions" });
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "true");
|
||||
expect(mySessions).toHaveAttribute("aria-controls", "session-scope-panel");
|
||||
expect(mySessions).toHaveAttribute("tabindex", "0");
|
||||
expect(mySessions).toHaveAttribute("data-tab-state", "active");
|
||||
expect(mySessions).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "false");
|
||||
expect(allSessions).toHaveAttribute("aria-controls", "session-scope-panel");
|
||||
expect(allSessions).toHaveAttribute("tabindex", "-1");
|
||||
expect(allSessions).toHaveAttribute("data-tab-state", "inactive");
|
||||
expect(allSessions).toHaveClass("border-border", "bg-card");
|
||||
expect(allSessions).not.toHaveClass("border-transparent", "bg-transparent");
|
||||
expect(screen.getByRole("tabpanel")).toHaveAttribute("aria-labelledby", "session-scope-mine-tab");
|
||||
await userEvent.click(allSessions);
|
||||
await waitFor(() => expect(scope).toBe("all"));
|
||||
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "false");
|
||||
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "true");
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "false");
|
||||
expect(mySessions).toHaveAttribute("data-tab-state", "inactive");
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "true");
|
||||
expect(allSessions).toHaveAttribute("data-tab-state", "active");
|
||||
expect(allSessions).toHaveClass("bg-[oklch(var(--nav-active))]");
|
||||
expect(screen.getByRole("tabpanel")).toHaveAttribute("aria-labelledby", "session-scope-all-tab");
|
||||
expect(await screen.findByText("Administrator view: all sessions")).toBeInTheDocument();
|
||||
expect(screen.getByText("Owner: Bob")).toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("session scope tabs support roving keyboard navigation", async () => {
|
||||
let scope = "";
|
||||
server.use(http.get("/api/sessions", ({ request }) => {
|
||||
scope = new URL(request.url).searchParams.get("scope") ?? "";
|
||||
return HttpResponse.json(LIST);
|
||||
}));
|
||||
wrap(adminUser);
|
||||
|
||||
const user = userEvent.setup();
|
||||
const tablist = await screen.findByRole("tablist", { name: "Session scope" });
|
||||
const mySessions = within(tablist).getByRole("tab", { name: "My sessions" });
|
||||
const allSessions = within(tablist).getByRole("tab", { name: "All sessions" });
|
||||
mySessions.focus();
|
||||
|
||||
await user.keyboard("{ArrowRight}");
|
||||
await waitFor(() => expect(scope).toBe("all"));
|
||||
expect(allSessions).toHaveFocus();
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "true");
|
||||
expect(mySessions).toHaveAttribute("tabindex", "-1");
|
||||
|
||||
await user.keyboard("{ArrowLeft}");
|
||||
await waitFor(() => expect(scope).toBe("mine"));
|
||||
expect(mySessions).toHaveFocus();
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "true");
|
||||
|
||||
await user.keyboard("{End}");
|
||||
expect(allSessions).toHaveFocus();
|
||||
expect(allSessions).toHaveAttribute("aria-selected", "true");
|
||||
await user.keyboard("{Home}");
|
||||
expect(mySessions).toHaveFocus();
|
||||
expect(mySessions).toHaveAttribute("aria-selected", "true");
|
||||
});
|
||||
|
||||
test("administrator confirms before deleting a same-named user's session", async () => {
|
||||
let deletes = 0;
|
||||
server.use(
|
||||
@@ -187,7 +249,7 @@ test("administrator confirms before deleting a same-named user's session", async
|
||||
}),
|
||||
);
|
||||
wrap(adminUser);
|
||||
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
|
||||
await userEvent.click(await screen.findByRole("tab", { name: "All sessions" }));
|
||||
await screen.findByText("Owner: Alice");
|
||||
await userEvent.click(screen.getByRole("checkbox", { name: "Select Attiva uno" }));
|
||||
await userEvent.click(screen.getByRole("button", { name: "Delete 1 selected sessions" }));
|
||||
@@ -213,7 +275,7 @@ test("administrator confirms before archiving a same-named user's session", asyn
|
||||
}),
|
||||
);
|
||||
wrap(adminUser);
|
||||
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
|
||||
await userEvent.click(await screen.findByRole("tab", { name: "All sessions" }));
|
||||
await screen.findByText("Owner: Alice");
|
||||
await userEvent.click(screen.getByRole("button", { name: "Session actions" }));
|
||||
await userEvent.click(await screen.findByText("Archive"));
|
||||
|
||||
+193
-90
@@ -17,8 +17,9 @@ import { StopConfirmDialog } from "./StopConfirmDialog";
|
||||
import { SteerInput, ComposerFooter } from "./SteerInput";
|
||||
import { WorkflowBar } from "./WorkflowBar";
|
||||
import { DatabaseManagementPage } from "./DatabaseManagementPage";
|
||||
import { Pencil, ArrowLeft, ArrowRight, Trash2 } from "lucide-react";
|
||||
import { Button } from "../components/ui/button";
|
||||
import { Pencil, ArrowLeft, ArrowRight, ChevronDown, Trash2 } from "lucide-react";
|
||||
import { Accordion } from "@base-ui/react/accordion";
|
||||
import { Button, buttonVariants } from "../components/ui/button";
|
||||
import { Checkbox } from "../components/ui/checkbox";
|
||||
import { Toaster } from "../components/ui/sonner";
|
||||
import { toast } from "sonner";
|
||||
@@ -34,7 +35,7 @@ import type { SessionScope, SessionSummary } from "../api/types";
|
||||
import { useAuthGeneration, useAuthUser } from "../auth/authState";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import type { CSSProperties } from "react";
|
||||
import type { CSSProperties, KeyboardEvent } from "react";
|
||||
import { captureAuthOperation, isAuthOperationCurrent, StaleAuthOperationError, type AuthOperationGuard } from "../auth/authOperation";
|
||||
|
||||
interface AppShellProps {
|
||||
@@ -43,6 +44,14 @@ interface AppShellProps {
|
||||
|
||||
type ActiveSurface = "core" | "database-management";
|
||||
type ActiveManagementPanel = "workspace" | "pi" | null;
|
||||
const SESSION_SCOPES: readonly SessionScope[] = ["mine", "all"];
|
||||
|
||||
function sessionScopeTabClass(selected: boolean): string {
|
||||
const base = "relative -mb-px flex h-8 w-full cursor-pointer items-center justify-center rounded-t-md border border-b-2 px-2 text-xs font-semibold tracking-[0.005em] outline-none focus-visible:z-10 focus-visible:ring-3 focus-visible:ring-[oklch(var(--nav-active-border)/0.28)]";
|
||||
return selected
|
||||
? `${base} border-[oklch(var(--nav-active-border))] bg-[oklch(var(--nav-active))] text-[oklch(var(--nav-active-foreground))] hover:bg-[oklch(var(--nav-active-hover))]`
|
||||
: `${base} border-border bg-card text-muted-foreground hover:border-muted-foreground/45 hover:bg-muted/55 hover:text-foreground`;
|
||||
}
|
||||
|
||||
type DatabaseManagementPresentation = "legacy" | "fleet";
|
||||
|
||||
@@ -129,8 +138,13 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
const [creatingSession, setCreatingSession] = useState(false);
|
||||
const [awaitingQuestion, setAwaitingQuestion] = useState(false);
|
||||
const [sessionScope, setSessionScope] = useState<SessionScope>("mine");
|
||||
const sessionScopeTabRefs = useRef<Record<SessionScope, HTMLButtonElement | null>>({
|
||||
mine: null,
|
||||
all: null,
|
||||
});
|
||||
const principal = authenticatedUser;
|
||||
const permissions = authenticatedUser?.permissions ?? [];
|
||||
const isAdmin = authenticatedUser?.isAdmin === true;
|
||||
const canReadAllSessions = permissions.includes("session.read_all");
|
||||
const canManageWorkspace = permissions.includes("workspace.manage");
|
||||
const canManageWorkspaceSecrets = permissions.includes("workspace.secrets.manage");
|
||||
@@ -164,6 +178,7 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
databaseNavigationRef.current = state;
|
||||
}, []);
|
||||
const [activeManagementPanel, setActiveManagementPanel] = useState<ActiveManagementPanel>(null);
|
||||
const [adminNavigationValue, setAdminNavigationValue] = useState<string[]>([]);
|
||||
const [activeOpen, setActiveOpen] = useState(true);
|
||||
const [archiveOpen, setArchiveOpen] = useState(false);
|
||||
const [renameTarget, setRenameTarget] = useState<SessionSummary | null>(null);
|
||||
@@ -236,6 +251,22 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
setSelectedSessionIds(selected ? new Set(sessions.map((session) => session.id)) : new Set());
|
||||
}
|
||||
|
||||
function handleSessionScopeTabKeyDown(
|
||||
event: KeyboardEvent<HTMLButtonElement>,
|
||||
current: SessionScope,
|
||||
) {
|
||||
const currentIndex = SESSION_SCOPES.indexOf(current);
|
||||
let target: SessionScope | undefined;
|
||||
if (event.key === "ArrowRight") target = SESSION_SCOPES[(currentIndex + 1) % SESSION_SCOPES.length];
|
||||
else if (event.key === "ArrowLeft") target = SESSION_SCOPES[(currentIndex - 1 + SESSION_SCOPES.length) % SESSION_SCOPES.length];
|
||||
else if (event.key === "Home") target = SESSION_SCOPES[0];
|
||||
else if (event.key === "End") target = SESSION_SCOPES.at(-1);
|
||||
if (!target) return;
|
||||
event.preventDefault();
|
||||
setSessionScope(target);
|
||||
sessionScopeTabRefs.current[target]?.focus();
|
||||
}
|
||||
|
||||
function canLeaveDatabaseManagement(): boolean {
|
||||
if (activeSurface !== "database-management") return true;
|
||||
if (databaseNavigationRef.current.busy) {
|
||||
@@ -627,6 +658,12 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
await logoutUser();
|
||||
}
|
||||
|
||||
const currentNavigation = activeSurface === "database-management"
|
||||
? "database"
|
||||
: activeManagementPanel ?? "core";
|
||||
const adminNavigationOpen = adminNavigationValue.includes("administration");
|
||||
const managementNavigationCurrent = currentNavigation !== "core";
|
||||
|
||||
return (
|
||||
<div
|
||||
ref={containerRef}
|
||||
@@ -808,115 +845,180 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
|
||||
<div className="flex flex-col gap-2 px-4 pb-3">
|
||||
<Button
|
||||
variant="default"
|
||||
variant={currentNavigation === "core" ? "navigationActive" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "core" ? "page" : undefined}
|
||||
data-navigation-state={currentNavigation === "core" ? "current" : "available"}
|
||||
onClick={startNewSession}
|
||||
>
|
||||
New session
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="w-full"
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("workspace");
|
||||
}}
|
||||
>
|
||||
Workspace management
|
||||
</Button>
|
||||
{canManageDatabase && (
|
||||
<Button
|
||||
variant={activeSurface === "database-management" ? "secondary" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={activeSurface === "database-management" ? "page" : undefined}
|
||||
onClick={() => {
|
||||
setActiveManagementPanel(null);
|
||||
setActiveSurface("database-management");
|
||||
}}
|
||||
{isAdmin && (
|
||||
<Accordion.Root
|
||||
value={adminNavigationValue}
|
||||
onValueChange={setAdminNavigationValue}
|
||||
className="rounded-lg border border-border/80 bg-card/45 p-1.5"
|
||||
>
|
||||
Database management
|
||||
</Button>
|
||||
)}
|
||||
{canManagePi && (
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="w-full"
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("pi");
|
||||
}}
|
||||
>
|
||||
Pi management
|
||||
</Button>
|
||||
<Accordion.Item value="administration">
|
||||
<Accordion.Header className="m-0">
|
||||
<Accordion.Trigger
|
||||
className={buttonVariants({
|
||||
variant: !adminNavigationOpen && managementNavigationCurrent ? "navigationActive" : "outline",
|
||||
size: "sm",
|
||||
className: "w-full justify-between px-2.5",
|
||||
})}
|
||||
data-navigation-state={!adminNavigationOpen && managementNavigationCurrent ? "current" : "available"}
|
||||
>
|
||||
<span>Administration</span>
|
||||
<ChevronDown
|
||||
aria-hidden="true"
|
||||
data-icon="inline-end"
|
||||
className={`transition-transform duration-150 motion-reduce:transition-none ${adminNavigationOpen ? "rotate-180" : ""}`}
|
||||
/>
|
||||
</Accordion.Trigger>
|
||||
</Accordion.Header>
|
||||
<Accordion.Panel className="grid gap-1.5 px-0.5 pt-1.5">
|
||||
<Button
|
||||
variant={currentNavigation === "database" ? "navigationActive" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "database" ? "page" : undefined}
|
||||
data-navigation-state={currentNavigation === "database" ? "current" : canManageDatabase ? "available" : "unavailable"}
|
||||
disabled={!canManageDatabase}
|
||||
title={canManageDatabase ? undefined : "Database management permission is required"}
|
||||
onClick={() => {
|
||||
setActiveManagementPanel(null);
|
||||
setActiveSurface("database-management");
|
||||
}}
|
||||
>
|
||||
Database management
|
||||
</Button>
|
||||
<div role="separator" aria-orientation="horizontal" className="mx-1 h-px bg-border/90" />
|
||||
<Button
|
||||
variant={currentNavigation === "workspace" ? "navigationActive" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "workspace" ? "page" : undefined}
|
||||
aria-expanded={activeManagementPanel === "workspace"}
|
||||
data-navigation-state={currentNavigation === "workspace" ? "current" : "available"}
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("workspace");
|
||||
}}
|
||||
>
|
||||
Workspace management
|
||||
</Button>
|
||||
<Button
|
||||
variant={currentNavigation === "pi" ? "navigationActive" : "outline"}
|
||||
size="sm"
|
||||
className="w-full"
|
||||
aria-current={currentNavigation === "pi" ? "page" : undefined}
|
||||
aria-expanded={activeManagementPanel === "pi"}
|
||||
data-navigation-state={currentNavigation === "pi" ? "current" : canManagePi ? "available" : "unavailable"}
|
||||
disabled={!canManagePi}
|
||||
title={canManagePi ? undefined : "Pi management permission is required"}
|
||||
onClick={() => {
|
||||
if (!canLeaveDatabaseManagement()) return;
|
||||
setActiveSurface("core");
|
||||
setActiveManagementPanel("pi");
|
||||
}}
|
||||
>
|
||||
Pi management
|
||||
</Button>
|
||||
</Accordion.Panel>
|
||||
</Accordion.Item>
|
||||
</Accordion.Root>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{canReadAllSessions && (
|
||||
<div className="px-4 pb-3">
|
||||
<div className="grid grid-cols-2 gap-1 rounded-lg bg-muted p-1" aria-label="Session scope">
|
||||
<Button
|
||||
variant={sessionScope === "mine" ? "secondary" : "ghost"}
|
||||
size="xs"
|
||||
aria-pressed={sessionScope === "mine"}
|
||||
<div className="px-4">
|
||||
<div
|
||||
role="tablist"
|
||||
aria-label="Session scope"
|
||||
aria-orientation="horizontal"
|
||||
className="grid grid-cols-2 border-b border-border"
|
||||
>
|
||||
<button
|
||||
ref={(node) => { sessionScopeTabRefs.current.mine = node; }}
|
||||
id="session-scope-mine-tab"
|
||||
type="button"
|
||||
role="tab"
|
||||
aria-selected={sessionScope === "mine"}
|
||||
aria-controls="session-scope-panel"
|
||||
tabIndex={sessionScope === "mine" ? 0 : -1}
|
||||
data-tab-state={sessionScope === "mine" ? "active" : "inactive"}
|
||||
className={sessionScopeTabClass(sessionScope === "mine")}
|
||||
onClick={() => setSessionScope("mine")}
|
||||
onKeyDown={(event) => handleSessionScopeTabKeyDown(event, "mine")}
|
||||
>
|
||||
My sessions
|
||||
</Button>
|
||||
<Button
|
||||
variant={showingAllSessions ? "secondary" : "ghost"}
|
||||
size="xs"
|
||||
aria-pressed={showingAllSessions}
|
||||
</button>
|
||||
<button
|
||||
ref={(node) => { sessionScopeTabRefs.current.all = node; }}
|
||||
id="session-scope-all-tab"
|
||||
type="button"
|
||||
role="tab"
|
||||
aria-selected={showingAllSessions}
|
||||
aria-controls="session-scope-panel"
|
||||
tabIndex={showingAllSessions ? 0 : -1}
|
||||
data-tab-state={showingAllSessions ? "active" : "inactive"}
|
||||
className={sessionScopeTabClass(showingAllSessions)}
|
||||
onClick={() => setSessionScope("all")}
|
||||
onKeyDown={(event) => handleSessionScopeTabKeyDown(event, "all")}
|
||||
>
|
||||
All sessions
|
||||
</Button>
|
||||
</button>
|
||||
</div>
|
||||
{showingAllSessions && (
|
||||
<p className="mt-2 text-xs font-medium text-amber-700 dark:text-amber-400">
|
||||
Administrator view: all sessions
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* L1 — rail title */}
|
||||
<div className="px-4 pb-1.5 pt-1">
|
||||
<span className="thot-label text-[0.8rem] font-bold tracking-[0.18em] text-primary">
|
||||
Sessions
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between px-4 pb-2">
|
||||
<label className="flex cursor-pointer items-center gap-2 text-xs font-medium text-muted-foreground hover:text-foreground">
|
||||
<Checkbox
|
||||
checked={allSessionsSelected}
|
||||
aria-label="Select all sessions"
|
||||
disabled={sessions.length === 0}
|
||||
onCheckedChange={(selected) => toggleAllSessions(selected === true)}
|
||||
/>
|
||||
<span>Select all</span>
|
||||
</label>
|
||||
{selectedSessions.length > 0 && (
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="xs"
|
||||
aria-label={`Delete ${selectedSessions.length} selected sessions`}
|
||||
onClick={() => {
|
||||
requestDelete(selectedSessions);
|
||||
}}
|
||||
>
|
||||
<Trash2 />
|
||||
Delete ({selectedSessions.length})
|
||||
</Button>
|
||||
<div
|
||||
id={canReadAllSessions ? "session-scope-panel" : undefined}
|
||||
role={canReadAllSessions ? "tabpanel" : undefined}
|
||||
aria-labelledby={canReadAllSessions ? `session-scope-${sessionScope}-tab` : undefined}
|
||||
className="flex min-h-0 flex-1 flex-col"
|
||||
>
|
||||
{canReadAllSessions && showingAllSessions && (
|
||||
<p className="mx-4 mb-2 mt-2 text-xs font-medium text-amber-700 dark:text-amber-400">
|
||||
Administrator view: all sessions
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex-1 overflow-y-auto px-2 pb-4">
|
||||
|
||||
{/* L1 — rail title */}
|
||||
<div className="px-4 pb-1.5 pt-1">
|
||||
<span className="thot-label text-[0.8rem] font-bold tracking-[0.18em] text-primary">
|
||||
Sessions
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between px-4 pb-2">
|
||||
<label className="flex cursor-pointer items-center gap-2 text-xs font-medium text-muted-foreground hover:text-foreground">
|
||||
<Checkbox
|
||||
checked={allSessionsSelected}
|
||||
aria-label="Select all sessions"
|
||||
disabled={sessions.length === 0}
|
||||
onCheckedChange={(selected) => toggleAllSessions(selected === true)}
|
||||
/>
|
||||
<span>Select all</span>
|
||||
</label>
|
||||
{selectedSessions.length > 0 && (
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="xs"
|
||||
aria-label={`Delete ${selectedSessions.length} selected sessions`}
|
||||
onClick={() => {
|
||||
requestDelete(selectedSessions);
|
||||
}}
|
||||
>
|
||||
<Trash2 />
|
||||
Delete ({selectedSessions.length})
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex-1 overflow-y-auto px-2 pb-4">
|
||||
{/* L2 — section toggle */}
|
||||
<button
|
||||
type="button"
|
||||
@@ -1002,6 +1104,7 @@ export function AppShell({ canLogout }: AppShellProps) {
|
||||
showOwner={showingAllSessions}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</aside>
|
||||
)}
|
||||
|
||||
@@ -29,6 +29,13 @@ function makeDatabase(overrides: Partial<CatalogDatabase> = {}): CatalogDatabase
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: {
|
||||
transport: "postgres_direct",
|
||||
configurationState: "ready",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
@@ -66,6 +73,9 @@ const orphan = makeDatabase({
|
||||
workspaceId: "retired",
|
||||
workspaceName: "Retired workspace",
|
||||
workspaceAvailable: false,
|
||||
workspaceRevision: null,
|
||||
workspaceEvidence: { sourceType: null, state: "workspace_unavailable" },
|
||||
runtimeBinding: null,
|
||||
});
|
||||
|
||||
function renderPage({
|
||||
@@ -192,7 +202,7 @@ const synchronizationScopes = [
|
||||
{ scope: "all", label: "Synchronize all" },
|
||||
] as const;
|
||||
|
||||
test("renders Fleet Ledger with real metrics, conceptual row tooltips, and contextual model selection", async () => {
|
||||
test("renders Fleet Ledger with real metrics, conceptual row tooltips, and persistent model selection", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
http.get("/api/catalog/metrics", () => HttpResponse.json({
|
||||
@@ -223,10 +233,15 @@ test("renders Fleet Ledger with real metrics, conceptual row tooltips, and conte
|
||||
expect(screen.queryByText("Thoth catalog · Fleet ledger")).not.toBeInTheDocument();
|
||||
expect(screen.queryByText("Inspect physical metadata, curate descriptions and control catalog operations.")).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Back to workspace" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
|
||||
expect(screen.getByRole("columnheader", { name: /Revision \/ Evidence/ })).toBeVisible();
|
||||
expect(screen.getByRole("columnheader", { name: /NL→SQL runtime/ })).toBeVisible();
|
||||
expect(screen.getByRole("columnheader", { name: /Metadata Catalog/ })).toBeVisible();
|
||||
const summary = screen.getByRole("region", { name: "Fleet summary" });
|
||||
expect(await within(summary).findByText("2,275")).toBeVisible();
|
||||
expect(within(summary).getByText("75%")).toBeVisible();
|
||||
expect(screen.queryByRole("combobox", { name: "Metadata description model" })).not.toBeInTheDocument();
|
||||
const modelSelector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
expect(modelSelector).toHaveValue("local-qwen");
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
const tablesAction = within(databaseRow).getByRole("button", { name: "View tables for Policlinico San Donato" });
|
||||
@@ -247,10 +262,103 @@ test("renders Fleet Ledger with real metrics, conceptual row tooltips, and conte
|
||||
const actionPicker = screen.getByRole("combobox", { name: "Batch action" });
|
||||
expect(within(actionPicker).getByRole("option", { name: /^Synchronize all(?:,|$)/ })).toBeVisible();
|
||||
await user.selectOptions(actionPicker, "generate-missing");
|
||||
expect(await screen.findByRole("combobox", { name: "Metadata description model" })).toHaveValue("local-qwen");
|
||||
expect(screen.getAllByRole("combobox", { name: "Metadata-generation LLM model" })).toHaveLength(1);
|
||||
expect(modelSelector).toHaveValue("local-qwen");
|
||||
|
||||
});
|
||||
|
||||
test("shows repository, NL→SQL runtime, and Metadata Catalog states independently", async () => {
|
||||
const configured = makeDatabase({ connectionStatus: "reachable", testedVersion: 3 });
|
||||
const needsRuntimeConfiguration = makeDatabase({
|
||||
...unconfigured,
|
||||
workspaceEvidence: { sourceType: "http", state: "configuration_required" },
|
||||
runtimeBinding: {
|
||||
transport: "rest_api",
|
||||
configurationState: "configuration_required",
|
||||
sessionTransportSupported: true,
|
||||
},
|
||||
});
|
||||
renderPage({ rows: [configured, needsRuntimeConfiguration, orphan], presentation: "fleet" });
|
||||
|
||||
const configuredRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
expect(within(configuredRow).getByText("Active revision aaaaaaa")).toBeVisible();
|
||||
expect(within(configuredRow).getByText("Evidence materialized · filesystem")).toBeVisible();
|
||||
expect(within(configuredRow).getByText("Ready")).toBeVisible();
|
||||
expect(within(configuredRow).getByText("Configured")).toBeVisible();
|
||||
expect(within(configuredRow).getByText("Connection reachable")).toBeVisible();
|
||||
|
||||
const needsConfigurationRow = screen.getByRole("row", { name: /Research laboratory/ });
|
||||
expect(within(needsConfigurationRow).getByText("Evidence credentials required · http")).toBeVisible();
|
||||
expect(within(needsConfigurationRow).getByText("Configuration required")).toBeVisible();
|
||||
expect(within(needsConfigurationRow).getByText("Not configured")).toBeVisible();
|
||||
|
||||
const orphanRow = screen.getByRole("row", { name: /Retired workspace/ });
|
||||
expect(within(orphanRow).getByText("Workspace missing")).toBeVisible();
|
||||
expect(within(orphanRow).getByText("Unavailable")).toBeVisible();
|
||||
expect(within(orphanRow).getByText("Orphaned configuration")).toBeVisible();
|
||||
});
|
||||
|
||||
test("keeps Fleet Ledger visible when a catalog response omits the Evidence projection", async () => {
|
||||
const legacyDatabase = makeDatabase();
|
||||
delete (legacyDatabase as Partial<CatalogDatabase>).workspaceEvidence;
|
||||
|
||||
renderPage({ rows: [legacyDatabase], presentation: "fleet" });
|
||||
|
||||
expect(await screen.findByRole("heading", { name: "Database management" })).toBeVisible();
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
expect(within(databaseRow).getByText("Evidence state unavailable")).toBeVisible();
|
||||
});
|
||||
|
||||
test("opens the relationship map directly from a Fleet database and restores focus on return", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/relationships", () => HttpResponse.json([])),
|
||||
);
|
||||
renderPage({
|
||||
rows: [makeDatabase({ connectionStatus: "reachable", testedVersion: 3 })],
|
||||
presentation: "fleet",
|
||||
});
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
const relationshipsAction = within(databaseRow).getByRole("button", {
|
||||
name: "View relationships for Policlinico San Donato",
|
||||
});
|
||||
expect(relationshipsAction.parentElement).toHaveAttribute("data-tooltip", "Relationships");
|
||||
expect(relationshipsAction).not.toHaveAttribute("title");
|
||||
|
||||
await user.click(relationshipsAction);
|
||||
expect(await screen.findByRole("region", {
|
||||
name: "Relationships for Policlinico San Donato",
|
||||
})).toBeVisible();
|
||||
expect(screen.getByText("Relationship map")).toBeVisible();
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Back to databases" }));
|
||||
await waitFor(() => expect(screen.getByRole("button", {
|
||||
name: "View relationships for Policlinico San Donato",
|
||||
})).toHaveFocus());
|
||||
});
|
||||
|
||||
test("offers catalog configuration directly on an unconfigured Fleet workspace", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderPage({ rows: [unconfigured], presentation: "fleet" });
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Research laboratory/ });
|
||||
expect(within(databaseRow).queryByRole("button", {
|
||||
name: "View relationships for Research laboratory",
|
||||
})).not.toBeInTheDocument();
|
||||
expect(within(databaseRow).queryByRole("button", {
|
||||
name: "View tables for Research laboratory",
|
||||
})).not.toBeInTheDocument();
|
||||
|
||||
await user.click(within(databaseRow).getByRole("button", {
|
||||
name: "Configure catalog for Research laboratory",
|
||||
}));
|
||||
|
||||
const drawer = await screen.findByRole("dialog", { name: "Configure catalog" });
|
||||
expect(within(drawer).getByLabelText("Workspace")).toHaveAttribute("readonly");
|
||||
expect(within(drawer).getByLabelText("Workspace")).toHaveValue("Research laboratory");
|
||||
});
|
||||
|
||||
test("keeps only the local back action while browsing Fleet Ledger columns", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
@@ -276,6 +384,32 @@ test("keeps only the local back action while browsing Fleet Ledger columns", asy
|
||||
expect(screen.queryByRole("button", { name: "Back to tables" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("uses the database back-action background treatment for the table back action", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([patientsTable])),
|
||||
http.get("/api/catalog/databases/:databaseId/tables/:tableId/columns", () => HttpResponse.json([])),
|
||||
);
|
||||
renderPage({ rows: [makeDatabase()], presentation: "fleet" });
|
||||
|
||||
const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ });
|
||||
await user.click(within(databaseRow).getByRole("button", {
|
||||
name: "View tables for Policlinico San Donato",
|
||||
}));
|
||||
const databaseBackAction = await screen.findByRole("button", { name: "Back to databases" });
|
||||
expect(databaseBackAction).toHaveClass("thot-fleet-back-action");
|
||||
const backgroundTreatment = (element: HTMLElement) => element.className
|
||||
.split(/\s+/)
|
||||
.filter((className) => /(^|:)bg-/.test(className))
|
||||
.sort();
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "View columns for patients" }));
|
||||
|
||||
const tableBackAction = await screen.findByRole("button", { name: "Back to tables" });
|
||||
expect(tableBackAction).toHaveClass("thot-fleet-back-action");
|
||||
expect(backgroundTreatment(tableBackAction)).toEqual(backgroundTreatment(databaseBackAction));
|
||||
});
|
||||
|
||||
test("reopens database synchronization history when no run is active", async () => {
|
||||
const user = userEvent.setup();
|
||||
const completed: CatalogSyncRun = {
|
||||
@@ -312,7 +446,7 @@ test("selects the configured metadata-description model by default", async () =>
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
await waitFor(() => expect(selector).toHaveValue("local-qwen"));
|
||||
});
|
||||
|
||||
@@ -320,7 +454,7 @@ test("keeps both run-history buttons visible beside the metadata-description sel
|
||||
const user = userEvent.setup();
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
const toolbar = screen.getByRole("group", { name: "Database management controls" });
|
||||
const metadataControls = screen.getByRole("group", { name: "Metadata description controls" });
|
||||
const actions = screen.getByRole("group", { name: "Database management actions" });
|
||||
@@ -344,7 +478,7 @@ test("keeps both run-history buttons visible beside the metadata-description sel
|
||||
expect(toolbar.lastElementChild).toBe(actions);
|
||||
expect(actions).toHaveClass("sm:justify-end");
|
||||
expect(actions).toContainElement(screen.getByRole("button", { name: "Refresh" }));
|
||||
expect(actions).toContainElement(screen.getByRole("button", { name: "Add database" }));
|
||||
expect(within(actions).queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("note", {
|
||||
name: "Metadata generation source data disclosure",
|
||||
})).not.toBeInTheDocument();
|
||||
@@ -371,7 +505,7 @@ test("changes the metadata-description model in page-local state", async () => {
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
await waitFor(() => expect(selector).toHaveValue("local-qwen"));
|
||||
await user.selectOptions(selector, "openai-mini");
|
||||
|
||||
@@ -385,10 +519,10 @@ test("explains application setup when no metadata-description model is configure
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
expect(selector).toBeDisabled();
|
||||
expect(await screen.findByText(
|
||||
"Configure a metadata-generation model in application setup to enable description generation.",
|
||||
"No metadata-generation LLM model is configured for this installation.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
@@ -399,11 +533,11 @@ test("uses a safe disabled setup state when metadata-description models are unav
|
||||
}, { status: 503 })));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
await waitFor(() => expect(selector).toHaveDisplayValue("Model choices unavailable"));
|
||||
expect(selector).toBeDisabled();
|
||||
expect(screen.getByText(
|
||||
"Configure a metadata-generation model in application setup to enable description generation.",
|
||||
"Metadata-generation LLM model choices are unavailable.",
|
||||
)).toBeVisible();
|
||||
expect(screen.queryByText("internal model registry details")).not.toBeInTheDocument();
|
||||
});
|
||||
@@ -417,11 +551,11 @@ test.each([null, "missing-model"])(
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
await waitFor(() => expect(selector).toHaveDisplayValue("No usable model configured"));
|
||||
expect(selector).toBeDisabled();
|
||||
expect(screen.getByText(
|
||||
"Configure a metadata-generation model in application setup to enable description generation.",
|
||||
"No metadata-generation LLM model is configured for this installation.",
|
||||
)).toBeVisible();
|
||||
},
|
||||
);
|
||||
@@ -441,7 +575,7 @@ test("renders only the public metadata-generation model contract", async () => {
|
||||
})));
|
||||
renderPage();
|
||||
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata description model" });
|
||||
const selector = await screen.findByRole("combobox", { name: "Metadata-generation LLM model" });
|
||||
await waitFor(() => expect(selector).toHaveValue("approved-model"));
|
||||
expect(within(selector).getAllByRole("option").map((option) => option.textContent))
|
||||
.toEqual(["Approved model"]);
|
||||
@@ -458,8 +592,9 @@ test("starts with a full-width list and applies the row action matrix", async ()
|
||||
expect(screen.getByRole("button", { name: "Delete Policlinico San Donato" })).toBeEnabled();
|
||||
|
||||
expect(screen.getByRole("button", { name: "View Research laboratory" })).toBeEnabled();
|
||||
expect(screen.getByRole("button", { name: "Edit Research laboratory" })).toBeEnabled();
|
||||
expect(screen.getByRole("button", { name: "Delete Research laboratory" })).toBeDisabled();
|
||||
expect(screen.getByRole("button", { name: "Configure catalog for Research laboratory" })).toBeEnabled();
|
||||
expect(screen.queryByRole("button", { name: "Edit Research laboratory" })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole("button", { name: "Delete Research laboratory" })).not.toBeInTheDocument();
|
||||
|
||||
expect(screen.getByRole("button", { name: "View Retired workspace" })).toBeEnabled();
|
||||
expect(screen.getByRole("button", { name: "Edit Retired workspace" })).toBeDisabled();
|
||||
@@ -558,7 +693,7 @@ test("keeps Fleet Generate Missing behind the source-data disclosure", async ()
|
||||
"generate-missing",
|
||||
);
|
||||
await waitFor(() => expect(screen.getByRole("combobox", {
|
||||
name: "Metadata description model",
|
||||
name: "Metadata-generation LLM model",
|
||||
})).toHaveValue("local-qwen"));
|
||||
await user.click(screen.getByRole("button", { name: "Run action" }));
|
||||
|
||||
@@ -951,7 +1086,7 @@ test("replaces the list with View and returns focus to the originating action",
|
||||
expect(screen.getByRole("button", { name: "View Policlinico San Donato" })).toBeVisible();
|
||||
});
|
||||
|
||||
test("opens an unconfigured row as Edit while creating its first saved configuration", async () => {
|
||||
test("configures the catalog for the selected workspace and locks that workspace", async () => {
|
||||
const user = userEvent.setup();
|
||||
let postBody: unknown;
|
||||
const saved = makeDatabase({
|
||||
@@ -971,16 +1106,16 @@ test("opens an unconfigured row as Edit while creating its first saved configura
|
||||
);
|
||||
renderPage({ rows: () => rows });
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "Edit Research laboratory" }));
|
||||
await user.click(await screen.findByRole("button", { name: "Configure catalog for Research laboratory" }));
|
||||
|
||||
expect(screen.getByRole("heading", { name: "Edit database" })).toBeVisible();
|
||||
expect(screen.getByLabelText("Workspace")).toBeDisabled();
|
||||
expect(screen.getByRole("heading", { name: "Configure catalog" })).toBeVisible();
|
||||
expect(screen.getByLabelText("Workspace")).toHaveAttribute("readonly");
|
||||
await user.selectOptions(screen.getByLabelText("Transport"), "rest_api");
|
||||
await user.type(screen.getByLabelText("Base URL"), "https://psd.example/api");
|
||||
expect(screen.getByLabelText("Diagnostic endpoint")).toHaveValue("/health");
|
||||
expect(screen.getByLabelText("Diagnostic endpoint")).toHaveAttribute("readonly");
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "Save database" }));
|
||||
await user.click(screen.getByRole("button", { name: "Save catalog configuration" }));
|
||||
|
||||
await waitFor(() => expect(postBody).toMatchObject({
|
||||
workspaceId: "lab",
|
||||
@@ -989,7 +1124,7 @@ test("opens an unconfigured row as Edit while creating its first saved configura
|
||||
expect(await screen.findByRole("heading", { name: "Edit database" })).toBeVisible();
|
||||
});
|
||||
|
||||
test("global Add offers only unconfigured workspaces and lets the operator choose one", async () => {
|
||||
test("has no global database creation path and scopes configuration to the chosen row", async () => {
|
||||
const user = userEvent.setup();
|
||||
const second = makeDatabase({
|
||||
...unconfigured,
|
||||
@@ -999,14 +1134,13 @@ test("global Add offers only unconfigured workspaces and lets the operator choos
|
||||
});
|
||||
renderPage({ rows: [makeDatabase(), unconfigured, second] });
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "Add database" }));
|
||||
expect(screen.queryByRole("button", { name: /Add database/i })).not.toBeInTheDocument();
|
||||
await user.click(await screen.findByRole("button", { name: "Configure catalog for Radiology" }));
|
||||
|
||||
expect(screen.getByRole("heading", { name: "Add database" })).toBeVisible();
|
||||
expect(screen.getByRole("button", { name: "Add database" })).toBeVisible();
|
||||
const selector = screen.getByLabelText("Workspace");
|
||||
expect(selector).toBeEnabled();
|
||||
expect(screen.queryByRole("option", { name: "Policlinico San Donato" })).not.toBeInTheDocument();
|
||||
await user.selectOptions(selector, "radiology");
|
||||
expect(screen.getByRole("heading", { name: "Configure catalog" })).toBeVisible();
|
||||
const workspace = screen.getByLabelText("Workspace");
|
||||
expect(workspace).toHaveAttribute("readonly");
|
||||
expect(workspace).toHaveValue("Radiology");
|
||||
expect(screen.getByDisplayValue("radiology_dwh")).toBeVisible();
|
||||
});
|
||||
|
||||
@@ -1064,7 +1198,8 @@ test("uses an in-page destructive form and returns the YAML workspace to Not con
|
||||
|
||||
await waitFor(() => expect(deleteCalled).toBe(true));
|
||||
expect(deleteVersion).toBe("3");
|
||||
expect(await screen.findByRole("button", { name: "Delete Policlinico San Donato" })).toBeDisabled();
|
||||
expect(await screen.findByRole("button", { name: "Configure catalog for Policlinico San Donato" })).toBeEnabled();
|
||||
expect(screen.queryByRole("button", { name: "Delete Policlinico San Donato" })).not.toBeInTheDocument();
|
||||
expect(screen.getByText("Not configured")).toBeVisible();
|
||||
});
|
||||
|
||||
@@ -2677,6 +2812,8 @@ test("navigates from a table to columns and from the database to physical relati
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{ position: 1, sourceColumnId: "1", sourceColumnName: "patient_id", targetColumnId: idColumn.id, targetColumnName: "id" }],
|
||||
origin: "physical",
|
||||
status: "active",
|
||||
lastSyncedDatabaseVersion: 3,
|
||||
lastSyncedAt: "2026-08-27T10:00:00Z",
|
||||
createdAt: "2026-08-27T10:00:00Z",
|
||||
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
useState,
|
||||
} from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { History, Plus, RefreshCw } from "lucide-react";
|
||||
import { History, RefreshCw } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import { Button } from "../components/ui/button";
|
||||
import { ApiError, apiErrorMessage } from "../api/client";
|
||||
@@ -209,11 +209,7 @@ export function DatabaseManagementPage({
|
||||
() => queryClient.invalidateQueries({ queryKey: ["catalog-metrics"] }),
|
||||
[queryClient],
|
||||
);
|
||||
const availableWorkspaces = useMemo(
|
||||
() => rows.filter((row) => row.workspaceAvailable && !row.configured),
|
||||
[rows],
|
||||
);
|
||||
const editable = screen.kind === "add" || screen.kind === "edit";
|
||||
const editable = screen.kind === "configure" || screen.kind === "edit";
|
||||
const configurationDirty = Boolean(
|
||||
editable
|
||||
&& draft
|
||||
@@ -221,8 +217,9 @@ export function DatabaseManagementPage({
|
||||
);
|
||||
const dirty = Boolean(editable && draft && (configurationDirty || hasSecretChanges(draft)));
|
||||
const busy = busyAction !== null;
|
||||
const navigationDirty = screen.kind === "tables" ? tablesNavigationState.dirty : dirty;
|
||||
const navigationBusy = screen.kind === "tables" ? tablesNavigationState.busy : busy;
|
||||
const catalogChildVisible = screen.kind === "tables" || screen.kind === "relationships";
|
||||
const navigationDirty = catalogChildVisible ? tablesNavigationState.dirty : dirty;
|
||||
const navigationBusy = catalogChildVisible ? tablesNavigationState.busy : busy;
|
||||
|
||||
useEffect(() => {
|
||||
onNavigationStateChange?.({ dirty: navigationDirty, busy: navigationBusy });
|
||||
@@ -263,9 +260,14 @@ export function DatabaseManagementPage({
|
||||
}, [queryClient]);
|
||||
|
||||
const restoreListFocus = useCallback(() => {
|
||||
const originLabel = originRef.current?.getAttribute("aria-label");
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) {
|
||||
originRef.current.focus();
|
||||
} else if (originLabel) {
|
||||
const matchingAction = [...document.querySelectorAll<HTMLButtonElement>("button")]
|
||||
.find((button) => button.getAttribute("aria-label") === originLabel);
|
||||
(matchingAction ?? searchInputRef.current)?.focus();
|
||||
} else {
|
||||
searchInputRef.current?.focus();
|
||||
}
|
||||
@@ -307,7 +309,6 @@ export function DatabaseManagementPage({
|
||||
mode: DatabaseFormMode,
|
||||
row: CatalogDatabase,
|
||||
origin: HTMLElement,
|
||||
workspaceLocked = false,
|
||||
) => {
|
||||
const nextDraft = draftFrom(row);
|
||||
originRef.current = origin;
|
||||
@@ -320,7 +321,6 @@ export function DatabaseManagementPage({
|
||||
setScreen({
|
||||
kind: mode,
|
||||
workspaceId: row.workspaceId,
|
||||
...(mode === "add" ? { workspaceLocked } : {}),
|
||||
});
|
||||
}, []);
|
||||
|
||||
@@ -329,7 +329,7 @@ export function DatabaseManagementPage({
|
||||
}, [openForm]);
|
||||
|
||||
const editRow = useCallback((row: CatalogDatabase, origin: HTMLButtonElement) => {
|
||||
openForm(row.configured ? "edit" : "add", row, origin, !row.configured);
|
||||
openForm(row.configured ? "edit" : "configure", row, origin);
|
||||
}, [openForm]);
|
||||
|
||||
const deleteRow = useCallback((row: CatalogDatabase, origin: HTMLButtonElement) => {
|
||||
@@ -349,8 +349,9 @@ export function DatabaseManagementPage({
|
||||
setScreen({ kind: "tables", workspaceId: row.workspaceId });
|
||||
}, []);
|
||||
|
||||
const openRelationships = useCallback((row: CatalogDatabase) => {
|
||||
const openRelationships = useCallback((row: CatalogDatabase, origin?: HTMLElement) => {
|
||||
if (!row.configured || !row.id) return;
|
||||
if (origin) originRef.current = origin;
|
||||
setDraft(null);
|
||||
setBaseline("");
|
||||
setFormSource(null);
|
||||
@@ -372,25 +373,6 @@ export function DatabaseManagementPage({
|
||||
setScreen({ kind: "view", workspaceId: row.workspaceId });
|
||||
}, []);
|
||||
|
||||
const addDatabase = useCallback((origin: HTMLElement) => {
|
||||
const workspace = availableWorkspaces[0];
|
||||
if (!workspace || !canManage) return;
|
||||
openForm("add", workspace, origin, false);
|
||||
}, [availableWorkspaces, canManage, openForm]);
|
||||
|
||||
const changeWorkspace = useCallback((workspaceId: string) => {
|
||||
const workspace = availableWorkspaces.find((row) => row.workspaceId === workspaceId);
|
||||
if (!workspace) return;
|
||||
if (dirty && !window.confirm("Discard changes and choose another workspace?")) return;
|
||||
const nextDraft = draftFrom(workspace);
|
||||
setDraft(nextDraft);
|
||||
setBaseline(configurationFingerprint(nextDraft));
|
||||
setFormSource({ id: workspace.id, version: workspace.version });
|
||||
setStale(false);
|
||||
setPartialSecretFailure(null);
|
||||
setScreen({ kind: "add", workspaceId, workspaceLocked: false });
|
||||
}, [availableWorkspaces, dirty]);
|
||||
|
||||
const changeField = useCallback((field: "databaseName" | "schema", value: string) => {
|
||||
setDraft((current) => current ? { ...current, [field]: value } : current);
|
||||
}, []);
|
||||
@@ -624,7 +606,7 @@ export function DatabaseManagementPage({
|
||||
showList();
|
||||
toast.info("This database configuration has already been deleted");
|
||||
} else if (screen.kind === "edit" && !latest.configured) {
|
||||
setScreen({ kind: "add", workspaceId: latest.workspaceId, workspaceLocked: true });
|
||||
setScreen({ kind: "configure", workspaceId: latest.workspaceId });
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
@@ -669,22 +651,40 @@ export function DatabaseManagementPage({
|
||||
}, [queryClient]);
|
||||
|
||||
const openDescriptionGenerationHistory = useCallback(() => {
|
||||
const run = observedActiveDescriptionGenerationRun
|
||||
?? descriptionGenerationRuns[0]
|
||||
?? activeDescriptionGenerationRun;
|
||||
const scopedRuns = activeRow ? descriptionGenerationRuns.filter((item) => item.databaseId === activeRow.id) : descriptionGenerationRuns;
|
||||
const scopedActiveRun = observedActiveDescriptionGenerationRun
|
||||
&& (!activeRow || observedActiveDescriptionGenerationRun.databaseId === activeRow.id)
|
||||
? observedActiveDescriptionGenerationRun
|
||||
: undefined;
|
||||
const scopedSelectedRun = activeDescriptionGenerationRun
|
||||
&& (!activeRow || activeDescriptionGenerationRun.databaseId === activeRow.id)
|
||||
? activeDescriptionGenerationRun
|
||||
: undefined;
|
||||
const run = scopedActiveRun
|
||||
?? scopedRuns[0]
|
||||
?? scopedSelectedRun;
|
||||
if (run) setActiveDescriptionGenerationRun(run);
|
||||
setSensitiveDataSuggestionHistoryDrawerOpen(false);
|
||||
setDescriptionGenerationDrawerOpen(true);
|
||||
}, [activeDescriptionGenerationRun, descriptionGenerationRuns, observedActiveDescriptionGenerationRun]);
|
||||
}, [activeDescriptionGenerationRun, activeRow, descriptionGenerationRuns, observedActiveDescriptionGenerationRun]);
|
||||
|
||||
const openSensitiveDataSuggestionHistory = useCallback(() => {
|
||||
const run = observedActiveSensitiveDataSuggestionRun
|
||||
?? sensitiveDataSuggestionRuns[0]
|
||||
?? activeSensitiveDataSuggestionRun;
|
||||
const scopedRuns = activeRow ? sensitiveDataSuggestionRuns.filter((item) => item.databaseId === activeRow.id) : sensitiveDataSuggestionRuns;
|
||||
const scopedActiveRun = observedActiveSensitiveDataSuggestionRun
|
||||
&& (!activeRow || observedActiveSensitiveDataSuggestionRun.databaseId === activeRow.id)
|
||||
? observedActiveSensitiveDataSuggestionRun
|
||||
: undefined;
|
||||
const scopedSelectedRun = activeSensitiveDataSuggestionRun
|
||||
&& (!activeRow || activeSensitiveDataSuggestionRun.databaseId === activeRow.id)
|
||||
? activeSensitiveDataSuggestionRun
|
||||
: undefined;
|
||||
const run = scopedActiveRun
|
||||
?? scopedRuns[0]
|
||||
?? scopedSelectedRun;
|
||||
if (run) setActiveSensitiveDataSuggestionRun(run);
|
||||
setDescriptionGenerationDrawerOpen(false);
|
||||
setSensitiveDataSuggestionHistoryDrawerOpen(true);
|
||||
}, [activeSensitiveDataSuggestionRun, observedActiveSensitiveDataSuggestionRun, sensitiveDataSuggestionRuns]);
|
||||
}, [activeRow, activeSensitiveDataSuggestionRun, observedActiveSensitiveDataSuggestionRun, sensitiveDataSuggestionRuns]);
|
||||
|
||||
const descriptionGenerationTerminated = useCallback(async (run: DescriptionGenerationRun) => {
|
||||
await Promise.all([
|
||||
@@ -902,6 +902,7 @@ export function DatabaseManagementPage({
|
||||
isLoading={metadataModelsQuery.isLoading}
|
||||
selectedModel={selectedMetadataModel}
|
||||
onSelectedModelChange={setSelectedMetadataModel}
|
||||
variant="compact"
|
||||
/>
|
||||
);
|
||||
|
||||
@@ -943,6 +944,7 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<DescriptionGenerationDrawer
|
||||
open={descriptionGenerationDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeDescriptionGenerationRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeDescriptionGenerationRun?.modelId,
|
||||
@@ -953,6 +955,7 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<SensitiveDataSuggestionHistoryDrawer
|
||||
open={sensitiveDataSuggestionHistoryDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeSensitiveDataSuggestionRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeSensitiveDataSuggestionRun?.modelId,
|
||||
@@ -980,7 +983,6 @@ export function DatabaseManagementPage({
|
||||
eyebrow="Catalog configuration"
|
||||
title={databaseFormTitle(
|
||||
fleetFormMode,
|
||||
fleetFormMode === "add" && Boolean(screen.kind === "add" && screen.workspaceLocked),
|
||||
"drawer",
|
||||
)}
|
||||
description={`${activeRow.workspaceName} · ${activeRow.workspaceId}`}
|
||||
@@ -993,8 +995,6 @@ export function DatabaseManagementPage({
|
||||
presentation="drawer"
|
||||
mode={fleetFormMode}
|
||||
row={activeRow}
|
||||
availableWorkspaces={availableWorkspaces}
|
||||
workspaceLocked={fleetFormMode === "add" && Boolean(screen.kind === "add" && screen.workspaceLocked)}
|
||||
draft={draft}
|
||||
dirty={dirty}
|
||||
canManage={canManage}
|
||||
@@ -1005,7 +1005,6 @@ export function DatabaseManagementPage({
|
||||
partialSecretFailure={partialSecretFailure}
|
||||
headingRef={formHeadingRef}
|
||||
onBack={backToList}
|
||||
onWorkspaceChange={changeWorkspace}
|
||||
onFieldChange={changeField}
|
||||
onTransportChange={changeTransport}
|
||||
onBindingChange={changeBinding}
|
||||
@@ -1020,6 +1019,8 @@ export function DatabaseManagementPage({
|
||||
onOpenRelationships={() => openRelationships(activeRow)}
|
||||
onSync={(scope) => void syncDatabase(scope)}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
activeSyncRun={currentActiveRun}
|
||||
/>
|
||||
</FleetLedgerDrawer>
|
||||
@@ -1040,10 +1041,11 @@ export function DatabaseManagementPage({
|
||||
onNestedNavigationChange={setTableNestedNavigationActive}
|
||||
onRunStarted={rememberSyncRun}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
onDescriptionGenerationRunStarted={rememberDescriptionGenerationRun}
|
||||
onSuggestSensitive={suggestActiveDatabaseSensitiveFields}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
metadataModelControl={fleetMetadataModelControl}
|
||||
/>
|
||||
) : screen.kind === "relationships" && relationshipsVisible ? (
|
||||
<DatabaseRelationships
|
||||
@@ -1055,6 +1057,10 @@ export function DatabaseManagementPage({
|
||||
onOpenTables={() => openTables(activeRow)}
|
||||
onRunStarted={rememberSyncRun}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
onNavigationStateChange={setTablesNavigationState}
|
||||
/>
|
||||
) : isError && rows.length === 0 ? (
|
||||
<div className="grid h-full place-items-center p-6">
|
||||
@@ -1078,6 +1084,7 @@ export function DatabaseManagementPage({
|
||||
onSearchChange={setSearch}
|
||||
onView={viewRow}
|
||||
onOpenTables={(row, origin) => openTables(row, origin)}
|
||||
onOpenRelationships={(row, origin) => openRelationships(row, origin)}
|
||||
onEdit={editRow}
|
||||
onDelete={deleteRow}
|
||||
onOpenSync={openSync}
|
||||
@@ -1088,7 +1095,6 @@ export function DatabaseManagementPage({
|
||||
onGenerateDescriptions={generateDatabaseDescriptions}
|
||||
onSuggestSensitive={suggestDatabaseSensitiveFields}
|
||||
onDeleteMetadataSelected={deleteSelectedMetadata}
|
||||
metadataModelControl={fleetMetadataModelControl}
|
||||
onRefresh={refreshList}
|
||||
/>
|
||||
);
|
||||
@@ -1118,28 +1124,7 @@ export function DatabaseManagementPage({
|
||||
)}
|
||||
actions={(
|
||||
<>
|
||||
<Button type="button" variant="outline" size="sm" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Description history" data-fleet-action="secondary" onClick={openDescriptionGenerationHistory}>
|
||||
<History aria-hidden="true" /> Description history
|
||||
{observedActiveDescriptionGenerationRun ? <span className="size-1.5 rounded-full bg-warning" aria-hidden="true" /> : null}
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Sensitive history" data-fleet-action="secondary" onClick={openSensitiveDataSuggestionHistory}>
|
||||
<History aria-hidden="true" /> Sensitive history
|
||||
{observedActiveSensitiveDataSuggestionRun ? <span className="size-1.5 rounded-full bg-warning" aria-hidden="true" /> : null}
|
||||
</Button>
|
||||
{(screen.kind === "list" || formVisible) ? (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
className="thot-fleet-nav-action thot-fleet-nav-action--below"
|
||||
data-tooltip="Add database"
|
||||
data-fleet-action="primary"
|
||||
disabled={!canManage || availableWorkspaces.length === 0}
|
||||
title={availableWorkspaces.length === 0 ? "Every available workspace is already configured" : undefined}
|
||||
onClick={(event) => addDatabase(event.currentTarget)}
|
||||
>
|
||||
<Plus aria-hidden="true" /> Add database
|
||||
</Button>
|
||||
) : null}
|
||||
{fleetMetadataModelControl}
|
||||
</>
|
||||
)}
|
||||
/>
|
||||
@@ -1205,40 +1190,16 @@ export function DatabaseManagementPage({
|
||||
selectedModel={selectedMetadataModel}
|
||||
onSelectedModelChange={setSelectedMetadataModel}
|
||||
/>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
className="whitespace-nowrap disabled:opacity-70"
|
||||
aria-label="Run descriptions generation history"
|
||||
disabled={!canManage}
|
||||
title={observedActiveDescriptionGenerationRun
|
||||
? "Description generation is active"
|
||||
: "View description generation history"}
|
||||
onClick={openDescriptionGenerationHistory}
|
||||
>
|
||||
<History />
|
||||
Run descriptions generation history
|
||||
{observedActiveDescriptionGenerationRun ? (
|
||||
<span aria-hidden="true" className="size-1.5 rounded-full bg-primary" />
|
||||
) : null}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
className="whitespace-nowrap disabled:opacity-70"
|
||||
aria-label="Run sensitive suggestions history"
|
||||
disabled={!canManage}
|
||||
title={observedActiveSensitiveDataSuggestionRun
|
||||
? "Sensitive suggestion generation is active"
|
||||
: "View sensitive suggestion history"}
|
||||
onClick={openSensitiveDataSuggestionHistory}
|
||||
>
|
||||
<History />
|
||||
Run sensitive suggestions history
|
||||
{observedActiveSensitiveDataSuggestionRun ? (
|
||||
<span aria-hidden="true" className="size-1.5 rounded-full bg-primary" />
|
||||
) : null}
|
||||
</Button>
|
||||
{screen.kind === "list" ? <>
|
||||
<Button type="button" variant="outline" className="whitespace-nowrap disabled:opacity-70" aria-label="Run descriptions generation history" disabled={!canManage} title="View description generation history" onClick={openDescriptionGenerationHistory}>
|
||||
<History /> Run descriptions generation history
|
||||
</Button>
|
||||
<Button type="button" variant="outline" className="whitespace-nowrap disabled:opacity-70" aria-label="Run sensitive suggestions history" disabled={!canManage} title={observedActiveSensitiveDataSuggestionRun ? "Sensitive suggestion generation is active" : "View sensitive suggestion history"} onClick={openSensitiveDataSuggestionHistory}>
|
||||
<History />
|
||||
{observedActiveSensitiveDataSuggestionRun ? <span aria-hidden="true" className="size-2 rounded-full bg-primary" /> : null}
|
||||
Run sensitive suggestions history
|
||||
</Button>
|
||||
</> : null}
|
||||
</div>
|
||||
{screen.kind === "list" ? (
|
||||
<div
|
||||
@@ -1249,14 +1210,6 @@ export function DatabaseManagementPage({
|
||||
<Button type="button" variant="outline" disabled={isFetching} onClick={() => void refreshList()}>
|
||||
<RefreshCw className={isFetching ? "animate-spin" : ""} /> Refresh
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
disabled={!canManage || availableWorkspaces.length === 0}
|
||||
title={availableWorkspaces.length === 0 ? "Every available workspace is already configured" : undefined}
|
||||
onClick={(event) => addDatabase(event.currentTarget)}
|
||||
>
|
||||
<Plus /> Add database
|
||||
</Button>
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
@@ -1308,8 +1261,6 @@ export function DatabaseManagementPage({
|
||||
<DatabaseForm
|
||||
mode={screen.kind as DatabaseFormMode}
|
||||
row={activeRow}
|
||||
availableWorkspaces={availableWorkspaces}
|
||||
workspaceLocked={screen.kind === "add" && Boolean(screen.workspaceLocked)}
|
||||
draft={draft}
|
||||
dirty={dirty}
|
||||
canManage={canManage}
|
||||
@@ -1320,7 +1271,6 @@ export function DatabaseManagementPage({
|
||||
partialSecretFailure={partialSecretFailure}
|
||||
headingRef={formHeadingRef}
|
||||
onBack={backToList}
|
||||
onWorkspaceChange={changeWorkspace}
|
||||
onFieldChange={changeField}
|
||||
onTransportChange={changeTransport}
|
||||
onBindingChange={changeBinding}
|
||||
@@ -1335,6 +1285,8 @@ export function DatabaseManagementPage({
|
||||
onOpenRelationships={() => openRelationships(activeRow)}
|
||||
onSync={(scope) => void syncDatabase(scope)}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
activeSyncRun={currentActiveRun}
|
||||
/>
|
||||
) : null}
|
||||
@@ -1352,6 +1304,8 @@ export function DatabaseManagementPage({
|
||||
onNavigationStateChange={setTablesNavigationState}
|
||||
onRunStarted={rememberSyncRun}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
onDescriptionGenerationRunStarted={rememberDescriptionGenerationRun}
|
||||
onSuggestSensitive={suggestActiveDatabaseSensitiveFields}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
@@ -1367,6 +1321,10 @@ export function DatabaseManagementPage({
|
||||
onOpenTables={() => openTables(activeRow)}
|
||||
onRunStarted={rememberSyncRun}
|
||||
onOpenSync={() => openSync(activeRow)}
|
||||
onOpenDescriptionHistory={openDescriptionGenerationHistory}
|
||||
onOpenSensitiveHistory={openSensitiveDataSuggestionHistory}
|
||||
onCatalogMetricsChanged={invalidateCatalogMetrics}
|
||||
onNavigationStateChange={setTablesNavigationState}
|
||||
/>
|
||||
) : null}
|
||||
</div>
|
||||
@@ -1381,6 +1339,7 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<DescriptionGenerationDrawer
|
||||
open={descriptionGenerationDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeDescriptionGenerationRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeDescriptionGenerationRun?.modelId,
|
||||
@@ -1391,6 +1350,7 @@ export function DatabaseManagementPage({
|
||||
/>
|
||||
<SensitiveDataSuggestionHistoryDrawer
|
||||
open={sensitiveDataSuggestionHistoryDrawerOpen}
|
||||
databaseId={activeRow?.id ?? null}
|
||||
run={activeSensitiveDataSuggestionRun}
|
||||
modelLabel={metadataModels.find(
|
||||
(model) => model.id === activeSensitiveDataSuggestionRun?.modelId,
|
||||
|
||||
@@ -249,6 +249,7 @@ export function CatalogSyncDrawer({
|
||||
key={item.id}
|
||||
type="button"
|
||||
aria-current={item.id === run?.id ? "true" : undefined}
|
||||
data-status={item.state}
|
||||
className="flex w-full items-center justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 aria-[current=true]:bg-primary/8"
|
||||
onClick={() => onRunChange(item)}
|
||||
>
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect, useMemo, useRef, useState, type ReactNode } from "react";
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import { Menu } from "@base-ui/react/menu";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
@@ -20,6 +20,7 @@ import {
|
||||
import type { DatabaseNavigationState } from "./model";
|
||||
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
|
||||
import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
|
||||
|
||||
interface Props {
|
||||
databaseId: string;
|
||||
@@ -34,7 +35,6 @@ interface Props {
|
||||
bindingReady?: boolean;
|
||||
catalogOperationActive?: boolean;
|
||||
onCatalogMetricsChanged?: () => void | Promise<void>;
|
||||
metadataModelControl?: ReactNode;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
|
||||
@@ -99,7 +99,6 @@ export function DatabaseColumns({
|
||||
bindingReady = true,
|
||||
catalogOperationActive = false,
|
||||
onCatalogMetricsChanged,
|
||||
metadataModelControl,
|
||||
presentation = "legacy",
|
||||
}: Props) {
|
||||
const queryClient = useQueryClient();
|
||||
@@ -307,7 +306,7 @@ export function DatabaseColumns({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one column."
|
||||
: !selectedMetadataModel
|
||||
? "Choose a metadata model first."
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
: descriptionGenerationActive || catalogOperationActive
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy
|
||||
@@ -341,7 +340,7 @@ export function DatabaseColumns({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one column."
|
||||
: !selectedMetadataModel
|
||||
? "Choose a metadata model first."
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
: descriptionGenerationActive || catalogOperationActive
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy
|
||||
@@ -468,7 +467,6 @@ export function DatabaseColumns({
|
||||
busyLabel={sensitiveAction === "suggest" ? "Suggesting…" : sensitiveAction === "save" ? "Saving…" : "Running…"}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Column action"
|
||||
/>
|
||||
</>
|
||||
@@ -501,7 +499,6 @@ export function DatabaseColumns({
|
||||
busyLabel={sensitiveAction === "save" ? "Saving…" : "Running…"}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Column action"
|
||||
/>
|
||||
<input className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search columns" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
|
||||
|
||||
@@ -14,6 +14,9 @@ const database: CatalogDatabase = {
|
||||
workspaceId: "psd-clinical",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: { transport: "postgres_direct", configurationState: "ready", sessionTransportSupported: true },
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "warehouse",
|
||||
|
||||
@@ -113,8 +113,6 @@ function SecretField({
|
||||
interface DatabaseFormProps {
|
||||
mode: DatabaseFormMode;
|
||||
row: CatalogDatabase;
|
||||
availableWorkspaces: CatalogDatabase[];
|
||||
workspaceLocked: boolean;
|
||||
draft: DatabaseFormDraft;
|
||||
dirty: boolean;
|
||||
canManage: boolean;
|
||||
@@ -125,7 +123,6 @@ interface DatabaseFormProps {
|
||||
partialSecretFailure: string | null;
|
||||
headingRef: RefObject<HTMLHeadingElement>;
|
||||
onBack: () => void;
|
||||
onWorkspaceChange: (workspaceId: string) => void;
|
||||
onFieldChange: (field: "databaseName" | "schema", value: string) => void;
|
||||
onTransportChange: (transport: DatabaseTransport) => void;
|
||||
onBindingChange: <K extends keyof DatabaseBinding>(key: K, value: DatabaseBinding[K]) => void;
|
||||
@@ -140,16 +137,17 @@ interface DatabaseFormProps {
|
||||
onOpenRelationships: () => void;
|
||||
onSync: (scope: CatalogSyncScope) => void;
|
||||
onOpenSync: () => void;
|
||||
onOpenDescriptionHistory?: () => void;
|
||||
onOpenSensitiveHistory?: () => void;
|
||||
activeSyncRun?: CatalogSyncRun;
|
||||
presentation?: "page" | "drawer";
|
||||
}
|
||||
|
||||
export function databaseFormTitle(
|
||||
mode: DatabaseFormMode,
|
||||
workspaceLocked: boolean,
|
||||
presentation: "page" | "drawer" = "page",
|
||||
): string {
|
||||
if (mode === "add") return workspaceLocked ? "Edit database" : "Add database";
|
||||
if (mode === "configure") return "Configure catalog";
|
||||
if (mode === "edit") return "Edit database";
|
||||
if (mode === "delete") return presentation === "drawer" ? "Remove configuration" : "Delete database";
|
||||
return "Database details";
|
||||
@@ -158,8 +156,6 @@ export function databaseFormTitle(
|
||||
export function DatabaseForm({
|
||||
mode,
|
||||
row,
|
||||
availableWorkspaces,
|
||||
workspaceLocked,
|
||||
draft,
|
||||
dirty,
|
||||
canManage,
|
||||
@@ -170,7 +166,6 @@ export function DatabaseForm({
|
||||
partialSecretFailure,
|
||||
headingRef,
|
||||
onBack,
|
||||
onWorkspaceChange,
|
||||
onFieldChange,
|
||||
onTransportChange,
|
||||
onBindingChange,
|
||||
@@ -185,12 +180,14 @@ export function DatabaseForm({
|
||||
onOpenRelationships,
|
||||
onSync,
|
||||
onOpenSync,
|
||||
onOpenDescriptionHistory,
|
||||
onOpenSensitiveHistory,
|
||||
activeSyncRun,
|
||||
presentation = "page",
|
||||
}: DatabaseFormProps) {
|
||||
const title = databaseFormTitle(mode, workspaceLocked, presentation);
|
||||
const title = databaseFormTitle(mode, presentation);
|
||||
const readOnly = mode === "view" || mode === "delete";
|
||||
const editable = mode === "add" || mode === "edit";
|
||||
const editable = mode === "configure" || mode === "edit";
|
||||
const busy = busyAction !== null;
|
||||
const testDisabled = !canManage
|
||||
|| !row.configured
|
||||
@@ -268,27 +265,9 @@ export function DatabaseForm({
|
||||
) : null}
|
||||
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
{mode === "add" ? (
|
||||
<Field label="Workspace">
|
||||
<select
|
||||
className={inputClass}
|
||||
aria-label="Workspace"
|
||||
value={draft.workspaceId}
|
||||
disabled={workspaceLocked || busy}
|
||||
onChange={(event) => onWorkspaceChange(event.target.value)}
|
||||
>
|
||||
{availableWorkspaces.map((workspace) => (
|
||||
<option key={workspace.workspaceId} value={workspace.workspaceId}>
|
||||
{workspace.workspaceName}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
</Field>
|
||||
) : (
|
||||
<Field label="Workspace">
|
||||
<input className={inputClass} value={row.workspaceName} readOnly />
|
||||
</Field>
|
||||
)}
|
||||
<Field label="Workspace">
|
||||
<input className={inputClass} value={row.workspaceName} readOnly />
|
||||
</Field>
|
||||
<Field label="Workspace ID">
|
||||
<input className={`${inputClass} font-mono text-xs`} value={row.workspaceId} readOnly />
|
||||
</Field>
|
||||
@@ -420,13 +399,13 @@ export function DatabaseForm({
|
||||
<TestTube2 /> {busyAction === "test" ? "Testing…" : "Test connection"}
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "add" || mode === "edit" ? (
|
||||
{mode === "configure" || mode === "edit" ? (
|
||||
<Button type="submit" disabled={saveDisabled}>
|
||||
<Save />
|
||||
{busyAction === "save"
|
||||
? "Saving…"
|
||||
: mode === "add"
|
||||
? workspaceLocked ? "Save database" : "Add database"
|
||||
: mode === "configure"
|
||||
? "Save catalog configuration"
|
||||
: "Save changes"}
|
||||
</Button>
|
||||
) : null}
|
||||
@@ -459,6 +438,16 @@ export function DatabaseForm({
|
||||
{activeSyncRun ? <RefreshCw className="animate-spin" /> : <History />} Sync history
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "view" && row.configured && onOpenDescriptionHistory ? (
|
||||
<Button type="button" variant="outline" size="sm" onClick={onOpenDescriptionHistory}>
|
||||
<History /> Description history
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "view" && row.configured && onOpenSensitiveHistory ? (
|
||||
<Button type="button" variant="outline" size="sm" onClick={onOpenSensitiveHistory}>
|
||||
<History /> Sensitive history
|
||||
</Button>
|
||||
) : null}
|
||||
{mode === "view" ? (
|
||||
<DatabaseSyncMenu
|
||||
disabled={!canManage || !bindingReady || busy || Boolean(activeSyncRun)}
|
||||
@@ -466,6 +455,8 @@ export function DatabaseForm({
|
||||
onSelect={onSync}
|
||||
/>
|
||||
) : null}
|
||||
{mode === "view" && row.configured && onOpenDescriptionHistory ? <Button type="button" variant="outline" onClick={onOpenDescriptionHistory}><History /> Description history</Button> : null}
|
||||
{mode === "view" && row.configured && onOpenSensitiveHistory ? <Button type="button" variant="outline" onClick={onOpenSensitiveHistory}><History /> Sensitive history</Button> : null}
|
||||
</div>
|
||||
</div>
|
||||
{row.lastTestedAt ? (
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect, useMemo, useRef, useState, type ReactNode, type RefObject } from "react";
|
||||
import { useEffect, useMemo, useRef, useState, type RefObject } from "react";
|
||||
import { Menu } from "@base-ui/react/menu";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
import {
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
} from "ag-grid-community";
|
||||
import "ag-grid-community/styles/ag-grid.css";
|
||||
import "ag-grid-community/styles/ag-theme-alpine.css";
|
||||
import { ChevronDown, Eye, History, Info, Pencil, RefreshCw, Rows3, Sparkles, Trash2, X } from "lucide-react";
|
||||
import { ChevronDown, Eye, History, Info, Link2, Pencil, RefreshCw, Rows3, Sparkles, Trash2, X } from "lucide-react";
|
||||
import { Button } from "../../components/ui/button";
|
||||
import type {
|
||||
CatalogDatabase,
|
||||
@@ -17,9 +17,9 @@ import type {
|
||||
CatalogSyncScope,
|
||||
DescriptionGenerationScope,
|
||||
} from "../../api/catalog-databases";
|
||||
import { statusLabel } from "./model";
|
||||
import { databaseSyncItemClass, databaseSyncScopes } from "./DatabaseSyncMenu";
|
||||
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
|
||||
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
|
||||
|
||||
ModuleRegistry.registerModules([AllCommunityModule]);
|
||||
|
||||
@@ -33,6 +33,7 @@ interface DatabaseGridProps {
|
||||
onSearchChange: (value: string) => void;
|
||||
onView: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onOpenTables?: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onOpenRelationships?: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onEdit: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onDelete: (row: CatalogDatabase, origin: HTMLButtonElement) => void;
|
||||
onOpenSync: (row: CatalogDatabase) => void;
|
||||
@@ -49,7 +50,6 @@ interface DatabaseGridProps {
|
||||
rows: CatalogDatabase[],
|
||||
target: CatalogDatabaseMetadataDeleteTarget,
|
||||
) => Promise<void>;
|
||||
metadataModelControl?: ReactNode;
|
||||
onRefresh?: () => void | Promise<void>;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
@@ -59,6 +59,7 @@ interface DatabaseGridContext {
|
||||
presentation: "legacy" | "fleet";
|
||||
onView: DatabaseGridProps["onView"];
|
||||
onOpenTables?: DatabaseGridProps["onOpenTables"];
|
||||
onOpenRelationships?: DatabaseGridProps["onOpenRelationships"];
|
||||
onEdit: DatabaseGridProps["onEdit"];
|
||||
onDelete: DatabaseGridProps["onDelete"];
|
||||
onOpenSync: DatabaseGridProps["onOpenSync"];
|
||||
@@ -91,18 +92,120 @@ function useCompactViewport(): boolean {
|
||||
return compact;
|
||||
}
|
||||
|
||||
function StatusPill({ row }: { row: CatalogDatabase }) {
|
||||
const tone = !row.workspaceAvailable
|
||||
? "border-destructive/30 bg-destructive/10 text-destructive"
|
||||
: !row.configured
|
||||
? "border-border bg-muted text-muted-foreground"
|
||||
: row.connectionStatus === "reachable"
|
||||
? "border-emerald-300/70 bg-emerald-50 text-emerald-800 dark:bg-emerald-950/30 dark:text-emerald-300"
|
||||
: row.connectionStatus === "failed"
|
||||
? "border-destructive/30 bg-destructive/10 text-destructive"
|
||||
: "border-amber-300/70 bg-amber-50 text-amber-900 dark:bg-amber-950/30 dark:text-amber-300";
|
||||
type StateTone = "neutral" | "info" | "success" | "warning" | "danger";
|
||||
|
||||
return <div className="flex items-center gap-1.5"><span className={`inline-flex max-w-full truncate rounded-full border px-2 py-0.5 text-[11px] font-semibold ${tone}`}>{statusLabel(row)}</span>{row.activeSyncRun ? <span className="inline-flex rounded-full border border-primary/30 bg-primary/8 px-2 py-0.5 text-[11px] font-semibold text-primary">Syncing</span> : null}</div>;
|
||||
const stateToneClass: Record<StateTone, string> = {
|
||||
neutral: "bg-muted-foreground/55",
|
||||
info: "bg-sky-500",
|
||||
success: "bg-emerald-500",
|
||||
warning: "bg-amber-500",
|
||||
danger: "bg-destructive",
|
||||
};
|
||||
|
||||
function StateCell({
|
||||
label,
|
||||
detail,
|
||||
tone,
|
||||
detailClassName = "text-muted-foreground",
|
||||
}: {
|
||||
label: string;
|
||||
detail: string;
|
||||
tone: StateTone;
|
||||
detailClassName?: string;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex h-full min-w-0 flex-col justify-center gap-0.5 py-1 leading-tight">
|
||||
<span className="flex min-w-0 items-center gap-1.5 text-xs font-semibold text-foreground">
|
||||
<span className={`size-1.5 shrink-0 rounded-full ${stateToneClass[tone]}`} aria-hidden="true" />
|
||||
<span className="truncate">{label}</span>
|
||||
</span>
|
||||
<span className={`truncate pl-3 text-[11px] ${detailClassName}`}>{detail}</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function evidenceStatus(row: CatalogDatabase): { label: string; tone: StateTone } {
|
||||
if (!row.workspaceEvidence) {
|
||||
return { label: "Evidence state unavailable", tone: "warning" };
|
||||
}
|
||||
switch (row.workspaceEvidence.state) {
|
||||
case "materialized_current_revision":
|
||||
return { label: "Evidence materialized", tone: "success" };
|
||||
case "configuration_required":
|
||||
return { label: "Evidence credentials required", tone: "warning" };
|
||||
case "configured_unverified":
|
||||
return { label: "Evidence configured, unverified", tone: "info" };
|
||||
case "workspace_unavailable":
|
||||
return { label: "Evidence unavailable", tone: "danger" };
|
||||
default:
|
||||
return { label: "No Evidence declared", tone: "neutral" };
|
||||
}
|
||||
}
|
||||
|
||||
function RevisionEvidenceCell({ row }: { row: CatalogDatabase }) {
|
||||
if (!row.workspaceRevision) {
|
||||
return <StateCell label="Workspace missing" detail="Revision and Evidence unavailable" tone="danger" />;
|
||||
}
|
||||
const evidence = evidenceStatus(row);
|
||||
const source = row.workspaceEvidence?.sourceType ? ` · ${row.workspaceEvidence.sourceType}` : "";
|
||||
return (
|
||||
<StateCell
|
||||
label={`Active revision ${row.workspaceRevision.commit.slice(0, 7)}`}
|
||||
detail={`${evidence.label}${source}`}
|
||||
tone={evidence.tone}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function transportLabel(row: CatalogDatabase): string {
|
||||
const transport = row.runtimeBinding?.transport;
|
||||
if (transport === "postgres_direct") return "Direct PostgreSQL";
|
||||
if (transport === "rest_api") return "REST API";
|
||||
if (transport === "ssh_tunnel") return "SSH tunnel";
|
||||
return "No runtime binding";
|
||||
}
|
||||
|
||||
function RuntimeBindingCell({ row }: { row: CatalogDatabase }) {
|
||||
if (!row.runtimeBinding) {
|
||||
return <StateCell label="Unavailable" detail="Workspace is not in the active revision" tone="danger" />;
|
||||
}
|
||||
if (!row.runtimeBinding.sessionTransportSupported) {
|
||||
return <StateCell label="Unsupported for NL→SQL" detail={transportLabel(row)} tone="danger" />;
|
||||
}
|
||||
if (row.runtimeBinding.configurationState === "configuration_required") {
|
||||
return <StateCell label="Configuration required" detail={transportLabel(row)} tone="warning" />;
|
||||
}
|
||||
return <StateCell label="Ready" detail={transportLabel(row)} tone="success" />;
|
||||
}
|
||||
|
||||
function catalogConnectionDetail(row: CatalogDatabase): { label: string; className?: string } {
|
||||
if (row.activeSyncRun) return { label: "Metadata sync in progress", className: "text-primary" };
|
||||
if (row.testedVersion !== undefined && row.testedVersion !== row.version) {
|
||||
return { label: "Connection retest required", className: "text-amber-700 dark:text-amber-300" };
|
||||
}
|
||||
if (row.connectionStatus === "reachable") return { label: "Connection reachable" };
|
||||
if (row.connectionStatus === "failed") {
|
||||
return { label: "Connection test failed", className: "text-destructive" };
|
||||
}
|
||||
return { label: "Connection not tested" };
|
||||
}
|
||||
|
||||
function MetadataCatalogCell({ row }: { row: CatalogDatabase }) {
|
||||
if (!row.workspaceAvailable) {
|
||||
return <StateCell label="Orphaned configuration" detail="Workspace missing from repository" tone="danger" />;
|
||||
}
|
||||
if (!row.configured) {
|
||||
return <StateCell label="Not configured" detail="Configure this workspace" tone="neutral" />;
|
||||
}
|
||||
const connection = catalogConnectionDetail(row);
|
||||
return (
|
||||
<StateCell
|
||||
label="Configured"
|
||||
detail={connection.label}
|
||||
detailClassName={connection.className}
|
||||
tone="success"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function DatabaseActionsCell({
|
||||
@@ -111,11 +214,48 @@ function DatabaseActionsCell({
|
||||
}: ICellRendererParams<CatalogDatabase, unknown, DatabaseGridContext>) {
|
||||
if (!data || !context) return null;
|
||||
|
||||
if (data.workspaceAvailable && !data.configured) {
|
||||
const configureReasonId = `database-configure-reason-${data.workspaceId}`;
|
||||
return (
|
||||
<div className="flex h-full items-center justify-end gap-1.5" onClick={(event) => event.stopPropagation()}>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon-lg"
|
||||
className={context.presentation === "fleet" ? "thot-fleet-icon-action" : undefined}
|
||||
data-tooltip={context.presentation === "fleet" ? "Details" : undefined}
|
||||
title={context.presentation === "legacy" ? `View ${data.workspaceName}` : undefined}
|
||||
aria-label={`View ${data.workspaceName}`}
|
||||
onClick={(event) => context.onView(data, event.currentTarget)}
|
||||
>
|
||||
{context.presentation === "fleet" ? <Info aria-hidden="true" /> : <Eye aria-hidden="true" />}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
className="whitespace-nowrap"
|
||||
aria-label={`Configure catalog for ${data.workspaceName}`}
|
||||
aria-describedby={!context.canManage ? configureReasonId : undefined}
|
||||
disabled={!context.canManage}
|
||||
onClick={(event) => context.onEdit(data, event.currentTarget)}
|
||||
>
|
||||
Configure catalog
|
||||
</Button>
|
||||
{!context.canManage ? (
|
||||
<span id={configureReasonId} className="sr-only">
|
||||
Database management permission is required.
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const editDisabled = !context.canManage || !data.workspaceAvailable;
|
||||
const deleteDisabled = !context.canManage || !data.configured;
|
||||
const editLabel = `Edit ${data.workspaceName}`;
|
||||
const editReasonId = `database-edit-reason-${data.workspaceId}`;
|
||||
const deleteReasonId = `database-delete-reason-${data.workspaceId}`;
|
||||
const relationshipsReasonId = `database-relationships-reason-${data.workspaceId}`;
|
||||
|
||||
return (
|
||||
<div className="flex h-full items-center justify-end gap-0.5" onClick={(event) => event.stopPropagation()}>
|
||||
@@ -138,6 +278,26 @@ function DatabaseActionsCell({
|
||||
</Button>
|
||||
</span>
|
||||
) : null}
|
||||
{context.presentation === "fleet" && context.onOpenRelationships ? (
|
||||
<span className="thot-fleet-icon-action inline-flex" data-tooltip="Relationships">
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon-lg"
|
||||
aria-label={`View relationships for ${data.workspaceName}`}
|
||||
aria-describedby={!data.configured || !data.id ? relationshipsReasonId : undefined}
|
||||
disabled={!data.configured || !data.id}
|
||||
onClick={(event) => context.onOpenRelationships?.(data, event.currentTarget)}
|
||||
>
|
||||
<Link2 aria-hidden="true" />
|
||||
</Button>
|
||||
{!data.configured || !data.id ? (
|
||||
<span id={relationshipsReasonId} className="sr-only">
|
||||
Save this database configuration before managing relationships.
|
||||
</span>
|
||||
) : null}
|
||||
</span>
|
||||
) : null}
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
@@ -193,6 +353,7 @@ export function DatabaseGrid({
|
||||
onSearchChange,
|
||||
onView,
|
||||
onOpenTables,
|
||||
onOpenRelationships,
|
||||
onEdit,
|
||||
onDelete,
|
||||
onOpenSync,
|
||||
@@ -203,7 +364,6 @@ export function DatabaseGrid({
|
||||
onGenerateDescriptions,
|
||||
onSuggestSensitive,
|
||||
onDeleteMetadataSelected,
|
||||
metadataModelControl,
|
||||
onRefresh,
|
||||
presentation = "legacy",
|
||||
}: DatabaseGridProps) {
|
||||
@@ -218,8 +378,8 @@ export function DatabaseGrid({
|
||||
Extract<DescriptionGenerationScope, "all" | "missing"> | null
|
||||
>(null);
|
||||
const context = useMemo<DatabaseGridContext>(
|
||||
() => ({ canManage, presentation, onView, onOpenTables, onEdit, onDelete, onOpenSync }),
|
||||
[canManage, presentation, onView, onOpenTables, onEdit, onDelete, onOpenSync],
|
||||
() => ({ canManage, presentation, onView, onOpenTables, onOpenRelationships, onEdit, onDelete, onOpenSync }),
|
||||
[canManage, presentation, onView, onOpenTables, onOpenRelationships, onEdit, onDelete, onOpenSync],
|
||||
);
|
||||
|
||||
const columnDefs = useMemo<ColDef<CatalogDatabase>[]>(() => {
|
||||
@@ -230,19 +390,38 @@ export function DatabaseGrid({
|
||||
flex: compact ? undefined : 1.35,
|
||||
width: compact ? 112 : undefined,
|
||||
};
|
||||
const revisionEvidence: ColDef<CatalogDatabase> = {
|
||||
headerName: "Revision / Evidence",
|
||||
minWidth: 215,
|
||||
flex: 1.2,
|
||||
valueGetter: ({ data }) => {
|
||||
if (!data) return "";
|
||||
const revision = data.workspaceRevision?.commit.slice(0, 7) ?? "workspace missing";
|
||||
return `${revision} ${evidenceStatus(data).label}`;
|
||||
},
|
||||
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => (
|
||||
data ? <RevisionEvidenceCell row={data} /> : null
|
||||
),
|
||||
};
|
||||
const runtimeBinding: ColDef<CatalogDatabase> = {
|
||||
headerName: "NL→SQL runtime",
|
||||
minWidth: 175,
|
||||
flex: 1,
|
||||
valueGetter: ({ data }) => {
|
||||
if (!data?.runtimeBinding) return "Unavailable";
|
||||
if (!data.runtimeBinding.sessionTransportSupported) return "Unsupported for NL→SQL";
|
||||
return data.runtimeBinding.configurationState === "ready" ? "Ready" : "Configuration required";
|
||||
},
|
||||
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => (
|
||||
data ? <RuntimeBindingCell row={data} /> : null
|
||||
),
|
||||
};
|
||||
const database: ColDef<CatalogDatabase> = {
|
||||
field: "databaseName", headerName: "Database", minWidth: 145, flex: 1,
|
||||
};
|
||||
const schema: ColDef<CatalogDatabase> = {
|
||||
field: "schema", headerName: "Schema", minWidth: 140, flex: 0.9,
|
||||
};
|
||||
const transport: ColDef<CatalogDatabase> = {
|
||||
field: "binding.transport",
|
||||
headerName: "Transport",
|
||||
minWidth: 135,
|
||||
flex: 0.9,
|
||||
valueFormatter: ({ value }) => String(value ?? "").replaceAll("_", " "),
|
||||
};
|
||||
const endpoint: ColDef<CatalogDatabase> = {
|
||||
headerName: "Endpoint",
|
||||
minWidth: 190,
|
||||
@@ -254,15 +433,17 @@ export function DatabaseGrid({
|
||||
return data.binding.host ? `${data.binding.host}:${data.binding.port ?? 5432}` : "";
|
||||
},
|
||||
};
|
||||
const status: ColDef<CatalogDatabase> = {
|
||||
headerName: "Status",
|
||||
minWidth: compact ? 96 : 130,
|
||||
width: compact ? 96 : undefined,
|
||||
flex: compact ? undefined : 0.8,
|
||||
cellClass: compact ? "thot-database-status-cell" : undefined,
|
||||
valueGetter: ({ data }) => (data ? statusLabel(data) : ""),
|
||||
const metadataCatalog: ColDef<CatalogDatabase> = {
|
||||
headerName: "Metadata Catalog",
|
||||
minWidth: 185,
|
||||
flex: 1.05,
|
||||
valueGetter: ({ data }) => {
|
||||
if (!data) return "";
|
||||
if (!data.workspaceAvailable) return "Orphaned configuration";
|
||||
return data.configured ? `Configured ${catalogConnectionDetail(data).label}` : "Not configured";
|
||||
},
|
||||
cellRenderer: ({ data }: ICellRendererParams<CatalogDatabase>) => (
|
||||
data ? <StatusPill row={data} /> : null
|
||||
data ? <MetadataCatalogCell row={data} /> : null
|
||||
),
|
||||
};
|
||||
const updated: ColDef<CatalogDatabase> = {
|
||||
@@ -272,7 +453,7 @@ export function DatabaseGrid({
|
||||
flex: 0.9,
|
||||
valueFormatter: ({ value }) => value ? new Date(String(value)).toLocaleString() : "",
|
||||
};
|
||||
const actionsWidth = presentation === "fleet" ? 196 : compact ? 148 : 164;
|
||||
const actionsWidth = presentation === "fleet" ? 232 : 210;
|
||||
const actions: ColDef<CatalogDatabase> = {
|
||||
colId: "actions",
|
||||
headerName: "Actions",
|
||||
@@ -291,8 +472,8 @@ export function DatabaseGrid({
|
||||
};
|
||||
|
||||
return compact
|
||||
? [workspace, status, database, schema, transport, endpoint, updated, actions]
|
||||
: [workspace, database, schema, transport, endpoint, status, updated, actions];
|
||||
? [workspace, revisionEvidence, runtimeBinding, metadataCatalog, database, schema, endpoint, updated, actions]
|
||||
: [workspace, revisionEvidence, runtimeBinding, database, schema, endpoint, metadataCatalog, updated, actions];
|
||||
}, [compact, presentation]);
|
||||
|
||||
const configuredCount = rows.filter((row) => row.configured).length;
|
||||
@@ -332,14 +513,14 @@ export function DatabaseGrid({
|
||||
: undefined);
|
||||
const generationReason = permissionReason
|
||||
?? (selectedRows.length !== 1 ? "Select exactly one database" : undefined)
|
||||
?? (!selectedMetadataModel ? "Choose a metadata model in the generation flow" : undefined)
|
||||
?? (!selectedMetadataModel ? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE : undefined)
|
||||
?? (descriptionGenerationActive ? "Wait for active description generation" : undefined)
|
||||
?? (selectedRows.some((row) => !row.configured || !row.id || row.activeSyncRun)
|
||||
? "The selected database must be configured and idle"
|
||||
: undefined);
|
||||
const sensitiveReason = permissionReason
|
||||
?? (selectedRows.length !== 1 ? "Select exactly one database" : undefined)
|
||||
?? (!selectedMetadataModel ? "Choose a metadata model in the suggestion flow" : undefined)
|
||||
?? (!selectedMetadataModel ? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE : undefined)
|
||||
?? (descriptionGenerationActive ? "Wait for the active metadata operation" : undefined)
|
||||
?? (selectedRows.some((row) => !row.configured || !row.id || row.activeSyncRun)
|
||||
? "The selected database must be configured and idle"
|
||||
@@ -538,12 +719,6 @@ export function DatabaseGrid({
|
||||
busyLabel={action === "suggest" ? "Suggesting…" : "Running…"}
|
||||
selectRef={actionSelectRef}
|
||||
onRun={runFleetAction}
|
||||
renderContext={(selectedAction) => (
|
||||
metadataModelControl
|
||||
&& ["generate-all", "generate-missing", "suggest-sensitive"].includes(selectedAction)
|
||||
? <div className="thot-fleet-metadata-model">{metadataModelControl}</div>
|
||||
: null
|
||||
)}
|
||||
onClear={() => {
|
||||
gridRef.current?.api.deselectAll();
|
||||
setSelectedRows([]);
|
||||
@@ -631,7 +806,7 @@ export function DatabaseGrid({
|
||||
</Button>
|
||||
) : null}</>}
|
||||
<span className="whitespace-nowrap text-xs tabular-nums text-muted-foreground">
|
||||
{configuredCount}/{rows.length}
|
||||
{configuredCount}/{rows.length} catalogs configured
|
||||
</span>
|
||||
{isFetching && !isLoading ? (
|
||||
<span className="text-xs text-muted-foreground" role="status">Refreshing</span>
|
||||
@@ -660,7 +835,7 @@ export function DatabaseGrid({
|
||||
if (pendingGenerationScope && action === null) setPendingGenerationScope(null);
|
||||
setSelectedRows(api.getSelectedRows());
|
||||
}}
|
||||
rowHeight={44}
|
||||
rowHeight={52}
|
||||
headerHeight={38}
|
||||
animateRows={false}
|
||||
/>
|
||||
|
||||
@@ -0,0 +1,379 @@
|
||||
import { render, screen, waitFor, within } from "@testing-library/react";
|
||||
import userEvent from "@testing-library/user-event";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { http, HttpResponse } from "msw";
|
||||
import { expect, test, vi } from "vitest";
|
||||
|
||||
import type {
|
||||
CatalogColumn,
|
||||
CatalogDatabase,
|
||||
CatalogRelationship,
|
||||
CatalogTable,
|
||||
} from "../../api/catalog-databases";
|
||||
import { Toaster } from "../../components/ui/sonner";
|
||||
import { server } from "../../test/msw";
|
||||
import { DatabaseRelationships } from "./DatabaseRelationships";
|
||||
|
||||
const databaseId = "11111111-1111-4111-8111-111111111111";
|
||||
|
||||
const database: CatalogDatabase = {
|
||||
id: databaseId,
|
||||
workspaceId: "psd",
|
||||
workspaceName: "Policlinico San Donato",
|
||||
workspaceAvailable: true,
|
||||
workspaceRevision: { commit: "a".repeat(40), blob: "b".repeat(40) },
|
||||
workspaceEvidence: { sourceType: "filesystem", state: "materialized_current_revision" },
|
||||
runtimeBinding: { transport: "postgres_direct", configurationState: "ready", sessionTransportSupported: true },
|
||||
configured: true,
|
||||
engine: "postgres",
|
||||
databaseName: "analytics",
|
||||
schema: "public",
|
||||
binding: { transport: "postgres_direct", port: 5432 },
|
||||
connectionStatus: "reachable",
|
||||
testedVersion: 3,
|
||||
version: 3,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
activeSyncRun: undefined,
|
||||
secrets: {
|
||||
password: false,
|
||||
apiKey: false,
|
||||
sshPrivateKey: false,
|
||||
sshPrivateKeyPassphrase: false,
|
||||
sshKnownHosts: false,
|
||||
tlsCa: false,
|
||||
},
|
||||
};
|
||||
|
||||
function relationship(
|
||||
id: string,
|
||||
origin: CatalogRelationship["origin"],
|
||||
status: CatalogRelationship["status"],
|
||||
sourceTableName: string,
|
||||
): CatalogRelationship {
|
||||
const physical = origin === "physical";
|
||||
return {
|
||||
id,
|
||||
databaseId,
|
||||
constraintName: physical ? `${sourceTableName}_user_id_fkey` : null,
|
||||
sourceTableId: `${id}-source-table`,
|
||||
sourceTableName,
|
||||
targetTableId: `${id}-target-table`,
|
||||
targetTableName: "users",
|
||||
updateRule: physical ? "NO ACTION" : null,
|
||||
deleteRule: physical ? "CASCADE" : null,
|
||||
deferrable: false,
|
||||
initiallyDeferred: false,
|
||||
columns: [{
|
||||
position: 1,
|
||||
sourceColumnId: `${id}-source-column`,
|
||||
sourceColumnName: "user_id",
|
||||
targetColumnId: `${id}-target-column`,
|
||||
targetColumnName: "id",
|
||||
}],
|
||||
origin,
|
||||
status,
|
||||
lastSyncedDatabaseVersion: physical ? 3 : null,
|
||||
lastSyncedAt: physical ? "2026-08-31T12:00:00.000Z" : null,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
}
|
||||
|
||||
function renderRelationships(rows: CatalogRelationship[], canManage = true) {
|
||||
server.use(http.get(
|
||||
"/api/catalog/databases/:databaseId/relationships",
|
||||
() => HttpResponse.json(rows),
|
||||
));
|
||||
const client = new QueryClient({ defaultOptions: { queries: { retry: false } } });
|
||||
const callbacks = {
|
||||
onBackToDatabases: vi.fn(),
|
||||
onOpenOverview: vi.fn(),
|
||||
onOpenTables: vi.fn(),
|
||||
onRunStarted: vi.fn(),
|
||||
onOpenSync: vi.fn(),
|
||||
};
|
||||
const view = render(
|
||||
<QueryClientProvider client={client}>
|
||||
<DatabaseRelationships
|
||||
presentation="fleet"
|
||||
database={database}
|
||||
canManage={canManage}
|
||||
{...callbacks}
|
||||
/>
|
||||
<Toaster duration={Infinity} />
|
||||
</QueryClientProvider>,
|
||||
);
|
||||
return { ...view, client, callbacks };
|
||||
}
|
||||
|
||||
test("shows one relationship map and filters active, excluded, and all relationships", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderRelationships([
|
||||
relationship("physical", "physical", "active", "visits"),
|
||||
relationship("generated", "generated", "active", "orders"),
|
||||
relationship("manual", "manual", "active", "payments"),
|
||||
relationship("excluded", "generated", "excluded", "legacy_orders"),
|
||||
]);
|
||||
|
||||
expect(await screen.findByText("visits")).toBeVisible();
|
||||
expect(screen.getByText("orders")).toBeVisible();
|
||||
expect(screen.getByText("payments")).toBeVisible();
|
||||
expect(screen.queryByText("legacy_orders")).not.toBeInTheDocument();
|
||||
expect(screen.getByText("Physical")).toBeVisible();
|
||||
expect(screen.getByText("Generated")).toBeVisible();
|
||||
expect(screen.getByText("Manual")).toBeVisible();
|
||||
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Relationship status" }), "excluded");
|
||||
expect(await screen.findByText("legacy_orders")).toBeVisible();
|
||||
await waitFor(() => expect(screen.queryByText("visits")).not.toBeInTheDocument());
|
||||
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Relationship status" }), "all");
|
||||
expect(await screen.findByText("visits")).toBeVisible();
|
||||
expect(screen.getByText("legacy_orders")).toBeVisible();
|
||||
});
|
||||
|
||||
test("adds a manual relationship from four catalog selections", async () => {
|
||||
const user = userEvent.setup();
|
||||
const orders: CatalogTable = {
|
||||
id: "22222222-2222-4222-8222-222222222222",
|
||||
databaseId,
|
||||
name: "orders",
|
||||
sourceComment: null,
|
||||
description: null,
|
||||
generatedDescription: null,
|
||||
version: 1,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
const users: CatalogTable = {
|
||||
...orders,
|
||||
id: "33333333-3333-4333-8333-333333333333",
|
||||
name: "users",
|
||||
};
|
||||
const sourceColumn: CatalogColumn = {
|
||||
id: "44444444-4444-4444-8444-444444444444",
|
||||
tableId: orders.id,
|
||||
name: "user_id",
|
||||
ordinalPosition: 1,
|
||||
dataType: "bigint",
|
||||
isNullable: false,
|
||||
defaultExpression: null,
|
||||
primaryKeyPosition: null,
|
||||
isPrimaryKey: false,
|
||||
isForeignKey: false,
|
||||
foreignKeyCount: 0,
|
||||
sourceComment: null,
|
||||
description: null,
|
||||
generatedDescription: null,
|
||||
sensitive: false,
|
||||
lastSyncedDatabaseVersion: 3,
|
||||
lastSyncedAt: "2026-08-31T12:00:00.000Z",
|
||||
version: 1,
|
||||
createdAt: "2026-08-31T12:00:00.000Z",
|
||||
updatedAt: "2026-08-31T12:00:00.000Z",
|
||||
};
|
||||
const targetColumn: CatalogColumn = {
|
||||
...sourceColumn,
|
||||
id: "55555555-5555-4555-8555-555555555555",
|
||||
tableId: users.id,
|
||||
name: "id",
|
||||
primaryKeyPosition: 1,
|
||||
isPrimaryKey: true,
|
||||
};
|
||||
let requestBody: unknown;
|
||||
server.use(
|
||||
http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([orders, users])),
|
||||
http.get(
|
||||
"/api/catalog/databases/:databaseId/tables/:tableId/columns",
|
||||
({ params }) => HttpResponse.json(params.tableId === orders.id ? [sourceColumn] : [targetColumn]),
|
||||
),
|
||||
http.post("/api/catalog/databases/:databaseId/relationships", async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json(relationship("created", "manual", "active", "orders"), { status: 201 });
|
||||
}),
|
||||
);
|
||||
renderRelationships([]);
|
||||
|
||||
await user.click(await screen.findByRole("button", { name: "Add relationship" }));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Add relationship" });
|
||||
expect(drawer).toHaveAttribute("aria-modal", "false");
|
||||
const submit = within(drawer).getByRole("button", { name: "Add relationship" });
|
||||
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Source table" }), orders.id);
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Source column" }), sourceColumn.id);
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Target table" }), users.id);
|
||||
await user.selectOptions(screen.getByRole("combobox", { name: "Target column" }), targetColumn.id);
|
||||
expect(submit).toBeEnabled();
|
||||
await user.click(submit);
|
||||
|
||||
await waitFor(() => expect(requestBody).toEqual({
|
||||
sourceColumnId: sourceColumn.id,
|
||||
targetColumnId: targetColumn.id,
|
||||
}));
|
||||
expect(await screen.findByText("Relationship added.")).toBeVisible();
|
||||
await waitFor(() => expect(screen.queryByRole("dialog", { name: "Add relationship" })).not.toBeInTheDocument());
|
||||
});
|
||||
|
||||
test("rebuilds generated relationships with progress and a result summary", async () => {
|
||||
const user = userEvent.setup();
|
||||
let finish: (() => void) | undefined;
|
||||
server.use(http.post(
|
||||
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
|
||||
async () => {
|
||||
await new Promise<void>((resolve) => { finish = resolve; });
|
||||
return HttpResponse.json({ added: 12, alreadyPresent: 7, excluded: 3, ambiguous: 2 });
|
||||
},
|
||||
));
|
||||
renderRelationships([]);
|
||||
|
||||
await user.selectOptions(
|
||||
await screen.findByRole("combobox", { name: "Relationship action" }),
|
||||
"rebuild-generated",
|
||||
);
|
||||
await user.click(screen.getByRole("button", { name: "Run action" }));
|
||||
expect(await screen.findByRole("button", { name: "Rebuilding…" })).toBeDisabled();
|
||||
|
||||
finish?.();
|
||||
expect(await screen.findByText(
|
||||
"Rebuild complete: 12 added, 7 already present, 3 excluded, 2 ambiguous.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
test("reports when rebuilding finds no new relationships", async () => {
|
||||
const user = userEvent.setup();
|
||||
server.use(http.post(
|
||||
"/api/catalog/databases/:databaseId/relationships/rebuild-generated",
|
||||
() => HttpResponse.json({ added: 0, alreadyPresent: 0, excluded: 0, ambiguous: 0 }),
|
||||
));
|
||||
renderRelationships([]);
|
||||
|
||||
await user.selectOptions(
|
||||
await screen.findByRole("combobox", { name: "Relationship action" }),
|
||||
"rebuild-generated",
|
||||
);
|
||||
await user.click(screen.getByRole("button", { name: "Run action" }));
|
||||
expect(await screen.findByText("Rebuild complete: no new relationships found.")).toBeVisible();
|
||||
});
|
||||
|
||||
test("excludes a generated relationship after an explanatory drawer confirmation", async () => {
|
||||
const user = userEvent.setup();
|
||||
const generated = relationship("generated", "generated", "active", "orders");
|
||||
let requestBody: unknown;
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json({ ...generated, status: "excluded" });
|
||||
},
|
||||
));
|
||||
renderRelationships([generated]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship orders.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
expect(within(drawer).getByText("Generated")).toBeVisible();
|
||||
await user.click(within(drawer).getByRole("button", { name: "Exclude" }));
|
||||
expect(within(drawer).getByText(
|
||||
"The relationship will move to Excluded. Rebuilds will not recreate it, and you can restore it later.",
|
||||
)).toBeVisible();
|
||||
const confirmExclude = within(drawer).getByRole("button", { name: "Exclude relationship" });
|
||||
await waitFor(() => expect(confirmExclude).toHaveFocus());
|
||||
await user.click(confirmExclude);
|
||||
|
||||
await waitFor(() => expect(requestBody).toEqual({ status: "excluded" }));
|
||||
expect(await screen.findByText(
|
||||
"Relationship excluded. Future rebuilds will leave it excluded.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
test("keeps physical relationships read-only in the details drawer", async () => {
|
||||
const user = userEvent.setup();
|
||||
renderRelationships([relationship("physical", "physical", "active", "visits")]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship visits.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
expect(within(drawer).getByText("This information is synchronized from the source schema and is read-only.")).toBeVisible();
|
||||
expect(within(drawer).queryByRole("button", { name: "Exclude" })).not.toBeInTheDocument();
|
||||
expect(within(drawer).queryByRole("button", { name: "Restore" })).not.toBeInTheDocument();
|
||||
expect(within(drawer).queryByRole("button", { name: "Delete permanently" })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
test("permanently deletes a logical relationship after warning that rebuild may recreate it", async () => {
|
||||
const user = userEvent.setup();
|
||||
const manual = relationship("manual", "manual", "active", "payments");
|
||||
let deletes = 0;
|
||||
server.use(http.delete(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
() => {
|
||||
deletes += 1;
|
||||
return new HttpResponse(null, { status: 204 });
|
||||
},
|
||||
));
|
||||
renderRelationships([manual]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship payments.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
await user.click(within(drawer).getByRole("button", { name: "Delete permanently" }));
|
||||
expect(within(drawer).getByText(
|
||||
"The relationship record will be erased. A future rebuild may infer it again.",
|
||||
)).toBeVisible();
|
||||
await user.click(within(drawer).getByRole("button", { name: "Delete permanently" }));
|
||||
|
||||
await waitFor(() => expect(deletes).toBe(1));
|
||||
expect(await screen.findByText(
|
||||
"Relationship deleted permanently. A future rebuild may infer it again.",
|
||||
)).toBeVisible();
|
||||
});
|
||||
|
||||
test("restores an excluded logical relationship", async () => {
|
||||
const user = userEvent.setup();
|
||||
const excluded = relationship("excluded", "generated", "excluded", "legacy_orders");
|
||||
let requestBody: unknown;
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
async ({ request }) => {
|
||||
requestBody = await request.json();
|
||||
return HttpResponse.json({ ...excluded, status: "active" });
|
||||
},
|
||||
));
|
||||
renderRelationships([excluded]);
|
||||
|
||||
await user.selectOptions(
|
||||
await screen.findByRole("combobox", { name: "Relationship status" }),
|
||||
"excluded",
|
||||
);
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship legacy_orders.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
await user.click(within(drawer).getByRole("button", { name: "Restore" }));
|
||||
|
||||
await waitFor(() => expect(requestBody).toEqual({ status: "active" }));
|
||||
expect(await screen.findByText("Relationship restored.")).toBeVisible();
|
||||
});
|
||||
|
||||
test("keeps relationship context open when a logical mutation fails", async () => {
|
||||
const user = userEvent.setup();
|
||||
const generated = relationship("generated-error", "generated", "active", "orders");
|
||||
server.use(http.patch(
|
||||
"/api/catalog/databases/:databaseId/relationships/:relationshipId",
|
||||
() => HttpResponse.json({ code: "relationship_not_found" }, { status: 404 }),
|
||||
));
|
||||
renderRelationships([generated]);
|
||||
|
||||
await user.click(await screen.findByRole("button", {
|
||||
name: "View relationship orders.user_id to users.id",
|
||||
}));
|
||||
const drawer = await screen.findByRole("dialog", { name: "Relationship details" });
|
||||
await user.click(within(drawer).getByRole("button", { name: "Exclude" }));
|
||||
await user.click(within(drawer).getByRole("button", { name: "Exclude relationship" }));
|
||||
|
||||
expect(await screen.findByText("The relationship no longer exists. Refresh and try again.")).toBeVisible();
|
||||
expect(screen.getByRole("dialog", { name: "Relationship details" })).toBeVisible();
|
||||
});
|
||||
@@ -1,19 +1,26 @@
|
||||
import { useMemo, useRef, useState, type ReactNode } from "react";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
import type { ColDef, ICellRendererParams } from "ag-grid-community";
|
||||
import { ArrowLeft, History, Info, Play, RefreshCw } from "lucide-react";
|
||||
import { ArrowLeft, History, Info, Play, Plus, RefreshCw } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import { Button } from "../../components/ui/button";
|
||||
import { apiErrorMessage } from "../../api/client";
|
||||
import { ApiError, apiErrorMessage } from "../../api/client";
|
||||
import {
|
||||
createCatalogRelationship,
|
||||
deleteCatalogRelationship,
|
||||
listCatalogRelationships,
|
||||
listCatalogColumns,
|
||||
listCatalogTables,
|
||||
rebuildGeneratedRelationships,
|
||||
setCatalogRelationshipStatus,
|
||||
startCatalogSync,
|
||||
type CatalogDatabase,
|
||||
type CatalogRelationship,
|
||||
type CatalogSyncRun,
|
||||
} from "../../api/catalog-databases";
|
||||
import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
import type { DatabaseNavigationState } from "./model";
|
||||
|
||||
interface Props {
|
||||
database: CatalogDatabase;
|
||||
@@ -23,7 +30,10 @@ interface Props {
|
||||
onOpenTables: () => void;
|
||||
onRunStarted: (run: CatalogSyncRun) => void;
|
||||
onOpenSync: () => void;
|
||||
metadataModelControl?: ReactNode;
|
||||
onOpenDescriptionHistory?: () => void;
|
||||
onOpenSensitiveHistory?: () => void;
|
||||
onCatalogMetricsChanged?: () => void | Promise<void>;
|
||||
onNavigationStateChange?: (state: DatabaseNavigationState) => void;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
|
||||
@@ -32,6 +42,19 @@ interface RelationshipGridContext {
|
||||
onView: (relationship: CatalogRelationship, origin: HTMLButtonElement) => void;
|
||||
}
|
||||
|
||||
type RelationshipStatusFilter = "active" | "excluded" | "all";
|
||||
|
||||
function titleCase(value: string): string {
|
||||
return value.charAt(0).toUpperCase() + value.slice(1);
|
||||
}
|
||||
|
||||
function relationshipLabel(relationship: CatalogRelationship): string {
|
||||
const pair = relationship.columns[0];
|
||||
const source = `${relationship.sourceTableName}${pair ? `.${pair.sourceColumnName}` : ""}`;
|
||||
const target = `${relationship.targetTableName}${pair ? `.${pair.targetColumnName}` : ""}`;
|
||||
return `${source} to ${target}`;
|
||||
}
|
||||
|
||||
function RelationshipActionsCell({ data, context }: ICellRendererParams<CatalogRelationship, unknown, RelationshipGridContext>) {
|
||||
if (!data || !context || context.presentation !== "fleet") return null;
|
||||
return (
|
||||
@@ -42,7 +65,7 @@ function RelationshipActionsCell({ data, context }: ICellRendererParams<CatalogR
|
||||
size="icon-lg"
|
||||
className="thot-fleet-icon-action"
|
||||
data-tooltip="Details"
|
||||
aria-label={`View relationship ${data.constraintName}`}
|
||||
aria-label={`View relationship ${relationshipLabel(data)}`}
|
||||
onClick={(event) => context.onView(data, event.currentTarget)}
|
||||
>
|
||||
<Info aria-hidden="true" />
|
||||
@@ -59,32 +82,86 @@ export function DatabaseRelationships({
|
||||
onOpenTables,
|
||||
onRunStarted,
|
||||
onOpenSync,
|
||||
onOpenDescriptionHistory,
|
||||
onOpenSensitiveHistory,
|
||||
onCatalogMetricsChanged,
|
||||
onNavigationStateChange,
|
||||
presentation = "legacy",
|
||||
}: Props) {
|
||||
const databaseId = database.id!;
|
||||
const queryClient = useQueryClient();
|
||||
const { data = [], error, isError, isLoading, isFetching, refetch } = useQuery({
|
||||
queryKey: ["catalog-relationships", databaseId],
|
||||
queryFn: () => listCatalogRelationships(databaseId),
|
||||
retry: false,
|
||||
});
|
||||
const [search, setSearch] = useState("");
|
||||
const [statusFilter, setStatusFilter] = useState<RelationshipStatusFilter>("active");
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [selectedAction, setSelectedAction] = useState<"" | "relationships" | "all">("");
|
||||
const [selectedAction, setSelectedAction] = useState<"" | "rebuild-generated" | "relationships" | "all">("");
|
||||
const [activeRelationshipId, setActiveRelationshipId] = useState<string | null>(null);
|
||||
const [addOpen, setAddOpen] = useState(false);
|
||||
const [sourceTableId, setSourceTableId] = useState("");
|
||||
const [sourceColumnId, setSourceColumnId] = useState("");
|
||||
const [targetTableId, setTargetTableId] = useState("");
|
||||
const [targetColumnId, setTargetColumnId] = useState("");
|
||||
const [addError, setAddError] = useState<string | null>(null);
|
||||
const [targetColumnError, setTargetColumnError] = useState<string | null>(null);
|
||||
const [pendingMutation, setPendingMutation] = useState<"exclude" | "delete" | null>(null);
|
||||
const originRef = useRef<HTMLButtonElement | null>(null);
|
||||
const statusFilterRef = useRef<HTMLSelectElement | null>(null);
|
||||
const bindingReady = database.connectionStatus === "reachable" && database.testedVersion === database.version;
|
||||
const activeRelationship = activeRelationshipId
|
||||
? data.find((relationship) => relationship.id === activeRelationshipId)
|
||||
: undefined;
|
||||
const visibleRelationships = useMemo(() => statusFilter === "all"
|
||||
? data
|
||||
: data.filter((relationship) => relationship.status === statusFilter), [data, statusFilter]);
|
||||
const tablesQuery = useQuery({
|
||||
queryKey: ["catalog-tables", databaseId],
|
||||
queryFn: () => listCatalogTables(databaseId),
|
||||
enabled: addOpen,
|
||||
retry: false,
|
||||
});
|
||||
const sourceColumnsQuery = useQuery({
|
||||
queryKey: ["catalog-columns", databaseId, sourceTableId],
|
||||
queryFn: () => listCatalogColumns(databaseId, sourceTableId),
|
||||
enabled: addOpen && Boolean(sourceTableId),
|
||||
retry: false,
|
||||
});
|
||||
const targetColumnsQuery = useQuery({
|
||||
queryKey: ["catalog-columns", databaseId, targetTableId],
|
||||
queryFn: () => listCatalogColumns(databaseId, targetTableId),
|
||||
enabled: addOpen && Boolean(targetTableId),
|
||||
retry: false,
|
||||
});
|
||||
const targetColumns = (targetColumnsQuery.data ?? []).filter((column) => column.isPrimaryKey);
|
||||
useEffect(() => {
|
||||
if (!pendingMutation) return;
|
||||
const timer = window.setTimeout(() => document
|
||||
.getElementById("relationship-mutation-confirm-button")?.focus(), 0);
|
||||
return () => window.clearTimeout(timer);
|
||||
}, [pendingMutation]);
|
||||
useEffect(() => {
|
||||
onNavigationStateChange?.({ dirty: false, busy });
|
||||
return () => onNavigationStateChange?.({ dirty: false, busy: false });
|
||||
}, [busy, onNavigationStateChange]);
|
||||
const selectedActionUnavailable = !selectedAction
|
||||
? null
|
||||
: !canManage
|
||||
? "You do not have permission to synchronize the catalog."
|
||||
: !bindingReady
|
||||
? "Test the current database binding first."
|
||||
: database.activeSyncRun
|
||||
? "Wait for the active synchronization to finish."
|
||||
? "You do not have permission to manage catalog relationships."
|
||||
: database.activeSyncRun
|
||||
? "Wait for the active synchronization to finish."
|
||||
: selectedAction !== "rebuild-generated" && !bindingReady
|
||||
? "Test the current database binding first."
|
||||
: null;
|
||||
const addUnavailable = !canManage
|
||||
? "You do not have permission to add catalog relationships."
|
||||
: database.activeSyncRun
|
||||
? "Wait for the active synchronization to finish."
|
||||
: busy
|
||||
? "Wait for the current relationship operation to finish."
|
||||
: null;
|
||||
|
||||
const synchronize = async (scope: "relationships" | "all") => {
|
||||
setBusy(true);
|
||||
@@ -97,28 +174,146 @@ export function DatabaseRelationships({
|
||||
|
||||
const openRelationship = (relationship: CatalogRelationship, origin: HTMLButtonElement) => {
|
||||
originRef.current = origin;
|
||||
setPendingMutation(null);
|
||||
setAddOpen(false);
|
||||
setActiveRelationshipId(relationship.id);
|
||||
};
|
||||
|
||||
const closeRelationship = () => {
|
||||
if (busy) return;
|
||||
setPendingMutation(null);
|
||||
setActiveRelationshipId(null);
|
||||
window.setTimeout(() => originRef.current?.focus(), 0);
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) originRef.current.focus();
|
||||
else statusFilterRef.current?.focus();
|
||||
}, 0);
|
||||
};
|
||||
|
||||
const changeRelationshipStatus = async (status: "active" | "excluded") => {
|
||||
if (!activeRelationship || activeRelationship.origin === "physical") return;
|
||||
setBusy(true);
|
||||
try {
|
||||
await setCatalogRelationshipStatus(databaseId, activeRelationship.id, status);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
setPendingMutation(null);
|
||||
setActiveRelationshipId(null);
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) originRef.current.focus();
|
||||
else statusFilterRef.current?.focus();
|
||||
}, 0);
|
||||
toast.success(status === "excluded"
|
||||
? "Relationship excluded. Future rebuilds will leave it excluded."
|
||||
: "Relationship restored.");
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
const permanentlyDeleteRelationship = async () => {
|
||||
if (!activeRelationship || activeRelationship.origin === "physical") return;
|
||||
setBusy(true);
|
||||
try {
|
||||
await deleteCatalogRelationship(databaseId, activeRelationship.id);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
setPendingMutation(null);
|
||||
setActiveRelationshipId(null);
|
||||
window.setTimeout(() => {
|
||||
if (originRef.current?.isConnected) originRef.current.focus();
|
||||
else statusFilterRef.current?.focus();
|
||||
}, 0);
|
||||
toast.success("Relationship deleted permanently. A future rebuild may infer it again.");
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
const resetAddForm = () => {
|
||||
setSourceTableId("");
|
||||
setSourceColumnId("");
|
||||
setTargetTableId("");
|
||||
setTargetColumnId("");
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
};
|
||||
|
||||
const closeAdd = () => {
|
||||
if (busy) return;
|
||||
setAddOpen(false);
|
||||
resetAddForm();
|
||||
};
|
||||
|
||||
const addRelationship = async () => {
|
||||
if (!sourceColumnId || !targetColumnId) return;
|
||||
setBusy(true);
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
try {
|
||||
await createCatalogRelationship(databaseId, sourceColumnId, targetColumnId);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
setAddOpen(false);
|
||||
resetAddForm();
|
||||
toast.success("Relationship added.");
|
||||
} catch (error) {
|
||||
if (error instanceof ApiError && error.code === "relationship_target_not_unique") {
|
||||
setTargetColumnError("The target column must be the only primary-key column of its table.");
|
||||
} else if (error instanceof ApiError && error.code === "relationship_type_incompatible") {
|
||||
setTargetColumnError("Source and target column types are not compatible.");
|
||||
} else if (error instanceof ApiError && error.code === "relationship_duplicate") {
|
||||
setAddError("This relationship already exists.");
|
||||
} else {
|
||||
setAddError(apiErrorMessage(error));
|
||||
}
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
};
|
||||
|
||||
const runSelectedAction = async () => {
|
||||
if (!selectedAction) return;
|
||||
await synchronize(selectedAction);
|
||||
if (selectedAction === "rebuild-generated") {
|
||||
setBusy(true);
|
||||
try {
|
||||
const result = await rebuildGeneratedRelationships(databaseId);
|
||||
await queryClient.invalidateQueries({ queryKey: ["catalog-relationships", databaseId], exact: true });
|
||||
await onCatalogMetricsChanged?.();
|
||||
const noFindings = result.added === 0
|
||||
&& result.alreadyPresent === 0
|
||||
&& result.excluded === 0
|
||||
&& result.ambiguous === 0;
|
||||
toast.success(noFindings
|
||||
? "Rebuild complete: no new relationships found."
|
||||
: `Rebuild complete: ${result.added} added, ${result.alreadyPresent} already present, ${result.excluded} excluded, ${result.ambiguous} ambiguous.`);
|
||||
} catch (error) {
|
||||
toast.error(apiErrorMessage(error));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
} else {
|
||||
await synchronize(selectedAction);
|
||||
}
|
||||
setSelectedAction("");
|
||||
};
|
||||
|
||||
const columns = useMemo<ColDef<CatalogRelationship>[]>(() => [
|
||||
{ field: "constraintName", headerName: "Constraint", minWidth: 220, flex: 1, cellClass: "font-mono text-xs" },
|
||||
...(presentation === "legacy" ? [{ field: "constraintName", headerName: "Constraint", minWidth: 220, flex: 1, cellClass: "font-mono text-xs" } satisfies ColDef<CatalogRelationship>] : []),
|
||||
{ field: "sourceTableName", headerName: "Source table", minWidth: 200, flex: 1 },
|
||||
{ headerName: "Source columns", minWidth: 200, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.sourceColumnName).join(", ") ?? "" },
|
||||
{ headerName: "Source column", minWidth: 180, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.sourceColumnName).join(", ") ?? "" },
|
||||
{ field: "targetTableName", headerName: "Target table", minWidth: 200, flex: 1 },
|
||||
{ headerName: "Target columns", minWidth: 200, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.targetColumnName).join(", ") ?? "" },
|
||||
{ field: "updateRule", headerName: "On update", minWidth: 125, width: 125 },
|
||||
{ field: "deleteRule", headerName: "On delete", minWidth: 125, width: 125 },
|
||||
{ headerName: "Target column", minWidth: 180, flex: 1, valueGetter: ({ data: row }) => row?.columns.map((pair) => pair.targetColumnName).join(", ") ?? "" },
|
||||
{ field: "origin", headerName: "Origin", minWidth: 120, width: 120, valueFormatter: ({ value }) => titleCase(String(value ?? "")) },
|
||||
{ field: "status", headerName: "Status", minWidth: 110, width: 110, valueFormatter: ({ value }) => titleCase(String(value ?? "")) },
|
||||
...(presentation === "legacy" ? [
|
||||
{ field: "updateRule", headerName: "On update", minWidth: 125, width: 125 },
|
||||
{ field: "deleteRule", headerName: "On delete", minWidth: 125, width: 125 },
|
||||
] satisfies ColDef<CatalogRelationship>[] : []),
|
||||
...(presentation === "fleet" ? [{ colId: "actions", headerName: "Actions", width: 64, minWidth: 64, maxWidth: 64, pinned: "right" as const, lockPinned: true, sortable: false, filter: false, resizable: false, suppressMovable: true, suppressHeaderMenuButton: true, cellRenderer: RelationshipActionsCell }] : []),
|
||||
], [presentation]);
|
||||
const context = useMemo<RelationshipGridContext>(() => ({ presentation, onView: openRelationship }), [data, presentation]);
|
||||
@@ -157,14 +352,27 @@ export function DatabaseRelationships({
|
||||
</div> : null}
|
||||
<div className={presentation === "fleet" ? "thot-fleet-grid-surface flex min-h-0 flex-1 flex-col overflow-hidden bg-card" : "mx-3 mb-4 mt-4 flex min-h-0 flex-1 flex-col overflow-hidden rounded-md border border-border bg-card sm:mx-5"}>
|
||||
<div className={`${presentation === "fleet" ? "thot-fleet-grid-toolbar " : ""}flex min-h-12 flex-wrap items-center gap-3 border-b border-border px-3 py-2`}>
|
||||
<span className="thot-label whitespace-nowrap">Physical relationships</span>
|
||||
<span className="thot-label whitespace-nowrap">Relationship map</span>
|
||||
<label className="sr-only" htmlFor="relationship-status-filter">Relationship status</label>
|
||||
<select
|
||||
ref={statusFilterRef}
|
||||
id="relationship-status-filter"
|
||||
className="h-8 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Relationship status"
|
||||
value={statusFilter}
|
||||
onChange={(event) => setStatusFilter(event.target.value as RelationshipStatusFilter)}
|
||||
>
|
||||
<option value="active">Active</option>
|
||||
<option value="excluded">Excluded</option>
|
||||
<option value="all">All</option>
|
||||
</select>
|
||||
<input className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search relationships" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{data.length}</span>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{visibleRelationships.length}</span>
|
||||
<Button type="button" variant="outline" disabled={isFetching || busy} onClick={() => void refetch()}><RefreshCw className={isFetching ? "animate-spin" : ""} />Refresh</Button>
|
||||
{presentation === "fleet" ? (
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<div className="thot-fleet-action-selector__scope" aria-live="polite">
|
||||
<strong>{selectedAction === "all" ? "This database" : selectedAction === "relationships" ? "All relationships in this database" : "This database"}</strong>
|
||||
<strong>{selectedAction === "relationships" ? "Physical relationships in this database" : "This database"}</strong>
|
||||
<span>Action scope</span>
|
||||
</div>
|
||||
<label className="sr-only" htmlFor="relationship-action">Relationship action</label>
|
||||
@@ -173,18 +381,37 @@ export function DatabaseRelationships({
|
||||
className="thot-fleet-action-selector__select"
|
||||
value={selectedAction}
|
||||
disabled={busy}
|
||||
onChange={(event) => setSelectedAction(event.target.value as "" | "relationships" | "all")}
|
||||
onChange={(event) => setSelectedAction(event.target.value as "" | "rebuild-generated" | "relationships" | "all")}
|
||||
>
|
||||
<option value="">Choose an action…</option>
|
||||
<optgroup label="Logical relationships">
|
||||
<option value="rebuild-generated">Rebuild generated relationships</option>
|
||||
</optgroup>
|
||||
<optgroup label="Synchronization">
|
||||
<option value="relationships">Synchronize all relationships in this database</option>
|
||||
<option value="relationships">Synchronize physical relationships</option>
|
||||
<option value="all">Synchronize the full schema for this database</option>
|
||||
</optgroup>
|
||||
</select>
|
||||
<Button type="button" size="sm" disabled={!selectedAction || Boolean(selectedActionUnavailable) || busy} aria-describedby={selectedActionUnavailable ? "relationship-action-reason" : undefined} onClick={() => void runSelectedAction()}><Play aria-hidden="true" />{busy ? "Running…" : "Run action"}</Button>
|
||||
<Button type="button" size="sm" disabled={!selectedAction || Boolean(selectedActionUnavailable) || busy} aria-describedby={selectedActionUnavailable ? "relationship-action-reason" : undefined} onClick={() => void runSelectedAction()}><Play aria-hidden="true" />{busy ? selectedAction === "rebuild-generated" ? "Rebuilding…" : "Running…" : "Run action"}</Button>
|
||||
<Button type="button" size="sm" variant="outline" className="thot-fleet-nav-action thot-fleet-nav-action--below" data-tooltip="Synchronization history" onClick={onOpenSync}>
|
||||
{database.activeSyncRun ? <RefreshCw className="animate-spin" aria-hidden="true" /> : <History aria-hidden="true" />} Sync history
|
||||
</Button>
|
||||
{onOpenDescriptionHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenDescriptionHistory}><History aria-hidden="true" /> Description history</Button> : null}
|
||||
{onOpenSensitiveHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenSensitiveHistory}><History aria-hidden="true" /> Sensitive history</Button> : null}
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
disabled={Boolean(addUnavailable)}
|
||||
aria-describedby={addUnavailable ? "relationship-add-reason" : undefined}
|
||||
onClick={() => {
|
||||
setActiveRelationshipId(null);
|
||||
setPendingMutation(null);
|
||||
setAddOpen(true);
|
||||
}}
|
||||
>
|
||||
<Plus aria-hidden="true" /> Add relationship
|
||||
</Button>
|
||||
{addUnavailable ? <span id="relationship-add-reason" className="sr-only">{addUnavailable}</span> : null}
|
||||
{selectedActionUnavailable ? <span id="relationship-action-reason" className="basis-full text-xs text-muted-foreground" role="status">{selectedActionUnavailable}</span> : null}
|
||||
</div>
|
||||
) : <Button type="button" disabled={!canManage || !bindingReady || busy || Boolean(database.activeSyncRun)} title={!bindingReady ? "Test the current database binding before synchronizing relationships" : undefined} onClick={() => void synchronize("relationships")}><RefreshCw />Sync relationships</Button>}
|
||||
@@ -192,26 +419,174 @@ export function DatabaseRelationships({
|
||||
{!bindingReady ? <div className="border-b border-border bg-muted/35 px-4 py-3 text-sm text-muted-foreground">{presentation === "fleet" ? "Test the current database binding in database details before synchronizing relationships." : "Test the current database binding from Overview before synchronizing relationships."}</div> : null}
|
||||
{fleetQueryError ?? <div className="relative min-h-[280px] flex-1">
|
||||
<div className={`thot-database-grid ${presentation === "fleet" ? "thot-fleet-ledger-grid " : ""}ag-theme-alpine absolute inset-0 h-full w-full`}>
|
||||
<AgGridReact<CatalogRelationship> rowData={data} columnDefs={columns} context={context} loading={isLoading} quickFilterText={search} defaultColDef={{ sortable: true, filter: true, resizable: true }} getRowId={({ data: row }) => row.id} rowHeight={44} headerHeight={38} animateRows={false} overlayNoRowsTemplate="No physical relationships synchronized for this database." />
|
||||
<AgGridReact<CatalogRelationship> rowData={visibleRelationships} columnDefs={columns} context={context} loading={isLoading} quickFilterText={search} defaultColDef={{ sortable: true, filter: true, resizable: true }} getRowId={({ data: row }) => row.id} rowHeight={44} headerHeight={38} animateRows={false} overlayNoRowsTemplate={statusFilter === "active" ? "No relationships yet. Rebuild from column names or add one manually." : statusFilter === "excluded" ? "No excluded relationships." : "No relationships for this database."} />
|
||||
</div>
|
||||
</div>}
|
||||
</div>
|
||||
{presentation === "fleet" && addOpen ? (
|
||||
<FleetLedgerDrawer
|
||||
open
|
||||
eyebrow="Logical relationship"
|
||||
title="Add relationship"
|
||||
description="Connect one source column to one primary-key target column."
|
||||
onClose={closeAdd}
|
||||
closeLabel="Close add relationship"
|
||||
busy={busy}
|
||||
footer={(
|
||||
<>
|
||||
<Button type="button" variant="outline" disabled={busy} onClick={closeAdd}>Cancel</Button>
|
||||
<Button
|
||||
type="button"
|
||||
disabled={busy || !sourceTableId || !sourceColumnId || !targetTableId || !targetColumnId}
|
||||
onClick={() => void addRelationship()}
|
||||
>
|
||||
{busy ? "Adding…" : "Add relationship"}
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
>
|
||||
<div className="grid gap-4 text-sm sm:grid-cols-2">
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Source table</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Source table"
|
||||
value={sourceTableId}
|
||||
disabled={busy || tablesQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setSourceTableId(event.target.value);
|
||||
setSourceColumnId("");
|
||||
setAddError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select source table…</option>
|
||||
{(tablesQuery.data ?? []).map((table) => <option key={table.id} value={table.id}>{table.name}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Source column</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Source column"
|
||||
value={sourceColumnId}
|
||||
disabled={busy || !sourceTableId || sourceColumnsQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setSourceColumnId(event.target.value);
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select source column…</option>
|
||||
{(sourceColumnsQuery.data ?? []).map((column) => <option key={column.id} value={column.id}>{column.name}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Target table</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Target table"
|
||||
value={targetTableId}
|
||||
disabled={busy || tablesQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setTargetTableId(event.target.value);
|
||||
setTargetColumnId("");
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select target table…</option>
|
||||
{(tablesQuery.data ?? []).map((table) => <option key={table.id} value={table.id}>{table.name}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="grid gap-1.5">
|
||||
<span className="font-semibold">Target column</span>
|
||||
<select
|
||||
className="h-9 rounded-md border border-input bg-background px-3 outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15"
|
||||
aria-label="Target column"
|
||||
aria-invalid={Boolean(targetColumnError)}
|
||||
aria-describedby={targetColumnError ? "target-column-error" : undefined}
|
||||
value={targetColumnId}
|
||||
disabled={busy || !targetTableId || targetColumnsQuery.isLoading}
|
||||
onChange={(event) => {
|
||||
setTargetColumnId(event.target.value);
|
||||
setAddError(null);
|
||||
setTargetColumnError(null);
|
||||
}}
|
||||
>
|
||||
<option value="">Select target column…</option>
|
||||
{targetColumns.map((column) => <option key={column.id} value={column.id}>{column.name}</option>)}
|
||||
</select>
|
||||
{targetColumnError ? <span id="target-column-error" className="text-xs text-destructive" role="alert">{targetColumnError}</span> : null}
|
||||
</label>
|
||||
{tablesQuery.isError || sourceColumnsQuery.isError || targetColumnsQuery.isError ? (
|
||||
<p className="sm:col-span-2 text-sm text-destructive" role="alert">Catalog tables or columns could not be loaded.</p>
|
||||
) : null}
|
||||
{addError ? <p className="sm:col-span-2 text-sm text-destructive" role="alert">{addError}</p> : null}
|
||||
</div>
|
||||
</FleetLedgerDrawer>
|
||||
) : null}
|
||||
{presentation === "fleet" && activeRelationship ? (
|
||||
<FleetLedgerDrawer
|
||||
open
|
||||
eyebrow="Physical relationship"
|
||||
eyebrow={`${titleCase(activeRelationship.origin)} relationship`}
|
||||
title="Relationship details"
|
||||
description="This information is synchronized from the source schema and is read-only."
|
||||
description={activeRelationship.origin === "physical"
|
||||
? "This information is synchronized from the source schema and is read-only."
|
||||
: activeRelationship.origin === "generated"
|
||||
? "This relationship was inferred from catalog column names."
|
||||
: "This relationship was added manually."}
|
||||
onClose={closeRelationship}
|
||||
closeLabel="Close relationship details"
|
||||
footer={<Button type="button" variant="outline" onClick={closeRelationship}>Close</Button>}
|
||||
busy={busy}
|
||||
footer={pendingMutation ? (
|
||||
<div
|
||||
className="flex min-w-0 flex-1 flex-wrap items-center gap-2"
|
||||
role="group"
|
||||
aria-labelledby="relationship-mutation-confirmation"
|
||||
onKeyDown={(event) => {
|
||||
if (event.key === "Escape" && !busy) setPendingMutation(null);
|
||||
}}
|
||||
>
|
||||
<p id="relationship-mutation-confirmation" className="mr-auto max-w-xl text-sm text-muted-foreground">
|
||||
{pendingMutation === "exclude"
|
||||
? "The relationship will move to Excluded. Rebuilds will not recreate it, and you can restore it later."
|
||||
: "The relationship record will be erased. A future rebuild may infer it again."}
|
||||
</p>
|
||||
<Button type="button" variant="ghost" disabled={busy} onClick={() => setPendingMutation(null)}>Cancel</Button>
|
||||
<Button
|
||||
id="relationship-mutation-confirm-button"
|
||||
type="button"
|
||||
variant={pendingMutation === "delete" ? "destructive" : "default"}
|
||||
disabled={busy}
|
||||
onClick={() => void (pendingMutation === "exclude"
|
||||
? changeRelationshipStatus("excluded")
|
||||
: permanentlyDeleteRelationship())}
|
||||
>
|
||||
{pendingMutation === "exclude" ? "Exclude relationship" : "Delete permanently"}
|
||||
</Button>
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<Button type="button" variant="outline" disabled={busy} onClick={closeRelationship}>Close</Button>
|
||||
{activeRelationship.origin !== "physical" && activeRelationship.status === "active" ? (
|
||||
<Button type="button" variant="outline" disabled={!canManage || busy} onClick={() => setPendingMutation("exclude")}>Exclude</Button>
|
||||
) : null}
|
||||
{activeRelationship.origin !== "physical" && activeRelationship.status === "excluded" ? (
|
||||
<Button type="button" disabled={!canManage || busy} onClick={() => void changeRelationshipStatus("active")}>Restore</Button>
|
||||
) : null}
|
||||
{activeRelationship.origin !== "physical" ? (
|
||||
<Button type="button" variant="destructive" disabled={!canManage || busy} onClick={() => setPendingMutation("delete")}>Delete permanently</Button>
|
||||
) : null}
|
||||
</>
|
||||
)}
|
||||
>
|
||||
<dl className="grid gap-4 text-sm sm:grid-cols-2">
|
||||
<div className="sm:col-span-2"><dt className="thot-label">Constraint</dt><dd className="mt-1 break-words font-mono text-xs">{activeRelationship.constraintName}</dd></div>
|
||||
<div><dt className="thot-label">Origin</dt><dd className="mt-1">{titleCase(activeRelationship.origin)}</dd></div>
|
||||
<div><dt className="thot-label">Status</dt><dd className="mt-1">{titleCase(activeRelationship.status)}</dd></div>
|
||||
{activeRelationship.constraintName ? <div className="sm:col-span-2"><dt className="thot-label">Constraint</dt><dd className="mt-1 break-words font-mono text-xs">{activeRelationship.constraintName}</dd></div> : null}
|
||||
<div><dt className="thot-label">Source table</dt><dd className="mt-1 break-words">{activeRelationship.sourceTableName}</dd></div>
|
||||
<div><dt className="thot-label">Target table</dt><dd className="mt-1 break-words">{activeRelationship.targetTableName}</dd></div>
|
||||
<div><dt className="thot-label">On update</dt><dd className="mt-1">{activeRelationship.updateRule ?? "Not specified"}</dd></div>
|
||||
<div><dt className="thot-label">On delete</dt><dd className="mt-1">{activeRelationship.deleteRule ?? "Not specified"}</dd></div>
|
||||
{activeRelationship.origin === "physical" ? <><div><dt className="thot-label">On update</dt><dd className="mt-1">{activeRelationship.updateRule ?? "Not specified"}</dd></div><div><dt className="thot-label">On delete</dt><dd className="mt-1">{activeRelationship.deleteRule ?? "Not specified"}</dd></div></> : null}
|
||||
<div className="sm:col-span-2">
|
||||
<dt className="thot-label">Column mapping</dt>
|
||||
<dd className="mt-2 overflow-hidden rounded-md border border-border">
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect, useMemo, useRef, useState, type ReactNode } from "react";
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import { Menu } from "@base-ui/react/menu";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { AgGridReact } from "ag-grid-react";
|
||||
@@ -25,6 +25,7 @@ import type { DatabaseNavigationState } from "./model";
|
||||
import { DatabaseColumns } from "./DatabaseColumns";
|
||||
import { FleetActionSelector, type FleetActionOption } from "./FleetActionSelector";
|
||||
import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
import { NO_METADATA_GENERATION_LLM_MODEL_MESSAGE } from "./MetadataGenerationModelSelector";
|
||||
|
||||
interface Props {
|
||||
database: CatalogDatabase;
|
||||
@@ -39,10 +40,11 @@ interface Props {
|
||||
onNestedNavigationChange?: (active: boolean) => void;
|
||||
onRunStarted: (run: CatalogSyncRun) => void;
|
||||
onOpenSync: () => void;
|
||||
onOpenDescriptionHistory?: () => void;
|
||||
onOpenSensitiveHistory?: () => void;
|
||||
onDescriptionGenerationRunStarted: (run: DescriptionGenerationRun) => void;
|
||||
onSuggestSensitive: (selection: SensitiveDataSuggestionRequest, scopeLabel: string) => Promise<void>;
|
||||
onCatalogMetricsChanged?: () => void | Promise<void>;
|
||||
metadataModelControl?: ReactNode;
|
||||
presentation?: "legacy" | "fleet";
|
||||
}
|
||||
|
||||
@@ -93,10 +95,11 @@ export function DatabaseTables({
|
||||
onNestedNavigationChange,
|
||||
onRunStarted,
|
||||
onOpenSync,
|
||||
onOpenDescriptionHistory,
|
||||
onOpenSensitiveHistory,
|
||||
onDescriptionGenerationRunStarted,
|
||||
onSuggestSensitive,
|
||||
onCatalogMetricsChanged,
|
||||
metadataModelControl,
|
||||
presentation = "legacy",
|
||||
}: Props) {
|
||||
const databaseId = database.id!;
|
||||
@@ -314,7 +317,7 @@ export function DatabaseTables({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one table."
|
||||
: !selectedMetadataModel
|
||||
? "Choose a metadata model first."
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
: descriptionGenerationActive || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy !== null
|
||||
@@ -377,7 +380,7 @@ export function DatabaseTables({
|
||||
: selectedIds.length === 0
|
||||
? "Select at least one table."
|
||||
: !selectedMetadataModel
|
||||
? "Choose a metadata model first."
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
: descriptionGenerationActive || Boolean(currentRun)
|
||||
? "Wait for the active catalog operation to finish."
|
||||
: busy !== null
|
||||
@@ -487,8 +490,9 @@ export function DatabaseTables({
|
||||
<div className="flex min-h-12 flex-wrap items-center gap-3 border-b border-border bg-muted/25 px-3 py-2">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
className="thot-fleet-icon-action border-primary/35 bg-primary/8 shadow-sm"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="thot-fleet-icon-action thot-fleet-back-action"
|
||||
disabled={navigationBusy}
|
||||
data-tooltip="Tables"
|
||||
aria-label="Back to tables"
|
||||
@@ -514,7 +518,6 @@ export function DatabaseTables({
|
||||
bindingReady={bindingReady}
|
||||
catalogOperationActive={Boolean(currentRun)}
|
||||
onCatalogMetricsChanged={onCatalogMetricsChanged}
|
||||
metadataModelControl={metadataModelControl}
|
||||
presentation="fleet"
|
||||
/>
|
||||
</section>
|
||||
@@ -712,7 +715,6 @@ export function DatabaseTables({
|
||||
selectRef={fleetActionSelectRef}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Table action"
|
||||
/>
|
||||
</>
|
||||
@@ -751,7 +753,6 @@ export function DatabaseTables({
|
||||
selectRef={fleetActionSelectRef}
|
||||
onRun={runFleetAction}
|
||||
onClear={clearSelection}
|
||||
renderContext={(action) => action === "generate-descriptions" || action === "suggest-sensitive" ? metadataModelControl : null}
|
||||
label="Table action"
|
||||
/>
|
||||
<input ref={searchInputRef} className="h-8 min-w-40 flex-1 rounded-md border border-input bg-background px-2.5 text-sm outline-none focus:border-primary/60 focus:ring-3 focus:ring-ring/15" aria-label="Search tables" placeholder="Search" value={search} onChange={(event) => setSearch(event.target.value)} />
|
||||
@@ -773,6 +774,8 @@ export function DatabaseTables({
|
||||
{currentRun ? <RefreshCw className="animate-spin" aria-hidden="true" /> : <History aria-hidden="true" />} Sync history
|
||||
</Button>
|
||||
) : null}
|
||||
{onOpenDescriptionHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenDescriptionHistory}><History aria-hidden="true" /> Description history</Button> : null}
|
||||
{onOpenSensitiveHistory ? <Button type="button" size="sm" variant="outline" onClick={onOpenSensitiveHistory}><History aria-hidden="true" /> Sensitive history</Button> : null}
|
||||
</div>
|
||||
{!bindingReady ? <div className="border-b border-border bg-muted/35 px-4 py-3 text-sm text-muted-foreground">Test the current database binding from Overview before synchronizing tables.</div> : null}
|
||||
{fleetQueryError ?? <div className="relative min-h-[280px] flex-1">
|
||||
|
||||
@@ -248,6 +248,33 @@ test("reopens a terminal run from newest-first persisted history", async () => {
|
||||
expect(within(drawer).getByText("3", { selector: "[data-count='generated']" })).toBeVisible();
|
||||
});
|
||||
|
||||
test("styles a successfully completed run with the success background", async () => {
|
||||
const completedRun: DescriptionGenerationRun = {
|
||||
...runningRun,
|
||||
status: "completed",
|
||||
total: 1,
|
||||
processed: 1,
|
||||
generated: 1,
|
||||
updatedAt: "2026-08-28T08:02:00Z",
|
||||
finishedAt: "2026-08-28T08:02:00Z",
|
||||
};
|
||||
server.use(
|
||||
http.get("/api/catalog/description-generation-runs", () => HttpResponse.json([completedRun])),
|
||||
http.get("/api/catalog/description-generation-runs/:runId", () => HttpResponse.json(completedRun)),
|
||||
http.get("/api/catalog/description-generation-runs/:runId/events-list", () => HttpResponse.json([])),
|
||||
);
|
||||
renderDrawer({ initialRun: completedRun });
|
||||
|
||||
const history = await screen.findByRole("region", {
|
||||
name: "Description generation history",
|
||||
});
|
||||
const completedHistoryItem = within(history).getByRole("button", {
|
||||
name: /Completed.*missing/i,
|
||||
});
|
||||
expect(completedHistoryItem).toHaveAttribute("data-status", "completed");
|
||||
expect(completedHistoryItem).toHaveClass("bg-[oklch(var(--success)/0.1)]");
|
||||
});
|
||||
|
||||
test("offers Unlock only for an apparently stale active run and explains the interruption", async () => {
|
||||
const user = userEvent.setup();
|
||||
const staleRun: DescriptionGenerationRun = {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { LoaderCircle, LockOpen, Square } from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
@@ -18,6 +18,7 @@ import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
|
||||
interface Props {
|
||||
open: boolean;
|
||||
databaseId?: string | null;
|
||||
run: DescriptionGenerationRun | null;
|
||||
modelLabel: string;
|
||||
onClose: () => void;
|
||||
@@ -62,6 +63,7 @@ function timestamp(value: string | null): string {
|
||||
|
||||
export function DescriptionGenerationDrawer({
|
||||
open,
|
||||
databaseId = null,
|
||||
run: initialRun,
|
||||
modelLabel,
|
||||
onClose,
|
||||
@@ -100,11 +102,15 @@ export function DescriptionGenerationDrawer({
|
||||
? 3_000
|
||||
: false,
|
||||
});
|
||||
const visibleHistory = useMemo(
|
||||
() => (historyQuery.data ?? []).filter((item) => !databaseId || item.databaseId === databaseId),
|
||||
[databaseId, historyQuery.data],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open || initialRun || !historyQuery.data?.[0]) return;
|
||||
onRunUpdate(historyQuery.data[0]);
|
||||
}, [historyQuery.data, initialRun, onRunUpdate, open]);
|
||||
if (!open || initialRun || !visibleHistory[0]) return;
|
||||
onRunUpdate(visibleHistory[0]);
|
||||
}, [initialRun, onRunUpdate, open, visibleHistory]);
|
||||
|
||||
const mergeEvents = useCallback((incoming: DescriptionGenerationEvent[]) => {
|
||||
if (incoming.length === 0) return;
|
||||
@@ -406,7 +412,7 @@ export function DescriptionGenerationDrawer({
|
||||
<div className="mb-2 flex items-center justify-between gap-3">
|
||||
<h3 className="thot-label">Recent runs</h3>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">
|
||||
{historyQuery.data?.length ?? 0}
|
||||
{visibleHistory.length}
|
||||
</span>
|
||||
</div>
|
||||
{historyQuery.isError ? (
|
||||
@@ -415,17 +421,18 @@ export function DescriptionGenerationDrawer({
|
||||
</p>
|
||||
) : historyQuery.isLoading ? (
|
||||
<p className="text-sm text-muted-foreground">Loading run history…</p>
|
||||
) : (historyQuery.data ?? []).length === 0 ? (
|
||||
) : visibleHistory.length === 0 ? (
|
||||
<p className="text-sm text-muted-foreground">No description generation runs yet.</p>
|
||||
) : (
|
||||
<div className="max-h-56 divide-y divide-border overflow-y-auto rounded-md border border-border">
|
||||
{(historyQuery.data ?? []).map((item) => (
|
||||
{visibleHistory.map((item) => (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
aria-label={`${statusLabel(item)} · ${item.scope.replaceAll("_", " ")}`}
|
||||
aria-current={item.id === run.id ? "true" : undefined}
|
||||
className="flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25"
|
||||
data-status={item.status}
|
||||
className={`flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25${item.status === "completed" ? " bg-[oklch(var(--success)/0.1)]" : ""}`}
|
||||
onClick={() => onRunUpdate(item)}
|
||||
>
|
||||
<span className="font-medium">{statusLabel(item)}</span>
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
import { readFileSync, readdirSync } from "node:fs";
|
||||
|
||||
import { expect, test } from "vitest";
|
||||
|
||||
const sourceDirectory = "src/shell/database-management";
|
||||
const styles = readFileSync(`${sourceDirectory}/FleetLedgerShell.css`, "utf8");
|
||||
|
||||
test("uses operational body text for every available-command list", () => {
|
||||
const availableCommandLists = readdirSync(sourceDirectory)
|
||||
.filter((fileName) => fileName.endsWith(".tsx") && !fileName.includes(".test."))
|
||||
.flatMap((fileName) => {
|
||||
const source = readFileSync(`${sourceDirectory}/${fileName}`, "utf8");
|
||||
const matches = source.match(
|
||||
/<select\b[\s\S]*?<option value="">Choose an action…<\/option>[\s\S]*?<\/select>/g,
|
||||
);
|
||||
return (matches ?? []).map((markup) => ({ fileName, markup }));
|
||||
});
|
||||
|
||||
expect(availableCommandLists.map(({ fileName }) => fileName).sort()).toEqual([
|
||||
"DatabaseRelationships.tsx",
|
||||
"FleetActionSelector.tsx",
|
||||
]);
|
||||
for (const { markup } of availableCommandLists) {
|
||||
expect(markup).toContain('className="thot-fleet-action-selector__select"');
|
||||
}
|
||||
|
||||
expect(styles).toMatch(
|
||||
/\.thot-fleet-action-selector__select\s*\{[^}]*font-size:\s*0\.9375rem;/s,
|
||||
);
|
||||
});
|
||||
@@ -274,7 +274,7 @@
|
||||
background: oklch(var(--card));
|
||||
}
|
||||
|
||||
.thot-fleet-ledger__navigation > button:first-child {
|
||||
.thot-fleet-back-action {
|
||||
min-height: 2.25rem;
|
||||
border: 1px solid oklch(var(--primary) / 0.28);
|
||||
color: oklch(var(--primary));
|
||||
@@ -283,7 +283,7 @@
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.thot-fleet-ledger__navigation > button:first-child:not(:disabled):hover {
|
||||
.thot-fleet-back-action:not(:disabled):hover {
|
||||
border-color: oklch(var(--primary) / 0.48);
|
||||
color: oklch(var(--primary));
|
||||
background: oklch(var(--primary) / 0.14);
|
||||
@@ -410,7 +410,7 @@
|
||||
border-radius: var(--radius);
|
||||
color: oklch(var(--foreground));
|
||||
background: oklch(var(--background));
|
||||
font-size: 0.8125rem;
|
||||
font-size: 0.9375rem;
|
||||
outline: none;
|
||||
}
|
||||
|
||||
@@ -431,24 +431,6 @@
|
||||
line-height: 1.4;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model {
|
||||
min-width: min(100%, 14rem);
|
||||
flex: 0 1 16rem;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model > div {
|
||||
min-width: 0;
|
||||
gap: 0.25rem;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model label {
|
||||
gap: 0.25rem;
|
||||
}
|
||||
|
||||
.thot-fleet-metadata-model select {
|
||||
height: 2rem;
|
||||
}
|
||||
|
||||
.thot-fleet-ledger__selection-summary {
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
@@ -582,6 +564,17 @@
|
||||
box-shadow: inset 3px 0 0 oklch(var(--primary));
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__history button[data-status="completed"],
|
||||
.thot-catalog-drawer__history button[data-status="succeeded"] {
|
||||
background: oklch(var(--success) / 0.1);
|
||||
}
|
||||
|
||||
.thot-catalog-drawer__history button[aria-current="true"][data-status="completed"],
|
||||
.thot-catalog-drawer__history button[aria-current="true"][data-status="succeeded"] {
|
||||
background: oklch(var(--success) / 0.1);
|
||||
box-shadow: inset 0 0 0 1px oklch(var(--success) / 0.5);
|
||||
}
|
||||
|
||||
.thot-fleet-grid-surface {
|
||||
position: relative;
|
||||
}
|
||||
@@ -623,24 +616,29 @@
|
||||
.thot-fleet-icon-action::after {
|
||||
position: absolute;
|
||||
z-index: 80;
|
||||
inset-inline-end: calc(100% + 5px);
|
||||
inset-block-start: 50%;
|
||||
inset-inline-end: 0;
|
||||
inset-block-start: calc(100% + 3px);
|
||||
width: max-content;
|
||||
max-width: 12rem;
|
||||
padding: 0.3rem 0.45rem;
|
||||
border: 1px solid oklch(var(--border));
|
||||
border-radius: calc(var(--radius) - 0.125rem);
|
||||
color: oklch(var(--popover-foreground));
|
||||
background: oklch(var(--popover));
|
||||
box-shadow: var(--shadow-sm);
|
||||
padding: 5px 8px;
|
||||
border: 1px solid oklch(42% 0.018 258);
|
||||
border-radius: 6px;
|
||||
color: oklch(96% 0.006 258);
|
||||
background: oklch(24% 0.014 258);
|
||||
box-shadow: 0 8px 20px -10px oklch(12% 0.02 258 / 0.72);
|
||||
content: attr(data-tooltip);
|
||||
font-size: 0.6875rem;
|
||||
font-weight: 650;
|
||||
line-height: 1.2;
|
||||
font-weight: 600;
|
||||
line-height: 1.25;
|
||||
text-align: center;
|
||||
white-space: normal;
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
transform: translate(2px, -50%);
|
||||
transition: opacity 60ms cubic-bezier(0.22, 1, 0.36, 1), transform 60ms cubic-bezier(0.22, 1, 0.36, 1);
|
||||
transform: translateY(-2px);
|
||||
transition:
|
||||
opacity 90ms linear,
|
||||
transform 120ms cubic-bezier(0.25, 1, 0.5, 1),
|
||||
visibility 0ms linear 120ms;
|
||||
visibility: hidden;
|
||||
}
|
||||
|
||||
@@ -677,7 +675,8 @@
|
||||
.thot-fleet-icon-action:focus-visible::after,
|
||||
.thot-fleet-icon-action:focus-within::after {
|
||||
opacity: 1;
|
||||
transform: translate(0, -50%);
|
||||
transform: translateY(0);
|
||||
transition-delay: 60ms;
|
||||
visibility: visible;
|
||||
}
|
||||
|
||||
|
||||
@@ -233,7 +233,7 @@ export function FleetLedgerBreadcrumb({
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="thot-fleet-nav-action"
|
||||
className="thot-fleet-nav-action thot-fleet-back-action"
|
||||
data-tooltip={back.label.replace(/^Back to\s+/i, "")}
|
||||
onClick={back.onBack}
|
||||
disabled={back.disabled}
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { describe, expect, test } from "vitest";
|
||||
|
||||
const pageSource = readFileSync(resolve(__dirname, "../DatabaseManagementPage.tsx"), "utf8");
|
||||
const componentSource = (name: string) => readFileSync(resolve(__dirname, name), "utf8");
|
||||
|
||||
describe("history action placement", () => {
|
||||
test("workspace header does not own database history actions", () => {
|
||||
const fleetHeader = pageSource.slice(pageSource.indexOf('title="Database management"'), pageSource.indexOf("kpis={"));
|
||||
expect(fleetHeader).not.toContain("Description history");
|
||||
expect(fleetHeader).not.toContain("Sensitive history");
|
||||
});
|
||||
|
||||
test("database views expose metadata histories beside database commands", () => {
|
||||
for (const file of ["DatabaseForm.tsx", "DatabaseTables.tsx", "DatabaseRelationships.tsx"]) {
|
||||
const source = componentSource(file);
|
||||
expect(source, file).toContain("Description history");
|
||||
expect(source, file).toContain("Sensitive history");
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,9 @@
|
||||
import { useId } from "react";
|
||||
import type { MetadataGenerationModels } from "../../api/catalog-databases";
|
||||
|
||||
export const NO_METADATA_GENERATION_LLM_MODEL_MESSAGE =
|
||||
"No metadata-generation LLM model is configured for this installation.";
|
||||
|
||||
interface Props {
|
||||
canManage: boolean;
|
||||
data?: MetadataGenerationModels;
|
||||
@@ -7,6 +11,7 @@ interface Props {
|
||||
isLoading: boolean;
|
||||
selectedModel: string;
|
||||
onSelectedModelChange: (modelId: string) => void;
|
||||
variant?: "default" | "compact";
|
||||
}
|
||||
|
||||
export function MetadataGenerationModelSelector({
|
||||
@@ -16,21 +21,35 @@ export function MetadataGenerationModelSelector({
|
||||
isLoading,
|
||||
selectedModel,
|
||||
onSelectedModelChange,
|
||||
variant = "default",
|
||||
}: Props) {
|
||||
const helpId = useId();
|
||||
const models = data?.models ?? [];
|
||||
const configuredDefault = data?.default ?? "";
|
||||
const hasUsableDefault = configuredDefault !== ""
|
||||
&& models.some((model) => model.id === configuredDefault);
|
||||
const noUsableModel = !isLoading && !isError && Boolean(data) && !hasUsableDefault;
|
||||
const unavailable = noUsableModel || (!isLoading && isError);
|
||||
const unavailableMessage = !isLoading && isError
|
||||
? "Metadata-generation LLM model choices are unavailable."
|
||||
: noUsableModel
|
||||
? NO_METADATA_GENERATION_LLM_MODEL_MESSAGE
|
||||
: null;
|
||||
const compact = variant === "compact";
|
||||
|
||||
return (
|
||||
<div className="grid min-w-56 max-w-lg gap-1.5">
|
||||
<label className="grid gap-1.5 text-xs font-medium text-foreground">
|
||||
<span>Metadata description model</span>
|
||||
<div className={compact
|
||||
? "flex min-w-0 flex-wrap items-center gap-x-2 gap-y-1"
|
||||
: "grid min-w-56 max-w-lg gap-1.5"}
|
||||
>
|
||||
<label className={compact
|
||||
? "flex min-w-0 items-center gap-2 text-xs font-medium text-foreground"
|
||||
: "grid gap-1.5 text-xs font-medium text-foreground"}
|
||||
>
|
||||
<span className={compact ? "whitespace-nowrap" : undefined}>Metadata-generation LLM model</span>
|
||||
<select
|
||||
className="h-9 w-full rounded-md border border-input bg-card px-3 text-sm outline-none transition focus:border-primary/60 focus:ring-3 focus:ring-ring/15 disabled:bg-muted disabled:text-muted-foreground"
|
||||
aria-label="Metadata description model"
|
||||
aria-describedby={unavailable ? "metadata-generation-model-help" : undefined}
|
||||
className={`${compact ? "h-8 min-w-52" : "h-9 w-full"} rounded-md border border-input bg-card px-3 text-sm outline-none transition focus:border-primary/60 focus:ring-3 focus:ring-ring/15 disabled:bg-muted disabled:text-muted-foreground`}
|
||||
aria-label="Metadata-generation LLM model"
|
||||
aria-describedby={unavailableMessage ? helpId : undefined}
|
||||
value={selectedModel}
|
||||
disabled={!canManage || isLoading || isError || !hasUsableDefault}
|
||||
onChange={(event) => onSelectedModelChange(event.target.value)}
|
||||
@@ -45,9 +64,9 @@ export function MetadataGenerationModelSelector({
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
{unavailable ? (
|
||||
<p id="metadata-generation-model-help" className="max-w-72 text-xs font-normal leading-4 text-muted-foreground">
|
||||
Configure a metadata-generation model in application setup to enable description generation.
|
||||
{unavailableMessage ? (
|
||||
<p id={helpId} className="max-w-72 text-xs font-normal leading-4 text-muted-foreground">
|
||||
{unavailableMessage}
|
||||
</p>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { describe, expect, test } from "vitest";
|
||||
|
||||
const sourceRoot = resolve(__dirname);
|
||||
|
||||
describe("Recent runs success styling contract", () => {
|
||||
test("every history renderer exposes its run status to the shared styles", () => {
|
||||
const renderers = [
|
||||
"DescriptionGenerationDrawer.tsx",
|
||||
"SensitiveDataSuggestionHistoryDrawer.tsx",
|
||||
"CatalogSyncDrawer.tsx",
|
||||
];
|
||||
|
||||
for (const renderer of renderers) {
|
||||
const source = readFileSync(resolve(sourceRoot, renderer), "utf8");
|
||||
expect(source, renderer).toMatch(/data-status=\{item\.(?:status|state)\}/);
|
||||
}
|
||||
});
|
||||
|
||||
test("shared history styles cover both successful status vocabularies", () => {
|
||||
const styles = readFileSync(resolve(sourceRoot, "FleetLedgerShell.css"), "utf8");
|
||||
expect(styles).toContain('button[data-status="completed"]');
|
||||
expect(styles).toContain('button[data-status="succeeded"]');
|
||||
expect(styles).toMatch(/oklch\(var\(--success\)/);
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect } from "react";
|
||||
import { useEffect, useMemo } from "react";
|
||||
import { useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { apiErrorMessage } from "../../api/client";
|
||||
import {
|
||||
@@ -11,6 +11,7 @@ import { FleetLedgerDrawer } from "./FleetLedgerShell";
|
||||
|
||||
interface Props {
|
||||
open: boolean;
|
||||
databaseId?: string | null;
|
||||
run: SensitiveDataSuggestionRun | null;
|
||||
modelLabel: string;
|
||||
onClose: () => void;
|
||||
@@ -46,6 +47,7 @@ function timestamp(value: string | null): string {
|
||||
|
||||
export function SensitiveDataSuggestionHistoryDrawer({
|
||||
open,
|
||||
databaseId = null,
|
||||
run: initialRun,
|
||||
modelLabel,
|
||||
onClose,
|
||||
@@ -78,11 +80,15 @@ export function SensitiveDataSuggestionHistoryDrawer({
|
||||
? 3_000
|
||||
: false,
|
||||
});
|
||||
const visibleHistory = useMemo(
|
||||
() => (historyQuery.data ?? []).filter((item) => !databaseId || item.databaseId === databaseId),
|
||||
[databaseId, historyQuery.data],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open || initialRun || !historyQuery.data?.[0]) return;
|
||||
onRunUpdate(historyQuery.data[0]);
|
||||
}, [historyQuery.data, initialRun, onRunUpdate, open]);
|
||||
if (!open || initialRun || !visibleHistory[0]) return;
|
||||
onRunUpdate(visibleHistory[0]);
|
||||
}, [initialRun, onRunUpdate, open, visibleHistory]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!run) return;
|
||||
@@ -199,23 +205,24 @@ export function SensitiveDataSuggestionHistoryDrawer({
|
||||
<div className="mb-2 flex items-center justify-between gap-3">
|
||||
<h3 className="thot-label">Recent runs</h3>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">
|
||||
{historyQuery.data?.length ?? 0}
|
||||
{visibleHistory.length}
|
||||
</span>
|
||||
</div>
|
||||
{historyQuery.isError ? (
|
||||
<p role="alert" className="text-sm text-destructive">{apiErrorMessage(historyQuery.error)}</p>
|
||||
) : historyQuery.isLoading ? (
|
||||
<p className="text-sm text-muted-foreground">Loading suggestion history…</p>
|
||||
) : (historyQuery.data ?? []).length === 0 ? (
|
||||
) : visibleHistory.length === 0 ? (
|
||||
<p className="text-sm text-muted-foreground">No sensitive suggestion runs yet.</p>
|
||||
) : (
|
||||
<div className="max-h-56 divide-y divide-border overflow-y-auto rounded-md border border-border">
|
||||
{(historyQuery.data ?? []).map((item) => (
|
||||
{visibleHistory.map((item) => (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
aria-label={`${statusLabel(item)} · ${item.scope.replaceAll("_", " ")}`}
|
||||
aria-current={item.id === run.id ? "true" : undefined}
|
||||
data-status={item.status}
|
||||
className="flex w-full items-start justify-between gap-3 px-3 py-2 text-left text-sm hover:bg-muted/50 focus-visible:outline-none focus-visible:ring-3 focus-visible:ring-ring/25"
|
||||
onClick={() => onRunUpdate(item)}
|
||||
>
|
||||
|
||||
@@ -4,7 +4,7 @@ import type {
|
||||
DatabaseConfiguration,
|
||||
} from "../../api/catalog-databases";
|
||||
|
||||
export type DatabaseFormMode = "add" | "view" | "edit" | "delete";
|
||||
export type DatabaseFormMode = "configure" | "view" | "edit" | "delete";
|
||||
|
||||
export type DatabaseScreen =
|
||||
| { kind: "list" }
|
||||
@@ -13,7 +13,6 @@ export type DatabaseScreen =
|
||||
| {
|
||||
kind: DatabaseFormMode;
|
||||
workspaceId: string;
|
||||
workspaceLocked?: boolean;
|
||||
};
|
||||
|
||||
export type DatabaseBusyAction = "save" | "test" | "delete" | "retry-secrets" | "reload" | null;
|
||||
|
||||
@@ -0,0 +1,403 @@
|
||||
import json
|
||||
from datetime import UTC, datetime
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from tht.cli import app
|
||||
from tht.config import load_config
|
||||
from tht.mschema.context import SchemaContextError, load_schema_context
|
||||
from tht.mschema.models import (
|
||||
Annotations,
|
||||
ColumnPhysical,
|
||||
ForeignKey,
|
||||
PhysicalSchema,
|
||||
TableAnnotation,
|
||||
TablePhysical,
|
||||
)
|
||||
from tht.mschema.render import to_mschema_text
|
||||
|
||||
|
||||
REVISION = "a" * 40
|
||||
RUNNER = CliRunner()
|
||||
|
||||
|
||||
def _workspace(tmp_path, snapshot: dict) -> object:
|
||||
artifacts = tmp_path / "artifacts"
|
||||
relationships = tmp_path / "effective-relationships.json"
|
||||
relationships.write_text(json.dumps(snapshot))
|
||||
physical = PhysicalSchema(
|
||||
database="warehouse",
|
||||
schema="analytics",
|
||||
introspected_at=datetime(2026, 1, 1, tzinfo=UTC),
|
||||
tables={
|
||||
"users": TablePhysical(
|
||||
columns={"id": ColumnPhysical(type="bigint", pk=True)},
|
||||
),
|
||||
"orders": TablePhysical(
|
||||
columns={
|
||||
"physical_user_id": ColumnPhysical(type="bigint"),
|
||||
"annotated_user_id": ColumnPhysical(type="bigint"),
|
||||
"generated_user_id": ColumnPhysical(type="bigint"),
|
||||
},
|
||||
foreign_keys=[
|
||||
ForeignKey(
|
||||
columns=["physical_user_id"],
|
||||
ref_table="users",
|
||||
ref_columns=["id"],
|
||||
)
|
||||
],
|
||||
),
|
||||
},
|
||||
)
|
||||
physical.to_yaml(artifacts / "mschema" / "physical.yaml")
|
||||
Annotations(
|
||||
tables={
|
||||
"orders": TableAnnotation(
|
||||
description="Customer orders",
|
||||
foreign_keys=[
|
||||
ForeignKey(
|
||||
columns=["annotated_user_id"],
|
||||
ref_table="users",
|
||||
ref_columns=["id"],
|
||||
)
|
||||
],
|
||||
)
|
||||
}
|
||||
).to_yaml(artifacts / "mschema" / "annotations.yaml")
|
||||
config = tmp_path / "runtime.yaml"
|
||||
config.write_text(
|
||||
f"""
|
||||
runtime_identity:
|
||||
workspace_id: demo
|
||||
workspace_revision: {REVISION}
|
||||
source_identity: workspace://demo
|
||||
database: {{database: warehouse, schema: analytics, user: reader, password: secret, transport: direct}}
|
||||
vector_db: {{database: vectors, schema: public, user: reader, password: secret}}
|
||||
embeddings: {{base_url: 'http://localhost:11434', model: 'qwen3-embedding:0.6b', dim: 1024}}
|
||||
paths:
|
||||
artifacts: {artifacts}
|
||||
indexes: {tmp_path / 'indexes'}
|
||||
sessions: {tmp_path / 'sessions'}
|
||||
effective_relationships: {relationships}
|
||||
"""
|
||||
)
|
||||
return load_config(config)
|
||||
|
||||
|
||||
def test_effective_snapshot_is_the_exclusive_relationship_source(tmp_path):
|
||||
cfg = _workspace(
|
||||
tmp_path,
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"workspaceId": "demo",
|
||||
"relationships": [
|
||||
{
|
||||
"sourceTable": "orders",
|
||||
"sourceColumns": ["generated_user_id"],
|
||||
"targetTable": "users",
|
||||
"targetColumns": ["id"],
|
||||
"origin": "generated",
|
||||
}
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
context = load_schema_context(cfg)
|
||||
rendered = to_mschema_text(
|
||||
context.physical,
|
||||
context.annotations,
|
||||
effective_relationships=context.effective_relationships,
|
||||
)
|
||||
|
||||
assert "-- Customer orders" in rendered
|
||||
assert "orders.generated_user_id=users.id" in rendered
|
||||
assert "orders.physical_user_id=users.id" not in rendered
|
||||
assert "orders.annotated_user_id=users.id" not in rendered
|
||||
|
||||
|
||||
def test_declared_effective_snapshot_must_exist(tmp_path):
|
||||
cfg = _workspace(
|
||||
tmp_path,
|
||||
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
|
||||
)
|
||||
cfg.paths.effective_relationships.unlink()
|
||||
|
||||
with pytest.raises(SchemaContextError, match="snapshot is missing"):
|
||||
load_schema_context(cfg)
|
||||
|
||||
|
||||
def test_data_root_override_preserves_the_runtime_relationship_snapshot_path(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
monkeypatch.setenv("THT_DATA_ROOT", str(tmp_path / "mounted-data"))
|
||||
monkeypatch.delenv("THT_HOME", raising=False)
|
||||
|
||||
cfg = _workspace(
|
||||
tmp_path,
|
||||
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
|
||||
)
|
||||
|
||||
assert cfg.paths.effective_relationships == tmp_path / "effective-relationships.json"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("snapshot", "message"),
|
||||
[
|
||||
(
|
||||
{"schemaVersion": 1, "workspaceId": "another", "relationships": []},
|
||||
"workspace does not match",
|
||||
),
|
||||
(
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"workspaceId": "demo",
|
||||
"relationships": [
|
||||
{
|
||||
"sourceTable": "missing_orders",
|
||||
"sourceColumns": ["user_id"],
|
||||
"targetTable": "users",
|
||||
"targetColumns": ["id"],
|
||||
"origin": "generated",
|
||||
}
|
||||
],
|
||||
},
|
||||
"endpoint table is absent",
|
||||
),
|
||||
(
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"workspaceId": "demo",
|
||||
"relationships": [
|
||||
{
|
||||
"sourceTable": "orders",
|
||||
"sourceColumns": ["missing_user_id"],
|
||||
"targetTable": "users",
|
||||
"targetColumns": ["id"],
|
||||
"origin": "manual",
|
||||
}
|
||||
],
|
||||
},
|
||||
"endpoint column is absent",
|
||||
),
|
||||
(
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"workspaceId": "demo",
|
||||
"relationships": [
|
||||
{
|
||||
"sourceTable": "orders",
|
||||
"sourceColumns": ["generated_user_id", "annotated_user_id"],
|
||||
"targetTable": "users",
|
||||
"targetColumns": ["id"],
|
||||
"origin": "physical",
|
||||
}
|
||||
],
|
||||
},
|
||||
"snapshot is invalid",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_effective_snapshot_rejects_wrong_workspace_and_orphan_endpoints(
|
||||
tmp_path, snapshot, message
|
||||
):
|
||||
cfg = _workspace(tmp_path, snapshot)
|
||||
|
||||
with pytest.raises(SchemaContextError, match=message):
|
||||
load_schema_context(cfg)
|
||||
|
||||
|
||||
def test_legacy_relationship_merge_remains_available_and_deduplicated(tmp_path):
|
||||
cfg = _workspace(
|
||||
tmp_path,
|
||||
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
|
||||
)
|
||||
cfg.paths.effective_relationships = None
|
||||
annotations = Annotations.from_yaml(
|
||||
cfg.paths.artifacts / "mschema" / "annotations.yaml"
|
||||
)
|
||||
annotations.tables["orders"].foreign_keys.append(
|
||||
ForeignKey(
|
||||
columns=["annotated_user_id"],
|
||||
ref_table="users",
|
||||
ref_columns=["id"],
|
||||
)
|
||||
)
|
||||
annotations.to_yaml(cfg.paths.artifacts / "mschema" / "annotations.yaml")
|
||||
|
||||
context = load_schema_context(cfg)
|
||||
rendered = to_mschema_text(context.physical, context.annotations)
|
||||
|
||||
assert context.effective_relationships is None
|
||||
assert rendered.count("orders.physical_user_id=users.id") == 1
|
||||
assert rendered.count("orders.annotated_user_id=users.id") == 1
|
||||
|
||||
|
||||
def test_schema_render_uses_the_effective_snapshot(tmp_path):
|
||||
_workspace(
|
||||
tmp_path,
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"workspaceId": "demo",
|
||||
"relationships": [
|
||||
{
|
||||
"sourceTable": "orders",
|
||||
"sourceColumns": ["generated_user_id"],
|
||||
"targetTable": "users",
|
||||
"targetColumns": ["id"],
|
||||
"origin": "manual",
|
||||
}
|
||||
],
|
||||
},
|
||||
)
|
||||
|
||||
result = RUNNER.invoke(
|
||||
app,
|
||||
["schema", "render", "--format", "mschema-text", "-c", str(tmp_path / "runtime.yaml")],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "orders.generated_user_id=users.id" in result.stdout
|
||||
assert "orders.physical_user_id=users.id" not in result.stdout
|
||||
assert "orders.annotated_user_id=users.id" not in result.stdout
|
||||
|
||||
|
||||
def test_schema_search_uses_the_effective_snapshot(tmp_path, monkeypatch):
|
||||
cfg = _workspace(
|
||||
tmp_path,
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"workspaceId": "demo",
|
||||
"relationships": [
|
||||
{
|
||||
"sourceTable": "orders",
|
||||
"sourceColumns": ["generated_user_id"],
|
||||
"targetTable": "users",
|
||||
"targetColumns": ["id"],
|
||||
"origin": "generated",
|
||||
}
|
||||
],
|
||||
},
|
||||
)
|
||||
physical = cfg.paths.artifacts / "mschema" / "physical.yaml"
|
||||
|
||||
class Searcher:
|
||||
def search(self, _vector, **_kwargs):
|
||||
return [
|
||||
SimpleNamespace(
|
||||
id="orders",
|
||||
kind="schema_table",
|
||||
ref="orders",
|
||||
title="orders",
|
||||
similarity=1.0,
|
||||
content="orders",
|
||||
metadata={},
|
||||
)
|
||||
]
|
||||
|
||||
class Embedder:
|
||||
def embed_query(self, _text):
|
||||
return [1.0, 0.0, 0.0]
|
||||
|
||||
import tht.cli.search_cmd as search_module
|
||||
import tht.cli.vector_cmd as vector_module
|
||||
import tht.evidence as evidence_module
|
||||
|
||||
monkeypatch.setattr(
|
||||
search_module,
|
||||
"_leased_dwh_snapshot",
|
||||
lambda _cfg, _ctx: SimpleNamespace(physical=physical, lsh_dir=tmp_path / "no-lsh"),
|
||||
)
|
||||
monkeypatch.setattr(vector_module, "require_vector_cfg", lambda _cfg: None)
|
||||
monkeypatch.setattr(vector_module, "open_searcher", lambda _cfg: Searcher())
|
||||
monkeypatch.setattr(vector_module, "make_embedder", lambda _cfg: Embedder())
|
||||
monkeypatch.setattr(evidence_module, "validate_corpus_workspace", lambda *_args: None)
|
||||
monkeypatch.setattr(evidence_module, "active_searcher", lambda _cfg, searcher, **_kwargs: searcher)
|
||||
|
||||
result = RUNNER.invoke(
|
||||
app,
|
||||
[
|
||||
"search",
|
||||
"find",
|
||||
"orders",
|
||||
"--kind",
|
||||
"schema",
|
||||
"--json",
|
||||
"-c",
|
||||
str(tmp_path / "runtime.yaml"),
|
||||
],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
payload = json.loads(result.stdout)
|
||||
assert "orders.generated_user_id=users.id" in payload["mschema"]
|
||||
assert "orders.physical_user_id=users.id" not in payload["mschema"]
|
||||
assert "orders.annotated_user_id=users.id" not in payload["mschema"]
|
||||
|
||||
|
||||
def test_schema_search_fails_closed_before_returning_no_candidates(tmp_path, monkeypatch):
|
||||
cfg = _workspace(
|
||||
tmp_path,
|
||||
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
|
||||
)
|
||||
cfg.paths.effective_relationships.unlink()
|
||||
physical = cfg.paths.artifacts / "mschema" / "physical.yaml"
|
||||
|
||||
import tht.cli.search_cmd as search_module
|
||||
import tht.cli.vector_cmd as vector_module
|
||||
import tht.evidence as evidence_module
|
||||
import tht.search as search_core
|
||||
|
||||
monkeypatch.setattr(
|
||||
search_module,
|
||||
"_leased_dwh_snapshot",
|
||||
lambda _cfg, _ctx: SimpleNamespace(physical=physical, lsh_dir=tmp_path / "no-lsh"),
|
||||
)
|
||||
monkeypatch.setattr(vector_module, "require_vector_cfg", lambda _cfg: None)
|
||||
monkeypatch.setattr(vector_module, "open_searcher", lambda _cfg: object())
|
||||
monkeypatch.setattr(vector_module, "make_embedder", lambda _cfg: object())
|
||||
monkeypatch.setattr(evidence_module, "validate_corpus_workspace", lambda *_args: None)
|
||||
monkeypatch.setattr(evidence_module, "active_searcher", lambda _cfg, searcher, **_kwargs: searcher)
|
||||
monkeypatch.setattr(search_core, "combined_search", lambda **_kwargs: [])
|
||||
|
||||
result = RUNNER.invoke(
|
||||
app,
|
||||
[
|
||||
"search",
|
||||
"find",
|
||||
"orders",
|
||||
"--kind",
|
||||
"schema",
|
||||
"--json",
|
||||
"-c",
|
||||
str(tmp_path / "runtime.yaml"),
|
||||
],
|
||||
)
|
||||
|
||||
assert result.exit_code == 1
|
||||
assert "effective relationship snapshot is missing" in result.stderr
|
||||
|
||||
|
||||
def test_suggest_fks_cannot_write_when_relationships_are_catalog_managed(tmp_path):
|
||||
_workspace(
|
||||
tmp_path,
|
||||
{"schemaVersion": 1, "workspaceId": "demo", "relationships": []},
|
||||
)
|
||||
annotations = tmp_path / "artifacts" / "mschema" / "annotations.yaml"
|
||||
before = annotations.read_text()
|
||||
|
||||
result = RUNNER.invoke(
|
||||
app,
|
||||
[
|
||||
"schema",
|
||||
"suggest-fks",
|
||||
"--write",
|
||||
"-c",
|
||||
str(tmp_path / "runtime.yaml"),
|
||||
],
|
||||
)
|
||||
|
||||
assert result.exit_code == 1
|
||||
assert "managed by the catalog" in result.stderr
|
||||
assert annotations.read_text() == before
|
||||
@@ -10,6 +10,12 @@ from tht.adapters.factory import build_dwh
|
||||
from tht.cli.config_cmd import CONFIG_OPT
|
||||
from tht.config import ConfigError, load_config
|
||||
from tht.db.sampling import is_text_type
|
||||
from tht.mschema.context import (
|
||||
SchemaContextError,
|
||||
annotations_path,
|
||||
load_schema_context,
|
||||
physical_path,
|
||||
)
|
||||
from tht.mschema.eligibility import classify_all
|
||||
|
||||
schema_app = typer.Typer(help="Gestione mschema (rappresentazione canonica dello schema)")
|
||||
@@ -40,20 +46,6 @@ def _load_config_or_exit(config: Path):
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
|
||||
def physical_path(cfg) -> Path:
|
||||
from tht.jobs.dwh_pipeline import resolve_dwh_snapshot
|
||||
|
||||
if not (cfg.paths.artifacts.parent / ".tht-dwh").exists():
|
||||
return cfg.paths.artifacts / "mschema" / "physical.yaml"
|
||||
return resolve_dwh_snapshot(cfg).physical
|
||||
|
||||
|
||||
def annotations_path(cfg) -> Path:
|
||||
if cfg.paths.annotations_root is not None:
|
||||
return cfg.paths.annotations_root / "mschema" / "annotations.yaml"
|
||||
return cfg.paths.artifacts / "mschema" / "annotations.yaml"
|
||||
|
||||
|
||||
def refresh_catalog(cfg, *, dwh=None, output_path: Path | None = None):
|
||||
"""Run the existing catalog algorithm and persist its canonical output."""
|
||||
target = dwh if dwh is not None else build_dwh(cfg)
|
||||
@@ -432,6 +424,21 @@ def suggest_fks_cmd(
|
||||
from tht.mschema.models import Annotations, PhysicalSchema, TableAnnotation
|
||||
|
||||
cfg = _load_config_or_exit(config)
|
||||
if write and cfg.paths.effective_relationships is not None:
|
||||
error = "effective relationships are managed by the catalog; --write is disabled"
|
||||
if json_output:
|
||||
_emit_json({
|
||||
"code": "catalog_relationships_managed",
|
||||
"error": error,
|
||||
"operation": "schema_suggest_fks",
|
||||
"schemaVersion": 1,
|
||||
"status": "failed",
|
||||
"workspaceId": cfg._workspace_id,
|
||||
"workspaceRevision": cfg._workspace_revision,
|
||||
})
|
||||
else:
|
||||
typer.secho(f"ERRORE: {error}", fg=typer.colors.RED, err=True)
|
||||
raise typer.Exit(code=1)
|
||||
phys_file = physical_path(cfg)
|
||||
if not phys_file.exists():
|
||||
if json_output:
|
||||
@@ -556,7 +563,6 @@ def render_cmd(
|
||||
"""Serializza mschema (physical + annotations) nel formato richiesto."""
|
||||
import json
|
||||
|
||||
from tht.mschema.models import Annotations, PhysicalSchema
|
||||
from tht.mschema.render import to_markdown, to_mschema_text, to_schema_dict
|
||||
|
||||
cfg = _load_config_or_exit(config)
|
||||
@@ -564,19 +570,40 @@ def render_cmd(
|
||||
if not phys_file.exists():
|
||||
typer.secho(
|
||||
f"ERRORE: {phys_file} non trovato. Esegui prima `tht schema introspect`.",
|
||||
fg=typer.colors.RED, err=True,
|
||||
fg=typer.colors.RED,
|
||||
err=True,
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
physical = PhysicalSchema.from_yaml(phys_file)
|
||||
annotations = Annotations.from_yaml(annotations_path(cfg))
|
||||
try:
|
||||
context = load_schema_context(cfg, physical_file=phys_file)
|
||||
except SchemaContextError as exc:
|
||||
typer.secho(f"ERRORE: {exc}", fg=typer.colors.RED, err=True)
|
||||
raise typer.Exit(code=1) from None
|
||||
table_filter = list(tables) if tables else None
|
||||
|
||||
if format == "markdown":
|
||||
out = to_markdown(physical, annotations)
|
||||
out = to_markdown(
|
||||
context.physical,
|
||||
context.annotations,
|
||||
effective_relationships=context.effective_relationships,
|
||||
)
|
||||
elif format == "mschema-text":
|
||||
out = to_mschema_text(physical, annotations, tables=table_filter)
|
||||
out = to_mschema_text(
|
||||
context.physical,
|
||||
context.annotations,
|
||||
tables=table_filter,
|
||||
effective_relationships=context.effective_relationships,
|
||||
)
|
||||
elif format == "schema-dict":
|
||||
out = json.dumps(to_schema_dict(physical, annotations), ensure_ascii=False, indent=2)
|
||||
out = json.dumps(
|
||||
to_schema_dict(
|
||||
context.physical,
|
||||
context.annotations,
|
||||
effective_relationships=context.effective_relationships,
|
||||
),
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
)
|
||||
else:
|
||||
typer.secho(f"ERRORE: formato sconosciuto: {format}", fg=typer.colors.RED, err=True)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
@@ -231,8 +231,7 @@ def search_cmd(
|
||||
)
|
||||
|
||||
if kind == "schema":
|
||||
from tht.cli.schema_cmd import annotations_path
|
||||
from tht.mschema.models import Annotations, PhysicalSchema
|
||||
from tht.mschema.context import SchemaContextError, load_schema_context
|
||||
from tht.mschema.render import to_mschema_text
|
||||
from tht.search import schema_tables
|
||||
|
||||
@@ -243,6 +242,11 @@ def search_cmd(
|
||||
fg=typer.colors.RED, err=True,
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
try:
|
||||
schema_context = load_schema_context(cfg, physical_file=phys_file)
|
||||
except SchemaContextError as exc:
|
||||
typer.secho(f"ERRORE: {exc}", fg=typer.colors.RED, err=True)
|
||||
raise typer.Exit(code=1) from None
|
||||
|
||||
candidates = combined_search(
|
||||
keyword=keyword, lsh_hits=lsh_hits,
|
||||
@@ -258,10 +262,13 @@ def search_cmd(
|
||||
typer.secho(f"Nessuna tabella candidata per '{keyword}'.", fg=typer.colors.YELLOW)
|
||||
return
|
||||
|
||||
physical = PhysicalSchema.from_yaml(phys_file)
|
||||
annotations = Annotations.from_yaml(annotations_path(cfg))
|
||||
selected = [t for t, _ in ranked]
|
||||
mschema = to_mschema_text(physical, annotations, tables=selected)
|
||||
mschema = to_mschema_text(
|
||||
schema_context.physical,
|
||||
schema_context.annotations,
|
||||
tables=selected,
|
||||
effective_relationships=schema_context.effective_relationships,
|
||||
)
|
||||
|
||||
if json_out:
|
||||
typer.echo(json.dumps(
|
||||
|
||||
@@ -327,6 +327,9 @@ class PathsConfig(BaseModel):
|
||||
# Revision-qualified curated FK annotations root (P5). When absent, legacy
|
||||
# `artifacts/mschema/annotations.yaml` remains the annotations source.
|
||||
annotations_root: Path | None = None
|
||||
# Runtime-only, backend-derived effective relationship snapshot. When present,
|
||||
# it is the exclusive FK source; it is not an authored workspace artifact.
|
||||
effective_relationships: Path | None = None
|
||||
|
||||
|
||||
class RuntimeIdentityConfig(BaseModel):
|
||||
@@ -676,6 +679,7 @@ def load_config(path: Path) -> Config:
|
||||
indexes=resolved.indexes,
|
||||
memory=cfg.paths.memory,
|
||||
annotations_root=cfg.paths.annotations_root,
|
||||
effective_relationships=cfg.paths.effective_relationships,
|
||||
)
|
||||
}
|
||||
)
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from pydantic import BaseModel, Field, ValidationError, model_validator
|
||||
|
||||
from tht.mschema.models import Annotations, ForeignKey, PhysicalSchema
|
||||
|
||||
|
||||
class SchemaContextError(ValueError):
|
||||
"""The effective schema inputs cannot be used safely."""
|
||||
|
||||
|
||||
RelationshipName = Annotated[str, Field(min_length=1)]
|
||||
|
||||
|
||||
class EffectiveRelationship(BaseModel):
|
||||
source_table: RelationshipName = Field(alias="sourceTable")
|
||||
source_columns: list[RelationshipName] = Field(alias="sourceColumns", min_length=1)
|
||||
target_table: RelationshipName = Field(alias="targetTable")
|
||||
target_columns: list[RelationshipName] = Field(alias="targetColumns", min_length=1)
|
||||
origin: Literal["physical", "generated", "manual"]
|
||||
|
||||
model_config = {"extra": "forbid", "populate_by_name": True}
|
||||
|
||||
@model_validator(mode="after")
|
||||
def columns_are_paired(self):
|
||||
if len(self.source_columns) != len(self.target_columns):
|
||||
raise ValueError("sourceColumns and targetColumns must have the same length")
|
||||
return self
|
||||
|
||||
|
||||
class EffectiveRelationshipSnapshot(BaseModel):
|
||||
schema_version: Literal[1] = Field(alias="schemaVersion")
|
||||
workspace_id: RelationshipName = Field(alias="workspaceId")
|
||||
relationships: list[EffectiveRelationship]
|
||||
|
||||
model_config = {"extra": "forbid", "populate_by_name": True}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SchemaContext:
|
||||
physical: PhysicalSchema
|
||||
annotations: Annotations
|
||||
# None means legacy physical + annotation FK merging. A dict, including an
|
||||
# empty one, means the catalog snapshot is the exclusive relationship source.
|
||||
effective_relationships: dict[str, list[ForeignKey]] | None
|
||||
|
||||
|
||||
def physical_path(cfg) -> Path:
|
||||
from tht.jobs.dwh_pipeline import resolve_dwh_snapshot
|
||||
|
||||
if not (cfg.paths.artifacts.parent / ".tht-dwh").exists():
|
||||
return cfg.paths.artifacts / "mschema" / "physical.yaml"
|
||||
return resolve_dwh_snapshot(cfg).physical
|
||||
|
||||
|
||||
def annotations_path(cfg) -> Path:
|
||||
if cfg.paths.annotations_root is not None:
|
||||
return cfg.paths.annotations_root / "mschema" / "annotations.yaml"
|
||||
return cfg.paths.artifacts / "mschema" / "annotations.yaml"
|
||||
|
||||
|
||||
def _load_effective_relationships(cfg, physical: PhysicalSchema) -> dict[str, list[ForeignKey]] | None:
|
||||
path = cfg.paths.effective_relationships
|
||||
if path is None:
|
||||
return None
|
||||
if not path.is_file():
|
||||
raise SchemaContextError(f"effective relationship snapshot is missing: {path}")
|
||||
try:
|
||||
raw = json.loads(path.read_text())
|
||||
snapshot = EffectiveRelationshipSnapshot.model_validate(raw)
|
||||
except (OSError, json.JSONDecodeError, ValidationError) as exc:
|
||||
raise SchemaContextError(f"effective relationship snapshot is invalid: {path}") from exc
|
||||
if snapshot.workspace_id != cfg._workspace_id:
|
||||
raise SchemaContextError(
|
||||
"effective relationship snapshot workspace does not match runtime workspace"
|
||||
)
|
||||
|
||||
by_table: dict[str, list[ForeignKey]] = {}
|
||||
seen: set[tuple[str, tuple[str, ...], str, tuple[str, ...]]] = set()
|
||||
for relationship in snapshot.relationships:
|
||||
source = physical.tables.get(relationship.source_table)
|
||||
target = physical.tables.get(relationship.target_table)
|
||||
if source is None or target is None:
|
||||
raise SchemaContextError(
|
||||
"effective relationship endpoint table is absent from physical schema: "
|
||||
f"{relationship.source_table}->{relationship.target_table}"
|
||||
)
|
||||
missing_source = [name for name in relationship.source_columns if name not in source.columns]
|
||||
missing_target = [name for name in relationship.target_columns if name not in target.columns]
|
||||
if missing_source or missing_target:
|
||||
missing = ", ".join(
|
||||
[f"{relationship.source_table}.{name}" for name in missing_source]
|
||||
+ [f"{relationship.target_table}.{name}" for name in missing_target]
|
||||
)
|
||||
raise SchemaContextError(
|
||||
f"effective relationship endpoint column is absent from physical schema: {missing}"
|
||||
)
|
||||
key = (
|
||||
relationship.source_table,
|
||||
tuple(relationship.source_columns),
|
||||
relationship.target_table,
|
||||
tuple(relationship.target_columns),
|
||||
)
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
by_table.setdefault(relationship.source_table, []).append(
|
||||
ForeignKey(
|
||||
columns=relationship.source_columns,
|
||||
ref_table=relationship.target_table,
|
||||
ref_columns=relationship.target_columns,
|
||||
)
|
||||
)
|
||||
return by_table
|
||||
|
||||
|
||||
def load_schema_context(cfg, *, physical_file: Path | None = None) -> SchemaContext:
|
||||
physical_file = physical_file or physical_path(cfg)
|
||||
if not physical_file.is_file():
|
||||
raise SchemaContextError(f"physical schema is missing: {physical_file}")
|
||||
try:
|
||||
physical = PhysicalSchema.from_yaml(physical_file)
|
||||
annotations = Annotations.from_yaml(annotations_path(cfg))
|
||||
except (OSError, ValidationError, ValueError) as exc:
|
||||
raise SchemaContextError("physical schema or annotations are invalid") from exc
|
||||
return SchemaContext(
|
||||
physical=physical,
|
||||
annotations=annotations,
|
||||
effective_relationships=_load_effective_relationships(cfg, physical),
|
||||
)
|
||||
@@ -7,16 +7,23 @@ MAX_EXAMPLES_IN_PROMPT = 5
|
||||
|
||||
|
||||
def table_foreign_keys(
|
||||
physical: PhysicalSchema, annotations: Annotations, table: str
|
||||
physical: PhysicalSchema,
|
||||
annotations: Annotations,
|
||||
table: str,
|
||||
effective_relationships: dict[str, list[ForeignKey]] | None = None,
|
||||
) -> list[ForeignKey]:
|
||||
"""FK fisiche + FK logiche dalle annotations (dedup su columns/ref)."""
|
||||
if effective_relationships is not None:
|
||||
return list(effective_relationships.get(table, []))
|
||||
fks = list(physical.tables[table].foreign_keys)
|
||||
ann = annotations.tables.get(table)
|
||||
if ann:
|
||||
seen = {(tuple(f.columns), f.ref_table, tuple(f.ref_columns)) for f in fks}
|
||||
for fk in ann.foreign_keys:
|
||||
if (tuple(fk.columns), fk.ref_table, tuple(fk.ref_columns)) not in seen:
|
||||
key = (tuple(fk.columns), fk.ref_table, tuple(fk.ref_columns))
|
||||
if key not in seen:
|
||||
fks.append(fk)
|
||||
seen.add(key)
|
||||
return fks
|
||||
|
||||
|
||||
@@ -47,6 +54,8 @@ def to_mschema_text(
|
||||
physical: PhysicalSchema,
|
||||
annotations: Annotations | None = None,
|
||||
tables: list[str] | None = None,
|
||||
*,
|
||||
effective_relationships: dict[str, list[ForeignKey]] | None = None,
|
||||
) -> str:
|
||||
"""Serializzazione testuale in stile ThothAI (【Schema】/【Foreign keys】)."""
|
||||
annotations = annotations or Annotations()
|
||||
@@ -73,7 +82,9 @@ def to_mschema_text(
|
||||
shown = ", ".join(column.examples[:MAX_EXAMPLES_IN_PROMPT])
|
||||
lines.append(f" -- Examples: {shown}")
|
||||
lines.append(");")
|
||||
for fk in table_foreign_keys(physical, annotations, table_name):
|
||||
for fk in table_foreign_keys(
|
||||
physical, annotations, table_name, effective_relationships
|
||||
):
|
||||
for src, dst in zip(fk.columns, fk.ref_columns):
|
||||
fk_lines.append(f"{table_name}.{src}={fk.ref_table}.{dst}")
|
||||
lines.extend(["", "【Foreign keys】", *fk_lines])
|
||||
@@ -81,7 +92,10 @@ def to_mschema_text(
|
||||
|
||||
|
||||
def to_schema_dict(
|
||||
physical: PhysicalSchema, annotations: Annotations | None = None
|
||||
physical: PhysicalSchema,
|
||||
annotations: Annotations | None = None,
|
||||
*,
|
||||
effective_relationships: dict[str, list[ForeignKey]] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Vista compatibile con le logiche AV-SQL (schema_dict)."""
|
||||
annotations = annotations or Annotations()
|
||||
@@ -103,13 +117,20 @@ def to_schema_dict(
|
||||
"primary_keys": [c for c in cols if table.columns[c].pk],
|
||||
"foreign_keys": [
|
||||
{"columns": fk.columns, "ref_table": fk.ref_table, "ref_columns": fk.ref_columns}
|
||||
for fk in table_foreign_keys(physical, annotations, table_name)
|
||||
for fk in table_foreign_keys(
|
||||
physical, annotations, table_name, effective_relationships
|
||||
)
|
||||
],
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
def to_markdown(physical: PhysicalSchema, annotations: Annotations | None = None) -> str:
|
||||
def to_markdown(
|
||||
physical: PhysicalSchema,
|
||||
annotations: Annotations | None = None,
|
||||
*,
|
||||
effective_relationships: dict[str, list[ForeignKey]] | None = None,
|
||||
) -> str:
|
||||
"""Report leggibile per il reviewer."""
|
||||
annotations = annotations or Annotations()
|
||||
lines = [
|
||||
@@ -146,7 +167,9 @@ def to_markdown(physical: PhysicalSchema, annotations: Annotations | None = None
|
||||
f"| {column_name} | {column.type} | {'sì' if column.nullable else 'no'} "
|
||||
f"| {'sì' if column.pk else ''} | {cdesc} | {examples} |"
|
||||
)
|
||||
fks = table_foreign_keys(physical, annotations, table_name)
|
||||
fks = table_foreign_keys(
|
||||
physical, annotations, table_name, effective_relationships
|
||||
)
|
||||
if fks:
|
||||
lines += ["", "Foreign keys:"]
|
||||
for fk in fks:
|
||||
|
||||
@@ -85,6 +85,7 @@ nav:
|
||||
- 0009 Sequential description generation: adr/0009-use-one-sequential-description-generation-run.md
|
||||
- 0010 Bounded source samples: adr/0010-allow-bounded-real-source-samples-for-description-generation.md
|
||||
- 0011 Sensitive data flag: adr/0011-gate-source-samples-with-a-sensitive-data-flag.md
|
||||
- 0012 Effective relationship authority: adr/0012-use-the-catalog-as-the-logical-relationship-authority.md
|
||||
- AI catalog description acceptance: testing/2026-08-29-ai-catalog-description-generation-acceptance.md
|
||||
- Design records:
|
||||
- Metadata catalog design: plans/2026-08-26-metadata-catalog-from-thothai.md
|
||||
|
||||
Reference in New Issue
Block a user