From 0736983bc57a59939d6c67375e1f0952c7a24650 Mon Sep 17 00:00:00 2001 From: Codex Date: Sun, 30 Aug 2026 12:14:23 +0200 Subject: [PATCH] feat: protect sensitive catalog samples --- CONTEXT.md | 17 +- PROJECT_STATE.md | 21 +- README.md | 16 +- backend/src/app.ts | 7 + .../catalog/description-generation-worker.ts | 72 ++++-- backend/src/catalog/memory-repository.ts | 11 +- backend/src/catalog/migrate.ts | 2 + .../migrations/006_sensitive_data_flag.ts | 12 + backend/src/catalog/repository.ts | 4 + .../src/catalog/sensitive-data-suggester.ts | 103 +++++++++ backend/src/catalog/synthetic-sample-value.ts | 67 ++++++ backend/src/catalog/types.ts | 2 + .../routes/catalog-description-generation.ts | 42 +++- backend/src/routes/catalog-schema.ts | 2 + ...alog-description-generation-routes.test.ts | 173 ++++++++++++++ ...alog-description-generation-worker.test.ts | 215 ++++++++++++++++-- ...description-generation.integration.test.ts | 2 + ...catalog-description-source-sampler.test.ts | 28 +++ .../catalog-repository.integration.test.ts | 45 +++- backend/test/catalog-schema-routes.test.ts | 20 +- .../catalog-synthetic-sample-value.test.ts | 29 +++ ...urce-samples-with-a-sensitive-data-flag.md | 19 ++ docs/operations/database-management.md | 24 +- frontend/src/api/catalog-databases.ts | 19 +- .../src/shell/DatabaseManagementPage.test.tsx | 195 ++++++++++++---- .../database-management/DatabaseColumns.tsx | 128 ++++++++++- .../MetadataGenerationModelSelector.tsx | 14 +- mkdocs.yml | 1 + 28 files changed, 1162 insertions(+), 128 deletions(-) create mode 100644 backend/src/catalog/migrations/006_sensitive_data_flag.ts create mode 100644 backend/src/catalog/sensitive-data-suggester.ts create mode 100644 backend/src/catalog/synthetic-sample-value.ts create mode 100644 backend/test/catalog-synthetic-sample-value.test.ts create mode 100644 docs/adr/0011-gate-source-samples-with-a-sensitive-data-flag.md diff --git a/CONTEXT.md b/CONTEXT.md index 9ca43d5c..f5a7d008 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -362,13 +362,18 @@ richiede autenticazione. Non appartiene al workspace ed è indipendente dalla co richiesta LiteLLM per conto del backend. Non è un servizio HTTP, non possiede il lifecycle della Description Generation Run e non è una CLI esposta agli utenti. -**Catalog Sample** — Un input transitorio composto da un massimo di cinque righe reali e dai -valori di esempio bounded letti da una Catalog Table per la generazione delle descrizioni. È -trattato come dato non fidato, non viene persistito e non diventa Catalog Metadata. +**Catalog Sample** — Un input transitorio composto da un massimo di cinque righe e da valori di +esempio bounded di una Catalog Table per la generazione delle descrizioni. Può contenere valori +reali oppure sintetici in base al Sensitive Data Flag della Catalog Column; non viene persistito +e non diventa Catalog Metadata. -**Sensitive Data Policy** — L'insieme di regole che classifica i valori sorgente protetti per -l'uso nei processi AI e ne prescrive l'esclusione o l'anonimizzazione. Non coincide con la sola -classificazione dei dati personali. +**Sensitive Data Flag** — La scelta binaria umana applicata a una Catalog Column: `true` protegge +i valori sorgente e `false` ne consente l'invio al modello. Il valore predefinito è `false`, anche +per le nuove colonne. + +**Sensitive Data Policy** — La regola che applica il Sensitive Data Flag ai Catalog Sample: +valori sintetici per una colonna protetta, valori reali per una colonna non protetta. L'AI può +suggerire il flag dai soli metadati tecnici, ma soltanto l'utente lo imposta. _Avoid_: PII filter, sample filter **Introspection Capability** — Una categoria di struttura fisica che una Database Binding diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 49d9fe97..506cbfb2 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -104,16 +104,18 @@ one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before loca sessions still consume the existing workspace configuration in this slice: database-management records do not yet change the NL→SQL handoff. The accepted design is recorded in `docs/plans/2026-08-26-metadata-catalog-from-thothai.md`, the snapshot contract under -`docs/contracts/`, and ADRs 0001–0010. +`docs/contracts/`, and ADRs 0001–0011. Semantic aliases, value descriptions, synonyms, concepts, and logical relationships remain deferred to their dedicated slices. -AI Description Generation preserves ThothAI's use of real source samples: up to five source rows -and five representative non-null example values may be sent transiently to the configured model -provider. The UI and operator documentation disclose this behavior. A required follow-up -improvement is a Sensitive Data Policy that classifies protected fields and excludes or anonymizes -their values before model calls. +AI Description Generation uses the catalog's human-owned Sensitive Data Flag. The flag defaults to +`false`, including for newly synchronized columns. An administrator may request an AI proposal based +only on structural metadata, but it remains an unsaved draft until the human reviews and saves it. +For unprotected columns, up to five source rows and five representative non-null values may be sent +transiently to the configured model provider. Protected columns are omitted from source reads and +replaced in the prompt by deterministic plausible values derived only from their metadata. Existing +descriptions are not regenerated when a flag changes. The accepted AI-description design is recorded in `docs/plans/2026-08-28-ai-catalog-description-generation.md`, with the formal specification in the @@ -141,6 +143,13 @@ point the next required design gate is to compare the catalog snapshot with the preprocessing/schema-linking contracts and plan the cutover; this follow-up must not be treated as optional cleanup or silently omitted. +**Deferred follow-up — Sensitive Data Policy in schema-linking.** The policy is first delivered +and tested in catalog description generation. Its enforcement for core schema-linking remains +out of scope until the current tickets are closed and the owner has completed the acceptance test. +At that gate, resume the design: `tht` must receive a read-only projection of the current Sensitive +Data Flags and exclude values from columns marked sensitive from every LSH result before it is +given to Pi. Do not start this integration before the owner gives final approval after that test. + ## Active deployment work and manual gates ### PSD server deployment program diff --git a/README.md b/README.md index 5676c628..7f81b54e 100644 --- a/README.md +++ b/README.md @@ -305,12 +305,16 @@ Configuration changes take effect after restart and do not use Pi settings or wo `llm_policy`. Before enabling Description Generation, approve the selected model provider for bounded source-data -disclosure. A request may send up to five real source rows and up to five representative distinct, -non-null example values for relevant columns. Samples are transient and are not stored in generation -runs, run logs, application logs, API responses, or catalog metadata; prompt and sample snapshots are -not retained. Automated Sensitive Data Policy filtering and anonymization are not currently provided. -A future Sensitive Data Policy is required to classify protected fields and exclude or anonymize -their values before model calls. +disclosure. Every catalog column has a **Sensitive** flag that defaults to `false`. Administrators can +request an AI proposal based only on structural metadata, then must review and save the resulting +checkboxes themselves. The proposal never reads column contents and is not persisted automatically. + +For unprotected columns, a request may send up to five real source rows and five representative +distinct, non-null example values. Protected columns are omitted from source reads and replaced in the +prompt by deterministic plausible values derived only from column metadata. Samples are transient and +are not stored in generation runs, run logs, application logs, API responses, or catalog metadata; +prompt and sample snapshots are not retained. A flag change applies to later generations and does not +regenerate existing descriptions. Description Generation is an interactive Database Management operation, not a user-facing CLI. The installation runs at most one sequential generation at a time. The run drawer exposes safe diff --git a/backend/src/app.ts b/backend/src/app.ts index 4ff1dab0..d3d56c0c 100644 --- a/backend/src/app.ts +++ b/backend/src/app.ts @@ -56,6 +56,7 @@ import { metadataGenerationModelRoutes } from "./routes/metadata-generation-mode import { catalogDescriptionConsolidationRoutes } from "./routes/catalog-description-consolidation.js"; import { PythonModelCompleter, type ModelCompleter } from "./catalog/model-completer.js"; import { DescriptionGenerationWorker } from "./catalog/description-generation-worker.js"; +import { SensitiveDataSuggester } from "./catalog/sensitive-data-suggester.js"; import { PostgresDescriptionSourceSampler, type DescriptionSourceSampler, @@ -179,6 +180,11 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc catalogOperationCoordinator, descriptionSourceSampler, ); + const sensitiveDataSuggester = new SensitiveDataSuggester( + catalogRepository, + metadataGenerationModels, + modelCompleter, + ); const catalogService = deps?.catalogService ?? new CatalogService( catalogRepository, workspaceRegistry, @@ -438,6 +444,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc catalogDescriptionGenerationRoutes(app, { repository: catalogRepository, worker: descriptionGenerationWorker, + sensitiveDataSuggester, }); settingsRoutes(app, { cfg: config, listModels, getSettings }); piManagementRoutes(app, { service: piManagement }); diff --git a/backend/src/catalog/description-generation-worker.ts b/backend/src/catalog/description-generation-worker.ts index b38ad5ac..233cf22e 100644 --- a/backend/src/catalog/description-generation-worker.ts +++ b/backend/src/catalog/description-generation-worker.ts @@ -12,6 +12,7 @@ import type { DescriptionSourceSampleValue, DescriptionTargetSourceSample, } from "./description-source-sampler.js"; +import { syntheticSampleValue } from "./synthetic-sample-value.js"; import { DescriptionGenerationRunActiveError, type CatalogColumn, @@ -306,35 +307,55 @@ function sourceSampleFor( ): PromptSourceSample | undefined { const targetId = target.kind === "column" ? target.column.id : target.table.id; const sample = samples.find((candidate) => candidate.targetId === targetId); - if (!sample) return undefined; - const relevantColumns = new Set( - target.kind === "column" ? [target.column.name] : target.columns.map((column) => column.name), - ); - const rows = sample.rows.slice(0, budget.rows).map((row) => { + const columns = target.kind === "column" ? [target.column] : target.columns; + const sourceRows = sample?.rows ?? []; + const hasSensitiveColumns = columns.some((column) => column.sensitive); + const hasNonSensitiveColumns = columns.some((column) => !column.sensitive); + const realRowCount = hasNonSensitiveColumns + ? Math.min(sourceRows.length, budget.rows) + : 0; + const rowCount = hasSensitiveColumns ? MAX_SAMPLE_ROWS_PER_REQUEST : realRowCount; + const rows = Array.from({ length: rowCount }, (_, rowIndex) => { + const row = rowIndex < realRowCount ? sourceRows[rowIndex] : undefined; + const sourceFields = new Map((row?.fields ?? []).map((field) => [field.name, field.value])); const fields: Array<{ name: string; value: PromptSampleValue }> = []; - const seen = new Set(); - for (const field of row.fields) { - if (!relevantColumns.has(field.name) || seen.has(field.name)) continue; - seen.add(field.name); + for (const column of columns) { + const value = column.sensitive + ? syntheticSampleValue(column, rowIndex + 1) + : sourceFields.get(column.name); + if (value === undefined) continue; fields.push({ - name: boundedJsonText(field.name, MAX_IDENTIFIER_JSON_BYTES), - value: promptSampleValue(field.value), + name: boundedJsonText(column.name, MAX_IDENTIFIER_JSON_BYTES), + value: promptSampleValue(value), }); if (fields.length === MAX_SAMPLE_FIELDS_PER_ROW) break; } return { fields }; }); - budget.rows -= rows.length; + budget.rows -= realRowCount; const representativeValues: PromptSourceSample["representativeValues"] = []; const seenColumns = new Set(); - let remainingRepresentativeValues = budget.representativeValues; - for (const examples of sample.representativeValues) { - if (remainingRepresentativeValues === 0) break; - if (!relevantColumns.has(examples.column) || seenColumns.has(examples.column)) continue; - seenColumns.add(examples.column); + let remainingRealRepresentativeValues = budget.representativeValues; + let remainingSyntheticRepresentativeValues = MAX_REPRESENTATIVE_VALUES_PER_REQUEST; + for (const column of columns) { + if (seenColumns.has(column.name)) continue; + const remainingRepresentativeValues = column.sensitive + ? remainingSyntheticRepresentativeValues + : remainingRealRepresentativeValues; + if (remainingRepresentativeValues === 0) continue; + const sourceExamples = sample?.representativeValues.find( + (examples) => examples.column === column.name, + ); + const exampleValues = column.sensitive + ? Array.from( + { length: Math.min(Math.max(rowCount, 1), remainingRepresentativeValues) }, + (_, index) => syntheticSampleValue(column, index + 1), + ) + : sourceExamples?.values ?? []; + seenColumns.add(column.name); const values: Array> = []; const seenValues = new Set(); - for (const value of examples.values) { + for (const value of exampleValues) { const normalized = promptSampleValue(value); if (normalized === null) continue; const key = JSON.stringify([typeof normalized, normalized]); @@ -345,11 +366,15 @@ function sourceSampleFor( } if (values.length > 0) { representativeValues.push({ - column: boundedJsonText(examples.column, MAX_IDENTIFIER_JSON_BYTES), + column: boundedJsonText(column.name, MAX_IDENTIFIER_JSON_BYTES), values, }); - remainingRepresentativeValues -= values.length; - budget.representativeValues -= values.length; + if (column.sensitive) { + remainingSyntheticRepresentativeValues -= values.length; + } else { + remainingRealRepresentativeValues -= values.length; + budget.representativeValues -= values.length; + } } if (representativeValues.length === MAX_SAMPLE_COLUMNS) break; } @@ -933,8 +958,9 @@ export class DescriptionGenerationWorker { targetId: target.kind === "column" ? target.column.id : target.table.id, tableName: target.table.name, columnNames: target.kind === "column" - ? [target.column.name] - : target.columns.map((column) => column.name), + ? target.column.sensitive ? [] : [target.column.name] + : target.columns.filter((column) => !column.sensitive) + .map((column) => column.name), })), signal, ); diff --git a/backend/src/catalog/memory-repository.ts b/backend/src/catalog/memory-repository.ts index 82701ba7..aed47bd6 100644 --- a/backend/src/catalog/memory-repository.ts +++ b/backend/src/catalog/memory-repository.ts @@ -202,10 +202,18 @@ export class MemoryCatalogRepository implements CatalogRepository { expectedVersion: number, description: string | null, generatedDescription: string | null, + sensitive?: boolean, ): Promise { const current = await this.getColumn(databaseId, tableId, columnId); if (!current || current.version !== expectedVersion) return undefined; - const updated = { ...current, description, generatedDescription, version: current.version + 1, updatedAt: new Date().toISOString() }; + const updated = { + ...current, + description, + generatedDescription, + sensitive: sensitive ?? current.sensitive, + version: current.version + 1, + updatedAt: new Date().toISOString(), + }; this.columns.set(columnId, updated); return structuredClone(updated); } @@ -621,6 +629,7 @@ export class MemoryCatalogRepository implements CatalogRepository { sourceComment: observed.sourceComment, description: null, generatedDescription: null, + sensitive: false, lastSyncedDatabaseVersion: expectedDatabaseVersion, lastSyncedAt: now, version: 1, diff --git a/backend/src/catalog/migrate.ts b/backend/src/catalog/migrate.ts index eeb3947f..b4467c5d 100644 --- a/backend/src/catalog/migrate.ts +++ b/backend/src/catalog/migrate.ts @@ -8,6 +8,7 @@ import * as catalogTablesMigration from "./migrations/002_catalog_tables.js"; import * as catalogSchemaSyncMigration from "./migrations/003_catalog_schema_sync.js"; import * as catalogRuntimeSequencePrivilegesMigration from "./migrations/004_catalog_runtime_sequence_privileges.js"; import * as descriptionGenerationRunsMigration from "./migrations/005_description_generation_runs.js"; +import * as sensitiveDataFlagMigration from "./migrations/006_sensitive_data_flag.js"; const connectionString = process.env.THT_CATALOG_MIGRATOR_DATABASE_URL; const host = process.env.THT_CATALOG_DB_HOST; @@ -38,6 +39,7 @@ const provider: MigrationProvider = { "003_catalog_schema_sync": catalogSchemaSyncMigration, "004_catalog_runtime_sequence_privileges": catalogRuntimeSequencePrivilegesMigration, "005_description_generation_runs": descriptionGenerationRunsMigration, + "006_sensitive_data_flag": sensitiveDataFlagMigration, }; }, }; diff --git a/backend/src/catalog/migrations/006_sensitive_data_flag.ts b/backend/src/catalog/migrations/006_sensitive_data_flag.ts new file mode 100644 index 00000000..84047183 --- /dev/null +++ b/backend/src/catalog/migrations/006_sensitive_data_flag.ts @@ -0,0 +1,12 @@ +import type { Kysely } from "kysely"; +import type { CatalogDatabase } from "../repository.js"; + +export async function up(db: Kysely): Promise { + await db.schema.alterTable("catalog_columns") + .addColumn("sensitive", "boolean", (column) => column.notNull().defaultTo(false)) + .execute(); +} + +export async function down(db: Kysely): Promise { + await db.schema.alterTable("catalog_columns").dropColumn("sensitive").execute(); +} diff --git a/backend/src/catalog/repository.ts b/backend/src/catalog/repository.ts index 405310ac..5aa41b0d 100644 --- a/backend/src/catalog/repository.ts +++ b/backend/src/catalog/repository.ts @@ -108,6 +108,7 @@ interface CatalogColumnTable { sourceComment: string | null; description: string | null; generatedDescription: string | null; + sensitive: Generated; lastSyncedDatabaseVersion: number | null; lastSyncedAt: Timestamp | null; version: Generated; @@ -290,6 +291,7 @@ function serializeColumn(row: Selectable, foreignKeyCount = sourceComment: row.sourceComment, description: row.description, generatedDescription: row.generatedDescription, + sensitive: row.sensitive, lastSyncedDatabaseVersion: row.lastSyncedDatabaseVersion, lastSyncedAt: row.lastSyncedAt === null ? null : new Date(row.lastSyncedAt).toISOString(), version: row.version, @@ -561,6 +563,7 @@ export class KyselyCatalogRepository implements CatalogRepository { expectedVersion: number, description: string | null, generatedDescription: string | null, + sensitive?: boolean, ): Promise { const belongs = await this.db.selectFrom("catalogTables").select("id") .where("id", "=", tableId).where("databaseId", "=", databaseId).executeTakeFirst(); @@ -568,6 +571,7 @@ export class KyselyCatalogRepository implements CatalogRepository { const row = await this.db.updateTable("catalogColumns").set({ description, generatedDescription, + ...(sensitive === undefined ? {} : { sensitive }), version: sql`version + 1`, updatedAt: sql`now()`, }).where("id", "=", columnId).where("tableId", "=", tableId) diff --git a/backend/src/catalog/sensitive-data-suggester.ts b/backend/src/catalog/sensitive-data-suggester.ts new file mode 100644 index 00000000..c99c6afb --- /dev/null +++ b/backend/src/catalog/sensitive-data-suggester.ts @@ -0,0 +1,103 @@ +import { z } from "zod"; +import type { MetadataGenerationModels } from "./metadata-generation-models.js"; +import type { ModelCompleter } from "./model-completer.js"; +import type { CatalogRepository } from "./types.js"; + +const MAX_COLUMNS = 10_000; +const responseSchema = z.object({ + suggestions: z.array(z.object({ + columnId: z.uuid(), + sensitive: z.boolean(), + }).strict()).max(MAX_COLUMNS), +}).strict(); + +export interface SensitiveDataSuggestion { + columnId: string; + sensitive: boolean; +} + +export class SensitiveDataSuggestionTargetNotFoundError extends Error { + constructor() { + super("database not found"); + this.name = "SensitiveDataSuggestionTargetNotFoundError"; + } +} + +export class SensitiveDataSuggestionInvalidResponseError extends Error { + constructor() { + super("sensitive-data suggestion response is invalid"); + this.name = "SensitiveDataSuggestionInvalidResponseError"; + } +} + +export class SensitiveDataSuggester { + constructor( + private readonly repository: CatalogRepository, + private readonly models: MetadataGenerationModels, + private readonly completer: ModelCompleter, + ) {} + + async suggest( + databaseId: string, + modelId: string, + signal: AbortSignal, + ): Promise { + const database = await this.repository.get(databaseId); + if (!database) throw new SensitiveDataSuggestionTargetNotFoundError(); + + const tables = await this.repository.listTables(databaseId); + const columns = (await Promise.all(tables.map(async (table) => ({ + table, + columns: await this.repository.listColumns(databaseId, table.id), + })))).flatMap(({ table, columns: tableColumns }) => tableColumns.map((column) => ({ + columnId: column.id, + table: table.name, + column: column.name, + dataType: column.dataType, + nullable: column.isNullable, + primaryKey: column.isPrimaryKey, + foreignKey: column.isForeignKey, + }))); + if (columns.length === 0) return []; + if (columns.length > MAX_COLUMNS) throw new SensitiveDataSuggestionInvalidResponseError(); + + const content = await this.completer.complete({ + model: this.models.resolve(modelId), + signal, + messages: [ + { + role: "system", + content: [ + "Classify whether each database column is likely to contain sensitive source values.", + "Use only the supplied structural metadata. Return strict JSON with this exact shape:", + '{"suggestions":[{"columnId":"uuid","sensitive":true}]}', + "Return every supplied column exactly once. Do not add explanations or markdown.", + ].join("\n"), + }, + { + role: "user", + content: JSON.stringify({ + database: database.databaseName, + schema: database.schema, + columns, + }), + }, + ], + }); + + try { + const parsed = responseSchema.parse(JSON.parse(content)); + const expected = new Set(columns.map((column) => column.columnId)); + const received = new Set(parsed.suggestions.map((suggestion) => suggestion.columnId)); + if (received.size !== parsed.suggestions.length + || received.size !== expected.size + || [...received].some((columnId) => !expected.has(columnId))) { + throw new SensitiveDataSuggestionInvalidResponseError(); + } + return parsed.suggestions; + } catch (error) { + if (error instanceof SensitiveDataSuggestionInvalidResponseError) throw error; + throw new SensitiveDataSuggestionInvalidResponseError(); + } + } +} diff --git a/backend/src/catalog/synthetic-sample-value.ts b/backend/src/catalog/synthetic-sample-value.ts new file mode 100644 index 00000000..6ba084d0 --- /dev/null +++ b/backend/src/catalog/synthetic-sample-value.ts @@ -0,0 +1,67 @@ +import type { DescriptionSourceSampleValue } from "./description-source-sampler.js"; + +const FIRST_NAMES = ["marta", "luca", "elena", "paolo", "giulia"] as const; +const LAST_NAMES = ["rossi", "bianchi", "conti", "romano", "ferrari"] as const; +const NUMERIC_TYPE = /(int|numeric|decimal|real|double|float|money)/; +const BOOLEAN_TYPE = /(bool)/; + +function nameAt(index: number): string { + const offset = Math.max(0, index - 1); + return `${FIRST_NAMES[offset % FIRST_NAMES.length]} ${LAST_NAMES[offset % LAST_NAMES.length]}`; +} + +export function syntheticSampleValue( + column: { name: string; dataType: string }, + index: number, +): DescriptionSourceSampleValue { + const ordinal = Math.max(1, index); + const name = column.name.toLocaleLowerCase("en-US"); + const type = column.dataType.toLocaleLowerCase("en-US"); + const person = nameAt(ordinal).split(" "); + const safeName = name.replace(/[^a-z0-9]+/g, "_").replace(/^_+|_+$/g, "") || "value"; + + if (type.endsWith("[]") || type.startsWith("_") || /\barray\b/.test(type)) { + if (NUMERIC_TYPE.test(type)) { + return `{${1000 + ordinal},${1001 + ordinal}}`; + } + if (BOOLEAN_TYPE.test(type)) return `{${ordinal % 2 === 1},${ordinal % 2 !== 1}}`; + return `{${safeName}_${String(ordinal).padStart(3, "0")},${safeName}_${String(ordinal + 1).padStart(3, "0")}}`; + } + if (/^jsonb?$/.test(type)) { + return JSON.stringify({ example: `${safeName}_${String(ordinal).padStart(3, "0")}`, sequence: ordinal }); + } + if (/(uuid|uniqueidentifier)/.test(type)) { + return `00000000-0000-4000-8000-${String(ordinal).padStart(12, "0")}`; + } + if (/\bcidr\b/.test(type)) return "192.0.2.0/24"; + if (/\binet\b/.test(type)) return `192.0.2.${((ordinal - 1) % 254) + 1}`; + if (/(timestamp|datetime)/.test(type)) { + return `2024-01-${String(Math.min(ordinal, 28)).padStart(2, "0")}T10:30:00.000Z`; + } + if (/\bdate\b/.test(type)) { + return `198${ordinal % 10}-01-${String(Math.min(ordinal, 28)).padStart(2, "0")}`; + } + if (/\btime\b/.test(type)) { + return `10:30:${String(ordinal % 60).padStart(2, "0")}`; + } + if (/\binterval\b/.test(type)) { + return `${ordinal} days ${String(ordinal % 24).padStart(2, "0")}:00:00`; + } + if (/\bbytea\b/.test(type)) return `\\x${ordinal.toString(16).padStart(8, "0")}`; + if (BOOLEAN_TYPE.test(type)) return ordinal % 2 === 1; + if (NUMERIC_TYPE.test(type)) return 1000 + ordinal; + + if (/e[-_]?mail/.test(name)) { + return `${person[0]}.${person[1]}@example.com`; + } + if (/(phone|mobile|cell|telefono|telefono_mobile|tel_)/.test(name)) { + return `+39 02 5550 ${String(1000 + ordinal).padStart(4, "0")}`; + } + if (/(first_?name|given_?name|nome)/.test(name)) return person[0]!; + if (/(last_?name|family_?name|surname|cognome)/.test(name)) return person[1]!; + if (/(full_?name|patient_?name|person_?name)/.test(name)) return nameAt(ordinal); + if (/(birth|dob|data_nascita)/.test(name)) { + return `198${ordinal % 10}-01-${String(Math.min(ordinal, 28)).padStart(2, "0")}`; + } + return `${safeName}_${String(ordinal).padStart(3, "0")}`; +} diff --git a/backend/src/catalog/types.ts b/backend/src/catalog/types.ts index 43e54ef7..caa36446 100644 --- a/backend/src/catalog/types.ts +++ b/backend/src/catalog/types.ts @@ -88,6 +88,7 @@ export interface CatalogColumn { sourceComment: string | null; description: string | null; generatedDescription: string | null; + sensitive: boolean; lastSyncedDatabaseVersion: number | null; lastSyncedAt: string | null; version: number; @@ -349,6 +350,7 @@ export interface CatalogRepository { expectedVersion: number, description: string | null, generatedDescription: string | null, + sensitive?: boolean, ): Promise; consolidateGeneratedDescriptions( databaseId: string, diff --git a/backend/src/routes/catalog-description-generation.ts b/backend/src/routes/catalog-description-generation.ts index 2da56655..bfbaa0f3 100644 --- a/backend/src/routes/catalog-description-generation.ts +++ b/backend/src/routes/catalog-description-generation.ts @@ -11,6 +11,12 @@ import { type DescriptionGenerationWorker, } from "../catalog/description-generation-worker.js"; import { MetadataGenerationModelUnavailableError } from "../catalog/metadata-generation-models.js"; +import { ModelCompletionProviderError } from "../catalog/model-completer.js"; +import { + SensitiveDataSuggester, + SensitiveDataSuggestionInvalidResponseError, + SensitiveDataSuggestionTargetNotFoundError, +} from "../catalog/sensitive-data-suggester.js"; import { CatalogOperationInProgressError, CatalogUnavailableError, @@ -22,6 +28,7 @@ import { const idSchema = z.uuid(); const modelIdSchema = z.string().regex(/^[a-z][a-z0-9._-]{0,63}$/); +const suggestionSchema = z.object({ modelId: modelIdSchema }).strict(); const selectedTargetIdsSchema = z.array(idSchema).min(1); const startSchema = z.discriminatedUnion("scope", [ z.object({ @@ -116,6 +123,19 @@ function safeError(reply: FastifyReply, error: unknown) { message: "The selected metadata-generation model is unavailable.", }); } + if (error instanceof SensitiveDataSuggestionTargetNotFoundError) { + return reply.code(404).send({ + code: "database_not_found", + message: "Database configuration was not found.", + }); + } + if (error instanceof SensitiveDataSuggestionInvalidResponseError + || error instanceof ModelCompletionProviderError) { + return reply.code(502).send({ + code: "sensitive_data_suggestion_failed", + message: "Sensitive-data suggestions could not be prepared.", + }); + } if (error instanceof DescriptionGenerationDuplicateTargetIdsError) { return reply.code(400).send({ code: "description_generation_target_ids_duplicate", @@ -172,8 +192,28 @@ function safeError(reply: FastifyReply, error: unknown) { export function catalogDescriptionGenerationRoutes( app: FastifyInstance, - deps: { repository: CatalogRepository; worker: DescriptionGenerationWorker }, + deps: { + repository: CatalogRepository; + worker: DescriptionGenerationWorker; + sensitiveDataSuggester: SensitiveDataSuggester; + }, ): void { + app.post("/catalog/databases/:databaseId/sensitive-data-suggestions", async (request, reply) => { + if (!manage(request, reply)) return reply; + try { + const databaseId = idSchema.parse((request.params as { databaseId?: unknown }).databaseId); + const input = suggestionSchema.parse(request.body); + const suggestions = await deps.sensitiveDataSuggester.suggest( + databaseId, + input.modelId, + new AbortController().signal, + ); + return { suggestions }; + } catch (error) { + return safeError(reply, error); + } + }); + app.post("/catalog/databases/:databaseId/description-generation-runs", async (request, reply) => { if (!manage(request, reply)) return reply; try { diff --git a/backend/src/routes/catalog-schema.ts b/backend/src/routes/catalog-schema.ts index b37a6a59..261d66c9 100644 --- a/backend/src/routes/catalog-schema.ts +++ b/backend/src/routes/catalog-schema.ts @@ -18,6 +18,7 @@ const metadataSchema = z.object({ version: z.number().int().positive(), description: z.string().max(20_000).nullable(), generatedDescription: z.string().max(20_000).nullable(), + sensitive: z.boolean().optional(), }).strict(); const createRunSchema = z.object({ version: z.number().int().positive(), @@ -117,6 +118,7 @@ export function catalogSchemaRoutes( input.version, normalized(input.description), normalized(input.generatedDescription), + input.sensitive, ); if (!updated) return reply.code(409).send({ code: "column_stale", message: "Column metadata changed. Reload and try again." }); return updated; diff --git a/backend/test/catalog-description-generation-routes.test.ts b/backend/test/catalog-description-generation-routes.test.ts index 79daa2c5..dbb47893 100644 --- a/backend/test/catalog-description-generation-routes.test.ts +++ b/backend/test/catalog-description-generation-routes.test.ts @@ -141,6 +141,83 @@ async function waitForTerminalRun(app: ReturnType, runId: strin throw new Error(`Description Generation Run ${runId} did not finish`); } +test("suggests sensitive flags from structural metadata without persisting them", async () => { + const modelCompleter = { + complete: vi.fn(async () => JSON.stringify({ + suggestions: [{ columnId: expect.any(String), sensitive: true }], + })), + }; + const { app, repository, database, column } = await setup(modelCompleter); + modelCompleter.complete.mockResolvedValueOnce(JSON.stringify({ + suggestions: [{ columnId: column.id, sensitive: true }], + })); + + try { + const response = await app.inject({ + method: "POST", + url: `/catalog/databases/${database.id}/sensitive-data-suggestions`, + payload: { modelId: configuredModel.id }, + }); + + expect(response.statusCode).toBe(200); + expect(response.json()).toEqual({ + suggestions: [{ columnId: column.id, sensitive: true }], + }); + expect(await repository.getColumn(database.id, column.tableId, column.id)) + .toMatchObject({ sensitive: false }); + + const request = modelCompleter.complete.mock.calls[0]![0] as ModelCompletionRequest; + const prompt = request.messages.map((message) => message.content).join("\n"); + expect(prompt).toContain("patients"); + expect(prompt).toContain("birth_date"); + expect(prompt).toContain("date"); + expect(prompt).not.toContain("Patient date of birth"); + expect(prompt).not.toContain("test-provider-secret"); + } finally { + await app.close(); + } +}); + +test.each(["malformed", "incomplete", "duplicate"] as const)( + "fails safely when sensitive-data suggestions are %s", + async (kind) => { + const modelCompleter: ModelCompleter = { + complete: vi.fn(async () => "unused"), + }; + const { app, repository, database, column } = await setup(modelCompleter); + const rawResponse = kind === "malformed" + ? "RAW_PROVIDER_RESPONSE_DO_NOT_EXPOSE_{" + : kind === "incomplete" + ? JSON.stringify({ suggestions: [] }) + : JSON.stringify({ + suggestions: [ + { columnId: column.id, sensitive: true }, + { columnId: column.id, sensitive: true }, + ], + }); + vi.mocked(modelCompleter.complete).mockResolvedValueOnce(rawResponse); + + try { + const response = await app.inject({ + method: "POST", + url: `/catalog/databases/${database.id}/sensitive-data-suggestions`, + payload: { modelId: configuredModel.id }, + }); + + expect(response.statusCode).toBe(502); + expect(response.json()).toEqual({ + code: "sensitive_data_suggestion_failed", + message: "Sensitive-data suggestions could not be prepared.", + }); + expect(response.body).not.toContain(rawResponse); + expect(await repository.getColumn(database.id, column.tableId, column.id)) + .toMatchObject({ sensitive: false }); + } finally { + await app.close(); + } + }, +); + interface SseFrame { id?: string; event?: string; @@ -398,6 +475,102 @@ test("keeps real source samples transient across the Fastify API and application } }); +test("never exposes a protected source value to the model, persistence, logs, or browser APIs", async () => { + let selectedColumnId = ""; + const protectedValue = "PROTECTED_SOURCE_VALUE_8f4c2a"; + const modelCompleter: ModelCompleter = { + complete: vi.fn(async () => JSON.stringify({ + results: [{ + targetId: selectedColumnId, + outcome: "generated", + description: "Data di nascita del paziente.", + }], + })), + }; + const descriptionSourceSampler: DescriptionSourceSampler = { + sample: vi.fn(async (_database, targets) => [{ + targetId: targets[0]!.targetId, + tableName: targets[0]!.tableName, + rows: [{ fields: [{ name: "birth_date", value: protectedValue }] }], + representativeValues: [{ column: "birth_date", values: [protectedValue] }], + }]), + }; + const { app, repository, database, table, column } = await setup( + modelCompleter, + {}, + "it", + descriptionSourceSampler, + ); + selectedColumnId = column.id; + await repository.updateColumnMetadata( + database.id, + table.id, + column.id, + column.version, + column.description, + column.generatedDescription, + true, + ); + const logSpies = [ + vi.spyOn(app.log, "info"), + vi.spyOn(app.log, "warn"), + vi.spyOn(app.log, "error"), + ]; + + try { + const start = await app.inject({ + method: "POST", + url: `/catalog/databases/${database.id}/description-generation-runs`, + payload: { + modelId: configuredModel.id, + scope: "selected_columns", + targetIds: [column.id], + }, + }); + expect(start.statusCode).toBe(202); + await waitForTerminalRun(app, start.json().id); + + expect(descriptionSourceSampler.sample).toHaveBeenCalledWith( + expect.anything(), + [expect.objectContaining({ targetId: column.id, columnNames: [] })], + expect.any(AbortSignal), + ); + const completionRequest = vi.mocked(modelCompleter.complete).mock.calls[0]![0]; + const providerPayload = JSON.stringify(completionRequest.messages); + expect(providerPayload).not.toContain(protectedValue); + expect(providerPayload).toContain("1981-01-01"); + + const apiResponses = await Promise.all([ + app.inject({ method: "GET", url: `/catalog/description-generation-runs/${start.json().id}` }), + app.inject({ + method: "GET", + url: `/catalog/description-generation-runs/${start.json().id}/events-list`, + }), + app.inject({ method: "GET", url: `/catalog/databases/${database.id}` }), + app.inject({ method: "GET", url: `/catalog/databases/${database.id}/tables` }), + app.inject({ + method: "GET", + url: `/catalog/databases/${database.id}/tables/${table.id}/columns`, + }), + ]); + expect(apiResponses.every((response) => response.statusCode === 200)).toBe(true); + expect(apiResponses.map((response) => response.body).join("\n")).not.toContain(protectedValue); + + const persisted = JSON.stringify({ + run: await repository.getDescriptionGenerationRun(start.json().id), + events: await repository.listDescriptionGenerationEvents(start.json().id), + database: await repository.get(database.id), + table: await repository.getTable(database.id, table.id), + column: await repository.getColumn(database.id, table.id, column.id), + }); + expect(persisted).not.toContain(protectedValue); + expect(JSON.stringify(logSpies.flatMap((spy) => spy.mock.calls))).not.toContain(protectedValue); + } finally { + for (const spy of logSpies) spy.mockRestore(); + await app.close(); + } +}); + test("wires the production sampler to the same injected CatalogPostgresAccess instance", async () => { let selectedColumnId = ""; const sampleSecret = "PRODUCTION_WIRING_SAMPLE_f2986a"; diff --git a/backend/test/catalog-description-generation-worker.test.ts b/backend/test/catalog-description-generation-worker.test.ts index c028541c..18749bd0 100644 --- a/backend/test/catalog-description-generation-worker.test.ts +++ b/backend/test/catalog-description-generation-worker.test.ts @@ -222,7 +222,7 @@ test("adds only bounded transient source samples to the model request", async () tables: [{ name: "patients", sourceComment: null }], columns: [{ tableName: "patients", - name: "status", + name: "patient_email", ordinalPosition: 1, dataType: "text", isNullable: true, @@ -243,9 +243,18 @@ test("adds only bounded transient source samples to the model request", async () }); const table = (await repository.listTables(database.id))[0]!; const columns = await repository.listColumns(database.id, table.id); - const column = columns.find((candidate) => candidate.name === "status")!; + const column = columns.find((candidate) => candidate.name === "patient_email")!; const ward = columns.find((candidate) => candidate.name === "ward")!; - const sampleSecret = "ONLY_IN_TRANSIENT_SAMPLE_7f29c8"; + await repository.updateColumnMetadata( + database.id, + table.id, + column.id, + column.version, + column.description, + column.generatedDescription, + true, + ); + const sampleSecret = "real.patient@hospital.invalid"; const sourceSampler: DescriptionSourceSampler = { sample: vi.fn(async () => [{ targetId: column.id, @@ -265,11 +274,14 @@ test("adds only bounded transient source samples to the model request", async () rows: [ { fields: [{ name: ward.name, value: "row-4" }] }, { fields: [{ name: ward.name, value: "row-5" }] }, - { fields: [{ name: ward.name, value: "row-6-must-be-omitted" }] }, + { fields: [{ name: ward.name, value: "row-6" }] }, + { fields: [{ name: ward.name, value: "row-7" }] }, + { fields: [{ name: ward.name, value: "row-8" }] }, + { fields: [{ name: ward.name, value: "row-9-must-be-omitted" }] }, ], representativeValues: [{ column: ward.name, - values: ["ward-1", "ward-2", "ward-3-must-be-omitted"], + values: ["ward-1", "ward-2", "ward-3", "ward-4", "ward-5", "ward-6-must-be-omitted"], }], }]), }; @@ -321,7 +333,7 @@ test("adds only bounded transient source samples to the model request", async () expect(sourceSampler.sample).toHaveBeenCalledWith( expect.objectContaining({ id: database.id, binding: database.binding }), [ - { targetId: column.id, tableName: table.name, columnNames: [column.name] }, + { targetId: column.id, tableName: table.name, columnNames: [] }, { targetId: ward.id, tableName: table.name, columnNames: [ward.name] }, ], expect.any(AbortSignal), @@ -338,21 +350,29 @@ test("adds only bounded transient source samples to the model request", async () targetContext.sourceSample?.representativeValues.flatMap((entry) => entry.values) ?? [] ), ); - expect(sampledRows).toHaveLength(5); - expect(representativeValues).toHaveLength(5); - expect(context.targets[0].sourceSample.rows).toHaveLength(3); - expect(context.targets[1].sourceSample.rows).toHaveLength(2); + expect(sampledRows).toHaveLength(10); + expect(representativeValues).toHaveLength(10); + expect(context.targets[0].sourceSample.rows).toHaveLength(5); + expect(context.targets[1].sourceSample.rows).toHaveLength(5); expect(context.targets[0].sourceSample.representativeValues).toEqual([{ column: column.name, - values: [sampleSecret, "two", "three"], + values: [ + "marta.rossi@example.com", + "luca.bianchi@example.com", + "elena.conti@example.com", + "paolo.romano@example.com", + "giulia.ferrari@example.com", + ], }]); expect(context.targets[1].sourceSample.representativeValues).toEqual([{ column: ward.name, - values: ["ward-1", "ward-2"], + values: ["ward-1", "ward-2", "ward-3", "ward-4", "ward-5"], }]); - expect(userMessage).toContain(sampleSecret); + expect(userMessage).not.toContain(sampleSecret); + expect(userMessage).not.toMatch(/synthetic|fake|fittizi/i); + expect(userMessage).toContain("marta.rossi@example.com"); expect(userMessage).not.toMatch( - /row-6-must-be-omitted|ward-3-must-be-omitted/, + /row-9-must-be-omitted|ward-6-must-be-omitted/, ); const persisted = JSON.stringify({ @@ -366,6 +386,173 @@ test("adds only bounded transient source samples to the model request", async () expect(persisted).not.toContain(sampleSecret); }); +test("gives every sensitive column synthetic context without consuming the real sample budget", async () => { + const repository = new MemoryCatalogRepository(); + const database = await repository.create({ + workspaceId: "psd-clinical", + engine: "postgres", + databaseName: "warehouse", + schema: "datawarehouse", + binding: { transport: "postgres_direct", host: "db.internal", port: 5432, username: "reader" }, + }); + await repository.applySchemaSync(database.id, database.version, "all", [], { + schemaVersion: 1, + capabilities: { tables: "available", columns: "available", relationships: "available" }, + tables: [{ name: "patients", sourceComment: null }], + columns: [{ + tableName: "patients", + name: "patient_email", + ordinalPosition: 1, + dataType: "text", + isNullable: true, + defaultExpression: null, + primaryKeyPosition: null, + sourceComment: null, + }, { + tableName: "patients", + name: "patient_phone", + ordinalPosition: 2, + dataType: "text", + isNullable: true, + defaultExpression: null, + primaryKeyPosition: null, + sourceComment: null, + }, { + tableName: "patients", + name: "ward", + ordinalPosition: 3, + dataType: "text", + isNullable: true, + defaultExpression: null, + primaryKeyPosition: null, + sourceComment: null, + }], + relationships: [], + }); + const table = (await repository.listTables(database.id))[0]!; + const columns = await repository.listColumns(database.id, table.id); + const email = columns.find((column) => column.name === "patient_email")!; + const phone = columns.find((column) => column.name === "patient_phone")!; + const ward = columns.find((column) => column.name === "ward")!; + for (const column of [email, phone]) { + await repository.updateColumnMetadata( + database.id, + table.id, + column.id, + column.version, + column.description, + column.generatedDescription, + true, + ); + } + const wardValues = ["ward-a", "ward-b", "ward-c", "ward-d", "ward-e"]; + const sourceSampler: DescriptionSourceSampler = { + sample: vi.fn(async (_database, targets) => targets.map((target) => { + if (target.columnNames.length === 0) { + return { + targetId: target.targetId, + tableName: target.tableName, + rows: [], + representativeValues: [], + }; + } + const columnName = target.columnNames[0]!; + return { + targetId: target.targetId, + tableName: target.tableName, + rows: wardValues.map((value) => ({ fields: [{ name: columnName, value }] })), + representativeValues: [{ column: columnName, values: wardValues }], + }; + })), + }; + const completer: ModelCompleter = { + complete: vi.fn(async (request) => { + const context = JSON.parse(request.messages[1]!.content.split("\n").slice(1).join("\n")); + return JSON.stringify({ + results: context.targets.map((target: { targetId: string }) => ({ + targetId: target.targetId, + outcome: "generated", + description: "Descrizione generata.", + })), + }); + }), + }; + const models: MetadataGenerationModels = { + catalog: () => ({ models: [{ id: "openai-mini", label: "OpenAI Mini" }], default: "openai-mini" }), + resolve: () => ({ + id: "openai-mini", + provider: "openai", + model: "gpt-4.1-mini", + apiKeyEnv: "OPENAI_API_KEY", + apiKey: "test-provider-secret", + }), + }; + const worker = new DescriptionGenerationWorker( + repository, + { + read: vi.fn(async () => ({ + workspace: { workspace: { language: "it" } }, + revision: {}, + })), + } as unknown as WorkspaceRegistry, + models, + completer, + new CatalogOperationCoordinator(), + sourceSampler, + ); + + const run = await worker.start( + database.id, + "openai-mini", + "selected_columns", + [email.id, phone.id, ward.id], + ); + await worker.waitForRun(run.id); + + expect(sourceSampler.sample).toHaveBeenCalledWith( + expect.objectContaining({ id: database.id }), + [ + { targetId: email.id, tableName: table.name, columnNames: [] }, + { targetId: phone.id, tableName: table.name, columnNames: [] }, + { targetId: ward.id, tableName: table.name, columnNames: [ward.name] }, + ], + expect.any(AbortSignal), + ); + const request = vi.mocked(completer.complete).mock.calls[0]![0] as ModelCompletionRequest; + const context = JSON.parse(request.messages[1]!.content.split("\n").slice(1).join("\n")); + const targets = new Map( + context.targets.map((target: { targetId: string }) => [target.targetId, target]), + ); + expect(targets.get(email.id)).toMatchObject({ + sourceSample: { + rows: expect.arrayContaining([ + { fields: [{ name: email.name, value: "marta.rossi@example.com" }] }, + ]), + representativeValues: [{ + column: email.name, + values: expect.arrayContaining(["marta.rossi@example.com"]), + }], + }, + }); + expect(targets.get(phone.id)).toMatchObject({ + sourceSample: { + rows: expect.arrayContaining([ + { fields: [{ name: phone.name, value: "+39 02 5550 1001" }] }, + ]), + representativeValues: [{ + column: phone.name, + values: expect.arrayContaining(["+39 02 5550 1001"]), + }], + }, + }); + expect(targets.get(ward.id)).toMatchObject({ + sourceSample: { + rows: wardValues.map((value) => ({ fields: [{ name: ward.name, value }] })), + representativeValues: [{ column: ward.name, values: wardValues }], + }, + }); +}); + test("continues metadata-only with one safe warning when source sampling is unavailable", async () => { const repository = new MemoryCatalogRepository(); const database = await repository.create({ diff --git a/backend/test/catalog-description-generation.integration.test.ts b/backend/test/catalog-description-generation.integration.test.ts index a4caad84..771d5e69 100644 --- a/backend/test/catalog-description-generation.integration.test.ts +++ b/backend/test/catalog-description-generation.integration.test.ts @@ -11,6 +11,7 @@ import { up as upDatabases } from "../src/catalog/migrations/001_workspace_datab import { up as upTables } from "../src/catalog/migrations/002_catalog_tables.js"; import { up as upSchemaSync } from "../src/catalog/migrations/003_catalog_schema_sync.js"; import { up as upDescriptionGeneration } from "../src/catalog/migrations/005_description_generation_runs.js"; +import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js"; import { KyselyCatalogRepository, type CatalogDatabase } from "../src/catalog/repository.js"; import { loadConfig } from "../src/config.js"; import type { WorkspaceRegistry } from "../src/workspaces/registry.js"; @@ -43,6 +44,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a await upDatabases(db); await upTables(db); await upSchemaSync(db); + await upSensitiveDataFlag(db); await upDescriptionGeneration(db); const repository = new KyselyCatalogRepository(db); const database = await repository.create({ diff --git a/backend/test/catalog-description-source-sampler.test.ts b/backend/test/catalog-description-source-sampler.test.ts index 06556032..07434e48 100644 --- a/backend/test/catalog-description-source-sampler.test.ts +++ b/backend/test/catalog-description-source-sampler.test.ts @@ -92,6 +92,34 @@ test("samples at most five source rows and five distinct non-null examples in a expect(end).toHaveBeenCalledOnce(); }); +test("does not issue a SELECT when a protected target has no source columns", async () => { + const query = vi.fn(async () => ({ rows: [] })); + const end = vi.fn(async () => undefined); + const access: CatalogPostgresAccess = { + connect: vi.fn(async () => ({ query, end }) as CatalogDatabaseClient), + }; + const sampler = new PostgresDescriptionSourceSampler(access); + + const samples = await sampler.sample(database, [{ + targetId: target.targetId, + tableName: target.tableName, + columnNames: [], + }], new AbortController().signal); + + expect(samples).toEqual([{ + targetId: target.targetId, + tableName: target.tableName, + rows: [], + representativeValues: [], + }]); + expect(query.mock.calls).toEqual([ + ["BEGIN TRANSACTION READ ONLY", []], + ["ROLLBACK", []], + ]); + expect(query.mock.calls.some(([sql]) => String(sql).startsWith("SELECT"))).toBe(false); + expect(end).toHaveBeenCalledOnce(); +}); + test("rolls back and closes the source connection when sampling fails", async () => { const query = vi.fn(async (sql: string) => { if (sql.startsWith("SELECT")) throw new Error("distinctive-source-secret"); diff --git a/backend/test/catalog-repository.integration.test.ts b/backend/test/catalog-repository.integration.test.ts index eb5a73d9..ecdf0cda 100644 --- a/backend/test/catalog-repository.integration.test.ts +++ b/backend/test/catalog-repository.integration.test.ts @@ -10,6 +10,7 @@ import { up as upTables } from "../src/catalog/migrations/002_catalog_tables.js" import { up as upSchemaSync } from "../src/catalog/migrations/003_catalog_schema_sync.js"; import { up as upRuntimeSequencePrivileges } from "../src/catalog/migrations/004_catalog_runtime_sequence_privileges.js"; import { up as upDescriptionGeneration } from "../src/catalog/migrations/005_description_generation_runs.js"; +import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js"; const dockerAvailable = spawnSync("docker", ["info"], { stdio: "ignore" }).status === 0; @@ -23,6 +24,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo await upDatabases(db); await upTables(db); await upSchemaSync(db); + await upSensitiveDataFlag(db); await sql`CREATE ROLE thothii_catalog_runtime`.execute(db); await upRuntimeSequencePrivileges(db); const sequencePrivilege = await sql<{ allowed: boolean }>` @@ -75,11 +77,47 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo }; expect(await repository.applySchemaSync(created.id, 1, "columns", [], fullColumnsSnapshot)) .toMatchObject({ created: 4, deleted: 0 }); - expect((await repository.listColumns(created.id, patients.id)).map((column) => column.name)) - .toEqual(["id", "name"]); + expect((await repository.listColumns(created.id, patients.id)).map((column) => ({ + name: column.name, + sensitive: column.sensitive, + }))).toEqual([ + { name: "id", sensitive: false }, + { name: "name", sensitive: false }, + ]); expect((await repository.listColumns(created.id, visits.id)).map((column) => column.name)) .toEqual(["id", "patient_id"]); + const patientName = (await repository.listColumns(created.id, patients.id)) + .find((column) => column.name === "name")!; + expect(await repository.updateColumnMetadata( + created.id, + patients.id, + patientName.id, + patientName.version, + patientName.description, + patientName.generatedDescription, + true, + )).toMatchObject({ sensitive: true }); + const refreshedColumnsSnapshot: ObservedSchemaSnapshot = { + ...fullColumnsSnapshot, + schemaVersion: 2, + columns: fullColumnsSnapshot.columns.map((column) => column.tableName === "patients" + && column.name === "name" + ? { ...column, sourceComment: "Sensitive patient name" } + : column), + }; + expect(await repository.applySchemaSync( + created.id, + 1, + "columns", + [patients.id], + refreshedColumnsSnapshot, + )).toMatchObject({ updated: 1 }); + expect(await repository.getColumn(created.id, patients.id, patientName.id)).toMatchObject({ + sensitive: true, + sourceComment: "Sensitive patient name", + }); + const reducedColumnsSnapshot: ObservedSchemaSnapshot = { ...fullColumnsSnapshot, columns: fullColumnsSnapshot.columns.filter((column) => column.name === "id"), @@ -135,6 +173,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository performs scoped metadata cl await upDatabases(db); await upTables(db); await upSchemaSync(db); + await upSensitiveDataFlag(db); const repository = new KyselyCatalogRepository(db); const database = await repository.create({ workspaceId: "cleanup-test", @@ -212,6 +251,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository atomically consolidates sel await upDatabases(db); await upTables(db); await upSchemaSync(db); + await upSensitiveDataFlag(db); const repository = new KyselyCatalogRepository(db); const database = await repository.create({ workspaceId: "consolidation-test", @@ -310,6 +350,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists globally exclusive await upDatabases(db); await upTables(db); await upSchemaSync(db); + await upSensitiveDataFlag(db); await upDescriptionGeneration(db); const repository = new KyselyCatalogRepository(db); const firstDatabase = await repository.create({ diff --git a/backend/test/catalog-schema-routes.test.ts b/backend/test/catalog-schema-routes.test.ts index 746e2114..c4cb3103 100644 --- a/backend/test/catalog-schema-routes.test.ts +++ b/backend/test/catalog-schema-routes.test.ts @@ -234,16 +234,30 @@ test("keeps generated descriptions editable and preserves them across synchroniz }); expect(editedTable.json()).toMatchObject({ description: null, generatedDescription: "Generated table draft" }); const idColumn = (await repository.listColumns(database.id, patients.id))[0]; + expect(idColumn.sensitive).toBe(false); const editedColumn = await app.inject({ method: "PATCH", url: `/catalog/databases/${database.id}/tables/${patients.id}/columns/${idColumn.id}`, - payload: { version: idColumn.version, description: "Reviewed key", generatedDescription: "Generated key draft" }, + payload: { + version: idColumn.version, + description: "Reviewed key", + generatedDescription: "Generated key draft", + sensitive: true, + }, + }); + expect(editedColumn.json()).toMatchObject({ + description: "Reviewed key", + generatedDescription: "Generated key draft", + sensitive: true, }); - expect(editedColumn.json()).toMatchObject({ description: "Reviewed key", generatedDescription: "Generated key draft" }); const second = await app.inject({ method: "POST", url: `/catalog/databases/${database.id}/sync-runs`, payload: { version: database.version, scope: "all", tableIds: [] } }); await waitFor(repository, second.json().id, "succeeded"); expect(await repository.getTable(database.id, patients.id)).toMatchObject({ generatedDescription: "Generated table draft" }); - expect(await repository.getColumn(database.id, patients.id, idColumn.id)).toMatchObject({ description: "Reviewed key", generatedDescription: "Generated key draft" }); + expect(await repository.getColumn(database.id, patients.id, idColumn.id)).toMatchObject({ + description: "Reviewed key", + generatedDescription: "Generated key draft", + sensitive: true, + }); }); test("consolidates non-empty generated table descriptions and reports skipped selections", async () => { diff --git a/backend/test/catalog-synthetic-sample-value.test.ts b/backend/test/catalog-synthetic-sample-value.test.ts new file mode 100644 index 00000000..f7587eff --- /dev/null +++ b/backend/test/catalog-synthetic-sample-value.test.ts @@ -0,0 +1,29 @@ +import { expect, test } from "vitest"; +import { syntheticSampleValue } from "../src/catalog/synthetic-sample-value.js"; + +test.each([ + ["text[]", "tags", 1, "{tags_001,tags_002}"], + ["json", "payload", 2, '{"example":"payload_002","sequence":2}'], + ["jsonb", "attributes", 3, '{"example":"attributes_003","sequence":3}'], + ["inet", "client_ip", 4, "192.0.2.4"], + ["cidr", "network", 5, "192.0.2.0/24"], + ["time without time zone", "opening_time", 6, "10:30:06"], + ["interval", "duration", 7, "7 days 07:00:00"], + ["bytea", "digest", 8, "\\x00000008"], +] as const)( + "creates a deterministic PostgreSQL-shaped value for %s", + (dataType, name, index, expected) => { + const column = { name, dataType }; + + expect(syntheticSampleValue(column, index)).toBe(expected); + expect(syntheticSampleValue(column, index)).toBe(expected); + }, +); + +test("uses the PostgreSQL type before birth-related name hints", () => { + expect(syntheticSampleValue({ name: "birth_year", dataType: "integer" }, 2)).toBe(1002); + expect(syntheticSampleValue({ name: "birth_date", dataType: "date" }, 2)) + .toBe("1982-01-02"); + expect(syntheticSampleValue({ name: "birth_recorded_at", dataType: "timestamp" }, 2)) + .toBe("2024-01-02T10:30:00.000Z"); +}); diff --git a/docs/adr/0011-gate-source-samples-with-a-sensitive-data-flag.md b/docs/adr/0011-gate-source-samples-with-a-sensitive-data-flag.md new file mode 100644 index 00000000..818719e8 --- /dev/null +++ b/docs/adr/0011-gate-source-samples-with-a-sensitive-data-flag.md @@ -0,0 +1,19 @@ +--- +status: accepted +--- + +# Gate source samples with a Sensitive Data Flag + +Each Catalog Column has one human-set `sensitive` boolean, defaulting to `false`. An AI may prefill +draft suggestions from structural metadata only, but the user decides and only the boolean is +persisted; there is no rationale, history, audit ledger, fingerprint, review state, or retroactive +regeneration of existing descriptions. + +Description generation extends ADR-0010 by sending bounded real values when `sensitive` is false +and deterministic plausible synthetic values when it is true, without identifying the synthetic +values to the model. The false default deliberately favors the expected stable schemas and the +minority of protected columns: a new column remains eligible for real sampling until a user marks +it sensitive. + +Applying the same flag to LSH value grounding is deferred until the current catalog tickets and +owner acceptance test are complete; `PROJECT_STATE.md` records that required follow-up gate. diff --git a/docs/operations/database-management.md b/docs/operations/database-management.md index 4b2e2614..df7695b3 100644 --- a/docs/operations/database-management.md +++ b/docs/operations/database-management.md @@ -48,9 +48,17 @@ the connection binding, or secrets. Deleting a table cascades to its columns and Generated descriptions can be requested for selected tables, selected columns, every eligible target, or targets with a missing generated description. The backend accepts one installation-wide -run and processes targets sequentially. It reads at most five source rows and five representative -non-null values per relevant source through a read-only connector, then sends that bounded sample -transiently to the configured model provider. +run and processes targets sequentially. Every catalog column has a **Sensitive** flag, which defaults +to `false`, including after a newly discovered column is synchronized. Before generation, an +administrator can ask the configured model to suggest flags from structural metadata only (database, +schema, table and column names, data types, nullability, primary keys, and foreign keys). Suggestions +remain an unsaved draft until a human reviews and saves them. + +For a column with `sensitive=false`, the worker may read at most five source rows and five +representative non-null values through a read-only connector. For `sensitive=true`, the source query +does not request that column's values; deterministic plausible values derived only from its name and +type take their place in the model prompt. The prompt does not identify those values as synthetic, so +the model can still describe the field as if it had received representative data. Each successful result is persisted immediately. Stop terminates the active helper but retains earlier results. A helper has at most one provider retry; three consecutively exhausted technical @@ -58,10 +66,12 @@ batches fail the run. Stale queued/running work is marked interrupted at startup unlocked only when no local worker/helper is live. There is no automatic resume and no public description-generation CLI. -Review generated text before copying it into the curated **Description** field. The sampling rule -is a deliberate data-disclosure boundary: do not use this facility for fields whose values must -not be sent to the configured provider until a Sensitive Data Policy is in place. +Review generated text before copying it into the curated **Description** field. Because the flag +defaults to `false`, an administrator must review the classification and mark protected fields before +starting generation. Changing a flag affects future generations only; existing generated or curated +descriptions are not regenerated. Real and substituted samples remain transient and are not persisted +or returned to the browser. -The decisions behind this surface are [ADRs 0001–0010](../adr/0001-postgres-metadata-catalog.md) +The decisions behind this surface are [ADRs 0001–0011](../adr/0001-postgres-metadata-catalog.md) and the detailed acceptance record is [AI catalog description generation acceptance](../testing/2026-08-29-ai-catalog-description-generation-acceptance.md). diff --git a/frontend/src/api/catalog-databases.ts b/frontend/src/api/catalog-databases.ts index 1024f95d..68e6f893 100644 --- a/frontend/src/api/catalog-databases.ts +++ b/frontend/src/api/catalog-databases.ts @@ -89,6 +89,7 @@ export interface CatalogColumn { sourceComment: string | null; description: string | null; generatedDescription: string | null; + sensitive: boolean; lastSyncedDatabaseVersion: number | null; lastSyncedAt: string | null; version: number; @@ -96,6 +97,15 @@ export interface CatalogColumn { updatedAt: string; } +export interface SensitiveDataSuggestion { + columnId: string; + sensitive: boolean; +} + +export interface SensitiveDataSuggestions { + suggestions: SensitiveDataSuggestion[]; +} + export interface CatalogRelationshipColumn { position: number; sourceColumnId: string; @@ -341,11 +351,18 @@ export const updateCatalogColumnMetadata = ( version: number, description: string | null, generatedDescription: string | null, + sensitive?: boolean, ) => apiFetch( `/catalog/databases/${encodeURIComponent(databaseId)}/tables/${encodeURIComponent(tableId)}/columns/${encodeURIComponent(columnId)}`, - { method: "PATCH", body: JSON.stringify({ version, description, generatedDescription }) }, + { method: "PATCH", body: JSON.stringify({ version, description, generatedDescription, sensitive }) }, ); +export const suggestSensitiveFields = (databaseId: string, modelId: string) => + apiFetch( + `/catalog/databases/${encodeURIComponent(databaseId)}/sensitive-data-suggestions`, + { method: "POST", body: JSON.stringify({ modelId }) }, + ); + export const listCatalogRelationships = (databaseId: string) => apiFetch(`/catalog/databases/${encodeURIComponent(databaseId)}/relationships`); diff --git a/frontend/src/shell/DatabaseManagementPage.test.tsx b/frontend/src/shell/DatabaseManagementPage.test.tsx index 85168777..68165082 100644 --- a/frontend/src/shell/DatabaseManagementPage.test.tsx +++ b/frontend/src/shell/DatabaseManagementPage.test.tsx @@ -286,10 +286,13 @@ test("discloses bounded transient source samples in database-wide and selected g name: "Metadata generation source data disclosure", }); expect(within(disclosure).getByText( - "Description generation may send up to five real source rows and up to five representative non-null example values to the selected model provider.", + "Description generation may send up to five source rows and up to five representative non-null example values to the selected model provider. Values from columns marked sensitive are replaced with plausible synthetic values before the request.", )).toBeVisible(); expect(within(disclosure).getByText( - "Samples are transient and are not stored in run logs or catalog metadata. Automated Sensitive Data Policy filtering and anonymization are not currently provided; they are planned for future work.", + "Values from unmarked columns may be sent unchanged. Samples are transient and are not stored in run logs or catalog metadata.", + )).toBeVisible(); + expect(within(disclosure).getByText( + "Sensitive-field suggestions use structural metadata only and remain unsaved until you choose Save sensitive fields.", )).toBeVisible(); await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" })); @@ -333,7 +336,7 @@ test.each(synchronizationScopes)( }); const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { name: synchronizationScopes[0].label })).toBeEnabled(); @@ -365,7 +368,7 @@ test("starts Generate Missing for one configured database without confirmation", renderPage({ rows: [makeDatabase()] }); const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate Missing" })); @@ -396,7 +399,7 @@ test("confirms Generate All replacement, supports cancel, and sends the database renderPage({ rows: [makeDatabase()] }); const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate All" })); @@ -434,7 +437,7 @@ test("keeps the database selected and safely explains when no descriptions are e renderPage({ rows: [makeDatabase()] }); const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate Missing" })); @@ -493,7 +496,7 @@ test.each([ ]) client.setQueryData(queryKey, []); const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: scope === "all" ? "Generate All" : "Generate Missing", @@ -568,7 +571,7 @@ test.each([ for (const database of rows.slice(0, selectedRows)) { const row = await screen.findByRole("row", { name: new RegExp(database.workspaceName) }); - await user.click(within(row).getByRole("checkbox")); + await user.click(within(row).getByRole("checkbox", { name: /toggle row selection/i })); } await user.click(screen.getByRole("button", { name: "Actions" })); @@ -599,13 +602,13 @@ test("disables database-wide generation while a description generation is active renderPage({ rows: [makeDatabase()] }); let databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate Missing" })); expect(await screen.findByRole("complementary", { name: "Description generation" })).toBeVisible(); databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { name: "Generate All" })) .toHaveAttribute("aria-disabled", "true"); @@ -630,8 +633,8 @@ test("selected database Actions confirms and deletes catalog tables for the full const firstRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); const secondRow = await screen.findByRole("row", { name: /Radiology/ }); - await user.click(within(firstRow).getByRole("checkbox")); - await user.click(within(secondRow).getByRole("checkbox")); + await user.click(within(firstRow).getByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(secondRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { name: "Delete all tables" })).toBeEnabled(); @@ -642,10 +645,10 @@ test("selected database Actions confirms and deletes catalog tables for the full expect(screen.getByText(/database configurations and source databases are unchanged/i)).toBeVisible(); expect(cleanupBody).toBeUndefined(); - await user.click(within(secondRow).getByRole("checkbox")); + await user.click(within(secondRow).getByRole("checkbox", { name: /toggle row selection/i })); expect(screen.queryByText(/Delete all catalog tables for/)).not.toBeInTheDocument(); expect(cleanupBody).toBeUndefined(); - await user.click(within(secondRow).getByRole("checkbox")); + await user.click(within(secondRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Delete all tables" })); await waitFor(() => expect(screen.getByRole("button", { name: "Delete catalog tables" })).toHaveFocus()); @@ -681,7 +684,7 @@ test("presents completed synchronization steps as success and skips unneeded con }); const databaseRow = await screen.findByRole("row", { name: /Policlinico San Donato/ }); - await user.click(within(databaseRow).getByRole("checkbox")); + await user.click(within(databaseRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Synchronize all columns" })); @@ -1047,6 +1050,7 @@ const patientIdColumn: CatalogColumn = { sourceComment: "Patient identifier", description: null, generatedDescription: null, + sensitive: false, lastSyncedDatabaseVersion: 3, lastSyncedAt: "2026-08-27T10:00:00Z", version: 1, @@ -1106,7 +1110,7 @@ test("selected table Actions exposes cleanup commands and sends synchronization await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" })); await user.click(screen.getByRole("tab", { name: "Tables" })); const tableRow = await screen.findByRole("row", { name: /patients/ }); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); const synchronizeColumns = await screen.findByRole("menuitem", { name: "Synchronize columns" }); @@ -1413,8 +1417,8 @@ test("moves selected generated table descriptions and reports copied and skipped await user.click(screen.getByRole("tab", { name: "Tables" })); const patientsRow = await screen.findByRole("row", { name: /patients/ }); const visitsRow = await screen.findByRole("row", { name: /visits/ }); - await user.click(within(patientsRow).getByRole("checkbox")); - await user.click(within(visitsRow).getByRole("checkbox")); + await user.click(within(patientsRow).getByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(visitsRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Move generated description to Description", @@ -1451,6 +1455,7 @@ test("selects columns, moves generated descriptions, and refreshes only the affe sourceComment: "Patient identifier", description: "Old identifier", generatedDescription: "Generated identifier", + sensitive: false, lastSyncedDatabaseVersion: 3, lastSyncedAt: "2026-08-27T10:00:00Z", version: 1, @@ -1491,7 +1496,7 @@ test("selects columns, moves generated descriptions, and refreshes only the affe const tableQuery = ["catalog-tables", patientsTable.databaseId] as const; const selectableRow = async (name: RegExp) => { const rows = await screen.findAllByRole("row", { name }); - const row = rows.find((candidate) => within(candidate).queryByRole("checkbox")); + const row = rows.find((candidate) => within(candidate).queryByRole("checkbox", { name: /toggle row selection/i })); expect(row).toBeDefined(); return row!; }; @@ -1501,8 +1506,8 @@ test("selects columns, moves generated descriptions, and refreshes only the affe await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const idRow = await selectableRow(/Patient identifier/); const nameRow = await selectableRow(/Keep curated patient name/); - await user.click(within(idRow).getByRole("checkbox")); - await user.click(within(nameRow).getByRole("checkbox")); + await user.click(within(idRow).getByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(nameRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Move generated description to Description", @@ -1549,9 +1554,9 @@ test("starts one selected column with the configured default model", async () => await user.click(screen.getByRole("tab", { name: "Tables" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); expect(columnRow).toBeDefined(); - await user.click(within(columnRow!).getByRole("checkbox")); + await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate description" })); @@ -1566,6 +1571,101 @@ test("starts one selected column with the configured default model", async () => expect(await screen.findByText("Description generation started for 1 column")).toBeVisible(); }); +test("reviews AI-sensitive-field suggestions as an editable draft and saves only changed columns", async () => { + const user = userEvent.setup(); + const idColumn = { ...patientIdColumn, sensitive: false }; + const nameColumn = { + ...patientIdColumn, + id: "eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee", + name: "name", + ordinalPosition: 2, + primaryKeyPosition: null, + isPrimaryKey: false, + description: "Patient name", + generatedDescription: "Name of the patient", + sensitive: false, + }; + let columns = [idColumn, nameColumn]; + let suggestionBody: unknown; + const patches: Array<{ columnId: string; body: unknown }> = []; + server.use( + http.get("/api/catalog/metadata-generation/models", () => HttpResponse.json({ + models: [{ id: "local-qwen", label: "Local Qwen" }], + default: "local-qwen", + })), + http.get("/api/catalog/databases/:databaseId/tables", () => HttpResponse.json([patientsTable])), + http.get( + "/api/catalog/databases/:databaseId/tables/:tableId/columns", + () => HttpResponse.json(columns), + ), + http.post( + "/api/catalog/databases/:databaseId/sensitive-data-suggestions", + async ({ request }) => { + suggestionBody = await request.json(); + return HttpResponse.json({ + suggestions: [ + { columnId: idColumn.id, sensitive: true }, + { columnId: nameColumn.id, sensitive: true }, + { columnId: "ffffffff-ffff-4fff-8fff-ffffffffffff", sensitive: true }, + ], + }); + }, + ), + http.patch( + "/api/catalog/databases/:databaseId/tables/:tableId/columns/:columnId", + async ({ params, request }) => { + const body = await request.json() as { + version: number; + description: string | null; + generatedDescription: string | null; + sensitive: boolean; + }; + patches.push({ columnId: String(params.columnId), body }); + const current = columns.find((column) => column.id === params.columnId)!; + const updated = { ...current, ...body, version: current.version + 1 }; + columns = columns.map((column) => column.id === updated.id ? updated : column); + return HttpResponse.json(updated); + }, + ), + ); + renderPage({ rows: [makeDatabase({ connectionStatus: "reachable", testedVersion: 3 })] }); + + await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" })); + await user.click(screen.getByRole("tab", { name: "Tables" })); + await user.click(await screen.findByRole("button", { name: "View columns for patients" })); + + const idSensitive = await screen.findByRole("checkbox", { name: "Sensitive data for id" }); + const nameSensitive = await screen.findByRole("checkbox", { name: "Sensitive data for name" }); + expect(idSensitive).not.toBeChecked(); + expect(nameSensitive).not.toBeChecked(); + + await user.click(screen.getByRole("button", { name: "Suggest sensitive fields" })); + + await waitFor(() => expect(suggestionBody).toEqual({ modelId: "local-qwen" })); + await waitFor(() => { + expect(screen.getByRole("checkbox", { name: "Sensitive data for id" })).toBeChecked(); + expect(screen.getByRole("checkbox", { name: "Sensitive data for name" })).toBeChecked(); + }); + expect(patches).toHaveLength(0); + + await user.click(screen.getByRole("checkbox", { name: "Sensitive data for name" })); + expect(screen.getByRole("checkbox", { name: "Sensitive data for name" })).not.toBeChecked(); + await user.click(screen.getByRole("button", { name: "Save sensitive fields" })); + + await waitFor(() => expect(patches).toEqual([{ + columnId: idColumn.id, + body: { + version: idColumn.version, + description: idColumn.description, + generatedDescription: idColumn.generatedDescription, + sensitive: true, + }, + }])); + expect(await screen.findByText("Sensitive fields saved")).toBeVisible(); + expect(screen.getByRole("checkbox", { name: "Sensitive data for id" })).toBeChecked(); + expect(screen.getByRole("checkbox", { name: "Sensitive data for name" })).not.toBeChecked(); +}); + test("polls a description run and renders its events in sequence order", async () => { const user = userEvent.setup(); const queuedRun = makeDescriptionGenerationRun(); @@ -1617,8 +1717,8 @@ test("polls a description run and renders its events in sequence order", async ( await user.click(screen.getByRole("tab", { name: "Tables" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); - await user.click(within(columnRow!).getByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate description" })); @@ -1699,8 +1799,8 @@ test("refreshes Catalog Tables and Catalog Columns after column generation compl await user.click(screen.getByRole("tab", { name: "Tables" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); - await user.click(within(columnRow!).getByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate description" })); @@ -1708,7 +1808,7 @@ test("refreshes Catalog Tables and Catalog Columns after column generation compl await waitFor(() => expect(columnReads).toBe(2)); await waitFor(() => expect(tableReads).toBe(2)); expect(within((await screen.findAllByRole("row", { name: /Stable patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox"))!).getByText("Stable patient identifier")).toBeVisible(); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i }))!).getByText("Stable patient identifier")).toBeVisible(); expect(client.getQueryState(unrelatedColumnsQuery)?.isInvalidated).toBe(true); }); @@ -1732,8 +1832,8 @@ test("keeps the selected column and shows a safe message when generation cannot await user.click(screen.getByRole("tab", { name: "Tables" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); - await user.click(within(columnRow!).getByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate description" })); @@ -1799,8 +1899,8 @@ test("shows a basic failed run without exposing private model or provider fields await user.click(screen.getByRole("tab", { name: "Tables" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); - await user.click(within(columnRow!).getByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate description" })); @@ -1851,10 +1951,10 @@ test("offers generation and consolidation for multiple selected columns", async await user.click(screen.getByRole("tab", { name: "Tables" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const idRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); const nameRow = (await screen.findAllByRole("row", { name: /Patient name/ })) - .find((row) => within(row).queryByRole("checkbox")); - await user.click(within(idRow!).getByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(idRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { name: "Generate description" })).toBeEnabled(); expect(screen.getByRole("menuitem", { @@ -1862,7 +1962,7 @@ test("offers generation and consolidation for multiple selected columns", async })).toBeEnabled(); await user.keyboard("{Escape}"); - await user.click(within(nameRow!).getByRole("checkbox")); + await user.click(within(nameRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); const generate = await screen.findByRole("menuitem", { name: "Generate descriptions" }); expect(generate).toBeEnabled(); @@ -1901,7 +2001,7 @@ test.each([ await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" })); await user.click(screen.getByRole("tab", { name: "Tables" })); const tableRow = await screen.findByRole("row", { name: /patients/ }); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { name: "Generate description" })) .toHaveAttribute("aria-disabled", "true"); @@ -1909,8 +2009,8 @@ test.each([ await user.click(screen.getByRole("button", { name: "Clear" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); - await user.click(within(columnRow!).getByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { name: "Generate description" })) @@ -1943,8 +2043,8 @@ test("disables another generation start while the selected-column run is active" await user.click(screen.getByRole("tab", { name: "Tables" })); await user.click(await screen.findByRole("button", { name: "View columns for patients" })); const columnRow = (await screen.findAllByRole("row", { name: /Patient identifier/ })) - .find((row) => within(row).queryByRole("checkbox")); - await user.click(within(columnRow!).getByRole("checkbox")); + .find((row) => within(row).queryByRole("checkbox", { name: /toggle row selection/i })); + await user.click(within(columnRow!).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Generate description" })); expect(await screen.findByRole("complementary", { name: "Description generation" })).toBeVisible(); @@ -1956,7 +2056,7 @@ test("disables another generation start while the selected-column run is active" await user.click(screen.getByRole("button", { name: "Back to tables" })); const tableRow = await screen.findByRole("row", { name: /patients/ }); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { name: "Generate description" })) .toHaveAttribute("aria-disabled", "true"); @@ -1982,7 +2082,7 @@ test("disables selected description consolidation without database.manage", asyn await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" })); await user.click(screen.getByRole("tab", { name: "Tables" })); const tableRow = await screen.findByRole("row", { name: /patients/ }); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); expect(await screen.findByRole("menuitem", { @@ -2011,7 +2111,7 @@ test("shows an operation conflict without clearing selected descriptions or refr await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" })); await user.click(screen.getByRole("tab", { name: "Tables" })); const tableRow = await screen.findByRole("row", { name: /patients/ }); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Move generated description to Description", @@ -2045,7 +2145,7 @@ test("selected table Actions confirms and deletes incoming and outgoing relation await user.click(await screen.findByRole("button", { name: "View Policlinico San Donato" })); await user.click(screen.getByRole("tab", { name: "Tables" })); const tableRow = await screen.findByRole("row", { name: /patients/ }); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Delete all relationships" })); @@ -2053,10 +2153,10 @@ test("selected table Actions confirms and deletes incoming and outgoing relation expect(screen.getByText(/incoming and outgoing relationships/i)).toBeVisible(); expect(cleanupBody).toBeUndefined(); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); expect(screen.queryByText("Delete all relationships for 1 table?")).not.toBeInTheDocument(); expect(cleanupBody).toBeUndefined(); - await user.click(within(tableRow).getByRole("checkbox")); + await user.click(within(tableRow).getByRole("checkbox", { name: /toggle row selection/i })); await user.click(screen.getByRole("button", { name: "Actions" })); await user.click(await screen.findByRole("menuitem", { name: "Delete all relationships" })); await user.click(screen.getByRole("button", { name: "Delete relationships" })); @@ -2133,6 +2233,7 @@ test("navigates from a table to columns and from the database to physical relati sourceComment: "Patient identifier", description: null, generatedDescription: null, + sensitive: false, lastSyncedDatabaseVersion: 3, lastSyncedAt: "2026-08-27T10:00:00Z", version: 1, diff --git a/frontend/src/shell/database-management/DatabaseColumns.tsx b/frontend/src/shell/database-management/DatabaseColumns.tsx index 3d5d2c51..6d335e9d 100644 --- a/frontend/src/shell/database-management/DatabaseColumns.tsx +++ b/frontend/src/shell/database-management/DatabaseColumns.tsx @@ -3,7 +3,7 @@ import { Menu } from "@base-ui/react/menu"; import { useQuery, useQueryClient } from "@tanstack/react-query"; import { AgGridReact } from "ag-grid-react"; import type { ColDef, ICellRendererParams } from "ag-grid-community"; -import { ChevronDown, KeyRound, Link2, Pencil, RefreshCw, Save, X } from "lucide-react"; +import { ChevronDown, KeyRound, Link2, Pencil, RefreshCw, Save, Sparkles, X } from "lucide-react"; import { toast } from "sonner"; import { Button } from "../../components/ui/button"; import { ApiError, apiErrorMessage } from "../../api/client"; @@ -11,6 +11,7 @@ import { consolidateCatalogDescriptions, listCatalogColumns, startDescriptionGenerationRun, + suggestSensitiveFields, updateCatalogColumnMetadata, type CatalogColumn, type CatalogTable, @@ -31,7 +32,9 @@ interface Props { interface GridContext { canManage: boolean; + busy: boolean; onEdit: (column: CatalogColumn, origin: HTMLButtonElement) => void; + onSensitiveChange: (column: CatalogColumn, sensitive: boolean) => void; } function KeyCell({ data }: ICellRendererParams) { @@ -55,6 +58,23 @@ function ActionCell({ data, context }: ICellRendererParams) { + if (!data || !context) return null; + return ( +
event.stopPropagation()}> + context.onSensitiveChange(data, event.target.checked)} + onClick={(event) => event.stopPropagation()} + /> +
+ ); +} + export function DatabaseColumns({ databaseId, table, @@ -77,6 +97,8 @@ export function DatabaseColumns({ const [editingId, setEditingId] = useState(null); const [description, setDescription] = useState(""); const [generatedDescription, setGeneratedDescription] = useState(""); + const [sensitiveDrafts, setSensitiveDrafts] = useState>({}); + const [sensitiveAction, setSensitiveAction] = useState<"suggest" | "save" | null>(null); const [baseline, setBaseline] = useState(""); const [version, setVersion] = useState(null); const [stale, setStale] = useState(false); @@ -86,7 +108,17 @@ export function DatabaseColumns({ const originRef = useRef(null); const active = editingId ? data.find((column) => column.id === editingId) : undefined; const fingerprint = JSON.stringify([description, generatedDescription]); - const dirty = Boolean(editingId && fingerprint !== baseline); + const editorDirty = Boolean(editingId && fingerprint !== baseline); + const changedSensitiveColumns = data.filter((column) => ( + Object.hasOwn(sensitiveDrafts, column.id) + && sensitiveDrafts[column.id] !== column.sensitive + )); + const dirty = editorDirty || changedSensitiveColumns.length > 0; + const displayedColumns = useMemo(() => data.map((column) => ( + Object.hasOwn(sensitiveDrafts, column.id) + ? { ...column, sensitive: sensitiveDrafts[column.id]! } + : column + )), [data, sensitiveDrafts]); useEffect(() => { onNavigationStateChange({ dirty, busy }); }, [busy, dirty, onNavigationStateChange]); useEffect(() => { @@ -107,7 +139,7 @@ export function DatabaseColumns({ }; const closeEditor = () => { if (busy) return; - if (dirty && !window.confirm("Discard unsaved column metadata?")) return; + if (editorDirty && !window.confirm("Discard unsaved column metadata?")) return; setEditingId(null); window.setTimeout(() => originRef.current?.focus(), 0); }; @@ -178,8 +210,74 @@ export function DatabaseColumns({ } finally { setBusy(false); } }; + const changeSensitive = (column: CatalogColumn, sensitive: boolean) => { + const persisted = data.find((candidate) => candidate.id === column.id); + if (!persisted) return; + setSensitiveDrafts((current) => { + const next = { ...current }; + if (sensitive === persisted.sensitive) delete next[column.id]; + else next[column.id] = sensitive; + return next; + }); + }; + + const suggestSensitive = async () => { + if (!selectedMetadataModel) return; + setBusy(true); + setSensitiveAction("suggest"); + try { + const result = await suggestSensitiveFields(databaseId, selectedMetadataModel); + const currentById = new Map(data.map((column) => [column.id, column])); + const next: Record = {}; + for (const suggestion of result.suggestions) { + const column = currentById.get(suggestion.columnId); + if (column && suggestion.sensitive !== column.sensitive) { + next[column.id] = suggestion.sensitive; + } + } + setSensitiveDrafts(next); + toast.success("Sensitive field suggestions ready for review"); + } catch (error) { + toast.error(apiErrorMessage(error)); + } finally { + setSensitiveAction(null); + setBusy(false); + } + }; + + const saveSensitive = async () => { + if (changedSensitiveColumns.length === 0) return; + setBusy(true); + setSensitiveAction("save"); + try { + const updated = await Promise.all(changedSensitiveColumns.map((column) => ( + updateCatalogColumnMetadata( + databaseId, + table.id, + column.id, + column.version, + column.description, + column.generatedDescription, + sensitiveDrafts[column.id], + ) + ))); + const updatedById = new Map(updated.map((column) => [column.id, column])); + queryClient.setQueryData(queryKey, (current = []) => current.map( + (column) => updatedById.get(column.id) ?? column, + )); + setSensitiveDrafts({}); + toast.success("Sensitive fields saved"); + } catch (error) { + toast.error(apiErrorMessage(error)); + } finally { + setSensitiveAction(null); + setBusy(false); + } + }; + const columns = useMemo[]>(() => [ { field: "ordinalPosition", headerName: "#", width: 64, maxWidth: 64, filter: "agNumberColumnFilter" }, + { field: "sensitive", headerName: "Sensitive", minWidth: 110, width: 110, sortable: false, filter: false, resizable: false, cellRenderer: SensitiveCell }, { field: "name", headerName: "Name", minWidth: 190, flex: 1, cellClass: "font-mono text-xs" }, { field: "dataType", headerName: "Type", minWidth: 150, flex: 0.8, cellClass: "font-mono text-xs" }, { headerName: "Keys", minWidth: 125, width: 125, sortable: false, filter: false, cellRenderer: KeyCell }, @@ -189,7 +287,12 @@ export function DatabaseColumns({ { field: "description", headerName: "Description", minWidth: 230, flex: 1.2, valueFormatter: ({ value }) => value ?? "" }, { colId: "actions", headerName: "", width: 64, maxWidth: 64, pinned: "right", sortable: false, filter: false, resizable: false, cellRenderer: ActionCell }, ], []); - const context = useMemo(() => ({ canManage, onEdit: edit }), [canManage, data]); + const context = useMemo(() => ({ + canManage, + busy, + onEdit: edit, + onSensitiveChange: changeSensitive, + }), [busy, canManage, data]); if (editingId && active) { return ( @@ -213,7 +316,7 @@ export function DatabaseColumns({
- +
); @@ -255,6 +358,19 @@ export function DatabaseColumns({ Catalog columns setSearch(event.target.value)} /> {data.length} + + {changedSensitiveColumns.length > 0 ? ( + + ) : null} @@ -264,7 +380,7 @@ export function DatabaseColumns({
ref={gridRef} - rowData={data} + rowData={displayedColumns} columnDefs={columns} context={context} loading={isLoading} diff --git a/frontend/src/shell/database-management/MetadataGenerationModelSelector.tsx b/frontend/src/shell/database-management/MetadataGenerationModelSelector.tsx index bb59cc46..f8186400 100644 --- a/frontend/src/shell/database-management/MetadataGenerationModelSelector.tsx +++ b/frontend/src/shell/database-management/MetadataGenerationModelSelector.tsx @@ -61,13 +61,17 @@ export function MetadataGenerationModelSelector({ className="space-y-1 text-xs font-normal leading-4 text-muted-foreground" >

- Description generation may send up to five real source rows and up to five representative - non-null example values to the selected model provider. + Description generation may send up to five source rows and up to five representative + non-null example values to the selected model provider. Values from columns marked + sensitive are replaced with plausible synthetic values before the request.

- Samples are transient and are not stored in run logs or catalog metadata. Automated - Sensitive Data Policy filtering and anonymization are not currently provided; they are - planned for future work. + Values from unmarked columns may be sent unchanged. Samples are transient and are not + stored in run logs or catalog metadata. +

+

+ Sensitive-field suggestions use structural metadata only and remain unsaved until you + choose Save sensitive fields.

diff --git a/mkdocs.yml b/mkdocs.yml index eb3108c8..3fcb9367 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -84,6 +84,7 @@ nav: - 0008 Manual catalog cleanup: adr/0008-allow-manual-catalog-metadata-cleanup.md - 0009 Sequential description generation: adr/0009-use-one-sequential-description-generation-run.md - 0010 Bounded source samples: adr/0010-allow-bounded-real-source-samples-for-description-generation.md + - 0011 Sensitive data flag: adr/0011-gate-source-samples-with-a-sensitive-data-flag.md - AI catalog description acceptance: testing/2026-08-29-ai-catalog-description-generation-acceptance.md - Design records: - Metadata catalog design: plans/2026-08-26-metadata-catalog-from-thothai.md