mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: protect sensitive catalog samples

Browse Source
This commit is contained in:
Codex
2026-08-30 12:14:23 +02:00
parent 6278ee9d81
commit 0736983bc5
28 changed files with 1162 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/adr/0011-gate-source-samples-with-a-sensitive-data-flag.md
+19
View File
@@ -0,0 +1,19 @@
---
status: accepted
---
# Gate source samples with a Sensitive Data Flag
Each Catalog Column has one human-set `sensitive` boolean, defaulting to `false`. An AI may prefill
draft suggestions from structural metadata only, but the user decides and only the boolean is
persisted; there is no rationale, history, audit ledger, fingerprint, review state, or retroactive
regeneration of existing descriptions.
Description generation extends ADR-0010 by sending bounded real values when `sensitive` is false
and deterministic plausible synthetic values when it is true, without identifying the synthetic
values to the model. The false default deliberately favors the expected stable schemas and the
minority of protected columns: a new column remains eligible for real sampling until a user marks
it sensitive.
Applying the same flag to LSH value grounding is deferred until the current catalog tickets and
owner acceptance test are complete; `PROJECT_STATE.md` records that required follow-up gate.
docs/operations/database-management.md
+17 -7
View File
@@ -48,9 +48,17 @@ the connection binding, or secrets. Deleting a table cascades to its columns and
Generated descriptions can be requested for selected tables, selected columns, every eligible
target, or targets with a missing generated description. The backend accepts one installation-wide
run and processes targets sequentially. It reads at most five source rows and five representative
non-null values per relevant source through a read-only connector, then sends that bounded sample
transiently to the configured model provider.
run and processes targets sequentially. Every catalog column has a **Sensitive** flag, which defaults
to `false`, including after a newly discovered column is synchronized. Before generation, an
administrator can ask the configured model to suggest flags from structural metadata only (database,
schema, table and column names, data types, nullability, primary keys, and foreign keys). Suggestions
remain an unsaved draft until a human reviews and saves them.
For a column with `sensitive=false`, the worker may read at most five source rows and five
representative non-null values through a read-only connector. For `sensitive=true`, the source query
does not request that column's values; deterministic plausible values derived only from its name and
type take their place in the model prompt. The prompt does not identify those values as synthetic, so
the model can still describe the field as if it had received representative data.
Each successful result is persisted immediately. Stop terminates the active helper but retains
earlier results. A helper has at most one provider retry; three consecutively exhausted technical
@@ -58,10 +66,12 @@ batches fail the run. Stale queued/running work is marked interrupted at startup
unlocked only when no local worker/helper is live. There is no automatic resume and no public
description-generation CLI.
Review generated text before copying it into the curated **Description** field. The sampling rule
is a deliberate data-disclosure boundary: do not use this facility for fields whose values must
not be sent to the configured provider until a Sensitive Data Policy is in place.
Review generated text before copying it into the curated **Description** field. Because the flag
defaults to `false`, an administrator must review the classification and mark protected fields before
starting generation. Changing a flag affects future generations only; existing generated or curated
descriptions are not regenerated. Real and substituted samples remain transient and are not persisted
or returned to the browser.
The decisions behind this surface are [ADRs 0001–0010](../adr/0001-postgres-metadata-catalog.md)
The decisions behind this surface are [ADRs 0001–0011](../adr/0001-postgres-metadata-catalog.md)
and the detailed acceptance record is
[AI catalog description generation acceptance](../testing/2026-08-29-ai-catalog-description-generation-acceptance.md).