ThothII/docs/adr/0011-gate-source-samples-with-a-sensitive-data-flag.md

status

status
accepted

Gate source samples with a Sensitive Data Flag

Each Catalog Column has one human-set sensitive boolean, defaulting to false. An AI may prefill draft suggestions from structural metadata only, but the user decides and the Catalog Column persists only the current boolean. It stores no rationale, prior flag values, proposal fingerprint, review state, or audit ledger, and changing the flag does not retroactively regenerate existing descriptions.

The user starts a suggestion from an explicit selection at database, table, or column level. A database request accepts exactly one selected database; a table or column request contains only the selected tables' columns or the selected columns, respectively. The backend divides that structural metadata into deterministic model requests of at most ten columns, also bounded by helper message size, and returns one combined draft for human review. No flag changes until the user saves the reviewed draft.

Each started suggestion attempt persists a separate Sensitive Data Suggestion Run containing its database, scope, selected model, lifecycle status, aggregate suggestion counts, timestamps, sanitized error summary, and ordered sanitized events. It does not persist selected target IDs, per-column proposals, prompts, raw provider output, or provider diagnostics. This is operational history of the AI attempt, not an audit trail of human flag decisions; reloading discards an unsaved proposal.

Description generation extends ADR-0010 by sending bounded real values when sensitive is false and deterministic plausible synthetic values when it is true, without identifying the synthetic values to the model. The false default deliberately favors the expected stable schemas and the minority of protected columns: a new column remains eligible for real sampling until a user marks it sensitive.

Applying the same flag to LSH value grounding is deferred until the current catalog tickets and owner acceptance test are complete; PROJECT_STATE.md records that required follow-up gate.