feat: sample sensitive columns progressively

This commit is contained in:
Codex
2026-09-03 10:25:05 +02:00
parent f114d0065a
commit 8e778b9edb
24 changed files with 1001 additions and 581 deletions
+9 -8
View File
@@ -107,16 +107,17 @@ explicitly unlocked; it never resumes automatically.
Sensitivity analysis is a synchronous administrative request and does not use the installation
model catalog. Database-specific adapters stream bounded normalized values from read-only source
connections; the TypeScript `SensitivityClassifier` is the single decision point for
`sensitive | non_sensitive | unknown`. Deterministic rules run first. A complete scan is attempted
for at most five seconds per table, then the adapter samples within the sixty-second request budget.
An optional offline GLiNER2 worker may add NER evidence on CPU for unresolved short text, but it
cannot make or persist the decision itself.
`sensitive | non_sensitive`. Deterministic rules run first. Tables up to 1,000 rows are fully
scanned; larger tables use breadth-first targets of 300, 1,000, and 3,000 values, with the last pass
limited to text-like columns. Source queries have five-second limits, but the request has no global
analysis deadline. An optional offline GLiNER2 worker may add NER evidence on CPU for unresolved
short text, but it cannot make or persist the decision itself.
Each attempt has its own durable run and ordered sanitized events, separate from Description
Generation because its lifecycle and counters differ. The run records the local policy version,
coverage aggregates, and sanitized rule identifiers. Proposed flags, source values, NER spans, and
worker diagnostics remain transient. Only an explicit administrator save changes the human-owned
Sensitive Data Flag.
Generation because its lifecycle and counters differ. The run records the local policy version and
aggregate decision counts. Coverage, rule identifiers, proposed flags, source values, NER spans,
and worker diagnostics remain transient. Only an explicit administrator save changes the
human-owned Sensitive Data Flag.
## Main backend classes