mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: classify sensitive columns locally

Browse Source
This commit is contained in:
Codex
2026-09-03 02:11:13 +02:00
parent 7b87e95427
commit f114d0065a
57 changed files with 4038 additions and 1149 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/database-management.md
+10 -7
View File
@@ -137,14 +137,17 @@ Generated descriptions can be requested for selected tables, selected columns, e
target, or targets with a missing generated description. The backend accepts one installation-wide
run and processes targets sequentially. Every catalog column has a **Sensitive** flag, which defaults
to `false`, including after a newly discovered column is synchronized. Before generation, an
administrator can ask the configured model to suggest flags from structural metadata only (database,
schema, table and column names, data types, nullability, primary keys, and foreign keys). Suggestions
remain an unsaved draft until a human reviews and saves them.
administrator can run local sensitivity analysis over the selected database, tables, or columns.
The analysis combines structural metadata with bounded read-only inspection of source values. It
uses no generative AI and no installation-catalog model. Assessments remain an unsaved draft until
a human reviews and saves them; the reviewer may reverse any proposal.
The page exposes separate histories for description generation and sensitive-field suggestions.
Sensitive-suggestion history stores the selected model, scope, status, aggregate counts, timestamps,
and sanitized events. It does not store the proposed per-column flags, prompts, raw model output, or
provider diagnostics; closing an unsaved review still discards that draft.
The page exposes separate histories for description generation and sensitivity analysis. Analysis
history stores the local policy version, scope, status, aggregate `sensitive`, `non_sensitive`, and
`unknown` counts, timestamps, and sanitized events. It does not store source values, per-column
proposals, NER spans, or worker diagnostics; closing an unsaved review discards that draft. The
rules, time bounds, and optional CPU-only NER profile are documented in
[Local sensitivity analysis](sensitivity-analysis.md).
For a column with `sensitive=false`, the worker may read at most five source rows and five
representative non-null values through a read-only connector. For `sensitive=true`, the source query