mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: protect sensitive catalog samples

Browse Source
This commit is contained in:
Codex
2026-08-30 12:14:23 +02:00
parent 6278ee9d81
commit 0736983bc5
28 changed files with 1162 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PROJECT_STATE.md
+15 -6
View File
@@ -104,16 +104,18 @@ one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before loca
sessions still consume the existing workspace configuration in this slice: database-management
records do not yet change the NL→SQL handoff. The accepted design is recorded in
`docs/plans/2026-08-26-metadata-catalog-from-thothai.md`, the snapshot contract under
`docs/contracts/`, and ADRs 0001–0010.
`docs/contracts/`, and ADRs 0001–0011.
Semantic aliases, value descriptions, synonyms, concepts, and logical relationships remain
deferred to their dedicated slices.
AI Description Generation preserves ThothAI's use of real source samples: up to five source rows
and five representative non-null example values may be sent transiently to the configured model
provider. The UI and operator documentation disclose this behavior. A required follow-up
improvement is a Sensitive Data Policy that classifies protected fields and excludes or anonymizes
their values before model calls.
AI Description Generation uses the catalog's human-owned Sensitive Data Flag. The flag defaults to
`false`, including for newly synchronized columns. An administrator may request an AI proposal based
only on structural metadata, but it remains an unsaved draft until the human reviews and saves it.
For unprotected columns, up to five source rows and five representative non-null values may be sent
transiently to the configured model provider. Protected columns are omitted from source reads and
replaced in the prompt by deterministic plausible values derived only from their metadata. Existing
descriptions are not regenerated when a flag changes.
The accepted AI-description design is recorded in
`docs/plans/2026-08-28-ai-catalog-description-generation.md`, with the formal specification in the
@@ -141,6 +143,13 @@ point the next required design gate is to compare the catalog snapshot with the
preprocessing/schema-linking contracts and plan the cutover; this follow-up must not be treated as
optional cleanup or silently omitted.
**Deferred follow-up — Sensitive Data Policy in schema-linking.** The policy is first delivered
and tested in catalog description generation. Its enforcement for core schema-linking remains
out of scope until the current tickets are closed and the owner has completed the acceptance test.
At that gate, resume the design: `tht` must receive a read-only projection of the current Sensitive
Data Flags and exclude values from columns marked sensitive from every LSH result before it is
given to Pi. Do not start this integration before the owner gives final approval after that test.
## Active deployment work and manual gates
### PSD server deployment program