docs: reconcile catalog run history and fleet state

2026-08-31 15:59:26 +02:00
parent ded66fde9c
commit b4b97436e1
7 changed files with 106 additions and 10 deletions
@@ -378,6 +378,14 @@ esplicitamente selezionate; le richieste ampie vengono divise in batch bounded,
 l'utente imposta i flag dopo aver rivisto la proposta completa.
 _Avoid_: PII filter, sample filter

+**Sensitive Data Suggestion Run** — Il tentativo amministrativo tracciato con cui il modello
+propone Sensitive Data Flag dai soli metadati strutturali. Conserva stato e conteggi aggregati,
+ma non i suggerimenti per colonna, che restano una proposta transitoria fino al salvataggio umano.
+
+**Sensitive Data Suggestion Event** — Una riga testuale ordinata e sanitizzata che registra
+l'avvio, l'esito o l'errore di una Sensitive Data Suggestion Run senza conservare prompt,
+risposte grezze del provider o proposte per colonna.
+
 **Introspection Capability** — Una categoria di struttura fisica che una Database Binding
 può osservare, come tabelle, colonne, relazioni, indici o enum. Una capability non disponibile
 è distinta da una capability osservata che non ha restituito elementi.
@@ -66,13 +66,21 @@ Git and are supplied only through installation-local protected files.
 ## Database management

 The database, table, and authoritative physical-schema catalog slices are implemented. Database
-management opens a responsive AG Grid master-detail surface, lists every YAML workspace, creates
-at most one PostgreSQL database configuration per workspace, edits direct PostgreSQL, REST API, or
-SSH-tunnel installation bindings, replaces write-only encrypted secrets, and tests supported
-connector bindings.
+Management now opens the Fleet Ledger presentation by default inside `AppShell`, lists every YAML
+workspace, creates at most one PostgreSQL database configuration per workspace, edits direct
+PostgreSQL, REST API, or SSH-tunnel installation bindings, replaces write-only encrypted secrets,
+and tests supported connector bindings. The surface keeps one responsive AG Grid visible at a time:
+databases lead to tables, tables lead to columns, and relationships are a sibling database view.
+Parent navigation remains explicit through the breadcrumb and emphasized back control.

-Configured databases use pure hierarchical navigation through `Overview`, `Tables`, and
-`Relationships`; a selected table has `Overview` and `Columns`. Physical membership, source
+Selection-scoped operations use one action selector plus an explicit **Run** control; ineligible
+actions remain visible with their disabled reason, while row-scoped actions stay in the pinned final
+column. The KPI strip reads installation-wide or selected-database aggregates from
+`GET /catalog/metrics`. Database configuration, metadata editors, synchronization history,
+description history, and sensitive-field review/history use the production APIs in right-side
+drawers rather than prototype fixtures; closing a history drawer does not stop its background run.
+
+Physical membership, source
 comments, column types/default/nullability/PK positions, and constraint-level ordered FK pairs are
 projections of the external schema. They cannot be created, renamed, or structurally edited by
 hand, but administrators can explicitly clear catalog tables, columns, or relationships without
@@ -82,6 +90,12 @@ relationships. Curated and generated descriptions are editable; generated descri
 and Database Management can generate or consolidate them for selected tables, selected columns,
 all targets, or only targets whose Generated Description is missing.

+The previous Database Management renderer remains a temporary comparison fallback for development
+and staging only: `?db-ui=legacy` is honored in Vite development or when
+`VITE_DB_MANAGEMENT_LEGACY=true`; it is not a production presentation. The standalone Fleet Ledger
+prototype on port `5173` also remains temporary until owner acceptance of the integrated surface,
+after which both migration aids can be removed.
+
 Schema refresh is one durable asynchronous engine with database-table, database-column,
 selected-table-column, relationship, and full-database actions. Database-level menus expose the
 table, all-column, relationship, and full scopes separately; selecting tables exposes column
@@ -115,6 +129,10 @@ only on structural metadata for one selected database, selected tables, or selec
 backend divides large scopes into deterministic model requests of at most ten columns, also bounded
 by helper message size, and combines their results, but the proposal remains an unsaved draft until
 the human reviews and saves it.
+Each started suggestion attempt records a separate Sensitive Data Suggestion Run with aggregate
+counters and safe ordered events. This operational history never stores per-column proposals,
+prompts, raw model output, or provider diagnostics; reloading still discards an unsaved review
+draft.
 For unprotected columns, up to five source rows and five representative non-null values may be sent
 transiently to the configured model provider. Protected columns are omitted from source reads and
 replaced in the prompt by deterministic plausible values derived only from their metadata. Existing
@@ -5,9 +5,10 @@ status: accepted
 # Gate source samples with a Sensitive Data Flag

 Each Catalog Column has one human-set `sensitive` boolean, defaulting to `false`. An AI may prefill
-draft suggestions from structural metadata only, but the user decides and only the boolean is
-persisted; there is no rationale, history, audit ledger, fingerprint, review state, or retroactive
-regeneration of existing descriptions.
+draft suggestions from structural metadata only, but the user decides and the Catalog Column
+persists only the current boolean. It stores no rationale, prior flag values, proposal fingerprint,
+review state, or audit ledger, and changing the flag does not retroactively regenerate existing
+descriptions.

 The user starts a suggestion from an explicit selection at database, table, or column level. A
 database request accepts exactly one selected database; a table or column request contains only the
@@ -16,6 +17,13 @@ metadata into deterministic model requests of at most ten columns, also bounded
 size, and returns one combined draft for human review. No flag changes until the user saves the
 reviewed draft.

+Each started suggestion attempt persists a separate Sensitive Data Suggestion Run containing its
+database, scope, selected model, lifecycle status, aggregate suggestion counts, timestamps,
+sanitized error summary, and ordered sanitized events. It does not persist selected target IDs,
+per-column proposals, prompts, raw provider output, or provider diagnostics. This is operational
+history of the AI attempt, not an audit trail of human flag decisions; reloading discards an unsaved
+proposal.
+
 Description generation extends ADR-0010 by sending bounded real values when `sensitive` is false
 and deterministic plausible synthetic values when it is true, without identifying the synthetic
 values to the model. The false default deliberately favors the expected stable schemas and the
@@ -5,3 +5,6 @@ This is a single-context repository.
 Before exploring, read `CONTEXT.md` at the repository root and the relevant decisions in
 `docs/adr/`. Use the project's terminology from `CONTEXT.md` in issue titles, proposals, and
 tests. Surface conflicts with an ADR instead of silently overriding it.
+
+When the vocabulary is missing or ambiguous, clarify the term with the `domain-modeling` skill and
+record the result in `CONTEXT.md` or an ADR before relying on it in implementation work.
@@ -9,7 +9,10 @@ Issues and specs for this repository live in the self-hosted Gitea repository:
  authenticated token with issue scope is available.
 - **Read and list issues**: use the Gitea web UI or authenticated API; include labels and comments.
 - **Apply or remove labels**: use the issue's label controls or the Gitea API.
- **Comment and close**: use the issue page or the Gitea API.
+- **Comment and close**: record the resolution on the issue before closing it, using the issue page
+  or the Gitea API.
+- In durable repository documents, cite an issue with its title and canonical link rather than only
+  its number.
 - Do not use `gh issue ...` for this repository: the `github` remote is a mirror, not the canonical
  issue tracker.

@@ -67,6 +67,26 @@ sequenceDiagram

 The backend uses `ThtRunner` for CLI subprocesses, `PiProcessManager` for one Pi process per session, `SessionBridge` to adapt RPC events, and `SseHub` to distribute them to clients.

+## Database Management frontend
+
+`AppShell` mounts Fleet Ledger as the default Database Management presentation. The controller keeps
+the existing React Query, AG Grid, permission, dirty-state, synchronization, SSE, and polling
+contracts; Fleet Ledger changes the information architecture without introducing a second catalog
+client. It shows exactly one grid at a time along the database → table → column hierarchy, with
+relationships as a sibling database view. An emphasized back control and breadcrumb move to the
+parent view.
+
+Selected-row operations are exposed through a single action selector and explicit **Run** control.
+Row-specific actions remain icon controls in the pinned final column. Configuration, metadata
+editing, synchronization, description generation, and sensitive-field review/history use the real
+catalog state and open in right-side drawers. A drawer can close independently of a durable run.
+
+The KPI strip calls `GET /catalog/metrics`: omitting `databaseId` returns installation-wide catalog
+aggregates, while supplying it scopes the same aggregate contract to the selected database. The
+previous renderer is reachable only as a temporary development/staging comparison with
+`?db-ui=legacy` when Vite development or `VITE_DB_MANAGEMENT_LEGACY=true` enables it. The standalone
+prototype on port `5173` remains outside `AppShell` only until the integrated surface is accepted.
+
 ## Catalog description generation

 Catalog description generation is a backend-owned administrative operation, separate from the
@@ -83,6 +103,12 @@ or second orchestration subsystem. A target receives at most one provider retry;
 exhausted technical batches fail the run. Stale work is marked interrupted at startup and must be
 explicitly unlocked; it never resumes automatically.

+Sensitive-field suggestion generation remains a synchronous administrative request, but each
+attempt has its own durable run and ordered sanitized events. This history is separate from
+Description Generation because its lifecycle and counters differ. Only execution metadata and
+aggregate counts are stored; proposed flags, prompts, raw model output, and provider diagnostics
+remain transient.
+
 ## Main backend classes

 The diagram shows the classes that form the bridge between the browser, Pi, and `tht`. Fastify routes receive requests and delegate to these services.
@@ -15,6 +15,31 @@ nullability, primary-key positions, and ordered foreign-key pairs are observatio
 schema and cannot be manually created, renamed, or structurally edited. Descriptions are the
 editable metadata.

+## Navigate the Fleet Ledger surface
+
+Database Management opens Fleet Ledger inside the normal application shell. Only one data grid is
+shown at a time: choose a database to see its tables, choose a table to see its columns, or open the
+database's relationships view. Use the emphasized back control or breadcrumb to return to the parent
+grid.
+
+The KPI strip reports tables, columns, sensitive columns, relationships, and description coverage.
+It uses `GET /catalog/metrics` without `databaseId` for installation totals and with `databaseId` for
+the current database. Choose a selection-scoped operation from the action selector and then press
+**Run**; unavailable operations remain listed with an explanation. Row-specific actions are the icon
+controls in the final column, and each navigation or action icon has an immediate conceptual tooltip.
+
+Configuration, object details, metadata editors, synchronization history, description history,
+sensitive-field review, and suggestion-run history open in right-side drawers backed by the
+production catalog APIs.
+Closing a history drawer does not cancel a durable background run. Existing permission checks,
+dirty/busy navigation guards, stale-state handling, and write-only secret behavior continue to
+apply.
+
+For temporary comparison in development or staging, add `?db-ui=legacy`; the parameter is honored
+only by Vite development or an environment explicitly configured with
+`VITE_DB_MANAGEMENT_LEGACY=true`. The separate prototype on port `5173` is not the application and
+remains available only until the integrated Fleet Ledger surface passes owner acceptance.
+
 ## Configure and test a database

 1. Open **Database Management** and choose a workspace.
@@ -54,6 +79,11 @@ administrator can ask the configured model to suggest flags from structural meta
 schema, table and column names, data types, nullability, primary keys, and foreign keys). Suggestions
 remain an unsaved draft until a human reviews and saves them.

+The page exposes separate histories for description generation and sensitive-field suggestions.
+Sensitive-suggestion history stores the selected model, scope, status, aggregate counts, timestamps,
+and sanitized events. It does not store the proposed per-column flags, prompts, raw model output, or
+provider diagnostics; closing an unsaved review still discards that draft.
+
 For a column with `sensitive=false`, the worker may read at most five source rows and five
 representative non-null values through a read-only connector. For `sensitive=true`, the source query
 does not request that column's values; deterministic plausible values derived only from its name and