From d504b1def1d453f73199631d2fc13b63008df597 Mon Sep 17 00:00:00 2001 From: Codex Date: Sat, 29 Aug 2026 16:46:48 +0200 Subject: [PATCH] docs(testing): record AI description acceptance --- ...talog-description-generation-acceptance.md | 96 +++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 97 insertions(+) create mode 100644 docs/testing/2026-08-29-ai-catalog-description-generation-acceptance.md diff --git a/docs/testing/2026-08-29-ai-catalog-description-generation-acceptance.md b/docs/testing/2026-08-29-ai-catalog-description-generation-acceptance.md new file mode 100644 index 00000000..102a7235 --- /dev/null +++ b/docs/testing/2026-08-29-ai-catalog-description-generation-acceptance.md @@ -0,0 +1,96 @@ +# AI Catalog Description Generation acceptance + +Date: 2026-08-29 + +Feature commit: `376dd5a` + +Result: **passed** + +## Scope + +This acceptance run covered the interactive generation and consolidation of Catalog Table and +Catalog Column descriptions. It used only an ephemeral PostgreSQL database and invented values; no +PSD database or other real business data was queried. + +The installation-level model catalog contained these entries, with `glm-53` as the default: + +| Selection ID | LiteLLM route | Authentication | +| --- | --- | --- | +| `deepseek-v4-pro` | `deepseek/deepseek-v4-pro` | Protected `DEEPSEEK_API_KEY` value already used by core | +| `glm-53` | `openai/glm-5.3` on the Z.AI coding endpoint | Protected `ZAI_API_KEY` value already used by core | +| `qwen-36` | `openai/qwen3.6-35b-a3b` on the VPN-only AritmoLab endpoint | No operator API key | + +Qwen was configured with `disableThinking: true`. The helper translated this into the endpoint's +chat-template option so reasoning prose could not precede the required JSON result. LiteLLM's +OpenAI-compatible client still receives a fixed, non-secret compatibility placeholder; no Qwen +credential is configured or required. + +Secret values were read only from the existing protected core authentication material and projected +into an ephemeral test bundle. No value, digest, prefix, or suffix is recorded here. + +## Provider and database checks + +- DeepSeek completed a real request successfully. +- GLM completed a real request successfully. +- Qwen's anonymous `/v1/models` endpoint exposed `qwen3.6-35b-a3b`, and a completion with thinking + disabled succeeded over the VPN. +- The disposable database used PostgreSQL `17.6-bookworm`, schema `acceptance`, and table + `prodotti_demo` with invented rows. +- The application role `ai_acceptance_reader` could execute `SELECT` but an `INSERT` failed with + PostgreSQL permission code `42501`, confirming the read-only boundary. + +## End-to-end result + +Four generation runs went through the application worker: + +1. DeepSeek generated the Catalog Column description for `categoria`. +2. GLM generated the Catalog Column description for `prezzo`. +3. Qwen generated the Catalog Column description for `attivo`. +4. GLM generated the Catalog Table description for `prodotti_demo`. + +Each run emitted exactly four safe activity events: queued, started, target generated, and +completed. The generated Italian descriptions were: + +- `categoria`: “Categoria merceologica dimostrativa del prodotto.” +- `prezzo`: “Prezzo del prodotto, espresso come valore numerico con decimali (es. 10.00, 12.50); + campo obbligatorio, non ammette valori nulli.” +- `attivo`: “Indica se il prodotto è attivo.” +- `prodotti_demo`: “Tabella di prodotti dimostrativi per il collaudo: per ogni prodotto registra un + codice univoco inventato (chiave primaria testuale, es. ALFA-DEMO), la categoria merceologica, il + prezzo numerico con decimali (es. 10.00, 12.50) e un flag booleano di attivazione; tutti i quattro + campi sono obbligatori.” + +The interactive consolidation action copied one generated description into `Description` and +reported `{copied: 1, skipped: 0}`. + +## Data and log safety assertions + +- Every provider request received bounded real source samples from the disposable table, within the + configured maximum of five rows and five representative values. +- Neither `sourceSample` nor `representativeValues` appeared in persisted catalog/run data. +- Protected credential values did not appear in worker logs, API responses, or persistence. +- Activity logs contained operational summaries only, without prompts, source rows, representative + values, or model responses. +- The generated table description echoed the invented sample `ALFA-DEMO`. This is acceptable for + this synthetic run but demonstrates why production policy must classify, exclude, or anonymize + sensitive values before model calls. That follow-up is tracked by + [issue #12](https://git.tylconsulting.it/mptyl/ThothII/issues/12). + +## Regression gates + +The following reproducible gates passed after the provider changes: + +| Gate | Command | Result | +| --- | --- | --- | +| Backend tests | `cd backend && npm test` | 93 files passed, 1 skipped; 1,243 tests passed, 40 skipped | +| Backend typecheck | `cd backend && npx tsc --noEmit -p .` | Passed | +| Backend production build | `cd backend && npm run build` | Passed | +| Harness tests | `cd harness && .venv/bin/pytest -q` | 1,138 passed, 4 deselected, 53 warnings | +| Harness lint | `harness/.venv/bin/ruff check harness` | Passed | +| Go tests | `env GOCACHE=/tmp/thothii-go-cache go test -p 1 ./...` | All packages passed | +| Frontend tests | `cd frontend && npm test` | 63 files and 532 tests passed | +| Frontend production build | `cd frontend && npm run build` | Passed; 4,860 modules transformed | +| Documentation | `./scripts/build-docs.sh` | Strict MkDocs build passed | + +The disposable database, temporary secret projection, and one-shot provider probe files were +removed after the successful run. diff --git a/mkdocs.yml b/mkdocs.yml index 23c0c92a..42096720 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -57,6 +57,7 @@ nav: - Architecture overview: architecture/overview.md - Components, modules, and flows: architecture/components.md - Evidence: evidence.md + - AI catalog description acceptance: testing/2026-08-29-ai-catalog-description-generation-acceptance.md - Authentication: architecture/authentication.md - Local authentication installation: install/authentication-local.md - Generic OIDC: install/authentication-oidc.md