docs(testing): record AI description acceptance

This commit is contained in:
Codex
2026-08-29 16:47:13 +02:00
parent 376dd5a09d
commit d504b1def1
2 changed files with 97 additions and 0 deletions
@@ -0,0 +1,96 @@
# AI Catalog Description Generation acceptance
Date: 2026-08-29
Feature commit: `376dd5a`
Result: **passed**
## Scope
This acceptance run covered the interactive generation and consolidation of Catalog Table and
Catalog Column descriptions. It used only an ephemeral PostgreSQL database and invented values; no
PSD database or other real business data was queried.
The installation-level model catalog contained these entries, with `glm-53` as the default:
| Selection ID | LiteLLM route | Authentication |
| --- | --- | --- |
| `deepseek-v4-pro` | `deepseek/deepseek-v4-pro` | Protected `DEEPSEEK_API_KEY` value already used by core |
| `glm-53` | `openai/glm-5.3` on the Z.AI coding endpoint | Protected `ZAI_API_KEY` value already used by core |
| `qwen-36` | `openai/qwen3.6-35b-a3b` on the VPN-only AritmoLab endpoint | No operator API key |
Qwen was configured with `disableThinking: true`. The helper translated this into the endpoint's
chat-template option so reasoning prose could not precede the required JSON result. LiteLLM's
OpenAI-compatible client still receives a fixed, non-secret compatibility placeholder; no Qwen
credential is configured or required.
Secret values were read only from the existing protected core authentication material and projected
into an ephemeral test bundle. No value, digest, prefix, or suffix is recorded here.
## Provider and database checks
- DeepSeek completed a real request successfully.
- GLM completed a real request successfully.
- Qwen's anonymous `/v1/models` endpoint exposed `qwen3.6-35b-a3b`, and a completion with thinking
disabled succeeded over the VPN.
- The disposable database used PostgreSQL `17.6-bookworm`, schema `acceptance`, and table
`prodotti_demo` with invented rows.
- The application role `ai_acceptance_reader` could execute `SELECT` but an `INSERT` failed with
PostgreSQL permission code `42501`, confirming the read-only boundary.
## End-to-end result
Four generation runs went through the application worker:
1. DeepSeek generated the Catalog Column description for `categoria`.
2. GLM generated the Catalog Column description for `prezzo`.
3. Qwen generated the Catalog Column description for `attivo`.
4. GLM generated the Catalog Table description for `prodotti_demo`.
Each run emitted exactly four safe activity events: queued, started, target generated, and
completed. The generated Italian descriptions were:
- `categoria`: “Categoria merceologica dimostrativa del prodotto.”
- `prezzo`: “Prezzo del prodotto, espresso come valore numerico con decimali (es. 10.00, 12.50);
campo obbligatorio, non ammette valori nulli.”
- `attivo`: “Indica se il prodotto è attivo.”
- `prodotti_demo`: “Tabella di prodotti dimostrativi per il collaudo: per ogni prodotto registra un
codice univoco inventato (chiave primaria testuale, es. ALFA-DEMO), la categoria merceologica, il
prezzo numerico con decimali (es. 10.00, 12.50) e un flag booleano di attivazione; tutti i quattro
campi sono obbligatori.”
The interactive consolidation action copied one generated description into `Description` and
reported `{copied: 1, skipped: 0}`.
## Data and log safety assertions
- Every provider request received bounded real source samples from the disposable table, within the
configured maximum of five rows and five representative values.
- Neither `sourceSample` nor `representativeValues` appeared in persisted catalog/run data.
- Protected credential values did not appear in worker logs, API responses, or persistence.
- Activity logs contained operational summaries only, without prompts, source rows, representative
values, or model responses.
- The generated table description echoed the invented sample `ALFA-DEMO`. This is acceptable for
this synthetic run but demonstrates why production policy must classify, exclude, or anonymize
sensitive values before model calls. That follow-up is tracked by
[issue #12](https://git.tylconsulting.it/mptyl/ThothII/issues/12).
## Regression gates
The following reproducible gates passed after the provider changes:
| Gate | Command | Result |
| --- | --- | --- |
| Backend tests | `cd backend && npm test` | 93 files passed, 1 skipped; 1,243 tests passed, 40 skipped |
| Backend typecheck | `cd backend && npx tsc --noEmit -p .` | Passed |
| Backend production build | `cd backend && npm run build` | Passed |
| Harness tests | `cd harness && .venv/bin/pytest -q` | 1,138 passed, 4 deselected, 53 warnings |
| Harness lint | `harness/.venv/bin/ruff check harness` | Passed |
| Go tests | `env GOCACHE=/tmp/thothii-go-cache go test -p 1 ./...` | All packages passed |
| Frontend tests | `cd frontend && npm test` | 63 files and 532 tests passed |
| Frontend production build | `cd frontend && npm run build` | Passed; 4,860 modules transformed |
| Documentation | `./scripts/build-docs.sh` | Strict MkDocs build passed |
The disposable database, temporary secret projection, and one-shot provider probe files were
removed after the successful run.
+1
View File
@@ -57,6 +57,7 @@ nav:
- Architecture overview: architecture/overview.md
- Components, modules, and flows: architecture/components.md
- Evidence: evidence.md
- AI catalog description acceptance: testing/2026-08-29-ai-catalog-description-generation-acceptance.md
- Authentication: architecture/authentication.md
- Local authentication installation: install/authentication-local.md
- Generic OIDC: install/authentication-oidc.md