docs(testing): record AI description acceptance
This commit is contained in:
Codex
@@ -0,0 +1,96 @@
|
||||
# AI Catalog Description Generation acceptance
|
||||
|
||||
Date: 2026-08-29
|
||||
|
||||
Feature commit: `376dd5a`
|
||||
|
||||
Result: **passed**
|
||||
|
||||
## Scope
|
||||
|
||||
This acceptance run covered the interactive generation and consolidation of Catalog Table and
|
||||
Catalog Column descriptions. It used only an ephemeral PostgreSQL database and invented values; no
|
||||
PSD database or other real business data was queried.
|
||||
|
||||
The installation-level model catalog contained these entries, with `glm-53` as the default:
|
||||
|
||||
| Selection ID | LiteLLM route | Authentication |
|
||||
| --- | --- | --- |
|
||||
| `deepseek-v4-pro` | `deepseek/deepseek-v4-pro` | Protected `DEEPSEEK_API_KEY` value already used by core |
|
||||
| `glm-53` | `openai/glm-5.3` on the Z.AI coding endpoint | Protected `ZAI_API_KEY` value already used by core |
|
||||
| `qwen-36` | `openai/qwen3.6-35b-a3b` on the VPN-only AritmoLab endpoint | No operator API key |
|
||||
|
||||
Qwen was configured with `disableThinking: true`. The helper translated this into the endpoint's
|
||||
chat-template option so reasoning prose could not precede the required JSON result. LiteLLM's
|
||||
OpenAI-compatible client still receives a fixed, non-secret compatibility placeholder; no Qwen
|
||||
credential is configured or required.
|
||||
|
||||
Secret values were read only from the existing protected core authentication material and projected
|
||||
into an ephemeral test bundle. No value, digest, prefix, or suffix is recorded here.
|
||||
|
||||
## Provider and database checks
|
||||
|
||||
- DeepSeek completed a real request successfully.
|
||||
- GLM completed a real request successfully.
|
||||
- Qwen's anonymous `/v1/models` endpoint exposed `qwen3.6-35b-a3b`, and a completion with thinking
|
||||
disabled succeeded over the VPN.
|
||||
- The disposable database used PostgreSQL `17.6-bookworm`, schema `acceptance`, and table
|
||||
`prodotti_demo` with invented rows.
|
||||
- The application role `ai_acceptance_reader` could execute `SELECT` but an `INSERT` failed with
|
||||
PostgreSQL permission code `42501`, confirming the read-only boundary.
|
||||
|
||||
## End-to-end result
|
||||
|
||||
Four generation runs went through the application worker:
|
||||
|
||||
1. DeepSeek generated the Catalog Column description for `categoria`.
|
||||
2. GLM generated the Catalog Column description for `prezzo`.
|
||||
3. Qwen generated the Catalog Column description for `attivo`.
|
||||
4. GLM generated the Catalog Table description for `prodotti_demo`.
|
||||
|
||||
Each run emitted exactly four safe activity events: queued, started, target generated, and
|
||||
completed. The generated Italian descriptions were:
|
||||
|
||||
- `categoria`: “Categoria merceologica dimostrativa del prodotto.”
|
||||
- `prezzo`: “Prezzo del prodotto, espresso come valore numerico con decimali (es. 10.00, 12.50);
|
||||
campo obbligatorio, non ammette valori nulli.”
|
||||
- `attivo`: “Indica se il prodotto è attivo.”
|
||||
- `prodotti_demo`: “Tabella di prodotti dimostrativi per il collaudo: per ogni prodotto registra un
|
||||
codice univoco inventato (chiave primaria testuale, es. ALFA-DEMO), la categoria merceologica, il
|
||||
prezzo numerico con decimali (es. 10.00, 12.50) e un flag booleano di attivazione; tutti i quattro
|
||||
campi sono obbligatori.”
|
||||
|
||||
The interactive consolidation action copied one generated description into `Description` and
|
||||
reported `{copied: 1, skipped: 0}`.
|
||||
|
||||
## Data and log safety assertions
|
||||
|
||||
- Every provider request received bounded real source samples from the disposable table, within the
|
||||
configured maximum of five rows and five representative values.
|
||||
- Neither `sourceSample` nor `representativeValues` appeared in persisted catalog/run data.
|
||||
- Protected credential values did not appear in worker logs, API responses, or persistence.
|
||||
- Activity logs contained operational summaries only, without prompts, source rows, representative
|
||||
values, or model responses.
|
||||
- The generated table description echoed the invented sample `ALFA-DEMO`. This is acceptable for
|
||||
this synthetic run but demonstrates why production policy must classify, exclude, or anonymize
|
||||
sensitive values before model calls. That follow-up is tracked by
|
||||
[issue #12](https://git.tylconsulting.it/mptyl/ThothII/issues/12).
|
||||
|
||||
## Regression gates
|
||||
|
||||
The following reproducible gates passed after the provider changes:
|
||||
|
||||
| Gate | Command | Result |
|
||||
| --- | --- | --- |
|
||||
| Backend tests | `cd backend && npm test` | 93 files passed, 1 skipped; 1,243 tests passed, 40 skipped |
|
||||
| Backend typecheck | `cd backend && npx tsc --noEmit -p .` | Passed |
|
||||
| Backend production build | `cd backend && npm run build` | Passed |
|
||||
| Harness tests | `cd harness && .venv/bin/pytest -q` | 1,138 passed, 4 deselected, 53 warnings |
|
||||
| Harness lint | `harness/.venv/bin/ruff check harness` | Passed |
|
||||
| Go tests | `env GOCACHE=/tmp/thothii-go-cache go test -p 1 ./...` | All packages passed |
|
||||
| Frontend tests | `cd frontend && npm test` | 63 files and 532 tests passed |
|
||||
| Frontend production build | `cd frontend && npm run build` | Passed; 4,860 modules transformed |
|
||||
| Documentation | `./scripts/build-docs.sh` | Strict MkDocs build passed |
|
||||
|
||||
The disposable database, temporary secret projection, and one-shot provider probe files were
|
||||
removed after the successful run.
|
||||
Reference in New Issue
Block a user