diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 0952344b..4689b78a 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -85,6 +85,12 @@ A later deployment does not prove every earlier acceptance item passed. ## Remaining acceptance and design gates +- The first installation acceptance uses an ad hoc + [Chinook workspace without Evidence](docs/testing/chinook-installation-smoke.md), + backed by a separate local PostgreSQL test container. Its fixture import, + read-only credentials and offline workspace validation were verified; full + installation/readiness and a human-reviewed question remain pending. Evidence + acceptance and the three curated examples are separate later work. - Fresh-machine Mac, Windows/WSL2 and Linux installation acceptance, including real DWH/model endpoints, remains a separate operator exercise. - Real IdP/portal login, logout, embedded interaction and PSD semantic acceptance diff --git a/docs/testing/chinook-installation-smoke.md b/docs/testing/chinook-installation-smoke.md new file mode 100644 index 00000000..cd550d3d --- /dev/null +++ b/docs/testing/chinook-installation-smoke.md @@ -0,0 +1,112 @@ +# First installation acceptance fixture: Chinook without Evidence + +Decision agreed on 2026-09-28: the first installer acceptance may use one ad hoc +workspace and a downloadable local database without Evidence. The three curated +example databases remain a separate deferred subproject. + +Use [Chinook v1.4.5](https://github.com/lerocha/chinook-database/releases/tag/v1.4.5), +the sample digital music store. The upstream PostgreSQL SQL asset contains both +schema and data: 11 related tables for artists, albums, tracks, customers, employees, +invoices and playlists. The source is licensed under +[MIT](https://github.com/lerocha/chinook-database/blob/master/LICENSE.md). +This fixture does not supply Evidence or claim that SQL schema information is Evidence. +It uses native PostgreSQL DDL/data, so SQLite type conversion is not involved. + +## Prepare the independent test database + +Run in Ubuntu WSL2 with Docker Desktop integration, or in a Linux/macOS Bash +terminal for a preliminary check. Docker, curl and OpenSSL must be available. +The first formal host acceptance remains Windows/WSL2. + +The helper below is in the ThothII repository for maintainers/testers; it is not +yet included in the downloadable operator bundle. It provisions only the test DWH, +which is a separate container from the installation's PostgreSQL Metadata Catalog. + +```bash +cd /path/to/ThothII +bash scripts/prepare-chinook-test.sh "$HOME/thothii-chinook-test" +``` + +Use a new directory outside any workspace repository. The helper pins the upstream +SQL version and checksum, creates private random passwords, starts PostgreSQL, +imports schema/data, and creates `thoth_reader` with SELECT privileges. Passwords +remain in the private directory, not in a workspace YAML or this repository. +An existing directory or container is refused; the helper never resets it. +An interrupted preparation leaves its private logs/container available for diagnosis. +Do not rerun the upstream import manually against another database: its opening +statements drop and recreate `chinook`. + +Default container: `thothii-test-chinook`. Host endpoint: `127.0.0.1:55432`. +Override names/port before the command if needed: + +```bash +CHINOOK_TEST_CONTAINER=my-chinook CHINOOK_TEST_PORT=55433 \ + bash scripts/prepare-chinook-test.sh "$HOME/my-chinook-test" +``` + +Expected import counts: 11 tables, 3,503 tracks, 412 invoices and 2,240 invoice lines. +The SQL download is 600,200 bytes. Its SHA-256 is +`e3fde5c1a5b51a2a91429a702c9ca6e69ba56e6c7f5e112724d70c3d03db695e`. + +Stop/start retains the database: + +```bash +docker stop thothii-test-chinook +docker start thothii-test-chinook +``` + +Only when deliberately discarding this test database, `docker rm -f -v +thothii-test-chinook` removes the container and its anonymous data volume. Private +files in the preparation directory are separate and remain until removed explicitly. + +## Prepare the workspace documents + +Using the native bundle, keep `tht` beside `tht-workspace-documents` and run: + +```bash +/path/to/bundle/bin/tht workspace prepare \ + --directory "$HOME/thothii-workspaces-test" \ + --id chinook-test --name "Chinook installation test" --language it +/path/to/bundle/bin/tht workspace validate \ + --directory "$HOME/thothii-workspaces-test" +``` + +The generated workspace has Evidence absent. Keep that configuration; no fake +Evidence document or credential should be added merely to satisfy installation. + +The later installation-local binding must select PostgreSQL, database `chinook`, +schema `public`, user `thoth_reader`, and reference the private `reader-password` +file. Do not use the PostgreSQL administrator as the ThothII data source account. +The host endpoint above is for host-side tools: `127.0.0.1` inside ThothII core +would refer to core itself. At integration, configure and verify connectivity from +both the preflight process and the core container; do not copy the host endpoint +unchanged into a runtime binding. That integration belongs to the remaining +installer/binding tickets and is not established by successful import here. + +## Acceptance scope + +After the remaining installer increments are available, verify prepared documents, +non-interactive setup using published images, database connection, schema sync, +required preprocessing/readiness and a real human-reviewed question. Then verify +stop/start preserves application state. These checks do not validate Evidence +ingestion, retrieval or interpretation; record those as not exercised for this fixture. + +Suggested practice questions, without target SQL: + +- Which five artists have the most tracks in the catalogue? +- Which countries have the largest total invoice amounts? +- Which customers bought tracks from more than three genres? + +These are practice prompts, not benchmark scores or a substitute for human review. +Windows/WSL2 acceptance is followed by Omarchy and macOS in separate steps. + +## Preliminary verification, 2026-09-28 + +The helper imported the pinned SQL into a fresh Linux amd64 PostgreSQL container +on the macOS development host. Counts matched the values above. A real TCP +connection using the generated `thoth_reader` password read the data; +`default_transaction_read_only` was `on`, and an UPDATE without matching rows +was still rejected with SQLSTATE `42501` even inside `BEGIN READ WRITE`. +Native workspace preparation and JSON validation reported `ok: true` and +`evidence: absent`, explicitly deferring Catalog binding, connectivity and runtime +preprocessing. These are fixture checks, not Windows or full-installer acceptance. diff --git a/scripts/prepare-chinook-test.sh b/scripts/prepare-chinook-test.sh new file mode 100644 index 00000000..7aa55249 --- /dev/null +++ b/scripts/prepare-chinook-test.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# Disposable acceptance fixture, separate from the ThothII Metadata Catalog. +set -euo pipefail +umask 077 + +if [[ $# -ne 1 || "$1" != /* || -e "$1" ]]; then + echo 'Usage: bash scripts/prepare-chinook-test.sh /absolute/NEW-private-directory' >&2 + exit 2 +fi +test_dir="$1" +container="${CHINOOK_TEST_CONTAINER:-thothii-test-chinook}" +port="${CHINOOK_TEST_PORT:-55432}" +[[ "$container" =~ ^[a-zA-Z0-9][a-zA-Z0-9_.-]+$ ]] || exit 2 +[[ "$port" =~ ^[1-9][0-9]{0,4}$ ]] && (( port <= 65535 )) || exit 2 +for executable in docker curl openssl; do command -v "$executable" >/dev/null; done +docker info >/dev/null +if docker container inspect "$container" >/dev/null 2>&1; then + echo "Container $container already exists; this command never resets an existing database." >&2 + exit 2 +fi +mkdir "$test_dir" +test_dir="$(cd "$test_dir" && pwd -P)" +sql="$test_dir/Chinook_PostgreSql.sql" +curl --fail --location --proto '=https' --tlsv1.2 --max-time 120 \ + 'https://github.com/lerocha/chinook-database/releases/download/v1.4.5/Chinook_PostgreSql.sql' \ + --output "$sql" +expected=e3fde5c1a5b51a2a91429a702c9ca6e69ba56e6c7f5e112724d70c3d03db695e +actual="$(openssl dgst -sha256 "$sql")" +[[ "${actual##* }" == "$expected" ]] || { echo 'Chinook checksum mismatch.' >&2; exit 1; } +openssl rand -hex 24 > "$test_dir/reader-password" +printf 'POSTGRES_PASSWORD=%s\n' "$(openssl rand -hex 24)" > "$test_dir/postgres.env" +printf '%s\n' "$container" > "$test_dir/container-name" + +# Use the same PostgreSQL linux/amd64 image as the initial installation prerelease. +# PostgreSQL gets its own anonymous data volume; stop/start preserves it. +docker run --detach --name "$container" --platform linux/amd64 \ + --publish "127.0.0.1:${port}:5432" \ + --env-file "$test_dir/postgres.env" \ + docker.io/library/postgres@sha256:45cd22f8d32e189d245403954882f88e7a8714301fda80dab6da90f1265b25a3 >/dev/null +ready=false +for ((attempt = 0; attempt < 90; attempt++)); do + # The entrypoint's temporary bootstrap server has only a Unix socket. + if docker exec "$container" pg_isready -h 127.0.0.1 -U postgres -d postgres >/dev/null 2>&1; then + ready=true + break + fi + sleep 1 +done +[[ "$ready" == true ]] || { echo "PostgreSQL did not start. Inspect docker logs $container." >&2; exit 1; } +# The upstream script drops/recreates chinook: it is used only in this NEW container. +docker exec -i "$container" psql -X -U postgres -d postgres -v ON_ERROR_STOP=1 < "$sql" > "$test_dir/import.log" 2>&1 +{ + printf "CREATE ROLE thoth_reader LOGIN PASSWORD '%s';\n" "$(cat "$test_dir/reader-password")" + cat <<'SQL' +GRANT CONNECT ON DATABASE chinook TO thoth_reader; +GRANT USAGE ON SCHEMA public TO thoth_reader; +GRANT SELECT ON ALL TABLES IN SCHEMA public TO thoth_reader; +ALTER ROLE thoth_reader SET default_transaction_read_only = on; +SQL +} | docker exec -i "$container" psql -X -U postgres -d chinook -v ON_ERROR_STOP=1 > "$test_dir/reader-setup.log" 2>&1 +docker exec "$container" psql -X -U postgres -d chinook -v ON_ERROR_STOP=1 -c \ + "SELECT (SELECT count(*) FROM information_schema.tables WHERE table_schema='public' AND table_type='BASE TABLE') AS tables, (SELECT count(*) FROM track) AS tracks, (SELECT count(*) FROM invoice) AS invoices, (SELECT count(*) FROM invoice_line) AS invoice_lines;" +printf '\nReady: container=%s, host=127.0.0.1, port=%s, database=chinook, schema=public, user=thoth_reader\n' "$container" "$port" +printf 'Protected password file: %s/reader-password\n' "$test_dir" +printf 'Use this read-only account for ThothII. Evidence is absent. No ThothII stack was started.\n'