From 5d037e97c4ff501222476a29f3a78ba35a41cb29 Mon Sep 17 00:00:00 2001 From: mptyl Date: Wed, 5 Aug 2026 06:58:19 +0200 Subject: [PATCH] refactor: remove portal deployment coupling --- .dockerignore | 5 +- AGENTS.md | 6 +- PROJECT_STATE.md | 20 ++- README.md | 93 +++++------- deploy/compose.production.yaml | 11 -- deploy/compose.psd-local.yaml.example | 52 ------- deploy/thothii.env.example | 3 +- docker-compose.dev.yml | 10 +- docker/core.Dockerfile | 7 +- docker/nginx.conf | 6 +- docs/architecture/overview.md | 4 +- .../server-compose.workspace-registry.yaml | 6 +- docs/installazione-docker-4-contesti.md | 62 ++++---- frontend/vite.config.ts | 6 +- harness/tests/test_local_compose_contract.py | 33 +++++ .../tests/test_psd_local_compose_contract.py | 133 ------------------ scripts/bootstrap-local-psd-docker-config.sh | 70 --------- scripts/docker-smoke.sh | 4 +- scripts/run-stack.sh | 72 ++-------- scripts/test-external-compose-lifecycle.sh | 11 ++ scripts/test-local-dev-routing.sh | 8 +- scripts/test-no-deployment-coupling.sh | 75 ++++++++++ scripts/test-pi-user-auth-compose.sh | 6 +- scripts/test-verify-line-endings.sh | 13 ++ scripts/verify-line-endings.sh | 1 + 25 files changed, 265 insertions(+), 452 deletions(-) delete mode 100644 deploy/compose.production.yaml delete mode 100644 deploy/compose.psd-local.yaml.example create mode 100644 harness/tests/test_local_compose_contract.py delete mode 100644 harness/tests/test_psd_local_compose_contract.py delete mode 100644 scripts/bootstrap-local-psd-docker-config.sh create mode 100755 scripts/test-no-deployment-coupling.sh diff --git a/.dockerignore b/.dockerignore index fc356502..492b9eef 100644 --- a/.dockerignore +++ b/.dockerignore @@ -15,7 +15,10 @@ !deploy/env/*.env.example deploy/thothii.env deploy/secrets/ -harness/workspaces/psd.yaml +harness/workspaces/*.yaml +!harness/workspaces/local.yaml +!harness/workspaces/tht.example.yaml +!harness/workspaces/tht-test.yaml **/*.log **/.DS_Store coverage/ diff --git a/AGENTS.md b/AGENTS.md index 34fc1175..4867fe6b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,8 +11,10 @@ detail. Design history lives in `docs/superpowers/specs/` and `docs/superpowers/ ## Commands -The repo has three independently-built layers. Run the **full stack** (real Pi + DWH, needs -VPN + `harness/.env` + `pi` on PATH) with `./scripts/run-stack.sh` (frontend :5173 → backend :8787). +The repo has three independently-built layers. Run the local Docker stack with +`./scripts/run-stack.sh` after creating `deploy/env/local.env`; it starts the base+local Compose +profile, whose core image contains Pi. DWH, vector DB, embedding, and LLM remain external +configuration endpoints. **harness/** (Python `tht` CLI + Pi gate extension) - Install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` (puts `tht` on PATH) diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index beed187c..be2df99b 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -1,8 +1,24 @@ # ThothII — Project State -> Starting-point snapshot for new sessions. Last updated: 2026-07-23 (session summary redesign live). +> Starting-point snapshot for new sessions. Last updated: 2026-08-05 (portable deployment decoupled). > Point a fresh session here ("read PROJECT_STATE.md") before substantial work. +## Portable deployment decoupling — LIVE 2026-08-05 + +- **Mandatory stack.** The supported Compose stack is exactly `frontend` plus `core`; use the + base file with `deploy/compose.local.yaml` or `deploy/compose.server.yaml`. `run-stack.sh` + invokes the base+local Compose command and the core image provides Pi, so no host Pi binary is + part of the launch contract. +- **External boundaries.** DWH, vector DB, embedding, LLM, and reverse-proxy services are + external configurable endpoints even when deployed on the same infrastructure. The two + superseded PSD/portal deployment overlays were removed. Workspace descriptors and migration + utilities remain separate from deployment runtime configuration. +- **Legacy PSD deployment ruling.** The PSD bootstrap was deleted because it generated the + retired overlay and was therefore deployment machinery, not a data migration utility. Its + remaining live contract checks were renamed for the generic local Compose profile. The coupling + gate rejects stale active deployment filenames and content while deliberately excluding + historical plans/specs, canonical workspace descriptors, and non-runtime migration helpers. + ## Portable Git workspace registry — source integration (2026-08-04) - **Source of truth and scope.** The canonical workspace repository is a generic Git remote, @@ -103,7 +119,7 @@ release; never re-enable filesystem persistence, restore the archive into production, or dual-write during rollback. -## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12 +## Historical deployment — Docker locale (Profile A, co-located) — superseded 2026-08-05 ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale). diff --git a/README.md b/README.md index db231057..6b50a234 100644 --- a/README.md +++ b/README.md @@ -4,57 +4,37 @@ ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fasti core. The portable deployment runs exactly two application services; data services remain external in this profile. -## Docker Compose: one-command startup +## Docker Compose: local startup -Requirements: Docker Engine with Compose v2. The default project starts only the two -application images; DWH, vector and embedding services can be remote or supplied by an -optional overlay. +Requirements: Docker Engine with Compose v2. The mandatory stack is exactly the `core` and +`frontend` application images. DWH, vector DB, embedding, and LLM services are external, +configurable endpoints—even when they are co-located with ThothII. From a fresh clone, run these commands from the repository root: ```sh -cp .env.example .env -cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets -chmod 600 deploy/secrets/thothii.secrets -# Edit .env (non-secret endpoints) and deploy/secrets/thothii.secrets (KEY=VALUE lines). -docker compose up --build -d +cp deploy/env/local.env.example deploy/env/local.env +# Edit deploy/env/local.env, including PI_AUTH_FILE and the external endpoint URLs. +docker compose --env-file deploy/env/local.env \ + -f compose.yaml -f deploy/compose.local.yaml up --build -d ``` -The root `.env` is loaded automatically by Compose. It defaults to `compose.yaml`, an empty -profile, and `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`; no `--env-file`, `-f`, or -`--profile` flag is required for the normal installation. Add or edit YAML workspace descriptors -under `deploy/workspaces/`; they are mounted read-only and relative `roots` resolve beneath -`/data/workspaces/`. Open (set `THOTH_HTTP_PORT` in -`.env` to choose another loopback port). +`./scripts/run-stack.sh` runs this same base+local command in the foreground. The core image +contains its Pi runtime; no host `pi` executable is used. For a server installation, copy and +fill `deploy/env/server.env.example`, then use `-f compose.yaml -f deploy/compose.server.yaml`. +Workspace descriptors come from the Git remote configured by `THT_WORKSPACE_GIT_REMOTE`; their +runtime endpoint and secret bindings remain installation-local. Open + (set `THOTH_HTTP_PORT` in `deploy/env/local.env` to choose another +loopback port). -The bundle contains only values, one per line (`THT_MODEL_API_KEY=...`, DWH/vector keys, and -the optional local-vector passwords). It is ignored by Git and never copied into either image. -Do not put credentials in `.env`, workspace YAML, URLs, or Compose interpolation values. +Credentials and certificates are local protected files. Do not put them in environment examples, +workspace YAML, URLs, or Compose interpolation values. The optional `local-vector` and +preprocessing overlays are development presets; they do not change the two-service mandatory +stack or the external-endpoint contract. -### Optional overlays - -Overlays are selected in `.env`, so the operational command remains the same. On Unix-like -systems use `:` between files; on Windows use `;`: - -```dotenv -# Remote DWH/vector/embedding services with authenticated reverse proxy: -COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml -COMPOSE_PROFILES= - -# Local pgvector (Mac/Windows or a standalone application server): -COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml -COMPOSE_PROFILES=local-vector -``` - -After changing `.env`, apply the selected configuration with `docker compose up --build -d`. -Preprocessing is an explicit opt-in preset: append -`deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml` and set -`COMPOSE_PROFILES=local-vector,preprocess`; then run the job with -`docker compose run --rm preprocess-evidence` or `preprocess-dwh`. - -Application state, including settings, sessions, artifacts, and indexes, lives in the named -`thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an -explicit destructive command such as `docker compose down --volumes` removes it. +Application state is split across the named `settings`, `pi-state`, `workspace-registry`, and +`sessions` volumes. `docker compose down` keeps them. Only an explicit destructive command such +as `docker compose down --volumes` removes them. The frontend depends on the core health check and proxies `/health` and `/api/*` to it. The application health endpoint intentionally checks process readiness only; external dependency @@ -110,17 +90,18 @@ application state; passwords are selected at runtime and are never passed as URL ## Preprocessing jobs and S3 Evidence -The included job workspaces target the local-vector profile. Put the four local-vector password -keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then select -the preprocessing preset in `.env`: +The included job workspaces target the optional local-vector profile. Put the four local-vector +password keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then +run the explicit preprocessing preset: -```dotenv -COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml:deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml -COMPOSE_PROFILES=local-vector,preprocess +```sh +docker compose --env-file deploy/env/local.env \ + -f compose.yaml -f deploy/compose.local.yaml -f deploy/compose.local-vector.yaml \ + -f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \ + --profile local-vector --profile preprocess run --rm preprocess-evidence ``` -Run `docker compose run --rm preprocess-evidence` or -`docker compose run --rm preprocess-dwh`. The overlay makes each job wait for the vector +Replace the final service with `preprocess-dwh` when required. The overlay makes each job wait for the vector database health check, role reconciliation, and a successful migration; no separate database startup or migration command is required. @@ -183,8 +164,8 @@ with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts thi rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other auth mode fails during core startup. -Production credentials use the one Compose secret bundle, not `.env`. Put the required keys in -`deploy/secrets/thothii.secrets` and select the production overlay in `.env`: +Production credentials use the one Compose secret bundle, not an environment example. Put the +required keys in `deploy/secrets/thothii.secrets` for the selected base+server installation: ```dotenv THT_MODEL_API_KEY=replace-me @@ -255,10 +236,10 @@ session store without upstream authentication, direct DB host/name/runtime user/ The migrator independently rejects every other TLS mode before reading its password secret or constructing a database URL. -Perform the cutover in one maintenance window, with the Task 4 portal proxy headers and Task 5 -backend principal parser deployed together. Neither change is safe to deploy independently: Task -4 clears the legacy identity header and Task 5 rejects it. Drain/stop active Pi work, enable a -maintenance response at the portal, then run the migrator once and inspect its pristine JSON: +Perform the cutover in one maintenance window, with the upstream identity-proxy headers and +backend principal parser deployed together. Neither change is safe to deploy independently: the +proxy clears the legacy identity header and the backend rejects it. Drain/stop active Pi work, +enable a maintenance response at the proxy, then run the migrator once and inspect its pristine JSON: ```sh docker compose -f compose.yaml -f deploy/compose.session-server.yaml \ diff --git a/deploy/compose.production.yaml b/deploy/compose.production.yaml deleted file mode 100644 index ab419743..00000000 --- a/deploy/compose.production.yaml +++ /dev/null @@ -1,11 +0,0 @@ -services: - core: - environment: - AUTH_MODE: upstream - THOTH_PUBLIC_EXPOSURE: "true" - THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME} - THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL} - THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL} - THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL} - THT_DOCS_ROOT: ${THT_DOCS_ROOT:-/data/workspaces/example/evidence-source} - THT_SECRETS_FILE: /run/secrets/thothii.secrets diff --git a/deploy/compose.psd-local.yaml.example b/deploy/compose.psd-local.yaml.example deleted file mode 100644 index 2a52b893..00000000 --- a/deploy/compose.psd-local.yaml.example +++ /dev/null @@ -1,52 +0,0 @@ -services: - core: - environment: - AUTH_MODE: ${AUTH_MODE:-none} - THOTH_PUBLIC_EXPOSURE: ${THOTH_PUBLIC_EXPOSURE:-false} - MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4} - PI_PROVIDER: ${PI_PROVIDER:?set PI_PROVIDER} - PI_MODEL: ${PI_MODEL:?set PI_MODEL} - PI_THINKING: ${PI_THINKING:-medium} - THT_PROFILE: ${THT_PROFILE:-workstation} - THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME} - THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL} - THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL} - THT_VEC_WRITE_REST_URL: ${THT_VEC_WRITE_REST_URL:?set THT_VEC_WRITE_REST_URL} - THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL} - THT_SECRETS_FILE: /run/secrets/thothii.secrets - THT_DOCS_ROOT: /data/workspaces/psd - THT_CONFIG: /app/harness/config/tht.yaml - extra_hosts: - - host.docker.internal:host-gateway - networks: !override - default: - aliases: [core, thothii-core] - volumes: - - thoth_data:/data - - thoth_pi_config:/home/thoth/.pi - - ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro - - ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro - - ${THT_SECRETS_FILE:?set THT_SECRETS_FILE}:/run/secrets/thothii.secrets:ro - - ${THT_PSD_WORKSPACE_HOST_PATH:?set THT_PSD_WORKSPACE_HOST_PATH}/evidence:/data/evidence:ro - - ./deploy/workspaces/psd.yaml:/app/harness/config/tht.yaml:ro - - ${THT_PSD_WORKSPACE_HOST_PATH:?set THT_PSD_WORKSPACE_HOST_PATH}:/data/workspaces/psd - - frontend: - build: - args: - VITE_BASE: / - VITE_BACKEND_URL: /api - ports: - - "127.0.0.1:8099:8080" - networks: !override - default: - aliases: [frontend, thothii-frontend] - -volumes: - thoth_data: - thoth_pi_config: - -networks: - default: - external: true - name: thothii_default diff --git a/deploy/thothii.env.example b/deploy/thothii.env.example index 255b34d7..1fb73b94 100644 --- a/deploy/thothii.env.example +++ b/deploy/thothii.env.example @@ -18,8 +18,9 @@ THT_VEC_PASSWORD=__CHANGE_ME__ THT_OLLAMA_URL=http://host.docker.internal:11434 # --- Backend --- -AUTH_MODE=none # none | mock | oidc (in embedded l'auth è al bordo del portale) +AUTH_MODE=none # none | mock | oidc (upstream auth is enforced at the proxy boundary) MAX_PI_PROCESSES=4 +THT_DEV_EVIDENCE_HOST_PATH=/absolute/path/to/evidence # --- Git-backed workspace registry (no secret values belong in this file) --- THT_WORKSPACE_REGISTRY_ROOT=/data/workspace-registry diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index 3bf0f38c..26443ba2 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -1,4 +1,4 @@ -# ThothII — deploy STANDALONE locale (dev / smoke test, senza portale). +# ThothII — deploy STANDALONE locale (dev / smoke test). # Rete propria + porte host per ispezione diretta. # docker compose -f docker-compose.dev.yml up -d --build # frontend: http://localhost:8090 backend: http://localhost:8787 @@ -40,10 +40,10 @@ services: extra_hosts: - "host.docker.internal:host-gateway" volumes: - - /home/chirone/thothii-data:/data + - dev-data:/data - workspace-registry:/data/workspace-registry - - /home/chirone/thothii-data/pi-config:/home/thoth/.pi - - /home/chirone/chirone/etl/docs/evidence:/data/evidence:ro + - dev-pi-state:/home/thoth/.pi + - ${THT_DEV_EVIDENCE_HOST_PATH:-./evidence}:/data/evidence:ro - ${THT_WORKSPACE_GIT_CREDENTIALS_FILE:-/dev/null}:/run/secrets/workspace-registry-git-credentials:ro - ${THT_WORKSPACE_GIT_CA_FILE:-/etc/ssl/certs/ca-certificates.crt}:/run/secrets/workspace-registry-git-ca:ro - ${THT_WORKSPACE_GIT_SSH_KEY_FILE:-/dev/null}:/run/secrets/workspace-registry-git-ssh-key:ro @@ -73,4 +73,6 @@ networks: driver: bridge volumes: + dev-data: + dev-pi-state: workspace-registry: diff --git a/docker/core.Dockerfile b/docker/core.Dockerfile index f22510dc..57ca8a9c 100644 --- a/docker/core.Dockerfile +++ b/docker/core.Dockerfile @@ -64,11 +64,10 @@ RUN python -m venv /opt/venv \ && /opt/venv/bin/pip install --no-cache-dir --upgrade pip \ && (cd /app/harness && /opt/venv/bin/pip install --no-cache-dir .) \ && cp /app/harness/workflow.yaml /opt/venv/lib/python3.12/site-packages/workflow.yaml -# Default locale e alias PSD convergono sul file canonico. Il CLI onora anche -# THT_CONFIG, quindi cambiare CWD non cambia l'identita' dello workspace. +# Il workspace locale predefinito converge sul file canonico. Il CLI onora anche THT_CONFIG, +# quindi cambiare CWD non cambia l'identita' dello workspace. RUN mkdir -p /app/harness/config \ - && cp --remove-destination /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml \ - && ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/psd.yaml + && cp --remove-destination /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml # PiProcessManager (backend) prepende harnessDir/.venv/bin al PATH del child Pi → symlink al venv reale RUN ln -s /opt/venv /app/harness/.venv diff --git a/docker/nginx.conf b/docker/nginx.conf index dcb83245..deb349d2 100644 --- a/docker/nginx.conf +++ b/docker/nginx.conf @@ -1,6 +1,4 @@ -# nginx per thothii-frontend: serve la SPA (modalità standalone) e reverse-proxy /api -> core. -# In modalità embedded il portale proxya /datamart-builder/assets/ qui (solo asset statici); -# il blocco /api non è usato in embedded (il portale hita core direttamente). +# nginx per thothii-frontend: serve la SPA e inoltra /api al core sulla rete Compose. server { listen 8080; server_name _; @@ -29,7 +27,7 @@ server { chunked_transfer_encoding on; } - # manifest.json servito (lo legge il template tag Django in embedded) + # manifest.json è servito come JSON. location = /manifest.json { default_type application/json; } diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 695f82d1..0ab59001 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -54,6 +54,8 @@ Il frontend renderizza questi widget-descriptor (registro in `src/widgets/`); il ## Come si lancia lo stack -Lo **stack completo** (Pi reale + DWH reale, serve VPN + `harness/.env` + `pi` sul PATH) si avvia con `./scripts/run-stack.sh` (frontend `:5173` → backend `:8787`). +Lo stack locale si avvia con `./scripts/run-stack.sh`, dopo aver creato +`deploy/env/local.env` da `deploy/env/local.env.example`. Il core Compose include Pi; DWH, +vector DB, embedding e LLM sono endpoint esterni configurati nel file locale. Comandi per singolo layer, test, lint: vedi il file `CLAUDE.md` nella radice del repo (guida operativa per Claude Code, tenuta sincronizzata con questa pagina). diff --git a/docs/install/examples/server-compose.workspace-registry.yaml b/docs/install/examples/server-compose.workspace-registry.yaml index 996800c9..fecc6dbb 100644 --- a/docs/install/examples/server-compose.workspace-registry.yaml +++ b/docs/install/examples/server-compose.workspace-registry.yaml @@ -45,13 +45,13 @@ services: - source: session_ca target: session_ca.pem networks: - - portal + - upstream restart: unless-stopped networks: - portal: + upstream: external: true - name: ${THT_PORTAL_NETWORK:-omics_portal_omics_network} + name: ${THT_UPSTREAM_NETWORK:-thothii-upstream} secrets: session_runtime_password: diff --git a/docs/installazione-docker-4-contesti.md b/docs/installazione-docker-4-contesti.md index 66018cc1..42d9207c 100644 --- a/docs/installazione-docker-4-contesti.md +++ b/docs/installazione-docker-4-contesti.md @@ -14,27 +14,28 @@ Servono Docker Engine/Compose v2 su Linux oppure Docker Desktop su macOS/Windows ```sh git clone ThothII cd ThothII -cp .env.example .env -mkdir -p deploy/secrets deploy/workspaces -cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets -chmod 600 deploy/secrets/thothii.secrets +cp deploy/env/local.env.example deploy/env/local.env ``` Modificare **solo** questi file interni al clone: | File | Cosa contiene | |---|---| -| `.env` | endpoint, database, provider, `COMPOSE_FILE` e `COMPOSE_PROFILES`; mai password/token | -| `deploy/secrets/thothii.secrets` | un bundle `NOME=VALORE`, mode host `0600` o `0400` | +| `deploy/env/local.env` | endpoint, database e path Pi locali; mai password/token | +| file protetti locali | credenziali e certificati, indicati dai binding del workspace | | `deploy/workspaces/.yaml` | adapter, endpoint non riservati, `roots` ed Evidence | -Il file `.env` viene caricato automaticamente da Docker Compose perché è nella radice del progetto. Il valore predefinito è `COMPOSE_FILE=compose.yaml`, con profili vuoti e `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`. Perciò, dopo aver compilato `.env`, il bundle e almeno il workspace, l'avvio normale è sempre: +Compilare `deploy/env/local.env`, incluso `PI_AUTH_FILE`, con gli endpoint esterni. L'avvio +normale usa esplicitamente il file base e l'overlay locale: ```sh -docker compose up --build -d +docker compose --env-file deploy/env/local.env \ + -f compose.yaml -f deploy/compose.local.yaml up --build -d ``` -Non occorre usare `--env-file`, `-f` o `--profile` per questa installazione. Verificare lo stato con `docker compose ps` e aprire . `docker compose down` conserva il volume `thoth_data`; usare `down --volumes` solo per un ambiente effimero. +Verificare lo stato con lo stesso comando Compose e aprire . Il core +include Pi; il binario Pi non deve essere installato sull'host. `docker compose down` conserva i +volumi; usare `down --volumes` solo per un ambiente effimero. ### Formato del bundle unico @@ -55,21 +56,13 @@ Inserire solo le chiavi necessarie al profilo scelto. Il bundle viene montato in Una catena CA PEM **non può essere inserita nel bundle**: contiene whitespace e viene rifiutata dal parser. Se un endpoint usa una CA privata, conservarla nel secret manager/host e aggiungere un override Compose revisionato che monti il file in `/run/secrets/ca-chain.pem` e imposti `THT_SSL_CA` (o il parametro dell'adapter). Il clone base non crea quel mount: questa è una limitazione intenzionale da considerare in fase di deployment. -### Overlay opzionali tramite `.env` +### Overlay opzionali espliciti -Gli overlay non cambiano il comando operativo. Impostare in `.env`: - -```dotenv -# DWH/vector/embedding remoti (server applicativo o server con i DB): -COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml -COMPOSE_PROFILES= - -# pgvector locale (Mac, Windows o server autonomo): -COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml -COMPOSE_PROFILES=local-vector -``` - -Su Windows usare `;` come separatore di `COMPOSE_FILE`. Per il preprocessing locale aggiungere `deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml` e impostare `COMPOSE_PROFILES=local-vector,preprocess`; poi usare `docker compose run --rm preprocess-evidence` oppure `docker compose run --rm preprocess-dwh`. +DWH/vector/embedding remoti restano endpoint del file locale o server. Per il solo preset di +sviluppo pgvector, aggiungere `-f deploy/compose.local-vector.yaml --profile local-vector` al +comando base. Per il preprocessing aggiungere anche +`-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml --profile preprocess`, +poi usare `docker compose run --rm preprocess-evidence` oppure `preprocess-dwh` con gli stessi argomenti. ## Workspace, adapter e Evidence @@ -116,11 +109,10 @@ il bind mount/runtime adapter corrispondente. Non inserire la password nel works ## 1. Server remoto insieme ai database e al vector DB -Usare quando il server Docker è nella stessa rete del DWH e del vector DB (containerizzati o meno). Il file `.env` può restare sul default, senza profili, impostando gli endpoint raggiungibili localmente: +Usare quando il server Docker è nella stessa rete del DWH e del vector DB (containerizzati o meno). +Compilare `deploy/env/local.env` con gli endpoint raggiungibili localmente: ```dotenv -COMPOSE_FILE=compose.yaml -COMPOSE_PROFILES= THT_DB_NAME=warehouse THT_DWH_REST_URL=https://dwh.internal.example THT_VEC_REST_URL=https://vectors.internal.example @@ -143,17 +135,15 @@ claim normalizzati `X-Thoth-Principal-Issuer`, `X-Thoth-Principal-Subject`, `X-Thoth-Principal-Display-Name` e `X-Thoth-Is-Admin` attesi dal core. Non esporre direttamente la porta pubblicata da nginx. -Se il server deve essere raggiungibile da altri host, sostituire `COMPOSE_FILE` con -`compose.yaml:deploy/compose.production.yaml`, configurare il proxy autenticato e impostare +Se il server deve essere raggiungibile da altri host, usare il profilo +`deploy/compose.server.yaml`, configurare il proxy autenticato e impostare `AUTH_MODE=upstream`/`THOTH_PUBLIC_EXPOSURE=true` come descritto nella sezione di trust boundary. ## 2. Mac locale -Installare Docker Desktop e, se usato, Ollama sul Mac. Nel `.env` selezionare il profilo locale: +Installare Docker Desktop e, se usato, Ollama sul Mac. In `deploy/env/local.env` impostare gli endpoint: ```dotenv -COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml -COMPOSE_PROFILES=local-vector THT_DB_NAME=warehouse THT_DWH_REST_URL=https://dwh.example.test THT_OLLAMA_URL=http://host.docker.internal:11434 @@ -173,11 +163,9 @@ Poi eseguire il comando standard `docker compose up --build -d`. Il primo avvio ## 3. PC Windows locale -Usare Docker Desktop con backend WSL2 e abilitare la condivisione della directory del clone. Modificare `.env` con il separatore Windows: +Usare Docker Desktop con backend WSL2 e abilitare la condivisione della directory del clone. Modificare `deploy/env/local.env`: ```dotenv -COMPOSE_FILE=compose.yaml;deploy/compose.local-vector.yaml -COMPOSE_PROFILES=local-vector THT_DB_NAME=warehouse THT_DWH_REST_URL=https://dwh.example.test THT_OLLAMA_URL=http://host.docker.internal:11434 @@ -195,11 +183,11 @@ Se un bind mount viene rifiutato, aggiungere la cartella del repository a Docker ## 4. Server applicativo distinto da DB ed Evidence -Usare il profilo production e consentire dal firewall solo le destinazioni necessarie: +Usare il profilo server e consentire dal firewall solo le destinazioni necessarie: ```dotenv -COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml -COMPOSE_PROFILES= +# Avvio: docker compose --env-file deploy/env/server.env \ +# -f compose.yaml -f deploy/compose.server.yaml up --build -d THT_DB_NAME=warehouse THT_DWH_REST_URL=https://dwh.example.test THT_VEC_REST_URL=https://vectors.example.test diff --git a/frontend/vite.config.ts b/frontend/vite.config.ts index 6b98f78a..53c8f70f 100644 --- a/frontend/vite.config.ts +++ b/frontend/vite.config.ts @@ -3,13 +3,13 @@ import react from "@vitejs/plugin-react"; import path from "path"; export default defineConfig(() => { - const embedBase = process.env.VITE_BASE; // "/datamart-builder/assets/" in embedded; undefined = standalone + const embedBase = process.env.VITE_BASE; const apiUpstream = process.env.THT_FRONTEND_API_UPSTREAM ?? "http://localhost:8787"; return { plugins: [react()], // base: prefisso pubblico degli asset. Default "/" (standalone). - // assetsDir vuoto in embedded → asset alla root di dist/ così il proxy - // /datamart-builder/assets/ → frontend-root mappa 1:1 (niente /assets/assets/). + // Con un prefisso personalizzato, gli asset restano alla root di dist/ per evitare + // di duplicare il segmento assets nel percorso pubblico. base: embedBase ?? "/", build: { manifest: true, outDir: "dist", assetsDir: embedBase ? "" : "assets" }, server: { diff --git a/harness/tests/test_local_compose_contract.py b/harness/tests/test_local_compose_contract.py new file mode 100644 index 00000000..e9ea5c7c --- /dev/null +++ b/harness/tests/test_local_compose_contract.py @@ -0,0 +1,33 @@ +from pathlib import Path + +import yaml + + +def test_local_compose_uses_the_generic_external_endpoint_contract(): + root = Path(__file__).resolve().parents[2] + compose = yaml.safe_load((root / "compose.yaml").read_text()) + local = yaml.safe_load((root / "deploy/compose.local.yaml").read_text()) + + assert set(compose["services"]) == {"core", "frontend"} + assert local["services"]["core"]["environment"]["AUTH_MODE"] == "none" + assert local["services"]["core"]["ports"] == ["127.0.0.1:${THOTH_CORE_HTTP_PORT:-8787}:8787"] + assert local["services"]["frontend"]["ports"] == ["127.0.0.1:${THOTH_HTTP_PORT:-8080}:8080"] + + environment = compose["services"]["core"]["environment"] + for name in ("THT_DWH_REST_URL", "THT_VEC_REST_URL", "THT_OLLAMA_URL", "THT_LLM_URL"): + assert name in environment + assert {"settings", "pi-state", "workspace-registry", "sessions"} <= set(compose["volumes"]) + + +def test_core_image_prepares_the_writable_pi_profile_before_mounting_config_files(): + root = Path(__file__).resolve().parents[2] + dockerfile = (root / "docker/core.Dockerfile").read_text() + + assert "mkdir -p /home/thoth/.pi/agent" in dockerfile + assert "chown -R thoth:thoth /home/thoth/.pi" in dockerfile + assert ( + "cp --remove-destination /app/harness/workspaces/local.yaml " + "/app/harness/config/tht.yaml" in dockerfile + ) + assert "ln -sf /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml" not in dockerfile + assert "ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/" not in dockerfile diff --git a/harness/tests/test_psd_local_compose_contract.py b/harness/tests/test_psd_local_compose_contract.py deleted file mode 100644 index c6a37333..00000000 --- a/harness/tests/test_psd_local_compose_contract.py +++ /dev/null @@ -1,133 +0,0 @@ -from pathlib import Path -import shutil -import subprocess - -import yaml - - -class ComposeLoader(yaml.SafeLoader): - pass - - -def _compose_override(loader, node): - if isinstance(node, yaml.MappingNode): - return loader.construct_mapping(node) - return loader.construct_sequence(node) - - -ComposeLoader.add_constructor("!override", _compose_override) - - -def test_psd_overlay_uses_generated_workspace_for_default_and_named_commands(): - root = Path(__file__).resolve().parents[2] - compose = yaml.load( - (root / "deploy/compose.psd-local.yaml.example").read_text(), - Loader=ComposeLoader, - ) - core = compose["services"]["core"] - - assert core["environment"]["THT_CONFIG"] == "/app/harness/config/tht.yaml" - assert core["environment"]["THT_SECRETS_FILE"] == "/run/secrets/thothii.secrets" - for name in ( - "THT_DB_NAME", "THT_DWH_REST_URL", "THT_VEC_REST_URL", - "THT_VEC_WRITE_REST_URL", "THT_OLLAMA_URL", "THT_PROFILE", - "PI_PROVIDER", "PI_MODEL", "PI_THINKING", - ): - assert name in core["environment"] - def target(volume): - if isinstance(volume, dict): - return volume["target"] - parts = volume.rsplit(":", 2) - return parts[-2] if parts[-1] in {"ro", "rw"} else parts[-1] - - targets = {target(volume) for volume in core["volumes"]} - assert "/app/harness/config/tht.yaml" in targets - assert "/app/harness/workspaces/psd.yaml" not in targets - assert "/data/workspaces/psd/config/tht.yaml" not in targets - assert "/data" in targets - assert "/home/thoth/.pi" in targets - assert "/home/thoth/.pi/agent/models.json" in targets - assert "/home/thoth/.pi/agent/settings.json" in targets - assert "/data/evidence" in targets - assert "/run/secrets/thothii.secrets" in targets - assert set(compose["volumes"]) == {"thoth_data", "thoth_pi_config"} - assert core["networks"]["default"]["aliases"] == ["core", "thothii-core"] - frontend = compose["services"]["frontend"] - assert frontend["ports"] == ["127.0.0.1:8099:8080"] - assert frontend["build"]["args"] == { - "VITE_BASE": "/", - "VITE_BACKEND_URL": "/api", - } - assert frontend["networks"] == { - "default": {"aliases": ["frontend", "thothii-frontend"]} - } - assert compose["networks"]["default"] == { - "external": True, - "name": "thothii_default", - } - - -def test_core_image_prepares_the_writable_pi_profile_before_mounting_config_files(): - root = Path(__file__).resolve().parents[2] - dockerfile = (root / "docker/core.Dockerfile").read_text() - - assert "mkdir -p /home/thoth/.pi/agent" in dockerfile - assert "chown -R thoth:thoth /home/thoth/.pi" in dockerfile - assert ( - "cp --remove-destination /app/harness/workspaces/local.yaml " - "/app/harness/config/tht.yaml" in dockerfile - ) - assert "ln -sf /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml" not in dockerfile - assert ( - "ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/psd.yaml" - in dockerfile - ) - - -def test_psd_bootstrap_materializes_the_base_compose_env_file(tmp_path): - source_root = Path(__file__).resolve().parents[2] - root = tmp_path / "ThothII" - (root / "scripts").mkdir(parents=True) - (root / "deploy/workspaces").mkdir(parents=True) - shutil.copy( - source_root / "scripts/bootstrap-local-psd-docker-config.sh", - root / "scripts/bootstrap-local-psd-docker-config.sh", - ) - shutil.copy( - source_root / "deploy/compose.psd-local.yaml.example", - root / "deploy/compose.psd-local.yaml.example", - ) - shutil.copy( - source_root / "deploy/workspaces/psd.yaml.example", - root / "deploy/workspaces/psd.yaml.example", - ) - source_env = tmp_path / "source.env" - source_env.write_text("\n".join([ - "THT_DB_NAME=postgres", - "THT_DWH_REST_URL=https://dwh.invalid/", - "THT_VEC_REST_URL=https://vec.invalid/read/", - "THT_VEC_WRITE_REST_URL=https://vec.invalid/write/", - "THT_DWH_API_KEY=dwh", - "THT_VEC_API_KEY=reader", - "THT_VEC_WRITE_API_KEY=writer", - "", - ])) - workspace = tmp_path / "workspace" - workspace.mkdir() - auth = tmp_path / "auth.json" - auth.write_text('{"zai":{"key":"model"}}') - - subprocess.run( - [ - "sh", str(root / "scripts/bootstrap-local-psd-docker-config.sh"), - str(source_env), str(workspace), str(auth), - ], - check=True, - capture_output=True, - text=True, - ) - - assert (root / "deploy/thothii.env").is_file() - assert "THT_SECRETS_FILE=./deploy/secrets/thothii.secrets" in ( - root / ".env" - ).read_text() diff --git a/scripts/bootstrap-local-psd-docker-config.sh b/scripts/bootstrap-local-psd-docker-config.sh deleted file mode 100644 index 81c54e04..00000000 --- a/scripts/bootstrap-local-psd-docker-config.sh +++ /dev/null @@ -1,70 +0,0 @@ -#!/bin/sh -set -eu - -root=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd) -source_env=${1:-"$root/../../harness/.env"} -workspace=${2:-"$root/../../../tht-workspace-psd"} -auth_file=${3:-"$HOME/.pi/agent/auth.json"} - -value() { - awk -F= -v key="$1" '$1 == key { sub(/^[^=]*=/, ""); sub(/[[:space:]].*$/, ""); print; exit }' "$source_env" -} -required() { - result=$(value "$1") - [ -n "$result" ] || { echo "missing $1 in local source configuration" >&2; exit 2; } - printf '%s' "$result" -} - -test -f "$source_env" -test -d "$workspace" -test -f "$auth_file" -ca=$(value THT_SSL_CA) -[ -z "$ca" ] || test -f "$ca" -model_key=$(jq -er '.zai.key' "$auth_file") -test -n "$model_key" - -umask 077 -mkdir -p "$root/deploy/secrets" "$root/deploy/workspaces" -: >"$root/deploy/thothii.env" -cp "$root/deploy/compose.psd-local.yaml.example" "$root/deploy/compose.psd-local.yaml" -cp "$root/deploy/workspaces/psd.yaml.example" "$root/deploy/workspaces/psd.yaml" -cat >"$root/.env" <"$root/deploy/secrets/thothii.secrets" <>"$root/deploy/compose.psd-local.yaml" <>"$root/deploy/secrets/thothii.secrets" -else - printf '%s\n' 'THT_CA=/etc/ssl/certs/ca-certificates.crt' >>"$root/deploy/secrets/thothii.secrets" -fi -chmod 600 "$root/.env" "$root/deploy/thothii.env" "$root/deploy/secrets/thothii.secrets" -echo "Local PSD Docker configuration materialized without printing secret values." diff --git a/scripts/docker-smoke.sh b/scripts/docker-smoke.sh index cfe357ca..53b3064c 100755 --- a/scripts/docker-smoke.sh +++ b/scripts/docker-smoke.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Smoke test del deploy standalone ThothII (core + frontend). -# Usa docker-compose.dev.yml (rete propria, porte host). Non tocca il portale. -# Prereq: deploy/thothii.env popolato + ruoli DB creati + pi-config + settings.json. +# Usa docker-compose.dev.yml (rete propria, porte host). +# Prereq: deploy/thothii.env popolato, endpoint esterni configurati e profilo Pi locale. set -euo pipefail cd "$(dirname "$0")/.." diff --git a/scripts/run-stack.sh b/scripts/run-stack.sh index 4befb659..3bf95077 100755 --- a/scripts/run-stack.sh +++ b/scripts/run-stack.sh @@ -1,68 +1,20 @@ #!/usr/bin/env bash -# run-stack.sh — avvia i 3 layer reali di ThothII per la validazione end-to-end. +# run-stack.sh — avvia lo stack Compose locale di ThothII in primo piano. # -# frontend (browser) → backend (Fastify :8787) → pi --mode rpc (GLM 5.2) → tht/harness → DWH +# Il core include Pi; DWH, vector DB, embedding e LLM sono endpoint esterni configurati +# in deploy/env/local.env. Non richiede un eseguibile Pi sull'host. # -# Prerequisiti: VPN attiva, `pi` su PATH (GLM 5.2 configurato), harness/.env popolato, -# harness/config/tht.yaml -> workspace cliente, deps installate nei 3 progetti. -# -# Uso: ./scripts/run-stack.sh -# Stop: Ctrl-C (termina backend + frontend; i processi pi figli del backend muoiono con esso). +# Preparazione: cp deploy/env/local.env.example deploy/env/local.env e compilare i valori. +# Uso: ./scripts/run-stack.sh [argomenti aggiuntivi per docker compose up] set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)" -HARNESS="$ROOT/harness" -BACKEND="$ROOT/backend" -FRONTEND="$ROOT/frontend" -THT_BIN="$HARNESS/.venv/bin/tht" -BACKEND_PORT="${BACKEND_PORT:-8787}" -FRONTEND_PORT="${FRONTEND_PORT:-5173}" +LOCAL_ENV_FILE="${THT_LOCAL_ENV_FILE:-$ROOT/deploy/env/local.env}" -# --- preflight --------------------------------------------------------------- -[ -f "$HARNESS/.env" ] || { echo "ERRORE: $HARNESS/.env mancante (credenziali DWH/vector)"; exit 1; } -[ -x "$THT_BIN" ] || { echo "ERRORE: $THT_BIN non trovato (esegui: cd harness && python -m venv .venv && pip install -e .)"; exit 1; } -command -v pi >/dev/null || { echo "ERRORE: 'pi' non sul PATH (configura @earendil-works/pi-coding-agent con GLM 5.2)"; exit 1; } -[ -e "$HARNESS/config/tht.yaml" ] || { echo "ERRORE: $HARNESS/config/tht.yaml mancante (symlink al workspace)"; exit 1; } +[[ -f "$LOCAL_ENV_FILE" ]] || { + echo "ERRORE: $LOCAL_ENV_FILE mancante. Copia deploy/env/local.env.example e configura gli endpoint." >&2 + exit 1 +} -# Carica le variabili del DWH/vector nell'ambiente: il backend le passa a pi e a tht. -set -a; . "$HARNESS/.env"; set +a - -# tht deve essere raggiungibile dal processo pi che il backend spawna. -export PATH="$HARNESS/.venv/bin:$PATH" - -echo "== ThothII stack ==" -echo " harness : $HARNESS (tht: $THT_BIN)" -echo " backend : http://localhost:$BACKEND_PORT" -echo " frontend: http://localhost:$FRONTEND_PORT" -echo " pi : $(command -v pi)" -echo - -# --- avvio ------------------------------------------------------------------- -pids=() -cleanup() { echo; echo "Arresto stack..."; for p in "${pids[@]}"; do kill "$p" 2>/dev/null || true; done; } -trap cleanup EXIT INT TERM - -# Backend: usa il pi reale + il tht del venv, cwd harness per lo spawn di pi. -( - cd "$BACKEND" - PORT="$BACKEND_PORT" \ - THT_HARNESS_DIR="$HARNESS" \ - THT_BIN="$THT_BIN" \ - PI_BIN="pi" \ - AUTH_MODE="none" \ - THT_DWH_PRECHECK="1" \ - npm run dev -) & -pids+=($!) - -# Frontend: il browser usa sempre /api; Vite lo inoltra al backend locale. -( - cd "$FRONTEND" - THT_FRONTEND_API_UPSTREAM="http://localhost:$BACKEND_PORT" \ - npm run dev -- --port "$FRONTEND_PORT" -) & -pids+=($!) - -echo "Stack avviato. Apri http://localhost:$FRONTEND_PORT e crea una nuova domanda." -echo "Ctrl-C per fermare." -wait +exec docker compose --env-file "$LOCAL_ENV_FILE" \ + -f "$ROOT/compose.yaml" -f "$ROOT/deploy/compose.local.yaml" up --build "$@" diff --git a/scripts/test-external-compose-lifecycle.sh b/scripts/test-external-compose-lifecycle.sh index 5fca2274..8b645ae7 100755 --- a/scripts/test-external-compose-lifecycle.sh +++ b/scripts/test-external-compose-lifecycle.sh @@ -6,6 +6,9 @@ project="thothii-external-lifecycle-$$" cleanup() { docker compose --project-name "$project" --profile external down --volumes >/dev/null 2>&1 || true; } trap cleanup EXIT HUP INT TERM +export THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git +export PI_AUTH_FILE=/dev/null + unset THT_VECTOR_BOOTSTRAP_PASSWORD_SECRET_FILE THT_VECTOR_MIGRATOR_PASSWORD_SECRET_FILE unset THT_VECTOR_READER_PASSWORD_SECRET_FILE THT_VECTOR_WRITER_PASSWORD_SECRET_FILE rendered=$(docker compose --project-name "$project" --profile external config) @@ -13,6 +16,10 @@ if printf '%s' "$rendered" | grep -q 'THT_VECTOR_.*PASSWORD_FILE\|vector_.*passw echo "external config contains local vector secret references" >&2 exit 1 fi +if printf '%s' "$rendered" | grep -Eqi 'omics_portal|chirone|localllm_default|/home/chirone|datamart-builder'; then + echo "external config contains deployment-specific coupling" >&2 + exit 1 +fi docker compose --project-name "$project" --profile external up --build --wait core core=$(docker compose --project-name "$project" --profile external ps -q core) inspect=$(docker inspect "$core") @@ -20,4 +27,8 @@ if printf '%s' "$inspect" | grep -q 'THT_VECTOR_.*PASSWORD_FILE\|/run/secrets/ve echo "external core inspect contains local vector secret references" >&2 exit 1 fi +if printf '%s' "$inspect" | grep -Eqi 'omics_portal|chirone|localllm_default|/home/chirone|datamart-builder'; then + echo "external core inspect contains deployment-specific coupling" >&2 + exit 1 +fi echo "external core lifecycle without local vector secrets passed." diff --git a/scripts/test-local-dev-routing.sh b/scripts/test-local-dev-routing.sh index 717853f9..b181b1da 100755 --- a/scripts/test-local-dev-routing.sh +++ b/scripts/test-local-dev-routing.sh @@ -54,13 +54,13 @@ if [ "$response" != '{"backend":"fastify","path":"/health"}' ]; then exit 1 fi -if ! rg -Fq 'THT_FRONTEND_API_UPSTREAM="http://localhost:$BACKEND_PORT"' "$ROOT/scripts/run-stack.sh"; then - echo "run-stack.sh must configure the Vite internal upstream from BACKEND_PORT" >&2 +if ! rg -Fq -- '-f "$ROOT/compose.yaml" -f "$ROOT/deploy/compose.local.yaml" up --build "$@"' "$ROOT/scripts/run-stack.sh"; then + echo "run-stack.sh must start the base+local Compose stack" >&2 exit 1 fi -if rg -q 'VITE_BACKEND_URL' "$ROOT/scripts/run-stack.sh"; then - echo "run-stack.sh must keep the browser base on /api" >&2 +if rg -q 'command -v pi|PI_BIN="pi"|PI_BIN=pi' "$ROOT/scripts/run-stack.sh"; then + echo "run-stack.sh must not require a host Pi binary" >&2 exit 1 fi diff --git a/scripts/test-no-deployment-coupling.sh b/scripts/test-no-deployment-coupling.sh new file mode 100755 index 00000000..510cf348 --- /dev/null +++ b/scripts/test-no-deployment-coupling.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "$0")/.." + +content_targets=( + .dockerignore + compose.yaml + docker-compose.dev.yml + deploy + docker + frontend/vite.config.ts + README.md + docs/install + docs/installazione-docker-4-contesti.md + .env.example + scripts/run-stack.sh + scripts/docker-smoke.sh +) + +matches=$( + rg -n -i \ + -g '!deploy/workspaces/**' \ + -g '!docker/session-migrate.sh' \ + -g '!docker/cutover-legacy-sessions.sh' \ + -g '!docker/smoke/**' \ + 'omics_portal|chirone|localllm_default|datamart-builder|compose\.production\.yaml|compose\.psd-local\.yaml' \ + "${content_targets[@]}" || true +) + +runtime_psd_matches=$( + rg -n -i \ + -g '!deploy/workspaces/**' \ + -g '!docker/session-migrate.sh' \ + -g '!docker/cutover-legacy-sessions.sh' \ + -g '!docker/smoke/**' \ + '\bpsd\b' \ + .dockerignore compose.yaml docker-compose.dev.yml deploy docker frontend/vite.config.ts \ + .env.example scripts/run-stack.sh scripts/docker-smoke.sh || true +) + +offenders=() +for superseded_file in \ + deploy/compose.production.yaml \ + deploy/compose.psd-local.yaml.example \ + deploy/compose.psd-local.yaml \ + scripts/bootstrap-local-psd-docker-config.sh \ + harness/tests/test_psd_local_compose_contract.py +do + [[ ! -e "$superseded_file" ]] || offenders+=("$superseded_file (forbidden active deployment filename)") +done + +if [[ -n "$matches" ]]; then + while IFS= read -r match; do + offenders+=("$match") + done <<<"$matches" +fi + +if [[ -n "$runtime_psd_matches" ]]; then + while IFS= read -r match; do + offenders+=("$match") + done <<<"$runtime_psd_matches" +fi + +if rg -n 'command -v pi|PI_BIN="pi"|PI_BIN=pi' scripts/run-stack.sh >/dev/null; then + offenders+=("scripts/run-stack.sh (requires a host Pi binary)") +fi + +if ((${#offenders[@]})); then + printf '%s\n' "active deployment coupling found:" >&2 + printf '%s\n' "${offenders[@]}" >&2 + exit 1 +fi + +echo "no active PSD, Chirone, or portal deployment coupling found." diff --git a/scripts/test-pi-user-auth-compose.sh b/scripts/test-pi-user-auth-compose.sh index a183730f..c50cb10c 100755 --- a/scripts/test-pi-user-auth-compose.sh +++ b/scripts/test-pi-user-auth-compose.sh @@ -9,7 +9,8 @@ auth_file="$tmp/auth.json" printf '%s\n' '{}' >"$auth_file" chmod 0600 "$auth_file" -rendered=$(PI_AUTH_FILE="$auth_file" docker compose config) +rendered=$(THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \ + PI_AUTH_FILE="$auth_file" docker compose config) printf '%s\n' "$rendered" | grep -q "source: $auth_file" printf '%s\n' "$rendered" | grep -q 'target: /home/thoth/.pi/agent/auth.json' printf '%s\n' "$rendered" | grep -A4 'target: /home/thoth/.pi/agent/auth.json' \ @@ -29,6 +30,7 @@ assert settings["enabledModels"] == [ PY grep -q '^ARG PI_VERSION=0.80.3$' docker/core.Dockerfile -grep -q '^PI_AUTH_FILE=$auth_file$' scripts/bootstrap-local-psd-docker-config.sh +grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/local.env.example +grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/server.env.example echo "Pi user-auth Compose contract passed." diff --git a/scripts/test-verify-line-endings.sh b/scripts/test-verify-line-endings.sh index c68ea4cc..5ea5c443 100755 --- a/scripts/test-verify-line-endings.sh +++ b/scripts/test-verify-line-endings.sh @@ -44,4 +44,17 @@ printf 'FROM scratch\n' > "$fixture_root/Dockerfile" "$repo_root/scripts/verify-line-endings.sh" "$fixture_root" +git_fixture="$fixture_root/git-worktree" +mkdir -p "$git_fixture" +git -C "$git_fixture" init -q +printf 'tracked then deleted\n' >"$git_fixture/deleted.md" +git -C "$git_fixture" add deleted.md +rm "$git_fixture/deleted.md" + +git_output="$(cd "$git_fixture" && "$repo_root/scripts/verify-line-endings.sh" 2>&1)" +if grep -Fq 'No such file or directory' <<<"$git_output"; then + echo "line-ending verifier tried to read a tracked deletion" >&2 + exit 1 +fi + echo "line-ending verifier tests passed" diff --git a/scripts/verify-line-endings.sh b/scripts/verify-line-endings.sh index 787c9967..b5ba2fc1 100755 --- a/scripts/verify-line-endings.sh +++ b/scripts/verify-line-endings.sh @@ -23,6 +23,7 @@ root="$(cd "$root" && pwd)" offenders=() while IFS= read -r -d '' file; do + [[ -f "$file" ]] || continue case "$file" in *.ps1) continue ;; esac