refactor: remove portal deployment coupling

This commit is contained in:
2026-08-05 06:58:19 +02:00
parent fd1fd2f802
commit 5d037e97c4
25 changed files with 265 additions and 452 deletions
+4 -1
View File
@@ -15,7 +15,10 @@
!deploy/env/*.env.example !deploy/env/*.env.example
deploy/thothii.env deploy/thothii.env
deploy/secrets/ deploy/secrets/
harness/workspaces/psd.yaml harness/workspaces/*.yaml
!harness/workspaces/local.yaml
!harness/workspaces/tht.example.yaml
!harness/workspaces/tht-test.yaml
**/*.log **/*.log
**/.DS_Store **/.DS_Store
coverage/ coverage/
+4 -2
View File
@@ -11,8 +11,10 @@ detail. Design history lives in `docs/superpowers/specs/` and `docs/superpowers/
## Commands ## Commands
The repo has three independently-built layers. Run the **full stack** (real Pi + DWH, needs The repo has three independently-built layers. Run the local Docker stack with
VPN + `harness/.env` + `pi` on PATH) with `./scripts/run-stack.sh` (frontend :5173 → backend :8787). `./scripts/run-stack.sh` after creating `deploy/env/local.env`; it starts the base+local Compose
profile, whose core image contains Pi. DWH, vector DB, embedding, and LLM remain external
configuration endpoints.
**harness/** (Python `tht` CLI + Pi gate extension) **harness/** (Python `tht` CLI + Pi gate extension)
- Install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` (puts `tht` on PATH) - Install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` (puts `tht` on PATH)
+18 -2
View File
@@ -1,8 +1,24 @@
# ThothII — Project State # ThothII — Project State
> Starting-point snapshot for new sessions. Last updated: 2026-07-23 (session summary redesign live). > Starting-point snapshot for new sessions. Last updated: 2026-08-05 (portable deployment decoupled).
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work. > Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
## Portable deployment decoupling — LIVE 2026-08-05
- **Mandatory stack.** The supported Compose stack is exactly `frontend` plus `core`; use the
base file with `deploy/compose.local.yaml` or `deploy/compose.server.yaml`. `run-stack.sh`
invokes the base+local Compose command and the core image provides Pi, so no host Pi binary is
part of the launch contract.
- **External boundaries.** DWH, vector DB, embedding, LLM, and reverse-proxy services are
external configurable endpoints even when deployed on the same infrastructure. The two
superseded PSD/portal deployment overlays were removed. Workspace descriptors and migration
utilities remain separate from deployment runtime configuration.
- **Legacy PSD deployment ruling.** The PSD bootstrap was deleted because it generated the
retired overlay and was therefore deployment machinery, not a data migration utility. Its
remaining live contract checks were renamed for the generic local Compose profile. The coupling
gate rejects stale active deployment filenames and content while deliberately excluding
historical plans/specs, canonical workspace descriptors, and non-runtime migration helpers.
## Portable Git workspace registry — source integration (2026-08-04) ## Portable Git workspace registry — source integration (2026-08-04)
- **Source of truth and scope.** The canonical workspace repository is a generic Git remote, - **Source of truth and scope.** The canonical workspace repository is a generic Git remote,
@@ -103,7 +119,7 @@
release; never re-enable filesystem persistence, restore the archive into production, or release; never re-enable filesystem persistence, restore the archive into production, or
dual-write during rollback. dual-write during rollback.
## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12 ## Historical deployment — Docker locale (Profile A, co-located) — superseded 2026-08-05
ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale). ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale).
+37 -56
View File
@@ -4,57 +4,37 @@ ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fasti
core. The portable deployment runs exactly two application services; data services remain core. The portable deployment runs exactly two application services; data services remain
external in this profile. external in this profile.
## Docker Compose: one-command startup ## Docker Compose: local startup
Requirements: Docker Engine with Compose v2. The default project starts only the two Requirements: Docker Engine with Compose v2. The mandatory stack is exactly the `core` and
application images; DWH, vector and embedding services can be remote or supplied by an `frontend` application images. DWH, vector DB, embedding, and LLM services are external,
optional overlay. configurable endpoints—even when they are co-located with ThothII.
From a fresh clone, run these commands from the repository root: From a fresh clone, run these commands from the repository root:
```sh ```sh
cp .env.example .env cp deploy/env/local.env.example deploy/env/local.env
cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets # Edit deploy/env/local.env, including PI_AUTH_FILE and the external endpoint URLs.
chmod 600 deploy/secrets/thothii.secrets docker compose --env-file deploy/env/local.env \
# Edit .env (non-secret endpoints) and deploy/secrets/thothii.secrets (KEY=VALUE lines). -f compose.yaml -f deploy/compose.local.yaml up --build -d
docker compose up --build -d
``` ```
The root `.env` is loaded automatically by Compose. It defaults to `compose.yaml`, an empty `./scripts/run-stack.sh` runs this same base+local command in the foreground. The core image
profile, and `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`; no `--env-file`, `-f`, or contains its Pi runtime; no host `pi` executable is used. For a server installation, copy and
`--profile` flag is required for the normal installation. Add or edit YAML workspace descriptors fill `deploy/env/server.env.example`, then use `-f compose.yaml -f deploy/compose.server.yaml`.
under `deploy/workspaces/`; they are mounted read-only and relative `roots` resolve beneath Workspace descriptors come from the Git remote configured by `THT_WORKSPACE_GIT_REMOTE`; their
`/data/workspaces/<workspace-name>`. Open <http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in runtime endpoint and secret bindings remain installation-local. Open
`.env` to choose another loopback port). <http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in `deploy/env/local.env` to choose another
loopback port).
The bundle contains only values, one per line (`THT_MODEL_API_KEY=...`, DWH/vector keys, and Credentials and certificates are local protected files. Do not put them in environment examples,
the optional local-vector passwords). It is ignored by Git and never copied into either image. workspace YAML, URLs, or Compose interpolation values. The optional `local-vector` and
Do not put credentials in `.env`, workspace YAML, URLs, or Compose interpolation values. preprocessing overlays are development presets; they do not change the two-service mandatory
stack or the external-endpoint contract.
### Optional overlays Application state is split across the named `settings`, `pi-state`, `workspace-registry`, and
`sessions` volumes. `docker compose down` keeps them. Only an explicit destructive command such
Overlays are selected in `.env`, so the operational command remains the same. On Unix-like as `docker compose down --volumes` removes them.
systems use `:` between files; on Windows use `;`:
```dotenv
# Remote DWH/vector/embedding services with authenticated reverse proxy:
COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml
COMPOSE_PROFILES=
# Local pgvector (Mac/Windows or a standalone application server):
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml
COMPOSE_PROFILES=local-vector
```
After changing `.env`, apply the selected configuration with `docker compose up --build -d`.
Preprocessing is an explicit opt-in preset: append
`deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml` and set
`COMPOSE_PROFILES=local-vector,preprocess`; then run the job with
`docker compose run --rm preprocess-evidence` or `preprocess-dwh`.
Application state, including settings, sessions, artifacts, and indexes, lives in the named
`thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an
explicit destructive command such as `docker compose down --volumes` removes it.
The frontend depends on the core health check and proxies `/health` and `/api/*` to it. The The frontend depends on the core health check and proxies `/health` and `/api/*` to it. The
application health endpoint intentionally checks process readiness only; external dependency application health endpoint intentionally checks process readiness only; external dependency
@@ -110,17 +90,18 @@ application state; passwords are selected at runtime and are never passed as URL
## Preprocessing jobs and S3 Evidence ## Preprocessing jobs and S3 Evidence
The included job workspaces target the local-vector profile. Put the four local-vector password The included job workspaces target the optional local-vector profile. Put the four local-vector
keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then select password keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then
the preprocessing preset in `.env`: run the explicit preprocessing preset:
```dotenv ```sh
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml:deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml docker compose --env-file deploy/env/local.env \
COMPOSE_PROFILES=local-vector,preprocess -f compose.yaml -f deploy/compose.local.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \
--profile local-vector --profile preprocess run --rm preprocess-evidence
``` ```
Run `docker compose run --rm preprocess-evidence` or Replace the final service with `preprocess-dwh` when required. The overlay makes each job wait for the vector
`docker compose run --rm preprocess-dwh`. The overlay makes each job wait for the vector
database health check, role reconciliation, and a successful migration; no separate database database health check, role reconciliation, and a successful migration; no separate database
startup or migration command is required. startup or migration command is required.
@@ -183,8 +164,8 @@ with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts thi
rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other
auth mode fails during core startup. auth mode fails during core startup.
Production credentials use the one Compose secret bundle, not `.env`. Put the required keys in Production credentials use the one Compose secret bundle, not an environment example. Put the
`deploy/secrets/thothii.secrets` and select the production overlay in `.env`: required keys in `deploy/secrets/thothii.secrets` for the selected base+server installation:
```dotenv ```dotenv
THT_MODEL_API_KEY=replace-me THT_MODEL_API_KEY=replace-me
@@ -255,10 +236,10 @@ session store without upstream authentication, direct DB host/name/runtime user/
The migrator independently rejects every other TLS mode before reading its password secret or The migrator independently rejects every other TLS mode before reading its password secret or
constructing a database URL. constructing a database URL.
Perform the cutover in one maintenance window, with the Task 4 portal proxy headers and Task 5 Perform the cutover in one maintenance window, with the upstream identity-proxy headers and
backend principal parser deployed together. Neither change is safe to deploy independently: Task backend principal parser deployed together. Neither change is safe to deploy independently: the
4 clears the legacy identity header and Task 5 rejects it. Drain/stop active Pi work, enable a proxy clears the legacy identity header and the backend rejects it. Drain/stop active Pi work,
maintenance response at the portal, then run the migrator once and inspect its pristine JSON: enable a maintenance response at the proxy, then run the migrator once and inspect its pristine JSON:
```sh ```sh
docker compose -f compose.yaml -f deploy/compose.session-server.yaml \ docker compose -f compose.yaml -f deploy/compose.session-server.yaml \
-11
View File
@@ -1,11 +0,0 @@
services:
core:
environment:
AUTH_MODE: upstream
THOTH_PUBLIC_EXPOSURE: "true"
THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME}
THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL}
THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL}
THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL}
THT_DOCS_ROOT: ${THT_DOCS_ROOT:-/data/workspaces/example/evidence-source}
THT_SECRETS_FILE: /run/secrets/thothii.secrets
-52
View File
@@ -1,52 +0,0 @@
services:
core:
environment:
AUTH_MODE: ${AUTH_MODE:-none}
THOTH_PUBLIC_EXPOSURE: ${THOTH_PUBLIC_EXPOSURE:-false}
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
PI_PROVIDER: ${PI_PROVIDER:?set PI_PROVIDER}
PI_MODEL: ${PI_MODEL:?set PI_MODEL}
PI_THINKING: ${PI_THINKING:-medium}
THT_PROFILE: ${THT_PROFILE:-workstation}
THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME}
THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL}
THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL}
THT_VEC_WRITE_REST_URL: ${THT_VEC_WRITE_REST_URL:?set THT_VEC_WRITE_REST_URL}
THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL}
THT_SECRETS_FILE: /run/secrets/thothii.secrets
THT_DOCS_ROOT: /data/workspaces/psd
THT_CONFIG: /app/harness/config/tht.yaml
extra_hosts:
- host.docker.internal:host-gateway
networks: !override
default:
aliases: [core, thothii-core]
volumes:
- thoth_data:/data
- thoth_pi_config:/home/thoth/.pi
- ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro
- ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro
- ${THT_SECRETS_FILE:?set THT_SECRETS_FILE}:/run/secrets/thothii.secrets:ro
- ${THT_PSD_WORKSPACE_HOST_PATH:?set THT_PSD_WORKSPACE_HOST_PATH}/evidence:/data/evidence:ro
- ./deploy/workspaces/psd.yaml:/app/harness/config/tht.yaml:ro
- ${THT_PSD_WORKSPACE_HOST_PATH:?set THT_PSD_WORKSPACE_HOST_PATH}:/data/workspaces/psd
frontend:
build:
args:
VITE_BASE: /
VITE_BACKEND_URL: /api
ports:
- "127.0.0.1:8099:8080"
networks: !override
default:
aliases: [frontend, thothii-frontend]
volumes:
thoth_data:
thoth_pi_config:
networks:
default:
external: true
name: thothii_default
+2 -1
View File
@@ -18,8 +18,9 @@ THT_VEC_PASSWORD=__CHANGE_ME__
THT_OLLAMA_URL=http://host.docker.internal:11434 THT_OLLAMA_URL=http://host.docker.internal:11434
# --- Backend --- # --- Backend ---
AUTH_MODE=none # none | mock | oidc (in embedded l'auth è al bordo del portale) AUTH_MODE=none # none | mock | oidc (upstream auth is enforced at the proxy boundary)
MAX_PI_PROCESSES=4 MAX_PI_PROCESSES=4
THT_DEV_EVIDENCE_HOST_PATH=/absolute/path/to/evidence
# --- Git-backed workspace registry (no secret values belong in this file) --- # --- Git-backed workspace registry (no secret values belong in this file) ---
THT_WORKSPACE_REGISTRY_ROOT=/data/workspace-registry THT_WORKSPACE_REGISTRY_ROOT=/data/workspace-registry
+6 -4
View File
@@ -1,4 +1,4 @@
# ThothII — deploy STANDALONE locale (dev / smoke test, senza portale). # ThothII — deploy STANDALONE locale (dev / smoke test).
# Rete propria + porte host per ispezione diretta. # Rete propria + porte host per ispezione diretta.
# docker compose -f docker-compose.dev.yml up -d --build # docker compose -f docker-compose.dev.yml up -d --build
# frontend: http://localhost:8090 backend: http://localhost:8787 # frontend: http://localhost:8090 backend: http://localhost:8787
@@ -40,10 +40,10 @@ services:
extra_hosts: extra_hosts:
- "host.docker.internal:host-gateway" - "host.docker.internal:host-gateway"
volumes: volumes:
- /home/chirone/thothii-data:/data - dev-data:/data
- workspace-registry:/data/workspace-registry - workspace-registry:/data/workspace-registry
- /home/chirone/thothii-data/pi-config:/home/thoth/.pi - dev-pi-state:/home/thoth/.pi
- /home/chirone/chirone/etl/docs/evidence:/data/evidence:ro - ${THT_DEV_EVIDENCE_HOST_PATH:-./evidence}:/data/evidence:ro
- ${THT_WORKSPACE_GIT_CREDENTIALS_FILE:-/dev/null}:/run/secrets/workspace-registry-git-credentials:ro - ${THT_WORKSPACE_GIT_CREDENTIALS_FILE:-/dev/null}:/run/secrets/workspace-registry-git-credentials:ro
- ${THT_WORKSPACE_GIT_CA_FILE:-/etc/ssl/certs/ca-certificates.crt}:/run/secrets/workspace-registry-git-ca:ro - ${THT_WORKSPACE_GIT_CA_FILE:-/etc/ssl/certs/ca-certificates.crt}:/run/secrets/workspace-registry-git-ca:ro
- ${THT_WORKSPACE_GIT_SSH_KEY_FILE:-/dev/null}:/run/secrets/workspace-registry-git-ssh-key:ro - ${THT_WORKSPACE_GIT_SSH_KEY_FILE:-/dev/null}:/run/secrets/workspace-registry-git-ssh-key:ro
@@ -73,4 +73,6 @@ networks:
driver: bridge driver: bridge
volumes: volumes:
dev-data:
dev-pi-state:
workspace-registry: workspace-registry:
+3 -4
View File
@@ -64,11 +64,10 @@ RUN python -m venv /opt/venv \
&& /opt/venv/bin/pip install --no-cache-dir --upgrade pip \ && /opt/venv/bin/pip install --no-cache-dir --upgrade pip \
&& (cd /app/harness && /opt/venv/bin/pip install --no-cache-dir .) \ && (cd /app/harness && /opt/venv/bin/pip install --no-cache-dir .) \
&& cp /app/harness/workflow.yaml /opt/venv/lib/python3.12/site-packages/workflow.yaml && cp /app/harness/workflow.yaml /opt/venv/lib/python3.12/site-packages/workflow.yaml
# Default locale e alias PSD convergono sul file canonico. Il CLI onora anche # Il workspace locale predefinito converge sul file canonico. Il CLI onora anche THT_CONFIG,
# THT_CONFIG, quindi cambiare CWD non cambia l'identita' dello workspace. # quindi cambiare CWD non cambia l'identita' dello workspace.
RUN mkdir -p /app/harness/config \ RUN mkdir -p /app/harness/config \
&& cp --remove-destination /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml \ && cp --remove-destination /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml
&& ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/psd.yaml
# PiProcessManager (backend) prepende harnessDir/.venv/bin al PATH del child Pi → symlink al venv reale # PiProcessManager (backend) prepende harnessDir/.venv/bin al PATH del child Pi → symlink al venv reale
RUN ln -s /opt/venv /app/harness/.venv RUN ln -s /opt/venv /app/harness/.venv
+2 -4
View File
@@ -1,6 +1,4 @@
# nginx per thothii-frontend: serve la SPA (modalità standalone) e reverse-proxy /api -> core. # nginx per thothii-frontend: serve la SPA e inoltra /api al core sulla rete Compose.
# In modalità embedded il portale proxya /datamart-builder/assets/ qui (solo asset statici);
# il blocco /api non è usato in embedded (il portale hita core direttamente).
server { server {
listen 8080; listen 8080;
server_name _; server_name _;
@@ -29,7 +27,7 @@ server {
chunked_transfer_encoding on; chunked_transfer_encoding on;
} }
# manifest.json servito (lo legge il template tag Django in embedded) # manifest.json è servito come JSON.
location = /manifest.json { location = /manifest.json {
default_type application/json; default_type application/json;
} }
+3 -1
View File
@@ -54,6 +54,8 @@ Il frontend renderizza questi widget-descriptor (registro in `src/widgets/`); il
## Come si lancia lo stack ## Come si lancia lo stack
Lo **stack completo** (Pi reale + DWH reale, serve VPN + `harness/.env` + `pi` sul PATH) si avvia con `./scripts/run-stack.sh` (frontend `:5173` → backend `:8787`). Lo stack locale si avvia con `./scripts/run-stack.sh`, dopo aver creato
`deploy/env/local.env` da `deploy/env/local.env.example`. Il core Compose include Pi; DWH,
vector DB, embedding e LLM sono endpoint esterni configurati nel file locale.
Comandi per singolo layer, test, lint: vedi il file `CLAUDE.md` nella radice del repo (guida operativa per Claude Code, tenuta sincronizzata con questa pagina). Comandi per singolo layer, test, lint: vedi il file `CLAUDE.md` nella radice del repo (guida operativa per Claude Code, tenuta sincronizzata con questa pagina).
@@ -45,13 +45,13 @@ services:
- source: session_ca - source: session_ca
target: session_ca.pem target: session_ca.pem
networks: networks:
- portal - upstream
restart: unless-stopped restart: unless-stopped
networks: networks:
portal: upstream:
external: true external: true
name: ${THT_PORTAL_NETWORK:-omics_portal_omics_network} name: ${THT_UPSTREAM_NETWORK:-thothii-upstream}
secrets: secrets:
session_runtime_password: session_runtime_password:
+25 -37
View File
@@ -14,27 +14,28 @@ Servono Docker Engine/Compose v2 su Linux oppure Docker Desktop su macOS/Windows
```sh ```sh
git clone <URL-REPOSITORY> ThothII git clone <URL-REPOSITORY> ThothII
cd ThothII cd ThothII
cp .env.example .env cp deploy/env/local.env.example deploy/env/local.env
mkdir -p deploy/secrets deploy/workspaces
cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets
chmod 600 deploy/secrets/thothii.secrets
``` ```
Modificare **solo** questi file interni al clone: Modificare **solo** questi file interni al clone:
| File | Cosa contiene | | File | Cosa contiene |
|---|---| |---|---|
| `.env` | endpoint, database, provider, `COMPOSE_FILE` e `COMPOSE_PROFILES`; mai password/token | | `deploy/env/local.env` | endpoint, database e path Pi locali; mai password/token |
| `deploy/secrets/thothii.secrets` | un bundle `NOME=VALORE`, mode host `0600` o `0400` | | file protetti locali | credenziali e certificati, indicati dai binding del workspace |
| `deploy/workspaces/<nome>.yaml` | adapter, endpoint non riservati, `roots` ed Evidence | | `deploy/workspaces/<nome>.yaml` | adapter, endpoint non riservati, `roots` ed Evidence |
Il file `.env` viene caricato automaticamente da Docker Compose perché è nella radice del progetto. Il valore predefinito è `COMPOSE_FILE=compose.yaml`, con profili vuoti e `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`. Perciò, dopo aver compilato `.env`, il bundle e almeno il workspace, l'avvio normale è sempre: Compilare `deploy/env/local.env`, incluso `PI_AUTH_FILE`, con gli endpoint esterni. L'avvio
normale usa esplicitamente il file base e l'overlay locale:
```sh ```sh
docker compose up --build -d docker compose --env-file deploy/env/local.env \
-f compose.yaml -f deploy/compose.local.yaml up --build -d
``` ```
Non occorre usare `--env-file`, `-f` o `--profile` per questa installazione. Verificare lo stato con `docker compose ps` e aprire <http://127.0.0.1:8080>. `docker compose down` conserva il volume `thoth_data`; usare `down --volumes` solo per un ambiente effimero. Verificare lo stato con lo stesso comando Compose e aprire <http://127.0.0.1:8080>. Il core
include Pi; il binario Pi non deve essere installato sull'host. `docker compose down` conserva i
volumi; usare `down --volumes` solo per un ambiente effimero.
### Formato del bundle unico ### Formato del bundle unico
@@ -55,21 +56,13 @@ Inserire solo le chiavi necessarie al profilo scelto. Il bundle viene montato in
Una catena CA PEM **non può essere inserita nel bundle**: contiene whitespace e viene rifiutata dal parser. Se un endpoint usa una CA privata, conservarla nel secret manager/host e aggiungere un override Compose revisionato che monti il file in `/run/secrets/ca-chain.pem` e imposti `THT_SSL_CA` (o il parametro dell'adapter). Il clone base non crea quel mount: questa è una limitazione intenzionale da considerare in fase di deployment. Una catena CA PEM **non può essere inserita nel bundle**: contiene whitespace e viene rifiutata dal parser. Se un endpoint usa una CA privata, conservarla nel secret manager/host e aggiungere un override Compose revisionato che monti il file in `/run/secrets/ca-chain.pem` e imposti `THT_SSL_CA` (o il parametro dell'adapter). Il clone base non crea quel mount: questa è una limitazione intenzionale da considerare in fase di deployment.
### Overlay opzionali tramite `.env` ### Overlay opzionali espliciti
Gli overlay non cambiano il comando operativo. Impostare in `.env`: DWH/vector/embedding remoti restano endpoint del file locale o server. Per il solo preset di
sviluppo pgvector, aggiungere `-f deploy/compose.local-vector.yaml --profile local-vector` al
```dotenv comando base. Per il preprocessing aggiungere anche
# DWH/vector/embedding remoti (server applicativo o server con i DB): `-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml --profile preprocess`,
COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml poi usare `docker compose run --rm preprocess-evidence` oppure `preprocess-dwh` con gli stessi argomenti.
COMPOSE_PROFILES=
# pgvector locale (Mac, Windows o server autonomo):
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml
COMPOSE_PROFILES=local-vector
```
Su Windows usare `;` come separatore di `COMPOSE_FILE`. Per il preprocessing locale aggiungere `deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml` e impostare `COMPOSE_PROFILES=local-vector,preprocess`; poi usare `docker compose run --rm preprocess-evidence` oppure `docker compose run --rm preprocess-dwh`.
## Workspace, adapter e Evidence ## Workspace, adapter e Evidence
@@ -116,11 +109,10 @@ il bind mount/runtime adapter corrispondente. Non inserire la password nel works
## 1. Server remoto insieme ai database e al vector DB ## 1. Server remoto insieme ai database e al vector DB
Usare quando il server Docker è nella stessa rete del DWH e del vector DB (containerizzati o meno). Il file `.env` può restare sul default, senza profili, impostando gli endpoint raggiungibili localmente: Usare quando il server Docker è nella stessa rete del DWH e del vector DB (containerizzati o meno).
Compilare `deploy/env/local.env` con gli endpoint raggiungibili localmente:
```dotenv ```dotenv
COMPOSE_FILE=compose.yaml
COMPOSE_PROFILES=
THT_DB_NAME=warehouse THT_DB_NAME=warehouse
THT_DWH_REST_URL=https://dwh.internal.example THT_DWH_REST_URL=https://dwh.internal.example
THT_VEC_REST_URL=https://vectors.internal.example THT_VEC_REST_URL=https://vectors.internal.example
@@ -143,17 +135,15 @@ claim normalizzati `X-Thoth-Principal-Issuer`, `X-Thoth-Principal-Subject`,
`X-Thoth-Principal-Display-Name` e `X-Thoth-Is-Admin` attesi dal core. Non esporre direttamente `X-Thoth-Principal-Display-Name` e `X-Thoth-Is-Admin` attesi dal core. Non esporre direttamente
la porta pubblicata da nginx. la porta pubblicata da nginx.
Se il server deve essere raggiungibile da altri host, sostituire `COMPOSE_FILE` con Se il server deve essere raggiungibile da altri host, usare il profilo
`compose.yaml:deploy/compose.production.yaml`, configurare il proxy autenticato e impostare `deploy/compose.server.yaml`, configurare il proxy autenticato e impostare
`AUTH_MODE=upstream`/`THOTH_PUBLIC_EXPOSURE=true` come descritto nella sezione di trust boundary. `AUTH_MODE=upstream`/`THOTH_PUBLIC_EXPOSURE=true` come descritto nella sezione di trust boundary.
## 2. Mac locale ## 2. Mac locale
Installare Docker Desktop e, se usato, Ollama sul Mac. Nel `.env` selezionare il profilo locale: Installare Docker Desktop e, se usato, Ollama sul Mac. In `deploy/env/local.env` impostare gli endpoint:
```dotenv ```dotenv
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml
COMPOSE_PROFILES=local-vector
THT_DB_NAME=warehouse THT_DB_NAME=warehouse
THT_DWH_REST_URL=https://dwh.example.test THT_DWH_REST_URL=https://dwh.example.test
THT_OLLAMA_URL=http://host.docker.internal:11434 THT_OLLAMA_URL=http://host.docker.internal:11434
@@ -173,11 +163,9 @@ Poi eseguire il comando standard `docker compose up --build -d`. Il primo avvio
## 3. PC Windows locale ## 3. PC Windows locale
Usare Docker Desktop con backend WSL2 e abilitare la condivisione della directory del clone. Modificare `.env` con il separatore Windows: Usare Docker Desktop con backend WSL2 e abilitare la condivisione della directory del clone. Modificare `deploy/env/local.env`:
```dotenv ```dotenv
COMPOSE_FILE=compose.yaml;deploy/compose.local-vector.yaml
COMPOSE_PROFILES=local-vector
THT_DB_NAME=warehouse THT_DB_NAME=warehouse
THT_DWH_REST_URL=https://dwh.example.test THT_DWH_REST_URL=https://dwh.example.test
THT_OLLAMA_URL=http://host.docker.internal:11434 THT_OLLAMA_URL=http://host.docker.internal:11434
@@ -195,11 +183,11 @@ Se un bind mount viene rifiutato, aggiungere la cartella del repository a Docker
## 4. Server applicativo distinto da DB ed Evidence ## 4. Server applicativo distinto da DB ed Evidence
Usare il profilo production e consentire dal firewall solo le destinazioni necessarie: Usare il profilo server e consentire dal firewall solo le destinazioni necessarie:
```dotenv ```dotenv
COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml # Avvio: docker compose --env-file deploy/env/server.env \
COMPOSE_PROFILES= # -f compose.yaml -f deploy/compose.server.yaml up --build -d
THT_DB_NAME=warehouse THT_DB_NAME=warehouse
THT_DWH_REST_URL=https://dwh.example.test THT_DWH_REST_URL=https://dwh.example.test
THT_VEC_REST_URL=https://vectors.example.test THT_VEC_REST_URL=https://vectors.example.test
+3 -3
View File
@@ -3,13 +3,13 @@ import react from "@vitejs/plugin-react";
import path from "path"; import path from "path";
export default defineConfig(() => { export default defineConfig(() => {
const embedBase = process.env.VITE_BASE; // "/datamart-builder/assets/" in embedded; undefined = standalone const embedBase = process.env.VITE_BASE;
const apiUpstream = process.env.THT_FRONTEND_API_UPSTREAM ?? "http://localhost:8787"; const apiUpstream = process.env.THT_FRONTEND_API_UPSTREAM ?? "http://localhost:8787";
return { return {
plugins: [react()], plugins: [react()],
// base: prefisso pubblico degli asset. Default "/" (standalone). // base: prefisso pubblico degli asset. Default "/" (standalone).
// assetsDir vuoto in embedded → asset alla root di dist/ così il proxy // Con un prefisso personalizzato, gli asset restano alla root di dist/ per evitare
// /datamart-builder/assets/ → frontend-root mappa 1:1 (niente /assets/assets/). // di duplicare il segmento assets nel percorso pubblico.
base: embedBase ?? "/", base: embedBase ?? "/",
build: { manifest: true, outDir: "dist", assetsDir: embedBase ? "" : "assets" }, build: { manifest: true, outDir: "dist", assetsDir: embedBase ? "" : "assets" },
server: { server: {
@@ -0,0 +1,33 @@
from pathlib import Path
import yaml
def test_local_compose_uses_the_generic_external_endpoint_contract():
root = Path(__file__).resolve().parents[2]
compose = yaml.safe_load((root / "compose.yaml").read_text())
local = yaml.safe_load((root / "deploy/compose.local.yaml").read_text())
assert set(compose["services"]) == {"core", "frontend"}
assert local["services"]["core"]["environment"]["AUTH_MODE"] == "none"
assert local["services"]["core"]["ports"] == ["127.0.0.1:${THOTH_CORE_HTTP_PORT:-8787}:8787"]
assert local["services"]["frontend"]["ports"] == ["127.0.0.1:${THOTH_HTTP_PORT:-8080}:8080"]
environment = compose["services"]["core"]["environment"]
for name in ("THT_DWH_REST_URL", "THT_VEC_REST_URL", "THT_OLLAMA_URL", "THT_LLM_URL"):
assert name in environment
assert {"settings", "pi-state", "workspace-registry", "sessions"} <= set(compose["volumes"])
def test_core_image_prepares_the_writable_pi_profile_before_mounting_config_files():
root = Path(__file__).resolve().parents[2]
dockerfile = (root / "docker/core.Dockerfile").read_text()
assert "mkdir -p /home/thoth/.pi/agent" in dockerfile
assert "chown -R thoth:thoth /home/thoth/.pi" in dockerfile
assert (
"cp --remove-destination /app/harness/workspaces/local.yaml "
"/app/harness/config/tht.yaml" in dockerfile
)
assert "ln -sf /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml" not in dockerfile
assert "ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/" not in dockerfile
@@ -1,133 +0,0 @@
from pathlib import Path
import shutil
import subprocess
import yaml
class ComposeLoader(yaml.SafeLoader):
pass
def _compose_override(loader, node):
if isinstance(node, yaml.MappingNode):
return loader.construct_mapping(node)
return loader.construct_sequence(node)
ComposeLoader.add_constructor("!override", _compose_override)
def test_psd_overlay_uses_generated_workspace_for_default_and_named_commands():
root = Path(__file__).resolve().parents[2]
compose = yaml.load(
(root / "deploy/compose.psd-local.yaml.example").read_text(),
Loader=ComposeLoader,
)
core = compose["services"]["core"]
assert core["environment"]["THT_CONFIG"] == "/app/harness/config/tht.yaml"
assert core["environment"]["THT_SECRETS_FILE"] == "/run/secrets/thothii.secrets"
for name in (
"THT_DB_NAME", "THT_DWH_REST_URL", "THT_VEC_REST_URL",
"THT_VEC_WRITE_REST_URL", "THT_OLLAMA_URL", "THT_PROFILE",
"PI_PROVIDER", "PI_MODEL", "PI_THINKING",
):
assert name in core["environment"]
def target(volume):
if isinstance(volume, dict):
return volume["target"]
parts = volume.rsplit(":", 2)
return parts[-2] if parts[-1] in {"ro", "rw"} else parts[-1]
targets = {target(volume) for volume in core["volumes"]}
assert "/app/harness/config/tht.yaml" in targets
assert "/app/harness/workspaces/psd.yaml" not in targets
assert "/data/workspaces/psd/config/tht.yaml" not in targets
assert "/data" in targets
assert "/home/thoth/.pi" in targets
assert "/home/thoth/.pi/agent/models.json" in targets
assert "/home/thoth/.pi/agent/settings.json" in targets
assert "/data/evidence" in targets
assert "/run/secrets/thothii.secrets" in targets
assert set(compose["volumes"]) == {"thoth_data", "thoth_pi_config"}
assert core["networks"]["default"]["aliases"] == ["core", "thothii-core"]
frontend = compose["services"]["frontend"]
assert frontend["ports"] == ["127.0.0.1:8099:8080"]
assert frontend["build"]["args"] == {
"VITE_BASE": "/",
"VITE_BACKEND_URL": "/api",
}
assert frontend["networks"] == {
"default": {"aliases": ["frontend", "thothii-frontend"]}
}
assert compose["networks"]["default"] == {
"external": True,
"name": "thothii_default",
}
def test_core_image_prepares_the_writable_pi_profile_before_mounting_config_files():
root = Path(__file__).resolve().parents[2]
dockerfile = (root / "docker/core.Dockerfile").read_text()
assert "mkdir -p /home/thoth/.pi/agent" in dockerfile
assert "chown -R thoth:thoth /home/thoth/.pi" in dockerfile
assert (
"cp --remove-destination /app/harness/workspaces/local.yaml "
"/app/harness/config/tht.yaml" in dockerfile
)
assert "ln -sf /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml" not in dockerfile
assert (
"ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/psd.yaml"
in dockerfile
)
def test_psd_bootstrap_materializes_the_base_compose_env_file(tmp_path):
source_root = Path(__file__).resolve().parents[2]
root = tmp_path / "ThothII"
(root / "scripts").mkdir(parents=True)
(root / "deploy/workspaces").mkdir(parents=True)
shutil.copy(
source_root / "scripts/bootstrap-local-psd-docker-config.sh",
root / "scripts/bootstrap-local-psd-docker-config.sh",
)
shutil.copy(
source_root / "deploy/compose.psd-local.yaml.example",
root / "deploy/compose.psd-local.yaml.example",
)
shutil.copy(
source_root / "deploy/workspaces/psd.yaml.example",
root / "deploy/workspaces/psd.yaml.example",
)
source_env = tmp_path / "source.env"
source_env.write_text("\n".join([
"THT_DB_NAME=postgres",
"THT_DWH_REST_URL=https://dwh.invalid/",
"THT_VEC_REST_URL=https://vec.invalid/read/",
"THT_VEC_WRITE_REST_URL=https://vec.invalid/write/",
"THT_DWH_API_KEY=dwh",
"THT_VEC_API_KEY=reader",
"THT_VEC_WRITE_API_KEY=writer",
"",
]))
workspace = tmp_path / "workspace"
workspace.mkdir()
auth = tmp_path / "auth.json"
auth.write_text('{"zai":{"key":"model"}}')
subprocess.run(
[
"sh", str(root / "scripts/bootstrap-local-psd-docker-config.sh"),
str(source_env), str(workspace), str(auth),
],
check=True,
capture_output=True,
text=True,
)
assert (root / "deploy/thothii.env").is_file()
assert "THT_SECRETS_FILE=./deploy/secrets/thothii.secrets" in (
root / ".env"
).read_text()
@@ -1,70 +0,0 @@
#!/bin/sh
set -eu
root=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
source_env=${1:-"$root/../../harness/.env"}
workspace=${2:-"$root/../../../tht-workspace-psd"}
auth_file=${3:-"$HOME/.pi/agent/auth.json"}
value() {
awk -F= -v key="$1" '$1 == key { sub(/^[^=]*=/, ""); sub(/[[:space:]].*$/, ""); print; exit }' "$source_env"
}
required() {
result=$(value "$1")
[ -n "$result" ] || { echo "missing $1 in local source configuration" >&2; exit 2; }
printf '%s' "$result"
}
test -f "$source_env"
test -d "$workspace"
test -f "$auth_file"
ca=$(value THT_SSL_CA)
[ -z "$ca" ] || test -f "$ca"
model_key=$(jq -er '.zai.key' "$auth_file")
test -n "$model_key"
umask 077
mkdir -p "$root/deploy/secrets" "$root/deploy/workspaces"
: >"$root/deploy/thothii.env"
cp "$root/deploy/compose.psd-local.yaml.example" "$root/deploy/compose.psd-local.yaml"
cp "$root/deploy/workspaces/psd.yaml.example" "$root/deploy/workspaces/psd.yaml"
cat >"$root/.env" <<EOF
COMPOSE_FILE=compose.yaml:deploy/compose.psd-local.yaml
COMPOSE_PROFILES=
THT_SECRETS_FILE=./deploy/secrets/thothii.secrets
THOTH_HTTP_PORT=8080
AUTH_MODE=none
THOTH_PUBLIC_EXPOSURE=false
MAX_PI_PROCESSES=4
PI_PROVIDER=zai
PI_MODEL=glm-5.2
PI_THINKING=medium
PI_AUTH_FILE=$auth_file
THT_PROFILE=workstation
THT_DB_NAME=$(required THT_DB_NAME)
THT_DWH_REST_URL=$(required THT_DWH_REST_URL)
THT_VEC_REST_URL=$(required THT_VEC_REST_URL)
THT_VEC_WRITE_REST_URL=$(required THT_VEC_WRITE_REST_URL)
THT_OLLAMA_URL=http://host.docker.internal:11434
THT_DOCS_ROOT=/data/workspaces/psd
THT_PSD_WORKSPACE_HOST_PATH=$workspace
EOF
cat >"$root/deploy/secrets/thothii.secrets" <<EOF
THT_MODEL_API_KEY=$model_key
THT_DWH_API_KEY=$(required THT_DWH_API_KEY)
THT_VEC_API_KEY=$(required THT_VEC_API_KEY)
THT_VEC_WRITE_API_KEY=$(required THT_VEC_WRITE_API_KEY)
EOF
if [ -n "$ca" ]; then
cat >>"$root/deploy/compose.psd-local.yaml" <<EOF
- type: bind
source: $ca
target: /run/secrets/ca-chain.pem
read_only: true
EOF
printf '%s\n' 'THT_CA=/run/secrets/ca-chain.pem' >>"$root/deploy/secrets/thothii.secrets"
else
printf '%s\n' 'THT_CA=/etc/ssl/certs/ca-certificates.crt' >>"$root/deploy/secrets/thothii.secrets"
fi
chmod 600 "$root/.env" "$root/deploy/thothii.env" "$root/deploy/secrets/thothii.secrets"
echo "Local PSD Docker configuration materialized without printing secret values."
+2 -2
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Smoke test del deploy standalone ThothII (core + frontend). # Smoke test del deploy standalone ThothII (core + frontend).
# Usa docker-compose.dev.yml (rete propria, porte host). Non tocca il portale. # Usa docker-compose.dev.yml (rete propria, porte host).
# Prereq: deploy/thothii.env popolato + ruoli DB creati + pi-config + settings.json. # Prereq: deploy/thothii.env popolato, endpoint esterni configurati e profilo Pi locale.
set -euo pipefail set -euo pipefail
cd "$(dirname "$0")/.." cd "$(dirname "$0")/.."
+12 -60
View File
@@ -1,68 +1,20 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# run-stack.sh — avvia i 3 layer reali di ThothII per la validazione end-to-end. # run-stack.sh — avvia lo stack Compose locale di ThothII in primo piano.
# #
# frontend (browser) → backend (Fastify :8787) → pi --mode rpc (GLM 5.2) → tht/harness → DWH # Il core include Pi; DWH, vector DB, embedding e LLM sono endpoint esterni configurati
# in deploy/env/local.env. Non richiede un eseguibile Pi sull'host.
# #
# Prerequisiti: VPN attiva, `pi` su PATH (GLM 5.2 configurato), harness/.env popolato, # Preparazione: cp deploy/env/local.env.example deploy/env/local.env e compilare i valori.
# harness/config/tht.yaml -> workspace cliente, deps installate nei 3 progetti. # Uso: ./scripts/run-stack.sh [argomenti aggiuntivi per docker compose up]
#
# Uso: ./scripts/run-stack.sh
# Stop: Ctrl-C (termina backend + frontend; i processi pi figli del backend muoiono con esso).
set -euo pipefail set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)" ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
HARNESS="$ROOT/harness" LOCAL_ENV_FILE="${THT_LOCAL_ENV_FILE:-$ROOT/deploy/env/local.env}"
BACKEND="$ROOT/backend"
FRONTEND="$ROOT/frontend"
THT_BIN="$HARNESS/.venv/bin/tht"
BACKEND_PORT="${BACKEND_PORT:-8787}"
FRONTEND_PORT="${FRONTEND_PORT:-5173}"
# --- preflight --------------------------------------------------------------- [[ -f "$LOCAL_ENV_FILE" ]] || {
[ -f "$HARNESS/.env" ] || { echo "ERRORE: $HARNESS/.env mancante (credenziali DWH/vector)"; exit 1; } echo "ERRORE: $LOCAL_ENV_FILE mancante. Copia deploy/env/local.env.example e configura gli endpoint." >&2
[ -x "$THT_BIN" ] || { echo "ERRORE: $THT_BIN non trovato (esegui: cd harness && python -m venv .venv && pip install -e .)"; exit 1; } exit 1
command -v pi >/dev/null || { echo "ERRORE: 'pi' non sul PATH (configura @earendil-works/pi-coding-agent con GLM 5.2)"; exit 1; } }
[ -e "$HARNESS/config/tht.yaml" ] || { echo "ERRORE: $HARNESS/config/tht.yaml mancante (symlink al workspace)"; exit 1; }
# Carica le variabili del DWH/vector nell'ambiente: il backend le passa a pi e a tht. exec docker compose --env-file "$LOCAL_ENV_FILE" \
set -a; . "$HARNESS/.env"; set +a -f "$ROOT/compose.yaml" -f "$ROOT/deploy/compose.local.yaml" up --build "$@"
# tht deve essere raggiungibile dal processo pi che il backend spawna.
export PATH="$HARNESS/.venv/bin:$PATH"
echo "== ThothII stack =="
echo " harness : $HARNESS (tht: $THT_BIN)"
echo " backend : http://localhost:$BACKEND_PORT"
echo " frontend: http://localhost:$FRONTEND_PORT"
echo " pi : $(command -v pi)"
echo
# --- avvio -------------------------------------------------------------------
pids=()
cleanup() { echo; echo "Arresto stack..."; for p in "${pids[@]}"; do kill "$p" 2>/dev/null || true; done; }
trap cleanup EXIT INT TERM
# Backend: usa il pi reale + il tht del venv, cwd harness per lo spawn di pi.
(
cd "$BACKEND"
PORT="$BACKEND_PORT" \
THT_HARNESS_DIR="$HARNESS" \
THT_BIN="$THT_BIN" \
PI_BIN="pi" \
AUTH_MODE="none" \
THT_DWH_PRECHECK="1" \
npm run dev
) &
pids+=($!)
# Frontend: il browser usa sempre /api; Vite lo inoltra al backend locale.
(
cd "$FRONTEND"
THT_FRONTEND_API_UPSTREAM="http://localhost:$BACKEND_PORT" \
npm run dev -- --port "$FRONTEND_PORT"
) &
pids+=($!)
echo "Stack avviato. Apri http://localhost:$FRONTEND_PORT e crea una nuova domanda."
echo "Ctrl-C per fermare."
wait
@@ -6,6 +6,9 @@ project="thothii-external-lifecycle-$$"
cleanup() { docker compose --project-name "$project" --profile external down --volumes >/dev/null 2>&1 || true; } cleanup() { docker compose --project-name "$project" --profile external down --volumes >/dev/null 2>&1 || true; }
trap cleanup EXIT HUP INT TERM trap cleanup EXIT HUP INT TERM
export THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
export PI_AUTH_FILE=/dev/null
unset THT_VECTOR_BOOTSTRAP_PASSWORD_SECRET_FILE THT_VECTOR_MIGRATOR_PASSWORD_SECRET_FILE unset THT_VECTOR_BOOTSTRAP_PASSWORD_SECRET_FILE THT_VECTOR_MIGRATOR_PASSWORD_SECRET_FILE
unset THT_VECTOR_READER_PASSWORD_SECRET_FILE THT_VECTOR_WRITER_PASSWORD_SECRET_FILE unset THT_VECTOR_READER_PASSWORD_SECRET_FILE THT_VECTOR_WRITER_PASSWORD_SECRET_FILE
rendered=$(docker compose --project-name "$project" --profile external config) rendered=$(docker compose --project-name "$project" --profile external config)
@@ -13,6 +16,10 @@ if printf '%s' "$rendered" | grep -q 'THT_VECTOR_.*PASSWORD_FILE\|vector_.*passw
echo "external config contains local vector secret references" >&2 echo "external config contains local vector secret references" >&2
exit 1 exit 1
fi fi
if printf '%s' "$rendered" | grep -Eqi 'omics_portal|chirone|localllm_default|/home/chirone|datamart-builder'; then
echo "external config contains deployment-specific coupling" >&2
exit 1
fi
docker compose --project-name "$project" --profile external up --build --wait core docker compose --project-name "$project" --profile external up --build --wait core
core=$(docker compose --project-name "$project" --profile external ps -q core) core=$(docker compose --project-name "$project" --profile external ps -q core)
inspect=$(docker inspect "$core") inspect=$(docker inspect "$core")
@@ -20,4 +27,8 @@ if printf '%s' "$inspect" | grep -q 'THT_VECTOR_.*PASSWORD_FILE\|/run/secrets/ve
echo "external core inspect contains local vector secret references" >&2 echo "external core inspect contains local vector secret references" >&2
exit 1 exit 1
fi fi
if printf '%s' "$inspect" | grep -Eqi 'omics_portal|chirone|localllm_default|/home/chirone|datamart-builder'; then
echo "external core inspect contains deployment-specific coupling" >&2
exit 1
fi
echo "external core lifecycle without local vector secrets passed." echo "external core lifecycle without local vector secrets passed."
+4 -4
View File
@@ -54,13 +54,13 @@ if [ "$response" != '{"backend":"fastify","path":"/health"}' ]; then
exit 1 exit 1
fi fi
if ! rg -Fq 'THT_FRONTEND_API_UPSTREAM="http://localhost:$BACKEND_PORT"' "$ROOT/scripts/run-stack.sh"; then if ! rg -Fq -- '-f "$ROOT/compose.yaml" -f "$ROOT/deploy/compose.local.yaml" up --build "$@"' "$ROOT/scripts/run-stack.sh"; then
echo "run-stack.sh must configure the Vite internal upstream from BACKEND_PORT" >&2 echo "run-stack.sh must start the base+local Compose stack" >&2
exit 1 exit 1
fi fi
if rg -q 'VITE_BACKEND_URL' "$ROOT/scripts/run-stack.sh"; then if rg -q 'command -v pi|PI_BIN="pi"|PI_BIN=pi' "$ROOT/scripts/run-stack.sh"; then
echo "run-stack.sh must keep the browser base on /api" >&2 echo "run-stack.sh must not require a host Pi binary" >&2
exit 1 exit 1
fi fi
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
content_targets=(
.dockerignore
compose.yaml
docker-compose.dev.yml
deploy
docker
frontend/vite.config.ts
README.md
docs/install
docs/installazione-docker-4-contesti.md
.env.example
scripts/run-stack.sh
scripts/docker-smoke.sh
)
matches=$(
rg -n -i \
-g '!deploy/workspaces/**' \
-g '!docker/session-migrate.sh' \
-g '!docker/cutover-legacy-sessions.sh' \
-g '!docker/smoke/**' \
'omics_portal|chirone|localllm_default|datamart-builder|compose\.production\.yaml|compose\.psd-local\.yaml' \
"${content_targets[@]}" || true
)
runtime_psd_matches=$(
rg -n -i \
-g '!deploy/workspaces/**' \
-g '!docker/session-migrate.sh' \
-g '!docker/cutover-legacy-sessions.sh' \
-g '!docker/smoke/**' \
'\bpsd\b' \
.dockerignore compose.yaml docker-compose.dev.yml deploy docker frontend/vite.config.ts \
.env.example scripts/run-stack.sh scripts/docker-smoke.sh || true
)
offenders=()
for superseded_file in \
deploy/compose.production.yaml \
deploy/compose.psd-local.yaml.example \
deploy/compose.psd-local.yaml \
scripts/bootstrap-local-psd-docker-config.sh \
harness/tests/test_psd_local_compose_contract.py
do
[[ ! -e "$superseded_file" ]] || offenders+=("$superseded_file (forbidden active deployment filename)")
done
if [[ -n "$matches" ]]; then
while IFS= read -r match; do
offenders+=("$match")
done <<<"$matches"
fi
if [[ -n "$runtime_psd_matches" ]]; then
while IFS= read -r match; do
offenders+=("$match")
done <<<"$runtime_psd_matches"
fi
if rg -n 'command -v pi|PI_BIN="pi"|PI_BIN=pi' scripts/run-stack.sh >/dev/null; then
offenders+=("scripts/run-stack.sh (requires a host Pi binary)")
fi
if ((${#offenders[@]})); then
printf '%s\n' "active deployment coupling found:" >&2
printf '%s\n' "${offenders[@]}" >&2
exit 1
fi
echo "no active PSD, Chirone, or portal deployment coupling found."
+4 -2
View File
@@ -9,7 +9,8 @@ auth_file="$tmp/auth.json"
printf '%s\n' '{}' >"$auth_file" printf '%s\n' '{}' >"$auth_file"
chmod 0600 "$auth_file" chmod 0600 "$auth_file"
rendered=$(PI_AUTH_FILE="$auth_file" docker compose config) rendered=$(THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
PI_AUTH_FILE="$auth_file" docker compose config)
printf '%s\n' "$rendered" | grep -q "source: $auth_file" printf '%s\n' "$rendered" | grep -q "source: $auth_file"
printf '%s\n' "$rendered" | grep -q 'target: /home/thoth/.pi/agent/auth.json' printf '%s\n' "$rendered" | grep -q 'target: /home/thoth/.pi/agent/auth.json'
printf '%s\n' "$rendered" | grep -A4 'target: /home/thoth/.pi/agent/auth.json' \ printf '%s\n' "$rendered" | grep -A4 'target: /home/thoth/.pi/agent/auth.json' \
@@ -29,6 +30,7 @@ assert settings["enabledModels"] == [
PY PY
grep -q '^ARG PI_VERSION=0.80.3$' docker/core.Dockerfile grep -q '^ARG PI_VERSION=0.80.3$' docker/core.Dockerfile
grep -q '^PI_AUTH_FILE=$auth_file$' scripts/bootstrap-local-psd-docker-config.sh grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/local.env.example
grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/server.env.example
echo "Pi user-auth Compose contract passed." echo "Pi user-auth Compose contract passed."
+13
View File
@@ -44,4 +44,17 @@ printf 'FROM scratch\n' > "$fixture_root/Dockerfile"
"$repo_root/scripts/verify-line-endings.sh" "$fixture_root" "$repo_root/scripts/verify-line-endings.sh" "$fixture_root"
git_fixture="$fixture_root/git-worktree"
mkdir -p "$git_fixture"
git -C "$git_fixture" init -q
printf 'tracked then deleted\n' >"$git_fixture/deleted.md"
git -C "$git_fixture" add deleted.md
rm "$git_fixture/deleted.md"
git_output="$(cd "$git_fixture" && "$repo_root/scripts/verify-line-endings.sh" 2>&1)"
if grep -Fq 'No such file or directory' <<<"$git_output"; then
echo "line-ending verifier tried to read a tracked deletion" >&2
exit 1
fi
echo "line-ending verifier tests passed" echo "line-ending verifier tests passed"
+1
View File
@@ -23,6 +23,7 @@ root="$(cd "$root" && pwd)"
offenders=() offenders=()
while IFS= read -r -d '' file; do while IFS= read -r -d '' file; do
[[ -f "$file" ]] || continue
case "$file" in case "$file" in
*.ps1) continue ;; *.ps1) continue ;;
esac esac