refactor: remove portal deployment coupling
This commit is contained in:
+4
-1
@@ -15,7 +15,10 @@
|
||||
!deploy/env/*.env.example
|
||||
deploy/thothii.env
|
||||
deploy/secrets/
|
||||
harness/workspaces/psd.yaml
|
||||
harness/workspaces/*.yaml
|
||||
!harness/workspaces/local.yaml
|
||||
!harness/workspaces/tht.example.yaml
|
||||
!harness/workspaces/tht-test.yaml
|
||||
**/*.log
|
||||
**/.DS_Store
|
||||
coverage/
|
||||
|
||||
@@ -11,8 +11,10 @@ detail. Design history lives in `docs/superpowers/specs/` and `docs/superpowers/
|
||||
|
||||
## Commands
|
||||
|
||||
The repo has three independently-built layers. Run the **full stack** (real Pi + DWH, needs
|
||||
VPN + `harness/.env` + `pi` on PATH) with `./scripts/run-stack.sh` (frontend :5173 → backend :8787).
|
||||
The repo has three independently-built layers. Run the local Docker stack with
|
||||
`./scripts/run-stack.sh` after creating `deploy/env/local.env`; it starts the base+local Compose
|
||||
profile, whose core image contains Pi. DWH, vector DB, embedding, and LLM remain external
|
||||
configuration endpoints.
|
||||
|
||||
**harness/** (Python `tht` CLI + Pi gate extension)
|
||||
- Install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` (puts `tht` on PATH)
|
||||
|
||||
+18
-2
@@ -1,8 +1,24 @@
|
||||
# ThothII — Project State
|
||||
|
||||
> Starting-point snapshot for new sessions. Last updated: 2026-07-23 (session summary redesign live).
|
||||
> Starting-point snapshot for new sessions. Last updated: 2026-08-05 (portable deployment decoupled).
|
||||
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
||||
|
||||
## Portable deployment decoupling — LIVE 2026-08-05
|
||||
|
||||
- **Mandatory stack.** The supported Compose stack is exactly `frontend` plus `core`; use the
|
||||
base file with `deploy/compose.local.yaml` or `deploy/compose.server.yaml`. `run-stack.sh`
|
||||
invokes the base+local Compose command and the core image provides Pi, so no host Pi binary is
|
||||
part of the launch contract.
|
||||
- **External boundaries.** DWH, vector DB, embedding, LLM, and reverse-proxy services are
|
||||
external configurable endpoints even when deployed on the same infrastructure. The two
|
||||
superseded PSD/portal deployment overlays were removed. Workspace descriptors and migration
|
||||
utilities remain separate from deployment runtime configuration.
|
||||
- **Legacy PSD deployment ruling.** The PSD bootstrap was deleted because it generated the
|
||||
retired overlay and was therefore deployment machinery, not a data migration utility. Its
|
||||
remaining live contract checks were renamed for the generic local Compose profile. The coupling
|
||||
gate rejects stale active deployment filenames and content while deliberately excluding
|
||||
historical plans/specs, canonical workspace descriptors, and non-runtime migration helpers.
|
||||
|
||||
## Portable Git workspace registry — source integration (2026-08-04)
|
||||
|
||||
- **Source of truth and scope.** The canonical workspace repository is a generic Git remote,
|
||||
@@ -103,7 +119,7 @@
|
||||
release; never re-enable filesystem persistence, restore the archive into production, or
|
||||
dual-write during rollback.
|
||||
|
||||
## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12
|
||||
## Historical deployment — Docker locale (Profile A, co-located) — superseded 2026-08-05
|
||||
|
||||
ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale).
|
||||
|
||||
|
||||
@@ -4,57 +4,37 @@ ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fasti
|
||||
core. The portable deployment runs exactly two application services; data services remain
|
||||
external in this profile.
|
||||
|
||||
## Docker Compose: one-command startup
|
||||
## Docker Compose: local startup
|
||||
|
||||
Requirements: Docker Engine with Compose v2. The default project starts only the two
|
||||
application images; DWH, vector and embedding services can be remote or supplied by an
|
||||
optional overlay.
|
||||
Requirements: Docker Engine with Compose v2. The mandatory stack is exactly the `core` and
|
||||
`frontend` application images. DWH, vector DB, embedding, and LLM services are external,
|
||||
configurable endpoints—even when they are co-located with ThothII.
|
||||
|
||||
From a fresh clone, run these commands from the repository root:
|
||||
|
||||
```sh
|
||||
cp .env.example .env
|
||||
cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets
|
||||
chmod 600 deploy/secrets/thothii.secrets
|
||||
# Edit .env (non-secret endpoints) and deploy/secrets/thothii.secrets (KEY=VALUE lines).
|
||||
docker compose up --build -d
|
||||
cp deploy/env/local.env.example deploy/env/local.env
|
||||
# Edit deploy/env/local.env, including PI_AUTH_FILE and the external endpoint URLs.
|
||||
docker compose --env-file deploy/env/local.env \
|
||||
-f compose.yaml -f deploy/compose.local.yaml up --build -d
|
||||
```
|
||||
|
||||
The root `.env` is loaded automatically by Compose. It defaults to `compose.yaml`, an empty
|
||||
profile, and `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`; no `--env-file`, `-f`, or
|
||||
`--profile` flag is required for the normal installation. Add or edit YAML workspace descriptors
|
||||
under `deploy/workspaces/`; they are mounted read-only and relative `roots` resolve beneath
|
||||
`/data/workspaces/<workspace-name>`. Open <http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in
|
||||
`.env` to choose another loopback port).
|
||||
`./scripts/run-stack.sh` runs this same base+local command in the foreground. The core image
|
||||
contains its Pi runtime; no host `pi` executable is used. For a server installation, copy and
|
||||
fill `deploy/env/server.env.example`, then use `-f compose.yaml -f deploy/compose.server.yaml`.
|
||||
Workspace descriptors come from the Git remote configured by `THT_WORKSPACE_GIT_REMOTE`; their
|
||||
runtime endpoint and secret bindings remain installation-local. Open
|
||||
<http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in `deploy/env/local.env` to choose another
|
||||
loopback port).
|
||||
|
||||
The bundle contains only values, one per line (`THT_MODEL_API_KEY=...`, DWH/vector keys, and
|
||||
the optional local-vector passwords). It is ignored by Git and never copied into either image.
|
||||
Do not put credentials in `.env`, workspace YAML, URLs, or Compose interpolation values.
|
||||
Credentials and certificates are local protected files. Do not put them in environment examples,
|
||||
workspace YAML, URLs, or Compose interpolation values. The optional `local-vector` and
|
||||
preprocessing overlays are development presets; they do not change the two-service mandatory
|
||||
stack or the external-endpoint contract.
|
||||
|
||||
### Optional overlays
|
||||
|
||||
Overlays are selected in `.env`, so the operational command remains the same. On Unix-like
|
||||
systems use `:` between files; on Windows use `;`:
|
||||
|
||||
```dotenv
|
||||
# Remote DWH/vector/embedding services with authenticated reverse proxy:
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml
|
||||
COMPOSE_PROFILES=
|
||||
|
||||
# Local pgvector (Mac/Windows or a standalone application server):
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml
|
||||
COMPOSE_PROFILES=local-vector
|
||||
```
|
||||
|
||||
After changing `.env`, apply the selected configuration with `docker compose up --build -d`.
|
||||
Preprocessing is an explicit opt-in preset: append
|
||||
`deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml` and set
|
||||
`COMPOSE_PROFILES=local-vector,preprocess`; then run the job with
|
||||
`docker compose run --rm preprocess-evidence` or `preprocess-dwh`.
|
||||
|
||||
Application state, including settings, sessions, artifacts, and indexes, lives in the named
|
||||
`thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an
|
||||
explicit destructive command such as `docker compose down --volumes` removes it.
|
||||
Application state is split across the named `settings`, `pi-state`, `workspace-registry`, and
|
||||
`sessions` volumes. `docker compose down` keeps them. Only an explicit destructive command such
|
||||
as `docker compose down --volumes` removes them.
|
||||
|
||||
The frontend depends on the core health check and proxies `/health` and `/api/*` to it. The
|
||||
application health endpoint intentionally checks process readiness only; external dependency
|
||||
@@ -110,17 +90,18 @@ application state; passwords are selected at runtime and are never passed as URL
|
||||
|
||||
## Preprocessing jobs and S3 Evidence
|
||||
|
||||
The included job workspaces target the local-vector profile. Put the four local-vector password
|
||||
keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then select
|
||||
the preprocessing preset in `.env`:
|
||||
The included job workspaces target the optional local-vector profile. Put the four local-vector
|
||||
password keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then
|
||||
run the explicit preprocessing preset:
|
||||
|
||||
```dotenv
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml:deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml
|
||||
COMPOSE_PROFILES=local-vector,preprocess
|
||||
```sh
|
||||
docker compose --env-file deploy/env/local.env \
|
||||
-f compose.yaml -f deploy/compose.local.yaml -f deploy/compose.local-vector.yaml \
|
||||
-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \
|
||||
--profile local-vector --profile preprocess run --rm preprocess-evidence
|
||||
```
|
||||
|
||||
Run `docker compose run --rm preprocess-evidence` or
|
||||
`docker compose run --rm preprocess-dwh`. The overlay makes each job wait for the vector
|
||||
Replace the final service with `preprocess-dwh` when required. The overlay makes each job wait for the vector
|
||||
database health check, role reconciliation, and a successful migration; no separate database
|
||||
startup or migration command is required.
|
||||
|
||||
@@ -183,8 +164,8 @@ with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts thi
|
||||
rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other
|
||||
auth mode fails during core startup.
|
||||
|
||||
Production credentials use the one Compose secret bundle, not `.env`. Put the required keys in
|
||||
`deploy/secrets/thothii.secrets` and select the production overlay in `.env`:
|
||||
Production credentials use the one Compose secret bundle, not an environment example. Put the
|
||||
required keys in `deploy/secrets/thothii.secrets` for the selected base+server installation:
|
||||
|
||||
```dotenv
|
||||
THT_MODEL_API_KEY=replace-me
|
||||
@@ -255,10 +236,10 @@ session store without upstream authentication, direct DB host/name/runtime user/
|
||||
The migrator independently rejects every other TLS mode before reading its password secret or
|
||||
constructing a database URL.
|
||||
|
||||
Perform the cutover in one maintenance window, with the Task 4 portal proxy headers and Task 5
|
||||
backend principal parser deployed together. Neither change is safe to deploy independently: Task
|
||||
4 clears the legacy identity header and Task 5 rejects it. Drain/stop active Pi work, enable a
|
||||
maintenance response at the portal, then run the migrator once and inspect its pristine JSON:
|
||||
Perform the cutover in one maintenance window, with the upstream identity-proxy headers and
|
||||
backend principal parser deployed together. Neither change is safe to deploy independently: the
|
||||
proxy clears the legacy identity header and the backend rejects it. Drain/stop active Pi work,
|
||||
enable a maintenance response at the proxy, then run the migrator once and inspect its pristine JSON:
|
||||
|
||||
```sh
|
||||
docker compose -f compose.yaml -f deploy/compose.session-server.yaml \
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
services:
|
||||
core:
|
||||
environment:
|
||||
AUTH_MODE: upstream
|
||||
THOTH_PUBLIC_EXPOSURE: "true"
|
||||
THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME}
|
||||
THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL}
|
||||
THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL}
|
||||
THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL}
|
||||
THT_DOCS_ROOT: ${THT_DOCS_ROOT:-/data/workspaces/example/evidence-source}
|
||||
THT_SECRETS_FILE: /run/secrets/thothii.secrets
|
||||
@@ -1,52 +0,0 @@
|
||||
services:
|
||||
core:
|
||||
environment:
|
||||
AUTH_MODE: ${AUTH_MODE:-none}
|
||||
THOTH_PUBLIC_EXPOSURE: ${THOTH_PUBLIC_EXPOSURE:-false}
|
||||
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
||||
PI_PROVIDER: ${PI_PROVIDER:?set PI_PROVIDER}
|
||||
PI_MODEL: ${PI_MODEL:?set PI_MODEL}
|
||||
PI_THINKING: ${PI_THINKING:-medium}
|
||||
THT_PROFILE: ${THT_PROFILE:-workstation}
|
||||
THT_DB_NAME: ${THT_DB_NAME:?set THT_DB_NAME}
|
||||
THT_DWH_REST_URL: ${THT_DWH_REST_URL:?set THT_DWH_REST_URL}
|
||||
THT_VEC_REST_URL: ${THT_VEC_REST_URL:?set THT_VEC_REST_URL}
|
||||
THT_VEC_WRITE_REST_URL: ${THT_VEC_WRITE_REST_URL:?set THT_VEC_WRITE_REST_URL}
|
||||
THT_OLLAMA_URL: ${THT_OLLAMA_URL:?set THT_OLLAMA_URL}
|
||||
THT_SECRETS_FILE: /run/secrets/thothii.secrets
|
||||
THT_DOCS_ROOT: /data/workspaces/psd
|
||||
THT_CONFIG: /app/harness/config/tht.yaml
|
||||
extra_hosts:
|
||||
- host.docker.internal:host-gateway
|
||||
networks: !override
|
||||
default:
|
||||
aliases: [core, thothii-core]
|
||||
volumes:
|
||||
- thoth_data:/data
|
||||
- thoth_pi_config:/home/thoth/.pi
|
||||
- ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro
|
||||
- ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro
|
||||
- ${THT_SECRETS_FILE:?set THT_SECRETS_FILE}:/run/secrets/thothii.secrets:ro
|
||||
- ${THT_PSD_WORKSPACE_HOST_PATH:?set THT_PSD_WORKSPACE_HOST_PATH}/evidence:/data/evidence:ro
|
||||
- ./deploy/workspaces/psd.yaml:/app/harness/config/tht.yaml:ro
|
||||
- ${THT_PSD_WORKSPACE_HOST_PATH:?set THT_PSD_WORKSPACE_HOST_PATH}:/data/workspaces/psd
|
||||
|
||||
frontend:
|
||||
build:
|
||||
args:
|
||||
VITE_BASE: /
|
||||
VITE_BACKEND_URL: /api
|
||||
ports:
|
||||
- "127.0.0.1:8099:8080"
|
||||
networks: !override
|
||||
default:
|
||||
aliases: [frontend, thothii-frontend]
|
||||
|
||||
volumes:
|
||||
thoth_data:
|
||||
thoth_pi_config:
|
||||
|
||||
networks:
|
||||
default:
|
||||
external: true
|
||||
name: thothii_default
|
||||
@@ -18,8 +18,9 @@ THT_VEC_PASSWORD=__CHANGE_ME__
|
||||
THT_OLLAMA_URL=http://host.docker.internal:11434
|
||||
|
||||
# --- Backend ---
|
||||
AUTH_MODE=none # none | mock | oidc (in embedded l'auth è al bordo del portale)
|
||||
AUTH_MODE=none # none | mock | oidc (upstream auth is enforced at the proxy boundary)
|
||||
MAX_PI_PROCESSES=4
|
||||
THT_DEV_EVIDENCE_HOST_PATH=/absolute/path/to/evidence
|
||||
|
||||
# --- Git-backed workspace registry (no secret values belong in this file) ---
|
||||
THT_WORKSPACE_REGISTRY_ROOT=/data/workspace-registry
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# ThothII — deploy STANDALONE locale (dev / smoke test, senza portale).
|
||||
# ThothII — deploy STANDALONE locale (dev / smoke test).
|
||||
# Rete propria + porte host per ispezione diretta.
|
||||
# docker compose -f docker-compose.dev.yml up -d --build
|
||||
# frontend: http://localhost:8090 backend: http://localhost:8787
|
||||
@@ -40,10 +40,10 @@ services:
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
volumes:
|
||||
- /home/chirone/thothii-data:/data
|
||||
- dev-data:/data
|
||||
- workspace-registry:/data/workspace-registry
|
||||
- /home/chirone/thothii-data/pi-config:/home/thoth/.pi
|
||||
- /home/chirone/chirone/etl/docs/evidence:/data/evidence:ro
|
||||
- dev-pi-state:/home/thoth/.pi
|
||||
- ${THT_DEV_EVIDENCE_HOST_PATH:-./evidence}:/data/evidence:ro
|
||||
- ${THT_WORKSPACE_GIT_CREDENTIALS_FILE:-/dev/null}:/run/secrets/workspace-registry-git-credentials:ro
|
||||
- ${THT_WORKSPACE_GIT_CA_FILE:-/etc/ssl/certs/ca-certificates.crt}:/run/secrets/workspace-registry-git-ca:ro
|
||||
- ${THT_WORKSPACE_GIT_SSH_KEY_FILE:-/dev/null}:/run/secrets/workspace-registry-git-ssh-key:ro
|
||||
@@ -73,4 +73,6 @@ networks:
|
||||
driver: bridge
|
||||
|
||||
volumes:
|
||||
dev-data:
|
||||
dev-pi-state:
|
||||
workspace-registry:
|
||||
|
||||
@@ -64,11 +64,10 @@ RUN python -m venv /opt/venv \
|
||||
&& /opt/venv/bin/pip install --no-cache-dir --upgrade pip \
|
||||
&& (cd /app/harness && /opt/venv/bin/pip install --no-cache-dir .) \
|
||||
&& cp /app/harness/workflow.yaml /opt/venv/lib/python3.12/site-packages/workflow.yaml
|
||||
# Default locale e alias PSD convergono sul file canonico. Il CLI onora anche
|
||||
# THT_CONFIG, quindi cambiare CWD non cambia l'identita' dello workspace.
|
||||
# Il workspace locale predefinito converge sul file canonico. Il CLI onora anche THT_CONFIG,
|
||||
# quindi cambiare CWD non cambia l'identita' dello workspace.
|
||||
RUN mkdir -p /app/harness/config \
|
||||
&& cp --remove-destination /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml \
|
||||
&& ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/psd.yaml
|
||||
&& cp --remove-destination /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml
|
||||
# PiProcessManager (backend) prepende harnessDir/.venv/bin al PATH del child Pi → symlink al venv reale
|
||||
RUN ln -s /opt/venv /app/harness/.venv
|
||||
|
||||
|
||||
+2
-4
@@ -1,6 +1,4 @@
|
||||
# nginx per thothii-frontend: serve la SPA (modalità standalone) e reverse-proxy /api -> core.
|
||||
# In modalità embedded il portale proxya /datamart-builder/assets/ qui (solo asset statici);
|
||||
# il blocco /api non è usato in embedded (il portale hita core direttamente).
|
||||
# nginx per thothii-frontend: serve la SPA e inoltra /api al core sulla rete Compose.
|
||||
server {
|
||||
listen 8080;
|
||||
server_name _;
|
||||
@@ -29,7 +27,7 @@ server {
|
||||
chunked_transfer_encoding on;
|
||||
}
|
||||
|
||||
# manifest.json servito (lo legge il template tag Django in embedded)
|
||||
# manifest.json è servito come JSON.
|
||||
location = /manifest.json {
|
||||
default_type application/json;
|
||||
}
|
||||
|
||||
@@ -54,6 +54,8 @@ Il frontend renderizza questi widget-descriptor (registro in `src/widgets/`); il
|
||||
|
||||
## Come si lancia lo stack
|
||||
|
||||
Lo **stack completo** (Pi reale + DWH reale, serve VPN + `harness/.env` + `pi` sul PATH) si avvia con `./scripts/run-stack.sh` (frontend `:5173` → backend `:8787`).
|
||||
Lo stack locale si avvia con `./scripts/run-stack.sh`, dopo aver creato
|
||||
`deploy/env/local.env` da `deploy/env/local.env.example`. Il core Compose include Pi; DWH,
|
||||
vector DB, embedding e LLM sono endpoint esterni configurati nel file locale.
|
||||
|
||||
Comandi per singolo layer, test, lint: vedi il file `CLAUDE.md` nella radice del repo (guida operativa per Claude Code, tenuta sincronizzata con questa pagina).
|
||||
|
||||
@@ -45,13 +45,13 @@ services:
|
||||
- source: session_ca
|
||||
target: session_ca.pem
|
||||
networks:
|
||||
- portal
|
||||
- upstream
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
portal:
|
||||
upstream:
|
||||
external: true
|
||||
name: ${THT_PORTAL_NETWORK:-omics_portal_omics_network}
|
||||
name: ${THT_UPSTREAM_NETWORK:-thothii-upstream}
|
||||
|
||||
secrets:
|
||||
session_runtime_password:
|
||||
|
||||
@@ -14,27 +14,28 @@ Servono Docker Engine/Compose v2 su Linux oppure Docker Desktop su macOS/Windows
|
||||
```sh
|
||||
git clone <URL-REPOSITORY> ThothII
|
||||
cd ThothII
|
||||
cp .env.example .env
|
||||
mkdir -p deploy/secrets deploy/workspaces
|
||||
cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets
|
||||
chmod 600 deploy/secrets/thothii.secrets
|
||||
cp deploy/env/local.env.example deploy/env/local.env
|
||||
```
|
||||
|
||||
Modificare **solo** questi file interni al clone:
|
||||
|
||||
| File | Cosa contiene |
|
||||
|---|---|
|
||||
| `.env` | endpoint, database, provider, `COMPOSE_FILE` e `COMPOSE_PROFILES`; mai password/token |
|
||||
| `deploy/secrets/thothii.secrets` | un bundle `NOME=VALORE`, mode host `0600` o `0400` |
|
||||
| `deploy/env/local.env` | endpoint, database e path Pi locali; mai password/token |
|
||||
| file protetti locali | credenziali e certificati, indicati dai binding del workspace |
|
||||
| `deploy/workspaces/<nome>.yaml` | adapter, endpoint non riservati, `roots` ed Evidence |
|
||||
|
||||
Il file `.env` viene caricato automaticamente da Docker Compose perché è nella radice del progetto. Il valore predefinito è `COMPOSE_FILE=compose.yaml`, con profili vuoti e `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`. Perciò, dopo aver compilato `.env`, il bundle e almeno il workspace, l'avvio normale è sempre:
|
||||
Compilare `deploy/env/local.env`, incluso `PI_AUTH_FILE`, con gli endpoint esterni. L'avvio
|
||||
normale usa esplicitamente il file base e l'overlay locale:
|
||||
|
||||
```sh
|
||||
docker compose up --build -d
|
||||
docker compose --env-file deploy/env/local.env \
|
||||
-f compose.yaml -f deploy/compose.local.yaml up --build -d
|
||||
```
|
||||
|
||||
Non occorre usare `--env-file`, `-f` o `--profile` per questa installazione. Verificare lo stato con `docker compose ps` e aprire <http://127.0.0.1:8080>. `docker compose down` conserva il volume `thoth_data`; usare `down --volumes` solo per un ambiente effimero.
|
||||
Verificare lo stato con lo stesso comando Compose e aprire <http://127.0.0.1:8080>. Il core
|
||||
include Pi; il binario Pi non deve essere installato sull'host. `docker compose down` conserva i
|
||||
volumi; usare `down --volumes` solo per un ambiente effimero.
|
||||
|
||||
### Formato del bundle unico
|
||||
|
||||
@@ -55,21 +56,13 @@ Inserire solo le chiavi necessarie al profilo scelto. Il bundle viene montato in
|
||||
|
||||
Una catena CA PEM **non può essere inserita nel bundle**: contiene whitespace e viene rifiutata dal parser. Se un endpoint usa una CA privata, conservarla nel secret manager/host e aggiungere un override Compose revisionato che monti il file in `/run/secrets/ca-chain.pem` e imposti `THT_SSL_CA` (o il parametro dell'adapter). Il clone base non crea quel mount: questa è una limitazione intenzionale da considerare in fase di deployment.
|
||||
|
||||
### Overlay opzionali tramite `.env`
|
||||
### Overlay opzionali espliciti
|
||||
|
||||
Gli overlay non cambiano il comando operativo. Impostare in `.env`:
|
||||
|
||||
```dotenv
|
||||
# DWH/vector/embedding remoti (server applicativo o server con i DB):
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml
|
||||
COMPOSE_PROFILES=
|
||||
|
||||
# pgvector locale (Mac, Windows o server autonomo):
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml
|
||||
COMPOSE_PROFILES=local-vector
|
||||
```
|
||||
|
||||
Su Windows usare `;` come separatore di `COMPOSE_FILE`. Per il preprocessing locale aggiungere `deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml` e impostare `COMPOSE_PROFILES=local-vector,preprocess`; poi usare `docker compose run --rm preprocess-evidence` oppure `docker compose run --rm preprocess-dwh`.
|
||||
DWH/vector/embedding remoti restano endpoint del file locale o server. Per il solo preset di
|
||||
sviluppo pgvector, aggiungere `-f deploy/compose.local-vector.yaml --profile local-vector` al
|
||||
comando base. Per il preprocessing aggiungere anche
|
||||
`-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml --profile preprocess`,
|
||||
poi usare `docker compose run --rm preprocess-evidence` oppure `preprocess-dwh` con gli stessi argomenti.
|
||||
|
||||
## Workspace, adapter e Evidence
|
||||
|
||||
@@ -116,11 +109,10 @@ il bind mount/runtime adapter corrispondente. Non inserire la password nel works
|
||||
|
||||
## 1. Server remoto insieme ai database e al vector DB
|
||||
|
||||
Usare quando il server Docker è nella stessa rete del DWH e del vector DB (containerizzati o meno). Il file `.env` può restare sul default, senza profili, impostando gli endpoint raggiungibili localmente:
|
||||
Usare quando il server Docker è nella stessa rete del DWH e del vector DB (containerizzati o meno).
|
||||
Compilare `deploy/env/local.env` con gli endpoint raggiungibili localmente:
|
||||
|
||||
```dotenv
|
||||
COMPOSE_FILE=compose.yaml
|
||||
COMPOSE_PROFILES=
|
||||
THT_DB_NAME=warehouse
|
||||
THT_DWH_REST_URL=https://dwh.internal.example
|
||||
THT_VEC_REST_URL=https://vectors.internal.example
|
||||
@@ -143,17 +135,15 @@ claim normalizzati `X-Thoth-Principal-Issuer`, `X-Thoth-Principal-Subject`,
|
||||
`X-Thoth-Principal-Display-Name` e `X-Thoth-Is-Admin` attesi dal core. Non esporre direttamente
|
||||
la porta pubblicata da nginx.
|
||||
|
||||
Se il server deve essere raggiungibile da altri host, sostituire `COMPOSE_FILE` con
|
||||
`compose.yaml:deploy/compose.production.yaml`, configurare il proxy autenticato e impostare
|
||||
Se il server deve essere raggiungibile da altri host, usare il profilo
|
||||
`deploy/compose.server.yaml`, configurare il proxy autenticato e impostare
|
||||
`AUTH_MODE=upstream`/`THOTH_PUBLIC_EXPOSURE=true` come descritto nella sezione di trust boundary.
|
||||
|
||||
## 2. Mac locale
|
||||
|
||||
Installare Docker Desktop e, se usato, Ollama sul Mac. Nel `.env` selezionare il profilo locale:
|
||||
Installare Docker Desktop e, se usato, Ollama sul Mac. In `deploy/env/local.env` impostare gli endpoint:
|
||||
|
||||
```dotenv
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml
|
||||
COMPOSE_PROFILES=local-vector
|
||||
THT_DB_NAME=warehouse
|
||||
THT_DWH_REST_URL=https://dwh.example.test
|
||||
THT_OLLAMA_URL=http://host.docker.internal:11434
|
||||
@@ -173,11 +163,9 @@ Poi eseguire il comando standard `docker compose up --build -d`. Il primo avvio
|
||||
|
||||
## 3. PC Windows locale
|
||||
|
||||
Usare Docker Desktop con backend WSL2 e abilitare la condivisione della directory del clone. Modificare `.env` con il separatore Windows:
|
||||
Usare Docker Desktop con backend WSL2 e abilitare la condivisione della directory del clone. Modificare `deploy/env/local.env`:
|
||||
|
||||
```dotenv
|
||||
COMPOSE_FILE=compose.yaml;deploy/compose.local-vector.yaml
|
||||
COMPOSE_PROFILES=local-vector
|
||||
THT_DB_NAME=warehouse
|
||||
THT_DWH_REST_URL=https://dwh.example.test
|
||||
THT_OLLAMA_URL=http://host.docker.internal:11434
|
||||
@@ -195,11 +183,11 @@ Se un bind mount viene rifiutato, aggiungere la cartella del repository a Docker
|
||||
|
||||
## 4. Server applicativo distinto da DB ed Evidence
|
||||
|
||||
Usare il profilo production e consentire dal firewall solo le destinazioni necessarie:
|
||||
Usare il profilo server e consentire dal firewall solo le destinazioni necessarie:
|
||||
|
||||
```dotenv
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml
|
||||
COMPOSE_PROFILES=
|
||||
# Avvio: docker compose --env-file deploy/env/server.env \
|
||||
# -f compose.yaml -f deploy/compose.server.yaml up --build -d
|
||||
THT_DB_NAME=warehouse
|
||||
THT_DWH_REST_URL=https://dwh.example.test
|
||||
THT_VEC_REST_URL=https://vectors.example.test
|
||||
|
||||
@@ -3,13 +3,13 @@ import react from "@vitejs/plugin-react";
|
||||
import path from "path";
|
||||
|
||||
export default defineConfig(() => {
|
||||
const embedBase = process.env.VITE_BASE; // "/datamart-builder/assets/" in embedded; undefined = standalone
|
||||
const embedBase = process.env.VITE_BASE;
|
||||
const apiUpstream = process.env.THT_FRONTEND_API_UPSTREAM ?? "http://localhost:8787";
|
||||
return {
|
||||
plugins: [react()],
|
||||
// base: prefisso pubblico degli asset. Default "/" (standalone).
|
||||
// assetsDir vuoto in embedded → asset alla root di dist/ così il proxy
|
||||
// /datamart-builder/assets/ → frontend-root mappa 1:1 (niente /assets/assets/).
|
||||
// Con un prefisso personalizzato, gli asset restano alla root di dist/ per evitare
|
||||
// di duplicare il segmento assets nel percorso pubblico.
|
||||
base: embedBase ?? "/",
|
||||
build: { manifest: true, outDir: "dist", assetsDir: embedBase ? "" : "assets" },
|
||||
server: {
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
def test_local_compose_uses_the_generic_external_endpoint_contract():
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
compose = yaml.safe_load((root / "compose.yaml").read_text())
|
||||
local = yaml.safe_load((root / "deploy/compose.local.yaml").read_text())
|
||||
|
||||
assert set(compose["services"]) == {"core", "frontend"}
|
||||
assert local["services"]["core"]["environment"]["AUTH_MODE"] == "none"
|
||||
assert local["services"]["core"]["ports"] == ["127.0.0.1:${THOTH_CORE_HTTP_PORT:-8787}:8787"]
|
||||
assert local["services"]["frontend"]["ports"] == ["127.0.0.1:${THOTH_HTTP_PORT:-8080}:8080"]
|
||||
|
||||
environment = compose["services"]["core"]["environment"]
|
||||
for name in ("THT_DWH_REST_URL", "THT_VEC_REST_URL", "THT_OLLAMA_URL", "THT_LLM_URL"):
|
||||
assert name in environment
|
||||
assert {"settings", "pi-state", "workspace-registry", "sessions"} <= set(compose["volumes"])
|
||||
|
||||
|
||||
def test_core_image_prepares_the_writable_pi_profile_before_mounting_config_files():
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
dockerfile = (root / "docker/core.Dockerfile").read_text()
|
||||
|
||||
assert "mkdir -p /home/thoth/.pi/agent" in dockerfile
|
||||
assert "chown -R thoth:thoth /home/thoth/.pi" in dockerfile
|
||||
assert (
|
||||
"cp --remove-destination /app/harness/workspaces/local.yaml "
|
||||
"/app/harness/config/tht.yaml" in dockerfile
|
||||
)
|
||||
assert "ln -sf /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml" not in dockerfile
|
||||
assert "ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/" not in dockerfile
|
||||
@@ -1,133 +0,0 @@
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
class ComposeLoader(yaml.SafeLoader):
|
||||
pass
|
||||
|
||||
|
||||
def _compose_override(loader, node):
|
||||
if isinstance(node, yaml.MappingNode):
|
||||
return loader.construct_mapping(node)
|
||||
return loader.construct_sequence(node)
|
||||
|
||||
|
||||
ComposeLoader.add_constructor("!override", _compose_override)
|
||||
|
||||
|
||||
def test_psd_overlay_uses_generated_workspace_for_default_and_named_commands():
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
compose = yaml.load(
|
||||
(root / "deploy/compose.psd-local.yaml.example").read_text(),
|
||||
Loader=ComposeLoader,
|
||||
)
|
||||
core = compose["services"]["core"]
|
||||
|
||||
assert core["environment"]["THT_CONFIG"] == "/app/harness/config/tht.yaml"
|
||||
assert core["environment"]["THT_SECRETS_FILE"] == "/run/secrets/thothii.secrets"
|
||||
for name in (
|
||||
"THT_DB_NAME", "THT_DWH_REST_URL", "THT_VEC_REST_URL",
|
||||
"THT_VEC_WRITE_REST_URL", "THT_OLLAMA_URL", "THT_PROFILE",
|
||||
"PI_PROVIDER", "PI_MODEL", "PI_THINKING",
|
||||
):
|
||||
assert name in core["environment"]
|
||||
def target(volume):
|
||||
if isinstance(volume, dict):
|
||||
return volume["target"]
|
||||
parts = volume.rsplit(":", 2)
|
||||
return parts[-2] if parts[-1] in {"ro", "rw"} else parts[-1]
|
||||
|
||||
targets = {target(volume) for volume in core["volumes"]}
|
||||
assert "/app/harness/config/tht.yaml" in targets
|
||||
assert "/app/harness/workspaces/psd.yaml" not in targets
|
||||
assert "/data/workspaces/psd/config/tht.yaml" not in targets
|
||||
assert "/data" in targets
|
||||
assert "/home/thoth/.pi" in targets
|
||||
assert "/home/thoth/.pi/agent/models.json" in targets
|
||||
assert "/home/thoth/.pi/agent/settings.json" in targets
|
||||
assert "/data/evidence" in targets
|
||||
assert "/run/secrets/thothii.secrets" in targets
|
||||
assert set(compose["volumes"]) == {"thoth_data", "thoth_pi_config"}
|
||||
assert core["networks"]["default"]["aliases"] == ["core", "thothii-core"]
|
||||
frontend = compose["services"]["frontend"]
|
||||
assert frontend["ports"] == ["127.0.0.1:8099:8080"]
|
||||
assert frontend["build"]["args"] == {
|
||||
"VITE_BASE": "/",
|
||||
"VITE_BACKEND_URL": "/api",
|
||||
}
|
||||
assert frontend["networks"] == {
|
||||
"default": {"aliases": ["frontend", "thothii-frontend"]}
|
||||
}
|
||||
assert compose["networks"]["default"] == {
|
||||
"external": True,
|
||||
"name": "thothii_default",
|
||||
}
|
||||
|
||||
|
||||
def test_core_image_prepares_the_writable_pi_profile_before_mounting_config_files():
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
dockerfile = (root / "docker/core.Dockerfile").read_text()
|
||||
|
||||
assert "mkdir -p /home/thoth/.pi/agent" in dockerfile
|
||||
assert "chown -R thoth:thoth /home/thoth/.pi" in dockerfile
|
||||
assert (
|
||||
"cp --remove-destination /app/harness/workspaces/local.yaml "
|
||||
"/app/harness/config/tht.yaml" in dockerfile
|
||||
)
|
||||
assert "ln -sf /app/harness/workspaces/local.yaml /app/harness/config/tht.yaml" not in dockerfile
|
||||
assert (
|
||||
"ln -sfn /app/harness/config/tht.yaml /app/harness/workspaces/psd.yaml"
|
||||
in dockerfile
|
||||
)
|
||||
|
||||
|
||||
def test_psd_bootstrap_materializes_the_base_compose_env_file(tmp_path):
|
||||
source_root = Path(__file__).resolve().parents[2]
|
||||
root = tmp_path / "ThothII"
|
||||
(root / "scripts").mkdir(parents=True)
|
||||
(root / "deploy/workspaces").mkdir(parents=True)
|
||||
shutil.copy(
|
||||
source_root / "scripts/bootstrap-local-psd-docker-config.sh",
|
||||
root / "scripts/bootstrap-local-psd-docker-config.sh",
|
||||
)
|
||||
shutil.copy(
|
||||
source_root / "deploy/compose.psd-local.yaml.example",
|
||||
root / "deploy/compose.psd-local.yaml.example",
|
||||
)
|
||||
shutil.copy(
|
||||
source_root / "deploy/workspaces/psd.yaml.example",
|
||||
root / "deploy/workspaces/psd.yaml.example",
|
||||
)
|
||||
source_env = tmp_path / "source.env"
|
||||
source_env.write_text("\n".join([
|
||||
"THT_DB_NAME=postgres",
|
||||
"THT_DWH_REST_URL=https://dwh.invalid/",
|
||||
"THT_VEC_REST_URL=https://vec.invalid/read/",
|
||||
"THT_VEC_WRITE_REST_URL=https://vec.invalid/write/",
|
||||
"THT_DWH_API_KEY=dwh",
|
||||
"THT_VEC_API_KEY=reader",
|
||||
"THT_VEC_WRITE_API_KEY=writer",
|
||||
"",
|
||||
]))
|
||||
workspace = tmp_path / "workspace"
|
||||
workspace.mkdir()
|
||||
auth = tmp_path / "auth.json"
|
||||
auth.write_text('{"zai":{"key":"model"}}')
|
||||
|
||||
subprocess.run(
|
||||
[
|
||||
"sh", str(root / "scripts/bootstrap-local-psd-docker-config.sh"),
|
||||
str(source_env), str(workspace), str(auth),
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
assert (root / "deploy/thothii.env").is_file()
|
||||
assert "THT_SECRETS_FILE=./deploy/secrets/thothii.secrets" in (
|
||||
root / ".env"
|
||||
).read_text()
|
||||
@@ -1,70 +0,0 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
root=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
|
||||
source_env=${1:-"$root/../../harness/.env"}
|
||||
workspace=${2:-"$root/../../../tht-workspace-psd"}
|
||||
auth_file=${3:-"$HOME/.pi/agent/auth.json"}
|
||||
|
||||
value() {
|
||||
awk -F= -v key="$1" '$1 == key { sub(/^[^=]*=/, ""); sub(/[[:space:]].*$/, ""); print; exit }' "$source_env"
|
||||
}
|
||||
required() {
|
||||
result=$(value "$1")
|
||||
[ -n "$result" ] || { echo "missing $1 in local source configuration" >&2; exit 2; }
|
||||
printf '%s' "$result"
|
||||
}
|
||||
|
||||
test -f "$source_env"
|
||||
test -d "$workspace"
|
||||
test -f "$auth_file"
|
||||
ca=$(value THT_SSL_CA)
|
||||
[ -z "$ca" ] || test -f "$ca"
|
||||
model_key=$(jq -er '.zai.key' "$auth_file")
|
||||
test -n "$model_key"
|
||||
|
||||
umask 077
|
||||
mkdir -p "$root/deploy/secrets" "$root/deploy/workspaces"
|
||||
: >"$root/deploy/thothii.env"
|
||||
cp "$root/deploy/compose.psd-local.yaml.example" "$root/deploy/compose.psd-local.yaml"
|
||||
cp "$root/deploy/workspaces/psd.yaml.example" "$root/deploy/workspaces/psd.yaml"
|
||||
cat >"$root/.env" <<EOF
|
||||
COMPOSE_FILE=compose.yaml:deploy/compose.psd-local.yaml
|
||||
COMPOSE_PROFILES=
|
||||
THT_SECRETS_FILE=./deploy/secrets/thothii.secrets
|
||||
THOTH_HTTP_PORT=8080
|
||||
AUTH_MODE=none
|
||||
THOTH_PUBLIC_EXPOSURE=false
|
||||
MAX_PI_PROCESSES=4
|
||||
PI_PROVIDER=zai
|
||||
PI_MODEL=glm-5.2
|
||||
PI_THINKING=medium
|
||||
PI_AUTH_FILE=$auth_file
|
||||
THT_PROFILE=workstation
|
||||
THT_DB_NAME=$(required THT_DB_NAME)
|
||||
THT_DWH_REST_URL=$(required THT_DWH_REST_URL)
|
||||
THT_VEC_REST_URL=$(required THT_VEC_REST_URL)
|
||||
THT_VEC_WRITE_REST_URL=$(required THT_VEC_WRITE_REST_URL)
|
||||
THT_OLLAMA_URL=http://host.docker.internal:11434
|
||||
THT_DOCS_ROOT=/data/workspaces/psd
|
||||
THT_PSD_WORKSPACE_HOST_PATH=$workspace
|
||||
EOF
|
||||
cat >"$root/deploy/secrets/thothii.secrets" <<EOF
|
||||
THT_MODEL_API_KEY=$model_key
|
||||
THT_DWH_API_KEY=$(required THT_DWH_API_KEY)
|
||||
THT_VEC_API_KEY=$(required THT_VEC_API_KEY)
|
||||
THT_VEC_WRITE_API_KEY=$(required THT_VEC_WRITE_API_KEY)
|
||||
EOF
|
||||
if [ -n "$ca" ]; then
|
||||
cat >>"$root/deploy/compose.psd-local.yaml" <<EOF
|
||||
- type: bind
|
||||
source: $ca
|
||||
target: /run/secrets/ca-chain.pem
|
||||
read_only: true
|
||||
EOF
|
||||
printf '%s\n' 'THT_CA=/run/secrets/ca-chain.pem' >>"$root/deploy/secrets/thothii.secrets"
|
||||
else
|
||||
printf '%s\n' 'THT_CA=/etc/ssl/certs/ca-certificates.crt' >>"$root/deploy/secrets/thothii.secrets"
|
||||
fi
|
||||
chmod 600 "$root/.env" "$root/deploy/thothii.env" "$root/deploy/secrets/thothii.secrets"
|
||||
echo "Local PSD Docker configuration materialized without printing secret values."
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Smoke test del deploy standalone ThothII (core + frontend).
|
||||
# Usa docker-compose.dev.yml (rete propria, porte host). Non tocca il portale.
|
||||
# Prereq: deploy/thothii.env popolato + ruoli DB creati + pi-config + settings.json.
|
||||
# Usa docker-compose.dev.yml (rete propria, porte host).
|
||||
# Prereq: deploy/thothii.env popolato, endpoint esterni configurati e profilo Pi locale.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
|
||||
+12
-60
@@ -1,68 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
# run-stack.sh — avvia i 3 layer reali di ThothII per la validazione end-to-end.
|
||||
# run-stack.sh — avvia lo stack Compose locale di ThothII in primo piano.
|
||||
#
|
||||
# frontend (browser) → backend (Fastify :8787) → pi --mode rpc (GLM 5.2) → tht/harness → DWH
|
||||
# Il core include Pi; DWH, vector DB, embedding e LLM sono endpoint esterni configurati
|
||||
# in deploy/env/local.env. Non richiede un eseguibile Pi sull'host.
|
||||
#
|
||||
# Prerequisiti: VPN attiva, `pi` su PATH (GLM 5.2 configurato), harness/.env popolato,
|
||||
# harness/config/tht.yaml -> workspace cliente, deps installate nei 3 progetti.
|
||||
#
|
||||
# Uso: ./scripts/run-stack.sh
|
||||
# Stop: Ctrl-C (termina backend + frontend; i processi pi figli del backend muoiono con esso).
|
||||
# Preparazione: cp deploy/env/local.env.example deploy/env/local.env e compilare i valori.
|
||||
# Uso: ./scripts/run-stack.sh [argomenti aggiuntivi per docker compose up]
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
|
||||
HARNESS="$ROOT/harness"
|
||||
BACKEND="$ROOT/backend"
|
||||
FRONTEND="$ROOT/frontend"
|
||||
THT_BIN="$HARNESS/.venv/bin/tht"
|
||||
BACKEND_PORT="${BACKEND_PORT:-8787}"
|
||||
FRONTEND_PORT="${FRONTEND_PORT:-5173}"
|
||||
LOCAL_ENV_FILE="${THT_LOCAL_ENV_FILE:-$ROOT/deploy/env/local.env}"
|
||||
|
||||
# --- preflight ---------------------------------------------------------------
|
||||
[ -f "$HARNESS/.env" ] || { echo "ERRORE: $HARNESS/.env mancante (credenziali DWH/vector)"; exit 1; }
|
||||
[ -x "$THT_BIN" ] || { echo "ERRORE: $THT_BIN non trovato (esegui: cd harness && python -m venv .venv && pip install -e .)"; exit 1; }
|
||||
command -v pi >/dev/null || { echo "ERRORE: 'pi' non sul PATH (configura @earendil-works/pi-coding-agent con GLM 5.2)"; exit 1; }
|
||||
[ -e "$HARNESS/config/tht.yaml" ] || { echo "ERRORE: $HARNESS/config/tht.yaml mancante (symlink al workspace)"; exit 1; }
|
||||
[[ -f "$LOCAL_ENV_FILE" ]] || {
|
||||
echo "ERRORE: $LOCAL_ENV_FILE mancante. Copia deploy/env/local.env.example e configura gli endpoint." >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Carica le variabili del DWH/vector nell'ambiente: il backend le passa a pi e a tht.
|
||||
set -a; . "$HARNESS/.env"; set +a
|
||||
|
||||
# tht deve essere raggiungibile dal processo pi che il backend spawna.
|
||||
export PATH="$HARNESS/.venv/bin:$PATH"
|
||||
|
||||
echo "== ThothII stack =="
|
||||
echo " harness : $HARNESS (tht: $THT_BIN)"
|
||||
echo " backend : http://localhost:$BACKEND_PORT"
|
||||
echo " frontend: http://localhost:$FRONTEND_PORT"
|
||||
echo " pi : $(command -v pi)"
|
||||
echo
|
||||
|
||||
# --- avvio -------------------------------------------------------------------
|
||||
pids=()
|
||||
cleanup() { echo; echo "Arresto stack..."; for p in "${pids[@]}"; do kill "$p" 2>/dev/null || true; done; }
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
# Backend: usa il pi reale + il tht del venv, cwd harness per lo spawn di pi.
|
||||
(
|
||||
cd "$BACKEND"
|
||||
PORT="$BACKEND_PORT" \
|
||||
THT_HARNESS_DIR="$HARNESS" \
|
||||
THT_BIN="$THT_BIN" \
|
||||
PI_BIN="pi" \
|
||||
AUTH_MODE="none" \
|
||||
THT_DWH_PRECHECK="1" \
|
||||
npm run dev
|
||||
) &
|
||||
pids+=($!)
|
||||
|
||||
# Frontend: il browser usa sempre /api; Vite lo inoltra al backend locale.
|
||||
(
|
||||
cd "$FRONTEND"
|
||||
THT_FRONTEND_API_UPSTREAM="http://localhost:$BACKEND_PORT" \
|
||||
npm run dev -- --port "$FRONTEND_PORT"
|
||||
) &
|
||||
pids+=($!)
|
||||
|
||||
echo "Stack avviato. Apri http://localhost:$FRONTEND_PORT e crea una nuova domanda."
|
||||
echo "Ctrl-C per fermare."
|
||||
wait
|
||||
exec docker compose --env-file "$LOCAL_ENV_FILE" \
|
||||
-f "$ROOT/compose.yaml" -f "$ROOT/deploy/compose.local.yaml" up --build "$@"
|
||||
|
||||
@@ -6,6 +6,9 @@ project="thothii-external-lifecycle-$$"
|
||||
cleanup() { docker compose --project-name "$project" --profile external down --volumes >/dev/null 2>&1 || true; }
|
||||
trap cleanup EXIT HUP INT TERM
|
||||
|
||||
export THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
||||
export PI_AUTH_FILE=/dev/null
|
||||
|
||||
unset THT_VECTOR_BOOTSTRAP_PASSWORD_SECRET_FILE THT_VECTOR_MIGRATOR_PASSWORD_SECRET_FILE
|
||||
unset THT_VECTOR_READER_PASSWORD_SECRET_FILE THT_VECTOR_WRITER_PASSWORD_SECRET_FILE
|
||||
rendered=$(docker compose --project-name "$project" --profile external config)
|
||||
@@ -13,6 +16,10 @@ if printf '%s' "$rendered" | grep -q 'THT_VECTOR_.*PASSWORD_FILE\|vector_.*passw
|
||||
echo "external config contains local vector secret references" >&2
|
||||
exit 1
|
||||
fi
|
||||
if printf '%s' "$rendered" | grep -Eqi 'omics_portal|chirone|localllm_default|/home/chirone|datamart-builder'; then
|
||||
echo "external config contains deployment-specific coupling" >&2
|
||||
exit 1
|
||||
fi
|
||||
docker compose --project-name "$project" --profile external up --build --wait core
|
||||
core=$(docker compose --project-name "$project" --profile external ps -q core)
|
||||
inspect=$(docker inspect "$core")
|
||||
@@ -20,4 +27,8 @@ if printf '%s' "$inspect" | grep -q 'THT_VECTOR_.*PASSWORD_FILE\|/run/secrets/ve
|
||||
echo "external core inspect contains local vector secret references" >&2
|
||||
exit 1
|
||||
fi
|
||||
if printf '%s' "$inspect" | grep -Eqi 'omics_portal|chirone|localllm_default|/home/chirone|datamart-builder'; then
|
||||
echo "external core inspect contains deployment-specific coupling" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "external core lifecycle without local vector secrets passed."
|
||||
|
||||
@@ -54,13 +54,13 @@ if [ "$response" != '{"backend":"fastify","path":"/health"}' ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! rg -Fq 'THT_FRONTEND_API_UPSTREAM="http://localhost:$BACKEND_PORT"' "$ROOT/scripts/run-stack.sh"; then
|
||||
echo "run-stack.sh must configure the Vite internal upstream from BACKEND_PORT" >&2
|
||||
if ! rg -Fq -- '-f "$ROOT/compose.yaml" -f "$ROOT/deploy/compose.local.yaml" up --build "$@"' "$ROOT/scripts/run-stack.sh"; then
|
||||
echo "run-stack.sh must start the base+local Compose stack" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if rg -q 'VITE_BACKEND_URL' "$ROOT/scripts/run-stack.sh"; then
|
||||
echo "run-stack.sh must keep the browser base on /api" >&2
|
||||
if rg -q 'command -v pi|PI_BIN="pi"|PI_BIN=pi' "$ROOT/scripts/run-stack.sh"; then
|
||||
echo "run-stack.sh must not require a host Pi binary" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
Executable
+75
@@ -0,0 +1,75 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
content_targets=(
|
||||
.dockerignore
|
||||
compose.yaml
|
||||
docker-compose.dev.yml
|
||||
deploy
|
||||
docker
|
||||
frontend/vite.config.ts
|
||||
README.md
|
||||
docs/install
|
||||
docs/installazione-docker-4-contesti.md
|
||||
.env.example
|
||||
scripts/run-stack.sh
|
||||
scripts/docker-smoke.sh
|
||||
)
|
||||
|
||||
matches=$(
|
||||
rg -n -i \
|
||||
-g '!deploy/workspaces/**' \
|
||||
-g '!docker/session-migrate.sh' \
|
||||
-g '!docker/cutover-legacy-sessions.sh' \
|
||||
-g '!docker/smoke/**' \
|
||||
'omics_portal|chirone|localllm_default|datamart-builder|compose\.production\.yaml|compose\.psd-local\.yaml' \
|
||||
"${content_targets[@]}" || true
|
||||
)
|
||||
|
||||
runtime_psd_matches=$(
|
||||
rg -n -i \
|
||||
-g '!deploy/workspaces/**' \
|
||||
-g '!docker/session-migrate.sh' \
|
||||
-g '!docker/cutover-legacy-sessions.sh' \
|
||||
-g '!docker/smoke/**' \
|
||||
'\bpsd\b' \
|
||||
.dockerignore compose.yaml docker-compose.dev.yml deploy docker frontend/vite.config.ts \
|
||||
.env.example scripts/run-stack.sh scripts/docker-smoke.sh || true
|
||||
)
|
||||
|
||||
offenders=()
|
||||
for superseded_file in \
|
||||
deploy/compose.production.yaml \
|
||||
deploy/compose.psd-local.yaml.example \
|
||||
deploy/compose.psd-local.yaml \
|
||||
scripts/bootstrap-local-psd-docker-config.sh \
|
||||
harness/tests/test_psd_local_compose_contract.py
|
||||
do
|
||||
[[ ! -e "$superseded_file" ]] || offenders+=("$superseded_file (forbidden active deployment filename)")
|
||||
done
|
||||
|
||||
if [[ -n "$matches" ]]; then
|
||||
while IFS= read -r match; do
|
||||
offenders+=("$match")
|
||||
done <<<"$matches"
|
||||
fi
|
||||
|
||||
if [[ -n "$runtime_psd_matches" ]]; then
|
||||
while IFS= read -r match; do
|
||||
offenders+=("$match")
|
||||
done <<<"$runtime_psd_matches"
|
||||
fi
|
||||
|
||||
if rg -n 'command -v pi|PI_BIN="pi"|PI_BIN=pi' scripts/run-stack.sh >/dev/null; then
|
||||
offenders+=("scripts/run-stack.sh (requires a host Pi binary)")
|
||||
fi
|
||||
|
||||
if ((${#offenders[@]})); then
|
||||
printf '%s\n' "active deployment coupling found:" >&2
|
||||
printf '%s\n' "${offenders[@]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "no active PSD, Chirone, or portal deployment coupling found."
|
||||
@@ -9,7 +9,8 @@ auth_file="$tmp/auth.json"
|
||||
printf '%s\n' '{}' >"$auth_file"
|
||||
chmod 0600 "$auth_file"
|
||||
|
||||
rendered=$(PI_AUTH_FILE="$auth_file" docker compose config)
|
||||
rendered=$(THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
|
||||
PI_AUTH_FILE="$auth_file" docker compose config)
|
||||
printf '%s\n' "$rendered" | grep -q "source: $auth_file"
|
||||
printf '%s\n' "$rendered" | grep -q 'target: /home/thoth/.pi/agent/auth.json'
|
||||
printf '%s\n' "$rendered" | grep -A4 'target: /home/thoth/.pi/agent/auth.json' \
|
||||
@@ -29,6 +30,7 @@ assert settings["enabledModels"] == [
|
||||
PY
|
||||
|
||||
grep -q '^ARG PI_VERSION=0.80.3$' docker/core.Dockerfile
|
||||
grep -q '^PI_AUTH_FILE=$auth_file$' scripts/bootstrap-local-psd-docker-config.sh
|
||||
grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/local.env.example
|
||||
grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/server.env.example
|
||||
|
||||
echo "Pi user-auth Compose contract passed."
|
||||
|
||||
@@ -44,4 +44,17 @@ printf 'FROM scratch\n' > "$fixture_root/Dockerfile"
|
||||
|
||||
"$repo_root/scripts/verify-line-endings.sh" "$fixture_root"
|
||||
|
||||
git_fixture="$fixture_root/git-worktree"
|
||||
mkdir -p "$git_fixture"
|
||||
git -C "$git_fixture" init -q
|
||||
printf 'tracked then deleted\n' >"$git_fixture/deleted.md"
|
||||
git -C "$git_fixture" add deleted.md
|
||||
rm "$git_fixture/deleted.md"
|
||||
|
||||
git_output="$(cd "$git_fixture" && "$repo_root/scripts/verify-line-endings.sh" 2>&1)"
|
||||
if grep -Fq 'No such file or directory' <<<"$git_output"; then
|
||||
echo "line-ending verifier tried to read a tracked deletion" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "line-ending verifier tests passed"
|
||||
|
||||
@@ -23,6 +23,7 @@ root="$(cd "$root" && pwd)"
|
||||
offenders=()
|
||||
|
||||
while IFS= read -r -d '' file; do
|
||||
[[ -f "$file" ]] || continue
|
||||
case "$file" in
|
||||
*.ps1) continue ;;
|
||||
esac
|
||||
|
||||
Reference in New Issue
Block a user