fix: close deployment decoupling review
This commit is contained in:
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
# Common non-secret Compose values. Select local.env or server.env with --env-file.
|
# Common non-secret Compose values. Select local.env or server.env with --env-file.
|
||||||
# Run Compose with both files explicitly, for example:
|
# Run Compose with both files explicitly, for example:
|
||||||
# docker compose -f compose.yaml -f deploy/compose.local.yaml up -d --build
|
# docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local.yaml up -d --build
|
||||||
|
|
||||||
MAX_PI_PROCESSES=4
|
MAX_PI_PROCESSES=4
|
||||||
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
||||||
|
|||||||
+12
-1
@@ -6,7 +6,8 @@
|
|||||||
## Portable deployment decoupling — LIVE 2026-08-05
|
## Portable deployment decoupling — LIVE 2026-08-05
|
||||||
|
|
||||||
- **Mandatory stack.** The supported Compose stack is exactly `frontend` plus `core`; use the
|
- **Mandatory stack.** The supported Compose stack is exactly `frontend` plus `core`; use the
|
||||||
base file with `deploy/compose.local.yaml` or `deploy/compose.server.yaml`. `run-stack.sh`
|
base file with `deploy/compose.local.yaml`, or with `deploy/compose.server.yaml` plus the
|
||||||
|
required public-server session overlay. `run-stack.sh`
|
||||||
invokes the base+local Compose command and the core image provides Pi, so no host Pi binary is
|
invokes the base+local Compose command and the core image provides Pi, so no host Pi binary is
|
||||||
part of the launch contract.
|
part of the launch contract.
|
||||||
- **External boundaries.** DWH, vector DB, embedding, LLM, and reverse-proxy services are
|
- **External boundaries.** DWH, vector DB, embedding, LLM, and reverse-proxy services are
|
||||||
@@ -18,6 +19,16 @@
|
|||||||
remaining live contract checks were renamed for the generic local Compose profile. The coupling
|
remaining live contract checks were renamed for the generic local Compose profile. The coupling
|
||||||
gate rejects stale active deployment filenames and content while deliberately excluding
|
gate rejects stale active deployment filenames and content while deliberately excluding
|
||||||
historical plans/specs, canonical workspace descriptors, and non-runtime migration helpers.
|
historical plans/specs, canonical workspace descriptors, and non-runtime migration helpers.
|
||||||
|
- **Fresh provider and secret contract.** Local, server, and standalone development mount the
|
||||||
|
protected Pi auth JSON plus tracked declarative model/settings files read-only under
|
||||||
|
`/home/thoth/.pi/agent`. The existing strict application bundle is a core-only Docker secret at
|
||||||
|
`/run/secrets/thothii.secrets`; operator env files contain only its absolute source path.
|
||||||
|
Provider readiness is exercised from a fresh Compose volume through model listing, configuration,
|
||||||
|
and sanitized credential status.
|
||||||
|
- **Install and scan closure.** Superseded copied one-service installation examples and the
|
||||||
|
provider-owned-network test are retired. Active manuals use the canonical base plus local/server
|
||||||
|
and optional overrides, while the category-based coupling scan covers runtime, Docker smoke,
|
||||||
|
install, operator, and positive deployment-test contracts and propagates scanner errors.
|
||||||
|
|
||||||
## Portable Git workspace registry — source integration (2026-08-04)
|
## Portable Git workspace registry — source integration (2026-08-04)
|
||||||
|
|
||||||
|
|||||||
@@ -14,14 +14,22 @@ From a fresh clone, run these commands from the repository root:
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
cp deploy/env/local.env.example deploy/env/local.env
|
cp deploy/env/local.env.example deploy/env/local.env
|
||||||
# Edit deploy/env/local.env, including PI_AUTH_FILE and the external endpoint URLs.
|
# Edit deploy/env/local.env, including PI_AUTH_FILE, THT_SECRETS_FILE, and external endpoints.
|
||||||
docker compose --env-file deploy/env/local.env \
|
docker compose --env-file deploy/env/local.env \
|
||||||
-f compose.yaml -f deploy/compose.local.yaml up --build -d
|
-f compose.yaml -f deploy/compose.local.yaml up --build -d
|
||||||
```
|
```
|
||||||
|
|
||||||
`./scripts/run-stack.sh` runs this same base+local command in the foreground. The core image
|
`./scripts/run-stack.sh` runs this same base+local command in the foreground. The core image
|
||||||
contains its Pi runtime; no host `pi` executable is used. For a server installation, copy and
|
contains its Pi runtime; no host `pi` executable is used. For a server installation:
|
||||||
fill `deploy/env/server.env.example`, then use `-f compose.yaml -f deploy/compose.server.yaml`.
|
|
||||||
|
```sh
|
||||||
|
cp deploy/env/server.env.example deploy/env/server.env
|
||||||
|
# Edit all absolute storage, Pi/secret/session files, and endpoint paths.
|
||||||
|
docker compose --env-file deploy/env/server.env \
|
||||||
|
-f compose.yaml -f deploy/compose.server.yaml \
|
||||||
|
-f deploy/compose.session-server.yaml.example up --build -d
|
||||||
|
```
|
||||||
|
|
||||||
Workspace descriptors come from the Git remote configured by `THT_WORKSPACE_GIT_REMOTE`; their
|
Workspace descriptors come from the Git remote configured by `THT_WORKSPACE_GIT_REMOTE`; their
|
||||||
runtime endpoint and secret bindings remain installation-local. Open
|
runtime endpoint and secret bindings remain installation-local. Open
|
||||||
<http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in `deploy/env/local.env` to choose another
|
<http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in `deploy/env/local.env` to choose another
|
||||||
@@ -164,15 +172,10 @@ with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts thi
|
|||||||
rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other
|
rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other
|
||||||
auth mode fails during core startup.
|
auth mode fails during core startup.
|
||||||
|
|
||||||
Production credentials use the one Compose secret bundle, not an environment example. Put the
|
Production credentials use the existing Compose secret-bundle contract, never environment values.
|
||||||
required keys in `deploy/secrets/thothii.secrets` for the selected base+server installation:
|
Copy `deploy/secrets/thothii.secrets.example` to a protected host file, include only the required
|
||||||
|
keys, and set its absolute path as `THT_SECRETS_FILE` in the operator env. Keep Pi's native
|
||||||
```dotenv
|
provider auth in the separate protected file named by `PI_AUTH_FILE`.
|
||||||
THT_MODEL_API_KEY=replace-me
|
|
||||||
THT_DWH_API_KEY=replace-me
|
|
||||||
THT_VEC_API_KEY=replace-me
|
|
||||||
THT_VEC_WRITE_API_KEY=replace-me
|
|
||||||
```
|
|
||||||
|
|
||||||
The bundle is mounted read-only as `/run/secrets/thothii.secrets` and must be mode `0600` or
|
The bundle is mounted read-only as `/run/secrets/thothii.secrets` and must be mode `0600` or
|
||||||
`0400` on the host. Docker's runtime `0444` mode is accepted only beneath `/run/secrets`; see
|
`0400` on the host. Docker's runtime `0444` mode is accepted only beneath `/run/secrets`; see
|
||||||
@@ -204,9 +207,9 @@ still scrubbed. Supporting them requires a future dedicated provider-specific co
|
|||||||
|
|
||||||
The server profile stores sessions and per-user preferences directly in PostgreSQL schema
|
The server profile stores sessions and per-user preferences directly in PostgreSQL schema
|
||||||
`thoth_sessions`; it does not use PostgREST, browser storage, a shared session directory, or a
|
`thoth_sessions`; it does not use PostgREST, browser storage, a shared session directory, or a
|
||||||
dual write. Start from [`deploy/compose.session-server.yaml.example`](deploy/compose.session-server.yaml.example)
|
dual write. Use [`deploy/compose.session-server.yaml.example`](deploy/compose.session-server.yaml.example)
|
||||||
and copy [`deploy/workspaces/server-sessions.yaml.example`](deploy/workspaces/server-sessions.yaml.example)
|
with the canonical base+server files and set `THT_SERVER_WORKSPACE_CONFIG` to an absolute,
|
||||||
to the untracked `deploy/workspaces/server-sessions.yaml` mounted into the core container.
|
protected copy of [`deploy/workspaces/server-sessions.yaml.example`](deploy/workspaces/server-sessions.yaml.example).
|
||||||
|
|
||||||
The runtime login needs membership in the no-login database role `thoth_sessions_runtime` only.
|
The runtime login needs membership in the no-login database role `thoth_sessions_runtime` only.
|
||||||
The distinct, one-shot migrator login needs migration authority and uses
|
The distinct, one-shot migrator login needs migration authority and uses
|
||||||
@@ -242,7 +245,8 @@ proxy clears the legacy identity header and the backend rejects it. Drain/stop a
|
|||||||
enable a maintenance response at the proxy, then run the migrator once and inspect its pristine JSON:
|
enable a maintenance response at the proxy, then run the migrator once and inspect its pristine JSON:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker compose -f compose.yaml -f deploy/compose.session-server.yaml \
|
docker compose --env-file deploy/env/server.env \
|
||||||
|
-f compose.yaml -f deploy/compose.server.yaml -f deploy/compose.session-server.yaml.example \
|
||||||
--profile session-migrate run --rm session-migrate
|
--profile session-migrate run --rm session-migrate
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ services:
|
|||||||
THT_WORKSPACE_GIT_AUTHOR_NAME: ${THT_WORKSPACE_GIT_AUTHOR_NAME:-Thoth Workspace Registry}
|
THT_WORKSPACE_GIT_AUTHOR_NAME: ${THT_WORKSPACE_GIT_AUTHOR_NAME:-Thoth Workspace Registry}
|
||||||
THT_WORKSPACE_GIT_AUTHOR_EMAIL: ${THT_WORKSPACE_GIT_AUTHOR_EMAIL:-thoth-workspace-registry@localhost}
|
THT_WORKSPACE_GIT_AUTHOR_EMAIL: ${THT_WORKSPACE_GIT_AUTHOR_EMAIL:-thoth-workspace-registry@localhost}
|
||||||
THT_WORKSPACE_SECRET_ROOTS: /run/secrets
|
THT_WORKSPACE_SECRET_ROOTS: /run/secrets
|
||||||
|
THT_SECRETS_FILE: /run/secrets/thothii.secrets
|
||||||
THT_DB_NAME: ${THT_DB_NAME:-}
|
THT_DB_NAME: ${THT_DB_NAME:-}
|
||||||
THT_DWH_REST_URL: ${THT_DWH_REST_URL:-}
|
THT_DWH_REST_URL: ${THT_DWH_REST_URL:-}
|
||||||
THT_VEC_REST_URL: ${THT_VEC_REST_URL:-}
|
THT_VEC_REST_URL: ${THT_VEC_REST_URL:-}
|
||||||
@@ -32,8 +33,13 @@ services:
|
|||||||
- settings:/data/settings
|
- settings:/data/settings
|
||||||
- pi-state:/home/thoth/.pi
|
- pi-state:/home/thoth/.pi
|
||||||
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
||||||
|
- ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro
|
||||||
|
- ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro
|
||||||
- workspace-registry:/data/workspace-registry
|
- workspace-registry:/data/workspace-registry
|
||||||
- sessions:/data/sessions
|
- sessions:/data/sessions
|
||||||
|
secrets:
|
||||||
|
- source: thothii_secrets
|
||||||
|
target: thothii.secrets
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8787/health"]
|
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8787/health"]
|
||||||
interval: 15s
|
interval: 15s
|
||||||
@@ -71,3 +77,7 @@ volumes:
|
|||||||
pi-state:
|
pi-state:
|
||||||
workspace-registry:
|
workspace-registry:
|
||||||
sessions:
|
sessions:
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
thothii_secrets:
|
||||||
|
file: "${THT_SECRETS_FILE:?set THT_SECRETS_FILE}"
|
||||||
|
|||||||
@@ -33,3 +33,6 @@ services:
|
|||||||
- thoth_data:/data
|
- thoth_data:/data
|
||||||
- ./deploy/workspaces:/app/harness/workspaces:ro
|
- ./deploy/workspaces:/app/harness/workspaces:ro
|
||||||
restart: "no"
|
restart: "no"
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
thoth_data:
|
||||||
|
|||||||
@@ -9,6 +9,8 @@ services:
|
|||||||
- ${THT_DATA_ROOT:?set THT_DATA_ROOT}:/data
|
- ${THT_DATA_ROOT:?set THT_DATA_ROOT}:/data
|
||||||
- ${THT_PI_STATE_ROOT:?set THT_PI_STATE_ROOT}:/home/thoth/.pi
|
- ${THT_PI_STATE_ROOT:?set THT_PI_STATE_ROOT}:/home/thoth/.pi
|
||||||
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
||||||
|
- ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro
|
||||||
|
- ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro
|
||||||
- ${THT_WORKSPACE_REGISTRY_ROOT:?set THT_WORKSPACE_REGISTRY_ROOT}:/data/workspace-registry
|
- ${THT_WORKSPACE_REGISTRY_ROOT:?set THT_WORKSPACE_REGISTRY_ROOT}:/data/workspace-registry
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ services:
|
|||||||
- source: session_ca
|
- source: session_ca
|
||||||
target: session_ca.pem
|
target: session_ca.pem
|
||||||
volumes:
|
volumes:
|
||||||
- ./deploy/workspaces:/app/harness/workspaces:ro
|
- ${THT_SERVER_WORKSPACE_CONFIG:?set THT_SERVER_WORKSPACE_CONFIG}:/app/harness/workspaces/server-sessions.yaml:ro
|
||||||
|
|
||||||
# Run manually during the maintenance window. It is not a dependency of core,
|
# Run manually during the maintenance window. It is not a dependency of core,
|
||||||
# so the application never gains the schema-changing migrator credential.
|
# so the application never gains the schema-changing migrator credential.
|
||||||
|
|||||||
@@ -1,40 +0,0 @@
|
|||||||
# Deprecated compatibility template; it is not loaded by Docker Compose automatically.
|
|
||||||
# New installations must copy ../.env.example to ../.env and run
|
|
||||||
# `docker compose up --build -d` from the repository root. Keep this file only for
|
|
||||||
# staged upgrades that still invoke `--env-file deploy/env.example` explicitly.
|
|
||||||
# Never put secret values in this file.
|
|
||||||
|
|
||||||
COMPOSE_FILE=compose.yaml
|
|
||||||
COMPOSE_PROFILES=
|
|
||||||
THT_SECRETS_FILE=deploy/secrets/thothii.secrets
|
|
||||||
|
|
||||||
PI_PROVIDER=
|
|
||||||
PI_MODEL=
|
|
||||||
PI_THINKING=
|
|
||||||
MAX_PI_PROCESSES=4
|
|
||||||
AUTH_MODE=none
|
|
||||||
|
|
||||||
# User-owned session storage. Keep local for the loopback-only development stack.
|
|
||||||
# The server-session overlay requires every THT_SESSION_* value below.
|
|
||||||
THT_SESSION_STORAGE=local
|
|
||||||
THT_SESSION_DB_HOST=
|
|
||||||
THT_SESSION_DB_PORT=5432
|
|
||||||
THT_SESSION_DB_NAME=
|
|
||||||
THT_SESSION_RUNTIME_USER=
|
|
||||||
THT_SESSION_RUNTIME_PASSWORD_SOURCE=
|
|
||||||
THT_SESSION_MIGRATOR_USER=
|
|
||||||
THT_SESSION_MIGRATOR_PASSWORD_SOURCE=
|
|
||||||
THT_SESSION_DB_SSLMODE=verify-full
|
|
||||||
THT_SESSION_CA_SOURCE=
|
|
||||||
|
|
||||||
THT_DB_NAME=
|
|
||||||
THT_DWH_REST_URL=
|
|
||||||
THT_VEC_REST_URL=
|
|
||||||
THT_OLLAMA_URL=
|
|
||||||
THT_DOCS_ROOT=/data/workspaces/example/evidence-source
|
|
||||||
|
|
||||||
THT_VECTOR_DATABASE=thoth
|
|
||||||
THT_VECTOR_BOOTSTRAP_USER=postgres
|
|
||||||
THT_VECTOR_MIGRATOR_USER=thoth_vector_migrator
|
|
||||||
THT_VECTOR_READER_USER=thoth_vector_reader
|
|
||||||
THT_VECTOR_WRITER_USER=thoth_vector_writer
|
|
||||||
Vendored
+1
@@ -4,6 +4,7 @@ THOTH_HTTP_PORT=8080
|
|||||||
THOTH_CORE_HTTP_PORT=8787
|
THOTH_CORE_HTTP_PORT=8787
|
||||||
MAX_PI_PROCESSES=4
|
MAX_PI_PROCESSES=4
|
||||||
PI_AUTH_FILE=/absolute/path/to/pi-auth.json
|
PI_AUTH_FILE=/absolute/path/to/pi-auth.json
|
||||||
|
THT_SECRETS_FILE=/absolute/path/to/thothii.secrets
|
||||||
|
|
||||||
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
||||||
THT_WORKSPACE_GIT_BRANCH=main
|
THT_WORKSPACE_GIT_BRANCH=main
|
||||||
|
|||||||
Vendored
+13
@@ -4,10 +4,12 @@ THOTH_SERVER_BIND=127.0.0.1
|
|||||||
THOTH_HTTP_PORT=8080
|
THOTH_HTTP_PORT=8080
|
||||||
MAX_PI_PROCESSES=4
|
MAX_PI_PROCESSES=4
|
||||||
PI_AUTH_FILE=/absolute/path/to/pi-auth.json
|
PI_AUTH_FILE=/absolute/path/to/pi-auth.json
|
||||||
|
THT_SECRETS_FILE=/absolute/path/to/thothii.secrets
|
||||||
|
|
||||||
THT_DATA_ROOT=/srv/thothii/data
|
THT_DATA_ROOT=/srv/thothii/data
|
||||||
THT_PI_STATE_ROOT=/srv/thothii/pi-state
|
THT_PI_STATE_ROOT=/srv/thothii/pi-state
|
||||||
THT_WORKSPACE_REGISTRY_ROOT=/srv/thothii/workspace-registry
|
THT_WORKSPACE_REGISTRY_ROOT=/srv/thothii/workspace-registry
|
||||||
|
THT_SERVER_WORKSPACE_CONFIG=/absolute/path/to/server-sessions.yaml
|
||||||
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
||||||
THT_WORKSPACE_GIT_BRANCH=main
|
THT_WORKSPACE_GIT_BRANCH=main
|
||||||
THT_WORKSPACE_GIT_AUTHOR_NAME="Thoth Workspace Registry"
|
THT_WORKSPACE_GIT_AUTHOR_NAME="Thoth Workspace Registry"
|
||||||
@@ -19,3 +21,14 @@ THT_VEC_REST_URL=https://vector.example.invalid
|
|||||||
THT_VEC_WRITE_REST_URL=https://vector-write.example.invalid
|
THT_VEC_WRITE_REST_URL=https://vector-write.example.invalid
|
||||||
THT_OLLAMA_URL=https://embeddings.example.invalid
|
THT_OLLAMA_URL=https://embeddings.example.invalid
|
||||||
THT_LLM_URL=https://llm.example.invalid
|
THT_LLM_URL=https://llm.example.invalid
|
||||||
|
|
||||||
|
# Public server session storage. Values are endpoints, roles, or protected source-file paths.
|
||||||
|
THT_SESSION_DB_HOST=sessions-db.example.invalid
|
||||||
|
THT_SESSION_DB_PORT=5432
|
||||||
|
THT_SESSION_DB_NAME=thoth_sessions
|
||||||
|
THT_SESSION_RUNTIME_USER=thoth_sessions_app
|
||||||
|
THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate
|
||||||
|
THT_SESSION_DB_SSLMODE=verify-full
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_SOURCE=/absolute/path/to/session-runtime-password
|
||||||
|
THT_SESSION_MIGRATOR_PASSWORD_SOURCE=/absolute/path/to/session-migrator-password
|
||||||
|
THT_SESSION_CA_SOURCE=/absolute/path/to/session-ca.pem
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ The file uses strict `KEY=VALUE` lines (comments and blank lines are allowed). T
|
|||||||
keys are `THT_MODEL_API_KEY`, `THT_DWH_API_KEY`, `THT_VEC_API_KEY`,
|
keys are `THT_MODEL_API_KEY`, `THT_DWH_API_KEY`, `THT_VEC_API_KEY`,
|
||||||
`THT_VEC_WRITE_API_KEY`, and the four `THT_VECTOR_*_PASSWORD` role passwords. Values must be
|
`THT_VEC_WRITE_API_KEY`, and the four `THT_VECTOR_*_PASSWORD` role passwords. Values must be
|
||||||
non-empty and contain no whitespace. Do not put secrets in the root `.env`, workspace YAML,
|
non-empty and contain no whitespace. Do not put secrets in the root `.env`, workspace YAML,
|
||||||
URLs, logs, or `docker compose config` output.
|
URLs, logs, or rendered Compose output.
|
||||||
|
|
||||||
Compose mounts the bundle read-only as `/run/secrets/thothii.secrets`. The host file must be a
|
Compose mounts the bundle read-only as `/run/secrets/thothii.secrets`. The host file must be a
|
||||||
regular non-symlink file with mode `0600` or `0400`; Docker's normal `0444` mode is accepted
|
regular non-symlink file with mode `0600` or `0400`; Docker's normal `0444` mode is accepted
|
||||||
@@ -20,7 +20,9 @@ only for the runtime mount beneath `/run/secrets`. The core runs as UID 10001. V
|
|||||||
without printing its contents:
|
without printing its contents:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker compose run --rm core sh -c 'id && test -r /run/secrets/thothii.secrets'
|
docker compose --env-file deploy/env/local.env \
|
||||||
|
-f compose.yaml -f deploy/compose.local.yaml \
|
||||||
|
run --rm core sh -c 'id && test -r /run/secrets/thothii.secrets'
|
||||||
```
|
```
|
||||||
|
|
||||||
A private CA PEM chain is not a bundle value: PEM whitespace is rejected by the strict parser.
|
A private CA PEM chain is not a bundle value: PEM whitespace is rejected by the strict parser.
|
||||||
@@ -31,9 +33,10 @@ Compose files intentionally do not create this mount.
|
|||||||
## Migration from separate secret files
|
## Migration from separate secret files
|
||||||
|
|
||||||
Older installations used `THT_*_SECRET_FILE` variables and one file per value. Migrate by
|
Older installations used `THT_*_SECRET_FILE` variables and one file per value. Migrate by
|
||||||
copying each value to its bundle key, validating with `docker compose config --quiet`, and only
|
copying each value to its bundle key, validating with the complete base+profile command, and only
|
||||||
then deleting the old files. The old variables remain a compatibility path for staged upgrades,
|
then deleting the old files. The old variables remain a compatibility path for staged upgrades,
|
||||||
but the documented and tested default is `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`.
|
but the documented and tested default is an absolute `THT_SECRETS_FILE` path to the protected
|
||||||
|
bundle.
|
||||||
|
|
||||||
The local-vector bootstrap rotation helper still accepts an old/new password file as its
|
The local-vector bootstrap rotation helper still accepts an old/new password file as its
|
||||||
maintenance interface. Run it only with files protected by `0600`, then copy the resulting
|
maintenance interface. Run it only with files protected by `0600`, then copy the resulting
|
||||||
|
|||||||
@@ -1,33 +0,0 @@
|
|||||||
# ThothII core — env di runtime (compose env_file).
|
|
||||||
# Copiare in deploy/thothii.env e completare. NON committare thothii.env.
|
|
||||||
|
|
||||||
# --- DWH (direct, ruolo read-only su schema datawarehouse) ---
|
|
||||||
THT_DB_HOST=host.docker.internal
|
|
||||||
THT_DB_PORT=5438
|
|
||||||
THT_DB_NAME=postgres
|
|
||||||
THT_DB_USER=thoth_dwh_reader
|
|
||||||
THT_DB_PASSWORD=__CHANGE_ME__
|
|
||||||
|
|
||||||
# --- Vector (direct, ruolo read+write su schema vectors; stessa istanza del DWH) ---
|
|
||||||
THT_VEC_HOST=host.docker.internal
|
|
||||||
THT_VEC_PORT=5438
|
|
||||||
THT_VEC_USER=thoth_vector_rw
|
|
||||||
THT_VEC_PASSWORD=__CHANGE_ME__
|
|
||||||
|
|
||||||
# --- Embeddings (Ollama sull'host, modello nomic-embed-text-v2-moe) ---
|
|
||||||
THT_OLLAMA_URL=http://host.docker.internal:11434
|
|
||||||
|
|
||||||
# --- Backend ---
|
|
||||||
AUTH_MODE=none # none | mock | oidc (upstream auth is enforced at the proxy boundary)
|
|
||||||
MAX_PI_PROCESSES=4
|
|
||||||
THT_DEV_EVIDENCE_HOST_PATH=/absolute/path/to/evidence
|
|
||||||
|
|
||||||
# --- Git-backed workspace registry (no secret values belong in this file) ---
|
|
||||||
THT_WORKSPACE_REGISTRY_ROOT=/data/workspace-registry
|
|
||||||
THT_WORKSPACE_GIT_BRANCH=main
|
|
||||||
THT_WORKSPACE_INSTALLATION_ID=server
|
|
||||||
# THT_WORKSPACE_GIT_REMOTE=ssh://git@your-git-host/your-org/thoth-workspaces.git
|
|
||||||
# THT_WORKSPACE_GIT_CREDENTIALS_FILE=/absolute/path/to/git-credentials
|
|
||||||
# THT_WORKSPACE_GIT_CA_FILE=/absolute/path/to/git-ca.pem
|
|
||||||
# THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/to/git-ssh-key
|
|
||||||
# THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/to/git-known-hosts
|
|
||||||
+33
-20
@@ -1,7 +1,7 @@
|
|||||||
# ThothII — deploy STANDALONE locale (dev / smoke test).
|
# ThothII standalone development/smoke stack.
|
||||||
# Rete propria + porte host per ispezione diretta.
|
# Run with the canonical local env file:
|
||||||
# docker compose -f docker-compose.dev.yml up -d --build
|
# docker compose --env-file deploy/env/local.env -f docker-compose.dev.yml up -d --build
|
||||||
# frontend: http://localhost:8090 backend: http://localhost:8787
|
# frontend: http://localhost:8090 backend: http://localhost:8787
|
||||||
name: thothii-dev
|
name: thothii-dev
|
||||||
|
|
||||||
services:
|
services:
|
||||||
@@ -10,19 +10,18 @@ services:
|
|||||||
context: .
|
context: .
|
||||||
dockerfile: docker/core.Dockerfile
|
dockerfile: docker/core.Dockerfile
|
||||||
image: thothii-core:local
|
image: thothii-core:local
|
||||||
env_file:
|
|
||||||
- path: deploy/thothii.env
|
|
||||||
required: false
|
|
||||||
environment:
|
environment:
|
||||||
HOST: 0.0.0.0
|
HOST: 0.0.0.0
|
||||||
PORT: "8787"
|
PORT: "8787"
|
||||||
THT_HARNESS_DIR: /app/harness
|
THT_HARNESS_DIR: /app/harness
|
||||||
THT_BIN: /opt/venv/bin/tht
|
THT_BIN: /opt/venv/bin/tht
|
||||||
PI_BIN: pi
|
PI_BIN: pi
|
||||||
AUTH_MODE: ${AUTH_MODE:-none}
|
AUTH_MODE: none
|
||||||
THT_SESSION_STORAGE: local
|
THT_SESSION_STORAGE: local
|
||||||
THT_HOME: /data/local-home
|
THT_HOME: /data/local-home
|
||||||
|
THT_DATA_ROOT: /data
|
||||||
SETTINGS_FILE: /data/settings/settings.json
|
SETTINGS_FILE: /data/settings/settings.json
|
||||||
|
THT_MAINTENANCE_FILE: /data/settings/maintenance.json
|
||||||
THT_WORKSPACE_REGISTRY_ROOT: /data/workspace-registry
|
THT_WORKSPACE_REGISTRY_ROOT: /data/workspace-registry
|
||||||
THT_WORKSPACE_GIT_REMOTE: ${THT_WORKSPACE_GIT_REMOTE:?set THT_WORKSPACE_GIT_REMOTE}
|
THT_WORKSPACE_GIT_REMOTE: ${THT_WORKSPACE_GIT_REMOTE:?set THT_WORKSPACE_GIT_REMOTE}
|
||||||
THT_WORKSPACE_GIT_BRANCH: ${THT_WORKSPACE_GIT_BRANCH:-main}
|
THT_WORKSPACE_GIT_BRANCH: ${THT_WORKSPACE_GIT_BRANCH:-main}
|
||||||
@@ -30,26 +29,35 @@ services:
|
|||||||
THT_WORKSPACE_GIT_AUTHOR_NAME: ${THT_WORKSPACE_GIT_AUTHOR_NAME:-Thoth Workspace Registry}
|
THT_WORKSPACE_GIT_AUTHOR_NAME: ${THT_WORKSPACE_GIT_AUTHOR_NAME:-Thoth Workspace Registry}
|
||||||
THT_WORKSPACE_GIT_AUTHOR_EMAIL: ${THT_WORKSPACE_GIT_AUTHOR_EMAIL:-thoth-workspace-registry@localhost}
|
THT_WORKSPACE_GIT_AUTHOR_EMAIL: ${THT_WORKSPACE_GIT_AUTHOR_EMAIL:-thoth-workspace-registry@localhost}
|
||||||
THT_WORKSPACE_SECRET_ROOTS: /run/secrets
|
THT_WORKSPACE_SECRET_ROOTS: /run/secrets
|
||||||
GIT_CONFIG_COUNT: "2"
|
THT_SECRETS_FILE: /run/secrets/thothii.secrets
|
||||||
GIT_CONFIG_KEY_0: credential.helper
|
THT_DB_NAME: ${THT_DB_NAME:-}
|
||||||
GIT_CONFIG_VALUE_0: store --file=/run/secrets/workspace-registry-git-credentials
|
THT_DWH_REST_URL: ${THT_DWH_REST_URL:-}
|
||||||
GIT_CONFIG_KEY_1: http.sslCAInfo
|
THT_VEC_REST_URL: ${THT_VEC_REST_URL:-}
|
||||||
GIT_CONFIG_VALUE_1: /run/secrets/workspace-registry-git-ca
|
THT_VEC_WRITE_REST_URL: ${THT_VEC_WRITE_REST_URL:-}
|
||||||
GIT_SSH_COMMAND: ssh -i /run/secrets/workspace-registry-git-ssh-key -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=/run/secrets/workspace-registry-git-known-hosts
|
THT_OLLAMA_URL: ${THT_OLLAMA_URL:-}
|
||||||
|
THT_LLM_URL: ${THT_LLM_URL:-}
|
||||||
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
||||||
extra_hosts:
|
extra_hosts:
|
||||||
- "host.docker.internal:host-gateway"
|
- "host.docker.internal:host-gateway"
|
||||||
volumes:
|
volumes:
|
||||||
- dev-data:/data
|
- dev-data:/data
|
||||||
- workspace-registry:/data/workspace-registry
|
|
||||||
- dev-pi-state:/home/thoth/.pi
|
- dev-pi-state:/home/thoth/.pi
|
||||||
|
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
||||||
|
- ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro
|
||||||
|
- ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro
|
||||||
|
- workspace-registry:/data/workspace-registry
|
||||||
- ${THT_DEV_EVIDENCE_HOST_PATH:-./evidence}:/data/evidence:ro
|
- ${THT_DEV_EVIDENCE_HOST_PATH:-./evidence}:/data/evidence:ro
|
||||||
- ${THT_WORKSPACE_GIT_CREDENTIALS_FILE:-/dev/null}:/run/secrets/workspace-registry-git-credentials:ro
|
secrets:
|
||||||
- ${THT_WORKSPACE_GIT_CA_FILE:-/etc/ssl/certs/ca-certificates.crt}:/run/secrets/workspace-registry-git-ca:ro
|
- source: thothii_secrets
|
||||||
- ${THT_WORKSPACE_GIT_SSH_KEY_FILE:-/dev/null}:/run/secrets/workspace-registry-git-ssh-key:ro
|
target: thothii.secrets
|
||||||
- ${THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE:-/dev/null}:/run/secrets/workspace-registry-git-known-hosts:ro
|
|
||||||
ports:
|
ports:
|
||||||
- "127.0.0.1:8787:8787"
|
- "127.0.0.1:8787:8787"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8787/health"]
|
||||||
|
interval: 15s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
start_period: 30s
|
||||||
restart: "no"
|
restart: "no"
|
||||||
networks: [thothii-net]
|
networks: [thothii-net]
|
||||||
|
|
||||||
@@ -64,7 +72,8 @@ services:
|
|||||||
ports:
|
ports:
|
||||||
- "127.0.0.1:8090:8080"
|
- "127.0.0.1:8090:8080"
|
||||||
depends_on:
|
depends_on:
|
||||||
- core
|
core:
|
||||||
|
condition: service_healthy
|
||||||
restart: "no"
|
restart: "no"
|
||||||
networks: [thothii-net]
|
networks: [thothii-net]
|
||||||
|
|
||||||
@@ -76,3 +85,7 @@ volumes:
|
|||||||
dev-data:
|
dev-data:
|
||||||
dev-pi-state:
|
dev-pi-state:
|
||||||
workspace-registry:
|
workspace-registry:
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
thothii_secrets:
|
||||||
|
file: "${THT_SECRETS_FILE:?set THT_SECRETS_FILE}"
|
||||||
|
|||||||
+2
-2
@@ -8,8 +8,8 @@ La documentazione è divisa in due aree:
|
|||||||
|
|
||||||
Come funziona il sistema: architettura, specifiche di design delle singole funzionalità, piani di implementazione, report di test. Parte da qui: [Panoramica dell'architettura](architecture/overview.md).
|
Come funziona il sistema: architettura, specifiche di design delle singole funzionalità, piani di implementazione, report di test. Parte da qui: [Panoramica dell'architettura](architecture/overview.md).
|
||||||
|
|
||||||
Per installare l'applicazione in Docker nei quattro contesti operativi, partendo dal comando
|
Per installare l'applicazione in Docker nei quattro contesti operativi, usando il file env,
|
||||||
predefinito `docker compose up --build -d` e dal bundle unico dei secret:
|
`compose.yaml`, l'overlay locale/server e il bundle di secret montato:
|
||||||
[Installazione Docker nei quattro contesti](installazione-docker-4-contesti.md).
|
[Installazione Docker nei quattro contesti](installazione-docker-4-contesti.md).
|
||||||
|
|
||||||
## Considerazioni Generali
|
## Considerazioni Generali
|
||||||
|
|||||||
@@ -1,13 +0,0 @@
|
|||||||
# Optional override for an HTTPS Git remote. Both source paths are required absolute paths to
|
|
||||||
# existing operator-managed files; neither file content belongs in the base Compose example.
|
|
||||||
services:
|
|
||||||
core:
|
|
||||||
environment:
|
|
||||||
GIT_CONFIG_COUNT: "2"
|
|
||||||
GIT_CONFIG_KEY_0: credential.helper
|
|
||||||
GIT_CONFIG_VALUE_0: store --file=/run/secrets/workspace-registry-git-credentials
|
|
||||||
GIT_CONFIG_KEY_1: http.sslCAInfo
|
|
||||||
GIT_CONFIG_VALUE_1: /run/secrets/workspace-registry-git-ca
|
|
||||||
volumes:
|
|
||||||
- ${THT_WORKSPACE_GIT_CREDENTIALS_FILE:?set THT_WORKSPACE_GIT_CREDENTIALS_FILE}:/run/secrets/workspace-registry-git-credentials:ro
|
|
||||||
- ${THT_WORKSPACE_GIT_CA_FILE:?set THT_WORKSPACE_GIT_CA_FILE}:/run/secrets/workspace-registry-git-ca:ro
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
# Optional override for an SSH Git remote. Source paths are required absolute operator-managed
|
|
||||||
# files. Host-key checking remains strict; do not add a fallback known-hosts or key mount.
|
|
||||||
services:
|
|
||||||
core:
|
|
||||||
environment:
|
|
||||||
GIT_SSH_COMMAND: ssh -i /run/secrets/workspace-registry-git-ssh-key -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=/run/secrets/workspace-registry-git-known-hosts
|
|
||||||
volumes:
|
|
||||||
- ${THT_WORKSPACE_GIT_SSH_KEY_FILE:?set THT_WORKSPACE_GIT_SSH_KEY_FILE}:/run/secrets/workspace-registry-git-ssh-key:ro
|
|
||||||
- ${THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE:?set THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE}:/run/secrets/workspace-registry-git-known-hosts:ro
|
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
# Standalone local registry example. Copy to an untracked operator directory and set the absolute
|
|
||||||
# THT_SOURCE_ROOT in .env. Add only the selected Git transport override from this directory.
|
|
||||||
name: thothii-workspace-registry-local
|
|
||||||
|
|
||||||
services:
|
|
||||||
core:
|
|
||||||
image: thothii-core:local
|
|
||||||
build:
|
|
||||||
context: ${THT_SOURCE_ROOT:?set THT_SOURCE_ROOT to the absolute ThothII source checkout}
|
|
||||||
dockerfile: docker/core.Dockerfile
|
|
||||||
env_file:
|
|
||||||
- path: ${THT_WORKSPACE_BINDINGS_ENV_FILE:?set THT_WORKSPACE_BINDINGS_ENV_FILE to an absolute THT_WS bindings file}
|
|
||||||
required: true
|
|
||||||
environment:
|
|
||||||
HOST: 0.0.0.0
|
|
||||||
PORT: "8787"
|
|
||||||
AUTH_MODE: none
|
|
||||||
THT_SESSION_STORAGE: local
|
|
||||||
THT_HOME: /data/local-home
|
|
||||||
SETTINGS_FILE: /data/settings/settings.json
|
|
||||||
THT_HARNESS_DIR: /app/harness
|
|
||||||
THT_BIN: /opt/venv/bin/tht
|
|
||||||
THT_WORKSPACE_REGISTRY_ROOT: /data/workspace-registry
|
|
||||||
THT_WORKSPACE_GIT_REMOTE: ${THT_WORKSPACE_GIT_REMOTE:-ssh://git@git.example.invalid/platform/thoth-workspaces.git}
|
|
||||||
THT_WORKSPACE_GIT_BRANCH: ${THT_WORKSPACE_GIT_BRANCH:-main}
|
|
||||||
THT_WORKSPACE_INSTALLATION_ID: ${THT_WORKSPACE_INSTALLATION_ID:-local-laptop}
|
|
||||||
THT_WORKSPACE_GIT_AUTHOR_NAME: ${THT_WORKSPACE_GIT_AUTHOR_NAME:-Thoth Workspace Registry}
|
|
||||||
THT_WORKSPACE_GIT_AUTHOR_EMAIL: ${THT_WORKSPACE_GIT_AUTHOR_EMAIL:-thoth-workspace-registry@localhost}
|
|
||||||
THT_WORKSPACE_SECRET_ROOTS: /run/secrets
|
|
||||||
ports:
|
|
||||||
- "127.0.0.1:8787:8787"
|
|
||||||
volumes:
|
|
||||||
- thoth-local-data:/data
|
|
||||||
- workspace-registry:/data/workspace-registry
|
|
||||||
restart: "no"
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
thoth-local-data: {}
|
|
||||||
workspace-registry: {}
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
# Server registry example. Copy to a reviewed, untracked operator directory and set absolute host
|
|
||||||
# paths and Git values in .env. Add a selected Git transport override from this directory.
|
|
||||||
name: thothii-workspace-registry-server
|
|
||||||
|
|
||||||
services:
|
|
||||||
core:
|
|
||||||
image: thothii-core:local
|
|
||||||
build:
|
|
||||||
context: ${THT_SOURCE_ROOT:?set THT_SOURCE_ROOT to the absolute ThothII source checkout}
|
|
||||||
dockerfile: docker/core.Dockerfile
|
|
||||||
env_file:
|
|
||||||
- path: ${THT_WORKSPACE_BINDINGS_ENV_FILE:?set THT_WORKSPACE_BINDINGS_ENV_FILE to an absolute THT_WS bindings file}
|
|
||||||
required: true
|
|
||||||
environment:
|
|
||||||
HOST: 0.0.0.0
|
|
||||||
PORT: "8787"
|
|
||||||
AUTH_MODE: upstream
|
|
||||||
THOTH_PUBLIC_EXPOSURE: "true"
|
|
||||||
THT_SESSION_STORAGE: postgres
|
|
||||||
THT_CONFIG: /app/harness/workspaces/server-sessions.yaml
|
|
||||||
THT_SESSION_DB_HOST: ${THT_SESSION_DB_HOST:?set THT_SESSION_DB_HOST}
|
|
||||||
THT_SESSION_DB_PORT: ${THT_SESSION_DB_PORT:-5432}
|
|
||||||
THT_SESSION_DB_NAME: ${THT_SESSION_DB_NAME:?set THT_SESSION_DB_NAME}
|
|
||||||
THT_SESSION_RUNTIME_USER: ${THT_SESSION_RUNTIME_USER:?set THT_SESSION_RUNTIME_USER}
|
|
||||||
THT_SESSION_RUNTIME_PASSWORD_FILE: /run/secrets/session_runtime_password
|
|
||||||
THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full}
|
|
||||||
THT_SESSION_DB_SSLROOTCERT: /run/secrets/session_ca.pem
|
|
||||||
THT_HARNESS_DIR: /app/harness
|
|
||||||
THT_BIN: /opt/venv/bin/tht
|
|
||||||
SETTINGS_FILE: /data/settings/settings.json
|
|
||||||
THT_WORKSPACE_REGISTRY_ROOT: /data/workspace-registry
|
|
||||||
THT_WORKSPACE_GIT_REMOTE: ${THT_WORKSPACE_GIT_REMOTE:-ssh://git@git.example.invalid/platform/thoth-workspaces.git}
|
|
||||||
THT_WORKSPACE_GIT_BRANCH: ${THT_WORKSPACE_GIT_BRANCH:-main}
|
|
||||||
THT_WORKSPACE_INSTALLATION_ID: ${THT_WORKSPACE_INSTALLATION_ID:-production-1}
|
|
||||||
THT_WORKSPACE_GIT_AUTHOR_NAME: ${THT_WORKSPACE_GIT_AUTHOR_NAME:-Thoth Workspace Registry}
|
|
||||||
THT_WORKSPACE_GIT_AUTHOR_EMAIL: ${THT_WORKSPACE_GIT_AUTHOR_EMAIL:-thoth-workspace-registry@localhost}
|
|
||||||
THT_WORKSPACE_SECRET_ROOTS: /run/secrets
|
|
||||||
volumes:
|
|
||||||
- ${THT_HOST_DATA_ROOT:-/srv/thothii/data}:/data
|
|
||||||
- ${THT_WORKSPACE_REGISTRY_HOST_PATH:-/srv/thothii/workspace-registry}:/data/workspace-registry
|
|
||||||
- ${THT_SERVER_WORKSPACE_CONFIG:?set THT_SERVER_WORKSPACE_CONFIG}:/app/harness/workspaces/server-sessions.yaml:ro
|
|
||||||
secrets:
|
|
||||||
- source: session_runtime_password
|
|
||||||
target: session_runtime_password
|
|
||||||
- source: session_ca
|
|
||||||
target: session_ca.pem
|
|
||||||
networks:
|
|
||||||
- upstream
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
networks:
|
|
||||||
upstream:
|
|
||||||
external: true
|
|
||||||
name: ${THT_UPSTREAM_NETWORK:-thothii-upstream}
|
|
||||||
|
|
||||||
secrets:
|
|
||||||
session_runtime_password:
|
|
||||||
file: ${THT_SESSION_RUNTIME_PASSWORD_SOURCE:?set THT_SESSION_RUNTIME_PASSWORD_SOURCE}
|
|
||||||
session_ca:
|
|
||||||
file: ${THT_SESSION_CA_SOURCE:?set THT_SESSION_CA_SOURCE}
|
|
||||||
@@ -1,13 +1,13 @@
|
|||||||
# Copy to an untracked operator file. This file contains only non-secret THT_WS_* bindings.
|
# Copy to an untracked operator file. This file contains only non-secret THT_WS_* bindings.
|
||||||
# Every *_FILE value is a container path supplied by the generated local connector override.
|
# Every *_FILE value is a container path supplied by the generated local connector override.
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=postgres_direct
|
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=postgres_direct
|
||||||
THT_WS_PSD_CLINICAL_DWH_HOST=dwh.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_DWH_HOST=dwh.internal.example
|
||||||
THT_WS_PSD_CLINICAL_DWH_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PORT=5432
|
||||||
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
|
||||||
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-clinical-dwh-password
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=pgvector_direct
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_TRANSPORT=pgvector_direct
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_HOST=vector.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_HOST=vector.internal.example
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_PORT=5432
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_USER=thoth_vector_reader
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_USER=thoth_vector_reader
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_PASSWORD_FILE=/run/secrets/psd-clinical-vector-password
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_PASSWORD_FILE=/run/secrets/north-star-research-vector-password
|
||||||
THT_WS_PSD_CLINICAL_EMBEDDING_BASE_URL=https://embeddings.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_EMBEDDING_BASE_URL=https://embeddings.internal.example
|
||||||
|
|||||||
@@ -34,14 +34,16 @@ workspaces/<workspace-id>.md
|
|||||||
For SSH, use a scoped deploy key, a verified `known_hosts` file, and strict host-key checking. For
|
For SSH, use a scoped deploy key, a verified `known_hosts` file, and strict host-key checking. For
|
||||||
HTTPS, use Git Credential Manager or a secret-manager-created credentials file. Mount a private
|
HTTPS, use Git Credential Manager or a secret-manager-created credentials file. Mount a private
|
||||||
HTTPS CA as its own file. Do not disable host or certificate verification. The base Compose file
|
HTTPS CA as its own file. Do not disable host or certificate verification. The base Compose file
|
||||||
does not mount a Git credential: add exactly one optional `git-ssh.workspace-registry.yaml` or
|
does not mount a Git credential: add exactly one optional `deploy/compose.git-ssh.yaml` or
|
||||||
`git-https.workspace-registry.yaml` override, so unused credential paths are never bind-mounted.
|
`deploy/compose.git-https.yaml` override, so unused credential paths are never bind-mounted.
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
THT_WORKSPACE_GIT_REMOTE=ssh://git@git.example.invalid/platform/thoth-workspaces.git
|
THT_WORKSPACE_GIT_REMOTE=ssh://git@git.example.invalid/platform/thoth-workspaces.git
|
||||||
THT_WORKSPACE_GIT_BRANCH=main
|
THT_WORKSPACE_GIT_BRANCH=main
|
||||||
THT_WORKSPACE_INSTALLATION_ID=local-laptop
|
THT_WORKSPACE_INSTALLATION_ID=local-laptop
|
||||||
THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
||||||
|
PI_AUTH_FILE=/absolute/path/installation-secrets/pi-auth.json
|
||||||
|
THT_SECRETS_FILE=/absolute/path/installation-secrets/thothii.secrets
|
||||||
THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/installation-secrets/git-ssh-key
|
THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/installation-secrets/git-ssh-key
|
||||||
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/installation-secrets/git-known-hosts
|
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/installation-secrets/git-known-hosts
|
||||||
THT_WORKSPACE_GIT_CA_FILE=/absolute/path/installation-secrets/git-ca.pem
|
THT_WORKSPACE_GIT_CA_FILE=/absolute/path/installation-secrets/git-ca.pem
|
||||||
@@ -69,12 +71,12 @@ state/ # active revision and registry state
|
|||||||
locks/ # short-lived publish locks
|
locks/ # short-lived publish locks
|
||||||
```
|
```
|
||||||
|
|
||||||
Installation variables are deterministic: `psd-clinical` becomes `PSD_CLINICAL`, and every name
|
Installation variables are deterministic: `north-star-research` becomes `NORTH_STAR_RESEARCH`, and every name
|
||||||
is `THT_WS_<NAMESPACE>_<ROLE>_<SUFFIX>`. Copy
|
is `THT_WS_<NAMESPACE>_<ROLE>_<SUFFIX>`. Copy
|
||||||
[the bindings env example](examples/workspace-bindings.env.example) to an untracked operator file
|
[the bindings env example](examples/workspace-bindings.env.example) to an untracked operator file
|
||||||
and set its absolute path as `THT_WORKSPACE_BINDINGS_ENV_FILE`. It is loaded only into `core`.
|
and set its absolute path as `THT_WORKSPACE_BINDINGS_ENV_FILE`. It is loaded only into `core`.
|
||||||
Credentials and certificates use `*_FILE` path variables that must point inside `/run/secrets`.
|
Credentials and certificates use `*_FILE` path variables that must point inside `/run/secrets`.
|
||||||
If declared, `THT_WS_PSD_CLINICAL_VECTOR_WRITER_API_KEY_FILE` is distinct from the vector reader
|
If declared, `THT_WS_NORTH_STAR_RESEARCH_VECTOR_WRITER_API_KEY_FILE` is distinct from the vector reader
|
||||||
file; a reader credential is never repurposed for writing.
|
file; a reader credential is never repurposed for writing.
|
||||||
|
|
||||||
## Direct PostgreSQL, REST, and SSH tunnel bindings
|
## Direct PostgreSQL, REST, and SSH tunnel bindings
|
||||||
@@ -87,41 +89,41 @@ copy or maintain a workspace-specific Compose override.
|
|||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
# Direct PostgreSQL and pgvector
|
# Direct PostgreSQL and pgvector
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=postgres_direct
|
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=postgres_direct
|
||||||
THT_WS_PSD_CLINICAL_DWH_HOST=dwh.example.invalid
|
THT_WS_NORTH_STAR_RESEARCH_DWH_HOST=dwh.example.invalid
|
||||||
THT_WS_PSD_CLINICAL_DWH_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PORT=5432
|
||||||
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
|
||||||
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=pgvector_direct
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_TRANSPORT=pgvector_direct
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_HOST=vector.example.invalid
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_HOST=vector.example.invalid
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_PORT=5432
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_USER=thoth_vector_reader
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_USER=thoth_vector_reader
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_PASSWORD_FILE=/run/secrets/psd-vector-reader
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_PASSWORD_FILE=/run/secrets/north-star-research-vector-password
|
||||||
THT_WS_PSD_CLINICAL_EMBEDDING_BASE_URL=https://embeddings.example.invalid
|
THT_WS_NORTH_STAR_RESEARCH_EMBEDDING_BASE_URL=https://embeddings.example.invalid
|
||||||
```
|
```
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
# REST; an API-key file is needed only for a declared bearer/x-api-key diagnostic.
|
# REST; an API-key file is needed only for a declared bearer/x-api-key diagnostic.
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api
|
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=rest_api
|
||||||
THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://dwh.example.invalid
|
THT_WS_NORTH_STAR_RESEARCH_DWH_BASE_URL=https://dwh.example.invalid
|
||||||
THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-dwh-api-key
|
THT_WS_NORTH_STAR_RESEARCH_DWH_API_KEY_FILE=/run/secrets/north-star-research-dwh-api-key
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=rest_api
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_TRANSPORT=rest_api
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_BASE_URL=https://vectors.example.invalid
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_BASE_URL=https://vectors.example.invalid
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_API_KEY_FILE=/run/secrets/psd-vector-api-key
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE=/run/secrets/north-star-research-vector-api-key
|
||||||
```
|
```
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
# SSH tunnel diagnostic only; runtime sessions are fail-closed in this release.
|
# SSH tunnel diagnostic only; runtime sessions are fail-closed in this release.
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=ssh_tunnel
|
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=ssh_tunnel
|
||||||
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
|
||||||
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_HOST=bastion.example.invalid
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_HOST=bastion.example.invalid
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_PORT=22
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_PORT=22
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_USER=thoth_tunnel
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_USER=thoth_tunnel
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/psd-dwh-tunnel-key
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/north-star-research-dwh-tunnel-key
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/psd-dwh-known-hosts
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/north-star-research-dwh-known-hosts
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_HOST=dwh.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_TARGET_HOST=dwh.internal.example
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_TARGET_PORT=5432
|
||||||
```
|
```
|
||||||
|
|
||||||
Repeat the SSH names for `VECTOR` where needed. REST diagnostics reject a private per-request CA
|
Repeat the SSH names for `VECTOR` where needed. REST diagnostics reject a private per-request CA
|
||||||
@@ -134,31 +136,36 @@ before creating sessions. Git pull/push over SSH remains fully supported and is
|
|||||||
|
|
||||||
## Bootstrap, first pull, and diagnostics
|
## Bootstrap, first pull, and diagnostics
|
||||||
|
|
||||||
Copy [the local Compose example](examples/local-compose.workspace-registry.yaml), exactly one
|
Use the repository's canonical `compose.yaml` plus `deploy/compose.local.yaml`; they always start
|
||||||
selected [SSH Git override](examples/git-ssh.workspace-registry.yaml) or [HTTPS Git override](examples/git-https.workspace-registry.yaml),
|
the mandatory `frontend` and `core` services. Do not copy or maintain a standalone application
|
||||||
and [the bindings env example](examples/workspace-bindings.env.example) into an untracked operator
|
Compose file. Copy [the bindings env example](examples/workspace-bindings.env.example) into an
|
||||||
directory. Keep `THT_SOURCE_ROOT` and the absolute `THT_WORKSPACE_BINDINGS_ENV_FILE` in its `.env`
|
untracked operator directory and create a protected operator env file from
|
||||||
for Compose interpolation; this keeps the copied Compose file buildable and confines `THT_WS_*`
|
`deploy/env/local.env.example`. It must contain absolute `PI_AUTH_FILE`,
|
||||||
values to `core`. A Compose `.env` file is not a shell environment, so do not import it into the
|
`THT_SECRETS_FILE`, `THT_WORKSPACE_BINDINGS_ENV_FILE`, and connector `*_SOURCE` paths.
|
||||||
maintenance shell. Instead, explicitly export the two non-secret paths before running the commands.
|
The Pi auth JSON, runtime secret bundle, and each connector credential remain separate protected
|
||||||
Create the host secret files named by the selected Git transport and every declared connector
|
host files and are mounted read-only; their contents never enter the operator env or rendered
|
||||||
`*_SOURCE`, then generate the connector override and render through the preflight wrapper. The
|
Compose.
|
||||||
wrapper is required: it rejects unsafe source paths and a combined SSH+HTTPS Git selection before
|
|
||||||
Compose runs.
|
Select exactly one repository Git transport override, `deploy/compose.git-ssh.yaml` or
|
||||||
|
`deploy/compose.git-https.yaml`. A Compose env file is not a shell environment, so export only the
|
||||||
|
non-secret paths required by the maintenance commands. Generate the connector override and render
|
||||||
|
through the preflight wrapper, which rejects unsafe paths and combined SSH+HTTPS selection.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
||||||
export THT_WORKSPACE_BINDINGS_ENV_FILE="$(pwd -P)/workspace-bindings.env"
|
export THT_OPERATOR_ENV=/absolute/path/to/operator/local.env
|
||||||
"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" --bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env .env --output connector-secrets.local.yaml
|
export THT_WORKSPACE_BINDINGS_ENV_FILE=/absolute/path/to/operator/workspace-bindings.env
|
||||||
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
|
export THT_CONNECTOR_OVERRIDE=/absolute/path/to/operator/connector-secrets.local.yaml
|
||||||
-f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml -f connector-secrets.local.yaml config --quiet
|
"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" --bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env "$THT_OPERATOR_ENV" --output "$THT_CONNECTOR_OVERRIDE"
|
||||||
|
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file "$THT_OPERATOR_ENV" \
|
||||||
|
-f "$THT_SOURCE_ROOT/compose.yaml" -f "$THT_SOURCE_ROOT/deploy/compose.local.yaml" \
|
||||||
|
-f "$THT_SOURCE_ROOT/deploy/compose.git-ssh.yaml" -f "$THT_CONNECTOR_OVERRIDE" config --quiet
|
||||||
```
|
```
|
||||||
|
|
||||||
From the operator directory:
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
|
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file "$THT_OPERATOR_ENV" \
|
||||||
-f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml -f connector-secrets.local.yaml up --build -d
|
-f "$THT_SOURCE_ROOT/compose.yaml" -f "$THT_SOURCE_ROOT/deploy/compose.local.yaml" \
|
||||||
|
-f "$THT_SOURCE_ROOT/deploy/compose.git-ssh.yaml" -f "$THT_CONNECTOR_OVERRIDE" up --build -d
|
||||||
curl --fail --silent http://127.0.0.1:8787/health
|
curl --fail --silent http://127.0.0.1:8787/health
|
||||||
curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
|
curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
|
||||||
curl --fail --silent http://127.0.0.1:8787/workspaces
|
curl --fail --silent http://127.0.0.1:8787/workspaces
|
||||||
@@ -169,7 +176,7 @@ Use `POST /workspace-registry/pull` to fetch later revisions. Run workspace diag
|
|||||||
required bindings are mounted. The optional writer probe uses a distinct writer file and removes
|
required bindings are mounted. The optional writer probe uses a distinct writer file and removes
|
||||||
its uniquely named temporary record; ordinary diagnostics are read-only.
|
its uniquely named temporary record; ordinary diagnostics are read-only.
|
||||||
|
|
||||||
To migrate an existing PSD descriptor, create/clone an empty private remote, set the absolute
|
To migrate an existing legacy descriptor, create/clone an empty private remote, set the absolute
|
||||||
`THT_SOURCE_ROOT`, transform with absolute paths, review the schema-v1 result, explicitly add
|
`THT_SOURCE_ROOT`, transform with absolute paths, review the schema-v1 result, explicitly add
|
||||||
vector database/schema and the complete schema-v2 contract, then commit/push. The transformer
|
vector database/schema and the complete schema-v2 contract, then commit/push. The transformer
|
||||||
never imports `${ENV}` values or secrets.
|
never imports `${ENV}` values or secrets.
|
||||||
@@ -177,7 +184,7 @@ never imports `${ENV}` values or secrets.
|
|||||||
```sh
|
```sh
|
||||||
THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
||||||
npm --prefix "$THT_SOURCE_ROOT/backend" run build
|
npm --prefix "$THT_SOURCE_ROOT/backend" run build
|
||||||
node "$THT_SOURCE_ROOT/backend/dist/workspaces/migrate-legacy.js" --input /absolute/path/psd.yaml --output /absolute/path/thoth-workspaces
|
node "$THT_SOURCE_ROOT/backend/dist/workspaces/migrate-legacy.js" --input /absolute/path/legacy.yaml --output /absolute/path/thoth-workspaces
|
||||||
```
|
```
|
||||||
|
|
||||||
## Publish, update, backup, outage recovery, and rollback
|
## Publish, update, backup, outage recovery, and rollback
|
||||||
|
|||||||
@@ -48,11 +48,13 @@ THT_WORKSPACE_GIT_CREDENTIALS_FILE=/srv/thothii/secrets/git-credentials
|
|||||||
THT_WORKSPACE_GIT_CA_FILE=/srv/thothii/secrets/git-ca.pem
|
THT_WORKSPACE_GIT_CA_FILE=/srv/thothii/secrets/git-ca.pem
|
||||||
THT_WORKSPACE_GIT_SSH_KEY_FILE=/srv/thothii/secrets/git-ssh-key
|
THT_WORKSPACE_GIT_SSH_KEY_FILE=/srv/thothii/secrets/git-ssh-key
|
||||||
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/srv/thothii/secrets/git-known-hosts
|
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/srv/thothii/secrets/git-known-hosts
|
||||||
|
PI_AUTH_FILE=/srv/thothii/secrets/pi-auth.json
|
||||||
|
THT_SECRETS_FILE=/srv/thothii/secrets/thothii.secrets
|
||||||
```
|
```
|
||||||
|
|
||||||
Use the credential file for HTTPS, or key and known-hosts for SSH. The base server Compose file
|
Use the credential file for HTTPS, or key and known-hosts for SSH. The base server Compose file
|
||||||
mounts neither transport; add exactly one [HTTPS override](examples/git-https.workspace-registry.yaml)
|
mounts neither transport; add exactly one `deploy/compose.git-https.yaml`
|
||||||
or [SSH override](examples/git-ssh.workspace-registry.yaml). Strict host-key checking stays enabled
|
or `deploy/compose.git-ssh.yaml` override. Strict host-key checking stays enabled
|
||||||
and Git stderr is not exposed by the API. Rotate by atomically replacing the secret file,
|
and Git stderr is not exposed by the API. Rotate by atomically replacing the secret file,
|
||||||
restarting `core`, and performing pull/status; never put the material in an environment variable or
|
restarting `core`, and performing pull/status; never put the material in an environment variable or
|
||||||
rendered Compose output.
|
rendered Compose output.
|
||||||
@@ -75,9 +77,9 @@ The runtime registry layout is persistent and must be backed up together:
|
|||||||
/data/workspace-registry/locks/
|
/data/workspace-registry/locks/
|
||||||
```
|
```
|
||||||
|
|
||||||
Variable names derive from the immutable ID: `psd-clinical` becomes `PSD_CLINICAL`, producing
|
Variable names derive from the immutable ID: `north-star-research` becomes `NORTH_STAR_RESEARCH`, producing
|
||||||
`THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE`. A declared vector writer uses the distinct
|
`THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE`. A declared vector writer uses the distinct
|
||||||
`THT_WS_PSD_CLINICAL_VECTOR_WRITER_API_KEY_FILE`; a reader file is never a writer substitute.
|
`THT_WS_NORTH_STAR_RESEARCH_VECTOR_WRITER_API_KEY_FILE`; a reader file is never a writer substitute.
|
||||||
Copy [the bindings env example](examples/workspace-bindings.env.example) to the protected operator
|
Copy [the bindings env example](examples/workspace-bindings.env.example) to the protected operator
|
||||||
directory. Every path-valued `*_FILE` entry needs an absolute host-only `*_SOURCE` path. Generate
|
directory. Every path-valued `*_FILE` entry needs an absolute host-only `*_SOURCE` path. Generate
|
||||||
the untracked connector override from those files during bootstrap; do not copy or maintain a
|
the untracked connector override from those files during bootstrap; do not copy or maintain a
|
||||||
@@ -90,41 +92,41 @@ dimensions, and distance as Git-shared identity.
|
|||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
# Direct PostgreSQL/pgvector with verified native TLS if a CA path is supplied.
|
# Direct PostgreSQL/pgvector with verified native TLS if a CA path is supplied.
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=postgres_direct
|
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=postgres_direct
|
||||||
THT_WS_PSD_CLINICAL_DWH_HOST=dwh.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_DWH_HOST=dwh.internal.example
|
||||||
THT_WS_PSD_CLINICAL_DWH_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PORT=5432
|
||||||
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
|
||||||
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=pgvector_direct
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_TRANSPORT=pgvector_direct
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_HOST=vector.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_HOST=vector.internal.example
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_PORT=5432
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_USER=thoth_vector_reader
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_USER=thoth_vector_reader
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_PASSWORD_FILE=/run/secrets/psd-vector-reader
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_PASSWORD_FILE=/run/secrets/north-star-research-vector-password
|
||||||
```
|
```
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
# REST needs API-key file paths only when the descriptor declares authenticated diagnostics.
|
# REST needs API-key file paths only when the descriptor declares authenticated diagnostics.
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api
|
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=rest_api
|
||||||
THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://dwh.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_DWH_BASE_URL=https://dwh.internal.example
|
||||||
THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-dwh-api-key
|
THT_WS_NORTH_STAR_RESEARCH_DWH_API_KEY_FILE=/run/secrets/north-star-research-dwh-api-key
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=rest_api
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_TRANSPORT=rest_api
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_BASE_URL=https://vectors.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_BASE_URL=https://vectors.internal.example
|
||||||
THT_WS_PSD_CLINICAL_VECTOR_API_KEY_FILE=/run/secrets/psd-vector-api-key
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE=/run/secrets/north-star-research-vector-api-key
|
||||||
THT_WS_PSD_CLINICAL_EMBEDDING_BASE_URL=https://embeddings.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_EMBEDDING_BASE_URL=https://embeddings.internal.example
|
||||||
```
|
```
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
# SSH tunnel diagnostic only; runtime sessions are fail-closed in this release.
|
# SSH tunnel diagnostic only; runtime sessions are fail-closed in this release.
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=ssh_tunnel
|
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=ssh_tunnel
|
||||||
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
|
||||||
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_HOST=bastion.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_HOST=bastion.internal.example
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_PORT=22
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_PORT=22
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_USER=thoth_tunnel
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_USER=thoth_tunnel
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/psd-dwh-tunnel-key
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/north-star-research-dwh-tunnel-key
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/psd-dwh-known-hosts
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/north-star-research-dwh-known-hosts
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_HOST=dwh.internal.example
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_TARGET_HOST=dwh.internal.example
|
||||||
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_PORT=5432
|
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_TARGET_PORT=5432
|
||||||
```
|
```
|
||||||
|
|
||||||
Repeat SSH variables for `VECTOR` when selected. REST diagnostics refuse private per-request CAs
|
Repeat SSH variables for `VECTOR` when selected. REST diagnostics refuse private per-request CAs
|
||||||
@@ -138,15 +140,17 @@ The Git registry itself may still use SSH normally.
|
|||||||
|
|
||||||
## Same-origin reverse proxy, bootstrap, and health
|
## Same-origin reverse proxy, bootstrap, and health
|
||||||
|
|
||||||
Copy [the server Compose example](examples/server-compose.workspace-registry.yaml) plus exactly one
|
Use the repository's canonical `compose.yaml` plus `deploy/compose.server.yaml`; they always
|
||||||
selected Git override to the protected operator directory. Set `THT_SOURCE_ROOT` to the absolute
|
start the mandatory `frontend` and `core` services. Do not copy or maintain a standalone
|
||||||
ThothII checkout; a copied file cannot use a relative build context. Copy
|
application Compose file. Review `deploy/workspaces/server-sessions.yaml.example`, materialize it
|
||||||
`deploy/workspaces/server-sessions.yaml.example` into that operator directory, review it, then set
|
as a protected host file, and set its absolute `THT_SERVER_WORKSPACE_CONFIG` path. Copy the
|
||||||
the absolute `THT_SERVER_WORKSPACE_CONFIG` path. Copy the bindings env example, then set absolute
|
bindings env example into the operator directory, then set absolute `PI_AUTH_FILE`,
|
||||||
`THT_WORKSPACE_BINDINGS_ENV_FILE` and connector `*_SOURCE` paths. The same `.env` must set
|
`THT_SECRETS_FILE`, `THT_WORKSPACE_BINDINGS_ENV_FILE`, and connector `*_SOURCE` paths.
|
||||||
|
The same operator env must set
|
||||||
`THT_SESSION_DB_HOST`, `THT_SESSION_DB_NAME`, `THT_SESSION_RUNTIME_USER`,
|
`THT_SESSION_DB_HOST`, `THT_SESSION_DB_NAME`, `THT_SESSION_RUNTIME_USER`,
|
||||||
`THT_SESSION_RUNTIME_PASSWORD_SOURCE`, and `THT_SESSION_CA_SOURCE`; the base Compose file wires
|
`THT_SESSION_RUNTIME_PASSWORD_SOURCE`, and `THT_SESSION_CA_SOURCE`;
|
||||||
`postgres`, `verify-full`, and the two Docker secret mount paths. This is the public server profile,
|
`deploy/compose.session-server.yaml.example` wires `postgres`, `verify-full`, and separate
|
||||||
|
runtime/CA Docker secret mount paths. This is the public server profile,
|
||||||
not a filesystem-session fallback. A Compose `.env` file is not a shell environment, so do not
|
not a filesystem-session fallback. A Compose `.env` file is not a shell environment, so do not
|
||||||
import it into the maintenance shell. Explicitly export the non-secret source and bindings paths
|
import it into the maintenance shell. Explicitly export the non-secret source and bindings paths
|
||||||
before running the commands below.
|
before running the commands below.
|
||||||
@@ -161,14 +165,24 @@ From a trusted maintenance shell:
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
|
||||||
export THT_WORKSPACE_BINDINGS_ENV_FILE="$(pwd -P)/workspace-bindings.env"
|
export THT_OPERATOR_ENV=/srv/thothii/operator/server.env
|
||||||
"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" --bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env .env --output connector-secrets.local.yaml
|
export THT_WORKSPACE_BINDINGS_ENV_FILE=/srv/thothii/operator/workspace-bindings.env
|
||||||
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
|
export THT_CONNECTOR_OVERRIDE=/srv/thothii/operator/connector-secrets.local.yaml
|
||||||
-f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml -f connector-secrets.local.yaml up --build -d
|
"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" --bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env "$THT_OPERATOR_ENV" --output "$THT_CONNECTOR_OVERRIDE"
|
||||||
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
|
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file "$THT_OPERATOR_ENV" \
|
||||||
-f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml -f connector-secrets.local.yaml exec -T core curl --fail --silent http://127.0.0.1:8787/health
|
-f "$THT_SOURCE_ROOT/compose.yaml" -f "$THT_SOURCE_ROOT/deploy/compose.server.yaml" \
|
||||||
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
|
-f "$THT_SOURCE_ROOT/deploy/compose.session-server.yaml.example" \
|
||||||
-f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml -f connector-secrets.local.yaml exec -T core curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
|
-f "$THT_SOURCE_ROOT/deploy/compose.git-ssh.yaml" -f "$THT_CONNECTOR_OVERRIDE" up --build -d
|
||||||
|
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file "$THT_OPERATOR_ENV" \
|
||||||
|
-f "$THT_SOURCE_ROOT/compose.yaml" -f "$THT_SOURCE_ROOT/deploy/compose.server.yaml" \
|
||||||
|
-f "$THT_SOURCE_ROOT/deploy/compose.session-server.yaml.example" \
|
||||||
|
-f "$THT_SOURCE_ROOT/deploy/compose.git-ssh.yaml" -f "$THT_CONNECTOR_OVERRIDE" \
|
||||||
|
exec -T core curl --fail --silent http://127.0.0.1:8787/health
|
||||||
|
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file "$THT_OPERATOR_ENV" \
|
||||||
|
-f "$THT_SOURCE_ROOT/compose.yaml" -f "$THT_SOURCE_ROOT/deploy/compose.server.yaml" \
|
||||||
|
-f "$THT_SOURCE_ROOT/deploy/compose.session-server.yaml.example" \
|
||||||
|
-f "$THT_SOURCE_ROOT/deploy/compose.git-ssh.yaml" -f "$THT_CONNECTOR_OVERRIDE" \
|
||||||
|
exec -T core curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
|
||||||
```
|
```
|
||||||
|
|
||||||
`/health` is liveness. Registry status verifies branch/head/degraded state and the active validated
|
`/health` is liveness. Registry status verifies branch/head/degraded state and the active validated
|
||||||
@@ -187,7 +201,7 @@ filesystem-consistent backup of `/srv/thothii/workspace-registry` plus `/srv/tho
|
|||||||
`/srv/thothii/secrets`. Render Compose, deploy the compatible image, verify health/status, then
|
`/srv/thothii/secrets`. Render Compose, deploy the compatible image, verify health/status, then
|
||||||
resume proxy traffic.
|
resume proxy traffic.
|
||||||
|
|
||||||
For PSD migration, use a temporary review clone and the legacy transformer with absolute paths.
|
For legacy descriptor migration, use a temporary review clone and the legacy transformer with absolute paths.
|
||||||
Its schema-v1 output is `migration_required`; explicitly supply vector database/schema, collection
|
Its schema-v1 output is `migration_required`; explicitly supply vector database/schema, collection
|
||||||
identity, diagnostics, and the reviewed v2 contract before commit. Never import `${ENV}` values or
|
identity, diagnostics, and the reviewed v2 contract before commit. Never import `${ENV}` values or
|
||||||
copy secret files.
|
copy secret files.
|
||||||
|
|||||||
@@ -15,6 +15,8 @@ Servono Docker Engine/Compose v2 su Linux oppure Docker Desktop su macOS/Windows
|
|||||||
git clone <URL-REPOSITORY> ThothII
|
git clone <URL-REPOSITORY> ThothII
|
||||||
cd ThothII
|
cd ThothII
|
||||||
cp deploy/env/local.env.example deploy/env/local.env
|
cp deploy/env/local.env.example deploy/env/local.env
|
||||||
|
cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets
|
||||||
|
chmod 600 deploy/secrets/thothii.secrets
|
||||||
```
|
```
|
||||||
|
|
||||||
Modificare **solo** questi file interni al clone:
|
Modificare **solo** questi file interni al clone:
|
||||||
@@ -25,7 +27,8 @@ Modificare **solo** questi file interni al clone:
|
|||||||
| file protetti locali | credenziali e certificati, indicati dai binding del workspace |
|
| file protetti locali | credenziali e certificati, indicati dai binding del workspace |
|
||||||
| `deploy/workspaces/<nome>.yaml` | adapter, endpoint non riservati, `roots` ed Evidence |
|
| `deploy/workspaces/<nome>.yaml` | adapter, endpoint non riservati, `roots` ed Evidence |
|
||||||
|
|
||||||
Compilare `deploy/env/local.env`, incluso `PI_AUTH_FILE`, con gli endpoint esterni. L'avvio
|
Compilare `deploy/env/local.env`, inclusi i path assoluti `PI_AUTH_FILE` e
|
||||||
|
`THT_SECRETS_FILE`, con gli endpoint esterni. L'avvio
|
||||||
normale usa esplicitamente il file base e l'overlay locale:
|
normale usa esplicitamente il file base e l'overlay locale:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
@@ -52,7 +55,7 @@ THT_VECTOR_READER_PASSWORD=...
|
|||||||
THT_VECTOR_WRITER_PASSWORD=...
|
THT_VECTOR_WRITER_PASSWORD=...
|
||||||
```
|
```
|
||||||
|
|
||||||
Inserire solo le chiavi necessarie al profilo scelto. Il bundle viene montato in sola lettura nel container come `/run/secrets/thothii.secrets`; il parser rifiuta duplicati, chiavi sconosciute, valori vuoti, symlink e permessi host troppo aperti. Non inserire secret in `.env`, nei workspace, negli URL o nell'output di `docker compose config`.
|
Inserire solo le chiavi necessarie al profilo scelto. Il bundle viene montato in sola lettura nel container come `/run/secrets/thothii.secrets`; il parser rifiuta duplicati, chiavi sconosciute, valori vuoti, symlink e permessi host troppo aperti. Non inserire secret in `.env`, nei workspace, negli URL o nell'output Compose renderizzato.
|
||||||
|
|
||||||
Una catena CA PEM **non può essere inserita nel bundle**: contiene whitespace e viene rifiutata dal parser. Se un endpoint usa una CA privata, conservarla nel secret manager/host e aggiungere un override Compose revisionato che monti il file in `/run/secrets/ca-chain.pem` e imposti `THT_SSL_CA` (o il parametro dell'adapter). Il clone base non crea quel mount: questa è una limitazione intenzionale da considerare in fase di deployment.
|
Una catena CA PEM **non può essere inserita nel bundle**: contiene whitespace e viene rifiutata dal parser. Se un endpoint usa una CA privata, conservarla nel secret manager/host e aggiungere un override Compose revisionato che monti il file in `/run/secrets/ca-chain.pem` e imposti `THT_SSL_CA` (o il parametro dell'adapter). Il clone base non crea quel mount: questa è una limitazione intenzionale da considerare in fase di deployment.
|
||||||
|
|
||||||
@@ -62,7 +65,8 @@ DWH/vector/embedding remoti restano endpoint del file locale o server. Per il so
|
|||||||
sviluppo pgvector, aggiungere `-f deploy/compose.local-vector.yaml --profile local-vector` al
|
sviluppo pgvector, aggiungere `-f deploy/compose.local-vector.yaml --profile local-vector` al
|
||||||
comando base. Per il preprocessing aggiungere anche
|
comando base. Per il preprocessing aggiungere anche
|
||||||
`-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml --profile preprocess`,
|
`-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml --profile preprocess`,
|
||||||
poi usare `docker compose run --rm preprocess-evidence` oppure `preprocess-dwh` con gli stessi argomenti.
|
poi ripetere l'intero comando base con l'azione `run --rm preprocess-evidence` oppure
|
||||||
|
`run --rm preprocess-dwh`.
|
||||||
|
|
||||||
## Workspace, adapter e Evidence
|
## Workspace, adapter e Evidence
|
||||||
|
|
||||||
@@ -124,8 +128,10 @@ THOTH_PUBLIC_EXPOSURE=false
|
|||||||
Riempire nel bundle le chiavi DWH/vector/model necessarie e avviare:
|
Riempire nel bundle le chiavi DWH/vector/model necessarie e avviare:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker compose up --build -d
|
docker compose --env-file deploy/env/local.env \
|
||||||
docker compose exec core /opt/venv/bin/tht doctor --json
|
-f compose.yaml -f deploy/compose.local.yaml up --build -d
|
||||||
|
docker compose --env-file deploy/env/local.env \
|
||||||
|
-f compose.yaml -f deploy/compose.local.yaml exec core /opt/venv/bin/tht doctor --json
|
||||||
```
|
```
|
||||||
|
|
||||||
Se si abilita l'overlay production, il proxy autenticato TLS deve essere l'unico listener pubblico
|
Se si abilita l'overlay production, il proxy autenticato TLS deve essere l'unico listener pubblico
|
||||||
@@ -159,7 +165,10 @@ THT_VECTOR_READER_PASSWORD=<valore casuale>
|
|||||||
THT_VECTOR_WRITER_PASSWORD=<valore casuale>
|
THT_VECTOR_WRITER_PASSWORD=<valore casuale>
|
||||||
```
|
```
|
||||||
|
|
||||||
Poi eseguire il comando standard `docker compose up --build -d`. Il primo avvio esegue reconciliation dei ruoli e migrazione pgvector. Per preprocessing, impostare il preset indicato sopra e usare `docker compose run --rm preprocess-evidence`/`preprocess-dwh`.
|
Poi eseguire il comando standard base+locale mostrato sopra. Il primo avvio esegue
|
||||||
|
reconciliation dei ruoli e migrazione pgvector. Per preprocessing, impostare il preset indicato
|
||||||
|
sopra e usare l'azione `run --rm preprocess-evidence` o `run --rm preprocess-dwh` con tutti
|
||||||
|
gli stessi file e profili.
|
||||||
|
|
||||||
## 3. PC Windows locale
|
## 3. PC Windows locale
|
||||||
|
|
||||||
@@ -175,8 +184,8 @@ THT_DOCS_ROOT=/data/source/evidence
|
|||||||
Creare `deploy/secrets/thothii.secrets` con un editor locale protetto (ACL leggibile solo dall'utente Docker) e le stesse quattro chiavi pgvector del profilo Mac. Non usare `ConvertFrom-SecureString`: il bundle deve contenere il valore in chiaro per il servizio, con accesso limitato al file. Da PowerShell, dalla radice del clone, eseguire:
|
Creare `deploy/secrets/thothii.secrets` con un editor locale protetto (ACL leggibile solo dall'utente Docker) e le stesse quattro chiavi pgvector del profilo Mac. Non usare `ConvertFrom-SecureString`: il bundle deve contenere il valore in chiaro per il servizio, con accesso limitato al file. Da PowerShell, dalla radice del clone, eseguire:
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
docker compose up --build -d
|
docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local.yaml up --build -d
|
||||||
docker compose ps
|
docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local.yaml ps
|
||||||
```
|
```
|
||||||
|
|
||||||
Se un bind mount viene rifiutato, aggiungere la cartella del repository a Docker Desktop → Settings → Resources → File Sharing. Per Ollama eseguito in WSL2 usare l'indirizzo raggiungibile dalla rete Docker invece di assumere `localhost`.
|
Se un bind mount viene rifiutato, aggiungere la cartella del repository a Docker Desktop → Settings → Resources → File Sharing. Per Ollama eseguito in WSL2 usare l'indirizzo raggiungibile dalla rete Docker invece di assumere `localhost`.
|
||||||
@@ -187,13 +196,25 @@ Usare il profilo server e consentire dal firewall solo le destinazioni necessari
|
|||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
# Avvio: docker compose --env-file deploy/env/server.env \
|
# Avvio: docker compose --env-file deploy/env/server.env \
|
||||||
# -f compose.yaml -f deploy/compose.server.yaml up --build -d
|
# -f compose.yaml -f deploy/compose.server.yaml \
|
||||||
|
# -f deploy/compose.session-server.yaml.example up --build -d
|
||||||
THT_DB_NAME=warehouse
|
THT_DB_NAME=warehouse
|
||||||
THT_DWH_REST_URL=https://dwh.example.test
|
THT_DWH_REST_URL=https://dwh.example.test
|
||||||
THT_VEC_REST_URL=https://vectors.example.test
|
THT_VEC_REST_URL=https://vectors.example.test
|
||||||
THT_OLLAMA_URL=https://embeddings.example.test
|
THT_OLLAMA_URL=https://embeddings.example.test
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Avviare e verificare con il profilo server completo:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose --env-file deploy/env/server.env \
|
||||||
|
-f compose.yaml -f deploy/compose.server.yaml \
|
||||||
|
-f deploy/compose.session-server.yaml.example up --build -d
|
||||||
|
docker compose --env-file deploy/env/server.env \
|
||||||
|
-f compose.yaml -f deploy/compose.server.yaml \
|
||||||
|
-f deploy/compose.session-server.yaml.example exec core /opt/venv/bin/tht doctor --json
|
||||||
|
```
|
||||||
|
|
||||||
Il DWH e il vector DB possono essere REST/HTTP oppure adapter diretti (`postgres_direct`, `pgvector_direct`) se il server ha connettività TCP. Le Evidence possono essere:
|
Il DWH e il vector DB possono essere REST/HTTP oppure adapter diretti (`postgres_direct`, `pgvector_direct`) se il server ha connettività TCP. Le Evidence possono essere:
|
||||||
|
|
||||||
- filesystem NFS/SMB montato sul server e presentato come root read-only;
|
- filesystem NFS/SMB montato sul server e presentato come root read-only;
|
||||||
@@ -208,8 +229,8 @@ Le variabili `THT_*_SECRET_FILE` e i file `dwh-api-key`, `vector-reader-api-key`
|
|||||||
|
|
||||||
1. creare `deploy/secrets/thothii.secrets` mode `0600`;
|
1. creare `deploy/secrets/thothii.secrets` mode `0600`;
|
||||||
2. copiare ogni valore nel nome chiave corrispondente (`THT_DWH_API_KEY`, `THT_VEC_API_KEY`, `THT_VEC_WRITE_API_KEY`, `THT_MODEL_API_KEY` o `THT_VECTOR_*_PASSWORD`), senza virgolette né newline;
|
2. copiare ogni valore nel nome chiave corrispondente (`THT_DWH_API_KEY`, `THT_VEC_API_KEY`, `THT_VEC_WRITE_API_KEY`, `THT_MODEL_API_KEY` o `THT_VECTOR_*_PASSWORD`), senza virgolette né newline;
|
||||||
3. rimuovere dal `.env` le variabili `_SECRET_FILE` e impostare `THT_SECRETS_FILE` al percorso del bundle (il default relativo è già corretto);
|
3. rimuovere dal `.env` le variabili `_SECRET_FILE` e impostare `THT_SECRETS_FILE` al percorso assoluto del bundle;
|
||||||
4. eseguire `docker compose config --quiet` e poi `docker compose up --build -d`;
|
4. renderizzare e avviare con il comando base+locale completo e il suo `--env-file`;
|
||||||
5. solo dopo la verifica, cancellare i vecchi file separati.
|
5. solo dopo la verifica, cancellare i vecchi file separati.
|
||||||
|
|
||||||
Una CA PEM resta un'eccezione esterna come descritto sopra. Provider Pi con credenziali composte (Bedrock, Azure OpenAI Responses, Cloudflare Workers AI/Gateway) restano rifiutati finché non viene implementato un adapter dedicato.
|
Una CA PEM resta un'eccezione esterna come descritto sopra. Provider Pi con credenziali composte (Bedrock, Azure OpenAI Responses, Cloudflare Workers AI/Gateway) restano rifiutati finché non viene implementato un adapter dedicato.
|
||||||
@@ -217,9 +238,12 @@ Una CA PEM resta un'eccezione esterna come descritto sopra. Provider Pi con cred
|
|||||||
## Controlli post-installazione
|
## Controlli post-installazione
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker compose config --quiet
|
docker compose --env-file deploy/env/local.env \
|
||||||
docker compose ps
|
-f compose.yaml -f deploy/compose.local.yaml config --quiet
|
||||||
docker compose exec core /opt/venv/bin/tht doctor --json
|
docker compose --env-file deploy/env/local.env \
|
||||||
|
-f compose.yaml -f deploy/compose.local.yaml ps
|
||||||
|
docker compose --env-file deploy/env/local.env \
|
||||||
|
-f compose.yaml -f deploy/compose.local.yaml exec core /opt/venv/bin/tht doctor --json
|
||||||
./scripts/docker-smoke.sh
|
./scripts/docker-smoke.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
# Workspace ThothII — Profilo A (server co-locato). DWH + vector BOTH direct, no REST.
|
# Workspace ThothII — Profilo A (server co-locato). DWH + vector BOTH direct, no REST.
|
||||||
# Segreti SOLO in env (compose env_file: deploy/thothii.env). Path assoluti interni al container (/data).
|
# Secret contents live only in protected mounted files; paths below are container-absolute.
|
||||||
language: it
|
language: it
|
||||||
|
|
||||||
database:
|
database:
|
||||||
|
|||||||
@@ -3,11 +3,11 @@ $ErrorActionPreference = "Continue"
|
|||||||
$repositoryRoot = Split-Path -Parent $PSScriptRoot
|
$repositoryRoot = Split-Path -Parent $PSScriptRoot
|
||||||
Set-Location $repositoryRoot
|
Set-Location $repositoryRoot
|
||||||
|
|
||||||
& docker compose -f compose.yaml -f deploy/compose.local.yaml build --pull
|
& docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local.yaml build --pull
|
||||||
$exitCode = $LASTEXITCODE
|
$exitCode = $LASTEXITCODE
|
||||||
|
|
||||||
if ($exitCode -eq 0) {
|
if ($exitCode -eq 0) {
|
||||||
Write-Output "Next: docker compose -f compose.yaml -f deploy/compose.local.yaml up -d"
|
Write-Output "Next: docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local.yaml up -d"
|
||||||
}
|
}
|
||||||
|
|
||||||
exit $exitCode
|
exit $exitCode
|
||||||
|
|||||||
@@ -3,11 +3,12 @@ set -u
|
|||||||
|
|
||||||
cd "$(dirname "$0")/.."
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
docker compose -f compose.yaml -f deploy/compose.local.yaml build --pull
|
docker compose --env-file deploy/env/local.env \
|
||||||
|
-f compose.yaml -f deploy/compose.local.yaml build --pull
|
||||||
status=$?
|
status=$?
|
||||||
|
|
||||||
if [[ "$status" -eq 0 ]]; then
|
if [[ "$status" -eq 0 ]]; then
|
||||||
printf '%s\n' 'Next: docker compose -f compose.yaml -f deploy/compose.local.yaml up -d'
|
printf '%s\n' 'Next: docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local.yaml up -d'
|
||||||
fi
|
fi
|
||||||
|
|
||||||
exit "$status"
|
exit "$status"
|
||||||
|
|||||||
@@ -1,21 +1,21 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Smoke test del deploy standalone ThothII (core + frontend).
|
# Smoke test del deploy standalone ThothII (core + frontend).
|
||||||
# Usa docker-compose.dev.yml (rete propria, porte host).
|
# Usa docker-compose.dev.yml (rete propria, porte host).
|
||||||
# Prereq: deploy/thothii.env popolato, endpoint esterni configurati e profilo Pi locale.
|
# Prereq: deploy/env/local.env popolato, endpoint esterni e file Pi/segreti configurati.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
cd "$(dirname "$0")/.."
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
DC="docker compose -f docker-compose.dev.yml"
|
DC=(docker compose --env-file deploy/env/local.env -f docker-compose.dev.yml)
|
||||||
WS="/app/harness/workspaces/local.yaml"
|
WS="/app/harness/workspaces/local.yaml"
|
||||||
|
|
||||||
echo "== ThothII standalone smoke =="
|
echo "== ThothII standalone smoke =="
|
||||||
$DC config --quiet
|
"${DC[@]}" config --quiet
|
||||||
|
|
||||||
echo "== Build =="
|
echo "== Build =="
|
||||||
$DC build
|
"${DC[@]}" build
|
||||||
|
|
||||||
echo "== Up (wait health) =="
|
echo "== Up (wait health) =="
|
||||||
$DC up -d --wait
|
"${DC[@]}" up -d --wait
|
||||||
|
|
||||||
echo "== Core health =="
|
echo "== Core health =="
|
||||||
curl -fsS http://localhost:8787/health && echo
|
curl -fsS http://localhost:8787/health && echo
|
||||||
@@ -24,12 +24,12 @@ echo "== Frontend serve =="
|
|||||||
curl -fsSI http://localhost:8090/ | head -1
|
curl -fsSI http://localhost:8090/ | head -1
|
||||||
|
|
||||||
echo "== Wiring check (config + DWH ping; -c è per-command) =="
|
echo "== Wiring check (config + DWH ping; -c è per-command) =="
|
||||||
$DC exec -T core tht config check -c "$WS" || \
|
"${DC[@]}" exec -T core tht config check -c "$WS" || \
|
||||||
echo "(config check non verde: verificare .env/ruoli DB)"
|
echo "(config check non verde: verificare .env/ruoli DB)"
|
||||||
$DC exec -T core tht db ping -c "$WS" || \
|
"${DC[@]}" exec -T core tht db ping -c "$WS" || \
|
||||||
echo "(db ping non verde: verificare ruolo thoth_dwh_reader + rete)"
|
echo "(db ping non verde: verificare ruolo thoth_dwh_reader + rete)"
|
||||||
|
|
||||||
echo "== Down =="
|
echo "== Down =="
|
||||||
$DC down
|
"${DC[@]}" down
|
||||||
|
|
||||||
echo "OK: smoke standalone passato."
|
echo "OK: smoke standalone passato."
|
||||||
|
|||||||
@@ -101,7 +101,13 @@ done < <(
|
|||||||
|
|
||||||
{
|
{
|
||||||
printf '%s\n' '# Generated by scripts/generate-connector-secrets-override.sh; keep this file untracked.'
|
printf '%s\n' '# Generated by scripts/generate-connector-secrets-override.sh; keep this file untracked.'
|
||||||
printf '%s\n' 'services:' ' core:' ' secrets:'
|
printf '%s\n' \
|
||||||
|
'services:' \
|
||||||
|
' core:' \
|
||||||
|
' env_file:' \
|
||||||
|
' - path: ${THT_WORKSPACE_BINDINGS_ENV_FILE:?set THT_WORKSPACE_BINDINGS_ENV_FILE}' \
|
||||||
|
' required: true' \
|
||||||
|
' secrets:'
|
||||||
for ((index = 0; index < ${#names[@]}; index += 1)); do
|
for ((index = 0; index < ${#names[@]}; index += 1)); do
|
||||||
printf ' - source: connector_secret_%d\n' "$((index + 1))"
|
printf ' - source: connector_secret_%d\n' "$((index + 1))"
|
||||||
printf ' target: %s\n' "${targets[index]}"
|
printf ' target: %s\n' "${targets[index]}"
|
||||||
|
|||||||
@@ -39,6 +39,13 @@ write_bundle() {
|
|||||||
}
|
}
|
||||||
write_bundle
|
write_bundle
|
||||||
export THT_SECRETS_FILE="$bundle"
|
export THT_SECRETS_FILE="$bundle"
|
||||||
|
printf '%s\n' '{}' >"$secret_dir/pi-auth.json"
|
||||||
|
chmod 0600 "$secret_dir/pi-auth.json"
|
||||||
|
operator_env="$secret_dir/operator.env"
|
||||||
|
printf '%s\n' \
|
||||||
|
'THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git' \
|
||||||
|
"PI_AUTH_FILE=$secret_dir/pi-auth.json" \
|
||||||
|
"THT_SECRETS_FILE=$bundle" >"$operator_env"
|
||||||
# The rotation helper has an old/new file interface; these are test-only
|
# The rotation helper has an old/new file interface; these are test-only
|
||||||
# scratch files and are never mounted into a Compose service.
|
# scratch files and are never mounted into a Compose service.
|
||||||
printf '%s' "$bootstrap_password" >"$secret_dir/bootstrap"
|
printf '%s' "$bootstrap_password" >"$secret_dir/bootstrap"
|
||||||
@@ -47,7 +54,7 @@ export THT_VECTOR_BOOTSTRAP_USER=thoth_bootstrap_smoke
|
|||||||
export THOTH_SMOKE_OWNER="$smoke_owner"
|
export THOTH_SMOKE_OWNER="$smoke_owner"
|
||||||
|
|
||||||
compose() {
|
compose() {
|
||||||
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
|
docker compose --env-file "$operator_env" -f compose.yaml -f deploy/compose.local-vector.yaml \
|
||||||
--project-name "$smoke_project" --profile local-vector "$@"
|
--project-name "$smoke_project" --profile local-vector "$@"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -58,6 +58,13 @@ bundle="$tmp/thothii.secrets"
|
|||||||
chmod 0600 "$bundle"
|
chmod 0600 "$bundle"
|
||||||
export THT_SECRETS_FILE="$bundle"
|
export THT_SECRETS_FILE="$bundle"
|
||||||
export THT_OLLAMA_URL=http://mock-embeddings:8081
|
export THT_OLLAMA_URL=http://mock-embeddings:8081
|
||||||
|
printf '%s\n' '{}' >"$tmp/pi-auth.json"
|
||||||
|
chmod 0600 "$tmp/pi-auth.json"
|
||||||
|
printf '%s\n' \
|
||||||
|
'THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git' \
|
||||||
|
"PI_AUTH_FILE=$tmp/pi-auth.json" \
|
||||||
|
"THT_SECRETS_FILE=$bundle" \
|
||||||
|
'THT_OLLAMA_URL=http://mock-embeddings:8081' >"$tmp/operator.env"
|
||||||
|
|
||||||
cat >"$tmp/smoke.yaml" <<YAML
|
cat >"$tmp/smoke.yaml" <<YAML
|
||||||
services:
|
services:
|
||||||
@@ -83,7 +90,7 @@ services:
|
|||||||
mock-embeddings: {condition: service_started}
|
mock-embeddings: {condition: service_started}
|
||||||
YAML
|
YAML
|
||||||
|
|
||||||
compose="docker compose -f compose.yaml -f deploy/compose.local-vector.yaml -f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml -f $tmp/smoke.yaml --project-name $project --profile local-vector --profile preprocess"
|
compose="docker compose --env-file $tmp/operator.env -f compose.yaml -f deploy/compose.local-vector.yaml -f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml -f $tmp/smoke.yaml --project-name $project --profile local-vector --profile preprocess"
|
||||||
$compose build preprocess-evidence
|
$compose build preprocess-evidence
|
||||||
if [ "${PREPROCESS_SMOKE_INJECT_FAILURE:-0}" = "1" ]; then
|
if [ "${PREPROCESS_SMOKE_INJECT_FAILURE:-0}" = "1" ]; then
|
||||||
sh -c 'exit 97'
|
sh -c 'exit 97'
|
||||||
|
|||||||
Executable
+99
@@ -0,0 +1,99 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Active installation manuals must drive the canonical two-service base+profile stack.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
tmp="$(mktemp -d "${TMPDIR%/}/thoth-canonical-install.XXXXXX")"
|
||||||
|
trap 'rm -rf "$tmp"' EXIT HUP INT TERM
|
||||||
|
|
||||||
|
for retired_example in \
|
||||||
|
"$root/docs/install/examples/local-compose.workspace-registry.yaml" \
|
||||||
|
"$root/docs/install/examples/server-compose.workspace-registry.yaml" \
|
||||||
|
"$root/docs/install/examples/git-ssh.workspace-registry.yaml" \
|
||||||
|
"$root/docs/install/examples/git-https.workspace-registry.yaml"; do
|
||||||
|
if [[ -e "$retired_example" ]]; then
|
||||||
|
echo "superseded one-service install example remains active: ${retired_example#"$root/"}" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
printf '%s\n' '{}' >"$tmp/pi-auth.json"
|
||||||
|
printf '%s\n' 'THT_MODEL_API_KEY=fixture-model-api-key' >"$tmp/thothii.secrets"
|
||||||
|
chmod 0600 "$tmp/pi-auth.json" "$tmp/thothii.secrets"
|
||||||
|
mkdir -p "$tmp/data" "$tmp/pi-state" "$tmp/workspace-registry"
|
||||||
|
printf '%s\n' 'fixture-session-password' >"$tmp/session-runtime-password"
|
||||||
|
printf '%s\n' 'fixture-session-migrator-password' >"$tmp/session-migrator-password"
|
||||||
|
printf '%s\n' 'fixture-session-ca' >"$tmp/session-ca.pem"
|
||||||
|
cp "$root/deploy/workspaces/server-sessions.yaml.example" "$tmp/server-sessions.yaml"
|
||||||
|
chmod 0600 "$tmp/session-runtime-password" "$tmp/session-migrator-password" "$tmp/session-ca.pem"
|
||||||
|
|
||||||
|
for profile in local server; do
|
||||||
|
env_file="$tmp/$profile.env"
|
||||||
|
{
|
||||||
|
printf '%s\n' \
|
||||||
|
'THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git' \
|
||||||
|
"PI_AUTH_FILE=$tmp/pi-auth.json" \
|
||||||
|
"THT_SECRETS_FILE=$tmp/thothii.secrets"
|
||||||
|
if [[ "$profile" == server ]]; then
|
||||||
|
printf '%s\n' \
|
||||||
|
"THT_DATA_ROOT=$tmp/data" \
|
||||||
|
"THT_PI_STATE_ROOT=$tmp/pi-state" \
|
||||||
|
"THT_WORKSPACE_REGISTRY_ROOT=$tmp/workspace-registry" \
|
||||||
|
"THT_SERVER_WORKSPACE_CONFIG=$tmp/server-sessions.yaml" \
|
||||||
|
'THT_SESSION_DB_HOST=sessions.example.invalid' \
|
||||||
|
'THT_SESSION_DB_NAME=thoth_sessions' \
|
||||||
|
'THT_SESSION_RUNTIME_USER=thoth_sessions_app' \
|
||||||
|
'THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate' \
|
||||||
|
"THT_SESSION_RUNTIME_PASSWORD_SOURCE=$tmp/session-runtime-password" \
|
||||||
|
"THT_SESSION_MIGRATOR_PASSWORD_SOURCE=$tmp/session-migrator-password" \
|
||||||
|
"THT_SESSION_CA_SOURCE=$tmp/session-ca.pem"
|
||||||
|
fi
|
||||||
|
} >"$env_file"
|
||||||
|
|
||||||
|
compose_files=(-f "$root/compose.yaml" -f "$root/deploy/compose.$profile.yaml")
|
||||||
|
if [[ "$profile" == server ]]; then
|
||||||
|
compose_files+=(-f "$root/deploy/compose.session-server.yaml.example")
|
||||||
|
fi
|
||||||
|
docker compose --env-file "$env_file" "${compose_files[@]}" \
|
||||||
|
config --format json >"$tmp/$profile.json"
|
||||||
|
node - "$tmp/$profile.json" "$profile" <<'NODE'
|
||||||
|
const fs = require("fs");
|
||||||
|
const [path, profile] = process.argv.slice(2);
|
||||||
|
const config = JSON.parse(fs.readFileSync(path, "utf8"));
|
||||||
|
if (Object.keys(config.services).sort().join(",") !== "core,frontend") {
|
||||||
|
throw new Error(profile + ": install stack must be exactly core,frontend");
|
||||||
|
}
|
||||||
|
if (!config.services.core.secrets?.some((secret) => secret.target === "thothii.secrets")) {
|
||||||
|
throw new Error(profile + ": install stack lacks the runtime secret bundle");
|
||||||
|
}
|
||||||
|
if (!config.services.core.volumes?.some(
|
||||||
|
(mount) => mount.target === "/home/thoth/.pi/agent/auth.json" && mount.read_only,
|
||||||
|
)) {
|
||||||
|
throw new Error(profile + ": install stack lacks the read-only Pi auth file");
|
||||||
|
}
|
||||||
|
if ((config.services.frontend.secrets || []).length !== 0) {
|
||||||
|
throw new Error(profile + ": frontend received runtime secrets");
|
||||||
|
}
|
||||||
|
if (profile === "server" && config.services.core.environment?.THT_SESSION_STORAGE !== "postgres") {
|
||||||
|
throw new Error("server: public startup must include the PostgreSQL session override");
|
||||||
|
}
|
||||||
|
if (JSON.stringify(config).includes("fixture-model-api-key")) {
|
||||||
|
throw new Error(profile + ": rendered Compose leaked a secret value");
|
||||||
|
}
|
||||||
|
NODE
|
||||||
|
done
|
||||||
|
|
||||||
|
for profile in local server; do
|
||||||
|
manual="$root/docs/install/$profile-workspace-registry.md"
|
||||||
|
grep -Fq -- '--env-file "$THT_OPERATOR_ENV"' "$manual" \
|
||||||
|
&& grep -Fq -- "-f \"\$THT_SOURCE_ROOT/compose.yaml\" -f \"\$THT_SOURCE_ROOT/deploy/compose.$profile.yaml\"" "$manual" || {
|
||||||
|
echo "$profile manual lacks the canonical base+profile command" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
if rg -q 'local-compose\.workspace-registry|server-compose\.workspace-registry' "$manual"; then
|
||||||
|
echo "$profile manual still references a superseded standalone Compose example" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "canonical install Compose contract passed."
|
||||||
Executable
+74
@@ -0,0 +1,74 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Fresh Compose flow: mounted Pi policy/auth must produce a selectable, credential-ready provider.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
tmp="$(mktemp -d "${TMPDIR%/}/thoth-provider-readiness.XXXXXX")"
|
||||||
|
project="thothii-provider-readiness-$$"
|
||||||
|
compose=(
|
||||||
|
docker compose --project-name "$project" --env-file "$tmp/local.env"
|
||||||
|
-f "$root/compose.yaml" -f "$root/deploy/compose.local.yaml"
|
||||||
|
)
|
||||||
|
cleanup() {
|
||||||
|
"${compose[@]}" down --volumes --remove-orphans >/dev/null 2>&1 || true
|
||||||
|
rm -rf "$tmp"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT HUP INT TERM
|
||||||
|
|
||||||
|
printf '%s\n' '{"zai":{"type":"api_key","key":"fixture-native-auth-key"}}' >"$tmp/pi-auth.json"
|
||||||
|
printf '%s\n' 'THT_MODEL_API_KEY=fixture-model-api-key' >"$tmp/thothii.secrets"
|
||||||
|
chmod 0600 "$tmp/pi-auth.json" "$tmp/thothii.secrets"
|
||||||
|
printf '%s\n' \
|
||||||
|
'THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git' \
|
||||||
|
"PI_AUTH_FILE=$tmp/pi-auth.json" \
|
||||||
|
"THT_SECRETS_FILE=$tmp/thothii.secrets" \
|
||||||
|
'THOTH_CORE_HTTP_PORT=0' \
|
||||||
|
'THOTH_HTTP_PORT=0' \
|
||||||
|
>"$tmp/local.env"
|
||||||
|
|
||||||
|
"${compose[@]}" up --detach --wait --wait-timeout 90 --build core
|
||||||
|
core_id="$("${compose[@]}" ps -q core)"
|
||||||
|
core_address="$("${compose[@]}" port core 8787 | head -n 1)"
|
||||||
|
|
||||||
|
"${compose[@]}" exec -T core sh -ceu '
|
||||||
|
test -r /home/thoth/.pi/agent/auth.json
|
||||||
|
test -r /home/thoth/.pi/agent/models.json
|
||||||
|
test -r /home/thoth/.pi/agent/settings.json
|
||||||
|
test -r /run/secrets/thothii.secrets
|
||||||
|
'
|
||||||
|
|
||||||
|
curl --fail --silent --show-error "http://$core_address/models" >"$tmp/models.json"
|
||||||
|
node - "$tmp/models.json" <<'NODE'
|
||||||
|
const fs = require("fs");
|
||||||
|
const body = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
|
||||||
|
if (!body.models?.some((model) => model.provider === "zai" && model.id === "glm-5.2")) {
|
||||||
|
throw new Error("fresh Compose did not expose the mounted Pi-enabled model");
|
||||||
|
}
|
||||||
|
NODE
|
||||||
|
|
||||||
|
curl --fail --silent --show-error -X PUT \
|
||||||
|
-H 'content-type: application/json' \
|
||||||
|
--data '{"provider":"zai","model":"glm-5.2","reasoning":"low"}' \
|
||||||
|
"http://$core_address/pi-management/config" >"$tmp/configured.json"
|
||||||
|
curl --fail --silent --show-error \
|
||||||
|
"http://$core_address/pi-management/status" >"$tmp/status.json"
|
||||||
|
node - "$tmp/status.json" <<'NODE'
|
||||||
|
const fs = require("fs");
|
||||||
|
const body = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
|
||||||
|
if (!body.ready || body.credentials !== "present") {
|
||||||
|
throw new Error("mounted Pi provider is not credential-ready");
|
||||||
|
}
|
||||||
|
if (body.config?.provider !== "zai" || body.config?.model !== "glm-5.2") {
|
||||||
|
throw new Error("Pi provider configuration was not persisted");
|
||||||
|
}
|
||||||
|
NODE
|
||||||
|
|
||||||
|
inspect="$(docker inspect "$core_id")"
|
||||||
|
for secret in fixture-native-auth-key fixture-model-api-key; do
|
||||||
|
if grep -Fq "$secret" <<<"$inspect"; then
|
||||||
|
echo "container inspection leaked $secret" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Compose provider-readiness contract passed."
|
||||||
@@ -48,7 +48,13 @@ for (const mount of (core.volumes || []).filter((item) => item.target?.startsWit
|
|||||||
const secretTargets = (core.secrets || []).map((secret) => secret.target).sort();
|
const secretTargets = (core.secrets || []).map((secret) => secret.target).sort();
|
||||||
const expectedSecrets = expectedSecretTargets ? expectedSecretTargets.split(",").filter(Boolean).sort() : [];
|
const expectedSecrets = expectedSecretTargets ? expectedSecretTargets.split(",").filter(Boolean).sort() : [];
|
||||||
if (secretTargets.join(",") !== expectedSecrets.join(",")) {
|
if (secretTargets.join(",") !== expectedSecrets.join(",")) {
|
||||||
throw new Error(`${name}: connector targets do not match generated THT_WS_*_FILE bindings`);
|
throw new Error(`${name}: Docker secret targets do not match the deployment contract`);
|
||||||
|
}
|
||||||
|
if (core.environment?.THT_SECRETS_FILE !== "/run/secrets/thothii.secrets") {
|
||||||
|
throw new Error(`${name}: core does not use the canonical /run/secrets bundle path`);
|
||||||
|
}
|
||||||
|
if ((config.services.frontend?.secrets || []).length !== 0) {
|
||||||
|
throw new Error(`${name}: frontend must not receive runtime secrets`);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (name === "ssh") {
|
if (name === "ssh") {
|
||||||
@@ -62,7 +68,7 @@ if (name === "https" && core.environment?.GIT_CONFIG_VALUE_1 !== "/run/secrets/w
|
|||||||
}
|
}
|
||||||
|
|
||||||
const rendered = JSON.stringify(config);
|
const rendered = JSON.stringify(config);
|
||||||
for (const secret of ["fixture-ssh-private-key", "fixture-ssh-known-hosts", "fixture-https-credentials", "fixture-https-ca", "fixture-dwh-password", "fixture-vector-api-key"]) {
|
for (const secret of ["fixture-model-api-key", "fixture-ssh-private-key", "fixture-ssh-known-hosts", "fixture-https-credentials", "fixture-https-ca", "fixture-dwh-password", "fixture-vector-api-key"]) {
|
||||||
if (rendered.includes(secret)) throw new Error(`${name}: rendered Compose leaked fixture secret value`);
|
if (rendered.includes(secret)) throw new Error(`${name}: rendered Compose leaked fixture secret value`);
|
||||||
}
|
}
|
||||||
NODE
|
NODE
|
||||||
@@ -97,6 +103,7 @@ assert_unsafe_source_rejected() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
write_secret "$fixture_root/pi-auth.json" 'fixture-pi-auth'
|
write_secret "$fixture_root/pi-auth.json" 'fixture-pi-auth'
|
||||||
|
write_secret "$fixture_root/thothii.secrets" 'THT_MODEL_API_KEY=fixture-model-api-key'
|
||||||
write_secret "$fixture_root/ssh-private-key" 'fixture-ssh-private-key'
|
write_secret "$fixture_root/ssh-private-key" 'fixture-ssh-private-key'
|
||||||
write_secret "$fixture_root/ssh-known-hosts" 'fixture-ssh-known-hosts'
|
write_secret "$fixture_root/ssh-known-hosts" 'fixture-ssh-known-hosts'
|
||||||
write_secret "$fixture_root/https-credentials" 'fixture-https-credentials'
|
write_secret "$fixture_root/https-credentials" 'fixture-https-credentials'
|
||||||
@@ -107,6 +114,8 @@ write_secret "$fixture_root/vector-api-key" 'fixture-vector-api-key'
|
|||||||
printf '%s\n' \
|
printf '%s\n' \
|
||||||
'THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git' \
|
'THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git' \
|
||||||
"PI_AUTH_FILE=$fixture_root/pi-auth.json" \
|
"PI_AUTH_FILE=$fixture_root/pi-auth.json" \
|
||||||
|
"THT_SECRETS_FILE=$fixture_root/thothii.secrets" \
|
||||||
|
"THT_WORKSPACE_BINDINGS_ENV_FILE=$fixture_root/workspace-bindings.env" \
|
||||||
"THT_WORKSPACE_GIT_SSH_KEY_FILE=$fixture_root/ssh-private-key" \
|
"THT_WORKSPACE_GIT_SSH_KEY_FILE=$fixture_root/ssh-private-key" \
|
||||||
"THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=$fixture_root/ssh-known-hosts" \
|
"THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=$fixture_root/ssh-known-hosts" \
|
||||||
"THT_WORKSPACE_GIT_CREDENTIALS_FILE=$fixture_root/https-credentials" \
|
"THT_WORKSPACE_GIT_CREDENTIALS_FILE=$fixture_root/https-credentials" \
|
||||||
@@ -127,13 +136,13 @@ connector_override="$fixture_root/compose.connector-secrets.local.yaml"
|
|||||||
--output "$connector_override"
|
--output "$connector_override"
|
||||||
|
|
||||||
render base
|
render base
|
||||||
assert_render_contract base '' ''
|
assert_render_contract base '' 'thothii.secrets'
|
||||||
render ssh -f "$root/deploy/compose.git-ssh.yaml"
|
render ssh -f "$root/deploy/compose.git-ssh.yaml"
|
||||||
assert_render_contract ssh '/run/secrets/workspace-registry-git-known-hosts,/run/secrets/workspace-registry-git-ssh-key' ''
|
assert_render_contract ssh '/run/secrets/workspace-registry-git-known-hosts,/run/secrets/workspace-registry-git-ssh-key' 'thothii.secrets'
|
||||||
render https -f "$root/deploy/compose.git-https.yaml"
|
render https -f "$root/deploy/compose.git-https.yaml"
|
||||||
assert_render_contract https '/run/secrets/workspace-registry-git-ca,/run/secrets/workspace-registry-git-credentials' ''
|
assert_render_contract https '/run/secrets/workspace-registry-git-ca,/run/secrets/workspace-registry-git-credentials' 'thothii.secrets'
|
||||||
render connector -f "$connector_override"
|
render connector -f "$connector_override"
|
||||||
assert_render_contract connector '' 'north-star-research-dwh-password,north-star-research-vector-api-key'
|
assert_render_contract connector '' 'north-star-research-dwh-password,north-star-research-vector-api-key,thothii.secrets'
|
||||||
|
|
||||||
assert_missing_source_rejected THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_SOURCE
|
assert_missing_source_rejected THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_SOURCE
|
||||||
assert_missing_source_rejected THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_SOURCE
|
assert_missing_source_rejected THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_SOURCE
|
||||||
|
|||||||
@@ -9,10 +9,13 @@ expected_pi_version=$(sed -n 's/^ARG PI_VERSION=//p' docker/core.Dockerfile)
|
|||||||
trap 'docker compose --project-name "$project" -f compose.yaml -f deploy/compose.local.yaml down --volumes --remove-orphans >/dev/null 2>&1 || true; rm -rf "$tmp"' EXIT HUP INT TERM
|
trap 'docker compose --project-name "$project" -f compose.yaml -f deploy/compose.local.yaml down --volumes --remove-orphans >/dev/null 2>&1 || true; rm -rf "$tmp"' EXIT HUP INT TERM
|
||||||
|
|
||||||
test -n "$expected_pi_version"
|
test -n "$expected_pi_version"
|
||||||
printf '{}\n' >"$tmp/pi-auth.json"
|
printf '%s\n' '{"zai":{"type":"api_key","key":"fixture-native-auth-key"}}' >"$tmp/pi-auth.json"
|
||||||
chmod 0600 "$tmp/pi-auth.json"
|
chmod 0600 "$tmp/pi-auth.json"
|
||||||
|
printf '%s\n' 'THT_MODEL_API_KEY=fixture-model-api-key' >"$tmp/thothii.secrets"
|
||||||
|
chmod 0600 "$tmp/thothii.secrets"
|
||||||
|
|
||||||
export PI_AUTH_FILE="$tmp/pi-auth.json"
|
export PI_AUTH_FILE="$tmp/pi-auth.json"
|
||||||
|
export THT_SECRETS_FILE="$tmp/thothii.secrets"
|
||||||
export THT_WORKSPACE_GIT_REMOTE="https://git.example.invalid/thothii/workspaces.git"
|
export THT_WORKSPACE_GIT_REMOTE="https://git.example.invalid/thothii/workspaces.git"
|
||||||
# Let Docker assign loopback ports so this isolated contract test never collides with an operator stack.
|
# Let Docker assign loopback ports so this isolated contract test never collides with an operator stack.
|
||||||
export THOTH_CORE_HTTP_PORT=0
|
export THOTH_CORE_HTTP_PORT=0
|
||||||
@@ -62,6 +65,10 @@ docker compose --project-name "$project" -f compose.yaml -f deploy/compose.local
|
|||||||
test "$(pi --version)" = "$PI_VERSION"
|
test "$(pi --version)" = "$PI_VERSION"
|
||||||
command -v pi >/dev/null
|
command -v pi >/dev/null
|
||||||
test ! -e /var/run/docker.sock
|
test ! -e /var/run/docker.sock
|
||||||
|
test -r /home/thoth/.pi/agent/auth.json
|
||||||
|
test -r /home/thoth/.pi/agent/models.json
|
||||||
|
test -r /home/thoth/.pi/agent/settings.json
|
||||||
|
test -r /run/secrets/thothii.secrets
|
||||||
touch /data/.task5-writable
|
touch /data/.task5-writable
|
||||||
rm /data/.task5-writable
|
rm /data/.task5-writable
|
||||||
if find /app /home /data -xdev \( -iname "*chirone*" -o -iname "*omics*portal*" \) -print -quit | grep -q .; then
|
if find /app /home /data -xdev \( -iname "*chirone*" -o -iname "*omics*portal*" \) -print -quit | grep -q .; then
|
||||||
|
|||||||
Executable
+66
@@ -0,0 +1,66 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Prevent active operator-facing startup examples from bypassing required env/profile inputs.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
cd "$root"
|
||||||
|
|
||||||
|
targets=(
|
||||||
|
README.md
|
||||||
|
.env.example
|
||||||
|
docker-compose.dev.yml
|
||||||
|
deploy/env.example
|
||||||
|
deploy/secrets/README.md
|
||||||
|
docs/install
|
||||||
|
docs/index.md
|
||||||
|
docs/installazione-docker-4-contesti.md
|
||||||
|
scripts/build-local.sh
|
||||||
|
scripts/build-local.ps1
|
||||||
|
scripts/docker-smoke.sh
|
||||||
|
scripts/local-vector-smoke.sh
|
||||||
|
scripts/preprocess-smoke.sh
|
||||||
|
scripts/vector-rotate-bootstrap-password.sh
|
||||||
|
)
|
||||||
|
|
||||||
|
existing=()
|
||||||
|
for target in "${targets[@]}"; do
|
||||||
|
[[ ! -e "$target" ]] || existing+=("$target")
|
||||||
|
done
|
||||||
|
|
||||||
|
set +e
|
||||||
|
matches="$(rg -n \
|
||||||
|
'docker compose (up|build|run|config|ps|exec|-f)|DC="docker compose -f|compose="docker compose -f' \
|
||||||
|
"${existing[@]}" 2>&1)"
|
||||||
|
rg_status=$?
|
||||||
|
set -e
|
||||||
|
case "$rg_status" in
|
||||||
|
0)
|
||||||
|
echo "active deployment command omits --env-file before its action/overrides:" >&2
|
||||||
|
printf '%s\n' "$matches" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
1) ;;
|
||||||
|
*)
|
||||||
|
printf '%s\n' "$matches" >&2
|
||||||
|
exit "$rg_status"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
for document in README.md docs/installazione-docker-4-contesti.md; do
|
||||||
|
grep -Fq -- '-f deploy/compose.session-server.yaml.example' "$document" || {
|
||||||
|
echo "$document omits the required public-server session override" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
done
|
||||||
|
for required in \
|
||||||
|
THT_SERVER_WORKSPACE_CONFIG \
|
||||||
|
THT_SESSION_RUNTIME_PASSWORD_SOURCE \
|
||||||
|
THT_SESSION_MIGRATOR_PASSWORD_SOURCE \
|
||||||
|
THT_SESSION_CA_SOURCE; do
|
||||||
|
grep -q "^$required=" deploy/env/server.env.example || {
|
||||||
|
echo "server env example omits $required" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "deployment command contract passed."
|
||||||
@@ -8,6 +8,7 @@ trap cleanup EXIT HUP INT TERM
|
|||||||
|
|
||||||
export THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
export THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
||||||
export PI_AUTH_FILE=/dev/null
|
export PI_AUTH_FILE=/dev/null
|
||||||
|
export THT_SECRETS_FILE=/dev/null
|
||||||
|
|
||||||
unset THT_VECTOR_BOOTSTRAP_PASSWORD_SECRET_FILE THT_VECTOR_MIGRATOR_PASSWORD_SECRET_FILE
|
unset THT_VECTOR_BOOTSTRAP_PASSWORD_SECRET_FILE THT_VECTOR_MIGRATOR_PASSWORD_SECRET_FILE
|
||||||
unset THT_VECTOR_READER_PASSWORD_SECRET_FILE THT_VECTOR_WRITER_PASSWORD_SECRET_FILE
|
unset THT_VECTOR_READER_PASSWORD_SECRET_FILE THT_VECTOR_WRITER_PASSWORD_SECRET_FILE
|
||||||
|
|||||||
Executable
+38
@@ -0,0 +1,38 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Model providers are external endpoints reached through the ordinary application network.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
tmp="$(mktemp -d "${TMPDIR%/}/thoth-external-llm.XXXXXX")"
|
||||||
|
trap 'rm -rf "$tmp"' EXIT HUP INT TERM
|
||||||
|
printf '%s\n' '{}' >"$tmp/pi-auth.json"
|
||||||
|
printf '%s\n' 'THT_MODEL_API_KEY=fixture-model-api-key' >"$tmp/thothii.secrets"
|
||||||
|
chmod 0600 "$tmp/pi-auth.json" "$tmp/thothii.secrets"
|
||||||
|
|
||||||
|
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
|
||||||
|
PI_AUTH_FILE="$tmp/pi-auth.json" \
|
||||||
|
THT_SECRETS_FILE="$tmp/thothii.secrets" \
|
||||||
|
THT_LLM_URL=https://llm.example.invalid/v1 \
|
||||||
|
docker compose -f "$root/compose.yaml" config --format json >"$tmp/config.json"
|
||||||
|
|
||||||
|
node - "$tmp/config.json" <<'NODE'
|
||||||
|
const fs = require("fs");
|
||||||
|
const config = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
|
||||||
|
if (Object.keys(config.services).sort().join(",") !== "core,frontend") {
|
||||||
|
throw new Error("external LLM deployment must retain the mandatory two-service stack");
|
||||||
|
}
|
||||||
|
if (Object.keys(config.networks || {}).join(",") !== "thothii") {
|
||||||
|
throw new Error("external LLM endpoint must not require a provider-owned Docker network");
|
||||||
|
}
|
||||||
|
if (config.services.core.environment?.THT_LLM_URL !== "https://llm.example.invalid/v1") {
|
||||||
|
throw new Error("core did not receive the generic external LLM endpoint");
|
||||||
|
}
|
||||||
|
const joins = (service, network) => Array.isArray(service.networks)
|
||||||
|
? service.networks.includes(network)
|
||||||
|
: Object.hasOwn(service.networks || {}, network);
|
||||||
|
if (!joins(config.services.core, "thothii") || !joins(config.services.frontend, "thothii")) {
|
||||||
|
throw new Error("frontend and core must share only the application network");
|
||||||
|
}
|
||||||
|
NODE
|
||||||
|
|
||||||
|
echo "external LLM network contract passed."
|
||||||
Executable
+84
@@ -0,0 +1,84 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Regression coverage for coupling-scan categories, exact exclusions, and scanner failures.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
fixture="$(mktemp -d "${TMPDIR%/}/thoth-coupling-scope.XXXXXX")"
|
||||||
|
trap 'rm -rf "$fixture"' EXIT HUP INT TERM
|
||||||
|
|
||||||
|
new_fixture() {
|
||||||
|
rm -rf "$fixture/repository"
|
||||||
|
mkdir -p \
|
||||||
|
"$fixture/repository/deploy/env" \
|
||||||
|
"$fixture/repository/deploy/workspaces" \
|
||||||
|
"$fixture/repository/docker/smoke" \
|
||||||
|
"$fixture/repository/docs/install" \
|
||||||
|
"$fixture/repository/docs/superpowers/plans" \
|
||||||
|
"$fixture/repository/frontend" \
|
||||||
|
"$fixture/repository/scripts"
|
||||||
|
|
||||||
|
printf '%s\n' 'services: {}' >"$fixture/repository/compose.yaml"
|
||||||
|
printf '%s\n' '# generic runtime image' >"$fixture/repository/docker/core.Dockerfile"
|
||||||
|
printf '%s\n' '# generic smoke' >"$fixture/repository/docker/smoke/core-smoke.sh"
|
||||||
|
printf '%s\n' '# generic install' >"$fixture/repository/docs/install/local.md"
|
||||||
|
printf '%s\n' 'THT_LLM_URL=https://llm.example.invalid' >"$fixture/repository/deploy/env/local.env.example"
|
||||||
|
printf '%s\n' '# generic launcher' >"$fixture/repository/scripts/run-stack.sh"
|
||||||
|
printf '%s\n' '// generic frontend configuration' >"$fixture/repository/frontend/vite.config.ts"
|
||||||
|
|
||||||
|
# These are the three intentionally allowed categories from the Task 10 boundary.
|
||||||
|
printf '%s\n' 'historical omics_portal and Chirone record' \
|
||||||
|
>"$fixture/repository/docs/superpowers/plans/legacy.md"
|
||||||
|
printf '%s\n' 'id: psd' >"$fixture/repository/deploy/workspaces/psd.yaml.example"
|
||||||
|
printf '%s\n' '# migrate PSD sessions from /home/chirone' \
|
||||||
|
>"$fixture/repository/docker/session-migrate.sh"
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_clean() {
|
||||||
|
"$root/scripts/test-no-deployment-coupling.sh" --root "$fixture/repository" >/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_detected() {
|
||||||
|
local relative_path="$1" content="$2" output status
|
||||||
|
new_fixture
|
||||||
|
mkdir -p "$(dirname "$fixture/repository/$relative_path")"
|
||||||
|
printf '%s\n' "$content" >"$fixture/repository/$relative_path"
|
||||||
|
set +e
|
||||||
|
output="$("$root/scripts/test-no-deployment-coupling.sh" --root "$fixture/repository" 2>&1)"
|
||||||
|
status=$?
|
||||||
|
set -e
|
||||||
|
if [[ $status -ne 1 ]] || ! grep -Fq "$relative_path" <<<"$output"; then
|
||||||
|
echo "coupling scan missed $relative_path" >&2
|
||||||
|
printf '%s\n' "$output" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
new_fixture
|
||||||
|
assert_clean
|
||||||
|
|
||||||
|
assert_detected compose.yaml 'services: # Chirone runtime coupling'
|
||||||
|
assert_detected docker/smoke/core-smoke.sh 'test -d /home/chirone'
|
||||||
|
assert_detected docs/install/local.md 'Install the PSD deployment profile.'
|
||||||
|
assert_detected deploy/env/local.env.example 'NETWORK=omics_portal'
|
||||||
|
assert_detected scripts/run-stack.sh 'exec datamart-builder'
|
||||||
|
assert_detected frontend/vite.config.ts 'const base = "/omics_portal";'
|
||||||
|
assert_detected scripts/test-qwen-network-config.sh 'require localllm_default'
|
||||||
|
assert_detected scripts/test-provider-network.sh 'if (!config.networks?.localllm_default?.external) exit 1'
|
||||||
|
assert_detected deploy/compose.psd-local.yaml 'services: {}'
|
||||||
|
|
||||||
|
new_fixture
|
||||||
|
mkdir -p "$fixture/bin"
|
||||||
|
printf '%s\n' '#!/bin/sh' 'exit 2' >"$fixture/bin/rg"
|
||||||
|
chmod +x "$fixture/bin/rg"
|
||||||
|
set +e
|
||||||
|
PATH="$fixture/bin:$PATH" "$root/scripts/test-no-deployment-coupling.sh" \
|
||||||
|
--root "$fixture/repository" >"$fixture/rg.out" 2>"$fixture/rg.err"
|
||||||
|
status=$?
|
||||||
|
set -e
|
||||||
|
if [[ $status -ne 2 ]]; then
|
||||||
|
echo "coupling scan masked an rg failure (status $status)" >&2
|
||||||
|
cat "$fixture/rg.out" "$fixture/rg.err" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "no-coupling scope regression tests passed."
|
||||||
@@ -1,69 +1,125 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
|
# Category-based guard for active build, runtime, install, and launch coupling.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
cd "$(dirname "$0")/.."
|
script_root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
|
scan_root="$script_root"
|
||||||
|
if [[ "${1:-}" == --root ]]; then
|
||||||
|
[[ $# -eq 2 ]] || { echo "usage: $0 [--root PATH]" >&2; exit 2; }
|
||||||
|
scan_root="$2"
|
||||||
|
elif [[ $# -ne 0 ]]; then
|
||||||
|
echo "usage: $0 [--root PATH]" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
[[ -d "$scan_root" ]] || { echo "coupling scan root is not a directory: $scan_root" >&2; exit 2; }
|
||||||
|
cd "$scan_root"
|
||||||
|
|
||||||
content_targets=(
|
runtime_files=()
|
||||||
.dockerignore
|
install_files=()
|
||||||
compose.yaml
|
operator_files=()
|
||||||
docker-compose.dev.yml
|
contract_test_files=()
|
||||||
deploy
|
add_file() {
|
||||||
docker
|
local array_name="$1" file="$2"
|
||||||
frontend/vite.config.ts
|
[[ ! -f "$file" ]] || eval "$array_name+=(\"\$file\")"
|
||||||
README.md
|
}
|
||||||
docs/install
|
|
||||||
docs/installazione-docker-4-contesti.md
|
|
||||||
.env.example
|
|
||||||
scripts/run-stack.sh
|
|
||||||
scripts/docker-smoke.sh
|
|
||||||
)
|
|
||||||
|
|
||||||
matches=$(
|
for file in .dockerignore compose.yaml docker-compose.dev.yml frontend/vite.config.ts; do
|
||||||
rg -n -i \
|
add_file runtime_files "$file"
|
||||||
-g '!deploy/workspaces/**' \
|
done
|
||||||
-g '!docker/session-migrate.sh' \
|
if [[ -d deploy ]]; then
|
||||||
-g '!docker/cutover-legacy-sessions.sh' \
|
while IFS= read -r -d '' file; do runtime_files+=("${file#./}"); done < <(
|
||||||
-g '!docker/smoke/**' \
|
find deploy -type f ! -path 'deploy/workspaces/*' -print0
|
||||||
'omics_portal|chirone|localllm_default|datamart-builder|compose\.production\.yaml|compose\.psd-local\.yaml' \
|
)
|
||||||
"${content_targets[@]}" || true
|
fi
|
||||||
)
|
if [[ -d docker ]]; then
|
||||||
|
while IFS= read -r -d '' file; do
|
||||||
|
case "$file" in
|
||||||
|
docker/session-migrate.sh|docker/cutover-legacy-sessions.sh) continue ;;
|
||||||
|
esac
|
||||||
|
runtime_files+=("${file#./}")
|
||||||
|
done < <(find docker -type f -print0)
|
||||||
|
fi
|
||||||
|
|
||||||
runtime_psd_matches=$(
|
for file in README.md .env.example docs/installazione-docker-4-contesti.md; do
|
||||||
rg -n -i \
|
add_file install_files "$file"
|
||||||
-g '!deploy/workspaces/**' \
|
done
|
||||||
-g '!docker/session-migrate.sh' \
|
if [[ -d docs/install ]]; then
|
||||||
-g '!docker/cutover-legacy-sessions.sh' \
|
while IFS= read -r -d '' file; do install_files+=("${file#./}"); done < <(
|
||||||
-g '!docker/smoke/**' \
|
find docs/install -type f -print0
|
||||||
'\bpsd\b' \
|
)
|
||||||
.dockerignore compose.yaml docker-compose.dev.yml deploy docker frontend/vite.config.ts \
|
fi
|
||||||
.env.example scripts/run-stack.sh scripts/docker-smoke.sh || true
|
|
||||||
)
|
if [[ -d scripts ]]; then
|
||||||
|
while IFS= read -r -d '' file; do
|
||||||
|
case "${file#scripts/}" in
|
||||||
|
test-no-deployment-coupling.sh|test-no-deployment-coupling-scope.sh) continue ;;
|
||||||
|
test-*.sh)
|
||||||
|
contract_test_files+=("${file#./}")
|
||||||
|
continue
|
||||||
|
;;
|
||||||
|
verify-*.sh) continue ;;
|
||||||
|
esac
|
||||||
|
operator_files+=("${file#./}")
|
||||||
|
done < <(find scripts -maxdepth 1 -type f -print0)
|
||||||
|
fi
|
||||||
|
|
||||||
offenders=()
|
offenders=()
|
||||||
for superseded_file in \
|
scan_category() {
|
||||||
|
local label="$1" pattern="$2"; shift 2
|
||||||
|
local output rg_status
|
||||||
|
(($#)) || return 0
|
||||||
|
set +e
|
||||||
|
output="$(rg -n -i --with-filename -- "$pattern" "$@" 2>&1)"
|
||||||
|
rg_status=$?
|
||||||
|
set -e
|
||||||
|
case "$rg_status" in
|
||||||
|
0)
|
||||||
|
while IFS= read -r match; do offenders+=("$label: $match"); done <<<"$output"
|
||||||
|
;;
|
||||||
|
1) ;;
|
||||||
|
*)
|
||||||
|
echo "coupling scan failed in $label (rg status $rg_status)" >&2
|
||||||
|
printf '%s\n' "$output" >&2
|
||||||
|
exit "$rg_status"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
for forbidden_file in \
|
||||||
deploy/compose.production.yaml \
|
deploy/compose.production.yaml \
|
||||||
deploy/compose.psd-local.yaml.example \
|
deploy/compose.psd-local.yaml.example \
|
||||||
deploy/compose.psd-local.yaml \
|
deploy/compose.psd-local.yaml \
|
||||||
scripts/bootstrap-local-psd-docker-config.sh \
|
scripts/bootstrap-local-psd-docker-config.sh \
|
||||||
harness/tests/test_psd_local_compose_contract.py
|
scripts/test-qwen-network-config.sh \
|
||||||
do
|
harness/tests/test_psd_local_compose_contract.py; do
|
||||||
[[ ! -e "$superseded_file" ]] || offenders+=("$superseded_file (forbidden active deployment filename)")
|
[[ ! -e "$forbidden_file" ]] \
|
||||||
|
|| offenders+=("active filename: $forbidden_file (superseded deployment contract)")
|
||||||
done
|
done
|
||||||
|
|
||||||
if [[ -n "$matches" ]]; then
|
forbidden='omics_portal|chirone|localllm_default|datamart-builder|compose\.production\.yaml|compose\.psd-local\.yaml|\bpsd\b'
|
||||||
while IFS= read -r match; do
|
scan_category runtime "$forbidden" "${runtime_files[@]}"
|
||||||
offenders+=("$match")
|
scan_category install "$forbidden" "${install_files[@]}"
|
||||||
done <<<"$matches"
|
scan_category operator "$forbidden" "${operator_files[@]}"
|
||||||
fi
|
# Contract tests legitimately quote forbidden names in negative assertions. Scan their positive
|
||||||
|
# deployment wiring constructs instead, so a provider-owned network or retired overlay cannot be
|
||||||
|
# required under a different test filename.
|
||||||
|
positive_contract='networks(\?|\.)?\.?localllm_default|services(\?|\.)?\.?core(\?|\.)?\.?networks(\?|\.)?\.?localllm_default|docker compose[^\n]*(compose\.psd-local|compose\.production)|THT_PSD_[A-Z0-9_]*='
|
||||||
|
scan_category contract-test "$positive_contract" "${contract_test_files[@]}"
|
||||||
|
|
||||||
if [[ -n "$runtime_psd_matches" ]]; then
|
if [[ -f scripts/run-stack.sh ]]; then
|
||||||
while IFS= read -r match; do
|
set +e
|
||||||
offenders+=("$match")
|
host_pi="$(rg -n 'command -v pi|PI_BIN="pi"|PI_BIN=pi' scripts/run-stack.sh 2>&1)"
|
||||||
done <<<"$runtime_psd_matches"
|
host_pi_status=$?
|
||||||
fi
|
set -e
|
||||||
|
case "$host_pi_status" in
|
||||||
if rg -n 'command -v pi|PI_BIN="pi"|PI_BIN=pi' scripts/run-stack.sh >/dev/null; then
|
0) offenders+=("operator: $host_pi") ;;
|
||||||
offenders+=("scripts/run-stack.sh (requires a host Pi binary)")
|
1) ;;
|
||||||
|
*)
|
||||||
|
echo "coupling scan failed in host-Pi contract (rg status $host_pi_status)" >&2
|
||||||
|
printf '%s\n' "$host_pi" >&2
|
||||||
|
exit "$host_pi_status"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if ((${#offenders[@]})); then
|
if ((${#offenders[@]})); then
|
||||||
|
|||||||
@@ -8,13 +8,42 @@ trap 'rm -rf "$tmp"' EXIT HUP INT TERM
|
|||||||
auth_file="$tmp/auth.json"
|
auth_file="$tmp/auth.json"
|
||||||
printf '%s\n' '{}' >"$auth_file"
|
printf '%s\n' '{}' >"$auth_file"
|
||||||
chmod 0600 "$auth_file"
|
chmod 0600 "$auth_file"
|
||||||
|
secrets_file="$tmp/thothii.secrets"
|
||||||
|
printf '%s\n' 'THT_MODEL_API_KEY=fixture-model-api-key' >"$secrets_file"
|
||||||
|
chmod 0600 "$secrets_file"
|
||||||
|
|
||||||
rendered=$(THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
|
rendered=$(THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
|
||||||
PI_AUTH_FILE="$auth_file" docker compose config)
|
PI_AUTH_FILE="$auth_file" THT_SECRETS_FILE="$secrets_file" docker compose config)
|
||||||
printf '%s\n' "$rendered" | grep -q "source: $auth_file"
|
printf '%s\n' "$rendered" | grep -q "source: $auth_file"
|
||||||
printf '%s\n' "$rendered" | grep -q 'target: /home/thoth/.pi/agent/auth.json'
|
printf '%s\n' "$rendered" | grep -q 'target: /home/thoth/.pi/agent/auth.json'
|
||||||
printf '%s\n' "$rendered" | grep -A4 'target: /home/thoth/.pi/agent/auth.json' \
|
printf '%s\n' "$rendered" | grep -A4 'target: /home/thoth/.pi/agent/auth.json' \
|
||||||
| grep -q 'read_only: true'
|
| grep -q 'read_only: true'
|
||||||
|
for target in \
|
||||||
|
/home/thoth/.pi/agent/models.json \
|
||||||
|
/home/thoth/.pi/agent/settings.json; do
|
||||||
|
printf '%s\n' "$rendered" | grep -q "target: $target"
|
||||||
|
printf '%s\n' "$rendered" | grep -A4 "target: $target" | grep -q 'read_only: true'
|
||||||
|
done
|
||||||
|
printf '%s\n' "$rendered" | grep -q "file: $secrets_file"
|
||||||
|
printf '%s\n' "$rendered" | grep -q 'target: thothii.secrets'
|
||||||
|
if grep -Fq 'fixture-model-api-key' <<<"$rendered"; then
|
||||||
|
echo "rendered base Compose leaked the model key" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
dev_rendered=$(THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
|
||||||
|
PI_AUTH_FILE="$auth_file" THT_SECRETS_FILE="$secrets_file" \
|
||||||
|
docker compose --env-file deploy/env/local.env.example -f docker-compose.dev.yml config)
|
||||||
|
printf '%s\n' "$dev_rendered" | grep -q "source: $auth_file"
|
||||||
|
printf '%s\n' "$dev_rendered" | grep -q 'target: /home/thoth/.pi/agent/auth.json'
|
||||||
|
printf '%s\n' "$dev_rendered" | grep -q 'target: /home/thoth/.pi/agent/models.json'
|
||||||
|
printf '%s\n' "$dev_rendered" | grep -q 'target: /home/thoth/.pi/agent/settings.json'
|
||||||
|
printf '%s\n' "$dev_rendered" | grep -q "file: $secrets_file"
|
||||||
|
printf '%s\n' "$dev_rendered" | grep -q 'target: thothii.secrets'
|
||||||
|
if grep -Fq 'fixture-model-api-key' <<<"$dev_rendered"; then
|
||||||
|
echo "rendered development Compose leaked the model key" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
python3 - <<'PY'
|
python3 - <<'PY'
|
||||||
import json
|
import json
|
||||||
@@ -32,5 +61,7 @@ PY
|
|||||||
grep -q '^ARG PI_VERSION=0.80.3$' docker/core.Dockerfile
|
grep -q '^ARG PI_VERSION=0.80.3$' docker/core.Dockerfile
|
||||||
grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/local.env.example
|
grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/local.env.example
|
||||||
grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/server.env.example
|
grep -q '^PI_AUTH_FILE=/absolute/path/to/pi-auth.json$' deploy/env/server.env.example
|
||||||
|
grep -q '^THT_SECRETS_FILE=/absolute/path/to/thothii.secrets$' deploy/env/local.env.example
|
||||||
|
grep -q '^THT_SECRETS_FILE=/absolute/path/to/thothii.secrets$' deploy/env/server.env.example
|
||||||
|
|
||||||
echo "Pi user-auth Compose contract passed."
|
echo "Pi user-auth Compose contract passed."
|
||||||
|
|||||||
@@ -4,7 +4,8 @@ set -eu
|
|||||||
cd "$(dirname "$0")/.."
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
tmp_bundle=$(mktemp)
|
tmp_bundle=$(mktemp)
|
||||||
trap 'rm -f "$tmp_bundle"' EXIT HUP INT TERM
|
tmp_auth=$(mktemp)
|
||||||
|
trap 'rm -f "$tmp_bundle" "$tmp_auth"' EXIT HUP INT TERM
|
||||||
cat >"$tmp_bundle" <<'EOF'
|
cat >"$tmp_bundle" <<'EOF'
|
||||||
THT_VECTOR_BOOTSTRAP_PASSWORD=test-bootstrap
|
THT_VECTOR_BOOTSTRAP_PASSWORD=test-bootstrap
|
||||||
THT_VECTOR_MIGRATOR_PASSWORD=test-migrator
|
THT_VECTOR_MIGRATOR_PASSWORD=test-migrator
|
||||||
@@ -12,7 +13,11 @@ THT_VECTOR_READER_PASSWORD=test-reader
|
|||||||
THT_VECTOR_WRITER_PASSWORD=test-writer
|
THT_VECTOR_WRITER_PASSWORD=test-writer
|
||||||
EOF
|
EOF
|
||||||
chmod 0600 "$tmp_bundle"
|
chmod 0600 "$tmp_bundle"
|
||||||
|
printf '%s\n' '{}' >"$tmp_auth"
|
||||||
|
chmod 0600 "$tmp_auth"
|
||||||
export THT_SECRETS_FILE="$tmp_bundle"
|
export THT_SECRETS_FILE="$tmp_bundle"
|
||||||
|
export PI_AUTH_FILE="$tmp_auth"
|
||||||
|
export THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git
|
||||||
|
|
||||||
local_files="-f compose.yaml -f deploy/compose.local-vector.yaml -f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml"
|
local_files="-f compose.yaml -f deploy/compose.local-vector.yaml -f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml"
|
||||||
local_json=$(docker compose $local_files --profile local-vector --profile preprocess config --format json)
|
local_json=$(docker compose $local_files --profile local-vector --profile preprocess config --format json)
|
||||||
|
|||||||
@@ -1,24 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
cd "$(dirname "$0")/.."
|
|
||||||
tmp=$(mktemp -d)
|
|
||||||
trap 'rm -rf "$tmp"' EXIT HUP INT TERM
|
|
||||||
mkdir -p "$tmp/deploy"
|
|
||||||
cp compose.yaml "$tmp/compose.yaml"
|
|
||||||
: >"$tmp/deploy/thothii.env"
|
|
||||||
|
|
||||||
docker compose --project-directory "$tmp" -f "$tmp/compose.yaml" config --format json >"$tmp/config.json"
|
|
||||||
node - "$tmp/config.json" <<'NODE'
|
|
||||||
const fs = require("fs");
|
|
||||||
const config = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
|
|
||||||
if (!config.networks?.localllm_default?.external) {
|
|
||||||
throw new Error("localllm_default must be an external network");
|
|
||||||
}
|
|
||||||
if (!config.services?.core?.networks?.localllm_default) {
|
|
||||||
throw new Error("core must join localllm_default");
|
|
||||||
}
|
|
||||||
if (config.services?.frontend?.networks?.localllm_default) {
|
|
||||||
throw new Error("frontend must not join the model network");
|
|
||||||
}
|
|
||||||
NODE
|
|
||||||
@@ -11,8 +11,12 @@ render_profile() {
|
|||||||
local env_file=$2
|
local env_file=$2
|
||||||
local compose_file=$3
|
local compose_file=$3
|
||||||
local rendered="$tmp/$profile.json"
|
local rendered="$tmp/$profile.json"
|
||||||
|
local -a files=(-f compose.yaml -f "$compose_file")
|
||||||
|
if [[ "$profile" == server ]]; then
|
||||||
|
files+=(-f deploy/compose.session-server.yaml.example)
|
||||||
|
fi
|
||||||
|
|
||||||
docker compose --env-file "$env_file" -f compose.yaml -f "$compose_file" \
|
docker compose --env-file "$env_file" "${files[@]}" \
|
||||||
config --format json >"$rendered"
|
config --format json >"$rendered"
|
||||||
|
|
||||||
node - "$rendered" "$profile" <<'NODE'
|
node - "$rendered" "$profile" <<'NODE'
|
||||||
@@ -38,6 +42,28 @@ const piAuthMounts = (config.services.core.volumes || []).filter(
|
|||||||
if (piAuthMounts.length !== 1 || piAuthMounts[0].type !== "bind" || !piAuthMounts[0].read_only) {
|
if (piAuthMounts.length !== 1 || piAuthMounts[0].type !== "bind" || !piAuthMounts[0].read_only) {
|
||||||
throw new Error("Pi auth must be one read-only file bind");
|
throw new Error("Pi auth must be one read-only file bind");
|
||||||
}
|
}
|
||||||
|
if (config.services.core.environment?.THT_SECRETS_FILE !== "/run/secrets/thothii.secrets") {
|
||||||
|
throw new Error("core must read the canonical runtime secret bundle from /run/secrets");
|
||||||
|
}
|
||||||
|
const runtimeSecrets = config.services.core.secrets || [];
|
||||||
|
const bundleSecrets = runtimeSecrets.filter(
|
||||||
|
(secret) => secret.source === "thothii_secrets" && secret.target === "thothii.secrets",
|
||||||
|
);
|
||||||
|
if (bundleSecrets.length !== 1) {
|
||||||
|
throw new Error("core must receive exactly one canonical runtime secret bundle");
|
||||||
|
}
|
||||||
|
if (profile === "local" && runtimeSecrets.length !== 1) {
|
||||||
|
throw new Error("local core must receive only the canonical runtime secret bundle");
|
||||||
|
}
|
||||||
|
if (profile === "server") {
|
||||||
|
const targets = new Set(runtimeSecrets.map((secret) => secret.target));
|
||||||
|
for (const target of ["session_runtime_password", "session_ca.pem"]) {
|
||||||
|
if (!targets.has(target)) throw new Error("server core lacks " + target);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ((config.services.frontend.secrets || []).length !== 0) {
|
||||||
|
throw new Error("frontend must not receive runtime secrets");
|
||||||
|
}
|
||||||
|
|
||||||
const ports = Object.fromEntries(
|
const ports = Object.fromEntries(
|
||||||
Object.entries(config.services).map(([name, service]) => [name, service.ports || []]),
|
Object.entries(config.services).map(([name, service]) => [name, service.ports || []]),
|
||||||
@@ -60,9 +86,12 @@ assert_remote_required() {
|
|||||||
local env_file=$1
|
local env_file=$1
|
||||||
local compose_file=$2
|
local compose_file=$2
|
||||||
local without_remote="$tmp/without-remote.env"
|
local without_remote="$tmp/without-remote.env"
|
||||||
|
local -a files=(-f compose.yaml -f "$compose_file")
|
||||||
|
[[ "$compose_file" != deploy/compose.server.yaml ]] \
|
||||||
|
|| files+=(-f deploy/compose.session-server.yaml.example)
|
||||||
grep -v '^THT_WORKSPACE_GIT_REMOTE=' "$env_file" >"$without_remote"
|
grep -v '^THT_WORKSPACE_GIT_REMOTE=' "$env_file" >"$without_remote"
|
||||||
|
|
||||||
if env -u THT_WORKSPACE_GIT_REMOTE docker compose --env-file "$without_remote" -f compose.yaml -f "$compose_file" \
|
if env -u THT_WORKSPACE_GIT_REMOTE docker compose --env-file "$without_remote" "${files[@]}" \
|
||||||
config --format json >"$tmp/missing-remote.out" 2>"$tmp/missing-remote.err"; then
|
config --format json >"$tmp/missing-remote.out" 2>"$tmp/missing-remote.err"; then
|
||||||
echo "Compose must require THT_WORKSPACE_GIT_REMOTE" >&2
|
echo "Compose must require THT_WORKSPACE_GIT_REMOTE" >&2
|
||||||
exit 1
|
exit 1
|
||||||
@@ -72,6 +101,7 @@ assert_remote_required() {
|
|||||||
|
|
||||||
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
|
THT_WORKSPACE_GIT_REMOTE=https://git.example.invalid/platform/thoth-workspaces.git \
|
||||||
PI_AUTH_FILE=/dev/null \
|
PI_AUTH_FILE=/dev/null \
|
||||||
|
THT_SECRETS_FILE=/dev/null \
|
||||||
docker compose -f compose.yaml config --format json >"$tmp/base.json"
|
docker compose -f compose.yaml config --format json >"$tmp/base.json"
|
||||||
node - "$tmp/base.json" <<'NODE'
|
node - "$tmp/base.json" <<'NODE'
|
||||||
const fs = require("fs");
|
const fs = require("fs");
|
||||||
@@ -98,6 +128,18 @@ const piAuthMounts = (config.services.core.volumes || []).filter(
|
|||||||
if (piAuthMounts.length !== 1 || piAuthMounts[0].type !== "bind" || !piAuthMounts[0].read_only) {
|
if (piAuthMounts.length !== 1 || piAuthMounts[0].type !== "bind" || !piAuthMounts[0].read_only) {
|
||||||
throw new Error("Pi auth must be one read-only file bind");
|
throw new Error("Pi auth must be one read-only file bind");
|
||||||
}
|
}
|
||||||
|
if (config.services.core.environment?.THT_SECRETS_FILE !== "/run/secrets/thothii.secrets") {
|
||||||
|
throw new Error("core must read the canonical runtime secret bundle from /run/secrets");
|
||||||
|
}
|
||||||
|
const runtimeSecrets = config.services.core.secrets || [];
|
||||||
|
if (runtimeSecrets.length !== 1
|
||||||
|
|| runtimeSecrets[0].source !== "thothii_secrets"
|
||||||
|
|| runtimeSecrets[0].target !== "thothii.secrets") {
|
||||||
|
throw new Error("core must receive exactly the canonical runtime secret bundle");
|
||||||
|
}
|
||||||
|
if ((config.services.frontend.secrets || []).length !== 0) {
|
||||||
|
throw new Error("frontend must not receive runtime secrets");
|
||||||
|
}
|
||||||
NODE
|
NODE
|
||||||
|
|
||||||
render_profile local deploy/env/local.env.example deploy/compose.local.yaml
|
render_profile local deploy/env/local.env.example deploy/compose.local.yaml
|
||||||
|
|||||||
@@ -9,20 +9,11 @@ trap 'rm -f "$output"' EXIT HUP INT TERM
|
|||||||
"$root/scripts/verify-workspace-install-docs.sh" --fixtures-only >"$output"
|
"$root/scripts/verify-workspace-install-docs.sh" --fixtures-only >"$output"
|
||||||
|
|
||||||
for fixture in \
|
for fixture in \
|
||||||
"local manual requires generated connector override and Compose preflight" \
|
"local manual canonical base+override references" \
|
||||||
"server manual requires generated connector override and Compose preflight" \
|
"server manual canonical base+override references" \
|
||||||
"local documented shell environment fixture" \
|
"canonical local base+override fixture" \
|
||||||
"server documented shell environment fixture" \
|
"canonical server base+override fixture" \
|
||||||
"copied local base fixture" \
|
"relative secret-source fixture rejected"; do
|
||||||
"copied server PostgreSQL/TLS fixture" \
|
|
||||||
"copied HTTPS Git override fixture" \
|
|
||||||
"copied SSH Git override fixture" \
|
|
||||||
"copied connector binding/secret fixture" \
|
|
||||||
"core process sees connector bindings and secret files" \
|
|
||||||
"non-path secret-file fixture rejected" \
|
|
||||||
"literal secret-source fixture rejected" \
|
|
||||||
"relative secret-source fixture rejected" \
|
|
||||||
"non-normalized secret-source fixture rejected"; do
|
|
||||||
grep -Fqx "$fixture passed" "$output" >/dev/null || {
|
grep -Fqx "$fixture passed" "$output" >/dev/null || {
|
||||||
echo "missing fixture verification: $fixture" >&2
|
echo "missing fixture verification: $fixture" >&2
|
||||||
cat "$output" >&2
|
cat "$output" >&2
|
||||||
@@ -37,7 +28,7 @@ for manual in \
|
|||||||
echo "installation manual does not publish a self-contained THT_SOURCE_ROOT export: $manual" >&2
|
echo "installation manual does not publish a self-contained THT_SOURCE_ROOT export: $manual" >&2
|
||||||
exit 1
|
exit 1
|
||||||
}
|
}
|
||||||
grep -Fq 'export THT_WORKSPACE_BINDINGS_ENV_FILE="$(pwd -P)/workspace-bindings.env"' "$manual" || {
|
grep -Fq 'export THT_WORKSPACE_BINDINGS_ENV_FILE=' "$manual" || {
|
||||||
echo "installation manual does not publish a self-contained bindings export: $manual" >&2
|
echo "installation manual does not publish a self-contained bindings export: $manual" >&2
|
||||||
exit 1
|
exit 1
|
||||||
}
|
}
|
||||||
@@ -47,7 +38,7 @@ for manual in \
|
|||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
if rg -n 'connector-secrets\.workspace-registry|docker compose' \
|
if rg -n 'local-compose\.workspace-registry|server-compose\.workspace-registry|connector-secrets\.workspace-registry|docker compose' \
|
||||||
"$root/docs/install/local-workspace-registry.md" \
|
"$root/docs/install/local-workspace-registry.md" \
|
||||||
"$root/docs/install/server-workspace-registry.md"; then
|
"$root/docs/install/server-workspace-registry.md"; then
|
||||||
echo "installation manuals still document a bypassed Compose or copied connector override path" >&2
|
echo "installation manuals still document a bypassed Compose or copied connector override path" >&2
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ trap 'rm -f "$replacement"' EXIT HUP INT TERM
|
|||||||
cp "$new_secret" "$replacement"
|
cp "$new_secret" "$replacement"
|
||||||
chmod 0600 "$replacement"
|
chmod 0600 "$replacement"
|
||||||
|
|
||||||
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
|
docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local-vector.yaml \
|
||||||
--project-name "$project" --profile local-vector run --rm --no-deps \
|
--project-name "$project" --profile local-vector run --rm --no-deps \
|
||||||
--user 0:0 \
|
--user 0:0 \
|
||||||
--entrypoint /opt/venv/bin/python \
|
--entrypoint /opt/venv/bin/python \
|
||||||
@@ -43,4 +43,4 @@ mv -f "$replacement" "$old_secret"
|
|||||||
trap - EXIT HUP INT TERM
|
trap - EXIT HUP INT TERM
|
||||||
|
|
||||||
echo "Deployment bootstrap secret atomically replaced only after verified database login."
|
echo "Deployment bootstrap secret atomically replaced only after verified database login."
|
||||||
echo "Re-run: docker compose -f compose.yaml -f deploy/compose.local-vector.yaml --project-name $project --profile local-vector up --wait vector-reconcile vector-migrate core"
|
echo "Re-run: docker compose --env-file deploy/env/local.env -f compose.yaml -f deploy/compose.local-vector.yaml --project-name $project --profile local-vector up --wait vector-reconcile vector-migrate core"
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Validate the installation manuals without reading an operator environment or production remote.
|
# Verify canonical local/server installation manuals and their base+override Compose paths.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||||
profile="${1:-}"
|
mode="${1:-}"
|
||||||
|
|
||||||
trim() {
|
trim() {
|
||||||
local value="$1"
|
local value="$1"
|
||||||
@@ -41,266 +41,13 @@ verify_path_variable_values() {
|
|||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
done <"$source"
|
done <"$source"
|
||||||
return 0
|
|
||||||
}
|
}
|
||||||
|
|
||||||
verify_server_public_contract() {
|
verify_manual() {
|
||||||
local server_example="$root/docs/install/examples/server-compose.workspace-registry.yaml"
|
local profile="$1" manual
|
||||||
for expected in \
|
manual="$root/docs/install/$profile-workspace-registry.md"
|
||||||
'THT_SESSION_STORAGE: postgres' \
|
local -a headings
|
||||||
'THT_SESSION_RUNTIME_PASSWORD_FILE: /run/secrets/session_runtime_password' \
|
if [[ "$profile" == local ]]; then
|
||||||
'THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full}' \
|
|
||||||
'THT_SESSION_DB_SSLROOTCERT: /run/secrets/session_ca.pem' \
|
|
||||||
'session_runtime_password:' \
|
|
||||||
'session_ca:'; do
|
|
||||||
grep -Fq "$expected" "$server_example" || {
|
|
||||||
echo "server Compose example lacks required public PostgreSQL/TLS contract: $expected" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
verify_manual_supported_path() {
|
|
||||||
local profile="$1" manual="$2"
|
|
||||||
local source_root_export='export THT_SOURCE_ROOT=/absolute/path/to/ThothII'
|
|
||||||
local bindings_export='export THT_WORKSPACE_BINDINGS_ENV_FILE="$(pwd -P)/workspace-bindings.env"'
|
|
||||||
local generator='"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" --bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env .env --output connector-secrets.local.yaml'
|
|
||||||
local wrapper='"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env'
|
|
||||||
|
|
||||||
grep -Fq "$source_root_export" "$manual" || {
|
|
||||||
echo "$profile manual does not export THT_SOURCE_ROOT for its shell commands" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
grep -Fq "$bindings_export" "$manual" || {
|
|
||||||
echo "$profile manual does not export THT_WORKSPACE_BINDINGS_ENV_FILE for its shell commands" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
grep -Fq "$generator" "$manual" || {
|
|
||||||
echo "$profile manual does not document the connector override generator" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
grep -Fq "$wrapper" "$manual" || {
|
|
||||||
echo "$profile manual does not document the Compose preflight wrapper" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
if grep -Eq 'connector-secrets\.workspace-registry|docker compose' "$manual"; then
|
|
||||||
echo "$profile manual documents a bypassed Compose or copied connector override path" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "$profile manual requires generated connector override and Compose preflight passed"
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_fixture() {
|
|
||||||
local name="$1" directory="$2"; shift 2
|
|
||||||
(
|
|
||||||
cd "$directory"
|
|
||||||
"$root/scripts/compose-with-preflight.sh" --env-file .env "$@" config --quiet
|
|
||||||
)
|
|
||||||
echo "$name passed"
|
|
||||||
}
|
|
||||||
|
|
||||||
prepare_binding_fixture() {
|
|
||||||
local directory="$1"
|
|
||||||
printf '%s\n' \
|
|
||||||
'THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=postgres_direct' \
|
|
||||||
'THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password' \
|
|
||||||
'THT_WS_NORTH_STAR_RESEARCH_VECTOR_TRANSPORT=rest_api' \
|
|
||||||
'THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE=/run/secrets/north-star-research-vector-api-key' \
|
|
||||||
>"$directory/workspace-bindings.env"
|
|
||||||
printf 'THT_WORKSPACE_BINDINGS_ENV_FILE=%s\n' "$directory/workspace-bindings.env" >>"$directory/.env"
|
|
||||||
}
|
|
||||||
|
|
||||||
verify_connector_fixture() {
|
|
||||||
local directory="$1" rendered project connector_override
|
|
||||||
project="thoth-install-connector-fixture-$$"
|
|
||||||
connector_override="$directory/connector-secrets.local.yaml"
|
|
||||||
"$root/scripts/generate-connector-secrets-override.sh" \
|
|
||||||
--bindings-env "$directory/workspace-bindings.env" --operator-env "$directory/.env" \
|
|
||||||
--output "$connector_override" >/dev/null
|
|
||||||
rendered="$(
|
|
||||||
cd "$directory"
|
|
||||||
"$root/scripts/compose-with-preflight.sh" --env-file .env \
|
|
||||||
-f compose.workspace-registry.yaml -f connector-secrets.local.yaml config
|
|
||||||
)"
|
|
||||||
for expected in \
|
|
||||||
'THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT: postgres_direct' \
|
|
||||||
'THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE: /run/secrets/north-star-research-dwh-password' \
|
|
||||||
'THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE: /run/secrets/north-star-research-vector-api-key' \
|
|
||||||
'target: north-star-research-dwh-password' \
|
|
||||||
'target: north-star-research-vector-api-key'; do
|
|
||||||
grep -Fq "$expected" <<<"$rendered" || {
|
|
||||||
echo "connector fixture does not give core required binding or secret target: $expected" >&2
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
done
|
|
||||||
echo "copied connector binding/secret fixture passed"
|
|
||||||
if ! (
|
|
||||||
cd "$directory"
|
|
||||||
"$root/scripts/compose-with-preflight.sh" --project-name "$project" --env-file .env \
|
|
||||||
-f compose.workspace-registry.yaml -f connector-secrets.local.yaml run --rm --no-deps --build --entrypoint sh core -c '
|
|
||||||
test "$THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT" = postgres_direct
|
|
||||||
test "$THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE" = /run/secrets/north-star-research-dwh-password
|
|
||||||
test "$THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE" = /run/secrets/north-star-research-vector-api-key
|
|
||||||
test -f "$THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE"
|
|
||||||
test -f "$THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE"
|
|
||||||
'
|
|
||||||
); then
|
|
||||||
(
|
|
||||||
cd "$directory"
|
|
||||||
"$root/scripts/compose-with-preflight.sh" --project-name "$project" --env-file .env \
|
|
||||||
-f compose.workspace-registry.yaml -f connector-secrets.local.yaml down --volumes --remove-orphans
|
|
||||||
) || true
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
(
|
|
||||||
cd "$directory"
|
|
||||||
"$root/scripts/compose-with-preflight.sh" --project-name "$project" --env-file .env \
|
|
||||||
-f compose.workspace-registry.yaml -f connector-secrets.local.yaml down --volumes --remove-orphans
|
|
||||||
)
|
|
||||||
echo "core process sees connector bindings and secret files passed"
|
|
||||||
}
|
|
||||||
|
|
||||||
verify_documented_operator_path() {
|
|
||||||
local profile="$1" directory="$2" documented_source_root="$3" connector_override
|
|
||||||
connector_override="$directory/connector-secrets.local.yaml"
|
|
||||||
(
|
|
||||||
cd "$directory"
|
|
||||||
unset THT_SOURCE_ROOT THT_WORKSPACE_BINDINGS_ENV_FILE
|
|
||||||
export THT_SOURCE_ROOT="$documented_source_root"
|
|
||||||
export THT_WORKSPACE_BINDINGS_ENV_FILE="$(pwd -P)/workspace-bindings.env"
|
|
||||||
"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" \
|
|
||||||
--bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env .env \
|
|
||||||
--output connector-secrets.local.yaml >/dev/null
|
|
||||||
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
|
|
||||||
-f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml \
|
|
||||||
-f connector-secrets.local.yaml config --quiet
|
|
||||||
)
|
|
||||||
echo "$profile documented shell environment fixture passed"
|
|
||||||
}
|
|
||||||
|
|
||||||
verify_copied_operator_fixtures() {
|
|
||||||
local fixture_root local_dir server_dir https_dir ssh_dir connector_dir
|
|
||||||
fixture_root="$(mktemp -d "${TMPDIR%/}/thoth-install-fixtures.XXXXXX")"
|
|
||||||
trap 'rm -rf "$fixture_root"' RETURN
|
|
||||||
local_dir="$fixture_root/local"; server_dir="$fixture_root/server"
|
|
||||||
https_dir="$fixture_root/https"; ssh_dir="$fixture_root/ssh"; connector_dir="$fixture_root/connector"
|
|
||||||
mkdir -p "$local_dir" "$server_dir" "$https_dir" "$ssh_dir" "$connector_dir"
|
|
||||||
|
|
||||||
cp "$root/docs/install/examples/local-compose.workspace-registry.yaml" "$local_dir/compose.workspace-registry.yaml"
|
|
||||||
printf 'THT_SOURCE_ROOT=%s\n' "$root" >"$local_dir/.env"
|
|
||||||
prepare_binding_fixture "$local_dir"
|
|
||||||
compose_fixture "copied local base fixture" "$local_dir" -f compose.workspace-registry.yaml
|
|
||||||
|
|
||||||
cp "$root/docs/install/examples/server-compose.workspace-registry.yaml" "$server_dir/compose.workspace-registry.yaml"
|
|
||||||
cp "$root/deploy/workspaces/server-sessions.yaml.example" "$server_dir/server-sessions.yaml"
|
|
||||||
: >"$server_dir/session-runtime-password"; : >"$server_dir/session-ca.pem"
|
|
||||||
printf '%s\n' \
|
|
||||||
"THT_SOURCE_ROOT=$root" \
|
|
||||||
"THT_SERVER_WORKSPACE_CONFIG=$server_dir/server-sessions.yaml" \
|
|
||||||
'THT_SESSION_DB_HOST=sessions.example.invalid' \
|
|
||||||
'THT_SESSION_DB_NAME=thoth_sessions' \
|
|
||||||
'THT_SESSION_RUNTIME_USER=thoth_sessions_app' \
|
|
||||||
"THT_SESSION_RUNTIME_PASSWORD_SOURCE=$server_dir/session-runtime-password" \
|
|
||||||
"THT_SESSION_CA_SOURCE=$server_dir/session-ca.pem" >"$server_dir/.env"
|
|
||||||
prepare_binding_fixture "$server_dir"
|
|
||||||
compose_fixture "copied server PostgreSQL/TLS fixture" "$server_dir" -f compose.workspace-registry.yaml
|
|
||||||
|
|
||||||
cp "$root/docs/install/examples/local-compose.workspace-registry.yaml" "$https_dir/compose.workspace-registry.yaml"
|
|
||||||
cp "$root/docs/install/examples/git-https.workspace-registry.yaml" "$https_dir/git-https.yaml"
|
|
||||||
: >"$https_dir/git-credentials"; : >"$https_dir/git-ca.pem"
|
|
||||||
printf '%s\n' \
|
|
||||||
"THT_SOURCE_ROOT=$root" \
|
|
||||||
"THT_WORKSPACE_GIT_CREDENTIALS_FILE=$https_dir/git-credentials" \
|
|
||||||
"THT_WORKSPACE_GIT_CA_FILE=$https_dir/git-ca.pem" >"$https_dir/.env"
|
|
||||||
prepare_binding_fixture "$https_dir"
|
|
||||||
compose_fixture "copied HTTPS Git override fixture" "$https_dir" -f compose.workspace-registry.yaml -f git-https.yaml
|
|
||||||
|
|
||||||
cp "$root/docs/install/examples/local-compose.workspace-registry.yaml" "$ssh_dir/compose.workspace-registry.yaml"
|
|
||||||
cp "$root/docs/install/examples/git-ssh.workspace-registry.yaml" "$ssh_dir/git-ssh.yaml"
|
|
||||||
: >"$ssh_dir/git-ssh-key"; : >"$ssh_dir/git-known-hosts"
|
|
||||||
printf '%s\n' \
|
|
||||||
"THT_SOURCE_ROOT=$root" \
|
|
||||||
"THT_WORKSPACE_GIT_SSH_KEY_FILE=$ssh_dir/git-ssh-key" \
|
|
||||||
"THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=$ssh_dir/git-known-hosts" >"$ssh_dir/.env"
|
|
||||||
prepare_binding_fixture "$ssh_dir"
|
|
||||||
compose_fixture "copied SSH Git override fixture" "$ssh_dir" -f compose.workspace-registry.yaml -f git-ssh.yaml
|
|
||||||
|
|
||||||
: >"$server_dir/git-ssh-key"; : >"$server_dir/git-known-hosts"
|
|
||||||
: >"$server_dir/dwh-password"; : >"$server_dir/vector-api-key"
|
|
||||||
printf '%s\n' \
|
|
||||||
"THT_WORKSPACE_GIT_SSH_KEY_FILE=$server_dir/git-ssh-key" \
|
|
||||||
"THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=$server_dir/git-known-hosts" \
|
|
||||||
"THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_SOURCE=$server_dir/dwh-password" \
|
|
||||||
"THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_SOURCE=$server_dir/vector-api-key" >>"$server_dir/.env"
|
|
||||||
cp "$root/docs/install/examples/git-ssh.workspace-registry.yaml" "$server_dir/git-ssh.workspace-registry.yaml"
|
|
||||||
: >"$ssh_dir/dwh-password"; : >"$ssh_dir/vector-api-key"
|
|
||||||
printf '%s\n' \
|
|
||||||
"THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_SOURCE=$ssh_dir/dwh-password" \
|
|
||||||
"THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_SOURCE=$ssh_dir/vector-api-key" >>"$ssh_dir/.env"
|
|
||||||
cp "$root/docs/install/examples/git-ssh.workspace-registry.yaml" "$ssh_dir/git-ssh.workspace-registry.yaml"
|
|
||||||
verify_documented_operator_path local "$ssh_dir" "$root"
|
|
||||||
verify_documented_operator_path server "$server_dir" "$root"
|
|
||||||
|
|
||||||
cp "$root/docs/install/examples/local-compose.workspace-registry.yaml" "$connector_dir/compose.workspace-registry.yaml"
|
|
||||||
: >"$connector_dir/dwh-password"; : >"$connector_dir/vector-password"
|
|
||||||
printf '%s\n' \
|
|
||||||
"THT_SOURCE_ROOT=$root" \
|
|
||||||
"THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_SOURCE=$connector_dir/dwh-password" \
|
|
||||||
"THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_SOURCE=$connector_dir/vector-password" >"$connector_dir/.env"
|
|
||||||
prepare_binding_fixture "$connector_dir"
|
|
||||||
verify_connector_fixture "$connector_dir"
|
|
||||||
|
|
||||||
printf 'THT_WS_EXAMPLE_DWH_PASSWORD_FILE=not-a-path\n' >"$fixture_root/non-path-secret.env"
|
|
||||||
if verify_path_variable_values "$fixture_root/non-path-secret.env" >/dev/null 2>&1; then
|
|
||||||
echo "non-path secret-file fixture was accepted" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "non-path secret-file fixture rejected passed"
|
|
||||||
|
|
||||||
printf 'THT_WS_EXAMPLE_DWH_PASSWORD_SOURCE=literal-value\n' >"$fixture_root/literal-source.env"
|
|
||||||
if verify_path_variable_values "$fixture_root/literal-source.env" >/dev/null 2>&1; then
|
|
||||||
echo "literal secret-source fixture was accepted" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "literal secret-source fixture rejected passed"
|
|
||||||
|
|
||||||
printf 'THT_WS_EXAMPLE_DWH_PASSWORD_SOURCE=installation-secrets/password\n' >"$fixture_root/relative-source.env"
|
|
||||||
if verify_path_variable_values "$fixture_root/relative-source.env" >/dev/null 2>&1; then
|
|
||||||
echo "relative secret-source fixture was accepted" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "relative secret-source fixture rejected passed"
|
|
||||||
|
|
||||||
printf 'THT_WS_EXAMPLE_DWH_PASSWORD_SOURCE=/srv/thothii/secrets/../password\n' >"$fixture_root/non-normalized-source.env"
|
|
||||||
if verify_path_variable_values "$fixture_root/non-normalized-source.env" >/dev/null 2>&1; then
|
|
||||||
echo "non-normalized secret-source fixture was accepted" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "non-normalized secret-source fixture rejected passed"
|
|
||||||
}
|
|
||||||
|
|
||||||
case "$profile" in
|
|
||||||
--fixtures-only)
|
|
||||||
[[ $# -eq 1 ]] || { echo "usage: $0 --fixtures-only" >&2; exit 2; }
|
|
||||||
verify_manual_supported_path local "$root/docs/install/local-workspace-registry.md"
|
|
||||||
verify_manual_supported_path server "$root/docs/install/server-workspace-registry.md"
|
|
||||||
verify_copied_operator_fixtures
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
--profile)
|
|
||||||
profile="${2:-}"
|
|
||||||
[[ $# -eq 2 ]] || { echo "usage: $0 --profile {local|server}" >&2; exit 2; }
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo "usage: $0 --profile {local|server}" >&2
|
|
||||||
exit 2
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
case "$profile" in
|
|
||||||
local)
|
|
||||||
manual="$root/docs/install/local-workspace-registry.md"
|
|
||||||
example="$root/docs/install/examples/local-compose.workspace-registry.yaml"
|
|
||||||
headings=(
|
headings=(
|
||||||
"Prerequisites"
|
"Prerequisites"
|
||||||
"Git remote: SSH and HTTPS"
|
"Git remote: SSH and HTTPS"
|
||||||
@@ -310,10 +57,7 @@ case "$profile" in
|
|||||||
"Publish, update, backup, outage recovery, and rollback"
|
"Publish, update, backup, outage recovery, and rollback"
|
||||||
"Troubleshooting"
|
"Troubleshooting"
|
||||||
)
|
)
|
||||||
;;
|
else
|
||||||
server)
|
|
||||||
manual="$root/docs/install/server-workspace-registry.md"
|
|
||||||
example="$root/docs/install/examples/server-compose.workspace-registry.yaml"
|
|
||||||
headings=(
|
headings=(
|
||||||
"Service account, storage, and firewall"
|
"Service account, storage, and firewall"
|
||||||
"Gitea and remote Git setup"
|
"Gitea and remote Git setup"
|
||||||
@@ -324,50 +68,186 @@ case "$profile" in
|
|||||||
"Pull, publish, upgrade, backup, and recovery"
|
"Pull, publish, upgrade, backup, and recovery"
|
||||||
"Troubleshooting and snapshot rollback"
|
"Troubleshooting and snapshot rollback"
|
||||||
)
|
)
|
||||||
|
fi
|
||||||
|
for heading in "${headings[@]}"; do
|
||||||
|
grep -Fqx "## $heading" "$manual" || {
|
||||||
|
echo "missing required heading in $profile manual: $heading" >&2
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
done
|
||||||
|
for expected in \
|
||||||
|
'export THT_SOURCE_ROOT=/absolute/path/to/ThothII' \
|
||||||
|
'--env-file "$THT_OPERATOR_ENV"' \
|
||||||
|
"-f \"\$THT_SOURCE_ROOT/compose.yaml\" -f \"\$THT_SOURCE_ROOT/deploy/compose.$profile.yaml\"" \
|
||||||
|
'"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh"'; do
|
||||||
|
grep -Fq -- "$expected" "$manual" || {
|
||||||
|
echo "$profile manual lacks canonical operator step: $expected" >&2
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
done
|
||||||
|
if rg -n 'local-compose\.workspace-registry|server-compose\.workspace-registry|connector-secrets\.workspace-registry|docker compose' "$manual"; then
|
||||||
|
echo "$profile manual documents a superseded or bypassed Compose path" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
verify_path_variable_values "$manual"
|
||||||
|
echo "$profile manual canonical base+override references passed"
|
||||||
|
}
|
||||||
|
|
||||||
|
write_private() {
|
||||||
|
local path="$1" value="$2"
|
||||||
|
printf '%s\n' "$value" >"$path"
|
||||||
|
chmod 0600 "$path"
|
||||||
|
}
|
||||||
|
|
||||||
|
verify_compose_fixtures() {
|
||||||
|
local fixture connector_override profile rendered
|
||||||
|
fixture="$(mktemp -d "${TMPDIR%/}/thoth-install-fixtures.XXXXXX")"
|
||||||
|
trap 'rm -rf "$fixture"' RETURN
|
||||||
|
mkdir -p "$fixture/data" "$fixture/pi-state" "$fixture/workspace-registry"
|
||||||
|
|
||||||
|
write_private "$fixture/pi-auth.json" '{"zai":{"type":"api_key","key":"fixture-native-auth-key"}}'
|
||||||
|
write_private "$fixture/thothii.secrets" 'THT_MODEL_API_KEY=fixture-model-api-key'
|
||||||
|
write_private "$fixture/git-ssh-key" 'fixture-git-ssh-key'
|
||||||
|
write_private "$fixture/git-known-hosts" 'fixture-git-known-hosts'
|
||||||
|
write_private "$fixture/dwh-password" 'fixture-dwh-password'
|
||||||
|
write_private "$fixture/vector-api-key" 'fixture-vector-api-key'
|
||||||
|
write_private "$fixture/session-runtime-password" 'fixture-session-runtime-password'
|
||||||
|
write_private "$fixture/session-migrator-password" 'fixture-session-migrator-password'
|
||||||
|
write_private "$fixture/session-ca.pem" 'fixture-session-ca'
|
||||||
|
cp "$root/deploy/workspaces/server-sessions.yaml.example" "$fixture/server-sessions.yaml"
|
||||||
|
|
||||||
|
printf '%s\n' \
|
||||||
|
'THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=postgres_direct' \
|
||||||
|
'THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password' \
|
||||||
|
'THT_WS_NORTH_STAR_RESEARCH_VECTOR_TRANSPORT=rest_api' \
|
||||||
|
'THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE=/run/secrets/north-star-research-vector-api-key' \
|
||||||
|
>"$fixture/workspace-bindings.env"
|
||||||
|
|
||||||
|
printf '%s\n' \
|
||||||
|
'THT_WORKSPACE_GIT_REMOTE=ssh://git@git.example.invalid/platform/thoth-workspaces.git' \
|
||||||
|
"PI_AUTH_FILE=$fixture/pi-auth.json" \
|
||||||
|
"THT_SECRETS_FILE=$fixture/thothii.secrets" \
|
||||||
|
"THT_WORKSPACE_BINDINGS_ENV_FILE=$fixture/workspace-bindings.env" \
|
||||||
|
"THT_WORKSPACE_GIT_SSH_KEY_FILE=$fixture/git-ssh-key" \
|
||||||
|
"THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=$fixture/git-known-hosts" \
|
||||||
|
"THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_SOURCE=$fixture/dwh-password" \
|
||||||
|
"THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_SOURCE=$fixture/vector-api-key" \
|
||||||
|
"THT_DATA_ROOT=$fixture/data" \
|
||||||
|
"THT_PI_STATE_ROOT=$fixture/pi-state" \
|
||||||
|
"THT_WORKSPACE_REGISTRY_ROOT=$fixture/workspace-registry" \
|
||||||
|
"THT_SERVER_WORKSPACE_CONFIG=$fixture/server-sessions.yaml" \
|
||||||
|
'THT_SESSION_DB_HOST=sessions.example.invalid' \
|
||||||
|
'THT_SESSION_DB_NAME=thoth_sessions' \
|
||||||
|
'THT_SESSION_RUNTIME_USER=thoth_sessions_app' \
|
||||||
|
'THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate' \
|
||||||
|
"THT_SESSION_RUNTIME_PASSWORD_SOURCE=$fixture/session-runtime-password" \
|
||||||
|
"THT_SESSION_MIGRATOR_PASSWORD_SOURCE=$fixture/session-migrator-password" \
|
||||||
|
"THT_SESSION_CA_SOURCE=$fixture/session-ca.pem" \
|
||||||
|
>"$fixture/operator.env"
|
||||||
|
|
||||||
|
connector_override="$fixture/connector-secrets.local.yaml"
|
||||||
|
"$root/scripts/generate-connector-secrets-override.sh" \
|
||||||
|
--bindings-env "$fixture/workspace-bindings.env" \
|
||||||
|
--operator-env "$fixture/operator.env" \
|
||||||
|
--output "$connector_override" >/dev/null
|
||||||
|
|
||||||
|
for profile in local server; do
|
||||||
|
rendered="$fixture/$profile.json"
|
||||||
|
files=(
|
||||||
|
-f "$root/compose.yaml"
|
||||||
|
-f "$root/deploy/compose.$profile.yaml"
|
||||||
|
)
|
||||||
|
if [[ "$profile" == server ]]; then
|
||||||
|
files+=(-f "$root/deploy/compose.session-server.yaml.example")
|
||||||
|
fi
|
||||||
|
files+=(
|
||||||
|
-f "$root/deploy/compose.git-ssh.yaml"
|
||||||
|
-f "$connector_override"
|
||||||
|
)
|
||||||
|
"$root/scripts/compose-with-preflight.sh" --env-file "$fixture/operator.env" \
|
||||||
|
"${files[@]}" config --format json >"$rendered"
|
||||||
|
|
||||||
|
node - "$rendered" "$profile" <<'NODE'
|
||||||
|
const fs = require("fs");
|
||||||
|
const [path, profile] = process.argv.slice(2);
|
||||||
|
const config = JSON.parse(fs.readFileSync(path, "utf8"));
|
||||||
|
if (Object.keys(config.services).sort().join(",") !== "core,frontend") {
|
||||||
|
throw new Error(profile + ": mandatory stack must be exactly core,frontend");
|
||||||
|
}
|
||||||
|
const core = config.services.core;
|
||||||
|
for (const target of [
|
||||||
|
"/home/thoth/.pi/agent/auth.json",
|
||||||
|
"/home/thoth/.pi/agent/models.json",
|
||||||
|
"/home/thoth/.pi/agent/settings.json",
|
||||||
|
]) {
|
||||||
|
if (!(core.volumes || []).some((mount) => mount.target === target && mount.read_only)) {
|
||||||
|
throw new Error(profile + ": missing read-only Pi mount " + target);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const [name, value] of Object.entries({
|
||||||
|
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE: "/run/secrets/north-star-research-dwh-password",
|
||||||
|
THT_WS_NORTH_STAR_RESEARCH_VECTOR_API_KEY_FILE: "/run/secrets/north-star-research-vector-api-key",
|
||||||
|
})) {
|
||||||
|
if (core.environment?.[name] !== value) throw new Error(profile + ": missing binding " + name);
|
||||||
|
}
|
||||||
|
const secretTargets = new Set((core.secrets || []).map((secret) => secret.target));
|
||||||
|
for (const target of [
|
||||||
|
"thothii.secrets",
|
||||||
|
"north-star-research-dwh-password",
|
||||||
|
"north-star-research-vector-api-key",
|
||||||
|
]) {
|
||||||
|
if (!secretTargets.has(target)) throw new Error(profile + ": missing secret target " + target);
|
||||||
|
}
|
||||||
|
if (profile === "server") {
|
||||||
|
for (const target of ["session_runtime_password", "session_ca.pem"]) {
|
||||||
|
if (!secretTargets.has(target)) throw new Error("server: missing session secret " + target);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ((config.services.frontend.secrets || []).length !== 0) {
|
||||||
|
throw new Error(profile + ": frontend received a runtime secret");
|
||||||
|
}
|
||||||
|
const rendered = JSON.stringify(config);
|
||||||
|
for (const value of [
|
||||||
|
"fixture-native-auth-key", "fixture-model-api-key", "fixture-git-ssh-key",
|
||||||
|
"fixture-git-known-hosts", "fixture-dwh-password", "fixture-vector-api-key",
|
||||||
|
"fixture-session-runtime-password", "fixture-session-migrator-password", "fixture-session-ca",
|
||||||
|
]) {
|
||||||
|
if (rendered.includes(value)) throw new Error(profile + ": rendered Compose leaked " + value);
|
||||||
|
}
|
||||||
|
NODE
|
||||||
|
echo "canonical $profile base+override fixture passed"
|
||||||
|
done
|
||||||
|
|
||||||
|
printf 'THT_WS_EXAMPLE_DWH_PASSWORD_SOURCE=relative/secret\n' >"$fixture/unsafe.env"
|
||||||
|
if verify_path_variable_values "$fixture/unsafe.env" >/dev/null 2>&1; then
|
||||||
|
echo "relative secret-source fixture was accepted" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
echo "relative secret-source fixture rejected passed"
|
||||||
|
}
|
||||||
|
|
||||||
|
case "$mode" in
|
||||||
|
--fixtures-only)
|
||||||
|
[[ $# -eq 1 ]] || { echo "usage: $0 --fixtures-only" >&2; exit 2; }
|
||||||
|
verify_manual local
|
||||||
|
verify_manual server
|
||||||
|
verify_compose_fixtures
|
||||||
|
;;
|
||||||
|
--profile)
|
||||||
|
profile="${2:-}"
|
||||||
|
[[ $# -eq 2 && "$profile" =~ ^(local|server)$ ]] \
|
||||||
|
|| { echo "usage: $0 --profile {local|server}" >&2; exit 2; }
|
||||||
|
verify_manual "$profile"
|
||||||
|
verify_compose_fixtures
|
||||||
|
echo "== Run isolated workspace-registry bootstrap and recovery smoke =="
|
||||||
|
(
|
||||||
|
cd "$root"
|
||||||
|
env -u WORKSPACE_GIT_REMOTE ./scripts/workspace-registry-smoke.sh
|
||||||
|
)
|
||||||
|
echo "$profile installation documentation verification passed"
|
||||||
;;
|
;;
|
||||||
*)
|
*)
|
||||||
echo "unknown documentation profile: $profile" >&2
|
echo "usage: $0 --fixtures-only | --profile {local|server}" >&2
|
||||||
exit 2
|
exit 2
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
[[ -f "$manual" ]] || { echo "missing $profile installation manual: $manual" >&2; exit 1; }
|
|
||||||
[[ -f "$example" ]] || { echo "missing $profile Compose example: $example" >&2; exit 1; }
|
|
||||||
|
|
||||||
for heading in "${headings[@]}"; do
|
|
||||||
grep -Fqx "## $heading" "$manual" >/dev/null || {
|
|
||||||
echo "missing required heading in $profile manual: $heading" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
done
|
|
||||||
|
|
||||||
grep -Fq "$(basename "$example")" "$manual" || {
|
|
||||||
echo "the $profile manual does not reference its Compose example" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# Values for secret-bearing variables must be paths. These patterns catch common accidental
|
|
||||||
# credentials while allowing declarative *_FILE bindings and explicitly empty assignments.
|
|
||||||
if grep -Ein '(^|[[:space:]])(password|api[_-]?key|token|secret)[[:space:]]*[:=][[:space:]]*[^[:space:]#]' \
|
|
||||||
"$manual" "$example" >/dev/null; then
|
|
||||||
echo "installation documentation contains a secret literal" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
verify_path_variable_values "$manual"
|
|
||||||
verify_path_variable_values "$example"
|
|
||||||
verify_path_variable_values "$root/docs/install/examples/git-https.workspace-registry.yaml"
|
|
||||||
verify_path_variable_values "$root/docs/install/examples/git-ssh.workspace-registry.yaml"
|
|
||||||
verify_path_variable_values "$root/docs/install/examples/workspace-bindings.env.example"
|
|
||||||
verify_server_public_contract
|
|
||||||
verify_manual_supported_path "$profile" "$manual"
|
|
||||||
|
|
||||||
echo "== Validate copied operator fixtures and documented optional Git transports =="
|
|
||||||
verify_copied_operator_fixtures
|
|
||||||
|
|
||||||
echo "== Run isolated workspace-registry bootstrap and recovery smoke =="
|
|
||||||
(
|
|
||||||
cd "$root"
|
|
||||||
env -u WORKSPACE_GIT_REMOTE ./scripts/workspace-registry-smoke.sh
|
|
||||||
)
|
|
||||||
|
|
||||||
echo "$profile installation documentation verification passed"
|
|
||||||
|
|||||||
Reference in New Issue
Block a user