mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: document one-command Docker installation

Browse Source
This commit is contained in:
marcopan
2026-07-12 11:44:00 +02:00
parent 2ab91b7c0d
commit 07967bf589
8 changed files with 303 additions and 443 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+74 -60
View File
@@ -4,24 +4,53 @@ ThothII is a human-reviewed NL-to-SQL workflow with a React frontend and a Fasti
core. The portable deployment runs exactly two application services; data services remain
external in this profile.
## Docker Compose: external services
## Docker Compose: one-command startup
Requirements: Docker Engine with Compose v2 and reachable DWH, vector, and embeddings
services.
Requirements: Docker Engine with Compose v2. The default project starts only the two
application images; DWH, vector and embedding services can be remote or supplied by an
optional overlay.
1. For local development only, copy `deploy/env.example` to `deploy/.env` and fill in runtime
credentials. The file is gitignored and is never copied into either image.
2. Add or edit YAML workspace descriptors under `deploy/workspaces/`. These files are mounted
read-only. Use relative `roots`; they resolve beneath `/data/workspaces/<workspace-name>`.
3. Start the external-service profile:
From a fresh clone, run these commands from the repository root:
```sh
docker compose -f compose.yaml -f deploy/compose.local.yaml \
--profile external up --build --wait
```
```sh
cp .env.example .env
cp deploy/secrets/thothii.secrets.example deploy/secrets/thothii.secrets
chmod 600 deploy/secrets/thothii.secrets
# Edit .env (non-secret endpoints) and deploy/secrets/thothii.secrets (KEY=VALUE lines).
docker compose up --build -d
```
4. Open <http://127.0.0.1:8080>. The published port is loopback-only. Set `THOTH_HTTP_PORT`
before starting to use another loopback port.
The root `.env` is loaded automatically by Compose. It defaults to `compose.yaml`, an empty
profile, and `THT_SECRETS_FILE=deploy/secrets/thothii.secrets`; no `--env-file`, `-f`, or
`--profile` flag is required for the normal installation. Add or edit YAML workspace descriptors
under `deploy/workspaces/`; they are mounted read-only and relative `roots` resolve beneath
`/data/workspaces/<workspace-name>`. Open <http://127.0.0.1:8080> (set `THOTH_HTTP_PORT` in
`.env` to choose another loopback port).
The bundle contains only values, one per line (`THT_MODEL_API_KEY=...`, DWH/vector keys, and
the optional local-vector passwords). It is ignored by Git and never copied into either image.
Do not put credentials in `.env`, workspace YAML, URLs, or Compose interpolation values.
### Optional overlays
Overlays are selected in `.env`, so the operational command remains the same. On Unix-like
systems use `:` between files; on Windows use `;`:
```dotenv
# Remote DWH/vector/embedding services with authenticated reverse proxy:
COMPOSE_FILE=compose.yaml:deploy/compose.production.yaml
COMPOSE_PROFILES=
# Local pgvector (Mac/Windows or a standalone application server):
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml
COMPOSE_PROFILES=local-vector
```
After changing `.env`, apply the selected configuration with `docker compose up --build -d`.
Preprocessing is an explicit opt-in preset: append
`deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml` and set
`COMPOSE_PROFILES=local-vector,preprocess`; then run the job with
`docker compose run --rm preprocess-evidence` or `preprocess-dwh`.
Application state, including settings, sessions, artifacts, and indexes, lives in the named
`thoth_data` volume mounted at `/data`. `docker compose down` keeps that volume. Only an
@@ -43,41 +72,30 @@ Each run uses a unique Compose project and removes that project's containers, ne
volume afterward. It never targets the fixed `thothii` operator project or its volume. Set
`SMOKE_PROJECT` to a different explicit project name for reproducible debugging, and set
`KEEP_SMOKE_RESOURCES=1` to retain that smoke project's resources for inspection; remove them
later with `docker compose --project-name "$SMOKE_PROJECT" --profile external down --volumes`.
later with `docker compose --project-name "$SMOKE_PROJECT" down --volumes`.
## Optional local pgvector and recovery
Start the persistent local vector profile with `docker compose -f compose.yaml -f
deploy/compose.local-vector.yaml --profile local-vector up --build --wait`. Its `vector_data`
volume is independent of application state. Reader, writer,
migrator, and bootstrap credentials remain separate; password files must be mode `0600` and
must not be passed as URL arguments.
The local-vector overlay reads `THT_VECTOR_BOOTSTRAP_PASSWORD`,
`THT_VECTOR_MIGRATOR_PASSWORD`, `THT_VECTOR_READER_PASSWORD`, and
`THT_VECTOR_WRITER_PASSWORD` from the same bundle. Its `vector_data` volume is independent of
application state; passwords are selected at runtime and are never passed as URL arguments.
## Preprocessing jobs and S3 Evidence
The included job workspaces target the local-vector profile. Point the four
`THT_VECTOR_*_PASSWORD_SECRET_FILE` variables at owner-only files, set `THT_OLLAMA_URL`, mount
Evidence at `/data/source/evidence`, then run the explicit overlays (which are inert for normal
runtime):
The included job workspaces target the local-vector profile. Put the four local-vector password
keys in the bundle, set `THT_OLLAMA_URL`, mount Evidence at `/data/source/evidence`, then select
the preprocessing preset in `.env`:
```sh
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \
--profile local-vector --profile preprocess build preprocess-evidence
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \
--profile local-vector --profile preprocess run --rm preprocess-evidence
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \
--profile local-vector --profile preprocess build preprocess-dwh
docker compose -f compose.yaml -f deploy/compose.local-vector.yaml \
-f deploy/compose.preprocess.yaml -f deploy/compose.preprocess-local-vector.yaml \
--profile local-vector --profile preprocess run --rm preprocess-dwh
```dotenv
COMPOSE_FILE=compose.yaml:deploy/compose.local-vector.yaml:deploy/compose.preprocess.yaml:deploy/compose.preprocess-local-vector.yaml
COMPOSE_PROFILES=local-vector,preprocess
```
The local preprocessing override makes each job wait for the vector database health check,
role reconciliation, and a successful migration. These commands are safe on a clean Compose
project; no separate database startup or migration command is required.
Run `docker compose run --rm preprocess-evidence` or
`docker compose run --rm preprocess-dwh`. The overlay makes each job wait for the vector
database health check, role reconciliation, and a successful migration; no separate database
startup or migration command is required.
S3 Evidence uses the optional `tht[s3]` dependency and canonical `s3://bucket/key` provenance.
AWS endpoints are used when no custom URL is supplied. Every custom endpoint is an explicit egress
@@ -138,36 +156,32 @@ with the organization's reviewed identity proxy. `AUTH_MODE=upstream` trusts thi
rejects requests without the identity header. Setting `THOTH_PUBLIC_EXPOSURE=true` with any other
auth mode fails during core startup.
Production credentials use Compose secrets, not `deploy/.env`. Create five files outside the
repository, restrict their host permissions, and point these variables to them:
Production credentials use the one Compose secret bundle, not `.env`. Put the required keys in
`deploy/secrets/thothii.secrets` and select the production overlay in `.env`:
```sh
export THT_DWH_API_KEY_SECRET_FILE=/secure/thoth/dwh-api-key
export THT_VEC_API_KEY_SECRET_FILE=/secure/thoth/vector-reader-api-key
export THT_VEC_WRITE_API_KEY_SECRET_FILE=/secure/thoth/vector-writer-api-key
export THT_CA_SECRET_FILE=/secure/thoth/ca-chain.pem
export THT_MODEL_API_KEY_SECRET_FILE=/secure/thoth/model-api-key
export THT_DB_NAME=warehouse
export THT_DWH_REST_URL=https://dwh.example.test
export THT_VEC_REST_URL=https://vectors.example.test
export THT_OLLAMA_URL=https://embeddings.example.test
docker compose -f compose.yaml -f deploy/compose.production.yaml \
--profile external up --build --wait
```dotenv
THT_MODEL_API_KEY=replace-me
THT_DWH_API_KEY=replace-me
THT_VEC_API_KEY=replace-me
THT_VEC_WRITE_API_KEY=replace-me
```
The secrets and public CA chain are mounted read-only under `/run/secrets` and must be readable by
the core's UID 10001. Host secret files must be `0600` or `0400`; Docker's runtime `0444` mount is
accepted only beneath `/run/secrets`. See [`deploy/secrets/README.md`](deploy/secrets/README.md) for
the verification command. The frontend remains on loopback; the authenticated host proxy is the
The bundle is mounted read-only as `/run/secrets/thothii.secrets` and must be mode `0600` or
`0400` on the host. Docker's runtime `0444` mode is accepted only beneath `/run/secrets`; see
[`deploy/secrets/README.md`](deploy/secrets/README.md). A PEM CA chain is deliberately not a
bundle value: PEM contains whitespace and is rejected by the strict parser. Keep the CA chain in
the host/secret-manager materialization and add a reviewed Compose override that mounts it at
`/run/secrets/ca-chain.pem` and sets `THT_SSL_CA` when a private CA is required. The base bundle
does not create that mount. The frontend remains on loopback; the authenticated host proxy is the
only public listener.
Set the selected model provider in application settings (or `PI_PROVIDER`). For each Pi spawn the
backend validates and reads `THT_MODEL_API_KEY_FILE`, then exposes its value only as the provider's
backend validates and reads `THT_MODEL_API_KEY` from the bundle, then exposes its value only as the provider's
recognized child variable (for example `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`, or
`ZAI_API_KEY`). Neither the generic file path nor deprecated `PI_PROVIDER_API_KEY` is inherited by
Pi. Local providers such as Ollama require no model key.
`THT_MODEL_API_KEY_FILE` supports Pi providers whose authentication is exactly one key:
`THT_MODEL_API_KEY` supports Pi providers whose authentication is exactly one key:
`ant-ling`, `anthropic`, `cerebras`, `deepseek`, `fireworks`, `github-copilot`, `google`
(including the `gemini` alias), `google-vertex` when using its API-key mode, `groq`,
`huggingface`, `kimi-coding`, `minimax`, `minimax-cn`, `mistral`, `moonshotai`,