mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
72e16dd5ea2bf6bc22f9d87c7ee6db23a6a3051f
ThothII/docs/install/local-workspace-registry.md
T

9.4 KiB
Raw Blame History

Local workspace-registry installation (Mac and PC)

This guide runs a single-user ThothII registry on Docker Desktop (macOS or Windows) or a local Linux Docker Engine. It is intentionally loopback-only. Git is shared; the checkout, connector bindings, credentials, and session data are local. Never put credentials in workspace YAML, Git, browser drafts, diagnostics, or .env.example.

Prerequisites

  • macOS: Docker Desktop, Git, and sufficient volume disk space. Git Credential Manager is useful for HTTPS sign-in.
  • Windows: Docker Desktop with WSL2, Git for Windows, and the clone enabled in Docker file sharing. Use absolute paths/WSL paths; PowerShell uses ; rather than : in COMPOSE_FILE.
  • Linux PC: Docker Engine, Compose plugin, Git, and a user permitted to run Docker.
  • Outbound access to the Git remote. A local installation needs no inbound firewall rule.

Keep the operator .env and installation-secrets/ outside the workspace-registry Git checkout. On macOS/Linux use mode 0600 for individual secret files. On Windows apply an ACL that grants read access only to the Docker Desktop user. Do not use an empty file to silently bypass a selected authentication method.

Git remote: SSH and HTTPS

Create one private repository such as thoth-workspaces.git. It contains canonical workspace definitions and generated artifacts only:

thoth-workspaces.yaml
workspaces/<workspace-id>.yaml
workspaces/<workspace-id>.env.example
workspaces/<workspace-id>.md

For SSH, use a scoped deploy key, a verified known_hosts file, and strict host-key checking. For HTTPS, use Git Credential Manager or a secret-manager-created credentials file. Mount a private HTTPS CA as its own file. Do not disable host or certificate verification.

THT_WORKSPACE_GIT_REMOTE=ssh://git@git.example.invalid/platform/thoth-workspaces.git
THT_WORKSPACE_GIT_BRANCH=main
THT_WORKSPACE_INSTALLATION_ID=local-laptop
THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/installation-secrets/git-ssh-key
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/installation-secrets/git-known-hosts
THT_WORKSPACE_GIT_CA_FILE=/absolute/path/installation-secrets/git-ca.pem

For HTTPS set THT_WORKSPACE_GIT_CREDENTIALS_FILE instead of the SSH key/known-hosts pair. Remote and branch are non-secret; every *_FILE is a local path whose content never enters Git or logs.

Shared Git values, local bindings, and secret files

Location Contains Never contains
Git workspace repository schema v2 YAML, generated binding names, LLM policy, model/index identity installation hostnames, keys, passwords, certificates, SSH keys
local .env remote, branch, installation ID, selected transport and endpoints secret contents
local secret directory Git credentials/key, known hosts, CA, connector secret files a copied registry checkout
Docker volumes registry checkout/snapshots/state/locks and local data host-only secret source files

The persistent volume is /data/workspace-registry:

repo/       # persistent Git checkout
snapshots/  # immutable validated revisions used by sessions
state/      # active revision and registry state
locks/      # short-lived publish locks

Installation variables are deterministic: psd-clinical becomes PSD_CLINICAL, and every name is THT_WS_<NAMESPACE>_<ROLE>_<SUFFIX>. Credentials and certificates use *_FILE path variables. If declared, THT_WS_PSD_CLINICAL_VECTOR_WRITER_API_KEY_FILE is distinct from the vector reader file; a reader credential is never repurposed for writing.

Direct PostgreSQL, REST, and SSH tunnel bindings

Set only fields for the selected transport. Canonical YAML keeps database/schema/collection, distance, embedding model, and dimensions shared in Git.

# Direct PostgreSQL and pgvector
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=postgres_direct
THT_WS_PSD_CLINICAL_DWH_HOST=dwh.example.invalid
THT_WS_PSD_CLINICAL_DWH_PORT=5432
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=pgvector_direct
THT_WS_PSD_CLINICAL_VECTOR_HOST=vector.example.invalid
THT_WS_PSD_CLINICAL_VECTOR_PORT=5432
THT_WS_PSD_CLINICAL_VECTOR_USER=thoth_vector_reader
THT_WS_PSD_CLINICAL_VECTOR_PASSWORD_FILE=/run/secrets/psd-vector-reader
THT_WS_PSD_CLINICAL_EMBEDDING_BASE_URL=https://embeddings.example.invalid

# REST; an API-key file is needed only for a declared bearer/x-api-key diagnostic.
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api
THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://dwh.example.invalid
THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-dwh-api-key
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=rest_api
THT_WS_PSD_CLINICAL_VECTOR_BASE_URL=https://vectors.example.invalid
THT_WS_PSD_CLINICAL_VECTOR_API_KEY_FILE=/run/secrets/psd-vector-api-key

# SSH tunnel; host-key verification and TLS target name remain mandatory.
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=ssh_tunnel
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
THT_WS_PSD_CLINICAL_DWH_SSH_HOST=bastion.example.invalid
THT_WS_PSD_CLINICAL_DWH_SSH_PORT=22
THT_WS_PSD_CLINICAL_DWH_SSH_USER=thoth_tunnel
THT_WS_PSD_CLINICAL_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/psd-dwh-tunnel-key
THT_WS_PSD_CLINICAL_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/psd-dwh-known-hosts
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_HOST=dwh.internal.example
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_PORT=5432

Repeat the SSH names for VECTOR where needed. REST diagnostics reject a private per-request CA rather than weakening TLS; use runtime-trusted HTTPS or verified direct/SSH native TLS. See the diagnostic protocol.

Bootstrap, first pull, and diagnostics

Copy the local Compose example into an untracked operator directory, create its local .env and mounted secret files, then render it before start.

docker compose -f docs/install/examples/local-compose.workspace-registry.yaml config --quiet

From the operator directory:

docker compose -f compose.workspace-registry.yaml up --build -d
curl --fail --silent http://127.0.0.1:8787/health
curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
curl --fail --silent http://127.0.0.1:8787/workspaces

The first status request clones, validates all descriptors, and atomically activates a snapshot. Use POST /workspace-registry/pull to fetch later revisions. Run workspace diagnostics only after required bindings are mounted. The optional writer probe uses a distinct writer file and removes its uniquely named temporary record; ordinary diagnostics are read-only.

To migrate an existing PSD descriptor, create/clone an empty private remote, transform with absolute paths, review the schema-v1 result, explicitly add vector database/schema and the complete schema-v2 contract, then commit/push. The transformer never imports ${ENV} values or secrets.

npm --prefix backend run build
node backend/dist/workspaces/migrate-legacy.js --input /absolute/path/psd.yaml --output /absolute/path/thoth-workspaces

Publish, update, backup, outage recovery, and rollback

Drafts live only in browser storage. Review the canonical field diff, validate/test locally, then publish. If conflicted, pull first and create a new reviewed field-level draft; never hand-edit the running repo/ volume. Before upgrading, record registry status, stop Compose, and take a timestamped ownership-preserving backup of both registry and local data volumes while excluding installation-secrets/. Render Compose, rebuild, start, and check status before resuming work.

After a valid bootstrap, remote outage retains the last valid snapshot and reports degraded: true. Pinned sessions continue. Repair network/authentication, pull, and confirm non-degraded status. To undo a bad remote change, create a reviewed Git revert/release branch, advance the remote through normal policy, pull, and confirm its new snapshot. Do not delete snapshots/ as rollback.

Troubleshooting

Stable code Meaning and safe action
workspace_invalid Invalid descriptor/path/snapshot; restore a reviewed canonical revision.
binding_missing A selected value or readable *_FILE is absent; fix the local binding/mount.
workspace_not_activatable Diagnostics cannot activate the workspace; correct the selected transport.
workspace_stale Checkout changed or is busy; stop competing pull/publish work.
workspace_conflict Draft base differs from Git; pull, resolve the diff, validate, republish.
git_unavailable Remote, path, network, or lock unavailable; preserve the degraded valid snapshot.
git_auth_failed Mounted SSH/HTTPS material rejected/unreadable; rotate or fix permissions without logging it.
git_non_fast_forward Checkout diverged; reconcile through the registry workflow.
git_push_rejected Remote policy rejected the change; review branch protection/hooks.
connector_unavailable DNS/TLS/auth/resource identity diagnostic failed; inspect local bindings and egress.
semantic_index_incompatible Collection/model/dimensions/distance differs from Git; perform an explicit index migration.

On macOS, restart Docker Desktop if a named volume disappears. On Windows, check WSL2 and Docker file sharing. A failed first bootstrap has no snapshot fallback: repair remote trust and retry; never create an unreviewed local registry repository.

Reference in New Issue View Git Blame Copy Permalink