186 lines
9.4 KiB
Markdown
186 lines
9.4 KiB
Markdown
# Local workspace-registry installation (Mac and PC)
|
|
|
|
This guide runs a single-user ThothII registry on Docker Desktop (macOS or Windows) or a local
|
|
Linux Docker Engine. It is intentionally loopback-only. Git is shared; the checkout, connector
|
|
bindings, credentials, and session data are local. Never put credentials in workspace YAML, Git,
|
|
browser drafts, diagnostics, or `.env.example`.
|
|
|
|
## Prerequisites
|
|
|
|
- macOS: Docker Desktop, Git, and sufficient volume disk space. Git Credential Manager is useful
|
|
for HTTPS sign-in.
|
|
- Windows: Docker Desktop with WSL2, Git for Windows, and the clone enabled in Docker file sharing.
|
|
Use absolute paths/WSL paths; PowerShell uses `;` rather than `:` in `COMPOSE_FILE`.
|
|
- Linux PC: Docker Engine, Compose plugin, Git, and a user permitted to run Docker.
|
|
- Outbound access to the Git remote. A local installation needs no inbound firewall rule.
|
|
|
|
Keep the operator `.env` and `installation-secrets/` outside the workspace-registry Git checkout.
|
|
On macOS/Linux use mode `0600` for individual secret files. On Windows apply an ACL that grants
|
|
read access only to the Docker Desktop user. Do not use an empty file to silently bypass a selected
|
|
authentication method.
|
|
|
|
## Git remote: SSH and HTTPS
|
|
|
|
Create one private repository such as `thoth-workspaces.git`. It contains canonical workspace
|
|
definitions and generated artifacts only:
|
|
|
|
```text
|
|
thoth-workspaces.yaml
|
|
workspaces/<workspace-id>.yaml
|
|
workspaces/<workspace-id>.env.example
|
|
workspaces/<workspace-id>.md
|
|
```
|
|
|
|
For SSH, use a scoped deploy key, a verified `known_hosts` file, and strict host-key checking. For
|
|
HTTPS, use Git Credential Manager or a secret-manager-created credentials file. Mount a private
|
|
HTTPS CA as its own file. Do not disable host or certificate verification.
|
|
|
|
```dotenv
|
|
THT_WORKSPACE_GIT_REMOTE=ssh://git@git.example.invalid/platform/thoth-workspaces.git
|
|
THT_WORKSPACE_GIT_BRANCH=main
|
|
THT_WORKSPACE_INSTALLATION_ID=local-laptop
|
|
THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/installation-secrets/git-ssh-key
|
|
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/installation-secrets/git-known-hosts
|
|
THT_WORKSPACE_GIT_CA_FILE=/absolute/path/installation-secrets/git-ca.pem
|
|
```
|
|
|
|
For HTTPS set `THT_WORKSPACE_GIT_CREDENTIALS_FILE` instead of the SSH key/known-hosts pair. Remote
|
|
and branch are non-secret; every `*_FILE` is a local path whose content never enters Git or logs.
|
|
|
|
## Shared Git values, local bindings, and secret files
|
|
|
|
| Location | Contains | Never contains |
|
|
| --- | --- | --- |
|
|
| Git workspace repository | schema v2 YAML, generated binding names, LLM policy, model/index identity | installation hostnames, keys, passwords, certificates, SSH keys |
|
|
| local `.env` | remote, branch, installation ID, selected transport and endpoints | secret contents |
|
|
| local secret directory | Git credentials/key, known hosts, CA, connector secret files | a copied registry checkout |
|
|
| Docker volumes | registry checkout/snapshots/state/locks and local data | host-only secret source files |
|
|
|
|
The persistent volume is `/data/workspace-registry`:
|
|
|
|
```text
|
|
repo/ # persistent Git checkout
|
|
snapshots/ # immutable validated revisions used by sessions
|
|
state/ # active revision and registry state
|
|
locks/ # short-lived publish locks
|
|
```
|
|
|
|
Installation variables are deterministic: `psd-clinical` becomes `PSD_CLINICAL`, and every name
|
|
is `THT_WS_<NAMESPACE>_<ROLE>_<SUFFIX>`. Credentials and certificates use `*_FILE` path variables.
|
|
If declared, `THT_WS_PSD_CLINICAL_VECTOR_WRITER_API_KEY_FILE` is distinct from the vector reader
|
|
file; a reader credential is never repurposed for writing.
|
|
|
|
## Direct PostgreSQL, REST, and SSH tunnel bindings
|
|
|
|
Set only fields for the selected transport. Canonical YAML keeps database/schema/collection,
|
|
distance, embedding model, and dimensions shared in Git.
|
|
|
|
```dotenv
|
|
# Direct PostgreSQL and pgvector
|
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=postgres_direct
|
|
THT_WS_PSD_CLINICAL_DWH_HOST=dwh.example.invalid
|
|
THT_WS_PSD_CLINICAL_DWH_PORT=5432
|
|
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
|
|
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
|
|
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=pgvector_direct
|
|
THT_WS_PSD_CLINICAL_VECTOR_HOST=vector.example.invalid
|
|
THT_WS_PSD_CLINICAL_VECTOR_PORT=5432
|
|
THT_WS_PSD_CLINICAL_VECTOR_USER=thoth_vector_reader
|
|
THT_WS_PSD_CLINICAL_VECTOR_PASSWORD_FILE=/run/secrets/psd-vector-reader
|
|
THT_WS_PSD_CLINICAL_EMBEDDING_BASE_URL=https://embeddings.example.invalid
|
|
```
|
|
|
|
```dotenv
|
|
# REST; an API-key file is needed only for a declared bearer/x-api-key diagnostic.
|
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api
|
|
THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://dwh.example.invalid
|
|
THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-dwh-api-key
|
|
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=rest_api
|
|
THT_WS_PSD_CLINICAL_VECTOR_BASE_URL=https://vectors.example.invalid
|
|
THT_WS_PSD_CLINICAL_VECTOR_API_KEY_FILE=/run/secrets/psd-vector-api-key
|
|
```
|
|
|
|
```dotenv
|
|
# SSH tunnel; host-key verification and TLS target name remain mandatory.
|
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=ssh_tunnel
|
|
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
|
|
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
|
|
THT_WS_PSD_CLINICAL_DWH_SSH_HOST=bastion.example.invalid
|
|
THT_WS_PSD_CLINICAL_DWH_SSH_PORT=22
|
|
THT_WS_PSD_CLINICAL_DWH_SSH_USER=thoth_tunnel
|
|
THT_WS_PSD_CLINICAL_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/psd-dwh-tunnel-key
|
|
THT_WS_PSD_CLINICAL_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/psd-dwh-known-hosts
|
|
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_HOST=dwh.internal.example
|
|
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_PORT=5432
|
|
```
|
|
|
|
Repeat the SSH names for `VECTOR` where needed. REST diagnostics reject a private per-request CA
|
|
rather than weakening TLS; use runtime-trusted HTTPS or verified direct/SSH native TLS. See the
|
|
[diagnostic protocol](../workspace-diagnostic-protocol.md).
|
|
|
|
## Bootstrap, first pull, and diagnostics
|
|
|
|
Copy [the local Compose example](examples/local-compose.workspace-registry.yaml) into an untracked
|
|
operator directory, create its local `.env` and mounted secret files, then render it before start.
|
|
|
|
<!-- verify:command -->
|
|
```sh
|
|
docker compose -f docs/install/examples/local-compose.workspace-registry.yaml config --quiet
|
|
```
|
|
|
|
From the operator directory:
|
|
|
|
```sh
|
|
docker compose -f compose.workspace-registry.yaml up --build -d
|
|
curl --fail --silent http://127.0.0.1:8787/health
|
|
curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
|
|
curl --fail --silent http://127.0.0.1:8787/workspaces
|
|
```
|
|
|
|
The first status request clones, validates all descriptors, and atomically activates a snapshot.
|
|
Use `POST /workspace-registry/pull` to fetch later revisions. Run workspace diagnostics only after
|
|
required bindings are mounted. The optional writer probe uses a distinct writer file and removes
|
|
its uniquely named temporary record; ordinary diagnostics are read-only.
|
|
|
|
To migrate an existing PSD descriptor, create/clone an empty private remote, transform with
|
|
absolute paths, review the schema-v1 result, explicitly add vector database/schema and the complete
|
|
schema-v2 contract, then commit/push. The transformer never imports `${ENV}` values or secrets.
|
|
|
|
```sh
|
|
npm --prefix backend run build
|
|
node backend/dist/workspaces/migrate-legacy.js --input /absolute/path/psd.yaml --output /absolute/path/thoth-workspaces
|
|
```
|
|
|
|
## Publish, update, backup, outage recovery, and rollback
|
|
|
|
Drafts live only in browser storage. Review the canonical field diff, validate/test locally, then
|
|
publish. If conflicted, pull first and create a new reviewed field-level draft; never hand-edit the
|
|
running `repo/` volume. Before upgrading, record registry status, stop Compose, and take a
|
|
timestamped ownership-preserving backup of both registry and local data volumes while excluding
|
|
`installation-secrets/`. Render Compose, rebuild, start, and check status before resuming work.
|
|
|
|
After a valid bootstrap, remote outage retains the last valid snapshot and reports `degraded: true`.
|
|
Pinned sessions continue. Repair network/authentication, pull, and confirm non-degraded status. To
|
|
undo a bad remote change, create a reviewed Git revert/release branch, advance the remote through
|
|
normal policy, pull, and confirm its new snapshot. Do not delete `snapshots/` as rollback.
|
|
|
|
## Troubleshooting
|
|
|
|
| Stable code | Meaning and safe action |
|
|
| --- | --- |
|
|
| `workspace_invalid` | Invalid descriptor/path/snapshot; restore a reviewed canonical revision. |
|
|
| `binding_missing` | A selected value or readable `*_FILE` is absent; fix the local binding/mount. |
|
|
| `workspace_not_activatable` | Diagnostics cannot activate the workspace; correct the selected transport. |
|
|
| `workspace_stale` | Checkout changed or is busy; stop competing pull/publish work. |
|
|
| `workspace_conflict` | Draft base differs from Git; pull, resolve the diff, validate, republish. |
|
|
| `git_unavailable` | Remote, path, network, or lock unavailable; preserve the degraded valid snapshot. |
|
|
| `git_auth_failed` | Mounted SSH/HTTPS material rejected/unreadable; rotate or fix permissions without logging it. |
|
|
| `git_non_fast_forward` | Checkout diverged; reconcile through the registry workflow. |
|
|
| `git_push_rejected` | Remote policy rejected the change; review branch protection/hooks. |
|
|
| `connector_unavailable` | DNS/TLS/auth/resource identity diagnostic failed; inspect local bindings and egress. |
|
|
| `semantic_index_incompatible` | Collection/model/dimensions/distance differs from Git; perform an explicit index migration. |
|
|
|
|
On macOS, restart Docker Desktop if a named volume disappears. On Windows, check WSL2 and Docker
|
|
file sharing. A failed first bootstrap has no snapshot fallback: repair remote trust and retry;
|
|
never create an unreviewed local registry repository.
|