ThothII/docs/install/local-workspace-registry.md

# Local workspace-registry installation (Mac and PC)

This guide runs a single-user ThothII registry on Docker Desktop (macOS or Windows) or a local
Linux Docker Engine. It is intentionally loopback-only. Git is shared; the checkout, connector
bindings, credentials, and session data are local. Never put credentials in workspace YAML, Git,
browser drafts, diagnostics, or `.env.example`.

## Prerequisites

- macOS: Docker Desktop, Git, and sufficient volume disk space. Git Credential Manager is useful
  for HTTPS sign-in.
- Windows: Docker Desktop with WSL2, Git for Windows, and the clone enabled in Docker file sharing.
  Use absolute paths/WSL paths; PowerShell uses `;` rather than `:` in `COMPOSE_FILE`.
- Linux PC: Docker Engine, Compose plugin, Git, and a user permitted to run Docker.
- Outbound access to the Git remote. A local installation needs no inbound firewall rule.

Keep the operator `.env` and `installation-secrets/` outside the workspace-registry Git checkout.
On macOS/Linux use mode `0600` for individual secret files. On Windows apply an ACL that grants
read access only to the Docker Desktop user. Do not use an empty file to silently bypass a selected
authentication method.

## Git remote: SSH and HTTPS

Create one private repository such as `thoth-workspaces.git`. It contains canonical workspace
definitions and generated artifacts only:

```text
thoth-workspaces.yaml
workspaces/<workspace-id>.yaml
workspaces/<workspace-id>.env.example
workspaces/<workspace-id>.md
```

For SSH, use a scoped deploy key, a verified `known_hosts` file, and strict host-key checking. For
HTTPS, use Git Credential Manager or a secret-manager-created credentials file. Mount a private
HTTPS CA as its own file. Do not disable host or certificate verification. The base Compose file
does not mount a Git credential: add exactly one optional `git-ssh.workspace-registry.yaml` or
`git-https.workspace-registry.yaml` override, so unused credential paths are never bind-mounted.

```dotenv
THT_WORKSPACE_GIT_REMOTE=ssh://git@git.example.invalid/platform/thoth-workspaces.git
THT_WORKSPACE_GIT_BRANCH=main
THT_WORKSPACE_INSTALLATION_ID=local-laptop
THT_SOURCE_ROOT=/absolute/path/to/ThothII
THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/installation-secrets/git-ssh-key
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/installation-secrets/git-known-hosts
THT_WORKSPACE_GIT_CA_FILE=/absolute/path/installation-secrets/git-ca.pem
```

For HTTPS set `THT_WORKSPACE_GIT_CREDENTIALS_FILE` instead of the SSH key/known-hosts pair. Remote
and branch are non-secret; every `*_FILE` is a local path whose content never enters Git or logs.

## Shared Git values, local bindings, and secret files

| Location | Contains | Never contains |
| --- | --- | --- |
| Git workspace repository | schema v2 YAML, generated binding names, LLM policy, model/index identity | installation hostnames, keys, passwords, certificates, SSH keys |
| local `.env` | remote, branch, installation ID, `THT_WORKSPACE_BINDINGS_ENV_FILE`, and secret source paths | secret contents or `THT_WS_*` values |
| workspace bindings env file | only `THT_WS_*` transport, endpoint, user, and `/run/secrets/...` path bindings | secret contents or unrelated application settings |
| local secret directory | Git credentials/key, known hosts, CA, connector secret files | a copied registry checkout |
| Docker volumes | registry checkout/snapshots/state/locks and local data | host-only secret source files |

The persistent volume is `/data/workspace-registry`:

```text
repo/       # persistent Git checkout
snapshots/  # immutable validated revisions used by sessions
state/      # active revision and registry state
locks/      # short-lived publish locks
```

Installation variables are deterministic: `psd-clinical` becomes `PSD_CLINICAL`, and every name
is `THT_WS_<NAMESPACE>_<ROLE>_<SUFFIX>`. Copy
[the bindings env example](examples/workspace-bindings.env.example) to an untracked operator file
and set its absolute path as `THT_WORKSPACE_BINDINGS_ENV_FILE`. It is loaded only into `core`.
Credentials and certificates use `*_FILE` path variables that must point inside `/run/secrets`.
If declared, `THT_WS_PSD_CLINICAL_VECTOR_WRITER_API_KEY_FILE` is distinct from the vector reader
file; a reader credential is never repurposed for writing.

## Direct PostgreSQL, REST, and SSH tunnel bindings

Set only fields for the selected transport in the dedicated bindings env file. Canonical YAML keeps
database/schema/collection, distance, embedding model, and dimensions shared in Git. Every
`*_FILE=/run/secrets/<target>` binding needs one matching host-only `*_SOURCE` path in operator
`.env`. Generate the untracked connector override from those two files during bootstrap; do not
copy or maintain a workspace-specific Compose override.

```dotenv
# Direct PostgreSQL and pgvector
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=postgres_direct
THT_WS_PSD_CLINICAL_DWH_HOST=dwh.example.invalid
THT_WS_PSD_CLINICAL_DWH_PORT=5432
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=pgvector_direct
THT_WS_PSD_CLINICAL_VECTOR_HOST=vector.example.invalid
THT_WS_PSD_CLINICAL_VECTOR_PORT=5432
THT_WS_PSD_CLINICAL_VECTOR_USER=thoth_vector_reader
THT_WS_PSD_CLINICAL_VECTOR_PASSWORD_FILE=/run/secrets/psd-vector-reader
THT_WS_PSD_CLINICAL_EMBEDDING_BASE_URL=https://embeddings.example.invalid
```

```dotenv
# REST; an API-key file is needed only for a declared bearer/x-api-key diagnostic.
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=rest_api
THT_WS_PSD_CLINICAL_DWH_BASE_URL=https://dwh.example.invalid
THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE=/run/secrets/psd-dwh-api-key
THT_WS_PSD_CLINICAL_VECTOR_TRANSPORT=rest_api
THT_WS_PSD_CLINICAL_VECTOR_BASE_URL=https://vectors.example.invalid
THT_WS_PSD_CLINICAL_VECTOR_API_KEY_FILE=/run/secrets/psd-vector-api-key
```

```dotenv
# SSH tunnel diagnostic only; runtime sessions are fail-closed in this release.
THT_WS_PSD_CLINICAL_DWH_TRANSPORT=ssh_tunnel
THT_WS_PSD_CLINICAL_DWH_USER=thoth_reader
THT_WS_PSD_CLINICAL_DWH_PASSWORD_FILE=/run/secrets/psd-dwh-reader
THT_WS_PSD_CLINICAL_DWH_SSH_HOST=bastion.example.invalid
THT_WS_PSD_CLINICAL_DWH_SSH_PORT=22
THT_WS_PSD_CLINICAL_DWH_SSH_USER=thoth_tunnel
THT_WS_PSD_CLINICAL_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/psd-dwh-tunnel-key
THT_WS_PSD_CLINICAL_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/psd-dwh-known-hosts
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_HOST=dwh.internal.example
THT_WS_PSD_CLINICAL_DWH_SSH_TARGET_PORT=5432
```

Repeat the SSH names for `VECTOR` where needed. REST diagnostics reject a private per-request CA
rather than weakening TLS; use runtime-trusted HTTPS or verified direct/SSH native TLS. See the
[diagnostic protocol](../workspace-diagnostic-protocol.md).

An SSH connector can prove installation reachability, host-key verification, authentication, and
target identity, but it intentionally returns `workspace_not_activatable`; select direct or REST
before creating sessions. Git pull/push over SSH remains fully supported and is independent.

## Bootstrap, first pull, and diagnostics

Copy [the local Compose example](examples/local-compose.workspace-registry.yaml), exactly one
selected [SSH Git override](examples/git-ssh.workspace-registry.yaml) or [HTTPS Git override](examples/git-https.workspace-registry.yaml),
and [the bindings env example](examples/workspace-bindings.env.example) into an untracked operator
directory. Keep `THT_SOURCE_ROOT` and the absolute `THT_WORKSPACE_BINDINGS_ENV_FILE` in its `.env`
for Compose interpolation; this keeps the copied Compose file buildable and confines `THT_WS_*`
values to `core`. A Compose `.env` file is not a shell environment, so do not import it into the
maintenance shell. Instead, explicitly export the two non-secret paths before running the commands.
Create the host secret files named by the selected Git transport and every declared connector
`*_SOURCE`, then generate the connector override and render through the preflight wrapper. The
wrapper is required: it rejects unsafe source paths and a combined SSH+HTTPS Git selection before
Compose runs.

```sh
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
export THT_WORKSPACE_BINDINGS_ENV_FILE="$(pwd -P)/workspace-bindings.env"
"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" --bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env .env --output connector-secrets.local.yaml
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
  -f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml -f connector-secrets.local.yaml config --quiet
```

From the operator directory:

```sh
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file .env \
  -f compose.workspace-registry.yaml -f git-ssh.workspace-registry.yaml -f connector-secrets.local.yaml up --build -d
curl --fail --silent http://127.0.0.1:8787/health
curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
curl --fail --silent http://127.0.0.1:8787/workspaces
```

The first status request clones, validates all descriptors, and atomically activates a snapshot.
Use `POST /workspace-registry/pull` to fetch later revisions. Run workspace diagnostics only after
required bindings are mounted. The optional writer probe uses a distinct writer file and removes
its uniquely named temporary record; ordinary diagnostics are read-only.

To migrate an existing PSD descriptor, create/clone an empty private remote, set the absolute
`THT_SOURCE_ROOT`, transform with absolute paths, review the schema-v1 result, explicitly add
vector database/schema and the complete schema-v2 contract, then commit/push. The transformer
never imports `${ENV}` values or secrets.

```sh
THT_SOURCE_ROOT=/absolute/path/to/ThothII
npm --prefix "$THT_SOURCE_ROOT/backend" run build
node "$THT_SOURCE_ROOT/backend/dist/workspaces/migrate-legacy.js" --input /absolute/path/psd.yaml --output /absolute/path/thoth-workspaces
```

## Publish, update, backup, outage recovery, and rollback

Drafts live only in browser storage. Review the canonical field diff, validate/test locally, then
publish. If conflicted, pull first and create a new reviewed field-level draft; never hand-edit the
running `repo/` volume. Before upgrading, record registry status, stop Compose, and take a
timestamped ownership-preserving backup of both registry and local data volumes while excluding
`installation-secrets/`. Render Compose, rebuild, start, and check status before resuming work.

After a valid bootstrap, remote outage retains the last valid snapshot and reports `degraded: true`.
Pinned sessions continue. Repair network/authentication, pull, and confirm non-degraded status. To
undo a bad remote change, create a reviewed Git revert/release branch, advance the remote through
normal policy, pull, and confirm its new snapshot. Do not delete `snapshots/` as rollback.

## Troubleshooting

| Stable code | Meaning and safe action |
| --- | --- |
| `workspace_invalid` | Invalid descriptor/path/snapshot; restore a reviewed canonical revision. |
| `binding_missing` | A selected value or readable `*_FILE` is absent; fix the local binding/mount. |
| `workspace_not_activatable` | Diagnostics cannot activate the workspace; correct the selected transport. |
| `workspace_stale` | Checkout changed or is busy; stop competing pull/publish work. |
| `workspace_conflict` | Draft base differs from Git; pull, resolve the diff, validate, republish. |
| `git_unavailable` | Remote, path, network, or lock unavailable; preserve the degraded valid snapshot. |
| `git_auth_failed` | Mounted SSH/HTTPS material rejected/unreadable; rotate or fix permissions without logging it. |
| `git_non_fast_forward` | Checkout diverged; reconcile through the registry workflow. |
| `git_push_rejected` | Remote policy rejected the change; review branch protection/hooks. |
| `connector_unavailable` | DNS/TLS/auth/resource identity diagnostic failed; inspect local bindings and egress. |
| `semantic_index_incompatible` | Collection/model/dimensions/distance differs from Git; perform an explicit index migration. |

On macOS, restart Docker Desktop if a named volume disappears. On Windows, check WSL2 and Docker
file sharing. A failed first bootstrap has no snapshot fallback: repair remote trust and retry;
never create an unreviewed local registry repository.