ThothII/docs/install/local-workspace-registry.md

# Local workspace-registry installation (Mac and PC)

Complete the [local PC/Mac/Linux installation](local.md) first. This guide continues with the
Git-backed workspace source of truth, installation-local connector bindings, and diagnostics. Use
the [Pi management manual](pi-management.md) for provider configuration and image recovery.

This guide runs a single-user ThothII registry on Docker Desktop (macOS or Windows) or a local
Linux Docker Engine. It is intentionally loopback-only. Git is shared; the checkout, DWH
bindings, credentials, and session data are local, while internal Qdrant/Ollama ship in the
Compose stack. Never put credentials in workspace YAML, Git, browser drafts, diagnostics, or
`.env.example`.

## Prerequisites

- macOS: Docker Desktop, Git, and sufficient volume disk space. Git Credential Manager is useful
  for HTTPS sign-in.
- Windows: Docker Desktop with WSL2, Git for Windows, and the clone enabled in Docker file sharing.
  Use absolute paths/WSL paths; PowerShell uses `;` rather than `:` in `COMPOSE_FILE`.
- Linux PC: Docker Engine, Compose plugin, Git, and a user permitted to run Docker.
- Outbound access to the Git remote. A local installation needs no inbound firewall rule.

Keep the operator `.env` and `installation-secrets/` outside the workspace-registry Git checkout.
On macOS/Linux use mode `0600` for individual secret files. On Windows apply an ACL that grants
read access only to the Docker Desktop user. Do not use an empty file to silently bypass a selected
authentication method.

## Git remote: SSH and HTTPS

Create one private repository such as `thoth-workspaces.git`. It contains canonical workspace
definitions and generated artifacts only:

```text
thoth-workspaces.yaml
workspaces/<workspace-id>.yaml
workspaces/<workspace-id>.env.example
workspaces/<workspace-id>.md
```

For SSH, use a scoped deploy key, a verified `known_hosts` file, and strict host-key checking. For
HTTPS, use Git Credential Manager or a secret-manager-created credentials file. Mount a private
HTTPS CA as its own file. Do not disable host or certificate verification. The base Compose file
does not mount a Git credential: add exactly one optional `deploy/compose.git-ssh.yaml` or
`deploy/compose.git-https.yaml` override, so unused credential paths are never bind-mounted.

```dotenv
THT_WORKSPACE_GIT_REMOTE=ssh://git@git.example.invalid/platform/thoth-workspaces.git
THT_WORKSPACE_GIT_BRANCH=main
THT_WORKSPACE_INSTALLATION_ID=local-laptop
THT_SOURCE_ROOT=/absolute/path/to/ThothII
PI_AUTH_FILE=/absolute/path/installation-secrets/pi-auth.json
THT_SECRETS_FILE=/absolute/path/installation-secrets/thothii.secrets
THT_WORKSPACE_GIT_SSH_KEY_FILE=/absolute/path/installation-secrets/git-ssh-key
THT_WORKSPACE_GIT_KNOWN_HOSTS_FILE=/absolute/path/installation-secrets/git-known-hosts
THT_WORKSPACE_GIT_CA_FILE=/absolute/path/installation-secrets/git-ca.pem
```

For HTTPS set `THT_WORKSPACE_GIT_CREDENTIALS_FILE` instead of the SSH key/known-hosts pair. Remote
and branch are non-secret; every `*_FILE` is a local path whose content never enters Git or logs.

## Shared Git values, local bindings, and secret files

| Location | Contains | Never contains |
| --- | --- | --- |
| Git workspace repository | schema v3 YAML, generated binding names, LLM policy, and semantic-index identity | installation hostnames, keys, passwords, certificates, SSH keys |
| local `.env` | remote, branch, installation ID, `THT_WORKSPACE_BINDINGS_ENV_FILE`, and secret source paths | secret contents or `THT_WS_*` values |
| workspace bindings env file | only `THT_WS_*` transport, endpoint, user, and `/run/secrets/...` path bindings | secret contents or unrelated application settings |
| local secret directory | Git credentials/key, known hosts, CA, connector secret files | a copied registry checkout |
| Docker volumes | registry checkout/snapshots/state/locks and local data | host-only secret source files |

The persistent volume is `/data/workspace-registry`:

```text
repo/       # persistent Git checkout
snapshots/  # immutable validated revisions used by sessions
state/      # active revision and registry state
locks/      # short-lived publish locks
```

Installation variables are deterministic: `north-star-research` becomes `NORTH_STAR_RESEARCH`, and every name
is `THT_WS_<NAMESPACE>_<ROLE>_<SUFFIX>`. Copy
[the bindings env example](examples/workspace-bindings.env.example) to an untracked operator file
and set its absolute path as `THT_WORKSPACE_BINDINGS_ENV_FILE`. It is loaded only into `core`.
Credentials and certificates use `*_FILE` path variables that must point inside `/run/secrets`.

## Direct PostgreSQL, REST, and SSH tunnel bindings

Set only fields for the selected transport in the dedicated bindings env file. Canonical YAML keeps
database/schema shared in Git. Every `*_FILE=/run/secrets/<target>` binding needs one matching
host-only `*_SOURCE` path in operator `.env`. Generate the untracked connector override from those
two files during bootstrap; do not copy or maintain a workspace-specific Compose override.

```dotenv
# Direct PostgreSQL
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=postgres_direct
THT_WS_NORTH_STAR_RESEARCH_DWH_HOST=dwh.example.invalid
THT_WS_NORTH_STAR_RESEARCH_DWH_PORT=5432
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
```

```dotenv
# REST; an API-key file is needed only for a declared bearer/x-api-key diagnostic.
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=rest_api
THT_WS_NORTH_STAR_RESEARCH_DWH_BASE_URL=https://dwh.example.invalid
THT_WS_NORTH_STAR_RESEARCH_DWH_API_KEY_FILE=/run/secrets/north-star-research-dwh-api-key
```

```dotenv
# SSH tunnel diagnostic only; runtime sessions are fail-closed in this release.
THT_WS_NORTH_STAR_RESEARCH_DWH_TRANSPORT=ssh_tunnel
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_HOST=bastion.example.invalid
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_PORT=22
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_USER=thoth_tunnel
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_PRIVATE_KEY_FILE=/run/secrets/north-star-research-dwh-tunnel-key
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_KNOWN_HOSTS_FILE=/run/secrets/north-star-research-dwh-known-hosts
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_TARGET_HOST=dwh.internal.example
THT_WS_NORTH_STAR_RESEARCH_DWH_SSH_TARGET_PORT=5432
```

REST diagnostics reject a private per-request CA rather than weakening TLS; use runtime-trusted
HTTPS or verified direct/SSH native TLS. See the
[diagnostic protocol](../workspace-diagnostic-protocol.md).

An SSH connector can prove installation reachability, host-key verification, authentication, and
target identity, but it intentionally returns `workspace_not_activatable`; select direct or REST
before creating sessions. Git pull/push over SSH remains fully supported and is independent.

## Bootstrap, first pull, and diagnostics

Use the repository's canonical `compose.yaml` plus `deploy/compose.local.yaml`; they always start
`frontend`, `core`, `qdrant`, `embedding`, and the one-shot `embedding-model-init`. This profile
is CPU-first. Add `THOTH_ENABLE_EMBEDDING_GPU=1` only on a Linux host that intentionally exposes a
supported GPU device to Docker. Qdrant is a derived but persistent index, while Ollama keeps a
local model cache for `qwen3-embedding:0.6b` (`1024` dimensions, cosine distance). Do not copy or
maintain a standalone application Compose file. Copy [the bindings env example](examples/workspace-bindings.env.example) into an
untracked operator directory and create a protected operator env file from
`deploy/env/local.env.example`. It must contain absolute `PI_AUTH_FILE`,
`THT_SECRETS_FILE`, `THT_WORKSPACE_BINDINGS_ENV_FILE`, and connector `*_SOURCE` paths.
The Pi auth JSON, runtime secret bundle, and each connector credential remain separate protected
host files and are mounted read-only; their contents never enter the operator env or rendered
Compose.

Select exactly one repository Git transport override, `deploy/compose.git-ssh.yaml` or
`deploy/compose.git-https.yaml`. A Compose env file is not a shell environment, so export only the
non-secret paths required by the maintenance commands. Generate the connector override and render
through the preflight wrapper, which rejects unsafe paths and combined SSH+HTTPS selection.
Record the selected Git and generated connector overrides in the operator
[`thothii-installation.yaml` example](examples/thothii-installation.local.yaml), using absolute
paths, so `thothctl` remains the ordinary lifecycle interface.

```sh
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
export THT_OPERATOR_ENV=/absolute/path/to/operator/local.env
export THT_WORKSPACE_BINDINGS_ENV_FILE=/absolute/path/to/operator/workspace-bindings.env
export THT_CONNECTOR_OVERRIDE=/absolute/path/to/operator/connector-secrets.local.yaml
"$THT_SOURCE_ROOT/scripts/generate-connector-secrets-override.sh" --bindings-env "$THT_WORKSPACE_BINDINGS_ENV_FILE" --operator-env "$THT_OPERATOR_ENV" --output "$THT_CONNECTOR_OVERRIDE"
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file "$THT_OPERATOR_ENV" \
  -f "$THT_SOURCE_ROOT/compose.yaml" -f "$THT_SOURCE_ROOT/deploy/compose.local.yaml" \
  -f "$THT_SOURCE_ROOT/deploy/compose.git-ssh.yaml" -f "$THT_CONNECTOR_OVERRIDE" config --quiet
```

```sh
"$THT_SOURCE_ROOT/scripts/compose-with-preflight.sh" --env-file "$THT_OPERATOR_ENV" \
  -f "$THT_SOURCE_ROOT/compose.yaml" -f "$THT_SOURCE_ROOT/deploy/compose.local.yaml" \
  -f "$THT_SOURCE_ROOT/deploy/compose.git-ssh.yaml" -f "$THT_CONNECTOR_OVERRIDE" up --build -d
curl --fail --silent http://127.0.0.1:8787/health
curl --fail --silent http://127.0.0.1:8787/workspace-registry/status
curl --fail --silent http://127.0.0.1:8787/workspaces
```

The first status request clones, validates all descriptors, and atomically activates a snapshot.
Use `POST /workspace-registry/pull` to fetch later revisions. Run workspace diagnostics only after
required DWH bindings are mounted. Schema-v3 diagnostics probe the internal Qdrant/Ollama
services through backend config; ordinary diagnostics are read-only.

Schema-v3 is the only operational descriptor format. Schema-v1/v2 descriptors remain
`migration_required` until an explicit reviewed migration writes schema version 3. One workspace owns one Qdrant collection; schema, Evidence, and Memory records share that collection and remain
isolated by payload `kind`.

To migrate an existing legacy descriptor, create/clone an empty private remote, set the absolute
`THT_SOURCE_ROOT`, transform with absolute paths, review the schema-v1 result, explicitly produce
the reviewed schema-v3 contract, then commit/push. The transformer never imports `${ENV}` values
or secrets.

```sh
THT_SOURCE_ROOT=/absolute/path/to/ThothII
npm --prefix "$THT_SOURCE_ROOT/backend" run build
node "$THT_SOURCE_ROOT/backend/dist/workspaces/migrate-legacy.js" --input /absolute/path/legacy.yaml --output /absolute/path/thoth-workspaces
```

## Publish, update, backup, outage recovery, and rollback

Drafts live only in browser storage. Review the canonical field diff, validate/test locally, then
publish. If conflicted, pull first and create a new reviewed field-level draft; never hand-edit the
running `repo/` volume. Before upgrading, record registry status, stop Compose, and take a
timestamped ownership-preserving backup of both registry and local data volumes while excluding
`installation-secrets/`. Render Compose, rebuild, start, and check status before resuming work.

After a valid bootstrap, remote outage retains the last valid snapshot and reports `degraded: true`.
Pinned sessions continue. Repair network/authentication, pull, and confirm non-degraded status. To
undo a bad remote change, create a reviewed Git revert/release branch, advance the remote through
normal policy, pull, and confirm its new snapshot. Do not delete `snapshots/` as rollback.

## Troubleshooting

| Stable code | Meaning and safe action |
| --- | --- |
| `workspace_invalid` | Invalid descriptor/path/snapshot; restore a reviewed canonical revision. |
| `binding_missing` | A selected value or readable `*_FILE` is absent; fix the local binding/mount. |
| `workspace_not_activatable` | Diagnostics cannot activate the workspace; correct the selected transport. |
| `workspace_stale` | Checkout changed or is busy; stop competing pull/publish work. |
| `workspace_conflict` | Draft base differs from Git; pull, resolve the diff, validate, republish. |
| `git_unavailable` | Remote, path, network, or lock unavailable; preserve the degraded valid snapshot. |
| `git_auth_failed` | Mounted SSH/HTTPS material rejected/unreadable; rotate or fix permissions without logging it. |
| `git_non_fast_forward` | Checkout diverged; reconcile through the registry workflow. |
| `git_push_rejected` | Remote policy rejected the change; review branch protection/hooks. |
| `connector_unavailable` | DNS/TLS/auth/resource identity diagnostic failed; inspect local bindings and egress. |
| `semantic_index_incompatible` | Collection/model/dimensions/distance differs from Git; perform an explicit index migration. |

On macOS, restart Docker Desktop if a named volume disappears. On Windows, check WSL2 and Docker
file sharing. A failed first bootstrap has no snapshot fallback: repair remote trust and retry;
never create an unreviewed local registry repository.