ThothII/docs/workspace-diagnostic-protocol.md

# Workspace diagnostic protocol

This is the operator contract for testing a workspace on one ThothII installation. The Git-shared
descriptor declares what can be checked; the installation supplies only the selected DWH
transport and local secret-file bindings. No secret value, certificate content, SSH key, or
response body belongs in the descriptor, generated `.env.example` files, or diagnostic output.

## Scope and safety rules

- Schema v3 is the only accepted workspace descriptor format.
- Schema v1 and v2 descriptors are rejected before diagnostics run. There is no in-product
  migrator or automatic conversion; the Git repository must already contain reviewed v3
  descriptors.
- One workspace owns one Qdrant collection.
- Qdrant and Ollama are internal services. Operators do not bind external vector or embedding
  transports for active manuals or supported diagnostics.
- Each diagnostic is bounded by the configured timeout. Redirects are rejected, response bodies
  stay inside the adapter, and browser-visible errors are limited to `binding_missing`,
  `connector_unavailable`, and `semantic_index_incompatible`.

## Canonical descriptor contract

```yaml
workspace:
  schema_version: 3
  id: psd-clinical
  name: PSD Clinical
  language: it

dwh:
  engine: postgres
  database: warehouse
  schema: datawarehouse
  supported_transports: [postgres_direct, rest_api, ssh_tunnel]

semantic_index:
  vector_store:
    engine: qdrant
    collection: psd-clinical
    dimensions: 1024
    distance: cosine
  embedding:
    provider: ollama_internal
    model: qwen3-embedding:0.6b
    dimensions: 1024

diagnostics:
  dwh_rest:
    method: POST
    path: /rpc/ping
    auth: bearer
    response: { database: database, schema: schema }
```

The semantic-index contract is fixed:

- `engine: qdrant`
- collection name equals the workspace-owned portable identifier
- `qwen3-embedding:0.6b`
- `1024` dimensions
- cosine distance

If any active collection reports a different model pairing, dimension, or distance, diagnostics
must return `semantic_index_incompatible` rather than silently rewriting data.

## Installation-local variable contract

Replace `<NAMESPACE>` with the immutable workspace ID converted to upper case with hyphens changed
to underscores. For example, `psd-clinical` becomes `PSD_CLINICAL`. Set only the variables for the
selected DWH transport. Every `*_FILE` value is an absolute path to a regular, readable file
inside an approved local secret root; it is never the secret itself.

| Connector and transport | Required local variables |
| --- | --- |
| DWH selection | `THT_WS_<NAMESPACE>_DWH_TRANSPORT` |
| DWH `postgres_direct` | `THT_WS_<NAMESPACE>_DWH_HOST`, `THT_WS_<NAMESPACE>_DWH_PORT`, `THT_WS_<NAMESPACE>_DWH_USER`, `THT_WS_<NAMESPACE>_DWH_PASSWORD_FILE`; optional `THT_WS_<NAMESPACE>_DWH_TLS_CA_FILE` |
| DWH `rest_api` | `THT_WS_<NAMESPACE>_DWH_BASE_URL`; `THT_WS_<NAMESPACE>_DWH_API_KEY_FILE` only for `bearer`/`x-api-key`; optional `THT_WS_<NAMESPACE>_DWH_TLS_CA_FILE` |
| DWH `ssh_tunnel` | `THT_WS_<NAMESPACE>_DWH_USER`, `THT_WS_<NAMESPACE>_DWH_PASSWORD_FILE`, `THT_WS_<NAMESPACE>_DWH_SSH_HOST`, `THT_WS_<NAMESPACE>_DWH_SSH_PORT`, `THT_WS_<NAMESPACE>_DWH_SSH_USER`, `THT_WS_<NAMESPACE>_DWH_SSH_PRIVATE_KEY_FILE`, `THT_WS_<NAMESPACE>_DWH_SSH_KNOWN_HOSTS_FILE`, `THT_WS_<NAMESPACE>_DWH_SSH_TARGET_HOST`, `THT_WS_<NAMESPACE>_DWH_SSH_TARGET_PORT`; optional `THT_WS_<NAMESPACE>_DWH_TLS_CA_FILE` |

There are no supported `THT_WS_<NAMESPACE>_VECTOR_*` or
`THT_WS_<NAMESPACE>_EMBEDDING_*` installation bindings in the active operator contract.

## DWH diagnostic

For direct PostgreSQL and SSH-tunnelled PostgreSQL, the diagnostic connects with the declared
`dwh.database`, checks TLS and authentication, then executes exactly:

```sql
SELECT current_database() AS database, current_schema() AS schema
```

Both returned values must equal the descriptor's DWH database and schema.

For REST, the descriptor-declared request is for example:

```text
POST <THT_WS_<NAMESPACE>_DWH_BASE_URL>/rpc/ping
Authorization: Bearer <content of DWH_API_KEY_FILE>
```

It has no request body. A 2xx response must be a JSON object whose declared `database` and
`schema` fields match the descriptor.

## Internal semantic-service diagnostic

Schema-v3 workspace diagnostics also verify the internal semantic infrastructure through backend
configuration:

- Qdrant must be reachable at the installation-owned internal URL.
- The workspace-owned collection must exist or be creatable with `1024` dimensions and cosine
  distance.
- Ollama must provide `qwen3-embedding:0.6b`.
- A bounded embed probe must return exactly `1024` dimensions.

These checks use the private Compose services and never require operator-supplied vector or
embedding URLs, transports, or credentials.

## SSH host verification and tunnel lifecycle

For DWH `ssh_tunnel`, the known-hosts file is mandatory and is verified before a connection is
accepted. The tunnel is a short-lived loopback forward for the diagnostic only. The effective
OpenSSH constraints are:

```text
-N -v
-o BatchMode=yes
-o ExitOnForwardFailure=yes
-o StrictHostKeyChecking=yes
-o UserKnownHostsFile=<ROLE>_SSH_KNOWN_HOSTS_FILE
-i <ROLE>_SSH_PRIVATE_KEY_FILE
-p <ROLE>_SSH_PORT
-L 127.0.0.1:<ephemeral-port>:<ROLE>_SSH_TARGET_HOST:<ROLE>_SSH_TARGET_PORT
<ROLE>_SSH_USER@<ROLE>_SSH_HOST
```

The local listener is `127.0.0.1` only. The process is terminated in cleanup after the direct
probe, on timeout, or on failure.

In this release, `ssh_tunnel` remains a diagnostic-only DWH transport. A successful probe is
followed by `workspace_not_activatable`, and `POST /sessions` rejects the workspace before
persisting a manifest or starting Pi. This restriction does not apply to SSH transport for the
workspace Git remote.

## Reader-only fallback

A workspace may be fully valid in Git but non-activatable locally when a required DWH binding,
secret file, host verification, TLS check, or declared DWH diagnostic fails. That state does not
alter the shared descriptor and does not permit a new session on that installation. It may still
be published and activated elsewhere with valid local bindings.