diff --git a/docs/plans/2026-08-14-read-only-workspace-runtime-secrets-design.md b/docs/plans/2026-08-14-read-only-workspace-runtime-secrets-design.md new file mode 100644 index 00000000..1e7083bd --- /dev/null +++ b/docs/plans/2026-08-14-read-only-workspace-runtime-secrets-design.md @@ -0,0 +1,189 @@ +# Read-only Workspace Repository and Runtime Secrets Design + +**Date:** 2026-08-14 +**Status:** Approved + +## Purpose + +ThothII consumes workspaces from one administrator-configured Git repository. Workspace authors +prepare and publish source outside ThothII. The application fetches, validates, and activates +repository revisions, but never edits, commits, pushes, imports, or exports workspace source. + +Runtime credentials are intentionally absent from Git. After a workspace has been read, ThothII +derives the required credentials from its connector and authentication choices and lets an +authorized user complete them in the web application. The values are encrypted and persisted by +the backend; the browser retains neither workspace content nor secrets. + +## Ownership boundaries + +### Workspace source + +The workspace source is an ordinary directory maintained outside the ThothII runtime. It contains +the catalog, each `workspace.yaml`, curated evidence, annotations, and other repository-owned +content. Authors validate it using source-side tooling and publish it through their normal Git +workflow to GitHub, GitLab, Gitea, or another standards-compatible server. + +### ThothII installation + +The installation descriptor selects the Git remote, branch, and one read-only authentication +transport. SSH uses a read-only deploy key plus pinned known hosts. HTTPS uses a read-only deploy +token and may provide a private CA. Secret values remain outside versioned configuration. + +The installer performs a sanitized `git ls-remote` preflight. Credentials embedded in a remote URL +are rejected. The API exposes only a normalized repository identity: host, repository path, branch, +transport, active commit, and synchronization state. + +### ThothII runtime + +The local Git checkout, candidate validation area, immutable snapshots, and active state are +application-owned. They are read-only from the workspace-management API. A pull fetches a candidate +revision, validates the complete repository, and atomically activates it only if valid. A failed +candidate never replaces the last valid active revision. + +ThothII never generates or reconciles files back into the checkout and never invokes Git commit or +push. Generated operational artifacts live under application data, not in the source repository. + +## Repository synchronization states + +A repository refresh has these states: + +- `syncing`: fetching and validating a candidate revision; +- `active`: the candidate passed validation and became the active immutable revision; +- `invalid_candidate`: Git succeeded but repository validation failed; the previous revision stays active; +- `unavailable`: Git or authentication failed; the previous revision stays active; +- `empty`: no valid revision has ever been activated. + +Validation is atomic at repository-commit level. A malformed catalog, descriptor, evidence tree, or +cross-file reference rejects the complete candidate revision. + +## Runtime secret model + +### Requirement discovery + +The workspace descriptor contains connector type, authentication method, and non-secret logical +configuration. It never contains secret values or host filesystem paths. Connector adapters define +the secret fields required by each supported authentication method. For example: + +- PostgreSQL `username_password` requires `username` and `password`; +- REST `bearer` requires `api_key`; +- SSH tunnel authentication requires the connector password and SSH private key; +- Evidence HTTP signed URLs and static S3 credentials contribute their own secret requirements. + +Requirements have stable identifiers scoped by workspace and connector. Labels, descriptions, +input kinds, and required/optional status come from trusted application code rather than repository +HTML or executable metadata. + +### Persistent encrypted store + +The backend owns a `WorkspaceSecretStore` abstraction. The first implementation is a local encrypted +vault in application-managed persistent storage. Each secret is encrypted with authenticated +encryption and bound to its installation, workspace, connector, and field identifier as associated +data. Plaintext values never appear in Git, API responses, logs, error messages, diagnostics, or +browser storage. + +The installation bootstraps one vault key independently from workspace content. Deployment tooling +owns its platform-specific provisioning; the workspace schema and GUI never contain filesystem +paths. The storage interface allows a future Vault, cloud secret manager, or OS keychain provider +without changing workspace descriptors or API consumers. + +When an existing file-oriented harness connector needs a credential, the backend materializes it as +a restrictive temporary file in an application-owned runtime directory. Its lifetime is tied to the +diagnostic or runtime lease and it is removed on release. Persistent storage contains ciphertext +only. + +### Secret API + +For a selected workspace the API returns requirement metadata and status only: + +```json +{ + "workspaceId": "psd-clinical", + "state": "configuration_required", + "requirements": [ + { + "id": "dwh.password", + "connector": "dwh", + "label": "Database password", + "input": "password", + "required": true, + "configured": false + } + ] +} +``` + +A write request contains values only for the selected requirement identifiers. The response returns +status, never values. A delete operation forgets a configured value. Authorization is deliberately +deferred; the current authenticated application user may manage runtime workspace secrets. + +Workspace readiness is derived as follows: + +- `invalid`: repository structure or descriptor is invalid; +- `configuration_required`: structurally valid but required runtime values are missing; +- `ready`: required values exist but connectivity has not yet passed or is stale; +- `verified`: the most recent connector diagnostic passed for the active revision and current secret generation. + +Changing or deleting a secret invalidates the previous diagnostic result. + +## Browser behavior + +Workspace management is a two-level read-only interface occupying at least 60 percent of viewport +width and height. + +Level 1 explains the source/runtime separation and displays: + +- normalized repository host and path; +- configured branch and read-only transport; +- active revision and last synchronization result; +- `Update workspace repository`, which fetches, validates, and conditionally activates a revision; +- the workspace list, with selection required for workspace-specific actions. + +There is no Import bundle, Export bundle, Create, Edit, Delete, Publish, or conflict-resolution +operation. There are no browser-persisted workspace drafts or preferences. + +Level 2 for the selected workspace explains and displays: + +- immutable source identity and validation result; +- required runtime configuration grouped by connector; +- secret-entry controls whose values are write-only; +- `Save secrets`, `Forget` per configured value, and `Test workspace connection`; +- clear consequences for each button and a reminder that source changes must be committed and pushed + by an author outside ThothII before repository update. + +The browser keeps form values only in component memory and clears them after submission or dialog +close. It never receives saved secret values. + +## Compatibility and migration + +Existing Git author settings, publish endpoints, bundle endpoints, generated-document +reconciliation, bootstrap catalog slots, and browser draft storage are removed. Existing environment +bindings may be read during a bounded migration period only to seed non-secret connector values; +secret file paths are not part of the new public workspace contract. + +Session manifests continue to pin an immutable validated workspace revision. An already running +session keeps its acquired runtime lease; new or resumed work resolves the current encrypted secret +generation and fails closed when required credentials are unavailable. + +## Failure handling and security + +- Repository and vault errors use stable sanitized codes and never echo remotes with user info, + credential paths, secret identifiers that are not safe to disclose, or secret values. +- Vault writes are atomic and authenticated; corrupted ciphertext fails closed. +- Secret comparison uses no read API. Updating a secret is always a blind replacement. +- The backend applies request-size and field-count limits and rejects unknown requirement IDs. +- Temporary plaintext files use restrictive permissions, trusted directories, no-follow opens, and + deterministic cleanup. +- Git credentials are installation-only, read-only, and never sent to the frontend. + +## Verification + +Backend tests cover repository read-only behavior, atomic candidate activation, remote sanitization, +vault encryption and corruption, requirement discovery, blind secret writes/deletes, materialization +cleanup, readiness transitions, and absence of publish/bundle routes. + +Frontend tests cover the two-level explanation, viewport dimensions, repository identity, selection +gating, dynamic secret forms, write-only behavior, status changes, and absence of local-storage, +import, export, editing, and publishing controls. + +Deployment and CLI tests cover required remote/branch configuration, one read-only Git transport, +sanitized remote preflight, vault-key provisioning, and removal of Git author/write configuration.