ThothII/docs/superpowers/specs/2026-08-20-dwh-rest-per-installation-auth-design.md

# DWH REST Per-Installation Authentication — Design

**Date:** 2026-08-20

**Status:** Approved by the owner

## Purpose

Replace the single static `X-API-Key` currently protecting the PSD `/dwh/` route with a reusable,
server-side authentication component that assigns one independently revocable credential to each
ThothII installation.

The design must preserve the existing portable connector contract. A ThothII installation on
macOS, Windows, or Linux continues to send `X-API-Key` over HTTPS when its selected transport is
`rest_api`. Installations using `postgres_direct` or `ssh_tunnel` are unaffected.

The component belongs in the ThothII source repository so that it can be reused on another DWH
server, but it is optional server infrastructure. It is not part of the portable application
runtime and is not started or stopped by `tht`.

## Confirmed context

- The exposed credential must be considered compromised and must eventually be revoked.
- The current PSD route accepts one shared static `X-API-Key` through Nginx `auth_request`.
- The Mac installation must continue to access PSD through REST.
- Future remote ThothII installations will also use REST and cannot depend on an SSH tunnel.
- The new ThothII installation on the PSD server will use `postgres_direct` with a dedicated,
  demonstrably read-only DWH role.
- The current self-issued TLS certificate remains in use. Clients that do not already trust its
  issuer require a separately delivered `TLS_CA_FILE` and must verify the approved fingerprint.
- There is no separate test environment. This is accepted and is not itself a blocker.
- Existing ThothII sessions, derived indexes, and model caches are test data. They are not migrated
  or protected by session-specific backup work.
- The legacy stack remains unchanged until the deployment program reaches its explicit stop gate.
  This is an authorization boundary, not a session-preservation requirement.

## Scope

This design includes:

- the reusable `dwh-auth` service and its administrative command;
- a protected file-based credential registry;
- Nginx `auth_request` integration;
- one credential identity per ThothII installation;
- creation, delivery, rotation, optional expiry, and revocation;
- a dual-key transition from the exposed shared credential;
- generic and PSD-specific operator documentation;
- automated tests and production-safe acceptance checks.

This design does not:

- change the ThothII REST header or connector protocol;
- add DWH credential administration to the portable `tht` CLI;
- place server credentials in a workspace repository;
- replace PostgreSQL grants, RLS, or the PostgREST read-only role boundary;
- preserve or migrate legacy ThothII sessions, Qdrant indexes, or Ollama caches;
- change the current certificate or resolve the separate `.it` versus `.com` public-origin issue;
- authorize any current server, Nginx, database, or service mutation.

## Alternatives considered

### Extend the portable `tht` CLI

This would provide a single operator command surface, but it would distribute server-only DWH
administration code to every macOS and Windows installation. The DWH route also has a lifecycle
independent from the local ThothII application. This option is rejected.

### Maintain a manual Nginx key map

This is initially small, but it encourages plaintext credentials in Nginx configuration and makes
atomic rotation, redacted inventory, and reliable revocation harder. This option is rejected.

### Use SQLite

SQLite is not a declared project dependency and is unnecessary for the expected registry size and
write rate. Introducing a database would add packaging, migration, backup, and corruption-recovery
work without improving the required contract. This option is rejected.

### Selected approach

Add a standalone, standard-library Go component with a protected file registry. On PSD it runs as
an independent `systemd` service and communicates with host Nginx through a Unix socket.

## Component and lifecycle boundary

The repository will contain a separate component, expected under `tools/dwh-auth/`, plus generic
deployment templates and documentation. Its build and tests are isolated from the portable
application entrypoints.

On PSD:

- `dwh-auth` runs under a dedicated, unprivileged system account;
- `systemd` owns its lifecycle;
- the service is not included in the canonical ThothII Compose stack;
- the registry resides outside both the Git checkout and ThothII application data;
- the service receives read-only access to active credential records;
- Nginx is the only runtime caller and reaches it through a permission-restricted Unix socket;
- stopping, updating, or replacing ThothII does not interrupt `/dwh/` authentication.

The same source may be deployed on another Linux DWH server. Platform-specific service packaging
beyond the approved PSD `systemd` deployment is not required by this implementation.

## Credential model

One credential identifies one ThothII installation, not one human user. An installation may have
more than one temporarily active generation during rotation.

The external key contains a version marker, a public key ID, and a random secret:

```text
thtdwh_v1.<public-key-id>.<random-secret>
```

Requirements:

- the public key ID is generated by the tool and contains only a strict safe alphabet;
- the secret is generated with the operating system cryptographic random source;
- the secret has at least 256 bits of entropy and uses an unambiguous transport-safe encoding;
- the complete header has a fixed maximum length;
- installation IDs are operator-provided, non-secret, unique labels and must not contain personal
  or clinical information;
- descriptions are optional non-secret operator metadata.

Because the secret is a uniformly random 256-bit value rather than a human password, the registry
stores a SHA-256 digest and verifies it with a constant-time comparison. No password hashing or
external cryptographic runtime dependency is needed.

## Protected file registry

The registry has versioned active and revoked records:

```text
<registry-root>/
  active/<public-key-id>.json
  revoked/<public-key-id>.json
```

An active record contains:

- schema version;
- public key ID;
- installation ID and optional description;
- encoded SHA-256 digest;
- creation timestamp;
- optional expiry timestamp, absent by default.

A revoked record additionally contains the revocation timestamp and a non-secret reason. The
original key never appears in either directory.

Multiple key IDs may refer to the same installation ID during a rotation. Creation uses an
exclusive temporary file, file flush, directory-safe atomic rename, restrictive ownership and
mode checks, and refusal of symlinks or unsafe paths. Revocation atomically moves a record from
`active` to `revoked` on the same filesystem. A malformed or unsafe record never authenticates.

The runtime service does not update registry files. Successful and failed use is recorded only in
the sanitized service journal, avoiding per-request writes and keeping the service's registry
access read-only.

## Administrative interface

The standalone binary provides a server-only administrative surface equivalent to:

```text
dwh-auth key create --installation-id ID --description TEXT --output ABSOLUTE_FILE
dwh-auth key import --installation-id ID --from-file ABSOLUTE_FILE
dwh-auth key list [--json]
dwh-auth key revoke --key-id ID --reason TEXT
dwh-auth key status --key-id ID [--json]
dwh-auth check
```

The exact syntax will be frozen in the implementation plan and tests. The interface must obey
these invariants:

- secrets are never accepted as command-line values;
- `create` writes the generated credential once to a new absolute file with restrictive access;
- `import` reads an existing protected file without displaying its content;
- normal stdout contains only non-secret identifiers, abbreviated fingerprints, status, and
  paths;
- JSON output is pristine and contains no credential value or digest;
- an existing output file is never overwritten;
- listing and status commands never expose digests;
- errors and logs pass explicit secret-leak regression tests.

## Provisioning and delivery

The operator creates one key for each installation and transfers it through an approved protected
channel, such as an organizational password manager, a secret manager or MDM, or authenticated
file transfer when the recipient has suitable access. Email, ordinary chat, and ticket bodies are
not approved delivery channels.

On the recipient installation:

- the normal interactive path saves the value through authenticated Workspace management, where
  ThothII keeps it in the installation-local encrypted workspace vault;
- a supported headless deployment may place it in a protected file referenced by `API_KEY_FILE`;
- the credential never enters the shared workspace repository or ordinary environment values;
- the private CA is delivered separately and configured with `TLS_CA_FILE` when the certificate
  chain is not already trusted;
- the operator verifies the documented certificate fingerprint before trusting the CA file.

After configuration, the recipient runs the existing workspace connection test against the
harmless `/rpc/ping` diagnostic. Only after the server and recipient both confirm the public key
ID may the one-time delivery copy be removed according to the organization's secret-handling
procedure.

## Expiry and rotation

Keys have no automatic expiry by default. This avoids unplanned outages for remote installations
that may not be continuously administered. An optional expiry timestamp is supported for sites
whose policy requires it; an expired key is rejected without fallback. Administrative status
output warns about approaching expiries without exposing secrets.

Normal zero-downtime rotation is:

1. create a second key generation for the installation;
2. deliver and configure it on the client;
3. verify `/rpc/ping` and the sanitized server key ID;
4. revoke the earlier key;
5. prove that the revoked key receives `401`;
6. remove temporary delivery material after recipient confirmation.

## Request flow and Nginx contract

```text
remote ThothII
  -> HTTPS /dwh/ with X-API-Key
  -> Nginx auth_request subrequest
  -> dwh-auth over a Unix socket
  -> valid: Nginx proxies the original request to internal PostgREST
  -> invalid or revoked: Nginx returns 401
  -> authenticator fault: Nginx returns 503
```

`dwh-auth` authenticates only. It does not proxy the original request, read a clinical response,
or connect to PostgreSQL. PostgREST remains inaccessible as a public direct backend.

Nginx integration must:

- forward only the bounded credential header to the internal verification endpoint;
- suppress the original request body in the auth subrequest;
- discard client-supplied identity or audit headers;
- accept only the authenticator's success result;
- return the same generic `401` for missing, malformed, unknown, expired, and revoked keys;
- map authenticator or registry faults to `503`, never to permissive access;
- apply bounded failure-rate protection;
- log only timestamp, result, request metadata already approved for the route, and the verified
  public key ID;
- preserve the existing PostgREST path and response contract for successful requests.

The authenticator returns a verified public key ID only after successful authentication. Nginx may
use that value in its sanitized access log but does not need to forward it to PostgREST.

## Fail-closed behavior

The following conditions deny access:

- header absent, duplicated, oversized, malformed, or encoded incorrectly;
- unknown public key ID;
- digest mismatch;
- revoked or expired record;
- unsafe permissions, symlink, malformed JSON, schema mismatch, or inconsistent record identity;
- unreadable registry;
- service startup or runtime failure.

Credential failures return a generic `401`. Infrastructure failures return `503` after the Nginx
mapping is applied. Neither response reveals whether an installation ID exists. Health reporting
is local-only and does not weaken the authentication decision.

## PSD dual-key transition

The exposed shared credential is represented temporarily as `legacy-shared`. It is imported only
from an approved protected source and is never placed in a command argument, terminal output,
document, or evidence file. Because the existing value does not use the new versioned key
format, it is stored as the only permitted `legacy_raw` record. The service compares the digest of
the complete opaque legacy header only for that reserved record. After `legacy-shared` is revoked,
no unversioned credential is accepted.

The production route does not switch to the new authenticator until all of the following hold:

1. protected backup and exact rollback instructions exist for the affected Nginx and service
   configuration;
2. the new service passes local synthetic checks while disconnected from the public route;
3. `legacy-shared` passes `/rpc/ping` through the candidate authentication path;
4. a new per-installation Mac key passes the same diagnostic;
5. a random invalid key is rejected;
6. `nginx -t` passes and the authorized operator approves reload.

After the Mac uses its new credential successfully for the agreed observation window,
`legacy-shared` is revoked. Acceptance requires a positive test with the new key and a negative
test proving `401` with the legacy key. The new secret must not appear in service, Nginx, shell, or
application logs.

Rollback during the dual-key window restores only the reviewed authentication route and service
configuration needed to keep REST clients working. It does not preserve or restore legacy
ThothII sessions, indexes, or model caches.

## PostgreSQL authorization boundary

Successful API-key authentication is not proof of read-only database authorization. The existing
PostgREST role, exposed schemas, function privileges, default privileges, and direct network
reachability remain subject to the catalog-only survey activity. Project execution cannot treat
`dwh-auth` as a substitute for a demonstrably read-only `dwh_reader` boundary.

## Verification strategy

Automated tests use synthetic credentials and no clinical data. They cover:

- key generation entropy and syntax;
- create, import, list, status, optional expiry, rotation, and revocation;
- constant-time digest verification;
- missing, duplicate, malformed, oversized, unknown, expired, and revoked credentials;
- corrupt records, unsafe modes, symlinks, path traversal, and partial files;
- atomic creation and revocation under concurrent reads;
- clean failure when the registry or socket is unavailable;
- absence of credential values and digests in human output, JSON, errors, and logs;
- Nginx `auth_request` integration against a synthetic upstream and `/rpc/ping` fixture;
- `401` for credential failures and `503` for infrastructure failures;
- unchanged successful request path and response body;
- independent service lifecycle from the ThothII stack.

Portability regression checks prove that:

- the existing REST connector still sends the same `X-API-Key` header;
- `postgres_direct` and `ssh_tunnel` configuration remain unchanged;
- canonical local and server ThothII Compose rendering does not acquire `dwh-auth`;
- macOS and Windows installation contracts do not require the new binary or files;
- existing `tht`, backend, frontend, and harness gates remain green in proportion to the touched
  source boundaries.

There is no separate PSD test environment. Local synthetic and integration tests precede a
bounded, reversible production rollout using only `/rpc/ping`. The absence of a test environment
does not trigger preservation work for legacy test sessions.

## Documentation deliverables

Implementation is incomplete until the following clear, secret-free documents exist and pass
their documentation checks:

1. a generic server installation and administration guide for `dwh-auth`;
2. a generic enrollment guide for issuing a key to a macOS, Windows, Linux, or headless ThothII
   installation;
3. exact GUI and `API_KEY_FILE` configuration paths;
4. a TLS guide covering `TLS_CA_FILE`, trusted delivery, fingerprint verification, renewal, and
   coordinated client updates;
5. a PSD runbook for backup, installation, legacy import, dual-key activation, Mac update,
   observation, revocation, negative testing, and rollback;
6. a troubleshooting table for `401`, `503`, TLS failures, revoked keys, and unreadable registry
   records;
7. sanitized evidence templates that never request secret values or digests;
8. links from the existing local, server, workspace, and final unified user manuals.

Examples use synthetic hostnames, IDs, fingerprints, and credentials. No real key, certificate
body, password, connection string, or clinical result is committed.

## Acceptance criteria

The design is implemented only when:

- each REST installation can be identified and revoked independently;
- no plaintext server credential is stored in the registry;
- the portable ThothII connector contract is unchanged;
- the PSD ThothII server continues to select `postgres_direct`;
- stopping ThothII does not stop `dwh-auth` or `/dwh/` authentication;
- the authenticator fails closed and its logs are sanitized;
- the dual-key production procedure proves both positive and negative outcomes;
- the exposed legacy credential is demonstrably rejected;
- the current TLS limitations are documented precisely rather than silently bypassed;
- database read-only authorization is independently verified;
- all documentation deliverables are reviewed and reproducible by an operator who did not write
  the implementation.

## Authorization gates

Approval of this design authorizes documentation and planning only. It does not authorize:

- reading or importing the legacy credential;
- creating real keys;
- installing a binary or `systemd` unit;
- writing the registry directory;
- modifying or reloading Nginx;
- querying or changing PostgreSQL;
- changing, stopping, or removing the legacy ThothII stack.

Those actions require the executable plan, the remaining survey gates, explicit mutation
authorization, and retained sanitized evidence.