docs: design Project A auth runtime projection

2026-08-21 20:23:29 +02:00
parent 042af932ee
commit a21e2c156d
1 changed files with 424 additions and 0 deletions
@@ -0,0 +1,424 @@
 # Project A Server Authentication Runtime Projection Design
 **Date:** 2026-08-21
 **Status:** approved
 **Scope:** Project A server-only authentication storage and publication
 **Source baseline:** `042af932ee7b686e8435aa1c5857a12c225e03c3`
 ## 1. Purpose
 Project A must use local ThothII authentication without creating a host account for the image's
 numeric identity `10001:10001`. The host operator runs `tht` through `sudo`, while the core runs in
 the container as `10001:10001`. Current code requires the authentication directory and files to be
 owned by the effective UID of the reader. A single physical directory therefore cannot be both the
 root-operated host store and the container-readable runtime store.
 This design separates one authoritative, root-only canonical store from a read-only runtime
 projection. Every successful `tht auth` mutation publishes and verifies a complete runtime
 generation automatically. The core never reads the canonical store and never observes a mixed
 `auth.yaml`/`users.yaml` pair.
 The change is opt-in and server-only. Existing Mac, Windows, and local Linux installations without
 a runtime projection retain their current files, CLI grammar, ownership rules, and behavior.
 ## 2. Non-goals and authorization boundary
 This source change does not:
 - start Project A or create any Project A container, network, or volume;
 - modify or reload Nginx;
 - stop, alter, or remove the legacy ThothII stack;
 - change DWH, ETL, Supabase, Authentik, Aritmolab, `dwh-auth`, or another shared service;
 - create a host user or group for numeric identity `10001`;
 - migrate legacy sessions, Qdrant data, Ollama data, or application settings;
 - expose passwords, password hashes, complete user records, or authentication YAML in logs.
 Implementation and automated verification use only the isolated source worktree and temporary
 fixtures. Applying the new contract to `/srv/thothii` remains a separate, explicitly authorized
 pre-start operation.
 ## 3. Installation contract
 The installation descriptor gains an optional nested runtime projection:
 ```yaml
 profile: server
 authentication:
  configDirectory: /srv/thothii/operator/auth
  runtimeProjection:
    directory: /srv/thothii/secrets/auth-runtime
    uid: 10001
    gid: 10001
 ```
 `authentication.configDirectory` remains the canonical store and must equal
 `THT_AUTH_CONFIG_ROOT`. When `runtimeProjection` is present:
 - the installation profile must be `server`;
 - the host platform must be Linux for mutation and publication operations;
 - `directory` must be an absolute canonical path distinct from `configDirectory`;
 - `uid` and `gid` must both equal decimal `10001`;
 - `directory` must equal `THT_AUTH_RUNTIME_ROOT` from the installation environment;
 - `sudo tht` is the only supported writer and publisher;
 - failure to establish effective UID 0 before a mutation or publication fails before changing
  canonical or runtime state.
 Projected server `tht setup`, including `--configure-only`, must also run with effective UID 0. It
 creates the canonical root and files directly as `root:root 0700/0600`; it must not first create an
 operator-owned store that a later `sudo tht` invocation cannot validate.
 When `runtimeProjection` is absent, `THT_AUTH_RUNTIME_ROOT` must also be absent and the existing
 single-directory behavior remains unchanged.
 The descriptor exposes separate accessors for the canonical authentication directory and the
 optional runtime-projection specification. Generic host validation continues to validate the
 canonical store against the effective host UID. Pre-start commands for a projected server
 additionally require a ready runtime projection consistent with the canonical revision.
 ## 4. Filesystem model
 ### 4.1 Canonical store
 The authoritative store remains compatible with the existing host auth implementation:
 ```text
 <configDirectory>/
  auth.yaml
  users.yaml                 # local mode only
  .auth.lock
  .auth-transaction.lock
 ```
 The directory is `root:root 0700`; all regular files are `root:root 0600`, have link count one,
 and are reached only through canonical, non-symlinked paths. `auth.yaml` and `users.yaml` retain
 their current strict schemas. The canonical store is the only data restored from an authentication
 backup and the only state accepted as authority by recovery.
 ### 4.2 Runtime projection
 The projection has this stable bind-mounted root:
 ```text
 <runtimeProjection.directory>/
  CURRENT
  generations/
    <generation-id>/
      manifest.json
      auth.yaml
      users.yaml             # local mode only
 ```
 The projection root, `generations`, and generation directories are `10001:10001 0700`. All regular
 files are `10001:10001 0600` with link count one. The privileged publisher sets these numeric IDs
 directly; it does not resolve or create a host account. The server Compose mount is read-only, so
 the core can read but cannot alter the projection even though its effective UID is `10001`.
 No symlink, hardlink, device, FIFO, socket, unexpected file, permissive mode, unexpected owner, or
 non-canonical path is accepted. Publishing uses descriptor-relative operations with no-follow
 checks, verifies the opened object before and after I/O, and fsyncs files and parent directories
 before making a generation ready.
 The runtime publisher and validator live in a projection-specific Linux package whose operations
 take explicit expected UID and GID values. The existing generic canonical `safeio` ownership rule
 continues to require owner equals effective UID and is not weakened to accommodate the projection.
 ### 4.3 Generation identity
 For each canonical snapshot, the publisher computes lowercase hexadecimal SHA-256 digests of the
 exact bounded bytes of `auth.yaml` and, in local mode, `users.yaml`. The generation ID is the
 lowercase SHA-256 of four UTF-8 lines joined by LF and terminated by one final LF:
 ```text
 thothii-auth-projection-v1
 mode=<local-or-oidc>
 auth=<64-lowercase-hex>
 users=<64-lowercase-hex-or-dash>
 ```
 The generation directory name is the 64-character generation ID. For a projected installation,
 the canonical revision is exactly `sha256:<generation-id>`; no second revision algorithm exists.
 Its strict `manifest.json` contains only schema version `1`, generation ID, mode, that canonical
 revision, exact filenames, byte sizes, and per-file SHA-256 digests. JSON is emitted canonically
 with a trailing newline and no unknown or duplicate fields. Non-projected installations retain the
 existing revision calculation and output for compatibility.
 Generation files are immutable after publication. An existing directory with the expected ID is
 reused only after complete owner, mode, type, size, content-digest, manifest, and canonical-revision
 verification. Any mismatch is an integrity failure; it is never repaired in place.
 `auth publish` can recover a corrupt target generation only while `CURRENT` is blocked. It
 revalidates the exact directory identity and performs a bounded descriptor-relative deletion of
 the whole target without following links, then publishes a newly staged complete directory under
 the expected generation ID. It never edits a published generation file in place. Deletion requires
 the target directory and every traversed entry to remain confined, owned by `10001:10001`, and to
 have the exact allowed directory/file type, mode, link count, and bounded name. File digests are
 checked when readable but are not required to match when digest failure is the reason the whole
 generation is being replaced. Any structurally unsafe or unrecognized entry is not deleted
 automatically and keeps the command in a sanitized integrity-failure state.
 Publisher-only temporary names are closed and bounded:
 - `generations/.stage-<transaction>-<generation>` is the only staging-directory form;
 - `.current-<transaction>.tmp` is the only selector temporary-file form;
 - no quarantine namespace exists.
 After acquiring the transaction lock and before new staging, `auth publish` scans only these two
 namespaces. A stage is automatically removed only when its transaction equals the transaction in a
 persisted blocked selector and the complete bounded no-follow ownership/type/mode/link validation
 passes. A selector temporary file may be removed for any syntactically valid transaction ID only
 after the same regular-file validation and strict bounded JSON decoding pass; this covers a crash
 before the blocked-selector rename. Entries with another name, an unrelated stage transaction,
 unsafe metadata, excessive contents, or an invalid document remain untouched and keep recovery
 blocked for explicit operator investigation. Recovery then creates a fresh transaction ID and
 rebuilds its stage from the canonical store.
 ## 5. Atomic selection and fail-closed state
 `CURRENT` is the only mutable runtime selector. It is a bounded, strict JSON document published by
 same-filesystem temporary-file creation, file fsync, atomic rename, and runtime-root fsync.
 A ready selector contains exactly:
 ```json
 {
  "version": 1,
  "state": "ready",
  "transaction": "<32-lowercase-hex>",
  "generation": "<64-lowercase-hex>",
  "previousGenerations": ["<zero-to-two-generation-ids>"]
 }
 ```
 A blocked selector contains exactly:
 ```json
 {
  "version": 1,
  "state": "blocked",
  "transaction": "<32-lowercase-hex>"
 }
 ```
 Transaction IDs are 128 random bits generated by the operating system. Missing, unsafe, malformed,
 unknown-version, `blocked`, or internally inconsistent selectors make runtime authentication
 unavailable. Only host `tht` can determine that a structurally valid projection is stale relative
 to the unmounted canonical store. The backend never falls back to the canonical directory, an
 older generation, an environment password, or upstream authentication.
 The backend runtime provider reads and validates `CURRENT`, resolves only a safe generation name
 under `generations`, and loads a complete immutable in-memory snapshot before returning: manifest,
 strict authentication configuration and, in local mode, the parsed local-user registry. All
 declared file digests and the generation identity are verified as part of that one load. The
 projected local-registry resolver consumes the in-memory registry associated with the exact loaded
 authentication snapshot; it never reopens `users.yaml` later. A request may therefore finish
 against the coherent ready generation it captured immediately before a transition, even if that
 generation is later removed. Every later request sees either `blocked` or the new coherent
 generation.
 The provider may cache an already verified complete immutable snapshot by its storage identity. It
 must re-read `CURRENT` for each authentication snapshot and must not cache a `blocked` or invalid
 state as ready. The in-memory snapshot is not serialized, logged, or exposed through diagnostics.
 ## 6. Automatic publication transaction
 The following commands are mutations:
 - `auth configure --mode local|oidc`;
 - `auth user add`;
 - `auth user set-password`;
 - `auth user enable` and `auth user disable`;
 - `auth user grant` and `auth user revoke`;
 - `auth user logout-all`.
 With a runtime projection configured, each mutation performs one serialized transaction:
 1. Validate descriptor, canonical and runtime parents, ownership policy, and root privilege. For
   initial configure or first publication, create missing roots with their exact required numeric
   owners and modes before any canonical credential is written.
 2. Acquire `.auth-transaction.lock` in the canonical root. The lock covers the complete operation
   and all publication or recovery work; existing `.auth.lock` remains the inner local-registry
   mutation lock.
 3. Load and verify the prior canonical revision when one exists. Except for initial configure,
   verify that the prior ready projection equals that canonical revision.
 4. Atomically publish a new `blocked` selector with a fresh transaction ID.
 5. Execute the existing canonical mutation and validate its complete resulting state.
 6. Stage the complete runtime generation in the runtime root, set exact numeric ownership and
   modes, fsync it, rename it to its immutable generation ID, and verify it through the same reader
   used by `auth publish`.
 7. Construct history as the previous ready generation followed by its prior history, de-duplicated
   and truncated to two entries. Atomically publish `CURRENT` as `ready` for the new generation.
 8. Re-read `CURRENT` and the complete generation, compare the runtime canonical revision with the
   current canonical revision, and return success only on exact equality.
 9. Remove generations not named by current plus its two-entry history, without following links.
   Cleanup occurs only after the new ready state is verified. A cleanup error is reported as an
   integrity failure but never rolls the selector back to an older credential state.
 If the canonical mutation fails and the publisher proves that canonical bytes are unchanged, it
 restores and verifies the previously saved ready selector. If canonical state changed or cannot be
 proved unchanged, `CURRENT` remains `blocked` and the command fails with a sanitized recovery
 instruction. If publication or final verification fails after a successful canonical mutation,
 `CURRENT` remains `blocked`.
 No mutating command reports success while canonical and runtime states differ.
 ## 7. CLI behavior and recovery
 ### 7.1 `tht auth publish`
 `auth publish` accepts no password and no content arguments. It acquires the transaction lock,
 treats the validated canonical store as authoritative, publishes `blocked`, builds or reuses the
 matching immutable generation, selects it as ready, verifies equality, applies bounded retention,
 and exits successfully only when the runtime projection is coherent.
 It is the supported recovery command after interruption, failed publication, runtime tampering, or
 restoration of a protected canonical backup. It does not recover a password, select an arbitrary
 old generation, or modify canonical users.
 ### 7.2 `tht auth status`
 Existing output remains compatible. For projected server installations, text and JSON output add
 only projection state, generation ID, canonical revision, and equality status. It never emits YAML,
 password hashes, user records, transaction lock contents, or environment values.
 ### 7.3 `tht auth check`
 Before invoking the existing container diagnostic, `auth check` verifies:
 - canonical auth validity;
 - runtime selector and generation integrity;
 - equality of canonical and projected revisions;
 - exact expected owners and modes.
 Only after these checks pass does it run the existing core diagnostic against the runtime mount.
 Output remains bounded and sanitized. `--json` remains pristine JSON.
 ### 7.4 Pre-start commands
 `start`, `update --check-only`, and relevant `doctor` checks fail closed when a configured runtime
 projection is absent, blocked, invalid, or stale. Initial `auth configure` and `auth publish` remain
 available to create or repair it. Read-only auth status remains available to diagnose a blocked
 projection without exposing secrets.
 ### 7.5 Backup restore
 A restore archive can replace canonical `auth.yaml` or `users.yaml`, so it participates in the
 same projection transaction. After archive preflight identifies any
 `authentication-configuration` entry and before the first restore mutation, `tht restore` must:
 1. require projected-server root privilege and acquire `.auth-transaction.lock`;
 2. validate the prior canonical and ready projection;
 3. atomically select `blocked`;
 4. perform the existing checkpointed restore or its checkpoint recovery;
 5. validate the final canonical store, publish its generation, and verify equality before selecting
   `ready`.
 If candidate restore fails and checkpoint recovery succeeds, the recovered canonical state is
 published before authentication becomes ready. If restore, checkpoint recovery, publication, or
 verification cannot complete, `CURRENT` remains blocked. A restore archive with no canonical
 authentication entries retains its current lifecycle and does not block authentication. A
 non-projected installation retains all existing restore behavior.
 ## 8. Compose and backend integration
 The base/local Compose contract remains unchanged: local installations mount
 `THT_AUTH_CONFIG_ROOT` at `/run/thothii-auth` and use the existing direct-file provider.
 Projected server installations add a dedicated
 `deploy/compose.auth-runtime-projection.yaml` override after the base profile and all declared
 operator overrides, but before the generated current-image override. `Installation.ComposeFiles()`
 inserts this file automatically whenever `runtimeProjection` is declared. The descriptor loader
 rejects manually listing the dedicated override, listing it without a projection, or otherwise
 duplicating it. This ordering and automatic selection prevent an operator override from silently
 restoring the canonical mount. Non-projected server installations do not include the new override
 and retain their existing direct-file behavior.
 The projection override:
 - requires host `THT_AUTH_RUNTIME_ROOT` as the bind source;
 - mounts it read-only at `/run/thothii-auth` for `core` only;
 - sets container environment `THT_AUTH_RUNTIME_PROJECTION_ROOT` to the literal
  `/run/thothii-auth`;
 - does not mount the canonical root anywhere;
 - does not add authentication environment or mounts to `workspace-maintenance`.
 `THT_AUTH_RUNTIME_ROOT` is only the host-side Compose bind source.
 `THT_AUTH_RUNTIME_PROJECTION_ROOT` is only the container-side provider selector and path. When the
 container variable is present, backend startup and every auth snapshot use the projected provider;
 the inherited `THT_AUTH_CONFIG_FILE` direct path is ignored and must retain its reviewed default
 value. A conflicting direct-file value is a configuration error. When the container variable is
 absent, the existing `THT_AUTH_CONFIG_FILE` behavior is unchanged. There is no runtime fallback
 between the two providers.
 ## 9. Security properties
 The implementation must preserve these invariants:
 - root is the sole canonical writer and runtime publisher on a projected server installation;
 - numeric `10001:10001` is a filesystem/container identity, not a new host account;
 - the core receives no write-capable auth mount;
 - canonical password hashes are copied only into a protected runtime generation and never logged;
 - a stale projection cannot silently authorize after a canonical security mutation fails to
  publish;
 - a selector cannot escape the runtime root or select a partial generation;
 - concurrent CLI mutations are serialized across both canonical mutation and publication;
 - interrupted work leaves either the prior verified ready selector, when canonical state is proven
  unchanged, or a persistent blocked selector;
 - recovery always derives runtime state from the canonical store;
 - local and cross-platform non-projected authentication remains behaviorally compatible.
 ## 10. Testing strategy
 Development follows strict test-driven development: each behavior is introduced by a focused test
 that fails for the intended reason before production code changes.
 Required automated coverage includes:
 1. Descriptor validation for valid server projection and rejection of local profile, non-Linux
   mutation, non-canonical/equal paths, missing environment match, and UID/GID other than `10001`.
 2. Regression coverage proving descriptors without a projection retain current Mac, Windows, and
   local Linux behavior.
 3. Secure publication primitives: exact owners/modes, link count, no-follow behavior, bounded
   reads, fsync/rename ordering, strict pointer/manifest schemas, and digest verification.
 4. Successful initial local and OIDC publication and automatic publication after every local-user
   mutation.
 5. Failure injection before blocking, during canonical mutation, at each stage/fsync/rename, during
   final verification, and during retention; expected ready restoration or persistent blocked state
   is asserted explicitly.
 6. Concurrent `tht` processes proving one transaction at a time and no mixed canonical/runtime
   result; rapid successive publications prove an in-flight request finishes from its complete
   in-memory snapshot after the referenced generation is pruned.
 7. Backend projected-provider tests for ready, blocked, absent, stale, malformed, symlinked,
   hardlinked, permissive, wrong-owner, altered-manifest, altered-file, and generation-switch cases.
 8. Login and session tests covering ordinary/admin login, password changes, enable/disable,
   grant/revoke, and `logout-all` through immutable generations.
 9. Compose rendering tests proving server core mounts only the runtime projection read-only and
   `workspace-maintenance` has no authentication mount or environment; automatic override ordering,
   duplicate refusal, and both host/container environment names are exact.
 10. Restore tests proving authentication-bearing candidate restore, checkpoint recovery, restore
    failure, and publication failure cannot leave an old ready projection; archives without auth
    retain existing behavior.
 11. Full Go tests with race detection and vet, backend Vitest plus TypeScript typecheck, Compose
    contract tests, documentation gates, and secret scanners using only synthetic credentials.
 No automated test may inspect the real `/srv/thothii` auth material, start the Project A stack, or
 mutate a live service.
 ## 11. Documentation and later operator gate
 The implementation updates server installation examples, local-auth documentation, Project A
 preflight instructions, recovery guidance, and tests that enforce those documents. The operator
 guide explains:
 - the canonical/projection distinction in plain language;
 - why no host account `10001` is created;
 - exact descriptor and environment fields;
 - automatic publication semantics;
 - the meaning of `blocked` and recovery with `sudo tht auth publish`;
 - backup and restoration of the canonical store only;
 - safe verification without printing credentials or hashes.
 After source implementation, review, and verification pass, applying the contract to Project A
 requires a new bounded pre-start authorization. That later gate will restore the existing generated
 auth data to canonical `root:root 0700/0600`, create the runtime projection through `sudo tht auth
 publish`, run read-only preflights, and stop before starting the new stack unless the owner provides
 separate start authorization.