docs: design Project A auth runtime projection
This commit is contained in:
User
@@ -0,0 +1,424 @@
|
||||
# Project A Server Authentication Runtime Projection Design
|
||||
|
||||
**Date:** 2026-08-21
|
||||
**Status:** approved
|
||||
**Scope:** Project A server-only authentication storage and publication
|
||||
**Source baseline:** `042af932ee7b686e8435aa1c5857a12c225e03c3`
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
Project A must use local ThothII authentication without creating a host account for the image's
|
||||
numeric identity `10001:10001`. The host operator runs `tht` through `sudo`, while the core runs in
|
||||
the container as `10001:10001`. Current code requires the authentication directory and files to be
|
||||
owned by the effective UID of the reader. A single physical directory therefore cannot be both the
|
||||
root-operated host store and the container-readable runtime store.
|
||||
|
||||
This design separates one authoritative, root-only canonical store from a read-only runtime
|
||||
projection. Every successful `tht auth` mutation publishes and verifies a complete runtime
|
||||
generation automatically. The core never reads the canonical store and never observes a mixed
|
||||
`auth.yaml`/`users.yaml` pair.
|
||||
|
||||
The change is opt-in and server-only. Existing Mac, Windows, and local Linux installations without
|
||||
a runtime projection retain their current files, CLI grammar, ownership rules, and behavior.
|
||||
|
||||
## 2. Non-goals and authorization boundary
|
||||
|
||||
This source change does not:
|
||||
|
||||
- start Project A or create any Project A container, network, or volume;
|
||||
- modify or reload Nginx;
|
||||
- stop, alter, or remove the legacy ThothII stack;
|
||||
- change DWH, ETL, Supabase, Authentik, Aritmolab, `dwh-auth`, or another shared service;
|
||||
- create a host user or group for numeric identity `10001`;
|
||||
- migrate legacy sessions, Qdrant data, Ollama data, or application settings;
|
||||
- expose passwords, password hashes, complete user records, or authentication YAML in logs.
|
||||
|
||||
Implementation and automated verification use only the isolated source worktree and temporary
|
||||
fixtures. Applying the new contract to `/srv/thothii` remains a separate, explicitly authorized
|
||||
pre-start operation.
|
||||
|
||||
## 3. Installation contract
|
||||
|
||||
The installation descriptor gains an optional nested runtime projection:
|
||||
|
||||
```yaml
|
||||
profile: server
|
||||
authentication:
|
||||
configDirectory: /srv/thothii/operator/auth
|
||||
runtimeProjection:
|
||||
directory: /srv/thothii/secrets/auth-runtime
|
||||
uid: 10001
|
||||
gid: 10001
|
||||
```
|
||||
|
||||
`authentication.configDirectory` remains the canonical store and must equal
|
||||
`THT_AUTH_CONFIG_ROOT`. When `runtimeProjection` is present:
|
||||
|
||||
- the installation profile must be `server`;
|
||||
- the host platform must be Linux for mutation and publication operations;
|
||||
- `directory` must be an absolute canonical path distinct from `configDirectory`;
|
||||
- `uid` and `gid` must both equal decimal `10001`;
|
||||
- `directory` must equal `THT_AUTH_RUNTIME_ROOT` from the installation environment;
|
||||
- `sudo tht` is the only supported writer and publisher;
|
||||
- failure to establish effective UID 0 before a mutation or publication fails before changing
|
||||
canonical or runtime state.
|
||||
|
||||
Projected server `tht setup`, including `--configure-only`, must also run with effective UID 0. It
|
||||
creates the canonical root and files directly as `root:root 0700/0600`; it must not first create an
|
||||
operator-owned store that a later `sudo tht` invocation cannot validate.
|
||||
|
||||
When `runtimeProjection` is absent, `THT_AUTH_RUNTIME_ROOT` must also be absent and the existing
|
||||
single-directory behavior remains unchanged.
|
||||
|
||||
The descriptor exposes separate accessors for the canonical authentication directory and the
|
||||
optional runtime-projection specification. Generic host validation continues to validate the
|
||||
canonical store against the effective host UID. Pre-start commands for a projected server
|
||||
additionally require a ready runtime projection consistent with the canonical revision.
|
||||
|
||||
## 4. Filesystem model
|
||||
|
||||
### 4.1 Canonical store
|
||||
|
||||
The authoritative store remains compatible with the existing host auth implementation:
|
||||
|
||||
```text
|
||||
<configDirectory>/
|
||||
auth.yaml
|
||||
users.yaml # local mode only
|
||||
.auth.lock
|
||||
.auth-transaction.lock
|
||||
```
|
||||
|
||||
The directory is `root:root 0700`; all regular files are `root:root 0600`, have link count one,
|
||||
and are reached only through canonical, non-symlinked paths. `auth.yaml` and `users.yaml` retain
|
||||
their current strict schemas. The canonical store is the only data restored from an authentication
|
||||
backup and the only state accepted as authority by recovery.
|
||||
|
||||
### 4.2 Runtime projection
|
||||
|
||||
The projection has this stable bind-mounted root:
|
||||
|
||||
```text
|
||||
<runtimeProjection.directory>/
|
||||
CURRENT
|
||||
generations/
|
||||
<generation-id>/
|
||||
manifest.json
|
||||
auth.yaml
|
||||
users.yaml # local mode only
|
||||
```
|
||||
|
||||
The projection root, `generations`, and generation directories are `10001:10001 0700`. All regular
|
||||
files are `10001:10001 0600` with link count one. The privileged publisher sets these numeric IDs
|
||||
directly; it does not resolve or create a host account. The server Compose mount is read-only, so
|
||||
the core can read but cannot alter the projection even though its effective UID is `10001`.
|
||||
|
||||
No symlink, hardlink, device, FIFO, socket, unexpected file, permissive mode, unexpected owner, or
|
||||
non-canonical path is accepted. Publishing uses descriptor-relative operations with no-follow
|
||||
checks, verifies the opened object before and after I/O, and fsyncs files and parent directories
|
||||
before making a generation ready.
|
||||
|
||||
The runtime publisher and validator live in a projection-specific Linux package whose operations
|
||||
take explicit expected UID and GID values. The existing generic canonical `safeio` ownership rule
|
||||
continues to require owner equals effective UID and is not weakened to accommodate the projection.
|
||||
|
||||
### 4.3 Generation identity
|
||||
|
||||
For each canonical snapshot, the publisher computes lowercase hexadecimal SHA-256 digests of the
|
||||
exact bounded bytes of `auth.yaml` and, in local mode, `users.yaml`. The generation ID is the
|
||||
lowercase SHA-256 of four UTF-8 lines joined by LF and terminated by one final LF:
|
||||
|
||||
```text
|
||||
thothii-auth-projection-v1
|
||||
mode=<local-or-oidc>
|
||||
auth=<64-lowercase-hex>
|
||||
users=<64-lowercase-hex-or-dash>
|
||||
```
|
||||
|
||||
The generation directory name is the 64-character generation ID. For a projected installation,
|
||||
the canonical revision is exactly `sha256:<generation-id>`; no second revision algorithm exists.
|
||||
Its strict `manifest.json` contains only schema version `1`, generation ID, mode, that canonical
|
||||
revision, exact filenames, byte sizes, and per-file SHA-256 digests. JSON is emitted canonically
|
||||
with a trailing newline and no unknown or duplicate fields. Non-projected installations retain the
|
||||
existing revision calculation and output for compatibility.
|
||||
|
||||
Generation files are immutable after publication. An existing directory with the expected ID is
|
||||
reused only after complete owner, mode, type, size, content-digest, manifest, and canonical-revision
|
||||
verification. Any mismatch is an integrity failure; it is never repaired in place.
|
||||
|
||||
`auth publish` can recover a corrupt target generation only while `CURRENT` is blocked. It
|
||||
revalidates the exact directory identity and performs a bounded descriptor-relative deletion of
|
||||
the whole target without following links, then publishes a newly staged complete directory under
|
||||
the expected generation ID. It never edits a published generation file in place. Deletion requires
|
||||
the target directory and every traversed entry to remain confined, owned by `10001:10001`, and to
|
||||
have the exact allowed directory/file type, mode, link count, and bounded name. File digests are
|
||||
checked when readable but are not required to match when digest failure is the reason the whole
|
||||
generation is being replaced. Any structurally unsafe or unrecognized entry is not deleted
|
||||
automatically and keeps the command in a sanitized integrity-failure state.
|
||||
|
||||
Publisher-only temporary names are closed and bounded:
|
||||
|
||||
- `generations/.stage-<transaction>-<generation>` is the only staging-directory form;
|
||||
- `.current-<transaction>.tmp` is the only selector temporary-file form;
|
||||
- no quarantine namespace exists.
|
||||
|
||||
After acquiring the transaction lock and before new staging, `auth publish` scans only these two
|
||||
namespaces. A stage is automatically removed only when its transaction equals the transaction in a
|
||||
persisted blocked selector and the complete bounded no-follow ownership/type/mode/link validation
|
||||
passes. A selector temporary file may be removed for any syntactically valid transaction ID only
|
||||
after the same regular-file validation and strict bounded JSON decoding pass; this covers a crash
|
||||
before the blocked-selector rename. Entries with another name, an unrelated stage transaction,
|
||||
unsafe metadata, excessive contents, or an invalid document remain untouched and keep recovery
|
||||
blocked for explicit operator investigation. Recovery then creates a fresh transaction ID and
|
||||
rebuilds its stage from the canonical store.
|
||||
|
||||
## 5. Atomic selection and fail-closed state
|
||||
|
||||
`CURRENT` is the only mutable runtime selector. It is a bounded, strict JSON document published by
|
||||
same-filesystem temporary-file creation, file fsync, atomic rename, and runtime-root fsync.
|
||||
|
||||
A ready selector contains exactly:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"state": "ready",
|
||||
"transaction": "<32-lowercase-hex>",
|
||||
"generation": "<64-lowercase-hex>",
|
||||
"previousGenerations": ["<zero-to-two-generation-ids>"]
|
||||
}
|
||||
```
|
||||
|
||||
A blocked selector contains exactly:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"state": "blocked",
|
||||
"transaction": "<32-lowercase-hex>"
|
||||
}
|
||||
```
|
||||
|
||||
Transaction IDs are 128 random bits generated by the operating system. Missing, unsafe, malformed,
|
||||
unknown-version, `blocked`, or internally inconsistent selectors make runtime authentication
|
||||
unavailable. Only host `tht` can determine that a structurally valid projection is stale relative
|
||||
to the unmounted canonical store. The backend never falls back to the canonical directory, an
|
||||
older generation, an environment password, or upstream authentication.
|
||||
|
||||
The backend runtime provider reads and validates `CURRENT`, resolves only a safe generation name
|
||||
under `generations`, and loads a complete immutable in-memory snapshot before returning: manifest,
|
||||
strict authentication configuration and, in local mode, the parsed local-user registry. All
|
||||
declared file digests and the generation identity are verified as part of that one load. The
|
||||
projected local-registry resolver consumes the in-memory registry associated with the exact loaded
|
||||
authentication snapshot; it never reopens `users.yaml` later. A request may therefore finish
|
||||
against the coherent ready generation it captured immediately before a transition, even if that
|
||||
generation is later removed. Every later request sees either `blocked` or the new coherent
|
||||
generation.
|
||||
|
||||
The provider may cache an already verified complete immutable snapshot by its storage identity. It
|
||||
must re-read `CURRENT` for each authentication snapshot and must not cache a `blocked` or invalid
|
||||
state as ready. The in-memory snapshot is not serialized, logged, or exposed through diagnostics.
|
||||
|
||||
## 6. Automatic publication transaction
|
||||
|
||||
The following commands are mutations:
|
||||
|
||||
- `auth configure --mode local|oidc`;
|
||||
- `auth user add`;
|
||||
- `auth user set-password`;
|
||||
- `auth user enable` and `auth user disable`;
|
||||
- `auth user grant` and `auth user revoke`;
|
||||
- `auth user logout-all`.
|
||||
|
||||
With a runtime projection configured, each mutation performs one serialized transaction:
|
||||
|
||||
1. Validate descriptor, canonical and runtime parents, ownership policy, and root privilege. For
|
||||
initial configure or first publication, create missing roots with their exact required numeric
|
||||
owners and modes before any canonical credential is written.
|
||||
2. Acquire `.auth-transaction.lock` in the canonical root. The lock covers the complete operation
|
||||
and all publication or recovery work; existing `.auth.lock` remains the inner local-registry
|
||||
mutation lock.
|
||||
3. Load and verify the prior canonical revision when one exists. Except for initial configure,
|
||||
verify that the prior ready projection equals that canonical revision.
|
||||
4. Atomically publish a new `blocked` selector with a fresh transaction ID.
|
||||
5. Execute the existing canonical mutation and validate its complete resulting state.
|
||||
6. Stage the complete runtime generation in the runtime root, set exact numeric ownership and
|
||||
modes, fsync it, rename it to its immutable generation ID, and verify it through the same reader
|
||||
used by `auth publish`.
|
||||
7. Construct history as the previous ready generation followed by its prior history, de-duplicated
|
||||
and truncated to two entries. Atomically publish `CURRENT` as `ready` for the new generation.
|
||||
8. Re-read `CURRENT` and the complete generation, compare the runtime canonical revision with the
|
||||
current canonical revision, and return success only on exact equality.
|
||||
9. Remove generations not named by current plus its two-entry history, without following links.
|
||||
Cleanup occurs only after the new ready state is verified. A cleanup error is reported as an
|
||||
integrity failure but never rolls the selector back to an older credential state.
|
||||
|
||||
If the canonical mutation fails and the publisher proves that canonical bytes are unchanged, it
|
||||
restores and verifies the previously saved ready selector. If canonical state changed or cannot be
|
||||
proved unchanged, `CURRENT` remains `blocked` and the command fails with a sanitized recovery
|
||||
instruction. If publication or final verification fails after a successful canonical mutation,
|
||||
`CURRENT` remains `blocked`.
|
||||
|
||||
No mutating command reports success while canonical and runtime states differ.
|
||||
|
||||
## 7. CLI behavior and recovery
|
||||
|
||||
### 7.1 `tht auth publish`
|
||||
|
||||
`auth publish` accepts no password and no content arguments. It acquires the transaction lock,
|
||||
treats the validated canonical store as authoritative, publishes `blocked`, builds or reuses the
|
||||
matching immutable generation, selects it as ready, verifies equality, applies bounded retention,
|
||||
and exits successfully only when the runtime projection is coherent.
|
||||
|
||||
It is the supported recovery command after interruption, failed publication, runtime tampering, or
|
||||
restoration of a protected canonical backup. It does not recover a password, select an arbitrary
|
||||
old generation, or modify canonical users.
|
||||
|
||||
### 7.2 `tht auth status`
|
||||
|
||||
Existing output remains compatible. For projected server installations, text and JSON output add
|
||||
only projection state, generation ID, canonical revision, and equality status. It never emits YAML,
|
||||
password hashes, user records, transaction lock contents, or environment values.
|
||||
|
||||
### 7.3 `tht auth check`
|
||||
|
||||
Before invoking the existing container diagnostic, `auth check` verifies:
|
||||
|
||||
- canonical auth validity;
|
||||
- runtime selector and generation integrity;
|
||||
- equality of canonical and projected revisions;
|
||||
- exact expected owners and modes.
|
||||
|
||||
Only after these checks pass does it run the existing core diagnostic against the runtime mount.
|
||||
Output remains bounded and sanitized. `--json` remains pristine JSON.
|
||||
|
||||
### 7.4 Pre-start commands
|
||||
|
||||
`start`, `update --check-only`, and relevant `doctor` checks fail closed when a configured runtime
|
||||
projection is absent, blocked, invalid, or stale. Initial `auth configure` and `auth publish` remain
|
||||
available to create or repair it. Read-only auth status remains available to diagnose a blocked
|
||||
projection without exposing secrets.
|
||||
|
||||
### 7.5 Backup restore
|
||||
|
||||
A restore archive can replace canonical `auth.yaml` or `users.yaml`, so it participates in the
|
||||
same projection transaction. After archive preflight identifies any
|
||||
`authentication-configuration` entry and before the first restore mutation, `tht restore` must:
|
||||
|
||||
1. require projected-server root privilege and acquire `.auth-transaction.lock`;
|
||||
2. validate the prior canonical and ready projection;
|
||||
3. atomically select `blocked`;
|
||||
4. perform the existing checkpointed restore or its checkpoint recovery;
|
||||
5. validate the final canonical store, publish its generation, and verify equality before selecting
|
||||
`ready`.
|
||||
|
||||
If candidate restore fails and checkpoint recovery succeeds, the recovered canonical state is
|
||||
published before authentication becomes ready. If restore, checkpoint recovery, publication, or
|
||||
verification cannot complete, `CURRENT` remains blocked. A restore archive with no canonical
|
||||
authentication entries retains its current lifecycle and does not block authentication. A
|
||||
non-projected installation retains all existing restore behavior.
|
||||
|
||||
## 8. Compose and backend integration
|
||||
|
||||
The base/local Compose contract remains unchanged: local installations mount
|
||||
`THT_AUTH_CONFIG_ROOT` at `/run/thothii-auth` and use the existing direct-file provider.
|
||||
|
||||
Projected server installations add a dedicated
|
||||
`deploy/compose.auth-runtime-projection.yaml` override after the base profile and all declared
|
||||
operator overrides, but before the generated current-image override. `Installation.ComposeFiles()`
|
||||
inserts this file automatically whenever `runtimeProjection` is declared. The descriptor loader
|
||||
rejects manually listing the dedicated override, listing it without a projection, or otherwise
|
||||
duplicating it. This ordering and automatic selection prevent an operator override from silently
|
||||
restoring the canonical mount. Non-projected server installations do not include the new override
|
||||
and retain their existing direct-file behavior.
|
||||
|
||||
The projection override:
|
||||
|
||||
- requires host `THT_AUTH_RUNTIME_ROOT` as the bind source;
|
||||
- mounts it read-only at `/run/thothii-auth` for `core` only;
|
||||
- sets container environment `THT_AUTH_RUNTIME_PROJECTION_ROOT` to the literal
|
||||
`/run/thothii-auth`;
|
||||
- does not mount the canonical root anywhere;
|
||||
- does not add authentication environment or mounts to `workspace-maintenance`.
|
||||
|
||||
`THT_AUTH_RUNTIME_ROOT` is only the host-side Compose bind source.
|
||||
`THT_AUTH_RUNTIME_PROJECTION_ROOT` is only the container-side provider selector and path. When the
|
||||
container variable is present, backend startup and every auth snapshot use the projected provider;
|
||||
the inherited `THT_AUTH_CONFIG_FILE` direct path is ignored and must retain its reviewed default
|
||||
value. A conflicting direct-file value is a configuration error. When the container variable is
|
||||
absent, the existing `THT_AUTH_CONFIG_FILE` behavior is unchanged. There is no runtime fallback
|
||||
between the two providers.
|
||||
|
||||
## 9. Security properties
|
||||
|
||||
The implementation must preserve these invariants:
|
||||
|
||||
- root is the sole canonical writer and runtime publisher on a projected server installation;
|
||||
- numeric `10001:10001` is a filesystem/container identity, not a new host account;
|
||||
- the core receives no write-capable auth mount;
|
||||
- canonical password hashes are copied only into a protected runtime generation and never logged;
|
||||
- a stale projection cannot silently authorize after a canonical security mutation fails to
|
||||
publish;
|
||||
- a selector cannot escape the runtime root or select a partial generation;
|
||||
- concurrent CLI mutations are serialized across both canonical mutation and publication;
|
||||
- interrupted work leaves either the prior verified ready selector, when canonical state is proven
|
||||
unchanged, or a persistent blocked selector;
|
||||
- recovery always derives runtime state from the canonical store;
|
||||
- local and cross-platform non-projected authentication remains behaviorally compatible.
|
||||
|
||||
## 10. Testing strategy
|
||||
|
||||
Development follows strict test-driven development: each behavior is introduced by a focused test
|
||||
that fails for the intended reason before production code changes.
|
||||
|
||||
Required automated coverage includes:
|
||||
|
||||
1. Descriptor validation for valid server projection and rejection of local profile, non-Linux
|
||||
mutation, non-canonical/equal paths, missing environment match, and UID/GID other than `10001`.
|
||||
2. Regression coverage proving descriptors without a projection retain current Mac, Windows, and
|
||||
local Linux behavior.
|
||||
3. Secure publication primitives: exact owners/modes, link count, no-follow behavior, bounded
|
||||
reads, fsync/rename ordering, strict pointer/manifest schemas, and digest verification.
|
||||
4. Successful initial local and OIDC publication and automatic publication after every local-user
|
||||
mutation.
|
||||
5. Failure injection before blocking, during canonical mutation, at each stage/fsync/rename, during
|
||||
final verification, and during retention; expected ready restoration or persistent blocked state
|
||||
is asserted explicitly.
|
||||
6. Concurrent `tht` processes proving one transaction at a time and no mixed canonical/runtime
|
||||
result; rapid successive publications prove an in-flight request finishes from its complete
|
||||
in-memory snapshot after the referenced generation is pruned.
|
||||
7. Backend projected-provider tests for ready, blocked, absent, stale, malformed, symlinked,
|
||||
hardlinked, permissive, wrong-owner, altered-manifest, altered-file, and generation-switch cases.
|
||||
8. Login and session tests covering ordinary/admin login, password changes, enable/disable,
|
||||
grant/revoke, and `logout-all` through immutable generations.
|
||||
9. Compose rendering tests proving server core mounts only the runtime projection read-only and
|
||||
`workspace-maintenance` has no authentication mount or environment; automatic override ordering,
|
||||
duplicate refusal, and both host/container environment names are exact.
|
||||
10. Restore tests proving authentication-bearing candidate restore, checkpoint recovery, restore
|
||||
failure, and publication failure cannot leave an old ready projection; archives without auth
|
||||
retain existing behavior.
|
||||
11. Full Go tests with race detection and vet, backend Vitest plus TypeScript typecheck, Compose
|
||||
contract tests, documentation gates, and secret scanners using only synthetic credentials.
|
||||
|
||||
No automated test may inspect the real `/srv/thothii` auth material, start the Project A stack, or
|
||||
mutate a live service.
|
||||
|
||||
## 11. Documentation and later operator gate
|
||||
|
||||
The implementation updates server installation examples, local-auth documentation, Project A
|
||||
preflight instructions, recovery guidance, and tests that enforce those documents. The operator
|
||||
guide explains:
|
||||
|
||||
- the canonical/projection distinction in plain language;
|
||||
- why no host account `10001` is created;
|
||||
- exact descriptor and environment fields;
|
||||
- automatic publication semantics;
|
||||
- the meaning of `blocked` and recovery with `sudo tht auth publish`;
|
||||
- backup and restoration of the canonical store only;
|
||||
- safe verification without printing credentials or hashes.
|
||||
|
||||
After source implementation, review, and verification pass, applying the contract to Project A
|
||||
requires a new bounded pre-start authorization. That later gate will restore the existing generated
|
||||
auth data to canonical `root:root 0700/0600`, create the runtime projection through `sudo tht auth
|
||||
publish`, run read-only preflights, and stop before starting the new stack unless the owner provides
|
||||
separate start authorization.
|
||||
Reference in New Issue
Block a user