229 lines
13 KiB
Markdown
229 lines
13 KiB
Markdown
# `thothctl pi` lifecycle contract
|
|
|
|
`thothctl` is the only component that drives Docker lifecycle operations. The `core` container
|
|
does not mount a Docker socket, and Pi is never updated in a running container.
|
|
|
|
## Inspection and configuration
|
|
|
|
```text
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi status
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi doctor
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi test
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi logs
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi configure
|
|
```
|
|
|
|
`status` executes the image-bundled `pi --version`. `doctor` compares that value with both the
|
|
container's `PI_VERSION` contract and the `io.thothii.pi.version` image label; a merely nonempty
|
|
version is not sufficient. `doctor` and `test` also require a healthy core, a successful Pi smoke,
|
|
valid settings, and an exact selected provider/model pair from the backend's available model
|
|
entries. `pi check` remains an alias for `pi test`. Logs are always a bounded, sanitized 200-line
|
|
snapshot; there is no follow mode.
|
|
|
|
On a TTY, `pi configure` presents numbered provider, model, and thinking choices. Providers and
|
|
models come from the backend's closed model list, and the model choices are restricted to the
|
|
selected provider. In non-interactive use, all choices must be explicit:
|
|
|
|
```text
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi configure \
|
|
--provider zai --model glm-5.2 --thinking medium
|
|
```
|
|
|
|
The helper snapshots exact settings-file existence and raw bytes, applies the new values atomically,
|
|
and verifies the readback and rendered-configuration digest. A helper, readback, or digest failure
|
|
restores those exact bytes when the prior file existed; on a clean installation it removes the new
|
|
file and verifies the absent/default state. Empty prior files are supported. The command reports
|
|
the actual host path from `PI_AUTH_FILE`; credentials remain in that protected host file and must
|
|
never be passed as flags.
|
|
|
|
Installation-managed Pi provider/model configuration is declarative only. Any JSON value beginning
|
|
with `!` is rejected recursively in the complete `models.json` before it can supply management
|
|
choices, and the exact selected provider/model and credential payload is checked again before the
|
|
isolated smoke files are written. The API returns only the fixed
|
|
`Pi provider/model configuration is invalid` message; rejected commands, paths, and secrets are
|
|
never included. Use `$NAME`/`${NAME}` environment references in `models.json`, or omit `apiKey` and
|
|
provide the selected credential through the protected `PI_AUTH_FILE`, `THT_MODEL_API_KEY_FILE`, or
|
|
`THT_SECRETS_FILE` contract. A literal leading exclamation mark uses Pi's `$!` escape. Direct
|
|
secret-file references are not a `models.json` feature: ThothII converts its managed key source to
|
|
the provider-native child environment, while `PI_AUTH_FILE` is mounted as Pi's protected credential
|
|
store.
|
|
|
|
## Supported Compose entry points and current image
|
|
|
|
Use `thothctl start`, `stop`, `status`, `logs`, and `doctor` for ordinary installation lifecycle
|
|
operations. All `thothctl` Compose commands automatically include the installation-specific
|
|
durable selector when it exists:
|
|
|
|
```text
|
|
<projectDirectory>/.thothctl/<installation-id>/current-image.yaml
|
|
```
|
|
|
|
This selector is part of the supported installation state: it keeps a verified Pi image selected
|
|
across a fresh `thothctl` process, stop/start, reconcile, and source checkout whose base image is
|
|
digest-pinned. Do not delete or hand-edit it. Direct raw `docker compose` lifecycle commands bypass
|
|
this protection and are unsupported. Advanced documented Compose rendering must use
|
|
`scripts/compose-with-preflight.sh` and include the same selector with `-f` when present; connector
|
|
secret overrides must never bypass that preflight wrapper.
|
|
|
|
## Reloading Pi configuration
|
|
|
|
Configuration reload is a separate lifecycle operation from an image update:
|
|
|
|
```text
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi restart --yes [--drain]
|
|
```
|
|
|
|
`--yes` is required after reviewing the planned core recreation. Restart activates the durable
|
|
maintenance gate before it checks sessions. Without `--drain`, active open sessions refuse the
|
|
command. With `--drain`, the command polls the authenticated session inventory until no active
|
|
sessions remain; it never terminates sessions and the wait is bounded.
|
|
|
|
Restart retains the exact current image and never builds, pulls, or upgrades an image. Before any
|
|
core mutation, it tags the captured running image ID with a transaction-scoped reference and
|
|
selects that reference through a lifecycle-only Compose override. A configured mutable tag moving
|
|
after capture therefore cannot change the restarted image. It recreates only `core` with
|
|
`--no-deps --force-recreate --no-build --pull never`; `frontend` and named volumes are not
|
|
recreated. Before reopening admission, it verifies health, the unchanged Pi version, the
|
|
provider/model/settings smoke, unchanged non-secret rendered configuration, the captured image
|
|
identity, and the complete persistence-mount fingerprint.
|
|
|
|
Restart and update keep separate recovery state:
|
|
|
|
```text
|
|
<projectDirectory>/.thothctl/<installation-id>/restart-state.json
|
|
<projectDirectory>/.thothctl/<installation-id>/update-state.json
|
|
```
|
|
|
|
The files are mode `0600` and share one installation lifecycle lock, so restart, update, and
|
|
rollback cannot race. Every mutating lifecycle command checks both files. Malformed or non-terminal
|
|
restart recovery state blocks update and rollback; malformed or incomplete update recovery state
|
|
blocks restart. A verified terminal restart state is cleaned up safely before a later mutation.
|
|
After core mutation, a restart failure leaves admission gated and preserves both
|
|
`restart-state.json` and its exact-image override; the operator must use status/logs and maintenance
|
|
recovery rather than deleting recovery material.
|
|
|
|
## Updating Pi
|
|
|
|
Every update requires a pinned version, an explicit source, and confirmation:
|
|
|
|
```text
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi update \
|
|
--version 0.81.0 --source build --yes
|
|
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi update \
|
|
--version 0.81.0 --source pull \
|
|
--image registry.example.invalid/thothii-core@sha256:<64-lowercase-hex-digits> --yes
|
|
```
|
|
|
|
`--source build` rebuilds only `core` with `PI_VERSION=<version>`. `--source pull` requires an
|
|
immutable digest reference; mutable tags, URL forms, and credential-bearing references are
|
|
rejected. `--source` is never inferred.
|
|
|
|
Before inventory, update activates the durable maintenance gate. Activation writes
|
|
`/data/settings/maintenance.json` in the mounted settings volume, closes admission, and waits for
|
|
all leases. A recreated candidate reads that marker at startup and therefore starts gated. The
|
|
loopback-only control endpoints cannot be reached through the frontend proxy and do not depend on
|
|
the configured authentication principal mode. Lost activation/deactivation responses are resolved
|
|
by querying gate status only when the original result is unknown. An explicit file or directory
|
|
durability failure is never converted to success by matching readback: the control API reports
|
|
`maintenance_durability_failed`, keeps or restores the safest durable marker state, and requires
|
|
recovery.
|
|
|
|
Open, unarchived sessions stop an update. After an operator has completed or otherwise drained
|
|
their work, `--drain` makes the command poll the authenticated bare-array
|
|
`GET /sessions?scope=all` response until no active sessions remain.
|
|
|
|
The configured `core.image` is never retagged or mutated. Each installation transaction creates
|
|
unique candidate and previous tags, including when two installations share a configured tag or
|
|
the configured image is digest-pinned. A temporary lifecycle-only Compose override selects those
|
|
tags for build, recreate, and rollback. Candidate build, pull, or candidate-tag failures happen
|
|
before `mutation_started` and therefore never recreate or roll back core. After verification, the
|
|
temporary candidate selector is atomically promoted to the durable current-image override.
|
|
Rollback atomically promotes the previous selector. Terminal cleanup removes only transaction
|
|
files and never deletes the durable selector.
|
|
|
|
Only `core` is recreated, with `--no-deps --force-recreate`; `frontend` is not recreated and no
|
|
volume-replacement flags are used. Verification checks health; exact requested Pi version at all
|
|
three declared boundaries (the candidate executable, `PI_VERSION` environment, and
|
|
`io.thothii.pi.version` image label); the provider/model/settings smoke; unchanged
|
|
non-secret rendered configuration; and the complete persistence-mount fingerprint.
|
|
|
|
## Recovery, rollback, and maintenance cleanup
|
|
|
|
Recovery state and lock diagnostics live under:
|
|
|
|
```text
|
|
<projectDirectory>/.thothctl/<installation-id>/update-state.json
|
|
<projectDirectory>/.thothctl/<installation-id>/restart-state.json
|
|
<projectDirectory>/.thothctl/<installation-id>/*.lock.owner.json
|
|
```
|
|
|
|
Each recovery file is mode `0600`. Update state records transaction-scoped image identities, mount
|
|
fingerprints, target version/source, configuration digest, phase, and timestamp; restart state
|
|
records the retained image and its verification inputs. Neither file contains credentials, endpoint
|
|
values, secret paths, Compose output, or logs. A cross-platform OS advisory file lock serializes
|
|
both lifecycle operations; a crashed owner releases the lock automatically. Owner metadata is
|
|
diagnostic only and cannot wedge acquisition if empty, partial, or stale.
|
|
|
|
Any post-candidate failure explicitly confirms or reactivates maintenance and rescans sessions
|
|
before compensation. Automatic rollback selects the transaction's previous image through the
|
|
lifecycle override and clears maintenance only after the previous image, configuration, mounts,
|
|
health, Pi smoke, and terminal recovery write are verified. Ambiguous compensation remains gated.
|
|
If the candidate core is stopped and cannot serve the maintenance endpoint, rollback proves that
|
|
state with Compose and writes the marker through a one-off previous-image `core` container sharing
|
|
the settings volume. It does not require the failed candidate, a host Node runtime, or the Docker
|
|
socket inside a container. The restored core is then recreated, verified, and rescanned before the
|
|
gate can open.
|
|
|
|
For a failed update with `update-state.json`, first run:
|
|
|
|
```text
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi rollback --yes
|
|
```
|
|
|
|
Rollback restores the image recorded in update state, but it checks restart state before making any
|
|
change. A failed, pending, or malformed restart state rejects rollback. A failed restart retains its
|
|
captured image and has no candidate image to roll back; first inspect status and logs, repair the
|
|
reported problem, then use maintenance recovery.
|
|
|
|
Inspect and clean a stale durable gate with:
|
|
|
|
```text
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi maintenance status
|
|
thothctl --installation /absolute/path/thothii-installation.yaml pi maintenance recover --yes
|
|
```
|
|
|
|
`maintenance recover` restores the captured restart image pin and lifecycle override when needed,
|
|
verifies and removes interrupted restart recovery material, and only then processes update state.
|
|
It completes an interrupted verified-image promotion, safely finalizes a preparation interrupted
|
|
before core mutation, and refuses other pending mutations. For terminal or absent recovery state,
|
|
it removes only a stale transaction override, verifies the running installation when the gate is
|
|
active, and only then removes the durable marker and reopens admission. It never removes
|
|
`current-image.yaml`. If rollback or recovery fails, leave the marker in place, preserve the
|
|
relevant recovery state and override, repair the reported Docker/configuration issue, and rerun
|
|
rollback or maintenance recovery.
|
|
|
|
Missing confirmation, invalid arguments, active sessions, and an interrupted transaction exit
|
|
`2`. Docker and verification failures exit nonzero with concise, redacted guidance. Direct
|
|
read-only/log commands preserve the original Docker child exit code.
|
|
|
|
## Go dependency security boundary
|
|
|
|
The supported toolchain is Go `1.26.5`, released 2026-07-07, with module language version
|
|
`1.26.0`. The Docker builder is pinned by both patch tag and the multi-platform manifest-list
|
|
digest:
|
|
|
|
```text
|
|
golang:1.26.5-bookworm@sha256:1ecb7edf62a0408027bd5729dfd6b1b8766e578e8df93995b225dfd0944eb651
|
|
```
|
|
|
|
That manifest provides both `linux/amd64` and `linux/arm64/v8` builders. Go's official release
|
|
history is the authority for the patch level (`https://go.dev/doc/devel/release`); the Docker
|
|
Official Image is the authority for the builder (`https://hub.docker.com/_/golang`).
|
|
`golang.org/x/sys`, used by the Windows durable-replace implementation, is pinned to `v0.47.0`.
|
|
The directly used `github.com/sirupsen/logrus` is pinned to `v1.9.1`, which removes
|
|
GO-2025-4188 from the imported package set. The build contract verifies the exact toolchain,
|
|
dependencies, digest, and all five supported target builds (Windows amd64, Darwin amd64/arm64, and
|
|
Linux amd64/arm64). `go mod verify`, tests including the race detector, `go vet`, and
|
|
`govulncheck` are release gates.
|