ThothII/docs/contracts/thothctl-pi.md

thothctl pi lifecycle contract

thothctl is the only component that drives Docker lifecycle operations. The core container does not mount a Docker socket, and Pi is never updated in a running container.

Inspection and configuration

thothctl pi status
thothctl pi doctor
thothctl pi test
thothctl pi logs
thothctl pi configure

When --installation is omitted, thothctl first uses THOTHII_INSTALLATION and otherwise discovers one valid thothii-installation.yaml in the current project tree, including an immediate deploy/* directory. Use --installation /absolute/path/thothii-installation.yaml as an explicit override when the descriptor is outside that tree or more than one installation is available.

status executes the image-bundled pi --version. doctor compares that value with both the container's PI_VERSION contract and the io.thothii.pi.version image label; a merely nonempty version is not sufficient. doctor and test also require a healthy core, a successful Pi smoke, valid settings, and an exact selected provider/model pair from the backend's available model entries. pi check remains an alias for pi test. Logs are always a bounded, sanitized 200-line snapshot; there is no follow mode.

On a TTY, pi configure presents numbered provider, model, and thinking choices. Providers and models come from the backend's closed model list, and the model choices are restricted to the selected provider. In non-interactive use, all choices must be explicit:

thothctl pi configure \
  --provider zai --model glm-5.2 --thinking medium

The helper snapshots exact settings-file existence and raw bytes, applies the new values atomically, and verifies the readback and rendered-configuration digest. A helper, readback, or digest failure restores those exact bytes when the prior file existed; on a clean installation it removes the new file and verifies the absent/default state. Empty prior files are supported. The command reports the actual host path from PI_AUTH_FILE; credentials remain in that protected host file and must never be passed as flags.

Installation-managed Pi provider/model configuration is declarative only. Any JSON value beginning with ! is rejected recursively in the complete models.json before it can supply management choices, and the exact selected provider/model and credential payload is checked again before the isolated smoke files are written. The API returns only the fixed Pi provider/model configuration is invalid message; rejected commands, paths, and secrets are never included. Use $NAME/${NAME} environment references in models.json, or omit apiKey and provide the selected credential through the protected PI_AUTH_FILE, THT_MODEL_API_KEY_FILE, or THT_SECRETS_FILE contract. A literal leading exclamation mark uses Pi's $! escape. Direct secret-file references are not a models.json feature: ThothII converts its managed key source to the provider-native child environment, while PI_AUTH_FILE is mounted as Pi's protected credential store.

Supported Compose entry points and current image

Use thothctl start, stop, status, logs, and doctor for ordinary installation lifecycle operations. All thothctl Compose commands automatically include the installation-specific durable selector when it exists:

<projectDirectory>/.thothctl/<installation-id>/current-image.yaml

This selector is part of the supported installation state: it keeps a verified Pi image selected across a fresh thothctl process, stop/start, reconcile, and source checkout whose base image is digest-pinned. Do not delete or hand-edit it. Direct raw docker compose lifecycle commands bypass this protection and are unsupported. Advanced documented Compose rendering must use scripts/compose-with-preflight.sh and include the same selector with -f when present; connector secret overrides must never bypass that preflight wrapper.

Reloading Pi configuration

Configuration reload is a separate lifecycle operation from an image update:

thothctl pi restart --yes [--drain]

--yes is required after reviewing the planned core recreation. Restart activates the durable maintenance gate before it checks sessions. Without --drain, active open sessions refuse the command. With --drain, the command polls the authenticated session inventory until no active sessions remain; it never terminates sessions and the wait is bounded.

Restart retains the exact current image and never builds, pulls, or upgrades an image. Before any core mutation, it tags the captured running image ID with a transaction-scoped reference and selects that reference through a lifecycle-only Compose override. A configured mutable tag moving after capture therefore cannot change the restarted image. It recreates only core with --no-deps --force-recreate --no-build --pull never; frontend and named volumes are not recreated. Before reopening admission, it verifies health, the unchanged Pi version, the provider/model/settings smoke, unchanged non-secret rendered configuration, the captured image identity, and the complete persistence-mount fingerprint.

Restart and update keep separate recovery state:

<projectDirectory>/.thothctl/<installation-id>/restart-state.json
<projectDirectory>/.thothctl/<installation-id>/update-state.json

The files are mode 0600 and share one installation lifecycle lock, so restart, update, and rollback cannot race. Every mutating lifecycle command checks both files. Malformed or non-terminal restart recovery state blocks update and rollback; malformed or incomplete update recovery state blocks restart. A verified terminal restart state is cleaned up safely before a later mutation. After core mutation, a restart failure leaves admission gated and preserves both restart-state.json and its exact-image override; the operator must use status/logs and maintenance recovery rather than deleting recovery material.

Updating Pi

The normal update uses the repository's pinned version and build source automatically:

thothctl pi update
thothctl pi update --version 0.81.0

With no --version, the command reads the single default ARG PI_VERSION=<version> from docker/core.Dockerfile in the selected project. The normal path confirms the explicit update command, drains active sessions without terminating them, builds the candidate, recreates only core, verifies it, and promotes it transactionally.

Advanced registry updates remain available and require an immutable digest:

thothctl pi update \
  --version 0.81.0 --source build --yes --drain

thothctl pi update \
  --version 0.81.0 --source pull \
  --image registry.example.invalid/thothii-core@sha256:<64-lowercase-hex-digits> --yes

--source build rebuilds only core with PI_VERSION=<version>. --source pull requires an immutable digest reference; mutable tags, URL forms, and credential-bearing references are rejected. --source is never inferred.

Before inventory, update activates the durable maintenance gate. Activation writes /data/settings/maintenance.json in the mounted settings volume, closes admission, and waits for all leases. A recreated candidate reads that marker at startup and therefore starts gated. The loopback-only control endpoints cannot be reached through the frontend proxy and do not depend on the configured authentication principal mode. Lost activation/deactivation responses are resolved by querying gate status only when the original result is unknown. An explicit file or directory durability failure is never converted to success by matching readback: the control API reports maintenance_durability_failed, keeps or restores the safest durable marker state, and requires recovery.

Open, unarchived sessions stop an update. After an operator has completed or otherwise drained their work, --drain makes the command poll the authenticated bare-array GET /sessions?scope=all response until no active sessions remain.

The configured core.image is never retagged or mutated. Each installation transaction creates unique candidate and previous tags, including when two installations share a configured tag or the configured image is digest-pinned. A temporary lifecycle-only Compose override selects those tags for build, recreate, and rollback. Candidate build, pull, or candidate-tag failures happen before mutation_started and therefore never recreate or roll back core. After verification, the temporary candidate selector is atomically promoted to the durable current-image override. Rollback atomically promotes the previous selector. Terminal cleanup removes only transaction files and never deletes the durable selector.

Only core is recreated, with --no-deps --force-recreate; frontend is not recreated and no volume-replacement flags are used. Verification checks health; exact requested Pi version at all three declared boundaries (the candidate executable, PI_VERSION environment, and io.thothii.pi.version image label); the provider/model/settings smoke; unchanged non-secret rendered configuration; and the complete persistence-mount fingerprint.

Recovery, rollback, and maintenance cleanup

Recovery state and lock diagnostics live under:

<projectDirectory>/.thothctl/<installation-id>/update-state.json
<projectDirectory>/.thothctl/<installation-id>/restart-state.json
<projectDirectory>/.thothctl/<installation-id>/*.lock.owner.json

Each recovery file is mode 0600. Update state records transaction-scoped image identities, mount fingerprints, target version/source, configuration digest, phase, and timestamp; restart state records the retained image and its verification inputs. Neither file contains credentials, endpoint values, secret paths, Compose output, or logs. A cross-platform OS advisory file lock serializes both lifecycle operations; a crashed owner releases the lock automatically. Owner metadata is diagnostic only and cannot wedge acquisition if empty, partial, or stale.

Any post-candidate failure explicitly confirms or reactivates maintenance and rescans sessions before compensation. Automatic rollback selects the transaction's previous image through the lifecycle override and clears maintenance only after the previous image, configuration, mounts, health, Pi smoke, and terminal recovery write are verified. Ambiguous compensation remains gated. If the candidate core is stopped and cannot serve the maintenance endpoint, rollback proves that state with Compose and writes the marker through a one-off previous-image core container sharing the settings volume. It does not require the failed candidate, a host Node runtime, or the Docker socket inside a container. The restored core is then recreated, verified, and rescanned before the gate can open.

For a failed update with update-state.json, first run:

thothctl pi rollback --yes

Rollback restores the image recorded in update state, but it checks restart state before making any change. A failed, pending, or malformed restart state rejects rollback. A failed restart retains its captured image and has no candidate image to roll back; first inspect status and logs, repair the reported problem, then use maintenance recovery.

Inspect and clean a stale durable gate with:

thothctl pi maintenance status
thothctl pi maintenance recover --yes

maintenance recover restores the captured restart image pin and lifecycle override when needed, verifies and removes interrupted restart recovery material, and only then processes update state. It completes an interrupted verified-image promotion, safely finalizes a preparation interrupted before core mutation, and refuses other pending mutations. For terminal or absent recovery state, it removes only a stale transaction override, verifies the running installation when the gate is active, and only then removes the durable marker and reopens admission. It never removes current-image.yaml. If rollback or recovery fails, leave the marker in place, preserve the relevant recovery state and override, repair the reported Docker/configuration issue, and rerun rollback or maintenance recovery.

Missing confirmation, invalid arguments, active sessions, and an interrupted transaction exit 2. Docker and verification failures exit nonzero with concise, redacted guidance. Direct read-only/log commands preserve the original Docker child exit code.

Go dependency security boundary

The supported toolchain is Go 1.26.5, released 2026-07-07, with module language version 1.26.0. The Docker builder is pinned by both patch tag and the multi-platform manifest-list digest:

golang:1.26.5-bookworm@sha256:1ecb7edf62a0408027bd5729dfd6b1b8766e578e8df93995b225dfd0944eb651

That manifest provides both linux/amd64 and linux/arm64/v8 builders. Go's official release history is the authority for the patch level (https://go.dev/doc/devel/release); the Docker Official Image is the authority for the builder (https://hub.docker.com/_/golang). golang.org/x/sys, used by the Windows durable-replace implementation, is pinned to v0.47.0. The directly used github.com/sirupsen/logrus is pinned to v1.9.1, which removes GO-2025-4188 from the imported package set. The build contract verifies the exact toolchain, dependencies, digest, and all five supported target builds (Windows amd64, Darwin amd64/arm64, and Linux amd64/arm64). go mod verify, tests including the race detector, go vet, and govulncheck are release gates.