feat: finish Pi and workspace management updates
This commit is contained in:
@@ -0,0 +1,86 @@
|
||||
# Simplified `thothctl` installation selection and Pi update Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Allow all existing `thothctl` commands to discover the installation descriptor automatically and make `thothctl pi update` use the checkout's pinned Pi version by default.
|
||||
|
||||
**Architecture:** Add a small config-level resolver that chooses one validated installation descriptor from an explicit flag, environment variable, or bounded upward search from the working directory. Keep the existing Pi lifecycle transaction intact; resolve only the requested version/source at the CLI boundary so the update engine retains its safety and recovery guarantees.
|
||||
|
||||
**Tech Stack:** Go 1.26, Docker Compose v2, existing `tools/thothctl` config and Pi lifecycle packages, Go tests.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-08-14-thothctl-discovery-and-pi-update-design.md`
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Preserve every existing `pi` subcommand, alias, safety check, and explicit invocation form.
|
||||
- `--installation` remains an explicit override and accepts only an absolute descriptor path named `thothii-installation.yaml`.
|
||||
- Automatic discovery must not recursively scan `.artifacts`, home directories, or unrelated descendants.
|
||||
- The default Pi version is the single `ARG PI_VERSION=<version>` in the selected project's `docker/core.Dockerfile`.
|
||||
- The default Pi update uses the existing transactional build path and must not install an arbitrary network “latest”.
|
||||
- All failures remain sanitized and must not reveal secret values.
|
||||
|
||||
## File Map
|
||||
|
||||
- Create `tools/thothctl/internal/config/discovery.go` and `discovery_test.go` for bounded descriptor resolution and safe diagnostics.
|
||||
- Modify `tools/thothctl/cmd/thothctl/main.go` and `main_test.go` for optional global selection, `pi update` defaults, help text, and dispatch.
|
||||
- Create `tools/thothctl/internal/pi/version.go` and `version_test.go` for reading the project Pi pin.
|
||||
- Modify `tools/thothctl/internal/pi/update.go` and `update_test.go` only if the default request needs a typed source/confirmation adjustment; keep lifecycle internals unchanged otherwise.
|
||||
- Modify `docs/contracts/thothctl-pi.md`, `docs/install/pi-management.md`, and relevant command-contract verification scripts.
|
||||
|
||||
### Task 1: Add bounded installation descriptor discovery
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/thothctl/internal/config/discovery.go`
|
||||
- Test: `tools/thothctl/internal/config/discovery_test.go`
|
||||
|
||||
**Interface:** `func Resolve(explicit string, environment func(string) string, workingDirectory string) (string, error)`.
|
||||
|
||||
- [x] Write failing tests for explicit-path precedence, `THOTHII_INSTALLATION`, `deploy/*/thothii-installation.yaml` discovery, parent discovery, `.artifacts` exclusion, invalid candidates, ambiguity, and no-candidate errors.
|
||||
- [x] Run `cd tools/thothctl && go test ./internal/config -run 'TestResolve' -count=1`; confirm RED because `Resolve` is absent.
|
||||
- [x] Implement a bounded upward walk. At each level inspect only the exact descriptor and immediate `deploy/*/thothii-installation.yaml` entries; skip `.artifacts`; require regular files; validate candidates through `config.Load`; deduplicate canonical paths; fail clearly on zero or multiple valid candidates.
|
||||
- [x] Re-run the focused tests and confirm GREEN.
|
||||
- [x] Refactor only after green, keeping path collection separate from candidate validation.
|
||||
|
||||
### Task 2: Make the global installation option optional
|
||||
|
||||
**Files:**
|
||||
- Modify: `tools/thothctl/cmd/thothctl/main.go`
|
||||
- Test: `tools/thothctl/cmd/thothctl/main_test.go`
|
||||
|
||||
- [x] Add failing CLI tests proving `thothctl pi status` works from a project tree, the environment variable is used, an explicit flag wins, ambiguity fails before Docker, and all existing commands retain their dispatch.
|
||||
- [x] Run the focused CLI tests and confirm RED because the current parser requires `--installation`.
|
||||
- [x] Parse optional `--installation`, call `config.Resolve` with `THOTHII_INSTALLATION` and the process working directory, and update help to `thothctl [--installation PATH] <command>`.
|
||||
- [x] Run `cd tools/thothctl && go test ./cmd/thothctl -count=1`; confirm GREEN.
|
||||
|
||||
### Task 3: Default `pi update` to the repository Pi pin
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/thothctl/internal/pi/version.go`
|
||||
- Test: `tools/thothctl/internal/pi/version_test.go`
|
||||
- Modify: `tools/thothctl/cmd/thothctl/main.go`
|
||||
- Test: `tools/thothctl/cmd/thothctl/main_test.go`
|
||||
|
||||
**Interface:** `func ReadPinnedVersion(projectDirectory string) (string, error)`.
|
||||
|
||||
- [x] Add failing tests for one valid Dockerfile pin, missing Dockerfile, duplicate default pins, malformed versions, and `pi update` without `--version`; retain explicit version and advanced pull tests.
|
||||
- [x] Run `cd tools/thothctl && go test ./internal/pi ./cmd/thothctl -run 'Test(ReadPinnedVersion|ParsePiUpdate|RunPiUpdate)' -count=1`; confirm RED.
|
||||
- [x] Read only `docker/core.Dockerfile`, require one default `ARG PI_VERSION=...`, validate it with the existing version grammar, and make the short request select build mode while preserving the lifecycle transaction.
|
||||
- [x] Re-run focused tests and confirm GREEN.
|
||||
|
||||
### Task 4: Update contracts without removing commands
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/contracts/thothctl-pi.md`
|
||||
- Modify: `docs/install/pi-management.md`
|
||||
- Modify: the documentation verification script that asserts the old mandatory update invocation.
|
||||
|
||||
- [x] Document automatic descriptor discovery, the explicit override, `thothctl pi update` as the normal path, `--version` as an explicit pin, and the advanced pull/digest form.
|
||||
- [x] Keep status, doctor, test/check, configure, restart, rollback, maintenance, and logs documented.
|
||||
- [x] Run the targeted documentation checks and `git diff --check`.
|
||||
|
||||
### Task 5: Full verification
|
||||
|
||||
- [x] Run `cd tools/thothctl && go test ./... -count=1`.
|
||||
- [x] Run `cd tools/thothctl && go build ./cmd/thothctl`.
|
||||
- [x] Run the relevant documentation contract script and inspect `git status --short`.
|
||||
- [x] Verify the help text contains the optional form and all existing commands; do not mutate the live Docker installation unless separately requested.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,44 @@
|
||||
# Simplified `thothctl` installation selection and Pi update design
|
||||
|
||||
## Decision
|
||||
|
||||
Keep every existing `thothctl pi` subcommand. Make the installation descriptor optional on the command line and resolve it automatically when the operator runs from the project tree. Keep `--installation <path>` as an explicit override for non-standard locations or multiple installations.
|
||||
|
||||
The normal Pi update becomes:
|
||||
|
||||
```sh
|
||||
thothctl pi update
|
||||
```
|
||||
|
||||
When `--version` is omitted, `thothctl` reads the single default `ARG PI_VERSION=<version>` from `docker/core.Dockerfile` in the selected installation's project directory and uses that pinned version with the existing transactional build/update path. An explicit `--version <version>` remains supported. The command does not fetch an arbitrary npm “latest”; the repository pin, lockfile, image labels, and executable must remain consistent.
|
||||
|
||||
## Installation resolution
|
||||
|
||||
Resolution order is:
|
||||
|
||||
1. an explicit `--installation <absolute-path>/thothii-installation.yaml`;
|
||||
2. `THOTHII_INSTALLATION`, when set to an absolute descriptor path;
|
||||
3. automatic discovery from the current working directory and its parents.
|
||||
|
||||
Automatic discovery examines only the exact descriptor at each directory level and the immediate `deploy/*/thothii-installation.yaml` locations. It never recursively scans `.artifacts`, home directories, or unrelated descendants. A candidate must be a regular file and must pass `config.Load`. One valid candidate is selected. No candidates or multiple valid candidates produce an actionable error that names the expected locations and explains how to use `--installation`.
|
||||
|
||||
The resolver is shared by all existing top-level commands, not only `pi`, so `thothctl status`, `start`, `stop`, `doctor`, `workspace`, and the Pi commands have the same invocation rules. Existing explicit invocations remain valid.
|
||||
|
||||
## Safety and compatibility
|
||||
|
||||
- No existing Pi subcommand is removed or renamed.
|
||||
- Existing advanced `pi update --source ... --image ... --yes --drain` syntax remains accepted for compatibility.
|
||||
- The short update path selects build mode and preserves the existing lifecycle lock, maintenance gate, session handling, candidate verification, image selector promotion, and rollback/recovery behavior.
|
||||
- The default update uses the current checkout's declared Pi pin; changing to a newer Pi release still requires updating the repository pin and lockfile in the normal source-update workflow.
|
||||
- Errors and automatic-discovery diagnostics never expose secret file contents.
|
||||
|
||||
## User-facing examples
|
||||
|
||||
```sh
|
||||
thothctl pi update
|
||||
thothctl pi update --version 0.81.0
|
||||
thothctl pi status
|
||||
thothctl pi restart --yes --drain
|
||||
thothctl --installation ~/operator/thothii-installation.yaml pi update
|
||||
```
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
# Unified `tht` CLI and Product Setup Design
|
||||
|
||||
Date: 2026-08-15
|
||||
|
||||
## Objective
|
||||
|
||||
Turn the repository into a product that can be cloned, bootstrapped once, and then operated with a
|
||||
single normal system command named `tht`. The operator must not build Go manually, create a `bin`
|
||||
directory, remember an installation descriptor path, or use raw Docker commands for normal
|
||||
installation, lifecycle, Pi updates, backup, or restore.
|
||||
|
||||
## Confirmed decisions
|
||||
|
||||
1. The only product command name is `tht`.
|
||||
2. `thothctl` is removed completely. There is no compatibility alias, wrapper, deprecation period,
|
||||
or second installed command.
|
||||
3. The host operator implementation remains a native Go binary so macOS, Linux, and Windows hosts
|
||||
do not require Go, Python, or a virtualenv.
|
||||
4. The existing Python workflow CLI remains inside `core` under the same command name `tht`.
|
||||
Backend and Pi continue to use it there. It is not installed on the host and is not presented in
|
||||
operator documentation. No `tht-runtime` command or namespace is introduced.
|
||||
5. The command audit is binding: 55 commands are `MAINTAIN`, 8 are `ENHANCE`, and 14 are `ERASE`.
|
||||
The detailed decision matrix is in
|
||||
`docs/reports/2026-08-15-tht-command-maintain-erase-enhance.md`.
|
||||
6. Workflow commands called by Pi, the gate, the backend, workspace maintenance, or the session
|
||||
migrator are machine contracts. Their semantics, pristine JSON output, stdin behavior, and exit
|
||||
codes are not changed merely to simplify the operator help.
|
||||
7. The public operator help stays small. Internal workflow commands do not appear in host help.
|
||||
8. `--installation` remains available as an optional override. Normal commands discover the
|
||||
installation from the current project root or worktree.
|
||||
9. `backup` and `restore` are introduced as first-class product commands.
|
||||
10. The final implementation is installed on this Mac and used to rebuild/restart the live stack on
|
||||
port 8080.
|
||||
|
||||
## Command boundary
|
||||
|
||||
### Host operator CLI
|
||||
|
||||
The native host command exposes:
|
||||
|
||||
```text
|
||||
tht setup
|
||||
tht version
|
||||
tht start [--build]
|
||||
tht stop
|
||||
tht status
|
||||
tht doctor [--json]
|
||||
tht logs
|
||||
tht update [--check-only] [--yes] [--drain]
|
||||
tht backup [--output PATH] [--include-secrets --yes] [--drain]
|
||||
tht restore ARCHIVE --yes [--drain]
|
||||
tht sessions migrate --yes
|
||||
tht remove [--yes ID...]
|
||||
tht pi ...
|
||||
tht workspace ...
|
||||
```
|
||||
|
||||
`tht pi` preserves `status`, `doctor`, `test`/`check`, `configure`, `restart`, `update`,
|
||||
`rollback`, `maintenance`, and `logs`. `tht workspace` preserves every currently implemented
|
||||
workspace operation, including the two vector operations missing from the current help.
|
||||
|
||||
### Container workflow CLI
|
||||
|
||||
The Python command tree remains the deterministic protocol used by Pi and the backend. The host
|
||||
installation does not expose these commands as operator shortcuts. This prevents a human operator
|
||||
from bypassing reviewer gates while avoiding a risky rewrite of the live workflow.
|
||||
|
||||
The `MAINTAIN`, `ENHANCE`, and `ERASE` decisions apply exactly as recorded in the command audit.
|
||||
`MAINTAIN` commands retain their current path. `ENHANCE` commands receive the approved safety or
|
||||
diagnostic improvements. `ERASE` commands disappear from Typer registration and active docs; domain
|
||||
functions still used by canonical pipelines remain internal libraries.
|
||||
|
||||
## Bootstrap and PATH installation
|
||||
|
||||
A freshly cloned repository necessarily needs one bootstrap action before `tht` exists:
|
||||
|
||||
```bash
|
||||
bash scripts/install-tht.sh
|
||||
```
|
||||
|
||||
Windows uses:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts/install-tht.ps1
|
||||
```
|
||||
|
||||
The scripts require Docker, build the correct native binary using the repository-pinned Docker
|
||||
builder, verify it, and install it atomically:
|
||||
|
||||
- macOS and Linux: `/usr/local/bin/tht`, requesting elevation only for the final atomic install;
|
||||
- Windows: `%LOCALAPPDATA%\ThothII\bin\tht.exe`, adding that directory to the user PATH when needed.
|
||||
|
||||
The user never selects a platform binary, runs Go, creates a `bin` directory, or invokes a
|
||||
project-relative executable. Re-running the installer upgrades the installed command idempotently.
|
||||
|
||||
## Project and worktree discovery
|
||||
|
||||
Every host command starts from the current working directory and walks parents until it finds the
|
||||
ThothII root contract (`compose.yaml`, `deploy/`, and the repository marker). A Git worktree root is
|
||||
treated exactly like the main checkout.
|
||||
|
||||
Installation resolution order is:
|
||||
|
||||
1. explicit `--installation PATH`;
|
||||
2. `THOTHII_INSTALLATION`;
|
||||
3. one valid `thothii-installation.yaml` in the current directory or immediate `deploy/*`;
|
||||
4. the same search while walking parent directories.
|
||||
|
||||
Zero candidates produce a setup-oriented error. Multiple candidates produce a bounded list and
|
||||
require `--installation`; no arbitrary recursive search is allowed.
|
||||
|
||||
## `tht setup`
|
||||
|
||||
`tht setup` is idempotent and defaults to the local profile on macOS, Windows, and workstation
|
||||
Linux. `--profile server` selects server behavior. Generated installation files live under
|
||||
`deploy/<installation-id>/`, where `deploy` is explicitly documented as a directory in the project
|
||||
or worktree root.
|
||||
|
||||
The setup flow:
|
||||
|
||||
1. verifies root/worktree identity, Docker, Compose, supported architecture, and line endings;
|
||||
2. creates or validates the installation descriptor and non-secret environment files;
|
||||
3. asks plain-language questions and stores secret file paths, never secret values in the
|
||||
descriptor;
|
||||
4. creates protected secret-file templates only after explicit confirmation and never overwrites an
|
||||
existing file;
|
||||
5. renders and validates Compose configuration;
|
||||
6. builds the ThothII images from the current checkout;
|
||||
7. starts the stack with `docker compose up --detach --remove-orphans`;
|
||||
8. waits for bounded health checks;
|
||||
9. runs installation diagnostics and Pi diagnostics;
|
||||
10. prints the URL and the exact descriptor selected.
|
||||
|
||||
`tht setup --configure-only` stops after validated configuration. `tht setup` never silently
|
||||
replaces a descriptor, environment file, secret file, or generated state belonging to another
|
||||
installation.
|
||||
|
||||
## Lifecycle and product update
|
||||
|
||||
- `tht start` starts the selected installation without rebuilding.
|
||||
- `tht start --build` builds current-checkout images before startup.
|
||||
- `tht stop` stops the installation while preserving state.
|
||||
- `tht update --check-only` retains its current non-mutating validation behavior.
|
||||
- `tht update` becomes the complete product update: lifecycle lock, active-session check/drain,
|
||||
backup checkpoint, image build from the current checkout, controlled recreation, health checks,
|
||||
diagnostics, and rollback to the recorded images when verification fails.
|
||||
|
||||
Product update and Pi update remain separate. `tht update` updates ThothII. `tht pi update` updates
|
||||
only Pi in `core`.
|
||||
|
||||
## Pi management
|
||||
|
||||
`tht pi update [--version VERSION]` makes the version optional. Without `--version`, it queries the
|
||||
latest stable version of the pinned Pi package from the authoritative package registry. A lookup
|
||||
failure stops before mutation and tells the operator to retry or supply `--version`; it never
|
||||
silently substitutes an older pin.
|
||||
|
||||
The command builds or pulls a candidate, verifies the Pi executable, `PI_VERSION`, and image label,
|
||||
recreates only `core`, checks health, runs the smoke test, preserves volumes, and rolls back on
|
||||
failure. Interactive terminals receive one clear confirmation; non-interactive execution requires
|
||||
`--yes`. Model/provider selection remains the responsibility of `tht pi configure` and is not a
|
||||
required argument to Pi update.
|
||||
|
||||
The project release pin in `docker/core.Dockerfile` remains the clean-build default. Installation
|
||||
update state records the selected newer image/version so ordinary restart does not revert it.
|
||||
|
||||
## Backup and restore
|
||||
|
||||
`tht backup` creates a versioned manifest and checksummed archive in
|
||||
`~/.thothii/backups/<installation-id>/` unless `--output` is supplied. It acquires the lifecycle
|
||||
lock, refuses active work unless `--drain` is accepted, obtains a consistent stopped snapshot, and
|
||||
restarts/verifies a previously running installation.
|
||||
|
||||
The backup includes:
|
||||
|
||||
- installation descriptor, non-secret environment/configuration, generated overrides, and source
|
||||
revision metadata;
|
||||
- installation-owned settings, Pi state, workspace registry, workspace secrets volume, sessions,
|
||||
Qdrant data, and embedding-model volume;
|
||||
- server bind roots returned by the installation preservation contract;
|
||||
- a manifest of external secret-file paths and digests.
|
||||
|
||||
Secret-file contents are excluded by default. `--include-secrets --yes` includes them and marks the
|
||||
archive sensitive; the file is created with owner-only permissions. A backup without secrets is
|
||||
restorable only when all referenced secret files still exist and match preflight requirements.
|
||||
|
||||
`tht restore ARCHIVE --yes` validates schema version, checksums, installation identity, target
|
||||
ownership, secret prerequisites, disk space, and stopped/quiescent state before mutation. It creates
|
||||
a rollback checkpoint, restores only manifest-listed paths/volumes, starts the stack when it was
|
||||
previously running, and runs health, `doctor`, `pi doctor`, and workspace inspection. Failure keeps
|
||||
the target in a recoverable stopped state and prints the checkpoint path.
|
||||
|
||||
## Pi Management frontend
|
||||
|
||||
The frontend uses only Docker-based instructions and only the command `tht`. The section:
|
||||
|
||||
- begins fully collapsed;
|
||||
- uses separate Linux, macOS, and Windows environment panels, with none open initially;
|
||||
- is shorter than the current panel and has a working vertical scrollbar;
|
||||
- begins with the exact concise wording `Using the host terminal`;
|
||||
- explains that `deploy` is a directory in the project/worktree root beside `compose.yaml`;
|
||||
- explains that `deploy/pi/models.json` and `deploy/pi/settings.json` are host files mounted
|
||||
read-only into `core`, so the operator edits the host files, not files inside the container;
|
||||
- explains credential-file concepts in plain language rather than presenting environment-variable
|
||||
names without context;
|
||||
- shows direct commands such as `tht pi configure`, `tht pi restart`, `tht pi update`,
|
||||
`tht pi status`, `tht pi doctor`, and `tht pi test`;
|
||||
- contains no Go build, `~/bin`, `./bin`, `./tht`, `thothctl`, or mandatory descriptor path.
|
||||
|
||||
The sanitized-log control must either display the bounded, sanitized `core` log response or fail
|
||||
with a visible error. The live model selector must reflect the mounted Pi configuration, including
|
||||
the existing GLM 5.3 change after `core` is recreated.
|
||||
|
||||
## Documentation
|
||||
|
||||
Active user, installation, architecture, CLI-contract, testing, and README documentation is
|
||||
rewritten around the bootstrap-plus-setup flow and the direct `tht` command. Historical
|
||||
`docs/superpowers` plans/specs remain historical records; the new spec supersedes them.
|
||||
|
||||
Documentation examples run from the project/worktree root, refer to the home directory as `~`, and
|
||||
do not teach manual Go builds or manually constructed installation paths for normal use. Advanced
|
||||
sections may document optional `--installation` and non-interactive flags.
|
||||
|
||||
## Safety and acceptance
|
||||
|
||||
- Existing user changes to `deploy/pi/models.json` and `deploy/pi/settings.json` are preserved.
|
||||
- Unrelated dirty-worktree files are not overwritten or committed accidentally.
|
||||
- JSON contracts remain pristine and secrets are sanitized from stdout, stderr, logs, archives, and
|
||||
failure messages.
|
||||
- Tests cover macOS/Linux shell installation, Windows PowerShell installation, root/worktree
|
||||
discovery, setup idempotency, lifecycle rollback, Pi latest-version lookup, command audit,
|
||||
backup/restore, frontend layout/copy, and active-document command examples.
|
||||
- Final acceptance installs `tht` on this Mac, verifies `command -v tht`, confirms `thothctl` is
|
||||
absent, updates the live stack, checks healthy services, opens port 8080, verifies Pi Management,
|
||||
and confirms GLM 5.3 is selectable.
|
||||
Reference in New Issue
Block a user