1160 lines
57 KiB
Markdown
1160 lines
57 KiB
Markdown
# Unified `tht` CLI Product Step Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||
|
||
**Goal:** Turn the repository's fragmented operator experience into one installable `tht` command that configures, builds, starts, updates, diagnoses, backs up, restores, and manages Pi, while preserving the indispensable NL-to-SQL workflow commands and simplifying the remaining command surface according to the approved maintain/erase/enhance audit.
|
||
|
||
**Architecture:** The host-facing command is the existing native Go operator CLI, renamed from `thothctl` to `tht` and installed on the operating-system `PATH`. It discovers the current ThothII repository or Git worktree and its installation descriptor automatically. The Python workflow CLI remains named `tht` inside the `core` container and continues to own sessions, decisions, documents, SQL, and persistence. Host operations call Docker Compose directly or, when workflow checks are needed, invoke the container-local Python CLI. There is no compatibility alias, wrapper, second public command, host Python virtual environment, or requirement to build the CLI manually.
|
||
|
||
**Tech Stack:** Go standard library, Docker Compose, Python/Typer, pytest, React 18, TypeScript, Vite, Vitest, Testing Library, Playwright, POSIX shell, PowerShell.
|
||
|
||
**Spec:** `docs/superpowers/specs/2026-08-15-unified-tht-cli-product-step-design.md`
|
||
|
||
## Global Constraints
|
||
|
||
- [ ] Expose exactly one public product command: `tht`. Remove `thothctl` rather than retaining an alias, wrapper, or deprecation period.
|
||
- [ ] Keep the Python workflow executable named `tht` inside `core`; do not introduce `tht-runtime` or a public `runtime` namespace.
|
||
- [ ] Preserve all 55 commands classified **MAINTAIN**, implement the 8 approved **ENHANCE** outcomes, and remove the 14 commands classified **ERASE**. The two audit reports are normative inputs.
|
||
- [ ] Make `--installation` optional on host commands. Explicit paths always win; otherwise discover the descriptor from the repository/worktree and then from the documented installation registry.
|
||
- [ ] Make `tht` callable from a project root or Git worktree root without `./`, `./bin/`, `~/bin/`, a shell wrapper, a Go build, or activation of a Python virtual environment.
|
||
- [ ] Make `tht setup` perform configuration, image build, container start, health verification, and diagnostics by default. `--configure-only` is the explicit opt-out before build/start.
|
||
- [ ] Make `tht pi update` resolve the latest stable Pi version when `--version` is omitted. A failed lookup must stop before any mutation; it must not silently reuse the Dockerfile pin.
|
||
- [ ] Preserve the user's current GLM 5.3 changes in `deploy/pi/models.json` and `deploy/pi/settings.json`; never replace those files with stale fixtures.
|
||
- [ ] Treat secrets as secret references by default. Do not print secret values, write them into tracked files, or include them in a backup unless the operator explicitly supplies both `--include-secrets` and `--yes`.
|
||
- [ ] Preserve unrelated dirty worktree changes and `.playwright-cli/`. Stage only files named by the current task.
|
||
- [ ] Keep frontend strings in English. Documentation may explain concepts in prose but all shown commands must be directly executable.
|
||
- [ ] Work test-first for every behavior change: add or tighten a failing test, run it to confirm the expected failure, implement the smallest complete change, rerun the focused test, then run the relevant suite.
|
||
- [ ] Do not deploy to the live Mac or restart port 8080 until all code, documentation, and automated gates pass.
|
||
|
||
## Approved Command-Surface Baseline
|
||
|
||
The implementation must end with this host-facing surface:
|
||
|
||
```text
|
||
tht setup [--configure-only] [--installation PATH]
|
||
tht version
|
||
tht start [--build] [--installation PATH]
|
||
tht stop [--installation PATH]
|
||
tht status [--installation PATH]
|
||
tht doctor [--json] [--installation PATH]
|
||
tht logs [SERVICE] [--installation PATH]
|
||
tht update [--check-only] [--yes] [--drain] [--installation PATH]
|
||
tht backup [--output PATH] [--include-secrets --yes] [--drain] [--installation PATH]
|
||
tht restore ARCHIVE --yes [--drain] [--installation PATH]
|
||
tht sessions migrate --yes [--installation PATH]
|
||
tht remove [--yes ID...] [--installation PATH]
|
||
tht pi <status|doctor|test|check|configure|restart|update|rollback|maintenance|logs> ...
|
||
tht workspace ...
|
||
```
|
||
|
||
`tht workspace ...` preserves every currently implemented native workspace operation, including
|
||
the existing inspect, preprocessing, schema, evidence, index, and vector operations. This plan does
|
||
not invent a second workspace-management surface merely to make the help tree look symmetrical.
|
||
|
||
The Python workflow command surface inside `core` is governed by the approved audit:
|
||
|
||
- **MAINTAIN:** 55 commands remain behaviorally and contractually available.
|
||
- **ENHANCE:** `config check`, `doctor`, `db fetch-ca`, `memory list`, `memory show`, `memory update`, `memory delete`, and `memory index` are improved or consolidated as described in Task 12.
|
||
- **ERASE:** `decision list`, `decision retract`, `cte list`, `sql explain`, `sql save`, `memory clear`, `memory migrate`, `evidence extract`, `evidence index`, `lsh build`, `lsh query`, `vector init`, `formula save`, and `formula list` are removed in Task 11.
|
||
|
||
---
|
||
|
||
## Task 1: Rename the Native Operator CLI and Its Build Artifacts
|
||
|
||
**Files:**
|
||
|
||
- Rename: `tools/thothctl/` → `tools/tht/`
|
||
- Rename: `tools/tht/cmd/thothctl/` → `tools/tht/cmd/tht/`
|
||
- Rename: `docker/thothctl.Dockerfile` → `docker/tht.Dockerfile`
|
||
- Rename: `scripts/build-thothctl.sh` → `scripts/build-tht.sh`
|
||
- Rename/update: existing `scripts/test-thothctl-*.sh` files → corresponding `scripts/test-tht-*.sh` files
|
||
- Modify: Go module/import paths and package references under `tools/tht/`
|
||
- Modify: `.gitignore`
|
||
|
||
### Steps
|
||
|
||
- [ ] Add a command-identity test in `tools/tht/cmd/tht/main_test.go` that invokes the existing `run` entry point, requires the help banner and error prefix to use `tht`, exercises the `version` path, and proves `thothctl` is not an alias.
|
||
- [ ] Run the focused test before renaming and confirm it fails because the current executable and root command are still `thothctl`.
|
||
|
||
```bash
|
||
cd tools/thothctl
|
||
go test ./cmd/thothctl -run 'TestRootCommandIdentity' -count=1
|
||
```
|
||
|
||
- [ ] Rename the directory, command package, Dockerfile, build script, smoke scripts, binary outputs, archive names, and image labels. Update imports mechanically, including the Go module path if it contains `/tools/thothctl`.
|
||
- [ ] Make `scripts/build-tht.sh` emit only `tht` binaries and archives such as `tht-darwin-arm64`, never `thothctl-*`.
|
||
- [ ] Remove all executable aliases and wrapper generation. Historical design documents may retain the old name as history; active source, tests, packaging, and user documentation may not.
|
||
- [ ] Run the renamed test and all Go tests.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./cmd/tht -run 'TestRootCommandIdentity' -count=1
|
||
go test ./...
|
||
```
|
||
|
||
- [ ] Run a scoped stale-name scan over the renamed implementation and build assets. Active documentation is intentionally updated in Task 14; do not partially rewrite it here.
|
||
|
||
```bash
|
||
rg -n 'thothctl|THOTHCTL' tools/tht docker scripts \
|
||
-g '!scripts/test-tht-command-docs.sh'
|
||
```
|
||
|
||
- [ ] Commit only the rename and mechanical identity changes.
|
||
|
||
```bash
|
||
git add -A -- tools/thothctl tools/tht docker/thothctl.Dockerfile docker/tht.Dockerfile \
|
||
scripts/build-thothctl.sh scripts/build-tht.sh \
|
||
scripts/test-thothctl-build-contract.sh scripts/test-tht-build-contract.sh \
|
||
scripts/thothctl-update-smoke.sh scripts/tht-update-smoke.sh .gitignore
|
||
git commit -m "refactor(cli): rename operator command to tht"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 2: Add Cross-Platform `tht` Installers
|
||
|
||
**Files:**
|
||
|
||
- Create: `scripts/install-tht.sh`
|
||
- Create: `scripts/install-tht.ps1`
|
||
- Create: `scripts/test-install-tht.sh`
|
||
- Create: `scripts/test-install-tht.ps1`
|
||
- Modify: `scripts/build-tht.sh`
|
||
|
||
### Installer contract
|
||
|
||
- macOS/Linux: use the repository-pinned Docker builder to produce the native binary for the current OS/architecture, install it as `/usr/local/bin/tht` by default, use elevation only for the final atomic install when required, and verify that the installed command is resolvable on `PATH`.
|
||
- Windows: install `tht.exe` under `%LOCALAPPDATA%\ThothII\bin`, add that directory to the current user's `PATH` when absent, and explain when a new terminal is required.
|
||
- Tests may override the destination with `THT_INSTALL_DIRECTORY`; production users are not instructed to set this variable.
|
||
- Re-running the installer replaces only the installed `tht` binary atomically and leaves installation data untouched.
|
||
|
||
### Steps
|
||
|
||
- [ ] Write shell installer tests that use a temporary `THT_INSTALL_DIRECTORY`, a fake build artifact, and a controlled `PATH`. Assert executable permissions, atomic replacement, `tht version`, and idempotency.
|
||
- [ ] Write PowerShell tests for the equivalent Windows behavior, including paths containing spaces and a pre-existing user `PATH` entry.
|
||
- [ ] Run both tests and confirm failure because the installers do not exist.
|
||
|
||
```bash
|
||
bash scripts/test-install-tht.sh
|
||
pwsh -NoProfile -File scripts/test-install-tht.ps1
|
||
```
|
||
|
||
- [ ] Implement `scripts/install-tht.sh` with explicit OS/architecture detection, a temporary staging directory, checksum validation when using a packaged artifact, and an atomic final rename.
|
||
- [ ] Implement `scripts/install-tht.ps1` with the same contract and user-level `PATH` update.
|
||
- [ ] Make both installers invoke `scripts/build-tht.sh` internally, which uses the repository-pinned Docker builder. The user must never install Go, select an artifact, or invoke a Go compiler.
|
||
- [ ] Rerun focused tests and package builds for Darwin arm64/amd64, Linux arm64/amd64, and Windows amd64.
|
||
|
||
```bash
|
||
bash scripts/test-install-tht.sh
|
||
pwsh -NoProfile -File scripts/test-install-tht.ps1
|
||
bash scripts/build-tht.sh --all
|
||
```
|
||
|
||
- [ ] Commit the installer slice.
|
||
|
||
```bash
|
||
git add scripts/install-tht.sh scripts/install-tht.ps1 scripts/test-install-tht.sh \
|
||
scripts/test-install-tht.ps1 scripts/build-tht.sh
|
||
git commit -m "feat(cli): install tht as a system command"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 3: Discover the Project, Worktree, and Installation Descriptor Automatically
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/project/discovery.go`
|
||
- Create: `tools/tht/internal/project/discovery_test.go`
|
||
- Modify: `tools/tht/internal/config/discovery.go`
|
||
- Modify: `tools/tht/internal/config/discovery_test.go`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
- Modify: `tools/tht/cmd/tht/main_test.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package project
|
||
|
||
type Root struct {
|
||
Path string
|
||
IsWorktree bool
|
||
}
|
||
|
||
func Discover(start string) (Root, error)
|
||
```
|
||
|
||
The descriptor resolver keeps its existing explicit-path support and applies this precedence:
|
||
|
||
1. `--installation PATH` supplied to the current command.
|
||
2. `THOTHII_INSTALLATION` when explicitly set by automation.
|
||
3. One valid `thothii-installation.yaml` in the current directory or its immediate `deploy/*` children.
|
||
4. The same bounded search while walking parent directories to the discovered repository/worktree root.
|
||
5. A concise actionable error suggesting `tht setup` when no descriptor exists, or listing bounded candidates and requiring optional `--installation` when more than one exists.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add table-driven tests for invocation from the repository root, a nested directory, a linked Git worktree, one immediate `deploy/<installation-id>/thothii-installation.yaml`, multiple immediate descriptors, a missing descriptor, `THOTHII_INSTALLATION`, and an explicit descriptor override.
|
||
- [ ] Add tests proving `tht`, `tht help`, `tht version`, and `tht setup` do not require a pre-existing descriptor.
|
||
- [ ] Run the focused tests and confirm the current resolver fails root/worktree and descriptor-free bootstrap cases.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/project ./internal/config ./cmd/tht -run 'TestDiscover|TestResolve|TestBootstrapCommands' -count=1
|
||
```
|
||
|
||
- [ ] Implement root detection using repository markers (`.git` file or directory, `compose.yaml`, `deploy/`, and the expected ThothII source layout). Resolve symlinks for identity while retaining the user's invocation path for messages.
|
||
- [ ] Extend descriptor discovery without changing explicit `--installation` semantics. Keep the search bounded to current/ancestor directories and immediate `deploy/*`; do not scan the home directory or fall back to `/Users/mp/thothii-installation.yaml`.
|
||
- [ ] Return ambiguity as an error listing installation IDs and descriptor paths without exposing secret values.
|
||
- [ ] Run all Go tests. Defer system-PATH smoke calls to Task 15 so the old Mac installation is not changed prematurely.
|
||
|
||
```bash
|
||
cd tools/tht && go test ./...
|
||
```
|
||
|
||
- [ ] Commit discovery behavior.
|
||
|
||
```bash
|
||
git add tools/tht/internal/project tools/tht/internal/config tools/tht/cmd/tht
|
||
git commit -m "feat(cli): discover ThothII projects and installations"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 4: Generate Setup Files Safely
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/setup/request.go`
|
||
- Create: `tools/tht/internal/setup/files.go`
|
||
- Create: `tools/tht/internal/setup/files_test.go`
|
||
- Modify: `.gitignore`
|
||
- Modify: `deploy/env/local.env.example`
|
||
- Modify: `deploy/psd/thothii-installation.yaml.example`
|
||
- Modify: `deploy/psd/operator.env.example`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package setup
|
||
|
||
type Request struct {
|
||
ProjectRoot string
|
||
InstallationID string
|
||
Profile string
|
||
ConfigureOnly bool
|
||
NonInteractive bool
|
||
}
|
||
|
||
type FilesResult struct {
|
||
DescriptorPath string
|
||
EnvironmentPath string
|
||
Created []string
|
||
}
|
||
|
||
func EnsureFiles(request Request, input io.Reader, output io.Writer) (FilesResult, error)
|
||
```
|
||
|
||
### Steps
|
||
|
||
- [ ] Add tests for a fresh checkout, a linked worktree, existing compatible files, conflicting files, interrupted writes, paths with spaces, and secret prompts. Assert that tracked examples are never modified.
|
||
- [ ] Require generated files to live under `deploy/<installation-id>/`, be ignored by Git except for tracked examples, and contain secret file references rather than secret values.
|
||
- [ ] Run the tests and confirm they fail because setup file generation is absent.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/setup -run 'TestEnsureFiles' -count=1
|
||
```
|
||
|
||
- [ ] Implement prompts for installation ID, deployment profile, externally reachable endpoints, workspace selection, and secret-file locations. Accept safe defaults in interactive mode and explicit flags/environment in non-interactive automation.
|
||
- [ ] Write `deploy/<installation-id>/thothii-installation.yaml` and `deploy/<installation-id>/operator.env` atomically with restrictive permissions where the platform supports them.
|
||
- [ ] Refuse to overwrite a conflicting descriptor or environment file. Report the exact file and corrective action.
|
||
- [ ] Create protected external secret-file templates only after explicit confirmation, never overwrite an existing secret file, and store only their paths in generated configuration.
|
||
- [ ] Rerun setup tests and verify the bounded discovery from Task 3 finds the generated descriptor and no generated file is tracked.
|
||
|
||
```bash
|
||
cd tools/tht && go test ./internal/setup -count=1
|
||
```
|
||
|
||
- [ ] Commit setup-file generation.
|
||
|
||
```bash
|
||
git add tools/tht/internal/setup tools/tht/cmd/tht/main.go \
|
||
deploy/env/local.env.example deploy/psd/thothii-installation.yaml.example \
|
||
deploy/psd/operator.env.example .gitignore
|
||
git commit -m "feat(setup): generate local installation configuration"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 5: Make `tht setup` Build, Start, and Verify the Product
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/setup/run.go`
|
||
- Create: `tools/tht/internal/setup/run_test.go`
|
||
- Modify: `tools/tht/internal/compose/runner.go`
|
||
- Modify: `tools/tht/internal/compose/runner_test.go`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
- Modify: `tools/tht/cmd/tht/main_test.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package setup
|
||
|
||
type Result struct {
|
||
DescriptorPath string
|
||
ProjectName string
|
||
Configured bool
|
||
Built bool
|
||
Started bool
|
||
Healthy bool
|
||
}
|
||
|
||
func Run(
|
||
ctx context.Context,
|
||
runner compose.Runner,
|
||
request Request,
|
||
input io.Reader,
|
||
output io.Writer,
|
||
) (Result, error)
|
||
```
|
||
|
||
### Ordered setup workflow
|
||
|
||
1. Discover and validate the repository/worktree.
|
||
2. Check Docker Engine, Docker Compose, supported architecture, and line-ending compatibility.
|
||
3. Generate or validate local configuration.
|
||
4. Run `docker compose config` against the resolved descriptor and profiles.
|
||
5. Stop successfully when `--configure-only` is set.
|
||
6. Build required images, including the `core` image containing Pi.
|
||
7. Start the stack with the installation-specific Compose project name.
|
||
8. Wait for frontend, core, qdrant, embedding, and one-shot model initialization health.
|
||
9. Run aggregate `tht doctor` and `tht pi doctor`.
|
||
10. Print the frontend URL and concise next actions.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add runner-fake tests asserting the exact order above, immediate stop for `--configure-only`, failure propagation, retryable health polling, and cleanup messaging after partial startup.
|
||
- [ ] Add CLI tests proving `tht setup` defaults to build/start and that only `--configure-only` disables those phases.
|
||
- [ ] Run focused tests and confirm failure.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/setup ./cmd/tht -run 'TestRun|TestSetupCommand' -count=1
|
||
```
|
||
|
||
- [ ] Implement orchestration using the existing descriptor/Compose abstractions. Do not duplicate command execution logic in the top-level argument dispatcher.
|
||
- [ ] Make health waits bounded and identify the failing service, last health state, and useful `tht logs <service>` command.
|
||
- [ ] Ensure setup can be rerun idempotently to repair/start an already configured checkout.
|
||
- [ ] Rerun focused and full Go suites.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/setup ./internal/compose ./cmd/tht -count=1
|
||
go test ./...
|
||
```
|
||
|
||
- [ ] Commit the complete setup workflow.
|
||
|
||
```bash
|
||
git add tools/tht/internal/setup tools/tht/internal/compose tools/tht/cmd/tht
|
||
git commit -m "feat(setup): build start and verify ThothII"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 6: Add `version`, Aggregate `doctor`, and `start --build`
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/version/info.go`
|
||
- Create: `tools/tht/internal/version/info_test.go`
|
||
- Create: `tools/tht/internal/doctor/report.go`
|
||
- Create: `tools/tht/internal/doctor/report_test.go`
|
||
- Create: `tools/tht/internal/service/service.go`
|
||
- Create: `tools/tht/internal/service/service_test.go`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
- Modify: `tools/tht/cmd/tht/main_test.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package doctor
|
||
|
||
type Check struct {
|
||
Name string `json:"name"`
|
||
Status string `json:"status"`
|
||
Detail string `json:"detail"`
|
||
}
|
||
|
||
type Report struct {
|
||
OK bool `json:"ok"`
|
||
Checks []Check `json:"checks"`
|
||
}
|
||
|
||
func Run(ctx context.Context, installation config.Installation, runner Runner) (Report, error)
|
||
```
|
||
|
||
### Required behavior
|
||
|
||
- `tht version` works without an installation and reports CLI semantic version, commit, build time, OS, and architecture. When an installation is discoverable, it may additionally report the deployed product and Pi versions.
|
||
- `tht doctor` aggregates descriptor validation, Compose availability/configuration, file permissions, required volume presence, service health, frontend/core reachability, workspace registry validity, container-local workflow diagnostics, and Pi diagnostics.
|
||
- `tht doctor --json` writes one pristine JSON document to stdout; all progress and warnings go to stderr.
|
||
- `tht start` starts without rebuilding. `tht start --build` runs the required build before Compose up and then performs bounded health checks.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add tests for descriptor-free `version`, deterministic JSON, unavailable Docker, stopped/running core, a failed workflow check, and a redacted secret path.
|
||
- [ ] Add service tests proving `--build` changes the runner sequence from `up` to `build → up → health`, while normal `start` remains `up → health`.
|
||
- [ ] Run focused tests and confirm failure.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/version ./internal/doctor ./internal/service ./cmd/tht \
|
||
-run 'TestVersion|TestDoctor|TestStart' -count=1
|
||
```
|
||
|
||
- [ ] Implement build metadata with linker defaults that remain useful in local source builds.
|
||
- [ ] Implement aggregate diagnostics as typed checks. Invoke the Python workflow `tht doctor --json` only through `docker compose exec -T core ...` when `core` is running; never require a host virtual environment.
|
||
- [ ] Implement `start --build` through the shared Compose runner.
|
||
- [ ] Verify JSON output and full tests.
|
||
|
||
```bash
|
||
cd tools/tht && go test ./...
|
||
go run ./cmd/tht version
|
||
go run ./cmd/tht doctor --json | jq -e '.ok != null and (.checks | type == "array")'
|
||
```
|
||
|
||
- [ ] Commit this operator-observability slice.
|
||
|
||
```bash
|
||
git add tools/tht/internal/version tools/tht/internal/doctor tools/tht/internal/service tools/tht/cmd/tht
|
||
git commit -m "feat(cli): add version diagnostics and build-aware start"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 7: Make `tht pi update` Resolve the Latest Stable Pi Version
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/pi/latest.go`
|
||
- Create: `tools/tht/internal/pi/latest_test.go`
|
||
- Modify: `tools/tht/internal/pi/update.go`
|
||
- Modify: `tools/tht/internal/pi/update_test.go`
|
||
- Modify: `tools/tht/internal/pi/commands.go`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package pi
|
||
|
||
type RegistryClient interface {
|
||
LatestStable(ctx context.Context, packageName string) (string, error)
|
||
}
|
||
|
||
func ResolveRequestedVersion(
|
||
ctx context.Context,
|
||
requested string,
|
||
packageName string,
|
||
registry RegistryClient,
|
||
) (string, error)
|
||
```
|
||
|
||
### Required behavior
|
||
|
||
- `tht pi update` means “install the latest stable Pi release available from the package registry.”
|
||
- `tht pi update --version X.Y.Z` installs that explicit stable version and skips latest-version discovery.
|
||
- Prerelease versions require an explicit full version; latest discovery ignores prereleases.
|
||
- Failure, malformed registry data, timeout, or an empty version stops before drain, Dockerfile mutation, image build, container replacement, or configuration changes.
|
||
- The command keeps the existing `--source build|pull`, immutable image digest, `--yes`, `--drain`, rollback, status, doctor, test, logs, and configure capabilities. `--installation` remains optional through Task 3 discovery.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add registry-client tests for a stable release, prerelease-only data, malformed JSON, timeout, package-not-found, and explicit version bypass.
|
||
- [ ] Change the update transaction test so an omitted version expects the resolved latest stable release rather than the current `ARG PI_VERSION` value from `docker/core.Dockerfile`.
|
||
- [ ] Add a no-mutation assertion around every discovery failure.
|
||
- [ ] Run focused tests and confirm the old default-to-Dockerfile behavior fails them.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/pi ./cmd/tht -run 'TestResolveRequestedVersion|TestPiUpdate' -count=1
|
||
```
|
||
|
||
- [ ] Implement a bounded HTTPS client for the registry used by the Pi package declared in `docker/core.Dockerfile`. Parse and validate strict semantic versions; do not shell out to a globally installed npm executable.
|
||
- [ ] Resolve the final version before acquiring a drain or update lock. Feed the resolved version into the existing transactional build/pull and rollback path.
|
||
- [ ] Leave the project release pin in `docker/core.Dockerfile` unchanged as the reproducible clean-build default. Record the selected newer image/version in installation update state so ordinary restart does not revert it; do not use or rewrite the pin as the meaning of “latest.”
|
||
- [ ] Rerun all Pi and Go tests, including explicit build and immutable pull paths.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/pi -count=1
|
||
go test ./...
|
||
```
|
||
|
||
- [ ] Commit latest-version resolution without altering the user's GLM files.
|
||
|
||
```bash
|
||
git add tools/tht/internal/pi tools/tht/cmd/tht/main.go
|
||
git commit -m "feat(pi): update to the latest stable release by default"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 8: Implement Transactional Installation Backups
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/backup/manifest.go`
|
||
- Create: `tools/tht/internal/backup/manifest_test.go`
|
||
- Create: `tools/tht/internal/backup/create.go`
|
||
- Create: `tools/tht/internal/backup/create_test.go`
|
||
- Create: `tools/tht/internal/lifecycle/lock.go`
|
||
- Create: `tools/tht/internal/lifecycle/lock_test.go`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
- Modify: `tools/tht/cmd/tht/main_test.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package backup
|
||
|
||
type Manifest struct {
|
||
SchemaVersion int `json:"schema_version"`
|
||
InstallationID string `json:"installation_id"`
|
||
CreatedAt time.Time `json:"created_at"`
|
||
SourceRevision string `json:"source_revision"`
|
||
IncludesSecrets bool `json:"includes_secrets"`
|
||
Entries []Entry `json:"entries"`
|
||
}
|
||
|
||
type CreateRequest struct {
|
||
Output string
|
||
IncludeSecrets bool
|
||
Confirm bool
|
||
Drain bool
|
||
}
|
||
|
||
func Create(ctx context.Context, installation config.Installation, request CreateRequest) (Result, error)
|
||
```
|
||
|
||
### Backup contents
|
||
|
||
- Effective installation descriptor and non-secret local environment/configuration files.
|
||
- Pi declarative host configuration, including `deploy/pi/models.json` and `deploy/pi/settings.json`.
|
||
- Named volumes: `settings`, `pi-state`, `workspace-registry`, `workspace-secrets`, `sessions`, `qdrant-data`, and `embedding-models`.
|
||
- Server preservation roots declared by the installation descriptor.
|
||
- A manifest containing checksums, logical ownership, source revision, image identities, Compose project name, volume metadata, and whether secret contents are present.
|
||
|
||
The installation-owned `workspace-secrets` volume is part of the consistent volume snapshot. External secret-file contents referenced by the installation are excluded by default; their paths and digests are recorded so restore can verify that they still exist. Including those external files requires `--include-secrets --yes`, writes the archive with mode `0600` where supported, and emits a clear custody warning without printing values.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add manifest tests for deterministic entry ordering, checksums, path normalization, secret markers, and schema-version validation.
|
||
- [ ] Add create tests for the seven required volumes, stopped and running installations, active sessions without `--drain`, explicit drain, custom output, default output, write failure, and cleanup of incomplete archives.
|
||
- [ ] Assert the default path is `~/.thothii/backups/<installation-id>/` and the final archive name contains a UTC timestamp and source revision.
|
||
- [ ] Run focused tests and confirm failure.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/backup ./internal/lifecycle ./cmd/tht \
|
||
-run 'TestManifest|TestCreate|TestBackupCommand' -count=1
|
||
```
|
||
|
||
- [ ] Implement a per-installation lifecycle lock shared by backup, restore, Pi update, and product update.
|
||
- [ ] Quiesce or stop mutable services before snapshotting. Refuse an unsafe live snapshot when active sessions exist and `--drain` was not supplied.
|
||
- [ ] Stream volume data into an archive through a minimal helper container; do not materialize secret data in the repository or process arguments.
|
||
- [ ] Write the manifest last, fsync the temporary archive, atomically rename it, and remove partial output on error.
|
||
- [ ] Run backup tests and inspect a fixture archive to verify that default backups contain no secret payloads.
|
||
|
||
```bash
|
||
cd tools/tht && go test ./internal/backup ./internal/lifecycle -count=1
|
||
go test ./...
|
||
```
|
||
|
||
- [ ] Commit backup support.
|
||
|
||
```bash
|
||
git add tools/tht/internal/backup tools/tht/internal/lifecycle tools/tht/cmd/tht
|
||
git commit -m "feat(cli): add transactional installation backups"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 9: Implement Validated Restore with a Recovery Checkpoint
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/backup/restore.go`
|
||
- Create: `tools/tht/internal/backup/restore_test.go`
|
||
- Create: `tools/tht/internal/backup/preflight.go`
|
||
- Create: `tools/tht/internal/backup/preflight_test.go`
|
||
- Modify: `tools/tht/internal/backup/create.go`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
- Modify: `tools/tht/cmd/tht/main_test.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package backup
|
||
|
||
type RestoreRequest struct {
|
||
Archive string
|
||
Confirm bool
|
||
Drain bool
|
||
}
|
||
|
||
type RestoreResult struct {
|
||
Checkpoint string
|
||
Restarted bool
|
||
Verified bool
|
||
}
|
||
|
||
func Restore(
|
||
ctx context.Context,
|
||
installation config.Installation,
|
||
request RestoreRequest,
|
||
) (RestoreResult, error)
|
||
```
|
||
|
||
### Restore contract
|
||
|
||
Before modifying the installation, restore must validate the archive schema, every checksum, path traversal safety, installation identity, secret policy, required free disk space, target ownership/permissions, volume mapping, and image/config compatibility. It then creates a non-secret recovery checkpoint of the current installation, acquires the lifecycle lock, drains or refuses active sessions, restores into controlled targets, restarts the stack only when it was previously running, and runs health, aggregate doctor, Pi doctor, and workspace inspection. A failure after mutation leaves the target in a recoverable stopped state and prints the checkpoint path; it must not compound damage with an unrequested automatic restore.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add adversarial archive tests for `../` traversal, absolute paths, symlink escapes, duplicate entries, checksum mismatch, unknown schema, wrong installation ID, secret-bearing archives without required protections, insufficient disk, and invalid volume ownership.
|
||
- [ ] Add transaction tests for success, failure before mutation, failure after one restored volume, failed restart, and failed health verification. Every post-mutation failure must stop the target, retain the checkpoint, and report a deterministic recovery command.
|
||
- [ ] Require positional archive plus `--yes`; do not allow an interactive typo to start restore without a complete preflight.
|
||
- [ ] Run focused tests and confirm failure.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/backup ./cmd/tht -run 'TestPreflight|TestRestore' -count=1
|
||
```
|
||
|
||
- [ ] Implement archive validation without extracting untrusted paths directly to final destinations.
|
||
- [ ] Create the rollback checkpoint through the same manifest/archive primitives as Task 8.
|
||
- [ ] Restore configuration and volumes in a deterministic order; retain the checkpoint path in both success and error messages.
|
||
- [ ] Verify with service health, `tht doctor`, `tht pi doctor`, and workspace inspection before declaring success.
|
||
- [ ] Rerun focused, package, and full Go tests.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/backup -count=1
|
||
go test ./...
|
||
```
|
||
|
||
- [ ] Commit restore support.
|
||
|
||
```bash
|
||
git add tools/tht/internal/backup tools/tht/cmd/tht
|
||
git commit -m "feat(cli): add validated restore with rollback"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 10: Turn `tht update` into a Full Product Update Transaction
|
||
|
||
**Files:**
|
||
|
||
- Create: `tools/tht/internal/productupdate/plan.go`
|
||
- Create: `tools/tht/internal/productupdate/plan_test.go`
|
||
- Create: `tools/tht/internal/productupdate/run.go`
|
||
- Create: `tools/tht/internal/productupdate/run_test.go`
|
||
- Modify: `tools/tht/internal/backup/create.go`
|
||
- Modify: `tools/tht/internal/lifecycle/lock.go`
|
||
- Modify: `tools/tht/cmd/tht/main.go`
|
||
- Modify: `tools/tht/cmd/tht/main_test.go`
|
||
|
||
### Interfaces
|
||
|
||
```go
|
||
package productupdate
|
||
|
||
type Request struct {
|
||
CheckOnly bool
|
||
Confirm bool
|
||
Drain bool
|
||
}
|
||
|
||
type Result struct {
|
||
PreviousImages map[string]string
|
||
CurrentImages map[string]string
|
||
Checkpoint string
|
||
RolledBack bool
|
||
}
|
||
|
||
func Plan(ctx context.Context, installation config.Installation) (UpdatePlan, error)
|
||
func Run(ctx context.Context, installation config.Installation, request Request) (Result, error)
|
||
```
|
||
|
||
### Transaction contract
|
||
|
||
- `tht update --check-only` validates compatibility and prints the exact image pull/build, migration, restart, and verification plan without mutating the Git checkout, containers, volumes, or configuration.
|
||
- `tht update --yes` updates the complete ThothII deployment according to the current checkout and installation descriptor: pull externally sourced images, build source-defined images, run required preflight/migrations, recreate changed services, and verify the complete product.
|
||
- Dirty source files are not reset, overwritten, committed, or pulled. Updating source from Git is deliberately outside this command; the operator chooses the checkout/revision and `tht update` deploys it.
|
||
- Before mutation, acquire the lifecycle lock, enforce active-session drain policy, and create a rollback checkpoint through Task 8.
|
||
- Record previous immutable image identities. A build/pull, migration, restart, health, aggregate-doctor, or Pi-doctor failure restores configuration/volumes as needed and recreates the previous images.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add plan tests for a no-op installation, changed built image, changed pulled image, migration required, incompatible descriptor, dirty worktree, and unavailable registry.
|
||
- [ ] Add transaction tests for every failure boundary and assert rollback uses recorded image digests rather than mutable tags.
|
||
- [ ] Add CLI tests for `--check-only`, required `--yes` before mutation, optional `--drain`, and optional `--installation`.
|
||
- [ ] Run focused tests and confirm the current check-only implementation cannot execute the transaction.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/productupdate ./cmd/tht \
|
||
-run 'TestUpdatePlan|TestProductUpdate|TestUpdateCommand' -count=1
|
||
```
|
||
|
||
- [ ] Implement pure planning first, then the transactional runner. Keep user confirmation outside the mutation core so tests can call it deterministically.
|
||
- [ ] Reuse Compose, lifecycle, backup, health, doctor, and Pi diagnostic components; do not introduce parallel shell orchestration.
|
||
- [ ] Make failure output state whether rollback completed and provide the retained checkpoint path.
|
||
- [ ] Rerun update, backup, Pi, and full Go suites.
|
||
|
||
```bash
|
||
cd tools/tht
|
||
go test ./internal/productupdate ./internal/update ./internal/backup ./internal/pi -count=1
|
||
go test ./...
|
||
```
|
||
|
||
- [ ] Commit the product-update transaction.
|
||
|
||
```bash
|
||
git add tools/tht/internal/productupdate tools/tht/internal/backup \
|
||
tools/tht/internal/lifecycle tools/tht/cmd/tht
|
||
git commit -m "feat(cli): update the full product transactionally"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 11: Apply the 14 Approved **ERASE** Decisions to the Python Workflow CLI
|
||
|
||
**Files:**
|
||
|
||
- Create: `harness/tests/fixtures/approved_cli_surface.json`
|
||
- Create: `harness/tests/test_cli_surface.py`
|
||
- Modify: `harness/tht/cli/decision_cmd.py`
|
||
- Modify: `harness/tht/cli/cte_cmd.py`
|
||
- Modify: `harness/tht/cli/sql_cmd.py`
|
||
- Modify: `harness/tht/cli/memory_cmd.py`
|
||
- Modify: `harness/tht/cli/evidence_cmd.py`
|
||
- Modify: `harness/tht/cli/lsh_cmd.py`
|
||
- Modify: `harness/tht/cli/vector_cmd.py`
|
||
- Modify: `harness/tht/cli/formula_cmd.py`
|
||
- Modify: `harness/tests/integration/test_gate_cli_signatures.py`
|
||
- Delete or rewrite: tests dedicated exclusively to erased CLI entry points, including `harness/tests/test_decision_retract_cli.py`
|
||
|
||
### Commands to erase
|
||
|
||
```text
|
||
decision list
|
||
decision retract
|
||
cte list
|
||
sql explain
|
||
sql save
|
||
memory clear
|
||
memory migrate
|
||
evidence extract
|
||
evidence index
|
||
lsh build
|
||
lsh query
|
||
vector init
|
||
formula save
|
||
formula list
|
||
```
|
||
|
||
Removing a command means removing its Typer registration, help entry, CLI-only parsing code, CLI-only tests, and active documentation. Underlying domain functions may remain only when a maintained workflow, preprocessing pipeline, or test imports them directly. Delete dead implementation only after a repository-wide reachability check.
|
||
|
||
### Steps
|
||
|
||
- [ ] Build `approved_cli_surface.json` from the two approved audit reports. It must enumerate all 55 maintained paths and the 8 enhanced paths and explicitly blacklist the 14 erased paths.
|
||
- [ ] Add a recursive Typer help test that compares the actual command tree to this fixture. Add integration assertions that every command invoked by `harness/.pi/extensions/tht-gate.js`, `harness/.pi/skills/tht-sessione/SKILL.md`, backend `ThtRunner`, preprocessing jobs, and deployment smoke tests is still present.
|
||
- [ ] Run the command-surface tests before removal and confirm they fail because the 14 erased commands are still exposed.
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/pytest -q tests/test_cli_surface.py tests/integration/test_gate_cli_signatures.py
|
||
```
|
||
|
||
- [ ] Remove the 14 registrations and CLI-only code. Preserve `phase reopen` as the supported correction/invalidation flow, `session show --json` as the ledger view, `sql set-final`/`sql export`, versioned `preprocess evidence`/`preprocess dwh`, collection reconciliation, `ollama ensure`, and `search find`.
|
||
- [ ] Delete or rewrite tests that assert the obsolete surface. Add negative tests requiring a nonzero exit and normal “no such command” message for every erased path.
|
||
- [ ] Use import and call-site scans before deleting any shared function.
|
||
|
||
```bash
|
||
rg -n 'decision_retract|cte_list|sql_explain|sql_save|memory_clear|memory_migrate|evidence_(extract|index)|lsh_(build|query)|vector_init|formula_(save|list)' \
|
||
harness backend frontend scripts docs
|
||
```
|
||
|
||
- [ ] Run the full non-L2 harness suite and the gate signature test.
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/ruff check .
|
||
.venv/bin/pytest -q
|
||
```
|
||
|
||
- [ ] Commit the approved surface reduction.
|
||
|
||
```bash
|
||
git add harness/tht/cli/decision_cmd.py harness/tht/cli/cte_cmd.py \
|
||
harness/tht/cli/sql_cmd.py harness/tht/cli/memory_cmd.py \
|
||
harness/tht/cli/evidence_cmd.py harness/tht/cli/lsh_cmd.py \
|
||
harness/tht/cli/vector_cmd.py harness/tht/cli/formula_cmd.py \
|
||
harness/tests/fixtures/approved_cli_surface.json harness/tests/test_cli_surface.py \
|
||
harness/tests/integration/test_gate_cli_signatures.py
|
||
git add -u -- harness/tests/test_decision_retract_cli.py
|
||
git commit -m "refactor(cli): remove obsolete workflow commands"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 12: Implement the 8 Approved **ENHANCE** Outcomes
|
||
|
||
**Files:**
|
||
|
||
- Modify: `harness/tht/cli/config_cmd.py`
|
||
- Modify: `harness/tht/cli/doctor_cmd.py`
|
||
- Modify: `harness/tht/cli/db_cmd.py`
|
||
- Modify: `harness/tht/cli/memory_cmd.py`
|
||
- Modify: `harness/tht/memory.py`
|
||
- Create: `harness/tests/test_cli_enhanced_surface.py`
|
||
- Modify: `harness/tests/test_doctor_cli.py`
|
||
- Modify: `harness/tests/test_cli_config_environment.py`
|
||
- Modify: `harness/tests/test_memory_metadata.py`
|
||
- Modify: `harness/tests/test_qdrant_cli_commands.py`
|
||
- Create: `tools/tht/internal/setup/tls.go`
|
||
- Create: `tools/tht/internal/setup/tls_test.go`
|
||
- Modify: `tools/tht/internal/setup/run.go`
|
||
- Modify: `tools/tht/internal/setup/run_test.go`
|
||
|
||
### Exact enhanced outcomes
|
||
|
||
1. `config check`: keep its validation as a reusable internal function and structured result, but make public `tht doctor` the normal single preflight. Avoid two competing operator diagnostics.
|
||
2. `doctor`: make it the non-mutating, multilayer diagnostic for installation, descriptor, Compose, storage, runtime configuration, DWH, Pi, Qdrant, and embedder, with readable output and pristine `--json`.
|
||
3. `db fetch-ca`: integrate CA retrieval into guided workspace setup. Show endpoint, certificate subject, validity, SHA-256 fingerprint, and destination before confirmation. Keep the standalone operation as an advanced TLS command, not a mandatory manual setup step.
|
||
4. `memory list`: make it an advanced paginated administrative view with filters for state, provenance, and indexing status plus `--json`; keep it out of concise/basic help.
|
||
5. `memory show`: display immutable provenance, source decision, mutable fields, index state, timestamps, and references needed for a safe correction.
|
||
6. `memory update`: permit only documented mutable fields, print a diff, require explicit confirmation, and reject attempts to alter identity or provenance.
|
||
7. `memory delete`: require an exact identifier and explicit confirmation, show registry/index impact, remove consistently from both, and verify absence afterward.
|
||
8. `memory index`: treat it as repair. Detect and report drift first; rebuild only with explicit confirmation; verify registry/index consistency afterward.
|
||
|
||
### Steps
|
||
|
||
- [ ] Add focused tests for every outcome above, including JSON purity, no mutation by doctor, TLS fingerprint confirmation, pagination bounds, provenance immutability, exact-ID deletion, drift-only repair, confirmation refusal, and partial index failure.
|
||
- [ ] Run the focused tests and confirm they fail against current behavior.
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/pytest -q tests/test_cli_enhanced_surface.py tests/test_doctor_cli.py \
|
||
tests/test_cli_config_environment.py tests/test_memory_metadata.py \
|
||
tests/test_qdrant_cli_commands.py
|
||
```
|
||
|
||
- [ ] Extract configuration validation into a typed result consumed by both container-local doctor and the host aggregate doctor from Task 6. Keep `config check` callable for automation but mark it advanced in help.
|
||
- [ ] Extend doctor without adding repair side effects. A failing layer changes exit status and report data but never changes files, volumes, indexes, or containers.
|
||
- [ ] Integrate TLS CA fetch into host workspace create/update setup with a two-stage inspect/confirm flow. Reject hostname mismatch and invalid/expired certificates before writing.
|
||
- [ ] Implement memory list/show/update/delete/index with a shared repository/index transaction boundary and postcondition checks.
|
||
- [ ] Rerun focused tests, then the full harness suite and Go workspace tests.
|
||
|
||
```bash
|
||
cd harness
|
||
.venv/bin/ruff check .
|
||
.venv/bin/pytest -q
|
||
cd ../tools/tht
|
||
go test ./internal/setup ./internal/doctor -count=1
|
||
go test ./...
|
||
```
|
||
|
||
- [ ] Commit enhanced diagnostics, TLS setup, and memory administration.
|
||
|
||
```bash
|
||
git add harness/tht/cli/config_cmd.py harness/tht/cli/doctor_cmd.py \
|
||
harness/tht/cli/db_cmd.py harness/tht/cli/memory_cmd.py harness/tht/memory.py \
|
||
harness/tests/test_cli_enhanced_surface.py harness/tests/test_doctor_cli.py \
|
||
harness/tests/test_cli_config_environment.py harness/tests/test_memory_metadata.py \
|
||
harness/tests/test_qdrant_cli_commands.py tools/tht/internal/setup/tls.go \
|
||
tools/tht/internal/setup/tls_test.go tools/tht/internal/setup/run.go \
|
||
tools/tht/internal/setup/run_test.go tools/tht/internal/doctor
|
||
git commit -m "feat(cli): harden diagnostics TLS and memory administration"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 13: Rewrite and Verify the Pi Management Frontend Guidance
|
||
|
||
**Files:**
|
||
|
||
- Modify: `frontend/src/shell/PiManagement.tsx`
|
||
- Modify: `frontend/src/shell/PiManagement.test.tsx`
|
||
- Modify: `frontend/src/api/pi-management.ts`
|
||
- Modify: `frontend/src/api/pi-management.test.ts`
|
||
- Modify if required by the verified behavior: `backend/src/pi/management.ts`
|
||
- Modify if required by the verified behavior: `backend/src/routes/pi-management.ts`
|
||
- Modify if required by the verified behavior: `backend/test/pi-management.test.ts`
|
||
- Modify if required by the verified behavior: `backend/test/routes-pi-management.test.ts`
|
||
|
||
### Information architecture and copy contract
|
||
|
||
- The whole section starts collapsed. After the operator expands it, the section title remains “Update Pi configuration and relaunch its container.”
|
||
- Its introduction starts exactly with “Using the host terminal”. It must not say “not this browser page”.
|
||
- Linux, macOS, and Windows are three separate disclosure tabs/panels. All are closed on initial render; opening one does not require another to remain open.
|
||
- The containing window is shorter than the current one and has an always-available vertical scrollbar. Mouse wheel, trackpad, keyboard, and touch scrolling must not be trapped.
|
||
- Guidance covers only Pi managed by ThothII Docker Compose. Remove every native/local Pi branch.
|
||
- Explain in plain language that `deploy` is a directory in the root of the ThothII project, at the same level as `compose.yaml`.
|
||
- Explain that `deploy/pi/models.json` and `deploy/pi/settings.json` are host files mounted read-only into `core`: edit them from the host project/worktree, never from inside the container.
|
||
- Break every explanation longer than two rendered lines into short paragraphs, numbered steps, bullets, field tables, or command blocks.
|
||
- Explain configuration fields before naming them:
|
||
- `baseUrl`: the provider endpoint used by Pi.
|
||
- `api`: the Pi adapter/protocol expected by that provider.
|
||
- `models`: the provider's available model objects and identifiers.
|
||
- `enabledModels`: every `provider/model` pair that Pi may select.
|
||
- credential/auth file settings: paths to host-side files containing provider credentials; never paste a secret into the page, command, Compose file, or tracked JSON.
|
||
- Show direct commands only: `tht pi configure`, `tht pi update`, `tht pi status`, `tht pi doctor`, `tht pi test`, `tht pi logs`, and `tht pi rollback`. Do not show `thothctl`, `~/bin`, `./bin`, `./tht`, shell wrappers, Go builds, or mandatory `--installation`.
|
||
- State that commands work from a repository root or Git worktree root once `tht` has been installed. Explain the optional `--installation PATH` only as an ambiguity/automation override.
|
||
- State that omitting `--version` from `tht pi update` installs the latest stable Pi release. Do not conflate the Pi package version with the provider/model selected by `tht pi configure`.
|
||
- Preserve and display GLM 5.3 when it is present in the live Pi configuration.
|
||
|
||
### Platform-specific command blocks
|
||
|
||
Each panel uses the native terminal syntax but the same operation sequence:
|
||
|
||
```text
|
||
1. Open Terminal/PowerShell in the ThothII repository or worktree root.
|
||
2. Edit deploy/pi/models.json and deploy/pi/settings.json on the host.
|
||
3. Run tht pi configure --provider <PROVIDER> --model <MODEL> --thinking <low|medium|high>.
|
||
4. Run tht pi update (or add --version X.Y.Z for an explicit Pi version).
|
||
5. Use tht pi restart when only configuration changed and no Pi package update is needed.
|
||
6. Run tht pi status, tht pi doctor, and tht pi test.
|
||
7. If needed, inspect tht pi logs or run tht pi rollback.
|
||
```
|
||
|
||
macOS and Windows explicitly require Docker Desktop to be running. Linux requires a running Docker Engine and permission to use Docker. PowerShell examples use PowerShell line continuation only when genuinely necessary; prefer one executable command per line.
|
||
|
||
### “Show sanitized logs” behavior
|
||
|
||
The button must request recent core/Pi diagnostic logs, display timestamps and severity where available, and redact credentials, authorization headers, API keys, tokens, cookies, connection strings, and secret file contents. It is diagnostic only: it must not change Pi or start/restart containers. Empty, unavailable, loading, success, and failure states must all be visible and understandable. If the current backend already satisfies this contract, change only tests/copy; otherwise make the smallest backend correction needed.
|
||
|
||
### Steps
|
||
|
||
- [ ] Extend component tests to require all platform panels closed initially, independent open/close state, a bounded scrollable region, exact introductory wording, structured copy, Docker-only guidance, root/worktree commands, field explanations, latest-Pi semantics, and absence of every obsolete command/path.
|
||
- [ ] Add accessibility tests for disclosure names, `aria-expanded`, focus order, keyboard activation, and scroll-region labeling.
|
||
- [ ] Extend API/backend tests for sanitized-log redaction and all UI states. Include adversarial fake logs containing every secret class listed above.
|
||
- [ ] Run focused tests and confirm current copy/layout fail the new contract.
|
||
|
||
```bash
|
||
cd frontend
|
||
npx vitest run src/shell/PiManagement.test.tsx src/api/pi-management.test.ts
|
||
cd ../backend
|
||
npx vitest run test/pi-management.test.ts test/routes-pi-management.test.ts
|
||
```
|
||
|
||
- [ ] Refactor the long instruction blob into data-driven platform sections and small semantic components. Keep state local to Pi Management unless there is an existing shared disclosure component.
|
||
- [ ] Apply a bounded height plus `overflow-y: auto`/`scroll` to the actual element containing the full section; remove ancestor wheel/overflow rules that block movement.
|
||
- [ ] Implement or verify sanitized-log behavior end to end without exposing raw secrets to the browser.
|
||
- [ ] Rerun focused tests, type checks, builds, and complete frontend/backend suites.
|
||
|
||
```bash
|
||
cd frontend
|
||
npx vitest run
|
||
npx tsc -b
|
||
npm run build
|
||
cd ../backend
|
||
npx vitest run
|
||
npx tsc --noEmit -p .
|
||
npm run build
|
||
```
|
||
|
||
- [ ] Commit the Pi Management UX slice without overwriting `deploy/pi/models.json` or `deploy/pi/settings.json`.
|
||
|
||
```bash
|
||
git add frontend/src/shell/PiManagement.tsx frontend/src/shell/PiManagement.test.tsx \
|
||
frontend/src/api/pi-management.ts frontend/src/api/pi-management.test.ts \
|
||
backend/src/pi/management.ts backend/src/routes/pi-management.ts \
|
||
backend/test/pi-management.test.ts backend/test/routes-pi-management.test.ts
|
||
git commit -m "feat(frontend): simplify Pi management guidance"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 14: Replace Active Installation, CLI, and Pi Documentation
|
||
|
||
**Files:**
|
||
|
||
- Rename: `docs/contracts/tht-pi.md` → `docs/contracts/tht-pi.md`
|
||
- Modify: `README.md`
|
||
- Modify: `PROJECT_STATE.md`
|
||
- Modify: `AGENTS.md`
|
||
- Modify: `docs/architecture/overview.md`
|
||
- Modify: `docs/guida-utente.md`
|
||
- Modify: `docs/install/local.md`
|
||
- Modify: `docs/install/server.md`
|
||
- Modify: `docs/install/pi-management.md`
|
||
- Modify: `docs/install/local-workspace-registry.md`
|
||
- Modify: `docs/install/server-workspace-registry.md`
|
||
- Modify: `docs/install/psd-workspace-setup.md`
|
||
- Modify: `docs/install/windows-line-endings.md`
|
||
- Modify: `docs/contracts/tht-dwh.md`
|
||
- Modify: `docs/contracts/workspace-preprocessing-cli.md`
|
||
- Modify: `docs/testing/p2-p6-manual-verification.md`
|
||
- Create: `scripts/test-tht-command-docs.sh`
|
||
- Modify: `scripts/verify-workspace-install-docs.sh`
|
||
- Modify: `scripts/test-verify-workspace-install-docs.sh`
|
||
|
||
### Required onboarding story
|
||
|
||
After cloning ThothII, the normal path is exactly:
|
||
|
||
```bash
|
||
# macOS or Linux, from the repository/worktree root
|
||
bash scripts/install-tht.sh
|
||
tht setup
|
||
```
|
||
|
||
```powershell
|
||
# Windows PowerShell, from the repository/worktree root
|
||
powershell -ExecutionPolicy Bypass -File scripts/install-tht.ps1
|
||
tht setup
|
||
```
|
||
|
||
The documentation must say that `tht setup` validates Docker, creates local configuration, builds images, starts containers, waits for health, and runs diagnostics. It must present `--configure-only` as the explicit way to stop before build/start. `scripts/run-stack.sh` may remain documented as an advanced contributor shortcut, not the primary user onboarding path.
|
||
|
||
### Steps
|
||
|
||
- [ ] Create a documentation contract test that scans active docs and scripts for forbidden user instructions: `thothctl`, `~/bin`, `./bin/tht`, `./tht`, `tht.sh`, `go build`, a required `--installation`, native Pi setup, or editing files inside `core`.
|
||
- [ ] Exclude historical `docs/superpowers/specs/`, `docs/superpowers/plans/`, `docs/plans/`, and `docs/reports/` from stale-name failure; history remains immutable context. Active docs and code examples are not excluded.
|
||
- [ ] Add positive assertions for both installer commands, `tht setup`, `setup --configure-only`, `start --build`, full `update`, `backup`, `restore`, Pi latest-version behavior, worktree discovery, and the three Pi platforms.
|
||
- [ ] Run documentation tests and confirm they fail against current active guidance.
|
||
|
||
```bash
|
||
bash scripts/test-tht-command-docs.sh
|
||
bash scripts/test-verify-workspace-install-docs.sh
|
||
```
|
||
|
||
- [ ] Rewrite active documents around one lifecycle: clone → install `tht` → `tht setup` → `tht doctor` → normal operation → `tht update`/`tht pi update` → `tht backup`/`tht restore`.
|
||
- [ ] Document the host/container command-name boundary once: users invoke the installed native `tht`; the backend and Pi gate invoke the Python `tht` inside `core`. Do not expose a second binary name.
|
||
- [ ] Document root/worktree discovery and optional `--installation` precedence with examples of ambiguity and automation, not as boilerplate on every command.
|
||
- [ ] Update the Pi contract and management guide with Docker-only host-file editing, declarative mounts, credential-file meaning, latest stable Pi default, rollback, and sanitized logs.
|
||
- [ ] Update command reference material from `approved_cli_surface.json`, explicitly omitting the 14 erased commands and marking enhanced administrative commands as advanced where applicable.
|
||
- [ ] Rerun documentation contracts and link checks.
|
||
|
||
```bash
|
||
bash scripts/test-tht-command-docs.sh
|
||
bash scripts/test-verify-workspace-install-docs.sh
|
||
rg -n '\]\([^)]*\.md(#[^)]*)?\)' README.md PROJECT_STATE.md AGENTS.md docs/install docs/contracts docs/architecture docs/testing
|
||
```
|
||
|
||
- [ ] Commit active documentation and contracts.
|
||
|
||
```bash
|
||
git add README.md PROJECT_STATE.md AGENTS.md docs/architecture docs/guida-utente.md \
|
||
docs/install docs/contracts docs/testing scripts/test-tht-command-docs.sh \
|
||
scripts/verify-workspace-install-docs.sh scripts/test-verify-workspace-install-docs.sh
|
||
git commit -m "docs: define the unified tht product workflow"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 15: Run Cross-Platform Gates, Install on This Mac, and Update the Live Stack
|
||
|
||
**Files:**
|
||
|
||
- Create: `docs/reports/2026-08-15-unified-tht-cli-acceptance.md`
|
||
- Modify only if a gate finds a defect: files owned by Tasks 1–14, with a new failing regression test first
|
||
|
||
### Automated acceptance matrix
|
||
|
||
- Go: all native CLI unit, contract, transaction, race, and cross-platform compile tests.
|
||
- Python: Ruff and full non-L2 pytest; run opt-in L2 only when its external GLM/DWH prerequisites are available and record the result separately.
|
||
- Backend: Vitest, TypeScript no-emit typecheck, production build.
|
||
- Frontend: Vitest, TypeScript build, production build, Playwright.
|
||
- Deployment: Compose config for base+local, server, GPU, HTTPS/SSH workspace, preprocessing, and session-server variants.
|
||
- Installer: macOS/Linux shell tests and Windows PowerShell tests; cross-build native binaries.
|
||
- Documentation: active command/copy contracts and link/path checks.
|
||
|
||
### Steps
|
||
|
||
- [ ] Run the complete automated matrix from the repository/worktree root and save command, revision, result, and meaningful skips in the acceptance report.
|
||
|
||
```bash
|
||
cd tools/tht && go test -race ./... && cd ../..
|
||
cd harness && .venv/bin/ruff check . && .venv/bin/pytest -q && cd ..
|
||
cd backend && npx vitest run && npx tsc --noEmit -p . && npm run build && cd ..
|
||
cd frontend && npx vitest run && npx tsc -b && npm run build && npm run e2e && cd ..
|
||
bash scripts/test-install-tht.sh
|
||
pwsh -NoProfile -File scripts/test-install-tht.ps1
|
||
bash scripts/test-tht-command-docs.sh
|
||
bash scripts/test-verify-workspace-install-docs.sh
|
||
bash scripts/unified-deployment-smoke.sh
|
||
```
|
||
|
||
- [ ] Cross-build and inspect all packaged binaries. Run Windows behavior tests in the existing Windows CI/VM path rather than claiming success from compilation alone.
|
||
|
||
```bash
|
||
bash scripts/build-tht.sh --all
|
||
```
|
||
|
||
- [ ] Before touching the Mac installation, resolve the exact existing executables with `command -v`, `type -a`, file metadata, and hashes. Remove only the obsolete development `thothctl` binary or symlink that was positively identified; do not remove any directory or installation data.
|
||
- [ ] Install the newly tested Mac binary with `bash scripts/install-tht.sh`, start a fresh terminal lookup, and verify that `tht` resolves without a relative path while `thothctl` no longer resolves.
|
||
|
||
```bash
|
||
command -v tht
|
||
type -a tht
|
||
tht version
|
||
tht help
|
||
```
|
||
|
||
- [ ] From `/Users/mp/projects/ThothII/.worktrees/p8-l2-live-session-smoke`, verify automatic worktree discovery and the optional installation override. Confirm no command searches for `/Users/mp/thothii-installation.yaml`.
|
||
- [ ] Inspect current sessions and container health. If no unsafe active work exists, update the live installation through the new transaction, using drain only as needed, then verify the stack.
|
||
|
||
```bash
|
||
tht update --check-only
|
||
tht update --yes --drain
|
||
tht status
|
||
tht doctor
|
||
tht pi status
|
||
tht pi doctor
|
||
tht pi test
|
||
```
|
||
|
||
- [ ] Verify port 8080 in a real browser with Playwright: open Pi Management; require all three platform panels closed initially; open each independently; scroll the bounded panel using wheel and keyboard; verify structured Linux/macOS/Windows text; verify only direct `tht` commands; click “Show sanitized logs” and inspect loading/success/empty/error behavior; confirm GLM 5.3 remains available; require no console errors, failed API calls, or exposed secrets.
|
||
- [ ] Compare the live Pi configuration and provider/model list before and after deployment to prove the existing GLM 5.3 changes were preserved.
|
||
- [ ] Record exact versions, image digests, test totals, manual observations, live URL, rollback checkpoint, and any explicitly deferred L2/Windows gate in `docs/reports/2026-08-15-unified-tht-cli-acceptance.md`.
|
||
- [ ] Run `git status --short` and a final diff audit. Confirm no unrelated files, secrets, `.playwright-cli/`, or stale model fixtures were staged.
|
||
- [ ] Commit only the acceptance report and regression fixes, if any.
|
||
|
||
```bash
|
||
git add docs/reports/2026-08-15-unified-tht-cli-acceptance.md
|
||
git commit -m "test: record unified tht acceptance"
|
||
```
|
||
|
||
---
|
||
|
||
## Approval Boundary
|
||
|
||
Approval of this plan authorizes implementation of Tasks 1–15 in order, including:
|
||
|
||
- replacing `thothctl` with the single installed command `tht` without compatibility aliases;
|
||
- installing the finished CLI on this Mac after automated gates pass;
|
||
- removing only the positively identified obsolete Mac development executable;
|
||
- updating/recreating the live Docker Compose services and verifying the GUI on port 8080;
|
||
- removing the 14 approved workflow CLI commands and implementing the 8 approved enhancements;
|
||
- changing active project documentation and Pi Management guidance as specified;
|
||
- creating backup/restore checkpoints needed to make updates recoverable.
|
||
|
||
Approval does **not** authorize deleting user data, secrets, workspaces, sessions, unrelated worktree changes, or historical design/audit documents. It does not authorize overwriting the current GLM 5.3 configuration. Any newly discovered decision that materially changes this architecture, command surface, data-safety policy, or live-deployment scope must return to the user for approval before implementation continues.
|
||
|
||
Implementation should stop for review after these four milestones:
|
||
|
||
1. Tasks 1–5: one installable command and complete setup.
|
||
2. Tasks 6–10: diagnostics, Pi/product updates, backup, restore, and rollback.
|
||
3. Tasks 11–14: approved command-surface changes, frontend, and documentation.
|
||
4. Task 15: Mac installation and live acceptance on port 8080.
|