docs: converge operator guidance on tht

This commit is contained in:
2026-08-19 16:12:11 +02:00
parent 32a17d83a9
commit 1184b6db16
29 changed files with 544 additions and 380 deletions
+10 -4
View File
@@ -13,7 +13,12 @@ detail. Design history lives in `docs/superpowers/specs/` and `docs/superpowers/
The repo has three independently-built layers. Run the local Docker stack with `./scripts/run-stack.sh` after creating `deploy/env/local.env`; it starts the base+local Compose profile with `frontend`, `core`, `qdrant`, `embedding`, and the one-shot `embedding-model-init`. The core image contains Pi. Qdrant and Ollama are internal Compose services; DWH and LLM remain external configuration endpoints.
**harness/** (Python `tht` CLI + Pi gate extension)
**Native host CLI `tht`** (`tools/tht/`)
- Operator surface: `setup`, `start`, `stop`, `status`, `doctor`, `auth`, `workspace`, and `pi`.
- Use `tht --installation <absolute-path>/thothii-installation.yaml <command>` for installation,
authentication, diagnostics, lifecycle, and workspace operations.
**harness/** (Python workflow `tht` CLI + Pi gate extension)
- Install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` (puts `tht` on PATH)
- Test: `.venv/bin/pytest -q` — `l2` (real GLM + remote DB) is opt-in via `addopts = -m 'not l2'`; `l0` (testcontainers) needs Docker
- Single test: `.venv/bin/pytest tests/test_session_mutations.py::test_set_name -v` (or `-k <pattern>`); include e2e with `-m l2`
@@ -37,8 +42,8 @@ No ESLint on the TS layers — `tsc` is the gate. Tests use vitest + MSW (no net
frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness → DWH (read-only)
```
- **The harness owns the workflow and all persistence.** `tht` (Python) is a deterministic
CLI; `harness/.pi/extensions/tht-gate.js` is a Pi extension that drives an **8-phase
- **The harness owns the workflow and all persistence.** The Python workflow CLI `tht` inside
`core` is deterministic; `harness/.pi/extensions/tht-gate.js` is a Pi extension that drives an **8-phase
NL→SQL workflow**. The single source of workflow truth is `harness/workflow.yaml`; the
orchestration rules the model must follow are `harness/.pi/skills/tht-sessione/SKILL.md`.
"Current phase" is computed by folding the decision ledger (`harness/tht/phase.py`), not
@@ -51,7 +56,8 @@ frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness →
There is no verbatim transcript store. A resumed Pi process rebuilds context from
`tht session show <id>` + the on-disk artifacts.
- **The backend is a thin bridge with no database.** `ThtRunner` shells `tht` subcommands;
- **The backend is a thin bridge with no database.** `ThtRunner` shells the Python workflow `tht`
subcommands inside `core`;
`PiProcessManager` runs one Pi child per session and bridges its RPC stream;
`SessionBridge` maps Pi RPC events → client events (`ui_request`/`text_delta`/`info`);
`SseHub` fans them out over SSE to the browser. App settings live in a JSON file
+26 -26
View File
@@ -4,7 +4,7 @@
> **Requisito finale del progetto (owner, 2026-08-11):** al termine dell'ultima fase tecnica deve
> essere prodotto un documento unico che guidi l'utente passo-passo su (1) come preparare il
> repository dei workspace su Git secondo le regole del progetto, (2) come usare gli strumenti di
> ThothII per il repository (app + CLI `thothctl`), (3) come usare l'applicazione ThothII di base
> ThothII per il repository (app + CLI `tht`), (3) come usare l'applicazione ThothII di base
> (sessioni, domande, gate). Il documento userà parole semplici ed esempi; i dettagli tecnici
> resteranno nei contratti esistenti. Esempio pratico completo: Policlinico San Donato.
> Last updated: 2026-08-18 (final-review fix round 2 recorded; native Windows authentication gate
@@ -14,8 +14,8 @@
### Authentication final-review fix round 2 — remediation PASS, release gates remain (2026-08-18)
- Frozen source is `2a9359071257f9b8a71d36ec2bbb25b161003f81` on `feat/thoth-auth`.
Source and evidence are separate commits; `.playwright-cli/` and `.thothctl/` remain the only
untracked paths.
Source and evidence are separate commits; generated local runtime-state directories remain
untracked and must not be staged.
- Local PASS on the frozen source: exact `safeio`/`backup`/`authstorage` tests, full Go race suite,
`go vet`, macOS host build, Windows amd64 package cross-compiles, and Windows CLI build.
- The lifecycle tests now use context-aware gate publication/release, bounded waits for stages,
@@ -60,7 +60,7 @@
lock and fails closed on conflicts.
- **Revision-scoped records:** schema and Evidence Qdrant point IDs, payloads and queries include
`workspace_revision`; memory/solved stay workspace-wide.
- **Operator contract:** `thothctl` now carries `effectiveConfigIdentity`/`configFingerprint`/
- **Operator contract:** `tht` now carries `effectiveConfigIdentity`/`configFingerprint`/
`inputFingerprint` in results; the operator config lease path is deterministic for the same
revision+identity.
- **Retained evidence:** `.artifacts/p3-integration/p3-da9428d84f152fe059d41a89436496b7/`
@@ -75,8 +75,8 @@
the 8 required keyword payload indexes) and adds missing indexes, but never mutates an
incompatible collection (`semantic_index_incompatible`); the operator path keeps
`require_existing` semantics.
- **Host CLI:** `thothctl workspace vector inspect` (read-only contract report) and
`thothctl workspace vector rebuild --workspace <id> --collection <name> --confirm <name> --destroy`
- **Host CLI:** `tht workspace vector inspect` (read-only contract report) and
`tht workspace vector rebuild --workspace <id> --collection <name> --confirm <name> --destroy`
(guarded delete/recreate of only the descriptor-owned collection, with durable state before
deletion and verification after recreation; mismatched confirmation or missing `--destroy`
→ exit 2).
@@ -84,7 +84,7 @@
(`qdrantEnsure` self-heal for admission; default `require_existing` elsewhere),
`backend/src/workspaces/runtime-config-lease.ts` (lease exposes `semanticQdrantUrl`),
`backend/src/workspace-maintenance.ts` + `preprocessing-service.ts` (`vector-inspect`/`vector-rebuild`
operator commands), `tools/thothctl/internal/workspaceops/operations.go` (+tests).
operator commands), `tools/tht/internal/workspaceops/operations.go` (+tests).
- **Automated acceptance:** PASS 11/11 (run `p4-466bbfdea9ef3111f36baa99fc2d64aa`,
report `.artifacts/p4-integration/p4-466bbfdea9ef3111f36baa99fc2d64aa/` retained via `--keep`,
bound to clean source commit `e056c19e6214254a9e3b2390e24c389920b84e95`): preflight, clean_state,
@@ -111,7 +111,7 @@
`paths.artifacts`/`indexes`/`memory`/`sessions` stay workspace-global (the binding-keyed DWH cache
at `artifacts.parent` is untouched); the harness resolves annotations from `annotations_root` with a
legacy fallback.
- **Review primitive:** `thothctl ... workspace schema accept --run <id> --yes` is the only human FK
- **Review primitive:** `tht ... workspace schema accept --run <id> --yes` is the only human FK
review path. It validates the current synced Git blob with the harness parser and records
`{ reviewedCandidatesDigest, annotationsDigest, workspaceRevision, blobId }`. Missing `--yes`, an
unknown run, an empty/malformed blob, or a non-matching candidate fails closed (`annotation_invalid`)
@@ -124,7 +124,7 @@
annotations.ts`, `backend/src/workspaces/git-repository.ts` (`annotationsObject`),
`backend/src/workspaces/registry.ts` (activation validation + sync), `backend/src/workspaces/
preprocessing-service.ts` (`acceptSchema` + continuation gate), `backend/src/workspace-maintenance.ts`
(`schema-accept`), `tools/thothctl/internal/workspaceops/operations.go` (+tests), `harness/tht/
(`schema-accept`), `tools/tht/internal/workspaceops/operations.go` (+tests), `harness/tht/
config.py` + `cli/schema_cmd.py` (`paths.annotations_root`), `docs/contracts/
workspace-preprocessing-cli.md`.
- **Gates:** backend **689/689** + tsc clean; Go build+test 9/9; harness focused schema/annotations
@@ -188,12 +188,12 @@
`deploy/psd/secrets/`). Real operator config is wired (gitignored): `deploy/psd/operator.env`,
`workspace-bindings.env`, `thothii-installation.yaml`, `connector-secrets.yaml` + `secrets/`
(DWH X-API-Key reused from the legacy `.env`; no CA — the DWH REST is public HTTPS).
- **Stack live:** started via `thothctl start` (project `thothii-70417a3e30ea`), all services
- **Stack live:** started via `tht start` (project `thothii-70417a3e30ea`), all services
healthy, `qwen3-embedding:0.6b` present; the registry cloned + activated `psd-clinical`
(`ready`); `thothctl workspace inspect` returns `ok` with descriptor/catalog/runtime identities.
Gotcha recorded: `thothctl` uses a per-descriptor Compose project name, so the stack must be
started with `thothctl start` (not a raw `compose-with-preflight.sh up`).
- **Preprocessing live (2026-08-13):** with VPN active, `thothctl workspace preprocess run
(`ready`); `tht workspace inspect` returns `ok` with descriptor/catalog/runtime identities.
Gotcha recorded: `tht` uses a per-descriptor Compose project name, so the stack must be
started with `tht start` (not a raw `compose-with-preflight.sh up`).
- **Preprocessing live (2026-08-13):** with VPN active, `tht workspace preprocess run
--workspace psd-clinical` **succeeded** against the real PSD DWH — DWH introspection + LSH
(163 tables / 2275 columns), FK review (no new candidates: the 42 KB curated annotations are
authoritative), schema index (2438 records) and filesystem Evidence index (36 docs / 43 chunks).
@@ -213,7 +213,7 @@
### Final aggregate P2–P6 verification — automated PASS, manual PENDING (2026-08-13)
- **Aggregate process goal:** one clean-state run exercises the complete DWH → FK → schema →
filesystem Evidence chain through `thothctl`/the operator surface, proves idempotency and
filesystem Evidence chain through `tht`/the operator surface, proves idempotency and
revision isolation, proves a second installation consumes the same Git workspace with its own
state, exercises unsafe-tree and bound negatives, and cleans only owned resources.
- **Automated acceptance:** PASS 12/12 (run `p2p6-ee542112c526ef0d4c25ddf6c8bc164b`, report
@@ -224,7 +224,7 @@
`scripts/p2p6-acceptance.sh` / `backend/scripts/p2p6-acceptance.mjs` (+unit test).
- **Full suites + builds (design §10):** harness **873 passed / 4 deselected** (with color disabled;
the forced-color environment splits `--help` flags and trips the gate-CLI consistency test only);
backend **698/698** + tsc + build; frontend **364/364** + `tsc -b` + build; `thothctl` Go
backend **698/698** + tsc + build; frontend **364/364** + `tsc -b` + build; `tht` Go
build+test **9/9**; `git diff --check` clean.
- **Manual acceptance:** PENDING — "Final aggregate P2–P6 verification" in
`docs/testing/p2-p6-manual-verification.md`.
@@ -233,14 +233,14 @@
- **`docs/guida-utente.md`** (Italian, simple words + examples) covers: (1) preparing the workspace
Git repository (catalog + schema-v3 descriptor + Evidence + curated annotations), (2) using the
ThothII tools for the repository (`thothctl` commands + read-only workspace management), and (3)
ThothII tools for the repository (`tht` commands + read-only workspace management), and (3)
using the base ThothII application (sessions, questions, gates). It ends with a complete
Policlinico San Donato walkthrough and links to the technical contracts.
- Registered in the MkDocs nav (`mkdocs.yml`). Owner review PENDING.
### P2 host preprocessing CLI — implementation complete, automated PASS, manual PENDING (2026-08-11)
- **Scope:** P2 (PRD D2, based on the P1.1 registry contract): the installed native `thothctl`
- **Scope:** P2 (PRD D2, based on the P1.1 registry contract): the installed native `tht`
binary is the only host interface for workspace preprocessing. Commands: `workspace inspect`,
`preprocess dwh`, `schema suggest-fks`, `schema check`, `index-schema`, `preprocess evidence`,
`preprocess run`, with the exact grammar, file-ingress bounds, result contract and exit codes in
@@ -378,7 +378,7 @@ P1.1 manual acceptance: PASS (owner approval 2026-08-11)
and proves exact cleanup of compose containers, volumes, networks, and only that smoke image.
Unified deployment smoke passed in **125.57s**; update-only rollback smoke
passed in **85.40s**; Linux server deployment smoke passed in **55.99s**. The previously
observed `thothctl` rollback failure did not recur.
observed `tht` rollback failure did not recur.
- **Task 13 image and manual-gate notes.** Verified pinned runtime images:
`qdrant/qdrant:v1.18.2@sha256:75eab8c4ba42096724fdcfde8b4de0b5713d529dde32f285a1f86fdcb2c9e50c`
and
@@ -397,12 +397,12 @@ P1.1 manual acceptance: PASS (owner approval 2026-08-11)
embedding endpoint coupling.
- **Final review fix verification.** Backend Vitest passed **477/477** plus TypeScript and build;
harness pytest passed **827 passed / 4 deselected** with the existing 74 warnings; touched Python
files are Ruff-clean. The complete `thothctl` Go suite, deterministic backup/restore safety test,
files are Ruff-clean. The complete `tht` Go suite, deterministic backup/restore safety test,
internal semantic Compose contract, no-deployment-coupling gate, CPU/offline semantic smoke, and
unified deployment smoke all pass after the final fix. The intermittent `thothctl` rollback failure was
unified deployment smoke all pass after the final fix. The intermittent `tht` rollback failure was
traced to Docker Desktop alternating equivalent bind sources between `/private/...` and
`/host_mnt/private/...`. Exact state-v4 source hashes remain unchanged; only fresh bind
observations made by a Darwin `thothctl` carry non-serialized aliases for the rollback
observations made by a Darwin `tht` carry non-serialized aliases for the rollback
comparison, so pre-fix recovery state remains readable and Linux `/host_mnt` paths remain
distinct. The rollback-only smoke passed twice consecutively after each fix revision, and the
subsequent full unified smoke passed with exact cleanup.
@@ -414,7 +414,7 @@ P1.1 manual acceptance: PASS (owner approval 2026-08-11)
- **Release coverage.** `scripts/unified-deployment-smoke.sh` gates the two-service render/build,
frontend-to-core routing, embedded pinned Pi, Git registry bootstrap, offline recreation, valid
update, invalid-update retention, and the four persistent stores. `scripts/thothctl-update-smoke.sh`
update, invalid-update retention, and the four persistent stores. `scripts/tht-update-smoke.sh`
independently exercises the bad-Pi update and automatic rollback path.
`scripts/server-deployment-smoke.sh` starts the server plus required session overlays with the
same smoke-built core/frontend images, disposable bind roots/secrets/session configuration,
@@ -422,7 +422,7 @@ P1.1 manual acceptance: PASS (owner approval 2026-08-11)
- **Isolation and disclosure boundary.** Every run generates a unique temporary root, Compose
project, container/image names, transaction image tags, and run label. The rollback fixture uses
an immutable `hello-world` digest whose preflight exits successfully, guaranteeing the stopped
core state required by `thothctl` compensation. Cleanup includes stopped project containers in
core state required by `tht` compensation. Cleanup includes stopped project containers in
its final ownership check immediately before teardown and removes only exact containers,
Compose resources, image references, control state, and temporary files. There is no global
prune. Failure diagnostics are bounded and sanitized, and all credentials/endpoints used by the
@@ -431,7 +431,7 @@ P1.1 manual acceptance: PASS (owner approval 2026-08-11)
- **Cross-platform CI contract.** `.github/workflows/deployment.yml` uses immutable action commits,
pinned supported Node and Go versions, runs LF/Compose/secret/coupling/docs/TypeScript gates on
Linux, runs each Linux Docker smoke once under its own outer timeout, and copies the Windows
source into a path containing spaces before building/invoking native `thothctl` and rendering
source into a path containing spaces before building/invoking native `tht` and rendering
Compose. The optional `windows_docker_startup` dispatch targets a labelled self-hosted Windows
Docker Desktop/WSL2 runner and performs bounded two-service startup and exact cleanup. No local
Windows or Windows Docker execution is claimed until that manual job is recorded.
@@ -439,7 +439,7 @@ P1.1 manual acceptance: PASS (owner approval 2026-08-11)
frontend **386/386** plus TypeScript, and harness **862 passed / 5 L2 deselected** are green.
Review round 1 ran each Docker smoke exactly once without retry. Unified (`103.86s`) and
update-only (`46.45s`) passed build/start, core/Pi/registry/persistence setup and the stopped
candidate preflight, but `thothctl` stopped before mutation at its active-session inventory gate.
candidate preflight, but `tht` stopped before mutation at its active-session inventory gate.
Round 2 replaces presence-only fixture checks with generated Compose renders plus the production
workspace resolver; this found and fixed missing explicit direct transport selections. The
clean-server preflight now atomically initializes the three hidden Pi-agent targets under the
+5 -5
View File
@@ -142,7 +142,7 @@ the teardown if any container, volume, or network has a foreign run label.
```sh
bash scripts/unified-deployment-smoke.sh
bash scripts/thothctl-update-smoke.sh
bash scripts/tht-update-smoke.sh
bash scripts/server-deployment-smoke.sh
```
@@ -151,10 +151,10 @@ registry, recreates with the Git remote offline, activates a valid Git update, r
content while retaining the valid snapshot, and checks the four persistence volumes. The unified
and update-only smokes
inject a digest-pinned non-core candidate under a deliberately mismatched Pi version and require
`thothctl pi update` to roll back while preserving settings, sessions, Pi state, registry revision,
`tht pi update` to roll back while preserving settings, sessions, Pi state, registry revision,
and mount identity. The rollback candidate is the digest-pinned `hello-world` executable: a
preflight proves that it exits successfully, so the failed replacement core satisfies
`thothctl`'s stopped-core compensation precondition. The server smoke uses the same smoke-built
`tht`'s stopped-core compensation precondition. The server smoke uses the same smoke-built
core/frontend images with the server and required session overlays, disposable bind roots and
secret files, upstream-auth checks, and a fail-closed `503` assertion for its deliberately
unavailable disposable session endpoint. No real provider, database credential, or repository
@@ -190,7 +190,7 @@ The deterministic native Windows contract is:
```
It checks Git's CRLF/LF attributes and bytes, copies tracked source into a temporary path containing
spaces, builds and invokes native Windows `thothctl` there, and renders exactly `core` plus
spaces, builds and invokes native Windows `tht` there, and renders exactly `core` plus
`frontend` without starting containers. On a supported self-hosted Windows Docker Desktop/WSL2
runner, dispatch the deployment workflow with `windows_docker_startup=true`; that job executes:
@@ -198,7 +198,7 @@ runner, dispatch the deployment workflow with `windows_docker_startup=true`; tha
.\scripts\test-windows-clone-contract.ps1 -DockerStartup
```
Startup mode adds bounded image build/two-service health startup, installation-aware `thothctl`
Startup mode adds bounded image build/two-service health startup, installation-aware `tht`
status, stopped-container-aware ownership checks, and exact cleanup. The ordinary hosted Windows
job remains deterministic and does not claim Docker startup.
+1 -1
View File
@@ -86,7 +86,7 @@ workspace://<workspace-id>@v1:<sha256 of the canonical effective configuration>
```
It is the same for the operator CLI and for application sessions, because both derive it from the
same rendered configuration. That is the guarantee that the work prepared by `thothctl` is exactly
same rendered configuration. That is the guarantee that the work prepared by `tht` is exactly
what the sessions will consume.
## Why a rerun can be instant or take minutes
@@ -1,19 +1,19 @@
# `thothctl pi` lifecycle contract
# `tht pi` lifecycle contract
`thothctl` is the only component that drives Docker lifecycle operations. The `core` container
`tht` is the only component that drives Docker lifecycle operations. The `core` container
does not mount a Docker socket, and Pi is never updated in a running container.
## Inspection and configuration
```text
thothctl pi status
thothctl pi doctor
thothctl pi test
thothctl pi logs
thothctl pi configure
tht pi status
tht pi doctor
tht pi test
tht pi logs
tht pi configure
```
When `--installation` is omitted, `thothctl` first uses `THOTHII_INSTALLATION` and otherwise
When `--installation` is omitted, `tht` first uses `THOTHII_INSTALLATION` and otherwise
discovers one valid `thothii-installation.yaml` in the current project tree, including an immediate
`deploy/*` directory. Use `--installation /absolute/path/thothii-installation.yaml` as an explicit
override when the descriptor is outside that tree or more than one installation is available.
@@ -30,7 +30,7 @@ models come from the backend's closed model list, and the model choices are rest
selected provider. In non-interactive use, all choices must be explicit:
```text
thothctl pi configure \
tht pi configure \
--provider zai --model glm-5.2 --thinking medium
```
@@ -55,16 +55,16 @@ store.
## Supported Compose entry points and current image
Use `thothctl start`, `stop`, `status`, `logs`, and `doctor` for ordinary installation lifecycle
operations. All `thothctl` Compose commands automatically include the installation-specific
Use `tht start`, `stop`, `status`, `logs`, and `doctor` for ordinary installation lifecycle
operations. All `tht` Compose commands automatically include the installation-specific
durable selector when it exists:
```text
<projectDirectory>/.thothctl/<installation-id>/current-image.yaml
<projectDirectory>/.tht/<installation-id>/current-image.yaml
```
This selector is part of the supported installation state: it keeps a verified Pi image selected
across a fresh `thothctl` process, stop/start, reconcile, and source checkout whose base image is
across a fresh `tht` process, stop/start, reconcile, and source checkout whose base image is
digest-pinned. Do not delete or hand-edit it. Direct raw `docker compose` lifecycle commands bypass
this protection and are unsupported. Advanced documented Compose rendering must use
`scripts/compose-with-preflight.sh` and include the same selector with `-f` when present; connector
@@ -75,7 +75,7 @@ secret overrides must never bypass that preflight wrapper.
Configuration reload is a separate lifecycle operation from an image update:
```text
thothctl pi restart --yes [--drain]
tht pi restart --yes [--drain]
```
`--yes` is required after reviewing the planned core recreation. Restart activates the durable
@@ -95,8 +95,8 @@ identity, and the complete persistence-mount fingerprint.
Restart and update keep separate recovery state:
```text
<projectDirectory>/.thothctl/<installation-id>/restart-state.json
<projectDirectory>/.thothctl/<installation-id>/update-state.json
<projectDirectory>/.tht/<installation-id>/restart-state.json
<projectDirectory>/.tht/<installation-id>/update-state.json
```
The files are mode `0600` and share one installation lifecycle lock, so restart, update, and
@@ -112,8 +112,8 @@ recovery rather than deleting recovery material.
The normal update uses the repository's pinned version and build source automatically:
```text
thothctl pi update
thothctl pi update --version 0.81.0
tht pi update
tht pi update --version 0.81.0
```
With no `--version`, the command reads the single default `ARG PI_VERSION=<version>` from
@@ -124,10 +124,10 @@ command, drains active sessions without terminating them, builds the candidate,
Advanced registry updates remain available and require an immutable digest:
```text
thothctl pi update \
tht pi update \
--version 0.81.0 --source build --yes --drain
thothctl pi update \
tht pi update \
--version 0.81.0 --source pull \
--image registry.example.invalid/thothii-core@sha256:<64-lowercase-hex-digits> --yes
```
@@ -170,9 +170,9 @@ non-secret rendered configuration; and the complete persistence-mount fingerprin
Recovery state and lock diagnostics live under:
```text
<projectDirectory>/.thothctl/<installation-id>/update-state.json
<projectDirectory>/.thothctl/<installation-id>/restart-state.json
<projectDirectory>/.thothctl/<installation-id>/*.lock.owner.json
<projectDirectory>/.tht/<installation-id>/update-state.json
<projectDirectory>/.tht/<installation-id>/restart-state.json
<projectDirectory>/.tht/<installation-id>/*.lock.owner.json
```
Each recovery file is mode `0600`. Update state records transaction-scoped image identities, mount
@@ -195,7 +195,7 @@ gate can open.
For a failed update with `update-state.json`, first run:
```text
thothctl pi rollback --yes
tht pi rollback --yes
```
Rollback restores the image recorded in update state, but it checks restart state before making any
@@ -206,8 +206,8 @@ reported problem, then use maintenance recovery.
Inspect and clean a stale durable gate with:
```text
thothctl pi maintenance status
thothctl pi maintenance recover --yes
tht pi maintenance status
tht pi maintenance recover --yes
```
`maintenance recover` restores the captured restart image pin and lifecycle override when needed,
+13 -13
View File
@@ -1,42 +1,42 @@
# Workspace preprocessing CLI contract
`thothctl` is the only supported host entrypoint for workspace preprocessing.
`tht` is the only supported host entrypoint for workspace preprocessing.
## Invocation
```text
thothctl --installation <absolute>/thothii-installation.yaml workspace inspect
tht --installation <absolute>/thothii-installation.yaml workspace inspect
--workspace <id> [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace preprocess dwh
tht --installation <absolute>/thothii-installation.yaml workspace preprocess dwh
--workspace <id> [--resume <32hex>] [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace schema suggest-fks
tht --installation <absolute>/thothii-installation.yaml workspace schema suggest-fks
--workspace <id>
[--from-sql <regular-file>]... [--assume <column=table>]...
[--output <new-file>] [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace schema check
tht --installation <absolute>/thothii-installation.yaml workspace schema check
--workspace <id>
[--annotations <regular-file> --reviewed-candidates <sha256:hex>]
[--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace schema accept
tht --installation <absolute>/thothii-installation.yaml workspace schema accept
--workspace <id> --run <32hex> --yes [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace index-schema
tht --installation <absolute>/thothii-installation.yaml workspace index-schema
--workspace <id> [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace preprocess evidence
tht --installation <absolute>/thothii-installation.yaml workspace preprocess evidence
--workspace <id> [--dry-run] [--resume <32hex>] [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace preprocess run
tht --installation <absolute>/thothii-installation.yaml workspace preprocess run
--workspace <id> [--resume <32hex>] [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace vector inspect
tht --installation <absolute>/thothii-installation.yaml workspace vector inspect
--workspace <id> [--json]
thothctl --installation <absolute>/thothii-installation.yaml workspace vector rebuild
tht --installation <absolute>/thothii-installation.yaml workspace vector rebuild
--workspace <id> --collection <name> --confirm <name> --destroy [--json]
```
@@ -120,7 +120,7 @@ thothctl --installation <absolute>/thothii-installation.yaml workspace vector re
## Container boundary
`thothctl` resolves the selected `core` image from the rendered installation, converts it to an immutable local image ID, writes a one-shot final override that pins both `core` and `workspace-maintenance` to that ID with `pull_policy: never`, and runs only:
`tht` resolves the selected `core` image from the rendered installation, converts it to an immutable local image ID, writes a one-shot final override that pins both `core` and `workspace-maintenance` to that ID with `pull_policy: never`, and runs only:
```text
docker compose run --rm --no-deps --no-TTY --name <owned-name> workspace-maintenance <fixed-command>
@@ -148,7 +148,7 @@ The request is streamed as one schema-versioned JSON document over stdin. Public
}
```
`thothctl --json` parses the operator stdout strictly and re-encodes only the public fields above.
`tht --json` parses the operator stdout strictly and re-encodes only the public fields above.
`evidence_materialization_required` is retained for pre-P6 compatibility; since P6, filesystem
Evidence is materialized at activation and preprocesses directly.
+16 -16
View File
@@ -112,41 +112,41 @@ Cosa cambia rispetto ai vecchi workspace (se ne avevi uno):
## Parte 2 — Usare gli strumenti ThothII per il repository
Ci sono **due** strumenti: l'**applicazione web** (gestione workspace) e la **CLI `thothctl`**
Ci sono **due** strumenti: l'**applicazione web** (gestione workspace) e la **CLI `tht`**
(preprocessing/operator). L'installazione completa è descritta nei manuali
`docs/install/local-workspace-registry.md` (macOS/Windows/Linux) e
`docs/install/server-workspace-registry.md`.
### 2.1 `thothctl` — comandi principali
### 2.1 `tht` — comandi principali
`thothctl` si invoca sempre con `--installation <percorso>/thothii-installation.yaml`. I comandi
`tht` si invoca sempre con `--installation <percorso>/thothii-installation.yaml`. I comandi
utili, nell'ordine tipico:
```bash
# 1) vedere lo stato di un workspace (revisione e identità)
thothctl --installation <install> workspace inspect --workspace <id> --json
tht --installation <install> workspace inspect --workspace <id> --json
# 2) introspezione del DWH (genera physical.yaml + LSH)
thothctl --installation <install> workspace preprocess dwh --workspace <id> --json
tht --installation <install> workspace preprocess dwh --workspace <id> --json
# 3) suggerire le join (FK) da SQL già approvato
thothctl --installation <install> workspace schema suggest-fks --workspace <id> --from-sql <query>.sql --output <candidati>.yaml --json
tht --installation <install> workspace schema suggest-fks --workspace <id> --from-sql <query>.sql --output <candidati>.yaml --json
# 4) dopo la revisione: pubblicare gli FK curati in Git e accettarli
thothctl --installation <install> workspace schema accept --workspace <id> --run <run-id> --yes --json
tht --installation <install> workspace schema accept --workspace <id> --run <run-id> --yes --json
# 5) indicizzare lo schema (Qdrant)
thothctl --installation <install> workspace index-schema --workspace <id> --json
tht --installation <install> workspace index-schema --workspace <id> --json
# 6) preprocessing dell'Evidence
thothctl --installation <install> workspace preprocess evidence --workspace <id> --json
tht --installation <install> workspace preprocess evidence --workspace <id> --json
# 7) catena completa (DWH → FK → schema → Evidence)
thothctl --installation <install> workspace preprocess run --workspace <id> --json
tht --installation <install> workspace preprocess run --workspace <id> --json
# 8) ispezione/ricostruzione della collection Qdrant (solo manutenzione)
thothctl --installation <install> workspace vector inspect --workspace <id> --json
thothctl --installation <install> workspace vector rebuild --workspace <id> --collection <nome> --confirm <nome> --destroy
tht --installation <install> workspace vector inspect --workspace <id> --json
tht --installation <install> workspace vector rebuild --workspace <id> --collection <nome> --confirm <nome> --destroy
```
Note importanti:
@@ -255,16 +255,16 @@ valida lo schema v3, materializza l'Evidence dal commit fissato e prepara la col
### Passo 1 — preprocessing
```bash
thothctl --installation ~/thothii-installation.yaml workspace preprocess dwh --workspace psd-clinical --json
thothctl --installation ~/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --json
tht --installation ~/thothii-installation.yaml workspace preprocess dwh --workspace psd-clinical --json
tht --installation ~/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --json
```
Se il run si ferma per le join (`manual_review_required`):
```bash
# il curatore rivede i candidati e pubblica psd-clinical/schema/annotations.yaml, poi:
thothctl --installation ~/thothii-installation.yaml workspace schema accept --workspace psd-clinical --run <run-id> --yes --json
thothctl --installation ~/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --resume <run-id> --json
tht --installation ~/thothii-installation.yaml workspace schema accept --workspace psd-clinical --run <run-id> --yes --json
tht --installation ~/thothii-installation.yaml workspace preprocess run --workspace psd-clinical --resume <run-id> --json
```
### Passo 2 — la domanda
+18 -4
View File
@@ -28,7 +28,7 @@ documented GPU prerequisites are satisfied. The embedding contract is fixed at
- A working local installation described by [local.md](local.md).
- A remote Git repository and a read-only deploy credential for this ThothII installation.
- A separate authoring clone in which a workspace curator can edit and publish source revisions.
- `thothctl` built with `bash scripts/build-thothctl.sh`.
- `tht` built with `bash scripts/build-tht.sh`.
## Prepare and publish a workspace source
@@ -51,6 +51,20 @@ Publishing is an author-side Git operation: validate the source, commit it, and
separate authoring clone to the configured branch. This is the only meaning of “publish” in the
workspace lifecycle. ThothII has no author identity and no Git write credential.
## Use the workspace from the application
After the installation is started, use Workspace management from the authenticated application:
1. Run **Update workspace repository** to fetch and validate the configured Git branch into the
application-owned registry. The operation is all-or-nothing and does not modify the authoring
clone.
2. Confirm that the installation-owned `workspace-secrets` storage remains outside the source
repository and contains no credentials in the workspace descriptors.
3. Select the workspace and run **Validate workspace** to verify the active descriptor, catalog,
Evidence, annotations, and runtime bindings.
4. Run **Test connections** only with the approved read-only DWH/Evidence test configuration.
Results are redacted and the workspace source remains unchanged.
<!-- workspace-descriptor-contract:start -->
Schema v3 is the only accepted workspace descriptor.
Schema v1 and v2 workspace descriptors are rejected before activation.
@@ -90,10 +104,10 @@ Use only the installation-aware lifecycle:
```bash
export THT_SOURCE_ROOT=/absolute/path/to/ThothII
THTCTL="$THT_SOURCE_ROOT/tools/thothctl/thothctl"
THT_BIN=tht
INSTALLATION=/absolute/path/to/operator/thothii-installation.yaml
"$THTCTL" --installation "$INSTALLATION" start
"$THTCTL" --installation "$INSTALLATION" doctor
"$THT_BIN" --installation "$INSTALLATION" start
"$THT_BIN" --installation "$INSTALLATION" doctor
```
At startup ThothII clones or fetches the configured repository into its application-managed
+51 -51
View File
@@ -13,10 +13,10 @@ Commands that contain example paths must be changed to absolute paths on your co
- **macOS:** use Terminal and Docker Desktop. Apple Silicon and Intel are supported by the local
image build.
- **Windows PowerShell:** use Docker Desktop with its WSL2 engine, Git for Windows, the Windows
build launcher, and `thothctl-windows-amd64.exe`.
build launcher, and `tht-windows-amd64.exe`.
- **Windows WSL2 (recommended):** enable Docker Desktop integration for your Linux distribution,
clone under `/home/<user>` rather than `/mnt/c`, and follow the Linux shell commands.
- **Linux PC:** use Docker Engine plus the Compose v2 plugin and the Linux `thothctl` binary.
- **Linux PC:** use Docker Engine plus the Compose v2 plugin and the Linux `tht` binary.
Windows users must also read [Windows and WSL2 line endings](windows-line-endings.md) before the
first build.
@@ -167,7 +167,7 @@ Then use `host.docker.internal` in the endpoint. `extra_hosts: host.docker.inter
is a host routing aid, not a bundled service. Prefer a real DNS name for independently operated
services; retain TLS and authentication even when co-located.
## Build ThothII and thothctl
## Build ThothII and tht
The canonical local Compose smoke uses the base file plus the local profile. Keep this exact
base+profile command available for install verification:
@@ -184,45 +184,45 @@ From the repository root, macOS/Linux/WSL2 users run:
```sh
bash scripts/build-local.sh
bash scripts/build-thothctl.sh
bash scripts/build-tht.sh
```
Native PowerShell users run:
```powershell
powershell -ExecutionPolicy Bypass -File scripts/build-local.ps1
& "C:\Program Files\Git\bin\bash.exe" scripts/build-thothctl.sh
& "C:\Program Files\Git\bin\bash.exe" scripts/build-tht.sh
```
The second command uses Docker to create native operator binaries under `dist/thothctl`; users do
not need to know or install Go. Select `thothctl-darwin-arm64` or `-amd64` on macOS,
`thothctl-linux-amd64` or `-arm64` on Linux/WSL2, and `thothctl-windows-amd64.exe` on Windows.
The second command uses Docker to create native operator binaries under `dist/tht`; users do
not need to know or install Go. Select `tht-darwin-arm64` or `-amd64` on macOS,
`tht-linux-amd64` or `-arm64` on Linux/WSL2, and `tht-windows-amd64.exe` on Windows.
Copy the selected file to the protected operator directory and, on macOS/Linux, run `chmod 0755`
on it.
## Start and verify
Set convenient variables (PowerShell users use `$THTCTL` and `$INSTALLATION` with `& $THTCTL`):
Every operator call has the form `thothctl --installation <absolute-descriptor> <command>`.
Set convenient variables (PowerShell users use `$THT_BIN` and `$INSTALLATION` with `& $THT_BIN`):
Every operator call has the form `tht --installation <absolute-descriptor> <command>`.
```sh
THTCTL=/absolute/path/to/thothii-operator/thothctl
THT_BIN=/absolute/path/to/thothii-operator/tht
INSTALLATION=/absolute/path/to/thothii-operator/thothii-installation.yaml
"$THTCTL" --installation "$INSTALLATION" update --check-only
"$THTCTL" --installation "$INSTALLATION" start
"$THTCTL" --installation "$INSTALLATION" status
"$THTCTL" --installation "$INSTALLATION" doctor
"$THT_BIN" --installation "$INSTALLATION" update --check-only
"$THT_BIN" --installation "$INSTALLATION" start
"$THT_BIN" --installation "$INSTALLATION" status
"$THT_BIN" --installation "$INSTALLATION" doctor
```
Native PowerShell uses the same order:
```powershell
$THTCTL = 'C:\Users\operator\thothii-operator\thothctl.exe'
$THT_BIN = 'C:\Users\operator\thothii-operator\tht.exe'
$INSTALLATION = 'C:\Users\operator\thothii-operator\thothii-installation.yaml'
& $THTCTL --installation $INSTALLATION update --check-only
& $THTCTL --installation $INSTALLATION start
& $THTCTL --installation $INSTALLATION status
& $THTCTL --installation $INSTALLATION doctor
& $THT_BIN --installation $INSTALLATION update --check-only
& $THT_BIN --installation $INSTALLATION start
& $THT_BIN --installation $INSTALLATION status
& $THT_BIN --installation $INSTALLATION doctor
```
Wait for both services, then check the same-origin frontend and direct loopback core:
@@ -230,8 +230,8 @@ Wait for both services, then check the same-origin frontend and direct loopback
```sh
curl --fail http://127.0.0.1:8080/health
curl --fail http://127.0.0.1:8787/health
"$THTCTL" --installation "$INSTALLATION" pi doctor
"$THTCTL" --installation "$INSTALLATION" pi test
"$THT_BIN" --installation "$INSTALLATION" pi doctor
"$THT_BIN" --installation "$INSTALLATION" pi test
```
Native PowerShell must call `curl.exe` explicitly; Windows PowerShell may otherwise resolve `curl`
@@ -240,11 +240,11 @@ to `Invoke-WebRequest`:
```powershell
curl.exe --fail --silent --show-error http://127.0.0.1:8080/health
curl.exe --fail --silent --show-error http://127.0.0.1:8787/health
& $THTCTL --installation $INSTALLATION pi doctor
& $THTCTL --installation $INSTALLATION pi test
& $THT_BIN --installation $INSTALLATION pi doctor
& $THT_BIN --installation $INSTALLATION pi test
```
Open <http://127.0.0.1:8080>. If a check fails, run `thothctl ... logs` or `pi logs`; these are
Open <http://127.0.0.1:8080>. If a check fails, run `tht ... logs` or `pi logs`; these are
bounded and sanitize declared secrets. Do not publish either loopback port.
## Update an installation
@@ -254,7 +254,7 @@ selected by the durable, installation-specific `current-image.yaml` after every
Therefore rebuilding `thothii-core:local` followed by `update --check-only` does not reconcile a
previous `pi update`: the old promoted core would remain selected.
Do not delete or edit the selector. `thothctl status` is the installation-aware selector test. If
Do not delete or edit the selector. `tht status` is the installation-aware selector test. If
the running core image is the base `thothii-core:local` image, no Pi update has promoted a durable
lifecycle image and an ordinary same-Pi-version source rebuild/start is supported. If status shows
a lifecycle image and the pulled Pi pin is unchanged, `pi update` would be a no-op and the procedure
@@ -284,14 +284,14 @@ if ! NEXT_PI_VERSION="$(sed -n 's/^ARG PI_VERSION=//p' docker/core.Dockerfile)";
abort_update "could not read the pulled Pi pin"
fi
[[ -n "$NEXT_PI_VERSION" && "$NEXT_PI_VERSION" != *$'\n'* ]] || abort_update "expected one pinned default PI_VERSION"
if ! INSTALLATION_STATUS="$("$THTCTL" --installation "$INSTALLATION" status)"; then
abort_update "thothctl status failed"
if ! INSTALLATION_STATUS="$("$THT_BIN" --installation "$INSTALLATION" status)"; then
abort_update "tht status failed"
fi
if ! RUNNING_PI_VERSION="$("$THTCTL" --installation "$INSTALLATION" pi status)"; then
abort_update "thothctl pi status failed"
if ! RUNNING_PI_VERSION="$("$THT_BIN" --installation "$INSTALLATION" pi status)"; then
abort_update "tht pi status failed"
fi
RUNNING_PI_VERSION="${RUNNING_PI_VERSION#Pi version: }"
[[ -n "$RUNNING_PI_VERSION" ]] || abort_update "thothctl pi status returned no version"
[[ -n "$RUNNING_PI_VERSION" ]] || abort_update "tht pi status returned no version"
COMPACT_STATUS="${INSTALLATION_STATUS//[[:space:]]/}"
USES_BASE_CORE=false
@@ -305,23 +305,23 @@ if [[ "$NEXT_PI_VERSION" == "$RUNNING_PI_VERSION" ]]; then
fi
if ! bash scripts/build-local.sh; then abort_update "the local image build failed"; fi
if ! bash scripts/build-thothctl.sh; then abort_update "the thothctl build failed"; fi
if ! "$THTCTL" --installation "$INSTALLATION" update --check-only; then
if ! bash scripts/build-tht.sh; then abort_update "the tht build failed"; fi
if ! "$THT_BIN" --installation "$INSTALLATION" update --check-only; then
abort_update "the installation render check failed"
fi
if [[ "$TRANSACTIONAL_PI_UPDATE" == true ]]; then
if ! "$THTCTL" --installation "$INSTALLATION" pi update \
if ! "$THT_BIN" --installation "$INSTALLATION" pi update \
--version "$NEXT_PI_VERSION" --source build --yes --drain; then
abort_update "the transactional core update failed"
fi
fi
if ! "$THTCTL" --installation "$INSTALLATION" start; then abort_update "installation start failed"; fi
if ! "$THT_BIN" --installation "$INSTALLATION" start; then abort_update "installation start failed"; fi
if ! curl --fail http://127.0.0.1:8080/health; then abort_update "frontend health check failed"; fi
if ! curl --fail http://127.0.0.1:8787/health; then abort_update "core health check failed"; fi
if ! FINAL_STATUS="$("$THTCTL" --installation "$INSTALLATION" status)"; then abort_update "final status failed"; fi
if ! FINAL_PI_STATUS="$("$THTCTL" --installation "$INSTALLATION" pi status)"; then abort_update "final pi status failed"; fi
if ! FINAL_STATUS="$("$THT_BIN" --installation "$INSTALLATION" status)"; then abort_update "final status failed"; fi
if ! FINAL_PI_STATUS="$("$THT_BIN" --installation "$INSTALLATION" pi status)"; then abort_update "final pi status failed"; fi
[[ "${FINAL_PI_STATUS#Pi version: }" == "$NEXT_PI_VERSION" ]] || abort_update "running Pi version does not match the pulled pin"
if ! "$THTCTL" --installation "$INSTALLATION" doctor; then abort_update "final doctor failed"; fi
if ! "$THT_BIN" --installation "$INSTALLATION" doctor; then abort_update "final doctor failed"; fi
require_clean_source
printf 'Built source revision: %s\n%s\n%s\n' "$SOURCE_REVISION" "$FINAL_STATUS" "$FINAL_PI_STATUS"
```
@@ -354,9 +354,9 @@ Assert-NativeSuccess 'source revision read'
$VersionLine = @(Select-String -Path docker/core.Dockerfile -Pattern '^ARG PI_VERSION=(.+)$')
if ($VersionLine.Count -ne 1) { throw 'Expected exactly one pinned default PI_VERSION.' }
$NextPiVersion = $VersionLine.Matches[0].Groups[1].Value
$InstallationStatus = @(& $THTCTL --installation $INSTALLATION status)
$InstallationStatus = @(& $THT_BIN --installation $INSTALLATION status)
Assert-NativeSuccess 'installation status'
$RunningPiStatus = (& $THTCTL --installation $INSTALLATION pi status)
$RunningPiStatus = (& $THT_BIN --installation $INSTALLATION pi status)
Assert-NativeSuccess 'Pi status'
$RunningPiVersion = $RunningPiStatus -replace '^Pi version:\s*', ''
if ([string]::IsNullOrWhiteSpace($RunningPiVersion)) { throw 'Pi status returned no version.' }
@@ -371,29 +371,29 @@ if ($NextPiVersion -eq $RunningPiVersion) {
}
powershell -ExecutionPolicy Bypass -File scripts/build-local.ps1
Assert-NativeSuccess 'local image build'
& "C:\Program Files\Git\bin\bash.exe" scripts/build-thothctl.sh
Assert-NativeSuccess 'thothctl build'
& $THTCTL --installation $INSTALLATION update --check-only
& "C:\Program Files\Git\bin\bash.exe" scripts/build-tht.sh
Assert-NativeSuccess 'tht build'
& $THT_BIN --installation $INSTALLATION update --check-only
Assert-NativeSuccess 'installation render check'
if ($TransactionalPiUpdate) {
& $THTCTL --installation $INSTALLATION pi update `
& $THT_BIN --installation $INSTALLATION pi update `
--version $NextPiVersion --source build --yes --drain
Assert-NativeSuccess 'transactional core update'
}
& $THTCTL --installation $INSTALLATION start
& $THT_BIN --installation $INSTALLATION start
Assert-NativeSuccess 'installation start'
curl.exe --fail --silent --show-error http://127.0.0.1:8080/health
Assert-NativeSuccess 'frontend health check'
curl.exe --fail --silent --show-error http://127.0.0.1:8787/health
Assert-NativeSuccess 'core health check'
$FinalStatus = @(& $THTCTL --installation $INSTALLATION status)
$FinalStatus = @(& $THT_BIN --installation $INSTALLATION status)
Assert-NativeSuccess 'final installation status'
$FinalPiStatus = (& $THTCTL --installation $INSTALLATION pi status)
$FinalPiStatus = (& $THT_BIN --installation $INSTALLATION pi status)
Assert-NativeSuccess 'final Pi status'
if (($FinalPiStatus -replace '^Pi version:\s*', '') -ne $NextPiVersion) {
throw 'Running Pi version does not match the pulled pin.'
}
& $THTCTL --installation $INSTALLATION doctor
& $THT_BIN --installation $INSTALLATION doctor
Assert-NativeSuccess 'final doctor'
Assert-CleanSource
Write-Output "Built source revision: $SourceRevision"
@@ -414,7 +414,7 @@ install a package in the running container.
Back up before source/Pi updates and test restoration periodically. First stop cleanly:
```sh
"$THTCTL" --installation "$INSTALLATION" stop
"$THT_BIN" --installation "$INSTALLATION" stop
docker volume ls --format '{{.Name}}' | grep '^thothii-'
```
@@ -477,7 +477,7 @@ before normal use. Never merge an archive into a non-empty volume.
## Data-preserving uninstall
Run `thothctl stop`, retain the installation descriptor at the same absolute path, and make one
Run `tht stop`, retain the installation descriptor at the same absolute path, and make one
verified backup set. In Docker Desktop, remove only this installation's stopped `core` and
`frontend` containers and optional local images; leave its four named volumes. On Linux, use the
containers' exact Compose project labels to remove only those stopped containers. Do not prune
@@ -485,7 +485,7 @@ global Docker data.
Do **not** run `docker compose down --volumes`: it deletes the application data this procedure is
meant to preserve. Keep the operator directory and protected secrets if you intend to reinstall.
Using the same descriptor path preserves the `thothctl` project identity and reconnects the same
Using the same descriptor path preserves the `tht` project identity and reconnects the same
named volumes after rebuilding the source checkout.
## Next: workspaces and Pi
+29 -29
View File
@@ -1,21 +1,21 @@
# Pi management
ThothII bundles Pi in the `core` image. Operators use the Pi Management page for safe application
defaults and the host-side `thothctl` CLI for lifecycle work. A local Pi installation is not
defaults and the host-side `tht` CLI for lifecycle work. A local Pi installation is not
required.
Run these commands from the root of the current ThothII checkout or worktree. `thothctl` discovers
Run these commands from the root of the current ThothII checkout or worktree. `tht` discovers
the valid installation descriptor in that project tree, so it uses the `deploy/` files belonging to
the checkout from which you run it. Do not use `~/bin`: `~` is the user home directory, not the
project root.
```sh
mkdir -p bin
go -C tools/thothctl build -o ../../bin/thothctl ./cmd/thothctl
THTCTL=./bin/thothctl
THT_BIN=tht
tht version --json
```
If `thothctl` is already on `PATH`, you may use `THTCTL=thothctl` instead. For an installation
If `tht` is not on `PATH`, install the native host CLI using the installation procedure in
`local.md` or `server.md`, then set `THT_BIN` to that installed binary. For an installation
stored elsewhere, set `THOTHII_INSTALLATION` or pass
`--installation <absolute-path>/thothii-installation.yaml` explicitly.
@@ -28,8 +28,8 @@ diagnostics; it never accepts or displays a credential, opens a terminal, or upd
Alternatively, use the CLI from an administrator terminal:
```sh
"$THTCTL" pi configure
"$THTCTL" pi configure --provider zai --model glm-5.2 --thinking medium
"$THT_BIN" pi configure
"$THT_BIN" pi configure --provider zai --model glm-5.2 --thinking medium
```
Use GUI Save defaults or CLI `pi configure`, not both for the same change. The CLI's interactive
@@ -39,11 +39,11 @@ methods store application defaults in backend installation settings, not in the
Useful read-only checks are:
```sh
"$THTCTL" pi status
"$THTCTL" pi doctor
"$THTCTL" pi test
"$THTCTL" pi check
"$THTCTL" pi logs
"$THT_BIN" pi status
"$THT_BIN" pi doctor
"$THT_BIN" pi test
"$THT_BIN" pi check
"$THT_BIN" pi logs
```
`pi check` is an alias for `pi test`; logs are a sanitized, bounded snapshot with no follow mode.
@@ -79,7 +79,7 @@ After changing the provider catalog, enabled-model policy, or selected credentia
running application with one confirmed restart:
```sh
"$THTCTL" pi restart --yes --drain
"$THT_BIN" pi restart --yes --drain
```
`--yes` confirms that core will be recreated. Without `--drain`, restart refuses active sessions;
@@ -91,8 +91,8 @@ image, and Compose is explicitly told never to build or pull. It will restart on
verifies health, Pi version, settings/model smoke, non-secret rendered configuration, and
persistence mounts before reopening admission.
Use `pi restart --yes` when there are already no active sessions. Do not substitute `thothctl stop`
and `thothctl start` or raw Compose commands for this reload workflow.
Use `pi restart --yes` when there are already no active sessions. Do not substitute `tht stop`
and `tht start` or raw Compose commands for this reload workflow.
## Update the bundled Pi version
@@ -101,20 +101,20 @@ uses the single `ARG PI_VERSION=...` pin in `docker/core.Dockerfile`, builds tha
active sessions to finish, and recreates only `core`:
```sh
"$THTCTL" pi update
"$THT_BIN" pi update
```
To build a specific version, pass `--version`; source, confirmation, and drain are automatic for
this normal build path:
```sh
"$THTCTL" pi update --version 0.81.0
"$THT_BIN" pi update --version 0.81.0
```
A registry update must use an immutable digest, never a mutable tag:
```sh
"$THTCTL" pi update \
"$THT_BIN" pi update \
--version 0.81.0 --source pull \
--image registry.example.invalid/thothii-core@sha256:<64-lowercase-hex-digits> \
--yes --drain
@@ -127,28 +127,28 @@ volumes.
## Recover a failed lifecycle operation
If a restart or update fails after core recreation, leave maintenance enabled and preserve the
reported recovery state and transaction override. Do not delete `.thothctl`, state files,
reported recovery state and transaction override. Do not delete `.tht`, state files,
containers, or volumes. Inspect status and sanitized logs:
```sh
"$THTCTL" pi maintenance status
"$THTCTL" pi status
"$THTCTL" pi logs
"$THT_BIN" pi maintenance status
"$THT_BIN" pi status
"$THT_BIN" pi logs
```
For a failed update, restore its prior image:
```sh
"$THTCTL" pi rollback --yes
"$THT_BIN" pi rollback --yes
```
For a failed restart, use maintenance recovery instead of rollback. After repairing the reported
Docker, disk, or configuration problem, use the same command to complete either safe recovery path:
```sh
"$THTCTL" pi maintenance recover --yes
"$THTCTL" pi doctor
"$THTCTL" pi test
"$THT_BIN" pi maintenance recover --yes
"$THT_BIN" pi doctor
"$THT_BIN" pi test
```
`pi rollback --yes` restores the prior update image. `pi maintenance recover --yes` checks both
@@ -158,5 +158,5 @@ installation gated and collect only the sanitized diagnostics.
## Direct support access
Raw Compose access is unsupported because it can bypass the installation-specific environment and
durable image selector. For support, use the installation-aware `thothctl pi status`,
`thothctl pi doctor`, `thothctl pi test`, and `thothctl pi logs` commands.
durable image selector. For support, use the installation-aware `tht pi status`,
`tht pi doctor`, `tht pi test`, and `tht pi logs` commands.
+11 -11
View File
@@ -8,7 +8,7 @@ manual identities, external L2, and the two parked restore-lock preconditions re
Task 15/release gates.
Guida operativa per collegare ThothII al DWH di PSD con il nuovo sistema (registry Git + descriptor
v3 + `thothctl`).
v3 + `tht`).
## Stato attuale (2026-08-13)
@@ -21,28 +21,28 @@ v3 + `thothctl`).
`thothii-installation.yaml` e i secret d'installazione in `secrets/` (pi-auth, secret bundle,
chiave SSH, known_hosts). L'API key DWH va completata nella gestione Workspace ed è conservata
nel vault cifrato del backend. Nessuna CA: il DWH REST usa HTTPS pubblico.
- **Stack avviato** (progetto `thothii-70417a3e30ea`, via `thothctl start`): `qdrant`, `embedding`
- **Stack avviato** (progetto `thothii-70417a3e30ea`, via `tht start`): `qdrant`, `embedding`
(con `qwen3-embedding:0.6b`), `core`, `frontend` sani. Il registry ha **clonato e attivato**
`psd-clinical` (stato `ready`).
- **`thothctl workspace inspect --workspace psd-clinical` = OK** (identità descrittore/catalogo
- **`tht workspace inspect --workspace psd-clinical` = OK** (identità descrittore/catalogo
risolte); la configurazione runtime va completata e testata dalla GUI.
- **Bloccante residuo: VPN.** `supabase-aritmolab.policlinicosandonato.it` non risolve
(`NXDOMAIN`) → il preprocessing DWH e le sessioni live non possono ancora partire.
## Avvio/arresto (canonico)
Usare `thothctl` (stesso project name, quindi stessi volumi named):
Usare `tht` (stesso project name, quindi stessi volumi named):
```bash
THOTHCTL=dist/thothctl/thothctl-darwin-arm64
"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" start
"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" workspace inspect --workspace psd-clinical --json
"$THOTHCTL" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" stop
tht=dist/tht/tht-darwin-arm64
"$tht" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" start
"$tht" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" workspace inspect --workspace psd-clinical --json
"$tht" --installation "$(pwd)/deploy/psd/thothii-installation.yaml" stop
```
> **Nota project name:** `thothctl` calcola un project name stabile dall'installation descriptor
> **Nota project name:** `tht` calcola un project name stabile dall'installation descriptor
> (`thothii-<hash>`); `docker compose` "a mano" usa invece `name: thothii` dal `compose.yaml`, quindi
> i volumi named non coinciderebbero. Perciò per lo stack si usa `thothctl start` (non
> i volumi named non coinciderebbero. Perciò per lo stack si usa `tht start` (non
> `compose-with-preflight.sh up`).
## Rimane: smoke live di una domanda (P8 L2)
@@ -57,5 +57,5 @@ Il preprocessing è già completato. Resta solo:
- Ristrutturazione del repo PSD nel layout P1.1 + validazione locale.
- Pubblicazione GitHub + deploy key read-only + configurazione Git d'installazione.
- Avvio stack + attivazione registry + `thothctl inspect` verde.
- Avvio stack + attivazione registry + `tht inspect` verde.
- **Preprocessing live completato** su PSD: DWH → FK → schema → Evidence, idempotente.
+3 -3
View File
@@ -67,10 +67,10 @@ management and are never exposed by the API.
Use the installation-aware controller described by `server.md`:
```bash
THTCTL=/srv/thothii/operator/thothctl
THT_BIN=/srv/thothii/operator/tht
INSTALLATION=/srv/thothii/operator/thothii-installation.yaml
"$THTCTL" --installation "$INSTALLATION" start
"$THTCTL" --installation "$INSTALLATION" doctor
"$THT_BIN" --installation "$INSTALLATION" start
"$THT_BIN" --installation "$INSTALLATION" doctor
```
The descriptor composes `compose.yaml`, `deploy/compose.server.yaml`, the server session-storage
+55 -54
View File
@@ -3,9 +3,10 @@
Server authentication uses generic OIDC with the reverse proxy preserving the configured public
origin and callback path. Follow the [OIDC guide](authentication-oidc.md), [Authentik guide](authentik.md)
when applicable, and the [authentication acceptance matrix](../testing/authentication-manual-acceptance.md).
The host authentication CLI is `tht`: Workspace Validate is static, `tht auth check` is live and
non-interactive, `--interactive` adds device-flow identity validation, and Workspace Test is the
aggregate live gate.
The host authentication CLI is `tht`. Use `tht --installation <descriptor> workspace inspect
--workspace <id> --json` for the active workspace snapshot, `tht --installation <descriptor>
auth check` for live non-interactive authentication diagnosis, `auth check --interactive` for
device-flow identity validation, and `tht ... doctor --json` for the aggregate installation gate.
This guide is for an installer with basic Linux administration and very basic Docker knowledge.
It deploys the same Compose distribution used on a local PC: the mandatory application is exactly
@@ -30,7 +31,7 @@ storage overlay and migration procedure before exposing a production installatio
value belongs in Git, images, browser storage, environment values, rendered Compose, or logs.
- The Git-backed workspace registry is the source of truth. Installation-local bindings identify
endpoints and secret-file paths; they do not replace the reviewed Git workspace descriptors.
- `thothctl` is the operator CLI for start, stop, status, health, logs, Pi lifecycle, drain, and
- `tht` is the operator CLI for start, stop, status, health, logs, Pi lifecycle, drain, and
rollback. Raw Compose lifecycle commands bypass installation state and are unsupported.
Read [server workspace-registry installation](server-workspace-registry.md),
@@ -51,7 +52,7 @@ sudo useradd --system --uid 10001 --user-group --home-dir /srv/thothii \
```
Use a dedicated `thothii-ops` group for the small set of human operators. A human who runs
`thothctl` must be in both `thothii-ops` (to traverse operator paths and read declared secret files
`tht` must be in both `thothii-ops` (to traverse operator paths and read declared secret files
for output redaction) and the host `docker` group (to invoke Docker). Docker-group membership is
effectively host-root access; grant both memberships only to reviewed administrators. The
non-login `thothii` account owns files and writable data but does not need Docker access.
@@ -61,7 +62,7 @@ sudo groupadd --system thothii-ops
sudo usermod --append --groups thothii-ops,docker "$USER"
```
Log out and back in before continuing; `id` must show both groups. Do not run `thothctl` through
Log out and back in before continuing; `id` must show both groups. Do not run `tht` through
`sudo -u thothii`: that account deliberately lacks Docker access. Do not grant the human direct
write access to runtime bind trees.
@@ -130,7 +131,7 @@ bridge gateway address or to a dedicated private host interface—never to `0.0.
the check pass. A stable internal DNS record routed through an authenticated private listener is
the preferred alternative.
After the first bounded start attempt, copy the exact core container name from `thothctl status`
After the first bounded start attempt, copy the exact core container name from `tht status`
into `CORE_NAME`, then derive—not guess—the network ID, Linux bridge interface, gateway, and
subnet. Compose networks normally use `br-<first-12-network-id>`; an explicit
`com.docker.network.bridge.name` option takes precedence:
@@ -163,7 +164,7 @@ sudo iptables -I INPUT 1 -i "$BRIDGE" -s "$SUBNET" -d "$GATEWAY" -p tcp --dport
sudo iptables -I INPUT 2 -d "$GATEWAY" -p tcp --dport "$EXTERNAL_PORT" -j REJECT
```
Confirm reachability with `thothctl pi test` for the configured LLM/Pi path and with the
Confirm reachability with `tht pi test` for the configured LLM/Pi path and with the
authenticated Workspace Diagnostics action for DWH, vector collection/embedding pairing, and
embedding endpoints. A timeout paired with `ss -lntp`, `ip address show dev "$BRIDGE"`, and the
firewall counters distinguishes a loopback bind from a subnet/interface rule failure. Do not add
@@ -200,7 +201,7 @@ the writable Pi-state root and then overlays protected `auth.json` plus tracked
`settings.json` read-only below it. Docker requires those three hidden target files to exist under
the host parent bind before startup. The initializer creates them atomically with UID/GID 10001,
mode `0600`, rejects symlink roots or targets, and never overwrites existing contents. It is safe to rerun
after restoring `pi-state`; run it before any `thothctl start`, Compose render/start, or Pi update.
after restoring `pi-state`; run it before any `tht start`, Compose render/start, or Pi update.
Copy the path-only server environment and installation descriptor:
@@ -224,7 +225,7 @@ image overrides go after them.
Create each installation credential (Pi/application, Git, and session storage) as an independent
regular file in `/srv/thothii/secrets`, owned by
UID 10001, group `thothii-ops`, and mode `0640`. Owner access lets the UID 10001 container read a
file mounted under `/run/secrets`; group access lets the reviewed human run `thothctl`. The
file mounted under `/run/secrets`; group access lets the reviewed human run `tht`. The
operator environment records only absolute `*_FILE` or `*_SOURCE` paths for those installation
credentials. DWH and Evidence values are entered later through Workspace management and persist
as ciphertext under `/data/workspace-secrets`; the frontend receives no secret values. Do not
@@ -253,7 +254,7 @@ sudo -u thothii cp /srv/thothii/operator/server.env deploy/env/local.env
bash scripts/build-local.sh
```
The printed local-profile start command is not the server start command; use `thothctl` below.
The printed local-profile start command is not the server start command; use `tht` below.
Alternatively, create a reviewed untracked override with release images pinned by immutable
digest. Mutable tags are not a production pin:
@@ -274,39 +275,39 @@ services:
Add that absolute file last in `overrides`. `core` and `session-migrate` must use the exact same
core digest; neither may retain a local build or `:local` image. Frontend uses its own exact digest.
Both images must come from one compatible release; the core image must retain the declared Pi
version labels checked by `thothctl pi doctor`. Pull access
version labels checked by `tht pi doctor`. Pull access
belongs in the host Docker credential store, not in Compose or the installation descriptor.
## Install thothctl
## Install tht
Build the operator binaries with Docker. No Go installation or Go knowledge is required:
```sh
cd /srv/thothii/source/ThothII
THT_THOTHCTL_OUTPUT_DIRECTORY=/srv/thothii/operator/build-output \
bash scripts/build-thothctl.sh
THT_THT_OUTPUT_DIRECTORY=/srv/thothii/operator/build-output \
bash scripts/build-tht.sh
sudo install -o root -g thothii-ops -m 0750 \
/srv/thothii/operator/build-output/thothctl-linux-amd64 \
/srv/thothii/operator/thothctl
/srv/thothii/operator/build-output/tht-linux-amd64 \
/srv/thothii/operator/tht
```
The source checkout stays read-only to the human. The explicit output directory is the only build
write boundary; the build script rejects relative or non-canonical output paths. After installation,
remove or retain `build-output` according to the site's reviewed artifact policy.
Use `thothctl-linux-arm64` on an ARM64 server. Set these variables in the maintenance shell; do
Use `tht-linux-arm64` on an ARM64 server. Set these variables in the maintenance shell; do
not source `server.env` as shell code:
```sh
THTCTL=/srv/thothii/operator/thothctl
THT_BIN=/srv/thothii/operator/tht
INSTALLATION=/srv/thothii/operator/thothii-installation.yaml
"$THTCTL" --help
"$THTCTL" --installation "$INSTALLATION" update --check-only
"$THT_BIN" --help
"$THT_BIN" --installation "$INSTALLATION" update --check-only
```
Every operator command includes the descriptor explicitly. This preserves the installation's
profile, overrides, project identity, and durable current-image selector. The general form is
`thothctl --installation /absolute/path/thothii-installation.yaml <command>`.
`tht --installation /absolute/path/thothii-installation.yaml <command>`.
## Start and verify readiness
@@ -317,8 +318,8 @@ selected core image after all installation overrides, so this procedure is ident
and pinned modes. It exits nonzero unless both arrays are empty:
```sh
"$THTCTL" --installation "$INSTALLATION" stop
"$THTCTL" --installation "$INSTALLATION" sessions migrate --yes
"$THT_BIN" --installation "$INSTALLATION" stop
"$THT_BIN" --installation "$INSTALLATION" sessions migrate --yes
```
Successful output has this shape (the `applied` list may contain versions on first use):
@@ -330,12 +331,12 @@ Successful output has this shape (the `applied` list may contain versions on fir
Only after seeing `"pending":[]` and `"drifted":[]`, start and verify:
```sh
"$THTCTL" --installation "$INSTALLATION" start
"$THTCTL" --installation "$INSTALLATION" status
"$THTCTL" --installation "$INSTALLATION" doctor
"$THT_BIN" --installation "$INSTALLATION" start
"$THT_BIN" --installation "$INSTALLATION" status
"$THT_BIN" --installation "$INSTALLATION" doctor
curl --fail http://127.0.0.1:8080/health
"$THTCTL" --installation "$INSTALLATION" pi doctor
"$THTCTL" --installation "$INSTALLATION" pi test
"$THT_BIN" --installation "$INSTALLATION" pi doctor
"$THT_BIN" --installation "$INSTALLATION" pi test
```
`/health` proves process liveness. Readiness additionally requires both healthy services, a valid
@@ -365,10 +366,10 @@ trusted proxy, and never expose `core`.
Configure only closed provider/model/reasoning choices. Credentials remain protected files:
```sh
"$THTCTL" --installation "$INSTALLATION" pi status
"$THTCTL" --installation "$INSTALLATION" pi configure
"$THTCTL" --installation "$INSTALLATION" pi doctor
"$THTCTL" --installation "$INSTALLATION" pi logs
"$THT_BIN" --installation "$INSTALLATION" pi status
"$THT_BIN" --installation "$INSTALLATION" pi configure
"$THT_BIN" --installation "$INSTALLATION" pi doctor
"$THT_BIN" --installation "$INSTALLATION" pi logs
```
Before an update, announce maintenance and ask users to finish active work. `--drain` closes new
@@ -376,9 +377,9 @@ admission and waits until no active sessions remain; it does not discard session
and registry-source examples are:
```sh
"$THTCTL" --installation "$INSTALLATION" pi update \
"$THT_BIN" --installation "$INSTALLATION" pi update \
--version 0.81.0 --source build --yes --drain
"$THTCTL" --installation "$INSTALLATION" pi update \
"$THT_BIN" --installation "$INSTALLATION" pi update \
--version 0.81.0 --source pull \
--image registry.example.com/thothii/core@sha256:<64-lowercase-hex-digits> \
--yes --drain
@@ -388,23 +389,23 @@ The transaction recreates only `core`, preserves volumes, verifies health/config
automatically attempts rollback after a post-mutation failure. For interrupted or ambiguous state:
```sh
"$THTCTL" --installation "$INSTALLATION" pi maintenance status
"$THTCTL" --installation "$INSTALLATION" pi rollback --yes
"$THTCTL" --installation "$INSTALLATION" pi maintenance recover --yes
"$THT_BIN" --installation "$INSTALLATION" pi maintenance status
"$THT_BIN" --installation "$INSTALLATION" pi rollback --yes
"$THT_BIN" --installation "$INSTALLATION" pi maintenance recover --yes
```
Leave maintenance active if rollback cannot be verified. Preserve `.thothctl/<installation-id>/`
Leave maintenance active if rollback cannot be verified. Preserve `.tht/<installation-id>/`
recovery state, repair the reported host/configuration issue, and rerun rollback or maintenance
recovery. Never delete or edit `current-image.yaml` or `update-state.json` to force progress.
## Back up and restore
Back up before source, workspace, session-schema, or Pi changes. Drain work, stop the installation,
record `git rev-parse HEAD`, image digests, and `thothctl status`, then archive the three bind trees
record `git rev-parse HEAD`, image digests, and `tht status`, then archive the three bind trees
with numeric ownership. Do not include live secrets in this ordinary archive.
```sh
"$THTCTL" --installation "$INSTALLATION" stop
"$THT_BIN" --installation "$INSTALLATION" stop
BACKUP=/srv/thothii-backups/2026-08-05
sudo install -d -o root -g root -m 0700 "$BACKUP"
sudo tar --numeric-owner --xattrs --acls -C /srv/thothii -czf "$BACKUP/runtime-data.tgz" \
@@ -447,14 +448,14 @@ merge an archive into a non-empty tree.
Begin with bounded, sanitized installation-aware commands:
```sh
"$THTCTL" --installation "$INSTALLATION" status
"$THTCTL" --installation "$INSTALLATION" doctor
"$THTCTL" --installation "$INSTALLATION" logs
"$THTCTL" --installation "$INSTALLATION" pi status
"$THTCTL" --installation "$INSTALLATION" pi doctor
"$THTCTL" --installation "$INSTALLATION" pi test
"$THTCTL" --installation "$INSTALLATION" pi logs
"$THTCTL" --installation "$INSTALLATION" pi maintenance status
"$THT_BIN" --installation "$INSTALLATION" status
"$THT_BIN" --installation "$INSTALLATION" doctor
"$THT_BIN" --installation "$INSTALLATION" logs
"$THT_BIN" --installation "$INSTALLATION" pi status
"$THT_BIN" --installation "$INSTALLATION" pi doctor
"$THT_BIN" --installation "$INSTALLATION" pi test
"$THT_BIN" --installation "$INSTALLATION" pi logs
"$THT_BIN" --installation "$INSTALLATION" pi maintenance status
```
Use the authenticated Workspace Management status and diagnostic actions for Git revision,
@@ -468,7 +469,7 @@ external service identity; and the proxy/identity provider for login failures.
## Data-preserving uninstall
Drain and stop through `thothctl`, take and verify one final backup, and disable the TLS proxy
Drain and stop through `tht`, take and verify one final backup, and disable the TLS proxy
route. Set `THT_BACKUP_ROOT=/srv/thothii-backups` in `server.env`; the removal command verifies the
filesystem identity of that backup root, all three bind trees, and every declared secret before
and after removing anything.
@@ -477,14 +478,14 @@ First run without confirmation. It displays the exact installation project, serv
name, container ID, and stopped state, then exits without mutation. Check every target:
```sh
"$THTCTL" --installation "$INSTALLATION" stop
"$THTCTL" --installation "$INSTALLATION" remove
"$THT_BIN" --installation "$INSTALLATION" stop
"$THT_BIN" --installation "$INSTALLATION" remove
```
If and only if both targets are the expected stopped `frontend` and `core` containers, confirm:
```sh
"$THTCTL" --installation "$INSTALLATION" remove --yes exact-core-id exact-frontend-id
"$THT_BIN" --installation "$INSTALLATION" remove --yes exact-core-id exact-frontend-id
```
Replace both example IDs with the values from the immediately preceding dry-run. The command
@@ -496,5 +497,5 @@ paths still identify the same filesystem objects. Keep `/srv/thothii/data`, `pi-
descriptor if reinstallation is possible. Do not prune global Docker data.
Do **not** run `docker compose down --volumes`; it deletes persistent application data. Reusing the
same protected descriptor path preserves the `thothctl` installation identity and allows a later
same protected descriptor path preserves the `tht` installation identity and allows a later
compatible source checkout to reconnect the retained state.
+1 -1
View File
@@ -21,7 +21,7 @@ bash scripts/verify-line-endings.sh
```
Keep Docker Desktop's integration enabled for that WSL distribution. Run the Linux build scripts
and the Linux `thothctl` binary from the same WSL shell.
and the Linux `tht` binary from the same WSL shell.
## Repository-local LF policy
@@ -1,5 +1,9 @@
# Internal Qdrant and Ollama Implementation Plan
> **Historical nomenclature:** this plan predates the native host CLI convergence. The current
> operator command is `tht`; any older `thothctl` smoke-script or rollback wording below is retained
> only as historical evidence.
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Make Qdrant and Ollama mandatory internal ThothII services while keeping the analytical
@@ -1,5 +1,9 @@
# Read-only Workspace Runtime Secrets Implementation Plan
> **Historical nomenclature:** this plan predates the native host CLI convergence. References to
> `thothctl` and `tools/thothctl` describe the implementation snapshot from which this plan was
> written; current operator commands and paths use native `tht` and `tools/tht`.
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Make workspace consumption strictly read-only while adding installation-scoped Git identity and persistent GUI-managed runtime secrets.
@@ -4,9 +4,9 @@
**Goal:** Validate local and OIDC authentication on macOS, deploy the exact feat/thoth-auth candidate to the Aritmolab/PSD server before merging it into main, and complete end-to-end acceptance with remote Authentik.
**Architecture:** Test the candidate first as a standalone local installation. Then install the same immutable Git revision on the existing PSD installation with the installation-aware thothctl lifecycle, leaving main untouched. Authentik provides OIDC login and a mandatory direct groups claim; ThothII maps exact external groups to roles and validates mapped groups through the Authentik catalog API.
**Architecture:** Test the candidate first as a standalone local installation. Then install the same immutable Git revision on the existing PSD installation with the installation-aware tht lifecycle, leaving main untouched. Authentik provides OIDC login and a mandatory direct groups claim; ThothII maps exact external groups to roles and validates mapped groups through the Authentik catalog API.
**Tech Stack:** macOS, Docker Desktop, Docker Compose, tht/thothctl, local Argon2id authentication, generic OIDC Authorization Code + PKCE, Authentik, PSD workspace registry, reverse proxy/TLS.
**Tech Stack:** macOS, Docker Desktop, Docker Compose, native host `tht` plus Python workflow `tht`, local Argon2id authentication, generic OIDC Authorization Code + PKCE, Authentik, PSD workspace registry, reverse proxy/TLS.
---
@@ -26,7 +26,7 @@ git status --short --untracked-files=all
Never deploy a moving branch name without checking its resolved SHA. Never put passwords, OIDC client secrets, Authentik API tokens, cookies, authorization headers, raw ID tokens, or password hashes in Git, shell history, screenshots, logs, or evidence.
Use protected operator values for <PUBLIC_URL>, <OIDC_ISSUER>, <AUTHENTIK_BASE_URL>, <OIDC_CLIENT_ID>, <THTCTL>, <INSTALLATION>, <WORKSPACE_ID>, and <OLD_SHA>.
Use protected operator values for <PUBLIC_URL>, <OIDC_ISSUER>, <AUTHENTIK_BASE_URL>, <OIDC_CLIENT_ID>, <THT_BIN>, <INSTALLATION>, <WORKSPACE_ID>, and <OLD_SHA>.
The Authentik contract is mandatory: a direct non-empty JSON array claim named groups; exact groups TOT Users and TOT Admin; mappings TOT Users -> user and TOT Admin -> admin; and a separate group-view-only API service account exposed only as THT_AUTHENTIK_API_TOKEN. Extra upstream groups are valid and silently ignored.
@@ -71,7 +71,7 @@ Expected: Docker Desktop and Compose are available and line-ending validation pa
```bash
bash scripts/build-local.sh
bash scripts/build-thothctl.sh
bash scripts/build-tht.sh
tht setup --profile local
```
@@ -176,12 +176,12 @@ Go/no-go: do not proceed to PSD if local login, role separation, logout, or reme
**Step 1: Capture live state**
```bash
THTCTL=<THTCTL>
THT_BIN=<THT_BIN>
INSTALLATION=<INSTALLATION>
"$THTCTL" --installation "$INSTALLATION" status
"$THTCTL" --installation "$INSTALLATION" doctor
"$THTCTL" --installation "$INSTALLATION" pi status
"$THTCTL" --installation "$INSTALLATION" pi doctor
"$THT_BIN" --installation "$INSTALLATION" status
"$THT_BIN" --installation "$INSTALLATION" doctor
"$THT_BIN" --installation "$INSTALLATION" pi status
"$THT_BIN" --installation "$INSTALLATION" pi doctor
git -C /srv/thothii/source/ThothII status --short --untracked-files=all
git -C /srv/thothii/source/ThothII rev-parse HEAD
```
@@ -190,14 +190,14 @@ Save the live SHA as <OLD_SHA> and capture image identities, workspace registry
**Step 2: Drain and back up**
Announce maintenance, close the reverse proxy or show its maintenance page, drain active work, and stop through thothctl. Create the protected, checksummed backup specified in docs/install/server.md, including runtime trees and PSD PostgreSQL/session data where applicable. Back up credentials separately. Never run docker compose down --volumes.
Announce maintenance, close the reverse proxy or show its maintenance page, drain active work, and stop through tht. Create the protected, checksummed backup specified in docs/install/server.md, including runtime trees and PSD PostgreSQL/session data where applicable. Back up credentials separately. Never run docker compose down --volumes.
**Step 3: Check preconditions**
```bash
git -C /srv/thothii/source/ThothII config --local core.autocrlf false
bash /srv/thothii/source/ThothII/scripts/verify-line-endings.sh
"$THTCTL" --installation "$INSTALLATION" update --check-only
"$THT_BIN" --installation "$INSTALLATION" update --check-only
```
Expected: descriptor, protected secrets, Pi-state mount, workspace repository binding, and Compose render remain valid before source changes.
@@ -207,7 +207,7 @@ Expected: descriptor, protected secrets, Pi-state mount, workspace repository bi
**Files:**
- Server source checkout: /srv/thothii/source/ThothII
- Server operator binary: protected THTCTL path
- Server operator binary: protected THT_BIN path
- Server installation descriptor and secret files: unchanged paths unless a reviewed auth update is required
**Step 1: Select the exact candidate**
@@ -226,21 +226,21 @@ Do not merge or rebase main. The running installation is intentionally based on
```bash
cd /srv/thothii/source/ThothII
bash scripts/build-local.sh
THT_THOTHCTL_OUTPUT_DIRECTORY=/srv/thothii/operator/build-output bash scripts/build-thothctl.sh
THT_THT_OUTPUT_DIRECTORY=/srv/thothii/operator/build-output bash scripts/build-tht.sh
```
Install the architecture-appropriate candidate thothctl only after its build succeeds. Keep the old operator binary recoverable.
Install the architecture-appropriate candidate tht only after its build succeeds. Keep the old operator binary recoverable.
**Step 3: Start and verify the candidate**
```bash
"$THTCTL" --installation "$INSTALLATION" update --check-only
"$THTCTL" --installation "$INSTALLATION" start --build
"$THTCTL" --installation "$INSTALLATION" status
"$THTCTL" --installation "$INSTALLATION" doctor --json
"$THT_BIN" --installation "$INSTALLATION" update --check-only
"$THT_BIN" --installation "$INSTALLATION" start --build
"$THT_BIN" --installation "$INSTALLATION" status
"$THT_BIN" --installation "$INSTALLATION" doctor --json
curl --fail http://127.0.0.1:8080/health
"$THTCTL" --installation "$INSTALLATION" pi doctor
"$THTCTL" --installation "$INSTALLATION" pi test
"$THT_BIN" --installation "$INSTALLATION" pi doctor
"$THT_BIN" --installation "$INSTALLATION" pi test
```
Expected: candidate frontend/core and internal services are healthy, no data volume was replaced, and the candidate SHA is recorded. Liveness alone is not release approval.
@@ -294,7 +294,7 @@ A missing mapped group must fail with redacted oidc_mapped_group_missing. Unmapp
If configuration requires process reload:
```bash
"$THTCTL" --installation "$INSTALLATION" pi restart --yes --drain
"$THT_BIN" --installation "$INSTALLATION" pi restart --yes --drain
```
Repeat authentication, doctor, and health checks. Do not substitute raw Compose commands.
@@ -329,13 +329,19 @@ During a controlled window, make discovery/JWKS unavailable or rename a mapped g
- Read: docs/install/server-workspace-registry.md
- Use: authenticated PSD browser sessions
**Step 1: Validate as administrator**
**Step 1: Validate the workspace and authentication from the host CLI**
In Workspace Management, select <WORKSPACE_ID> and run Validate workspace source. Expected: Git descriptor, Evidence/catalog invariants, and complete authentication readiness pass; result is redacted and identifies the workspace revision.
```bash
"$THT_BIN" --installation "$INSTALLATION" \
workspace inspect --workspace "$WORKSPACE_ID" --json
"$THT_BIN" --installation "$INSTALLATION" auth check --json
```
**Step 2: Test connections**
Expected: the workspace registry is ready, authentication readiness passes, and output is redacted while identifying the active workspace revision.
Run Test workspace connections for the selected workspace. Expected: protected DWH/Evidence secrets are used, status is redacted, and workspace source is unchanged.
**Step 2: Verify the application boundary**
Open the configured public URL, authenticate with the approved identity, and verify that the application reaches the selected workspace without unexpected `401`/`403` responses. Keep DWH/Evidence connection tests read-only and use only the existing approved smoke question.
**Step 3: Test ordinary-user authorization**
@@ -356,7 +362,7 @@ Do not run mutating production queries. Preserve only a session ID and redacted
**Files:**
- Create: docs/testing/evidence/2026-08-18-thothii-authentication-psd-acceptance.md or the approved external evidence location
- Read: docs/install/server.md and docs/contracts/thothctl-pi.md
- Read: docs/install/server.md and docs/contracts/tht-pi.md
**Step 1: Mandatory gates**
@@ -370,7 +376,7 @@ Do not run mutating production queries. Preserve only a session ID and redacted
| Candidate deploy | Server candidate status/doctor/health pass |
| Authentik | Direct groups claim, issuer/JWKS, secrets, catalog, and mapped groups pass |
| OIDC authorization | ordinary, admin, unmapped, malformed, logout, and outage cases pass |
| Workspace integration | Validate workspace source and test connections pass |
| Workspace integration | `tht workspace inspect` and `tht auth check` pass; the authenticated application reaches the selected workspace |
| PSD smoke | One harmless known-good session completes |
| Hygiene | No secrets, tokens, cookies, hashes, or raw claims in evidence |
@@ -380,9 +386,9 @@ Include candidate SHA, old SHA, timestamps, commands, browser cases, redacted di
**Step 3: Roll back a failed candidate**
1. Keep the proxy closed and preserve .thothctl/<installation-id>/ recovery state.
2. Do not use thothctl pi rollback as the whole-application rollback; it addresses only Pi lifecycle images.
3. Stop with thothctl.
1. Keep the proxy closed and preserve .tht/<installation-id>/ recovery state.
2. Do not use tht pi rollback as the whole-application rollback; it addresses only Pi lifecycle images.
3. Stop with tht.
4. Return the source checkout to <OLD_SHA>, rebuild old application/operator artifacts, and start through the same descriptor.
5. Run update --check-only, status, doctor, health, Pi smoke, workspace diagnostics, and one harmless session.
6. For ambiguous recovery, leave maintenance active and follow pi maintenance status / pi maintenance recover --yes. Never delete volumes, selectors, or recovery files to force progress.
@@ -400,4 +406,3 @@ Merge only after PSD owner acceptance, exact-SHA evidence, no unresolved auth/wo
## Handoff checklist
Deliver the redacted report, local result/SHA, PSD candidate SHA/images, Authentik provider and group mapping confirmation, workspace validation/connection results, backup/rollback status, and an explicit READY TO MERGE or NOT READY TO MERGE decision.
@@ -0,0 +1,92 @@
# Tht Documentation Convergence Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Align current documentation and documentation smoke checks with the converged native host CLI `tht`, while preserving historical references only where they describe past decisions or evidence.
**Architecture:** Treat `tools/tht/cmd/tht/main.go` as the canonical host CLI surface for installation, authentication, diagnostics, lifecycle, and workspace operations. Keep the Python `harness/.venv/bin/tht` distinction explicit for the workflow runtime, and update current operator/test instructions to invoke the native `tht` with `--installation`.
**Tech Stack:** Markdown documentation, shell smoke tests, Go CLI command surface, repository search-based verification.
---
### Task 1: Classify current and historical legacy CLI references
**Files:**
- Inspect: `README.md`, `PROJECT_STATE.md`, `AGENTS.md`, `docs/**`, `scripts/**`
- Reference: `tools/tht/cmd/tht/main.go`
**Step 1:** Build a complete occurrence inventory with a case-insensitive search for the former host CLI name and classify every match.
**Step 2:** Classify each occurrence as current operator documentation, documentation smoke expectation, executable/script contract, or historical design/evidence.
**Step 3:** Record the classification in the implementation notes before editing.
### Task 2: Update canonical operator and installation documentation
**Files:**
- Modify: `README.md`
- Modify: `AGENTS.md`
- Modify: `PROJECT_STATE.md`
- Modify: `docs/guida-utente.md`
- Modify: `docs/contracts/workspace-preprocessing-cli.md`
- Rename/update: `docs/contracts/tht-pi.md` as the current `tht` Pi contract
- Modify: relevant installation and architecture pages that expose operator commands
**Step 1:** Replace current host/operator invocations with `tht --installation ...`.
**Step 2:** Document the distinction between the native host CLI `tht` and the Python harness CLI invoked by the backend/runtime.
**Step 3:** Update command examples for `start`, `status`, `doctor`, `auth`, `workspace`, and `pi`.
**Step 4:** Add a short historical note only where a document must explain the former name.
### Task 3: Rewrite authentication acceptance and manual test instructions
**Files:**
- Modify: `docs/testing/authentication-manual-acceptance.md`
- Modify: `docs/plans/2026-08-18-thothii-authentication-acceptance-and-psd-deployment.md`
- Modify: `docs/install/authentication-local.md`
- Modify: `docs/install/authentication-oidc.md`
- Modify: `docs/install/authentik.md`
**Step 1:** Make `tht auth status`, `tht auth check`, `tht auth check --interactive`, and `tht doctor --json` the canonical terminal preflight.
**Step 2:** Use `tht status`, `tht start`, and `tht workspace inspect --workspace psd-clinical --json` for PSD deployment checks.
**Step 3:** Clarify that the P8 L2 gate is authentication-to-application integration through the first reviewer gate.
**Step 4:** Retain the prior functional test suite as a baseline and add only the authentication boundary smoke required for this acceptance.
### Task 4: Align documentation smoke tests
**Files:**
- Modify: `scripts/auth-docs-smoke.sh`
- Modify: `scripts/test-auth-docs-smoke.sh`
- Inspect/update: any current smoke script whose user-facing command examples still require the legacy CLI name
**Step 1:** Replace forbidden/current command assertions with `tht` equivalents.
**Step 2:** Preserve negative checks for obsolete authentication CLI wording.
**Step 3:** Run the positive and negative documentation fixtures.
### Task 5: Preserve or annotate historical material
**Files:**
- Inspect the historical discovery specification for context, without treating it as current operator documentation.
- Inspect: dated reports and archived acceptance scripts
**Step 1:** Do not rewrite historical titles, commit evidence, or old implementation names solely to erase history.
**Step 2:** Add a concise “historical nomenclature” note where an archived document could otherwise be mistaken for current instructions.
### Task 6: Verify the convergence
**Step 1:** Run `scripts/auth-docs-smoke.sh` and `scripts/test-auth-docs-smoke.sh`.
**Step 2:** Search active documentation for remaining legacy CLI references.
**Step 3:** Confirm every remaining match is either an explicit historical note, an ignored runtime directory name, or a non-document executable compatibility artifact.
**Step 4:** Run `git diff --check` and report the exact files changed plus any intentionally retained historical references.
@@ -371,7 +371,7 @@ git commit -m "feat: add cross-platform thothctl"
- Test: `tools/thothctl/internal/pi/commands_test.go`
- Test: `tools/thothctl/internal/pi/update_test.go`
- Modify: `tools/thothctl/cmd/thothctl/main.go`
- Create: `docs/contracts/thothctl-pi.md`
- Create: `docs/contracts/tht-pi.md`
**Interfaces:**
- Produces: `pi status|doctor|configure|test|update|logs`.
@@ -405,7 +405,7 @@ Record current image ID, pull/build requested pinned version, recreate only `cor
```sh
docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./internal/pi -v
docker run --rm -v "$PWD:/src" -w /src/tools/thothctl golang:1.24 go test ./...
git add tools/thothctl docs/contracts/thothctl-pi.md
git add tools/thothctl docs/contracts/tht-pi.md
git commit -m "feat: manage embedded pi with thothctl"
```
@@ -31,7 +31,7 @@
- Modify the `Target.Source` comment in `tools/thothctl/internal/pi/state.go` when `"restart"` becomes a valid non-image lifecycle source.
- Modify `tools/thothctl/cmd/thothctl/main.go` and its test for usage, parsing, dispatch, and recovery.
- Modify `frontend/src/shell/PiManagement.tsx` and its test for the structured workflow.
- Modify `docs/contracts/thothctl-pi.md`, `docs/install/pi-management.md`, `docs/general/pi-configuration.md`, and `scripts/verify-workspace-install-docs.sh`.
- Modify `docs/contracts/tht-pi.md`, `docs/install/pi-management.md`, `docs/general/pi-configuration.md`, and `scripts/verify-workspace-install-docs.sh`.
- Modify `README.md` only if it claims to list the complete Pi command surface.
---
@@ -440,7 +440,7 @@ git commit -m "feat(thothctl): expose Pi restart command"
### Task 4: Update contracts, operator docs, and documentation gates
**Files:**
- Modify: `docs/contracts/thothctl-pi.md`
- Modify: `docs/contracts/tht-pi.md`
- Modify: `docs/install/pi-management.md`
- Modify: `docs/general/pi-configuration.md`
- Modify: `scripts/verify-workspace-install-docs.sh:1160-1188`
@@ -484,7 +484,7 @@ Expected: FAIL because restart and separated workflows are not documented.
- [ ] **Step 3: Rewrite the two authoritative operator documents**
In `docs/contracts/thothctl-pi.md`, document `pi restart --yes [--drain]`, confirmation, maintenance, bounded drain, current-image retention, core-only recreation, verification, separate restart state, shared lock, and recovery.
In `docs/contracts/tht-pi.md`, document `pi restart --yes [--drain]`, confirmation, maintenance, bounded drain, current-image retention, core-only recreation, verification, separate restart state, shared lock, and recovery.
In `docs/install/pi-management.md`, use these headings:
@@ -519,7 +519,7 @@ Expected: both exit 0.
- [ ] **Step 6: Commit**
```bash
git add docs/contracts/thothctl-pi.md docs/install/pi-management.md docs/general/pi-configuration.md scripts/verify-workspace-install-docs.sh
git add docs/contracts/tht-pi.md docs/install/pi-management.md docs/general/pi-configuration.md scripts/verify-workspace-install-docs.sh
git commit -m "docs: clarify Pi reload and update workflows"
```
@@ -25,7 +25,7 @@
- Modify `tools/thothctl/cmd/thothctl/main.go` and `main_test.go` for optional global selection, `pi update` defaults, help text, and dispatch.
- Create `tools/thothctl/internal/pi/version.go` and `version_test.go` for reading the project Pi pin.
- Modify `tools/thothctl/internal/pi/update.go` and `update_test.go` only if the default request needs a typed source/confirmation adjustment; keep lifecycle internals unchanged otherwise.
- Modify `docs/contracts/thothctl-pi.md`, `docs/install/pi-management.md`, and relevant command-contract verification scripts.
- Modify `docs/contracts/tht-pi.md`, `docs/install/pi-management.md`, and relevant command-contract verification scripts.
### Task 1: Add bounded installation descriptor discovery
@@ -70,7 +70,7 @@
### Task 4: Update contracts without removing commands
**Files:**
- Modify: `docs/contracts/thothctl-pi.md`
- Modify: `docs/contracts/tht-pi.md`
- Modify: `docs/install/pi-management.md`
- Modify: the documentation verification script that asserts the old mandatory update invocation.
@@ -988,7 +988,7 @@ git commit -m "feat(frontend): simplify Pi management guidance"
**Files:**
- Rename: `docs/contracts/thothctl-pi.md` → `docs/contracts/tht-pi.md`
- Rename: `docs/contracts/tht-pi.md` → `docs/contracts/tht-pi.md`
- Modify: `README.md`
- Modify: `PROJECT_STATE.md`
- Modify: `AGENTS.md`
@@ -170,7 +170,7 @@ GUI workflow.
## Documentation changes
- Update `docs/contracts/thothctl-pi.md` with the restart safety and recovery contract.
- Update `docs/contracts/tht-pi.md` with the restart safety and recovery contract.
- Update `docs/install/pi-management.md` to separate GUI defaults, configuration reload, version
update, and failure recovery.
- Add a prominent ThothII-operator note to `docs/general/pi-configuration.md`: native Pi paths
@@ -16,10 +16,20 @@ than inferring a PASS.
staged/revalidated inside that lock immediately before extraction, and checkpointing requires
an opaque installation-bound transaction capability. Manual acceptance never substitutes for
those automated concurrency and mutation tests.
2. Run Workspace Validate first; it is the static authentication gate. Run `tht auth check` for
live non-interactive diagnosis, then `tht auth check --interactive` where Device Authorization
is available, then Workspace Test for aggregate live validation.
3. Run `tht doctor --json` and confirm this exact report order: `descriptor`, `files`, `docker`,
2. Set the installation and workspace identifiers, then inspect the active workspace with the
native host CLI. This replaces the former Workspace Validate/Test wording:
```bash
export THT_BIN=tht
export INSTALLATION=/absolute/path/to/thothii-installation.yaml
export WORKSPACE_ID=psd-clinical
"$THT_BIN" --installation "$INSTALLATION" \
workspace inspect --workspace "$WORKSPACE_ID" --json
```
3. Run `"$THT_BIN" --installation "$INSTALLATION" auth check --json` for live non-interactive
diagnosis, then `auth check --interactive` where Device Authorization is available.
4. Run `"$THT_BIN" --installation "$INSTALLATION" doctor --json` and confirm this exact report order: `descriptor`, `files`, `docker`,
`compose`, `configuration`, `authentication`, `services`, `core-http`, `frontend-http`,
`workspace-registry`, `workflow`, `pi`.
4. Confirm the exact direct `groups` claim for both identities and the mappings `TOT Users → user`
@@ -39,6 +49,7 @@ than inferring a PASS.
| Catalog token is wrong or lacks group-view-only access | Live check fails redacted with `oidc_group_catalog_unauthorized`. |
| Mapped group is renamed | The next check fails closed until configuration and provider agree. |
| Token adds an unrelated group | Login and authorization are unchanged; no warning is emitted. |
| Authenticated PSD identity creates a known-good session | SSE connects, the session is created, and the first reviewer gate appears without unexpected `401`/`403` responses. |
| Backend restarts with Remember me | Remembered local session survives within its TTL. |
| Password/role/enable revision changes | Affected local sessions are rejected and reauthentication is required. |
| CSRF or cross-origin mutation is attempted | Request is rejected. |
+20 -20
View File
@@ -18,7 +18,7 @@
**Status:** P2 implementation complete; automated integration PASS; manual acceptance PENDING.
Manual goal: from a clean local installation, use only `thothctl` on the host to inspect one
Manual goal: from a clean local installation, use only `tht` on the host to inspect one
registry workspace and execute the controlled REST-DWH/HTTP-Evidence preprocessing path without a
host Python or Node runtime. Use a fresh operator root and a fresh fixture Git remote; never reuse
the automated `.artifacts/p2-integration/**` state.
@@ -26,15 +26,15 @@ the automated `.artifacts/p2-integration/**` state.
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
```bash
thothctl --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --resume <run-id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
thothctl --installation <abs>/thothii-installation.yaml workspace schema check --workspace <id> --annotations <reviewed>.yaml --reviewed-candidates <sha256:hex> --json
thothctl --installation <abs>/thothii-installation.yaml workspace index-schema --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --json
tht --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --resume <run-id> --json
tht --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
tht --installation <abs>/thothii-installation.yaml workspace schema check --workspace <id> --annotations <reviewed>.yaml --reviewed-candidates <sha256:hex> --json
tht --installation <abs>/thothii-installation.yaml workspace index-schema --workspace <id> --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --json
```
Checks:
@@ -67,7 +67,7 @@ safe, and that search records are revision-scoped. See `docs/contracts/tht-dwh.m
Checks:
1. run `thothctl ... workspace preprocess dwh` twice with only an Evidence/content change between
1. run `tht ... workspace preprocess dwh` twice with only an Evidence/content change between
them: the second run reports `unchanged` and does not re-introspect;
2. change a DWH-affecting field (host/port/database/schema/user/collection) in the descriptor,
push, pull: the next run refuses the old generation and regenerates, with a clear
@@ -115,9 +115,9 @@ Checks to complete during P4 manual acceptance (decision: **PASS** (owner approv
`record_kind`, `vector_generation`, `workspace_id`, `workspace_revision`).
2. A pre-existing collection with incompatible dimensions/distance (e.g. 768-dim or dot)
is refused with `semantic_index_incompatible` and is never mutated.
3. `thothctl ... workspace vector inspect --workspace <id> --json` reports the collection
3. `tht ... workspace vector inspect --workspace <id> --json` reports the collection
contract without mutation (pristine JSON, exit 0).
4. `thothctl ... workspace vector rebuild --workspace <id> --collection <name>
4. `tht ... workspace vector rebuild --workspace <id> --collection <name>
--confirm <name> --destroy` deletes and recreates the descriptor-owned collection and
verifies the recreated contract; a mismatched `--confirm` or a missing `--destroy` is
refused (exit 2) without touching the collection.
@@ -135,10 +135,10 @@ pushing curated content from the operator CLI.
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
```bash
thothctl --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
tht --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
# curate the candidate into <id>/schema/annotations.yaml in the author clone, then commit/push/pull
thothctl --installation <abs>/thothii-installation.yaml workspace schema accept --workspace <id> --run <run-id> --yes --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --resume <run-id> --json
tht --installation <abs>/thothii-installation.yaml workspace schema accept --workspace <id> --run <run-id> --yes --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --resume <run-id> --json
```
Checks:
@@ -173,10 +173,10 @@ aggregate-limit failures without partial publication.
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
```bash
thothctl --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json # idempotent rerun
tht --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
tht --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json # idempotent rerun
```
Checks:
+4 -2
View File
@@ -22,6 +22,8 @@ docs=(
"$root/docs/install/psd-workspace-setup.md"
"$root/docs/install/reverse-proxy-caddy.md"
"$root/docs/install/reverse-proxy-nginx.md"
"$root/docs/contracts/tht-pi.md"
"$root/docs/contracts/workspace-preprocessing-cli.md"
"$root/docs/guida-utente.md"
"$root/docs/index.md"
"$root/README.md"
@@ -138,8 +140,8 @@ for relative, language, forbidden, required in [
raise SystemExit(f"auth docs smoke: {relative} omits scoped deprecated upstream auth")
PY
if rg -n -i --pcre2 '\bthothii-admin\b|\bthothctl\b[^\r\n]{0,256}\bauth\b' "$corpus"; then
echo "auth docs smoke: forbidden authentication CLI wording" >&2
if rg -n -i --pcre2 '\bthothii-admin\b|\bthothctl\b' "$corpus"; then
echo "auth docs smoke: forbidden obsolete host CLI wording" >&2
exit 1
fi
if rg -n -i --pcre2 -- '--password(?!-file)\b(?:[[:space:]]+|=)\S+' "$corpus"; then
+4 -2
View File
@@ -17,6 +17,8 @@ files=(
docs/install/psd-workspace-setup.md
docs/install/reverse-proxy-caddy.md
docs/install/reverse-proxy-nginx.md
docs/contracts/tht-pi.md
docs/contracts/workspace-preprocessing-cli.md
docs/guida-utente.md
docs/index.md
README.md
@@ -65,10 +67,10 @@ echo "auth docs positive fixture passed"
expect_rejected thothctl-intervening \
'Run thothctl --installation <descriptor> --json auth check.' \
'forbidden authentication CLI wording'
'forbidden obsolete host CLI wording'
expect_rejected alternate-admin \
'Run thothii-admin users list.' \
'forbidden authentication CLI wording'
'forbidden obsolete host CLI wording'
expect_rejected password-option \
'Run tht auth user add demo --password example-value.' \
'plaintext password option'
@@ -86,11 +86,11 @@ for required in \
exit 1
}
done
grep -Fq '"$THTCTL" --help' "$server_guide" || {
grep -Fq '"$THT_BIN" --help' "$server_guide" || {
echo "server guide lacks plain tht --help" >&2
exit 1
}
if grep -Fq '"$THTCTL" --installation "$INSTALLATION" --help' "$server_guide"; then
if grep -Fq '"$THT_BIN" --installation "$INSTALLATION" --help' "$server_guide"; then
echo "server guide still uses installation-scoped --help" >&2
exit 1
fi
@@ -106,7 +106,7 @@ for manual in "$root/docs/install/local-workspace-registry.md"; do
fi
done
grep -Fq 'THTCTL=/srv/thothii/operator/tht' \
grep -Fq 'THT_BIN=/srv/thothii/operator/tht' \
"$root/docs/install/server-workspace-registry.md" || {
echo "server installation manual does not use the installation-aware operator CLI" >&2
exit 1
@@ -645,6 +645,9 @@ expect_guide_rejected() {
if [[ "$validator" == verify_windows_line_endings_guide ]]; then
mkdir -p "$fixture_root/scripts"
cp "$root/scripts/verify-line-endings.sh" "$fixture_root/scripts/verify-line-endings.sh"
elif [[ "$validator" == verify_pi_management_guide ]]; then
mkdir -p "$fixture_root/docs/contracts"
cp "$root/docs/contracts/tht-pi.md" "$fixture_root/docs/contracts/tht-pi.md"
fi
node - "$fixture_root/$relative_path" "$mutation" <<'NODE'
const fs = require("fs");
@@ -910,11 +913,11 @@ expect_guide_rejected \
expect_guide_rejected \
"Nginx additional frontend bypass location" verify_reverse_proxy_nginx_guide \
"$root/docs/install/reverse-proxy-nginx.md" docs/install/reverse-proxy-nginx.md nginx-additional-bypass \
"Nginx frontend upstream location bypasses complete authentication contract"
"Nginx direct OIDC mode contains an additional frontend bypass location"
expect_guide_rejected \
"Nginx frontend auth directives only in comments" verify_reverse_proxy_nginx_guide \
"$root/docs/install/reverse-proxy-nginx.md" docs/install/reverse-proxy-nginx.md nginx-comment-only-auth \
"Nginx frontend upstream location bypasses complete authentication contract"
"Nginx direct OIDC mode contains an additional frontend bypass location"
expect_guide_rejected \
"Caddy identity without authentication" verify_reverse_proxy_caddy_guide \
"$root/docs/install/reverse-proxy-caddy.md" docs/install/reverse-proxy-caddy.md caddy-no-auth \
+82 -62
View File
@@ -851,9 +851,9 @@ if (/\|\|\s*true|;\s*true\b/.test(updateShell)) throw new Error("POSIX source up
requirePattern("POSIX source update does not fail closed: source pull", updateShell,
/if ! git pull --ff-only; then abort_update/);
requirePattern("POSIX source update does not fail closed: installation status", updateShell,
/if ! INSTALLATION_STATUS="\$\("\$THTCTL" --installation "\$INSTALLATION" status\)"; then/);
/if ! INSTALLATION_STATUS="\$\("\$THT_BIN" --installation "\$INSTALLATION" status\)"; then/);
requirePattern("POSIX source update does not fail closed: Pi status", updateShell,
/if ! RUNNING_PI_VERSION="\$\("\$THTCTL" --installation "\$INSTALLATION" pi status\)"; then/);
/if ! RUNNING_PI_VERSION="\$\("\$THT_BIN" --installation "\$INSTALLATION" pi status\)"; then/);
requirePattern("POSIX source update does not fail closed: local build", updateShell,
/if ! bash scripts\/build-local\.sh; then/);
requirePattern("POSIX source update does not fail closed: tht build", updateShell,
@@ -861,12 +861,12 @@ requirePattern("POSIX source update does not fail closed: tht build", updateShel
requirePattern("POSIX source update lacks the same-version/no-selector path", updateShell,
/if \[\[ "\$NEXT_PI_VERSION" == "\$RUNNING_PI_VERSION" \]\]; then[\s\S]*"\$USES_BASE_CORE" == true[\s\S]*TRANSACTIONAL_PI_UPDATE=false/);
for (const [label, pattern] of [
["installation start", /if ! "\$THTCTL" --installation "\$INSTALLATION" start; then/],
["installation start", /if ! "\$THT_BIN" --installation "\$INSTALLATION" start; then/],
["frontend health", /if ! curl --fail http:\/\/127\.0\.0\.1:8080\/health; then/],
["core health", /if ! curl --fail http:\/\/127\.0\.0\.1:8787\/health; then/],
["final status", /if ! FINAL_STATUS="\$\("\$THTCTL" --installation "\$INSTALLATION" status\)"; then/],
["final Pi status", /if ! FINAL_PI_STATUS="\$\("\$THTCTL" --installation "\$INSTALLATION" pi status\)"; then/],
["final doctor", /if ! "\$THTCTL" --installation "\$INSTALLATION" doctor; then/],
["final status", /if ! FINAL_STATUS="\$\("\$THT_BIN" --installation "\$INSTALLATION" status\)"; then/],
["final Pi status", /if ! FINAL_PI_STATUS="\$\("\$THT_BIN" --installation "\$INSTALLATION" pi status\)"; then/],
["final doctor", /if ! "\$THT_BIN" --installation "\$INSTALLATION" doctor; then/],
]) requirePattern(`POSIX source update does not fail closed: ${label}`, updateShell, pattern);
const provenance = updateShell.indexOf("printf 'Built source revision:");
if (provenance < updateShell.lastIndexOf("require_clean_source") ||
@@ -883,14 +883,14 @@ requireTokens("native PowerShell source update", updatePowerShell, [
]);
for (const [command, step] of [
["git pull --ff-only", "source pull"],
["$InstallationStatus = @(& $THTCTL --installation $INSTALLATION status)", "installation status"],
["$RunningPiStatus = (& $THTCTL --installation $INSTALLATION pi status)", "Pi status"],
["$InstallationStatus = @(& $THT_BIN --installation $INSTALLATION status)", "installation status"],
["$RunningPiStatus = (& $THT_BIN --installation $INSTALLATION pi status)", "Pi status"],
["powershell -ExecutionPolicy Bypass -File scripts/build-local.ps1", "local image build"],
["& \"C:\\Program Files\\Git\\bin\\bash.exe\" scripts/build-tht.sh", "tht build"],
["curl.exe --fail --silent --show-error http://127.0.0.1:8080/health", "frontend health check"],
["curl.exe --fail --silent --show-error http://127.0.0.1:8787/health", "core health check"],
["$FinalPiStatus = (& $THTCTL --installation $INSTALLATION pi status)", "final Pi status"],
["& $THTCTL --installation $INSTALLATION doctor", "final doctor"],
["$FinalPiStatus = (& $THT_BIN --installation $INSTALLATION pi status)", "final Pi status"],
["& $THT_BIN --installation $INSTALLATION doctor", "final doctor"],
]) {
const commandAt = updatePowerShell.indexOf(command);
const checkAt = updatePowerShell.indexOf(`Assert-NativeSuccess '${step}'`, commandAt);
@@ -963,7 +963,7 @@ NODE
(
cd "$update_fixture/project"
env PATH="$update_fixture/bin:$PATH" CALLS="$calls" FAIL_STEP="$fixture_step" \
THTCTL="$update_fixture/bin/tht" INSTALLATION="$update_fixture/installation.yaml" \
THT_BIN="$update_fixture/bin/tht" INSTALLATION="$update_fixture/installation.yaml" \
/bin/bash "$update_script"
) >"$output" 2>&1
status=$?
@@ -1339,7 +1339,6 @@ verify_reverse_proxy_nginx_guide() {
}
require_headings "$guide" "Nginx reverse-proxy guide" \
"Trust boundary" \
"Example configuration" \
"Validate and reload" \
"Test authentication and SSE"
require_text "$guide" "Nginx reverse-proxy guide" \
@@ -1437,16 +1436,69 @@ function nginxLocations(text) {
return locations;
}
const locations = nginxLocations(effectiveBlock);
const authLocations = locations.filter((location) => location.selector === "= /_authenticate");
if (authLocations.length !== 1) {
throw new Error("Nginx proxy must define exactly one authentication location");
}
const authLocation = authLocations[0].body;
const frontendLocations = locations.filter((location) =>
/proxy_pass\s+http:\/\/127\.0\.0\.1:8080\s*;/.test(location.body));
if (frontendLocations.length === 0) {
throw new Error("Nginx proxy lacks a frontend upstream location");
}
const authenticatedFrontendLocations = frontendLocations.filter((location) =>
/auth_request\s+\/_authenticate\s*;/.test(location.body));
const directFrontendLocations = frontendLocations.filter((location) =>
!/auth_request\s+\/_authenticate\s*;/.test(location.body));
if (directFrontendLocations.length > 1) {
throw new Error("Nginx direct OIDC mode contains an additional frontend bypass location");
}
for (const frontendLocation of frontendLocations) {
for (const token of [
"proxy_http_version 1.1;", "proxy_buffering off;", "proxy_cache off;",
"proxy_read_timeout 3600s;",
]) {
if (!frontendLocation.body.includes(token)) {
throw new Error(`Nginx proxy lacks structural token: ${token}`);
}
}
}
if (authenticatedFrontendLocations.length > 0) {
const authLocations = locations.filter((location) => location.selector === "= /_authenticate");
if (authLocations.length !== 1) {
throw new Error("Nginx proxy must define exactly one authentication location for upstream mode");
}
const authLocation = authLocations[0].body;
for (const [label, publicName, variable, upstream] of identities) {
const trustedName = publicName === "Is-Admin" ? "Is-Admin" : publicName;
const publicClear = new RegExp(`proxy_set_header\\s+X-Thoth-${escaped(publicName)}\\s+"";`);
const trustedHeader = `X-Thoth-Trusted-${trustedName}`;
const trustedClear = new RegExp(`proxy_set_header\\s+${escaped(trustedHeader)}\\s+"";`);
const authPublicAt = authLocation.search(publicClear);
const authTrustedAt = authLocation.search(trustedClear);
if (authPublicAt < 0) {
throw new Error(`Nginx auth location does not clear inbound ${label} identity`);
}
if (authTrustedAt < 0) {
throw new Error(`Nginx auth location does not clear inbound trusted ${label} identity`);
}
for (const frontendLocation of authenticatedFrontendLocations) {
const frontendPublicAt = frontendLocation.body.search(publicClear);
if (frontendPublicAt < 0) {
throw new Error(`Nginx frontend location does not clear inbound ${label} identity`);
}
const normalizedFrontend = frontendLocation.body.replace(/\s+/g, " ");
const captureAt = normalizedFrontend.search(new RegExp(`auth_request_set\\s+\\$${variable}\\s+\\$upstream_http_${upstream};`));
if (captureAt < 0) {
throw new Error(`Nginx frontend location does not capture authenticated ${label} identity`);
}
const mapAt = normalizedFrontend.search(new RegExp(`proxy_set_header\\s+${escaped(trustedHeader)}\\s+\\$${variable};`));
if (mapAt < 0) {
throw new Error(`Nginx frontend location does not map authenticated ${label} identity`);
}
}
if (new RegExp(`auth_request_set\\s+\\$${variable}|proxy_set_header\\s+${escaped(trustedHeader)}\\s+\\$${variable};`).test(authLocation)) {
throw new Error(`Nginx auth location performs a forbidden ${label} capture or mapping`);
}
}
} else if (directFrontendLocations.length === 0) {
throw new Error("Nginx proxy lacks a direct or authenticated frontend path");
}
for (const location of locations) {
const upstreams = [...location.body.matchAll(/proxy_pass\s+([^;]+);/g)].map((match) => match[1].trim());
for (const upstream of upstreams) {
@@ -1455,41 +1507,9 @@ for (const location of locations) {
throw new Error(`Nginx location proxies to an unreviewed upstream: ${upstream}`);
}
}
for (const frontendLocation of frontendLocations) {
for (const frontendLocation of authenticatedFrontendLocations) {
if (!/auth_request\s+\/_authenticate\s*;/.test(frontendLocation.body)) {
throw new Error("Nginx frontend upstream location bypasses complete authentication contract");
}
}
for (const [label, publicName, variable, upstream] of identities) {
const trustedName = publicName === "Is-Admin" ? "Is-Admin" : publicName;
const publicClear = new RegExp(`proxy_set_header\\s+X-Thoth-${escaped(publicName)}\\s+"";`);
const trustedHeader = `X-Thoth-Trusted-${trustedName}`;
const trustedClear = new RegExp(`proxy_set_header\\s+${escaped(trustedHeader)}\\s+"";`);
const authPublicAt = authLocation.search(publicClear);
const authTrustedAt = authLocation.search(trustedClear);
if (authPublicAt < 0) {
throw new Error(`Nginx auth location does not clear inbound ${label} identity`);
}
if (authTrustedAt < 0) {
throw new Error(`Nginx auth location does not clear inbound trusted ${label} identity`);
}
for (const frontendLocation of frontendLocations) {
const frontendPublicAt = frontendLocation.body.search(publicClear);
if (frontendPublicAt < 0) {
throw new Error(`Nginx frontend location does not clear inbound ${label} identity`);
}
const normalizedFrontend = frontendLocation.body.replace(/\s+/g, " ");
const captureAt = normalizedFrontend.search(new RegExp(`auth_request_set\\s+\\$${variable}\\s+\\$upstream_http_${upstream};`));
if (captureAt < 0) {
throw new Error(`Nginx frontend location does not capture authenticated ${label} identity`);
}
const mapAt = normalizedFrontend.search(new RegExp(`proxy_set_header\\s+${escaped(trustedHeader)}\\s+\\$${variable};`));
if (mapAt < 0) {
throw new Error(`Nginx frontend location does not map authenticated ${label} identity`);
}
}
if (new RegExp(`auth_request_set\\s+\\$${variable}|proxy_set_header\\s+${escaped(trustedHeader)}\\s+\\$${variable};`).test(authLocation)) {
throw new Error(`Nginx auth location performs a forbidden ${label} capture or mapping`);
throw new Error("Nginx authenticated frontend path bypasses complete authentication contract");
}
}
NODE
@@ -1504,21 +1524,20 @@ verify_reverse_proxy_caddy_guide() {
}
require_headings "$guide" "Caddy reverse-proxy guide" \
"Trust boundary" \
"Example configuration" \
"Validate and reload" \
"Test authentication and SSE"
require_text "$guide" "Caddy reverse-proxy guide" \
"Forwarding identity headers alone does not authenticate a user" \
"authentication gateway" \
"2xx" \
"automatic HTTPS" \
"Caddy terminates TLS" \
"frontend"
node - "$guide" <<'NODE'
const fs = require("fs");
const source = fs.readFileSync(process.argv[2], "utf8");
const block = [...source.matchAll(/```caddyfile\n([\s\S]*?)```/g)].map((match) => match[1]).join("\n");
const tokens = [
"thoth.example.com {", "route {",
"thoth.example.invalid {", "route {",
"forward_auth auth-gateway:4180 {", "uri /verify", "copy_headers {",
"reverse_proxy 127.0.0.1:8080 {", "flush_interval -1",
];
@@ -1730,15 +1749,15 @@ verify_manual() {
'export THT_SOURCE_ROOT=/absolute/path/to/ThothII'
'thothii-installation.yaml'
'workspaceRepository'
'"$THTCTL" --installation "$INSTALLATION" start'
'"$THTCTL" --installation "$INSTALLATION" doctor'
'"$THT_BIN" --installation "$INSTALLATION" start'
'"$THT_BIN" --installation "$INSTALLATION" doctor'
)
else
expected_steps=(
'THTCTL=/srv/thothii/operator/tht'
'THT_BIN=/srv/thothii/operator/tht'
'INSTALLATION=/srv/thothii/operator/thothii-installation.yaml'
'"$THTCTL" --installation "$INSTALLATION" start'
'"$THTCTL" --installation "$INSTALLATION" doctor'
'"$THT_BIN" --installation "$INSTALLATION" start'
'"$THT_BIN" --installation "$INSTALLATION" doctor'
'docs/install/examples/thothii-installation.server.yaml'
'compose.yaml'
'deploy/compose.server.yaml'
@@ -1829,7 +1848,7 @@ for required in (
ui = (root / "frontend/src/shell/WorkspaceManager.tsx").read_text()
for required in (
"Create a local workspace",
"Create a workspace repository",
"Update workspace repository",
"No workspace selection is required",
"Temporary files are deleted after the test",
@@ -2024,8 +2043,9 @@ if (Object.keys(config.services).sort().join(",") !== "core,embedding,embedding-
}
const core = config.services.core;
const frontend = config.services.frontend;
if (core.environment?.AUTH_MODE !== "upstream" || core.environment?.THOTH_PUBLIC_EXPOSURE !== "true") {
throw new Error("server installation example must fail closed behind upstream authentication");
if (core.environment?.THOTH_PUBLIC_EXPOSURE !== "true" ||
core.environment?.THT_AUTH_CONFIG_FILE !== "/run/thothii-auth/auth.yaml") {
throw new Error("server installation example must expose only the authenticated frontend");
}
if ((core.ports || []).length !== 0) throw new Error("server installation example published core");
const ports = frontend.ports || [];