mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: finalize P2 manuals, project state, and install-doc service contract

Browse Source
This commit is contained in:
marcopan
2026-08-11 20:09:11 +02:00
parent de5de36f9a
commit 3c5c2e8afd
5 changed files with 82 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PROJECT_STATE.md
+24
View File
@@ -3,6 +3,30 @@
> Starting-point snapshot for new sessions. Last updated: 2026-08-10 (final verification).
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
### P2 host preprocessing CLI — implementation complete, automated PASS, manual PENDING (2026-08-11)
- **Scope:** P2 (PRD D2, based on the P1.1 registry contract): the installed native `thothctl`
binary is the only host interface for workspace preprocessing. Commands: `workspace inspect`,
`preprocess dwh`, `schema suggest-fks`, `schema check`, `index-schema`, `preprocess evidence`,
`preprocess run`, with the exact grammar, file-ingress bounds, result contract and exit codes in
`docs/contracts/workspace-preprocessing-cli.md`.
- **Operator:** `workspace-maintenance` is a profile-gated Compose service sharing the core image,
with no Pi auth/state, no backend/Pi/frontend listener, no Git credentials, and a compiled Node
entrypoint (`backend/src/workspace-maintenance.ts`) driving the existing harness engine through
pristine JSON machine interfaces (`schema_cmd.py`, `vector_cmd.py`, `preprocess_cmd.py`).
- **Boundaries honored:** FK review is digest-bound (candidate digest == persisted artifact; a
review accepted for the same candidate content counts); Qdrant collections are never created by
the product path (`require_existing` + pre-provisioned fixture, P4 owns lifecycle); filesystem
Evidence stops with `evidence_materialization_required` (P6); HTTP Evidence enforces an
installation private-host allowlist; `ssh_tunnel` stays fail-closed (P10); cross-revision DWH
reuse is explicitly P3.
- **Retained evidence:** `.artifacts/p2-integration/p2-b109757b26388a5ed6b1d173dee86584/`
(11/11 checks PASS), bound to clean source commit
`de5de36f9a4edfd4fbebf277822090871ccdd61f`.
- **Manual gate:** P2 walkthrough in `docs/testing/p2-p6-manual-verification.md`; decision
**PENDING** and recorded independently. P3 and later start only after an explicit new
authorization.
### P1.1 workspace-directory registry — automated integration PASS, manual PENDING (2026-08-11)
- **Scope:** P1 correction (not preprocessing). Root curator-owned catalog `thoth-workspaces.yaml`;
docs/install/local-workspace-registry.md
+11
View File
@@ -19,6 +19,17 @@ Compose stack. Never put credentials in workspace YAML, Git, browser drafts, dia
| Qdrant | Internal | Mandatory private Compose semantic service; persistent `qdrant-data` volume. |
| Ollama embedding | Internal | Mandatory private Compose semantic service for `qwen3-embedding:0.6b`. |
## Host preprocessing (P2)
The installed native `thothctl` is the only host entrypoint for workspace preprocessing
(introspection+LSH, FK review, schema indexing, HTTP Evidence). Use
`thothctl --installation <thothii-installation.yaml> workspace <command> --workspace <id> [--json]`
per `docs/contracts/workspace-preprocessing-cli.md` and the P2 walkthrough in
`docs/testing/p2-p6-manual-verification.md`. Preprocessing runs through the profile-gated
`workspace-maintenance` Compose service; it never starts a backend/Pi/frontend listener and never
attaches Git credentials.
## Prerequisites
- macOS: Docker Desktop, Git, and sufficient volume disk space. Git Credential Manager is useful
docs/install/server-workspace-registry.md
+11
View File
@@ -14,6 +14,17 @@ proxy; never publish the core port directly.
| Qdrant | Internal | Mandatory private Compose semantic service; persistent `qdrant-data` volume. |
| Ollama embedding | Internal | Mandatory private Compose semantic service for `qwen3-embedding:0.6b`. |
## Host preprocessing (P2)
The installed native `thothctl` is the only host entrypoint for workspace preprocessing
(introspection+LSH, FK review, schema indexing, HTTP Evidence). Use
`thothctl --installation <thothii-installation.yaml> workspace <command> --workspace <id> [--json]`
per `docs/contracts/workspace-preprocessing-cli.md` and the P2 walkthrough in
`docs/testing/p2-p6-manual-verification.md`. Preprocessing runs through the profile-gated
`workspace-maintenance` Compose service; it never starts a backend/Pi/frontend listener and never
attaches Git credentials.
## Service account, storage, and firewall
Create a dedicated host service account and an operator root such as `/srv/thothii`. The core
docs/testing/p2-p6-manual-verification.md
+33 -13
View File
@@ -16,25 +16,45 @@
## P2 — Host preprocessing CLI
**Status:** instructions to be finalized by P2 implementation; not yet runnable.
**Status:** P2 implementation complete; automated integration PASS; manual acceptance PENDING.
Manual goal: from a clean local installation, use only `thothctl` on the host to inspect one
registry workspace and execute the controlled REST-DWH/HTTP-Evidence preprocessing path without a
host Python or Node runtime.
host Python or Node runtime. Use a fresh operator root and a fresh fixture Git remote; never reuse
the automated `.artifacts/p2-integration/**` state.
Checks to fill during P2:
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
1. installation/render preflight;
2. workspace inspection and exact revision display;
3. DWH preprocessing and resume;
4. FK machine output and manual-review checkpoint;
5. schema check/index;
6. HTTP Evidence dry-run and real run;
7. idempotent rerun;
8. filesystem Evidence stable deferred error;
9. secret scan and exact cleanup.
```bash
thothctl --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --resume <run-id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
thothctl --installation <abs>/thothii-installation.yaml workspace schema check --workspace <id> --annotations <reviewed>.yaml --reviewed-candidates <sha256:hex> --json
thothctl --installation <abs>/thothii-installation.yaml workspace index-schema --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --json
```
Decision: **PENDING**.
Checks:
1. installation/render preflight (`inspect` returns exact revision + catalog/descriptor digests);
2. DWH introspection+LSH succeeds, rerun is `unchanged`, `--resume <run-id>` is `unchanged`/`succeeded`;
3. `schema suggest-fks` returns pristine JSON with `suggestedFksYaml` and a `manual_review_required`
block (exit 3) when candidates exist; the suggested YAML digest equals the reported digest;
4. `schema check --annotations <reviewed> --reviewed-candidates <digest>` succeeds after review;
5. `index-schema` counts against a pre-provisioned compatible collection and rerun is `unchanged`;
6. HTTP Evidence `--dry-run` returns `dry_run`, the real run publishes, rerun is `unchanged`, an input
mutation produces a new generation/ACTIVE;
7. filesystem Evidence returns a stable `evidence_materialization_required` block with no partial
corpus/vector publication;
8. negatives: missing workspace (`workspace_not_activatable`), resume of a nonexistent run
(`preprocessing_resume_mismatch`), invalid annotations digest (`annotation_invalid`), no-Evidence
skip warning, no collection creation, no backend/Pi/frontend listener;
9. secret scan over retained artifacts and exact owned-resource cleanup.
Decision: **PENDING** (independent manual gate; automation never records PASS).
## P3 — Effective config and `.tht-dwh`
scripts/verify-workspace-install-docs.sh
+3 -1
View File
@@ -176,9 +176,11 @@ verify_compose_internal_semantic_contract() {
import sys, yaml, pathlib
doc = yaml.safe_load(pathlib.Path(sys.argv[1]).read_text())
services = doc["services"]
expected = {"core", "frontend", "qdrant", "embedding", "embedding-model-init"}
expected = {"core", "frontend", "qdrant", "embedding", "embedding-model-init", "workspace-maintenance"}
if set(services) != expected:
raise SystemExit(f"compose.yaml services mismatch: {sorted(services)}")
if doc["services"]["workspace-maintenance"].get("profiles") != ["workspace-maintenance"]:
raise SystemExit("workspace-maintenance must be profile-gated and absent from default startup")
core = services["core"]
env = core["environment"]
for key, value in {