docs: finalize P2 manuals, project state, and install-doc service contract
This commit is contained in:
@@ -3,6 +3,30 @@
|
|||||||
> Starting-point snapshot for new sessions. Last updated: 2026-08-10 (final verification).
|
> Starting-point snapshot for new sessions. Last updated: 2026-08-10 (final verification).
|
||||||
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
|
||||||
|
|
||||||
|
### P2 host preprocessing CLI — implementation complete, automated PASS, manual PENDING (2026-08-11)
|
||||||
|
|
||||||
|
- **Scope:** P2 (PRD D2, based on the P1.1 registry contract): the installed native `thothctl`
|
||||||
|
binary is the only host interface for workspace preprocessing. Commands: `workspace inspect`,
|
||||||
|
`preprocess dwh`, `schema suggest-fks`, `schema check`, `index-schema`, `preprocess evidence`,
|
||||||
|
`preprocess run`, with the exact grammar, file-ingress bounds, result contract and exit codes in
|
||||||
|
`docs/contracts/workspace-preprocessing-cli.md`.
|
||||||
|
- **Operator:** `workspace-maintenance` is a profile-gated Compose service sharing the core image,
|
||||||
|
with no Pi auth/state, no backend/Pi/frontend listener, no Git credentials, and a compiled Node
|
||||||
|
entrypoint (`backend/src/workspace-maintenance.ts`) driving the existing harness engine through
|
||||||
|
pristine JSON machine interfaces (`schema_cmd.py`, `vector_cmd.py`, `preprocess_cmd.py`).
|
||||||
|
- **Boundaries honored:** FK review is digest-bound (candidate digest == persisted artifact; a
|
||||||
|
review accepted for the same candidate content counts); Qdrant collections are never created by
|
||||||
|
the product path (`require_existing` + pre-provisioned fixture, P4 owns lifecycle); filesystem
|
||||||
|
Evidence stops with `evidence_materialization_required` (P6); HTTP Evidence enforces an
|
||||||
|
installation private-host allowlist; `ssh_tunnel` stays fail-closed (P10); cross-revision DWH
|
||||||
|
reuse is explicitly P3.
|
||||||
|
- **Retained evidence:** `.artifacts/p2-integration/p2-b109757b26388a5ed6b1d173dee86584/`
|
||||||
|
(11/11 checks PASS), bound to clean source commit
|
||||||
|
`de5de36f9a4edfd4fbebf277822090871ccdd61f`.
|
||||||
|
- **Manual gate:** P2 walkthrough in `docs/testing/p2-p6-manual-verification.md`; decision
|
||||||
|
**PENDING** and recorded independently. P3 and later start only after an explicit new
|
||||||
|
authorization.
|
||||||
|
|
||||||
### P1.1 workspace-directory registry — automated integration PASS, manual PENDING (2026-08-11)
|
### P1.1 workspace-directory registry — automated integration PASS, manual PENDING (2026-08-11)
|
||||||
|
|
||||||
- **Scope:** P1 correction (not preprocessing). Root curator-owned catalog `thoth-workspaces.yaml`;
|
- **Scope:** P1 correction (not preprocessing). Root curator-owned catalog `thoth-workspaces.yaml`;
|
||||||
|
|||||||
@@ -19,6 +19,17 @@ Compose stack. Never put credentials in workspace YAML, Git, browser drafts, dia
|
|||||||
| Qdrant | Internal | Mandatory private Compose semantic service; persistent `qdrant-data` volume. |
|
| Qdrant | Internal | Mandatory private Compose semantic service; persistent `qdrant-data` volume. |
|
||||||
| Ollama embedding | Internal | Mandatory private Compose semantic service for `qwen3-embedding:0.6b`. |
|
| Ollama embedding | Internal | Mandatory private Compose semantic service for `qwen3-embedding:0.6b`. |
|
||||||
|
|
||||||
|
|
||||||
|
## Host preprocessing (P2)
|
||||||
|
|
||||||
|
The installed native `thothctl` is the only host entrypoint for workspace preprocessing
|
||||||
|
(introspection+LSH, FK review, schema indexing, HTTP Evidence). Use
|
||||||
|
`thothctl --installation <thothii-installation.yaml> workspace <command> --workspace <id> [--json]`
|
||||||
|
per `docs/contracts/workspace-preprocessing-cli.md` and the P2 walkthrough in
|
||||||
|
`docs/testing/p2-p6-manual-verification.md`. Preprocessing runs through the profile-gated
|
||||||
|
`workspace-maintenance` Compose service; it never starts a backend/Pi/frontend listener and never
|
||||||
|
attaches Git credentials.
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- macOS: Docker Desktop, Git, and sufficient volume disk space. Git Credential Manager is useful
|
- macOS: Docker Desktop, Git, and sufficient volume disk space. Git Credential Manager is useful
|
||||||
|
|||||||
@@ -14,6 +14,17 @@ proxy; never publish the core port directly.
|
|||||||
| Qdrant | Internal | Mandatory private Compose semantic service; persistent `qdrant-data` volume. |
|
| Qdrant | Internal | Mandatory private Compose semantic service; persistent `qdrant-data` volume. |
|
||||||
| Ollama embedding | Internal | Mandatory private Compose semantic service for `qwen3-embedding:0.6b`. |
|
| Ollama embedding | Internal | Mandatory private Compose semantic service for `qwen3-embedding:0.6b`. |
|
||||||
|
|
||||||
|
|
||||||
|
## Host preprocessing (P2)
|
||||||
|
|
||||||
|
The installed native `thothctl` is the only host entrypoint for workspace preprocessing
|
||||||
|
(introspection+LSH, FK review, schema indexing, HTTP Evidence). Use
|
||||||
|
`thothctl --installation <thothii-installation.yaml> workspace <command> --workspace <id> [--json]`
|
||||||
|
per `docs/contracts/workspace-preprocessing-cli.md` and the P2 walkthrough in
|
||||||
|
`docs/testing/p2-p6-manual-verification.md`. Preprocessing runs through the profile-gated
|
||||||
|
`workspace-maintenance` Compose service; it never starts a backend/Pi/frontend listener and never
|
||||||
|
attaches Git credentials.
|
||||||
|
|
||||||
## Service account, storage, and firewall
|
## Service account, storage, and firewall
|
||||||
|
|
||||||
Create a dedicated host service account and an operator root such as `/srv/thothii`. The core
|
Create a dedicated host service account and an operator root such as `/srv/thothii`. The core
|
||||||
|
|||||||
@@ -16,25 +16,45 @@
|
|||||||
|
|
||||||
## P2 — Host preprocessing CLI
|
## P2 — Host preprocessing CLI
|
||||||
|
|
||||||
**Status:** instructions to be finalized by P2 implementation; not yet runnable.
|
**Status:** P2 implementation complete; automated integration PASS; manual acceptance PENDING.
|
||||||
|
|
||||||
Manual goal: from a clean local installation, use only `thothctl` on the host to inspect one
|
Manual goal: from a clean local installation, use only `thothctl` on the host to inspect one
|
||||||
registry workspace and execute the controlled REST-DWH/HTTP-Evidence preprocessing path without a
|
registry workspace and execute the controlled REST-DWH/HTTP-Evidence preprocessing path without a
|
||||||
host Python or Node runtime.
|
host Python or Node runtime. Use a fresh operator root and a fresh fixture Git remote; never reuse
|
||||||
|
the automated `.artifacts/p2-integration/**` state.
|
||||||
|
|
||||||
Checks to fill during P2:
|
Commands (contract: `docs/contracts/workspace-preprocessing-cli.md`):
|
||||||
|
|
||||||
1. installation/render preflight;
|
```bash
|
||||||
2. workspace inspection and exact revision display;
|
thothctl --installation <abs>/thothii-installation.yaml workspace inspect --workspace <id> --json
|
||||||
3. DWH preprocessing and resume;
|
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --json
|
||||||
4. FK machine output and manual-review checkpoint;
|
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess dwh --workspace <id> --resume <run-id> --json
|
||||||
5. schema check/index;
|
thothctl --installation <abs>/thothii-installation.yaml workspace schema suggest-fks --workspace <id> --from-sql <file>.sql --output <candidates>.yaml --json
|
||||||
6. HTTP Evidence dry-run and real run;
|
thothctl --installation <abs>/thothii-installation.yaml workspace schema check --workspace <id> --annotations <reviewed>.yaml --reviewed-candidates <sha256:hex> --json
|
||||||
7. idempotent rerun;
|
thothctl --installation <abs>/thothii-installation.yaml workspace index-schema --workspace <id> --json
|
||||||
8. filesystem Evidence stable deferred error;
|
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --dry-run --json
|
||||||
9. secret scan and exact cleanup.
|
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess evidence --workspace <id> --json
|
||||||
|
thothctl --installation <abs>/thothii-installation.yaml workspace preprocess run --workspace <id> --json
|
||||||
|
```
|
||||||
|
|
||||||
Decision: **PENDING**.
|
Checks:
|
||||||
|
|
||||||
|
1. installation/render preflight (`inspect` returns exact revision + catalog/descriptor digests);
|
||||||
|
2. DWH introspection+LSH succeeds, rerun is `unchanged`, `--resume <run-id>` is `unchanged`/`succeeded`;
|
||||||
|
3. `schema suggest-fks` returns pristine JSON with `suggestedFksYaml` and a `manual_review_required`
|
||||||
|
block (exit 3) when candidates exist; the suggested YAML digest equals the reported digest;
|
||||||
|
4. `schema check --annotations <reviewed> --reviewed-candidates <digest>` succeeds after review;
|
||||||
|
5. `index-schema` counts against a pre-provisioned compatible collection and rerun is `unchanged`;
|
||||||
|
6. HTTP Evidence `--dry-run` returns `dry_run`, the real run publishes, rerun is `unchanged`, an input
|
||||||
|
mutation produces a new generation/ACTIVE;
|
||||||
|
7. filesystem Evidence returns a stable `evidence_materialization_required` block with no partial
|
||||||
|
corpus/vector publication;
|
||||||
|
8. negatives: missing workspace (`workspace_not_activatable`), resume of a nonexistent run
|
||||||
|
(`preprocessing_resume_mismatch`), invalid annotations digest (`annotation_invalid`), no-Evidence
|
||||||
|
skip warning, no collection creation, no backend/Pi/frontend listener;
|
||||||
|
9. secret scan over retained artifacts and exact owned-resource cleanup.
|
||||||
|
|
||||||
|
Decision: **PENDING** (independent manual gate; automation never records PASS).
|
||||||
|
|
||||||
## P3 — Effective config and `.tht-dwh`
|
## P3 — Effective config and `.tht-dwh`
|
||||||
|
|
||||||
|
|||||||
@@ -176,9 +176,11 @@ verify_compose_internal_semantic_contract() {
|
|||||||
import sys, yaml, pathlib
|
import sys, yaml, pathlib
|
||||||
doc = yaml.safe_load(pathlib.Path(sys.argv[1]).read_text())
|
doc = yaml.safe_load(pathlib.Path(sys.argv[1]).read_text())
|
||||||
services = doc["services"]
|
services = doc["services"]
|
||||||
expected = {"core", "frontend", "qdrant", "embedding", "embedding-model-init"}
|
expected = {"core", "frontend", "qdrant", "embedding", "embedding-model-init", "workspace-maintenance"}
|
||||||
if set(services) != expected:
|
if set(services) != expected:
|
||||||
raise SystemExit(f"compose.yaml services mismatch: {sorted(services)}")
|
raise SystemExit(f"compose.yaml services mismatch: {sorted(services)}")
|
||||||
|
if doc["services"]["workspace-maintenance"].get("profiles") != ["workspace-maintenance"]:
|
||||||
|
raise SystemExit("workspace-maintenance must be profile-gated and absent from default startup")
|
||||||
core = services["core"]
|
core = services["core"]
|
||||||
env = core["environment"]
|
env = core["environment"]
|
||||||
for key, value in {
|
for key, value in {
|
||||||
|
|||||||
Reference in New Issue
Block a user