feat: complete catalog-driven preprocessing
Publish documentation / publish (push) Successful in 2m12s

This commit is contained in:
Codex
2026-09-06 17:49:35 +02:00
parent 8707ae1d46
commit cffa60772e
141 changed files with 5898 additions and 3015 deletions
+24 -12
View File
@@ -106,16 +106,18 @@ a remote user's partial list. The isolated deployment exercise is
`./scripts/verify-workspace-install-docs.sh --profile local` or `--profile server`.
<!-- workspace-descriptor-contract:start -->
Schema v4 is the only accepted workspace descriptor. Schema v1, v2, and v3 workspace descriptors
are rejected before activation. Candidate snapshot validation therefore makes activation or a pull
fail atomically while the prior valid snapshot remains active. One workspace owns one Qdrant collection;
schema, Evidence, and Memory records share that collection and stay separated by indexed payload
`kind`.
Schema v4 is the only accepted workspace descriptor. It contains workspace identity and optional
Evidence configuration only; PostgreSQL Metadata Catalog owns every database fact and binding.
Schema v1, v2, and v3 descriptors are rejected before activation. Candidate snapshot validation
therefore makes activation or a pull fail atomically while the prior valid snapshot remains active.
Each workspace owns separate Qdrant `reference` and `memory` collections: Schema, relationships, and
Evidence are replaceable reference data; Memory and solved questions have a persistent lifecycle.
<!-- workspace-descriptor-contract:end -->
<!-- non-workspace-migration:start -->
Convert a v3 descriptor before publication by setting `workspace.schema_version` to `4` and
removing `llm_policy` and `semantic_index`; no database or Evidence field changes.
Create a clean v4 descriptor containing only `workspace` and optional `evidence`. Do not copy the
legacy database, diagnostics, `llm_policy`, or `semantic_index` blocks; configure the database in
Database Management.
<!-- non-workspace-migration:end -->
For NL→SQL runtime sessions, connector `ssh_tunnel` bindings remain diagnostic-only: their bounded
@@ -223,15 +225,23 @@ job remains deterministic and does not claim Docker startup.
## Workspace preprocessing and S3 Evidence
Run preprocessing through the native host CLI and the installation descriptor:
For an interactive run, select the workspace, expand **Administration** in the right sidebar, and
use its **Preprocessing** control. The control explains any unmet prerequisite and exposes only the
latest safe failure diagnostic. For unattended operation, use the native host CLI and installation
descriptor:
```sh
tht --installation /absolute/path/thothii-installation.yaml workspace preprocess evidence
tht --installation /absolute/path/thothii-installation.yaml workspace preprocess dwh
tht --installation /absolute/path/thothii-installation.yaml \
workspace preprocess run --workspace <workspace-id>
tht --installation /absolute/path/thothii-installation.yaml \
workspace preprocess clear --workspace <workspace-id>
```
The CLI starts the profile-gated `workspace-maintenance` service and enforces the workspace,
secret, Qdrant, and embedding contracts. See [Evidence](docs/evidence.md) and the
The one-shot command starts the profile-gated `workspace-maintenance` service, reads database
metadata from PostgreSQL, and rebuilds LSH plus schema/Evidence vectors. The clear command removes
those derived artifacts while preserving the separate Memory collection. The core remains unavailable
until preprocessing completes. See [Evidence](docs/evidence.md) and the
[workspace preprocessing CLI contract](docs/contracts/workspace-preprocessing-cli.md).
S3 Evidence uses the optional `tht[s3]` dependency and canonical `s3://bucket/key` provenance.
@@ -347,6 +357,8 @@ The server profile stores sessions and per-user preferences directly in PostgreS
dual write. Use [`deploy/compose.session-server.yaml.example`](deploy/compose.session-server.yaml.example)
with the canonical base+server files and set `THT_SERVER_WORKSPACE_CONFIG` to an absolute,
protected copy of [`deploy/workspaces/server-sessions.yaml.example`](deploy/workspaces/server-sessions.yaml.example).
That file is an installation runtime template, not an authored workspace descriptor; database
bindings are injected from the PostgreSQL Metadata Catalog for each runtime lease.
The runtime login needs membership in the no-login database role `thoth_sessions_runtime` only.
The distinct, one-shot migrator login needs migration authority and uses