feat: complete catalog-driven preprocessing
Publish documentation / publish (push) Successful in 2m12s

This commit is contained in:
Codex
2026-09-06 17:49:35 +02:00
parent 8707ae1d46
commit cffa60772e
141 changed files with 5898 additions and 3015 deletions
+27 -13
View File
@@ -1,6 +1,6 @@
# ThothII — Project State
Last updated: 2026-09-04.
Last updated: 2026-09-06.
This file is the short operational snapshot. Stable commands and the architecture mental model
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
@@ -51,21 +51,36 @@ The canonical authoring, validation, publication, materialization, and preproces
documented in `docs/evidence.md`. The governing contracts are
`docs/contracts/workspace-evidence-v3.md` and
`docs/contracts/workspace-preprocessing-cli.md`.
The incremental server procedure for the `260906-preprocessing-complete` release is
`docs/operations/server-handoff-260906-preprocessing-complete.md`.
## Workspace preprocessing and configuration
The native host CLI `tht` is the operator surface. Workspace preprocessing runs through:
```sh
tht --installation /absolute/path/thothii-installation.yaml workspace preprocess evidence
tht --installation /absolute/path/thothii-installation.yaml workspace preprocess dwh
tht --installation /absolute/path/thothii-installation.yaml \
workspace preprocess run --workspace <workspace-id>
```
These commands use the profile-gated `workspace-maintenance` service. The former standalone
preprocessing Compose fixtures are retired.
This complete one-shot command uses the profile-gated `workspace-maintenance` service. Partial DWH,
schema, Evidence, and vector mutation commands are retired.
Workspace descriptors use schema v4 and contain only database, Evidence, diagnostics, and binding
concerns; model, provider, embedding, and vector-store configuration is installation-owned. For
The right Administration sidebar invokes that same operation for the selected workspace. It shows
only current readiness or the latest bounded failure diagnostic; there is no preprocessing history.
Known non-ready state disables **New session**, while backend admission remains authoritative.
The same control exposes an inline-confirmed **Clear** action to remove replaceable reference
vectors, LSH, corpus, and checkpoints while preserving the separate Memory collection. The host CLI
equivalent is `workspace preprocess clear`.
Each workspace now uses `<workspace>-reference` for Schema, relationships, and Evidence and
`<workspace>-memory` for `memory` and `solved_question`. Clear and preprocessing own only the former.
LSH ownership additionally binds the Catalog database ID and Metadata Content Revision, so derived
values cannot be reused across database identities or Catalog revisions.
Workspace descriptors use schema v4 and contain only workspace identity and optional Evidence.
PostgreSQL Metadata Catalog owns database identity, binding, schema, descriptions, sensitivity, and
relationships; model, provider, embedding, and vector-store configuration is installation-owned. For
PSD, workspace content and runtime roots point to the
separate uncommitted repository `/Users/mp/projects/tht-workspace-psd`. Secrets remain outside
Git and are supplied only through installation-local protected files.
@@ -166,12 +181,11 @@ SSH is not yet enabled for NL→SQL session runtime.
The catalog runs in the internal `catalog-db` PostgreSQL service. Kysely migrations are an explicit
one-shot `catalog-migrate` operation; `scripts/run-stack.sh` runs it before local startup. Runtime
sessions now consume the Catalog's active effective relationship map through an immutable JSON
snapshot tied to the runtime-config lease. The harness uses that snapshot as its exclusive
relationship source while retaining Git-pinned annotations for descriptive metadata; legacy
runtimes without a snapshot keep the previous merge behavior. The accepted design is recorded in
`docs/plans/2026-08-26-metadata-catalog-from-thothai.md`, the snapshot contract under
`docs/contracts/`, and ADRs 0001–0012.
sessions consume an immutable Catalog JSON snapshot tied to the runtime-config lease. It contains
the tables, columns, effective descriptions, sensitivity flags, and active relationships used by
the harness; PostgreSQL is the exclusive runtime authority for database metadata. Authored
workspace YAML remains limited to workspace identity and optional Evidence configuration. The
accepted design is recorded in ADR 0016 and the contracts under `docs/contracts/`.
Semantic aliases, value descriptions, synonyms, and concepts remain deferred to their dedicated
slices.