This commit is contained in:
@@ -2,6 +2,18 @@
|
||||
|
||||
`tht` is the only supported host entrypoint for workspace preprocessing.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
OP["Operator"] --> INVOKE["tht workspace preprocess"]
|
||||
INVOKE --> VALIDATE["Validate descriptor\nand paths"]
|
||||
VALIDATE --> SOURCE["Read source and\ncurated workspace"]
|
||||
SOURCE --> NORMALIZE["Normalize and chunk"]
|
||||
NORMALIZE --> INDEX["Update vector and\nBM25 indexes"]
|
||||
INDEX --> VERIFY["Verify collection\nand generation"]
|
||||
VERIFY --> READY["Generation ready"]
|
||||
VALIDATE -->|"invalid"| STOP["Exit with diagnostic"]
|
||||
```
|
||||
|
||||
## Invocation
|
||||
|
||||
```text
|
||||
@@ -58,6 +70,20 @@ tht --installation <absolute>/thothii-installation.yaml workspace vector rebuild
|
||||
performs the guarded rebuild; rebuild state is written before deletion and the
|
||||
collection is verified after recreation. No prefix matching or global Qdrant
|
||||
mutation is performed.
|
||||
|
||||
## Additive BM25 for Evidence
|
||||
|
||||
- Only `workspace preprocess evidence` (and the Evidence portion of `workspace preprocess run`)
|
||||
may add the named sparse vector `bm25` with Qdrant modifier `idf`.
|
||||
- The upgrade uses Qdrant's additive named-vector operation. It preserves the existing unnamed
|
||||
dense vector and never deletes, renames, or rebuilds the shared collection.
|
||||
- Session readiness remains read-only with respect to BM25. Schema, Memory, and solved-question
|
||||
records therefore continue to use their existing dense-only points during and after an Evidence
|
||||
upgrade.
|
||||
- A missing `bm25` is added and reread before Evidence preprocessing starts. An existing definition
|
||||
other than `modifier: idf` fails as `semantic_index_incompatible` without any collection mutation.
|
||||
If a later Evidence candidate fails, the compatible additive schema remains in place; it does not
|
||||
make the dense-only records unavailable.
|
||||
```
|
||||
|
||||
## Curated FK annotations (P5)
|
||||
@@ -92,7 +118,13 @@ tht --installation <absolute>/thothii-installation.yaml workspace vector rebuild
|
||||
before any bytes are written and no partial root is published.
|
||||
- `preprocess evidence` and `preprocess run` operate directly on the materialized root; the
|
||||
temporary `evidence_materialization_required` stop is retired (the code remains only for
|
||||
pre-P6 compatibility). HTTP/S3 Evidence is unchanged.
|
||||
pre-P6 compatibility). For `evidence.schema_version: 2`, runtime acquisition receives exactly
|
||||
`curated/**/*.md`; `source/` and support files remain in the materialized tree for traceability.
|
||||
HTTP/S3 Evidence is unchanged.
|
||||
- The curator validates Evidence before merge. Preprocessing validates the pinned curated corpus
|
||||
again before it constructs a candidate generation, so an invalid revision is never indexed.
|
||||
- The runtime writes only its immutable materialized snapshot and derived index state. It never
|
||||
writes, stages, commits, or pushes the workspace authoring repository.
|
||||
- Materialized roots are retained with their commit-addressed snapshot directory and removed only
|
||||
when the revision becomes unreferenced.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user