docs: publish English public documentation

docs/contracts/workspace-evidence-v3.md

@@ -5,21 +5,58 @@ workspace descriptor. Evidence is optional: a valid v3 descriptor without it rem
 When present, `evidence` is strict: it contains `source` and a defaulted strict `policy`; every
 source variant and the policy reject unknown keys.
 
+```mermaid
+flowchart LR
+    REGISTRY["Workspace registry"] --> DESCRIPTOR["Evidence descriptor"]
+    DESCRIPTOR --> FILESYSTEM["Filesystem adapter"]
+    DESCRIPTOR --> HTTP["HTTP adapter"]
+    DESCRIPTOR --> S3["S3 adapter"]
+    FILESYSTEM --> CURATED["Curated markdown"]
+    HTTP --> CURATED
+    S3 --> CURATED
+    CURATED --> VALIDATE["Validate schema\nand provenance"]
+    VALIDATE --> PREPROCESS["Preprocess pinned\nrevision"]
+    PREPROCESS --> GENERATION["Versioned generation"]
+    GENERATION --> ACTIVE["Active corpus"]
+```
+
 ## Filesystem source
 
 A filesystem source uses the exact URI `<workspace.id>/evidence`. `patterns` is a nonempty list of
-unique, normalized relative POSIX globs. Its defaults are `patterns: ["**/*.md"]` and
+unique, normalized relative POSIX globs. The Evidence-local `schema_version` defaults to `1` for
+compatibility, where an omitted filesystem pattern defaults to `patterns: ["**/*.md"]` and
 `max_bytes: 10485760`.
 
+`evidence.schema_version: 2` declares the source/curated authoring layout. Its omitted filesystem
+pattern defaults to `patterns: ["curated/**/*.md"]`; if declared, the only accepted v2 filesystem
+pattern list is exactly `patterns: ["curated/**/*.md"]`. A v2 descriptor that selects `source/`,
+spans both `source/` and `curated/`, uses a broader curated glob, or selects a non-Markdown file is
+rejected. Explicit safe legacy filesystem patterns remain supported under Evidence version 1. HTTP
+and S3 sources do not use filesystem layout patterns and retain their existing contracts.
+
+The v2 authoring tree is:
+
+```text
+evidence/
+├── source/       # preserved original material
+├── curated/      # reviewed Evidence Units indexed at runtime
+├── manifest.yaml
+└── evaluation.yaml
+```
+
+`source/`, the manifest, the evaluation set, and other support files are materialized for
+traceability but never acquired by v2 runtime preprocessing.
+
 ### Example: filesystem
 
 ```yaml
 evidence:
+  schema_version: 2
   source:
     type: filesystem
     uri: example/evidence
     patterns:
-      - "**/*.md"
+      - "curated/**/*.md"
     max_bytes: 10485760
   policy:
     max_chunk_chars: 4000
@@ -152,8 +189,10 @@ catalog metadata exactly. Every catalog entry must have its descriptor at that s
 catalog-only entries are invalid and reject the complete candidate revision.
 
 Workspace source changes only through curator Git commit/push in a separate authoring clone,
-followed by an installation pull. The API never writes `thoth-workspaces.yaml`,
-`<id>/workspace.yaml`, `<id>/schema/**`, or `<id>/evidence/**`.
+followed by an installation pull. Curator validation occurs before merge; activation and
+preprocessing consume only the merged, pinned commit. The API and runtime never write
+`thoth-workspaces.yaml`, `<id>/workspace.yaml`, `<id>/schema/**`, or `<id>/evidence/**` in the
+authoring repository.
 
 ## Registry revision and phase ownership
 
@@ -164,7 +203,7 @@ followed by an installation pull. The API never writes `thoth-workspaces.yaml`,
 | Repository consumer | ThothII fetches and validates a complete candidate, atomically activates it only on success, and never edits, commits, or pushes repository content. |
 | Runtime secrets | Workspace management returns configured/missing status only; decrypted values exist only for the lifetime of a diagnostic or runtime lease. |
 | P1.1 | Validates the lexical URI `<id>/evidence` and proves the declared filesystem root object is a Git tree at that same commit; it does not recursively inspect nested symlinks. Evidence materialization stays out of scope for P1.1. |
-| P6 | Owns commit-addressed materialization, realpath and recursive containment, nested-symlink checks, and race checks. |
+| P6 | Owns commit-addressed materialization of the complete Evidence tree, realpath and recursive containment, nested-symlink checks, and race checks. |
 
 P1.1 performs no acquisition, extraction, preprocessing/indexing, embeddings, Qdrant writes,
 active-snapshot retention, or GC.
@@ -179,10 +218,3 @@ tht config check -c <path>
 ```
 
 Stop after validation. P2/P6 later owns preprocessing and materialization.
-
-## Acceptance states
-
-These gates are independent and are not implied by this documentation contract.
-
-automated integration: PENDING
-manual acceptance: PENDING

docs/contracts/workspace-preprocessing-cli.md

@@ -2,6 +2,18 @@
 
 `tht` is the only supported host entrypoint for workspace preprocessing.
 
+```mermaid
+flowchart LR
+    OP["Operator"] --> INVOKE["tht workspace preprocess"]
+    INVOKE --> VALIDATE["Validate descriptor\nand paths"]
+    VALIDATE --> SOURCE["Read source and\ncurated workspace"]
+    SOURCE --> NORMALIZE["Normalize and chunk"]
+    NORMALIZE --> INDEX["Update vector and\nBM25 indexes"]
+    INDEX --> VERIFY["Verify collection\nand generation"]
+    VERIFY --> READY["Generation ready"]
+    VALIDATE -->|"invalid"| STOP["Exit with diagnostic"]
+```
+
 ## Invocation
 
 ```text
@@ -58,6 +70,20 @@ tht --installation <absolute>/thothii-installation.yaml workspace vector rebuild
   performs the guarded rebuild; rebuild state is written before deletion and the
   collection is verified after recreation. No prefix matching or global Qdrant
   mutation is performed.
+
+## Additive BM25 for Evidence
+
+- Only `workspace preprocess evidence` (and the Evidence portion of `workspace preprocess run`)
+  may add the named sparse vector `bm25` with Qdrant modifier `idf`.
+- The upgrade uses Qdrant's additive named-vector operation. It preserves the existing unnamed
+  dense vector and never deletes, renames, or rebuilds the shared collection.
+- Session readiness remains read-only with respect to BM25. Schema, Memory, and solved-question
+  records therefore continue to use their existing dense-only points during and after an Evidence
+  upgrade.
+- A missing `bm25` is added and reread before Evidence preprocessing starts. An existing definition
+  other than `modifier: idf` fails as `semantic_index_incompatible` without any collection mutation.
+  If a later Evidence candidate fails, the compatible additive schema remains in place; it does not
+  make the dense-only records unavailable.
 ```
 
 ## Curated FK annotations (P5)
@@ -92,7 +118,13 @@ tht --installation <absolute>/thothii-installation.yaml workspace vector rebuild
   before any bytes are written and no partial root is published.
 - `preprocess evidence` and `preprocess run` operate directly on the materialized root; the
   temporary `evidence_materialization_required` stop is retired (the code remains only for
-  pre-P6 compatibility). HTTP/S3 Evidence is unchanged.
+  pre-P6 compatibility). For `evidence.schema_version: 2`, runtime acquisition receives exactly
+  `curated/**/*.md`; `source/` and support files remain in the materialized tree for traceability.
+  HTTP/S3 Evidence is unchanged.
+- The curator validates Evidence before merge. Preprocessing validates the pinned curated corpus
+  again before it constructs a candidate generation, so an invalid revision is never indexed.
+- The runtime writes only its immutable materialized snapshot and derived index state. It never
+  writes, stages, commits, or pushes the workspace authoring repository.
 - Materialized roots are retained with their commit-addressed snapshot directory and removed only
   when the revision becomes unreferenced.