feat: complete catalog-driven preprocessing

docs/architecture/overview.md

@@ -48,8 +48,8 @@ A session is a directory under `sessions/` (the workspace defines the path): `se
 - `PiProcessManager` runs one Pi child process per session and bridges its RPC stream.
 - `SessionBridge` maps Pi RPC events to client events (`ui_request` / `text_delta` / `info`).
 - `SseHub` distributes these events to the browser over SSE.
-- `CatalogService` joins authoritative YAML workspace identities with installation-local database
-  configurations stored in PostgreSQL through Kysely.
+- `CatalogService` joins authoritative YAML workspace identities and Evidence settings with the
+  sole database authority: installation-local PostgreSQL Metadata Catalog records.
 - `CatalogTableService` reconciles persisted Catalog Tables with a successful external schema scan;
   `ConcreteCatalogTableIntrospector` isolates direct PostgreSQL, typed REST, and SSH-tunnel access.
 - `DescriptionGenerationWorker` admits one installation-wide run and processes its table or column
@@ -60,8 +60,9 @@ A session is a directory under `sessions/` (the workspace defines the path): `se
 
 Application settings keep only workspace and thinking preferences in `backend/data/settings.json`;
 provider/model defaults come from the generated Installation Model Catalog. Session state remains in
-harness phase documents. PostgreSQL stores only the administrative database catalog, bindings, observed tables,
-curated and generated descriptions, description-generation runs, and sanitized run events.
+harness phase documents. PostgreSQL stores the administrative database catalog, bindings, observed tables,
+curated and generated descriptions, description-generation runs, sanitized run events, and the
+authoritative preprocessing state/revisions.
 Connector secrets remain write-only in the encrypted workspace secret store; model credentials
 remain in the protected installation secret bundle. Catalog SSH support is limited to connection
 tests, table synchronization, and bounded description-generation sampling; it does not change the
@@ -86,18 +87,23 @@ only `curated/**/*.md` from the immutable revision root to preprocessing. Source
 evaluation data remain available for traceability. The runtime never modifies, stages, commits, or
 publishes the authoring repository.
 
-Before indexing, the curated corpus from the pinned revision is validated. The shared Qdrant
-collection keeps the unnamed dense vector used by Schema and Memory. Evidence preprocessing may
-add only the sparse `bm25` vector with `idf`, without deleting, renaming, or recreating the collection.
-`workspace preprocess evidence` and the Evidence part of `workspace preprocess run` are the only public
-operations that perform this upgrade.
+Before indexing, the curated corpus from the pinned revision is validated. Each workspace has two
+physical Qdrant collections with different lifecycles. `reference` contains Schema, relationships,
+and Evidence and may be replaced or cleared by preprocessing; `memory` contains `memory` and
+`solved_question` records and is never preprocessing output. Only the `reference` collection has the
+sparse `bm25` vector with `idf`, and only the Evidence stage writes sparse values.
+
+The Administration control can clear the replaceable reference collection, LSH, corpus, and derived
+checkpoints. The operation preserves the memory collection and makes preprocessing required before
+the core can admit a new session.
 
 ## Recurring points of attention
 
 - `tht -c`/`--config` is a **per-command** option. It must follow the subcommand, never precede it (`ThtRunner.buildArgv` enforces this).
 - `--json` output must be plain JSON on stdout. It is a machine-readable contract.
 - UI strings are in English. Document *content* stays in the workspace language because it is the actual data; only chrome and labels are in English.
-- Each workspace defines its DWH target and working directories. Secrets remain in protected installation files, not in the workspace repository.
+- Each workspace defines identity and optional Evidence only. The PostgreSQL Metadata Catalog defines
+  its DWH target and binding; secrets remain in the protected workspace secret store.
 - Settings are global (`backend/data/settings.json`: workspace/thinking); provider/model choices are
   canonical catalog references selected per session.
 - **Resume**: a resumable session returns to its last incomplete phase. The backend rejects resume with 409 when `finalized` or `archived`; `PiProcessManager.spawnFor` must send `/riprendi-sessione <id>` for resume and `/nuova-domanda` for a new session. The wrong prompt silently turns a resume into a new question.