feat: implement memory and evidence administration with guided repairs

Add PostgreSQL-backed memory, editable evidence with source review and activation, and human-approved archive repairs across the harness, API, and UI. Include migrations, deployment support, regression coverage, and validation documentation.

Refresh permissions from validated session roles so existing administrator logins can access newly deployed archive management features.

PROJECT_STATE.md

@@ -1,6 +1,6 @@
 # ThothII — Project State
 
-Last updated: 2026-09-06.
+Last updated: 2026-09-10.
 
 This file is the short operational snapshot. Stable commands and the architecture mental model
 live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
@@ -26,6 +26,165 @@ metadata catalog for administrative database configuration. The frontend renders
 and keeps the live transcript in memory. See
 `docs/architecture/components.md` for the detailed component and data-flow map.
 
+## Memory M1–M3, Evidence E1–E3 and joint workflow repair X1 implemented
+
+Browser authorization now derives permissions from validated session roles using the current
+catalog. This fixes Memory/Evidence navigation remaining disabled for administrators whose
+remembered login predates those permissions; refreshing the page loads the updated permissions.
+
+The owner requested two independent administration projects: Memory management and Evidence
+management. Their navigation entries must sit immediately after Database management, as peers;
+neither page belongs to Database management. Both require complete browsing, filtering, and CRUD
+without an active core session. Shared requirements and the two project briefs are linked from
+`docs/plans/2026-09-08-memory-evidence-administration.md`. Memory administration is implemented;
+Evidence administration is implemented through external file editing and explicit consolidation.
+Evidence editing requires an explicit evolution of the current authoring/publication contract.
+The M1 specification is at `docs/plans/2026-09-08-memory-m1-spec.md` and published as
+[M1 — Archivio autorevole e amministrazione delle Memory Card](https://git.tylconsulting.it/mptyl/ThothII/issues/27)
+with the `ready-for-agent` and `enhancement` labels.
+It covers the authoritative store, administrative CRUD, projection recovery, and current
+Memory/exemplar producer integration. Its accepted test boundaries are the public harness
+service, Fastify APIs, and the AppShell page, joined by a focused real-stack browser path.
+The owner confirmed those boundaries and authorized publication on 2026-09-08.
+M1 is implemented locally: PostgreSQL authority in `thoth_memory`, versioned installation
+migrations, admin CRUD for all four families, structured dependencies and links, explicit
+projection recovery, verified recall and current workflow producers. The page requires no
+active session or DWH binding. The existing `catalog-migrate` preparation service now runs
+Memory migrations as well. Existing installations need that preparation before using M1;
+the initial implementation did not deploy or migrate the owner's stacks.
+The integrated browser check passed with real authentication, Fastify, ThtRunner, harness,
+PostgreSQL and Qdrant; embeddings were deterministic and unrelated Pi/session activity used
+test fixtures. See `docs/plans/2026-09-08-memory-m1-validation.md` for results and commands.
+M2 is implemented locally: Memory dense/BM25 fusion, physical and business scope filters,
+bounded outgoing-link expansion and joint ranking, all resolved against current PostgreSQL
+authority. Migration `002_hybrid_projection.sql` makes old dense projections pending until
+explicit retry/rebuild; Reference remains separate. The real retrieval check uses the configured
+`qwen3-embedding:0.6b` model, a separate Ollama process with a read-only model-volume mount,
+and isolated PostgreSQL/Qdrant resources. See `docs/plans/2026-09-09-memory-m2-validation.md`.
+M3 is implemented: editable F8 summary grounded in effective approved decisions, explicit
+updates with concurrent-edit protection, selected-card/link transactions and durable review
+receipts. Finalization no longer saves exemplars implicitly. SQL rules and explained errors
+are consulted in the existing F4/F6/F7 gates. Successful Catalog physical synchronization
+performs dependency cleanup, preserving the original removals for recovery across restarts.
+Migration `003_review_receipts.sql` is required. Validation includes a real GLM 5.3 generation
+case against synthetic PostgreSQL data; see `docs/plans/2026-09-09-memory-m3-validation.md`.
+Evidence administration and the joint X1 conflict-repair increment are now implemented.
+The same local preview was subsequently rebuilt with M3 and migration 003 applied.
+Core and frontend now include the final Memory review and physical dependency cleanup.
+On 2026-09-09, at the owner's request, the local PSD Docker installation
+`thothii-18998cca7b0a` was updated from this worktree. Core/frontend images were rebuilt,
+Catalog and Memory migrations completed, and all five services became healthy. The UI is at
+`http://127.0.0.1:8080`, using the existing local authentication and persistent volumes.
+The `psd-clinical` authoritative Memory archive is initially empty (no legacy import).
+The existing installation configuration remains in `/Users/mp/projects/ThothII/deploy/psd/`;
+`/private/tmp/thothii-memory-preview.sh` invokes its Compose files with a final build-context
+and migration-command override from this worktree. A future build from the main checkout
+will use that checkout's code, so retain the worktree override until the changes are integrated.
+Memory revision history and compatibility with existing development sessions are not requirements.
+The agreed Memory scope includes reusable domain clarifications, SQL construction rules, solved
+questions, and explained, approved mistakes to avoid. This extends the current runtime's
+`concept_clarified`-only reusable Memory contract. The owner also approved an editable final summary
+for proposed additions/updates and Memory consumption in the relevant existing review gates.
+Further agreed behavior includes persistent corrections for Memory/Evidence conflicts through
+explicit choices, deletion of Memory with invalid dependencies after successful physical schema
+synchronization, and bounded functional/regression tests instead of a general quality benchmark.
+In-house Memory evolution is approved, including hybrid Qdrant retrieval and explicit card links
+traversed in core, without a dedicated graph database. Both capabilities belong to the current
+scope. PostgreSQL is the agreed authority for Memory Cards, links, and schema dependencies;
+Qdrant is a rebuildable index. Links are reviewed with cards and editable in Administration;
+deleting a card removes its incident links while preserving the other cards. The accepted,
+implemented architecture is recorded in
+`docs/adr/0018-use-postgres-for-memory-and-qdrant-for-retrieval.md`.
+For Evidence administration, Q12 rejects a separate editorial publication workflow.
+The final R0 direction uses external editors and a manual consolidation command that
+validates structure, reports required corrections, updates derived metadata, and
+activates the local Evidence index. The operator then checks the diff and runs Git
+commit/push manually. No watcher, automated Git, or web editor is required.
+The owner's later simplification
+instruction removes the requirement to support administrative edits while core work
+is in progress. Deliberate corrections made by the workflow itself remain supported.
+The context specialist writes Evidence drafts independently of the installation and
+without PostgreSQL access; the system refines them and stores them locally for use
+and maintenance. The revised Q11 recommendation separates external drafts from a
+durable local canonical file archive, removing automatic commit/push from CRUD.
+The owner has now accepted local files and requires discoverable, editable Markdown
+for nontechnical domain specialists, with no JSONL management surface. Editors on
+Mac/PC or vim/nano on the server are the chosen R0 editing surface. Evidence
+management keeps browsing/filtering/detail and clearly identifies the persistent
+working tree, each Markdown file's actual host path, and the manual commands.
+E1 implements editable Curated Evidence v4, deterministic legacy conversion, persistent
+local files and a consolidation API with immutable candidates, manual provenance,
+deletion records and recoverable activation. E2 connects its installed command, runtime source
+selection and administration page. The operator completes Git steps manually.
+All 35 PSD units were converted on an isolated copy with identical IDs and typed content.
+Tests exercise visible edits through normalization, indexing and recall with real Qdrant;
+see `docs/plans/2026-09-09-evidence-e1-validation.md` and
+`docs/contracts/curated-evidence-v4.md`. E2 is now running on the local Docker preview:
+all 35 PSD units were converted and indexed through the installed command. The editable
+host archive is `/Users/mp/projects/ThothII/deploy/psd/evidence-registry/repo/psd-clinical/evidence`.
+The original registry-volume checkout and original author repository were retained.
+The installation descriptor now includes `workspace-bindings.yaml` and `evidence-host.yaml`
+so native maintenance uses the same volumes and checkout as the preview. Descriptor backup:
+`/private/tmp/thothii-installation-before-e2.yaml`; previous native binary: `/private/tmp/tht-before-e2`.
+The launcher remains `bash /private/tmp/thothii-memory-preview.sh`; its worktree build override
+is still required until integration. Runtime lease filenames now include rendered bytes so
+upgrading the Evidence renderer does not collide with old immutable configs; Catalog
+input fingerprints and readiness are unchanged. See `docs/plans/2026-09-09-evidence-e2-validation.md`.
+E3 adds explicit source acquisition/refinement and durable comparisons in Evidence management.
+Local drafts go in `evidence/incoming/`; original local documents and configured HTTP/S3
+sources are reacquired only by **Import or refresh sources** or installed
+`tht workspace evidence refresh --workspace <id>`. Keep/replace decisions, including the
+native `workspace evidence decide` command, activate through the existing archive/index
+path and have durable retry state. Acquired source versions and remote provenance stay
+local; old documentary lineage can coexist with current manual declarations.
+The installed preview's 35 PSD sources were verified unchanged with no new proposals.
+The previous native CLI is backed up at `/private/tmp/tht-before-e3`.
+See `docs/plans/2026-09-09-evidence-e3-validation.md` for tests and validation limits.
+X1 adds `reviewer_archive_repair`: closed alternatives with complete before/after content,
+explicit rejection/reformulation, admin-only application, durable session receipts and
+retry of saved but inactive corrections. The coordinator uses the canonical Memory and
+Evidence services; phase approval remains separate. Migration `004_archive_repairs.sql`
+is required. See `docs/contracts/archive-repair.md` for authorization and recovery limits.
+X1 validation is recorded in `docs/plans/2026-09-09-archive-repair-x1-validation.md`:
+both archives were corrected and retrieved through the actual CLI with real PostgreSQL
+and Qdrant; desktop/mobile widget behavior and permission failures were verified separately.
+The local preview images include X1 and migration 004 is applied. Follow-up technical
+acceptance passed with configured GLM 5.3 generating closed conflict alternatives and
+the chosen Memory correction persisted and retrieved. An authenticated browser test
+also verified both administration pages, navigation order, Evidence filtering, workspace
+404s, desktop/mobile layout, and Evidence survival after Memory deletion. All approved
+technical increments/checks are complete. A real PSD domain-conflict session remains
+the reviewer's semantic acceptance check; automated cases did not modify PSD knowledge.
+The joint browser inspection also fixed mobile archive navigation: below 768px,
+Memory and Evidence keep the full content width and open navigation in the shared
+accessible dialog. Desktop retains its sidebar; the local frontend image includes this fix.
+The owner accepted the remaining simplifications and requested explicit clarification
+of the core format change and the simple terminal-based Git check. E1 must adapt
+the Evidence parser, renderer, authoring, validation, and normalization; convert
+existing files and reindex; and verify that visible edits reach core consumption.
+Preserve the internal typed model where possible. This precedes the administrative
+page and is not merely a presentation change. Human inspection uses normal Git
+status and optional line-level diff commands; no custom diff viewer or mandatory
+double review is required.
+The suggestion of making PostgreSQL the Evidence authority was withdrawn after this
+clarification; it was never implemented or accepted as a replacement for Q11.
+Q13 keeps manual corrections active when updated sources contradict them, until an
+administrator resolves the comparison; deleted Evidence must not be regenerated
+automatically. Q14 allows direct manual creation and records a manual declaration
+as the current source, preserving any original document as distinct provenance.
+Q15 refreshes external sources only on explicit administrator request; normal saves
+and lookups do not reacquire them. These decisions, now implemented through E1–E3, are recorded in
+`docs/adr/0019-author-evidence-in-app-with-automatic-activation.md`.
+The decision-by-decision review is recorded in
+`docs/plans/2026-09-08-memory-evidence-simplification-review.md`; it distinguishes the
+new constraints from the revised technical recommendations. It also specifies a
+sequential save with minimal durable retry state and direct Memory cleanup after
+successful schema synchronization, without new queues or event infrastructure.
+The delivery order remains Memory, Evidence, and persistent conflict repair between
+both modules. Local curated Evidence must survive preprocessing Clear and be backed
+up as primary data; the Qdrant projection remains rebuildable.
+No runtime gate change or administrative page is implemented yet.
+
 ## Evidence restructuring — accepted
 
 The evidence restructuring and PSD migration completed real acceptance on 2026-08-25.