Files
ThothII/docs/adr/0019-author-evidence-in-app-with-automatic-activation.md
T
Codex 82e2c91f42
Publish documentation / publish (push) Successful in 1m27s
feat: implement memory and evidence administration with guided repairs
Add PostgreSQL-backed memory, editable evidence with source review and activation, and human-approved archive repairs across the harness, API, and UI. Include migrations, deployment support, regression coverage, and validation documentation.

Refresh permissions from validated session roles so existing administrator logins can access newly deployed archive management features.
2026-09-10 10:31:34 +02:00

153 lines
9.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
status: accepted
date: 2026-09-08
---
# Maintain local Evidence with external editors and manual consolidation
Evidence management must provide complete access to registered Evidence and a
maintenance path, as accepted in Q11–Q15 and revised during simplification. The owner subsequently
clarified that a context specialist writes drafts independently of the installation
and without PostgreSQL access; the system refines and stores them locally for use
and maintenance. Implementation is pending.
The storage mechanics of Q11 were revisited in the
[simplification review](../plans/2026-09-08-memory-evidence-simplification-review.md).
The original choice combined the workspace repository as Evidence authority with
application-managed Git writes. The owner accepted the separation of external draft
files/repositories from a durable local canonical file archive and removes
commit/push from normal CRUD. The management surface must expose discoverable,
editable Markdown for domain specialists, without JSONL editing. The owner chose
external editors for release 0: their preferred Mac/PC editor or vim/nano on the
server. No web editor or content form is required. The page keeps browsing,
filtering, and detail, with actual host file paths and maintenance instructions.
The owner also requested manual consolidation followed by manual Git diff, commit,
and push, relying on operator discipline rather than automation. The owner accepted
the remaining simplifications and requested explicit clarification of the core
format change and the simple terminal-based Git check. The original application Git writer
is not the final implementation prescription. A suggestion
to move Evidence authority into PostgreSQL was withdrawn after the clarification.
Memory remains a distinct module with its own PostgreSQL authority under
[ADR 0018](0018-use-postgres-for-memory-and-qdrant-for-retrieval.md).
## Manual consolidation and Git follow-up
The external-editor decision replaces the form's single Save with saving files and
running one consolidation command. The command checks expected structure, required
fields, types, identifiers, references, and provenance. It reports the affected file
and the necessary correction. Invalid input stops activation; valid input updates
derived metadata, the corpus, and the Evidence index. New and deleted files use the
same path. An inaccessible archive must not be interpreted as deleted Evidence.
Structural validation does not prove semantic correctness or rerun model refinement
to rewrite manually edited rules.
The core reads only successfully consolidated active content, never the editable
working files directly. Candidate validation and activation must preserve the last
valid corpus on failure, or report unavailability if its integrity cannot be
guaranteed. Unconsolidated edits must not mix new text with old retrieval data.
Consolidation reports full local success only once the updated content is available
to the core. If persistence succeeds but activation fails, the command reports the
partial outcome and can be rerun without duplicating units or losing edits.
The operation does not imply atomicity across
authoritative storage and Qdrant; their failure and recovery behavior requires
explicit implementation. An interrupted operation must not leave a partial corpus
advertised as ready for subsequent use.
The maintained archive is a persistent Git working tree containing the local
Evidence; it may reuse the workspace repository. Draft sources and curated units
remain distinct even when in the same repository. Setup and the page identify the
repository, working tree, actual host paths, branch, and configured remote. A
rebuildable runtime snapshot is not the editing location.
Before consolidation, the operator checks Git status, including inspection of new
untracked files. Normal terminal Git diff commands are available for line-level
inspection when needed; no custom viewer or mandatory double review is required.
After consolidation, the operator stages the Evidence and required metadata changes,
commits, and pushes using normal Git commands. No watcher, automatic Git writes, retrying push, pull, merge,
or dedicated web execution control is required. Missing credentials, unconfigured
upstreams, and Git conflicts are handled by the operator.
Consolidation activates local changes before commit/push. If the operator omits the
Git steps or push fails, the local update remains effective while transfer to the
remote is incomplete. Pushing does not automatically update other installations.
The system relies on operator discipline to complete the sequence; it does not add
a separate editorial publication state machine or require a second reviewer.
Approved conflict repairs from the core still call the same persistence and
activation service directly. They do not require an external editor or manual
consolidation before the session can use its own approved correction. Their local
file changes are included in the operator's subsequent Git maintenance.
## Administration without concurrent core work
The owner's simplification instruction replaces the earlier Q12 requirement to
refresh open sessions after administrative edits. Core activity can be assumed
absent during administration, or its overlap can be ignored. No dedicated live
update, session notification, restart, maintenance mode, or reader coordination
is required. Completed administrative changes apply to subsequent work.
Deliberate writes from the workflow itself still exist: approved conflict repairs
and the final Memory summary use the same persistence services and handle their
outcome before proceeding. In particular, a session must be able to use its own
approved Evidence correction. This does not require updating all other sessions
or rewriting previously approved decisions, artifacts, or SQL.
## Manual corrections survive source updates
When an updated source contradicts an administrator's correction, the saved manual
Evidence remains active. Evidence management shows the conflicting content for an
explicit decision and subsequent save. The source's newer content does not
automatically override the correction. The owner accepts that the manual rule can
remain in use until that comparison is resolved.
Deleted Evidence must not silently reappear after preparation or reindexing. The
authoring implementation must preserve both manual corrections and suppression of
deleted units through source refreshes. The current protection for uncommitted Git
changes is insufficient: it does not preserve already committed manual corrections
against regeneration, and retirement currently removes the unit without recording
suppression for later preparation.
## Manual creation and explicit source refresh
Administrators can create Evidence without providing an external document. In R0,
they add Markdown using the documented example and consolidate it. The application
records a manual declaration as the managed source of the current statement.
Explicit consolidation approves that declaration; it does not
claim independent documentary verification.
A correction that changes a unit's meaning uses the same explicit manual origin.
The original document remains linked for provenance and source-change detection,
but its excerpt is not presented as support for a rule it does not contain. The
canonical contract must distinguish the current supporting source from the original
document rather than silently retaining outdated support metadata.
External sources are reacquired only when an administrator requests a source
refresh. Ordinary saves and session lookups use the acquired local content; they
do not poll sources or fetch their current versions. A remote change is therefore
detected at the next requested refresh, when the manual-precedence rule applies.
The revised storage recommendation reads the specialist's repository as an input;
ordinary local edits do not write back to it or refresh its content automatically.
## Implementation consequences
The existing [Evidence lifecycle](../evidence.md) and
[Workspace Evidence v3 contract](../contracts/workspace-evidence-v3.md) require
explicit evolution for editable local Markdown, manual consolidation, and manual
precedence. Source provenance and coherent canonical metadata remain requirements;
manual changes must not be presented as statements supported by an unrelated source
excerpt. Under the accepted local-file direction, curated files are primary
installation data preserved by Clear and backed up separately from rebuildable
indexes. Visible Markdown content must become authoritative for editing; operators
must not maintain hidden duplicate text, hashes, or manifest entries themselves.
This is a core Evidence contract change: parser, renderer, authoring, validation,
normalization, and their integration with indexing and recall must be adapted
together. E1 includes a new Curated unit format version, conversion of existing
Evidence, reindexing, and end-to-end verification that visible edits reach core
consumption. Preserve the internal typed model where possible; the change does
not require redesigning the NL-to-SQL workflow phases.
Local edits do not automatically change external drafts or another installation;
Git versioning and transfer occur through the explicit manual follow-up.
The [Evidence management project](../plans/2026-09-08-evidence-management.md)
records the implementation sequence and acceptance checks.