Codex
82e2c91f42
Publish documentation / publish (push) Successful in 1m27s
Add PostgreSQL-backed memory, editable evidence with source review and activation, and human-approved archive repairs across the harness, API, and UI. Include migrations, deployment support, regression coverage, and validation documentation. Refresh permissions from validated session roles so existing administrator logins can access newly deployed archive management features.
101 lines
6.4 KiB
Markdown
101 lines
6.4 KiB
Markdown
# Evidence E3 — validation
|
|
|
|
Date: 2026-09-09. Explicit source import/refresh and decisions are implemented. X1,
|
|
the integration of deliberate Memory/Evidence corrections into workflow gates, remains next.
|
|
|
|
## Delivered behavior
|
|
|
|
The independent Evidence page now offers **Sources and imports**. An operator copies
|
|
a specialist's draft into `evidence/incoming/`, then explicitly imports/refreshes.
|
|
Original local Markdown and configured HTTP/S3 sources use existing read-only adapters.
|
|
Acquisition retains raw bytes, source identity and versioned normalized documents.
|
|
The existing Pi authoring refiner prepares typed, editable v4 proposals.
|
|
|
|
Unchanged hashes skip refinement. All source acquisitions/refinements must succeed
|
|
before saving a new set of comparisons. Missing sources are recorded as unavailable,
|
|
never interpreted as permission to delete. No runtime lookup, ordinary consolidation
|
|
or preprocessing triggers remote refresh once the local archive is initialized.
|
|
|
|
The administrator sees current and proposed units, scope, content, excerpts, review
|
|
items and explicit retirement IDs. **Keep local Evidence** records the retained wording
|
|
as a manual declaration with original lineage. **Use proposed Evidence** adopts the
|
|
proposal and its source version. Both save and activate through the existing archive
|
|
and corpus pipeline; review items block adoption. Comparisons use optimistic checks
|
|
on affected file bytes. Interrupted decisions have a durable replay journal and retry
|
|
without reacquisition, while intervening external edits are preserved and reported.
|
|
|
|
Deleted IDs remain reserved. New model-generated identities from sources with curated
|
|
deletions are also conservatively suppressed; surviving IDs can still receive reviewed
|
|
updates. Deliberate new knowledge can be authored as a manual file. This mechanical
|
|
protection does not depend on the model detecting semantic duplication or contradictions.
|
|
|
|
Installed commands are `workspace evidence refresh` and `workspace evidence decide`,
|
|
alongside E2 consolidation. Decision envelopes carry a source identity, comparison
|
|
revision and keep/replace choice. Extra URLs, arbitrary paths, forged actors and unknown
|
|
fields are rejected at the public API/CLI boundary. HTTP requests bind the authenticated
|
|
curator. Source operations do not mutate Catalog readiness or run DWH/schema stages.
|
|
The Python source worker is internal; the workflow CLI's visible surface is preserved.
|
|
|
|
## Checks
|
|
|
|
- Complete backend suite: **1,359 passed**, 40 skipped. Complete frontend suite:
|
|
**641 passed**. Both TypeScript checks passed; native Go CLI/workspace tests passed.
|
|
- Harness regression run: **1,256 passed**, one skipped and five deselected, with
|
|
portable-path tests run separately without `THT_HOME`. All **24 focused import,
|
|
CLI-surface and portable-path checks** passed. These
|
|
cover import, unchanged refresh, access failure, missing source, manual correction,
|
|
keep/replace, deletion suppression, stale comparisons, failure/retry and interrupted
|
|
journal writes. Ruff passed on the changed Python implementation and tests.
|
|
- The real-Qdrant test traverses the actual harness source CLI with deterministic
|
|
refinement/embedding boundaries: import is absent from recall before a decision,
|
|
accepted content becomes searchable, refreshed proposals preserve active manual
|
|
corrections, replacement removes the former text, deletion remains absent after
|
|
another refresh, and unrelated Schema/Memory canaries survive.
|
|
- Source contract fixtures cover controlled HTTP and S3 identities, exact acquired
|
|
bytes, and acquisition call counts. Existing adapter tests retain transport/egress
|
|
coverage. The test does not claim to exercise a live S3 account.
|
|
- React interaction tests verify explicit refresh, comparison content, exact decisions,
|
|
saved-decision retry and failure feedback. Route tests cover admin authorization,
|
|
workspace isolation, strict inputs and principal attribution. Service tests verify
|
|
the trusted config file descriptor and absence of Catalog mutation.
|
|
|
|
## Local preview
|
|
|
|
Core/frontend were rebuilt for the existing `thothii-18998cca7b0a` stack. Its persistent
|
|
archive and data volumes are retained. The native `/usr/local/bin/tht` was updated;
|
|
the previous executable is at `/private/tmp/tht-before-e3`.
|
|
|
|
The installed refresh command ran against `psd-clinical` successfully: **35 unchanged
|
|
sources, zero changed, zero pending comparisons**. All 35 source hashes matched their
|
|
existing units, so this probe required no refinement and changed no active Evidence.
|
|
Source registry metadata was saved locally; no Git commit or push was performed.
|
|
|
|
A separate synthetic draft was passed to the configured Pi/model inside core. It
|
|
produced one domain proposal with one review item, which was not activated. The probe
|
|
exposed a deployment issue: Python wheel modules and Pi skills live in different
|
|
directories. The refiner now resolves resources through `THT_HARNESS_DIR`, with the
|
|
source-tree location as its development fallback; a regression test covers this layout.
|
|
|
|
The corrected installed worker was then exercised end to end in a temporary workspace
|
|
inside core, using the real configured Pi/model and a synthetic `incoming/orders.md`.
|
|
It returned success, one changed source, one comparison and one proposal with a review
|
|
item. The active snapshot remained absent. The temporary directory was removed on exit;
|
|
the probe did not open the DWH or activate an index. Strict docs build and
|
|
`git diff --check` also passed.
|
|
|
|
The in-app browser still showed the login page with the prior credential error.
|
|
Authenticated visual verification remains manual; no credentials were reset or retried.
|
|
|
|
## Operational instructions and boundaries
|
|
|
|
See [Import drafts and refresh sources](../contracts/curated-evidence-v4.md#import-drafts-and-refresh-sources)
|
|
for commands, local paths, review, retry and backup/Git requirements. Preserve the
|
|
complete Evidence tree, including source comparisons, journals and acquired versions.
|
|
Local activation and transfer to the remote Git repository remain separate operator steps.
|
|
|
|
No web content editor, automatic Git, background watcher, new job queue, general
|
|
retrieval benchmark or automatic source merge was introduced. Review/refinement is
|
|
sequential and bounded by per-source limits plus 200 documents/100 MiB per refresh.
|
|
New changed-source decisions and failure scenarios use isolated test data; live PSD
|
|
curated content was kept unchanged. X1 is not included in this increment.
|