test(workflow): complete observable baseline (#21)

This commit is contained in:
2026-08-24 00:40:11 +02:00
parent dc9726cb35
commit d15bb59c3d
4 changed files with 626 additions and 173 deletions
+41 -14
View File
@@ -12,20 +12,26 @@ baseline green without weakening its assertions.
### Pi gate
Run `npm test` from the harness package.
From `harness/`, run `npm test` with the repository's supported Node 24 runtime.
The gate suite fixes:
- the registered Pi tool names and their required and optional parameters;
- the exact workflow definition and injected session skill bytes;
- the complete registered Pi tool schemas, including nested types and enum-like constraints;
- the semantic workflow definition and the exact injected session skill bytes;
- widget descriptors and reviewer response semantics;
- F1 clarification and explicitly accepted open ambiguity;
- F2 Memory applied, deselected, and absent;
- F3 rewritten question and assumptions, including mutation failure ordering;
- F4 Evidence acceptance and rejection;
- F4 Evidence used, accepted, rejected, and legacy-without-corpus projections;
- F8 Memory promotion accepted, declined, and absent, including mutation failure ordering;
- resume reconstruction for the touched F1, F2, F3, F4, and F8 states;
- artifact payload compatibility, anti-bypass behavior, and final phase closing.
The baseline intentionally checks widget structure and domain content without freezing the
pre-existing Italian chrome emitted by the gate. Repository policy requires UI chrome and labels
to migrate to English in their owning workstream; this contract must not turn that mismatch into a
new compatibility requirement.
### Harness CLI and persistence
Run the default pytest suite from the harness package. The suite fixes:
@@ -42,7 +48,18 @@ working local Docker daemon and remain part of the default suite when Docker is
### Backend bridge
Run the backend test suite followed by TypeScript typechecking. The suite fixes:
From `backend/`, the passing automated baseline is:
```sh
npx vitest run test/tht-runner.test.ts test/pi-process-manager.test.ts \
test/session-bridge.test.ts test/sse-hub.test.ts test/sse-route.test.ts \
test/routes-sessions.test.ts test/e2e-f1.test.ts \
test/workspace-preprocessing-service.test.ts test/evidence-materialization.test.ts
npx tsc --noEmit -p .
npm run build
```
These suites fix:
- CLI argument ordering and JSON/error propagation across the runner boundary;
- new-session versus resume Pi prompts;
@@ -52,7 +69,18 @@ Run the backend test suite followed by TypeScript typechecking. The suite fixes:
### Frontend client
Run the frontend test suite followed by TypeScript typechecking. The suite fixes:
From `frontend/`, the passing automated baseline is:
```sh
npx vitest run src/store/sessionStore.test.ts src/stream/useSessionStream.test.tsx \
src/widgets/registry.test.tsx src/widgets/SelectWidget.test.tsx \
src/widgets/MultiselectWidget.test.tsx src/widgets/ArtifactWidget.test.tsx \
src/shell/f1-loop.test.tsx src/shell/SessionDocumentsPanel.test.tsx
npx tsc -b
npm run build
```
These suites fix:
- widget registry and gate response payloads;
- `ui_request`, `text_delta`, activity, usage, and lifecycle event reduction;
@@ -78,18 +106,17 @@ question to finalization, followed by resume verification, belongs to the final
ticket. If its environment or credentials are unavailable, it must remain recorded as a pending
manual gate rather than being reported as passed.
## Pre-existing full-suite exceptions
## Full-suite diagnostic exceptions
The workflow baseline and every focused seam above pass on the source commit from which this
branch was created. Two unrelated full-suite failures also reproduce unchanged on that base
checkout and are therefore recorded rather than hidden or repaired in this refactoring ticket:
Every command defined above as part of the automated baseline exits successfully. Running the
broader backend and frontend suites is still useful as a diagnostic, but those full suites are not
the executable acceptance gate for this ticket because two unrelated failures reproduce unchanged
on the source commit from which this branch was created:
- the backend authentication runtime-projection suite currently rejects ten positive fixtures
with its fail-closed public error;
- one frontend application-shell authentication test does not render the expected trusted-upstream
display name.
The focused backend workflow suite, backend typecheck and build, focused frontend workflow suite,
frontend typecheck and build, complete harness pytest suite, Ruff, and complete Pi gate suite all
pass. These two exceptions must remain visible until their owning workstream resolves them; they
must not be used to relax any workflow assertion.
These two exceptions must remain visible until their owning workstream resolves them; they must not
be used to relax any workflow assertion or to describe a nonzero command as a passing baseline.