docs: plan model activity signal filter

2026-07-15 03:15:53 +02:00
parent eef4cf0f50
commit e9cbc2b701
1 changed files with 495 additions and 0 deletions
@@ -0,0 +1,495 @@
+# Model Activity Signal Filter Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Keep the left Model activity panel focused on prompt, genuine reasoning, meaningful status, and reviewer gates while hiding assistant narration, tool lifecycle, and turn lifecycle rows.
+
+**Architecture:** Preserve the complete chronological `activityLog` and every existing backend/SSE/store contract. Add a strict allowlist at the `ModelActivityPanel` rendering boundary, derive the visible tail for stable bottom-follow behavior, and leave the central transcript untouched.
+
+**Tech Stack:** React 18, TypeScript, Zustand, Vitest, Testing Library, Vite, Docker Compose, nginx-unprivileged.
+
+## Global Constraints
+
+- Visible kinds are exactly `prompt`, `thinking`, `status`, and `gate`.
+- Hidden kinds are exactly `assistant`, `tool`, and `lifecycle`; unknown future kinds are hidden by default.
+- Do not filter by text, language, tool name, or heuristic content matching.
+- Do not change backend emission, SSE subscriptions, `sessionStore` ingestion, the central transcript, or workflow state.
+- Base the empty state and bottom-follow behavior on visible entries only.
+- Preserve warning/error labels, Markdown rendering, phase labels, accessibility, close behavior, and the 48 px near-bottom threshold.
+- Do not touch the user-owned untracked `.vite/` directory.
+
+---
+
+## File map
+
+- `frontend/src/shell/ModelActivityPanel.tsx`: owns the visibility predicate, visible projection, rendering, empty state, and bottom-follow dependency.
+- `frontend/src/shell/ModelActivityPanel.test.tsx`: proves the allowlist, transcript isolation, hidden-only empty state, Markdown/status rendering, and scroll behavior.
+- `brain/codebase/workflow-ui-contracts.md`: records the durable distinction between the complete internal log and its user-facing projection.
+- `PROJECT_STATE.md`: records final verification, container image/asset, and Qwen smoke evidence.
+
+### Task 1: Filter the user-facing activity projection
+
+**Files:**
+- Modify: `frontend/src/shell/ModelActivityPanel.test.tsx`
+- Modify: `frontend/src/shell/ModelActivityPanel.tsx`
+
+**Interfaces:**
+- Consumes: `ActivityEntry` and `ActivityKind` from `frontend/src/api/types.ts`; the unchanged `activityLog` array from `useSessionStore`.
+- Produces: `isVisibleModelActivity(entry: ActivityEntry): boolean`; a panel projection containing only prompt/thinking/status/gate rows.
+
+- [ ] **Step 1: Write failing behavior tests before production code**
+
+In `frontend/src/shell/ModelActivityPanel.test.tsx`, import the activity kind type and the module namespace so the not-yet-created predicate can fail as an assertion rather than a module-load error:
+
+```tsx
+import type { ActivityEntry, ActivityKind } from "../api/types";
+import * as activityPanelModule from "./ModelActivityPanel";
+import { isNearBottom, ModelActivityPanel } from "./ModelActivityPanel";
+```
+
+Replace the existing complete-sequence test with the approved mixed-sequence contract:
+
+```tsx
+test("renders only prompt, thinking, status, and gate from a mixed F1 sequence", () => {
+  const store = useSessionStore.getState();
+  store.setPhase("F1");
+  store.setLastUserEntry({ kind: "input", text: "How many patients?" });
+  store.applyEvent({ type: "system_event", event: "agent_start" });
+  store.applyEvent({
+    type: "activity_event",
+    activity: { kind: "tool", toolCallId: "tool-1", toolName: "bash", status: "running" },
+  });
+  store.applyEvent({
+    type: "activity_event",
+    activity: { kind: "tool", toolCallId: "tool-1", toolName: "bash", status: "completed" },
+  });
+  store.applyEvent({ type: "text_delta", text: "Let me run another bash search." });
+  store.applyEvent({ type: "activity_delta", text: "Inspecting **schema**." });
+  store.applyEvent({ type: "info", level: "warning", text: "Retrying schema lookup" });
+  store.applyEvent({
+    type: "ui_request",
+    ui_request: { id: "gate-1", widget: "select", phase: "F1_review", title: "Confirm cohort" },
+  });
+
+  render(<ModelActivityPanel onClose={vi.fn()} />);
+
+  expect(screen.getAllByRole("article")).toHaveLength(4);
+  expect(screen.getByText("How many patients?")).toBeInTheDocument();
+  expect(screen.getByText("schema").tagName).toBe("STRONG");
+  expect(screen.getByText("Retrying schema lookup")).toBeInTheDocument();
+  expect(screen.getByText("Confirm cohort")).toBeInTheDocument();
+  expect(screen.queryByText("Agent started")).not.toBeInTheDocument();
+  expect(screen.queryByText("bash")).not.toBeInTheDocument();
+  expect(screen.queryByText("completed")).not.toBeInTheDocument();
+  expect(screen.queryByText("Let me run another bash search.")).not.toBeInTheDocument();
+  expect(useSessionStore.getState().transcript).toEqual([
+    { role: "assistant", text: "Let me run another bash search." },
+  ]);
+});
+```
+
+Add direct allowlist/default-deny and hidden-only empty-state tests:
+
+```tsx
+test("uses an explicit default-deny activity-kind allowlist", () => {
+  const predicate = (activityPanelModule as unknown as {
+    isVisibleModelActivity?: (entry: ActivityEntry) => boolean;
+  }).isVisibleModelActivity;
+  expect(predicate).toBeTypeOf("function");
+  if (!predicate) return;
+
+  for (const kind of ["prompt", "thinking", "status", "gate"] satisfies ActivityKind[]) {
+    expect(predicate({ kind, phase: "F1", text: kind })).toBe(true);
+  }
+  for (const kind of ["assistant", "tool", "lifecycle"] satisfies ActivityKind[]) {
+    expect(predicate({ kind, phase: "F1", text: kind })).toBe(false);
+  }
+  expect(predicate({ kind: "future-kind" as ActivityKind, phase: "F1", text: "future" })).toBe(false);
+});
+
+test("shows the empty state when the raw log contains only hidden entries", () => {
+  useSessionStore.setState({
+    activityLog: [
+      { kind: "assistant", phase: "F1", text: "Let me try another command." },
+      { kind: "tool", phase: "F1", text: "bash", toolCallId: "tool-1", status: "completed" },
+      { kind: "lifecycle", phase: "F1", text: "Turn end" },
+    ],
+  });
+
+  render(<ModelActivityPanel onClose={vi.fn()} />);
+
+  expect(screen.getByText("No activity yet.")).toBeInTheDocument();
+  expect(screen.queryAllByRole("article")).toHaveLength(0);
+});
+```
+
+Change the assistant/thinking Markdown test so only reasoning is rendered:
+
+```tsx
+test("renders thinking markdown without assistant narration", () => {
+  useSessionStore.setState({
+    activityLog: [
+      { kind: "thinking", phase: "F2", text: "Considering **history**." },
+      { kind: "assistant", phase: "F2", text: "Use *current* records." },
+    ],
+  });
+
+  render(<ModelActivityPanel onClose={vi.fn()} />);
+
+  expect(screen.getByText("history").tagName).toBe("STRONG");
+  expect(screen.queryByText("current")).not.toBeInTheDocument();
+  expect(screen.getAllByRole("article")).toHaveLength(1);
+});
+```
+
+Add the hidden-update scroll regression:
+
+```tsx
+test("does not bottom-follow when only a hidden event arrives", () => {
+  useSessionStore.setState({
+    activityLog: [{ kind: "prompt", phase: "F1", text: "Visible prompt" }],
+  });
+  render(<ModelActivityPanel onClose={vi.fn()} />);
+  const viewport = screen.getByTestId("activity-scroll");
+  setScrollGeometry(viewport, { scrollHeight: 400, clientHeight: 100, scrollTop: 300 });
+
+  act(() => useSessionStore.getState().applyEvent({
+    type: "activity_event",
+    activity: { kind: "tool", toolCallId: "tool-2", toolName: "bash", status: "running" },
+  }));
+
+  expect(viewport.scrollTop).toBe(300);
+});
+```
+
+Update both existing scroll tests to start with a visible prompt and append a visible status rather
+than calling `recordLifecycle`, which is intentionally hidden:
+
+```tsx
+useSessionStore.setState({
+  activityLog: [{ kind: "prompt", phase: "F1", text: "First" }],
+});
+// ...existing geometry and scroll setup...
+act(() => useSessionStore.getState().applyEvent({ type: "info", text: "Second" }));
+```
+
+- [ ] **Step 2: Run the focused test and verify RED**
+
+Run:
+
+```bash
+cd frontend
+npx vitest run src/shell/ModelActivityPanel.test.tsx
+```
+
+Expected: FAIL because the predicate is absent, the panel renders seven mixed-sequence rows instead
+of four, hidden-only input does not show the empty state, assistant text remains visible, and a
+hidden tool append moves the bottom-follow viewport.
+
+- [ ] **Step 3: Implement the strict rendering-boundary filter**
+
+In `frontend/src/shell/ModelActivityPanel.tsx`, extend the type import and add the pure predicate:
+
+```tsx
+import type { ActivityEntry, ActivityKind } from "../api/types";
+
+const VISIBLE_MODEL_ACTIVITY_KINDS: ReadonlySet<ActivityKind> = new Set([
+  "prompt",
+  "thinking",
+  "status",
+  "gate",
+]);
+
+export function isVisibleModelActivity(entry: ActivityEntry): boolean {
+  return VISIBLE_MODEL_ACTIVITY_KINDS.has(entry.kind);
+}
+```
+
+Inside `ModelActivityPanel`, derive the projection and a stable visible tail:
+
+```tsx
+const activityLog = useSessionStore((s) => s.activityLog);
+const visibleActivity = activityLog.filter(isVisibleModelActivity);
+const visibleCount = visibleActivity.length;
+const visibleTail = visibleActivity.at(-1);
+const scrollRef = useRef<HTMLDivElement>(null);
+const followRef = useRef(true);
+```
+
+Change the layout effect dependency so hidden appends do not run it, while a streamed visible tail
+still changes object identity and follows correctly:
+
+```tsx
+useLayoutEffect(() => {
+  const viewport = scrollRef.current;
+  if (viewport && followRef.current) viewport.scrollTop = viewport.scrollHeight;
+}, [visibleCount, visibleTail]);
+```
+
+Finally, use `visibleActivity` for both empty-state and row rendering:
+
+```tsx
+{visibleActivity.length === 0 ? (
+  <p className="py-3 text-sm text-muted-foreground">No activity yet.</p>
+) : (
+  visibleActivity.map((entry, index) => (
+    <ActivityRow key={`${index}-${entry.toolCallId ?? entry.kind}`} entry={entry} />
+  ))
+)}
+```
+
+Do not modify `sessionStore.ts`, `useSessionStream.ts`, backend files, or `ActivityEntry`.
+
+- [ ] **Step 4: Run focused GREEN and adjacent store regressions**
+
+Run:
+
+```bash
+cd frontend
+npx vitest run src/shell/ModelActivityPanel.test.tsx src/store/sessionStore.test.ts
+npx tsc -b
+```
+
+Expected: both test files pass, the complete store fold still retains all kinds, and TypeScript
+exits 0.
+
+- [ ] **Step 5: Inspect scope and commit Task 1**
+
+Run:
+
+```bash
+git diff --check
+git diff -- frontend/src/shell/ModelActivityPanel.tsx frontend/src/shell/ModelActivityPanel.test.tsx
+git status --short
+git add frontend/src/shell/ModelActivityPanel.tsx frontend/src/shell/ModelActivityPanel.test.tsx
+git commit -m "fix(frontend): hide model activity execution noise"
+```
+
+Expected: only the panel and its test are committed; `.vite/` remains untracked and untouched.
+
+### Task 2: Verify, document, deploy, and smoke the filtered panel
+
+**Files:**
+- Modify: `brain/codebase/workflow-ui-contracts.md`
+- Modify: `PROJECT_STATE.md`
+
+**Interfaces:**
+- Consumes: Task 1's `isVisibleModelActivity` contract and production frontend bundle.
+- Produces: deployed frontend image/entry evidence and a durable record of the complete-log versus visible-projection distinction.
+
+- [ ] **Step 1: Run the complete frontend verification gate**
+
+Run:
+
+```bash
+cd frontend
+npx vitest run
+npx tsc -b
+npm run build
+cd ..
+git diff --check
+```
+
+Expected: all frontend tests pass (baseline before this task: 285 tests), TypeScript/build exit 0,
+and only the repository's pre-existing MSW/ref/chunk-size warnings remain.
+
+- [ ] **Step 2: Record the running asset and runtime state before deployment**
+
+Run:
+
+```bash
+docker inspect -f '{{.Name}}|{{.State.Status}}|{{if .State.Health}}{{.State.Health.Status}}{{else}}no-healthcheck{{end}}|{{.Image}}|{{.State.StartedAt}}' thothii-core-1 thothii-frontend-1 omics_portal-web-1
+docker exec thothii-frontend-1 cat /usr/share/nginx/html/.vite/manifest.json | rg -n -B 3 '"isEntry": true'
+docker top thothii-core-1 -eo pid,args
+```
+
+Expected: core is healthy, frontend/portal are running, the active entry is recorded, and no
+`pi --mode rpc` process is active before the frontend-only recreate.
+
+- [ ] **Step 3: Build and recreate only the frontend service**
+
+Run from the repository root:
+
+```bash
+docker compose build frontend
+docker compose up -d --no-deps --force-recreate --wait --wait-timeout 60 frontend
+docker inspect -f '{{.Name}}|{{.State.Status}}|{{.Image}}|{{.State.StartedAt}}' thothii-frontend-1
+docker exec thothii-frontend-1 cat /usr/share/nginx/html/.vite/manifest.json | rg -n -B 3 '"isEntry": true'
+```
+
+Expected: the build exits 0, frontend is running on the newly built image, and the active Vite
+entry differs from the value recorded in Step 2.
+
+If and only if the entry changed, refresh the portal's indefinite manifest cache:
+
+```bash
+docker restart omics_portal-web-1
+docker inspect -f '{{.Name}}|{{.State.Status}}|{{.State.StartedAt}}' omics_portal-web-1
+```
+
+Expected: only `omics_portal-web-1` receives a new start timestamp; core and unrelated portal
+services are not restarted.
+
+- [ ] **Step 4: Run a real no-COT Qwen smoke with exact settings restore**
+
+Run the probe inside the core container so it can reach the private backend listener. It preserves
+the full settings object, creates one uniquely named session, waits for sanitized tool activity and
+the first gate, asserts Qwen emitted no reasoning delta, cleans up only its own session, and restores
+settings in `finally`:
+
+```bash
+docker exec -i thothii-core-1 node - <<'NODE'
+const base = "http://127.0.0.1:8787";
+const canonical = (value) => JSON.stringify(value, Object.keys(value).sort());
+async function request(path, options = {}) {
+  const response = await fetch(base + path, {
+    ...options,
+    headers: { "content-type": "application/json", ...(options.headers ?? {}) },
+  });
+  if (!response.ok) throw new Error(`${options.method ?? "GET"} ${path}: ${response.status}`);
+  return response.status === 204 ? null : response.json();
+}
+
+const saved = await request("/settings");
+const stamp = new Date().toISOString().replaceAll(/[:.]/g, "-");
+const question = `activity-filter-smoke-${stamp}: quanti record sono disponibili?`;
+let sessionId = null;
+let reader = null;
+let activityDeltaCount = 0;
+let toolCount = 0;
+let gateSeen = false;
+let timeout = null;
+let cleanupError = null;
+
+try {
+  await request("/settings", {
+    method: "PUT",
+    body: JSON.stringify({
+      ...saved,
+      provider: "local-qwen",
+      model: "qwen3.6-35b-a3b",
+      thinking: "low",
+    }),
+  });
+  const created = await request("/sessions", {
+    method: "POST",
+    body: JSON.stringify({ question }),
+  });
+  sessionId = created.id;
+
+  const response = await fetch(`${base}/sessions/${encodeURIComponent(sessionId)}/events`);
+  if (!response.ok || !response.body) throw new Error(`SSE open failed: ${response.status}`);
+  reader = response.body.getReader();
+  timeout = setTimeout(() => { void reader.cancel("Qwen smoke timeout"); }, 180_000);
+  const decoder = new TextDecoder();
+  let buffer = "";
+  let eventName = "message";
+  let dataLines = [];
+
+  while (!gateSeen) {
+    const read = await reader.read();
+    if (read.done) throw new Error("SSE ended before the first gate");
+    buffer += decoder.decode(read.value, { stream: true });
+    const lines = buffer.split("\n");
+    buffer = lines.pop() ?? "";
+    for (const rawLine of lines) {
+      const line = rawLine.replace(/\r$/, "");
+      if (line.startsWith("event:")) eventName = line.slice(6).trim();
+      else if (line.startsWith("data:")) dataLines.push(line.slice(5).trimStart());
+      else if (line === "") {
+        if (dataLines.length > 0) {
+          const payload = JSON.parse(dataLines.join("\n"));
+          if (eventName === "activity_delta") activityDeltaCount += 1;
+          if (eventName === "activity_event") {
+            const keys = Object.keys(payload.activity ?? {}).sort();
+            const allowed = ["kind", "status", "toolCallId", "toolName"];
+            if (JSON.stringify(keys) !== JSON.stringify(allowed)) {
+              throw new Error(`forbidden tool fields: ${keys.join(",")}`);
+            }
+            toolCount += 1;
+          }
+          if (eventName === "ui_request") gateSeen = true;
+        }
+        eventName = "message";
+        dataLines = [];
+      }
+    }
+  }
+
+  if (toolCount === 0) throw new Error("no sanitized tool activity observed");
+  if (activityDeltaCount !== 0) throw new Error(`expected no COT, got ${activityDeltaCount}`);
+  console.log(JSON.stringify({ sessionId, activityDeltaCount, toolCount, gateSeen }));
+} finally {
+  if (timeout) clearTimeout(timeout);
+  if (reader) await reader.cancel().catch(() => undefined);
+  if (sessionId) {
+    try {
+      const close = await fetch(`${base}/sessions/${encodeURIComponent(sessionId)}/close`, { method: "POST" });
+      if (!close.ok) throw new Error(`smoke close failed: ${close.status}`);
+      const remove = await fetch(`${base}/sessions/${encodeURIComponent(sessionId)}`, { method: "DELETE" });
+      if (remove.status !== 204) throw new Error(`smoke delete failed: ${remove.status}`);
+    } catch (error) {
+      cleanupError = error;
+    }
+  }
+  await request("/settings", { method: "PUT", body: JSON.stringify(saved) });
+  const restored = await request("/settings");
+  const sessions = await request("/sessions");
+  if (canonical(restored) !== canonical(saved)) throw new Error("settings restore mismatch");
+  if (sessionId && sessions.some((session) => session.id === sessionId)) {
+    throw new Error("smoke session still exists");
+  }
+  if (cleanupError) throw cleanupError;
+  console.log(JSON.stringify({ settingsRestored: true, smokePresent: false }));
+}
+NODE
+```
+
+Expected: `activityDeltaCount` is 0, `toolCount` is greater than 0, `gateSeen` is true, tool payloads
+contain only the four public fields, settings restore is exact, and the smoke session is absent.
+
+- [ ] **Step 5: Update durable project state with the new projection contract and evidence**
+
+In `brain/codebase/workflow-ui-contracts.md`, replace the complete-panel statement with:
+
+```markdown
+- `activityLog` remains the complete in-memory chronological fold of prompt, thinking, assistant,
+  sanitized tool lifecycle, reviewer gates, status, and turn lifecycle. The left Model activity
+  panel is a strict projection: it renders only prompt, thinking, status, and gate; assistant,
+  tool, lifecycle, and unknown future kinds are hidden. The central transcript and workflow state
+  continue to consume their existing events independently.
+```
+
+In `PROJECT_STATE.md`, add a dated resolved-state paragraph containing the exact final frontend test
+count, TypeScript/build result, new frontend image id, old/new Vite entry, portal restart decision,
+Qwen smoke session id/counts, exact settings restore, cleanup, and absence of residual Pi runtime.
+
+- [ ] **Step 6: Recheck runtime, review scope, and commit the evidence**
+
+Run:
+
+```bash
+docker inspect -f '{{.Name}}|{{.State.Status}}|{{if .State.Health}}{{.State.Health.Status}}{{else}}no-healthcheck{{end}}|{{.Image}}' thothii-core-1 thothii-frontend-1 omics_portal-web-1
+docker top thothii-core-1 -eo pid,args
+docker logs --since 15m --tail 300 thothii-core-1
+docker logs --since 15m --tail 200 thothii-frontend-1
+git diff --check
+git status --short
+git diff -- brain/codebase/workflow-ui-contracts.md PROJECT_STATE.md
+git add brain/codebase/workflow-ui-contracts.md PROJECT_STATE.md
+git commit -m "docs: record filtered model activity deployment"
+```
+
+Expected: containers are healthy/running, no smoke/Pi process remains, logs contain no raw tool
+payloads/credentials/fatal crash, only the two documentation files are committed, and `.vite/`
+remains untouched.
+
+---
+
+## Final review gate
+
+Before integration, request an independent review of the complete implementation range. The review
+must verify the allowlist is at the rendering boundary, assistant text remains in the central
+transcript, hidden updates cannot move the panel scroll, unknown kinds default to hidden, tests
+exercise real store/panel behavior, and deployment evidence matches the live containers. Fix every
+Critical or Important finding before declaring the work complete.