diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index 66502815..d35ca67f 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -12,6 +12,18 @@ Authentik, internal catalog/embedding services, and the PSD workspace repository `docs/operations/server-upgrade-gitea-workspace-v2.md`. Treat its operator gates and rollback requirements as mandatory; do not replace the running server stack in place. +## Session composer and empty Memory fix prepared — 2026-09-14 + +The embedded shell now caps its height at the portal mount height, keeping steering +and Stop & save visible. Empty authoritative Memory archives return zero results +without requiring embedding/BM25; SQL-rule embedding is lazy and shared. The live +empty PSD archive reproduces 503 with the current image and succeeds with the +candidate. 54 Memory tests, 90 frontend tests, five browser scenarios and both image +builds passed. Candidate core/frontend tags are `49333a2d-session-memory-fix`; +production recreation awaits the release-window confirmation required by the server +handoff. See `docs/reports/2026-09-14-session-layout-memory-fix.md` for evidence, +prepared rollback and resolved deployment commands. + ## Full/embedded shell and bilingual interface Full/embedded shell, EN/IT UI, immutable session interaction language, dark theme, diff --git a/docs/reports/2026-09-14-session-layout-memory-fix.md b/docs/reports/2026-09-14-session-layout-memory-fix.md new file mode 100644 index 00000000..1260fa43 --- /dev/null +++ b/docs/reports/2026-09-14-session-layout-memory-fix.md @@ -0,0 +1,106 @@ +# Correzione input sessione embedded e Memory vuota + +## Stato + +Correzioni implementate e immagini candidate costruite; **rilascio operativo non +ancora eseguito**. Il runbook `docs/operations/server-codex-handoff.md` richiede +conferma della finestra prima della ricreazione dei servizi operativi. + +## Cause e correzioni + +- La nuova `.thot-host` usa `100dvh`; Omics limita invece `#root` allo spazio + sotto il topbar e ne nasconde l'overflow. La prova browser riproduce controlli + che terminano a 821 px con il contenitore che termina a 782 px. La shell + embedded ora ha `max-height: 100%`, così rispetta il contenitore; il fallback + alla viewport e il contratto `--thoth-app-height` restano utilizzabili. +- Nell'installazione reale PostgreSQL contiene zero Memory Card e la collection + `psd-clinical-memory` zero punti, senza vettore sparse BM25. Il workflow + interrogava comunque embedding/Qdrant e trasformava `BM25 collection + configuration mismatch` in `memory_unavailable`, status 503. Il recupero ora + restituisce `[]` dopo aver verificato in PostgreSQL che l'archivio è vuoto. + Gli errori dell'archivio autorevole continuano a propagarsi. `memory rules` + calcola il vettore soltanto se serve e lo condivide fra le due famiglie. + +Nessuna migrazione, modifica di card, indice, DWH o autenticazione è necessaria. + +## Verifiche eseguite + +- Riproduzione live: `tht memory search` restituisceva 503. +- Confronto delle immagini attuale/candidata, con PostgreSQL e Qdrant reali e + montaggi read-only: attuale exit 1/status 503; candidata exit 0/`[]`. La + configurazione diagnostica non contiene credenziali DWH e non interroga il DWH. +- 54 test Memory superati, 1 skipped, 2 deselected secondo la configurazione + pytest; inclusi test PostgreSQL/Qdrant reali e regressione CLI archivio vuoto. +- 90 test frontend superati: shell host, composer, creazione e gestione sessioni. +- 5 scenari Playwright superati: sessione attiva embedded a 390/1280 px con + input multilinea e apertura del dialogo di arresto; composer full dopo + navigazione amministrativa; geometria host con header/rail. +- TypeScript, build frontend, Ruff sui file Python modificati, `git diff --check` + e scansione layout superati. Entrambe le build Docker completate; + smoke della configurazione frontend superato. +- Screenshot verificati in `frontend/test-results/visual-review/`. + +## Consegna candidata + +Base operativa verificata e pulita: `49333a2d35664b7237c3ddc2a9f10a605dcc84ce`. +La patch `/tmp/thoth-session-memory-fix.patch` supera `git apply --check` +contro `/srv/thothii-v2/source/ThothII`. + +| Immagine candidata | ID | +| --- | --- | +| `thothii-v2-core:49333a2d-session-memory-fix` | `sha256:ff4c435abd67c57e1e91e6e560dae73e67350ca499a5aedca3ffa517b9f59ee0` | +| `thothii-v2-frontend:49333a2d-session-memory-fix` | `sha256:35933317769f3e12953f3b144d51f8fc2e7e9f7c4830eba80cc626f757f5bf0d` | + +Copie protette di operator.env, descriptor e override sono già in +`/srv/thothii-v2/backups/20260914-session-memory-fix` (directory 0700). +Le immagini correnti sono conservate anche con tag +`before-session-memory-fix-20260914`. Non è stato fatto un nuovo backup dei dati: +questa preparazione non ha modificato dati e non sostituisce un backup coerente +nella finestra operativa, se richiesto dal runbook di rilascio. + +## Comandi risolti per la finestra da confermare + +Verificare nuovamente lo stato delle sessioni e gestire quelle attive prima del +riavvio. Il seguente launcher conserva progetto, env file e ordine degli override: + +```bash +thoth_fix_compose() { + sudo docker compose --project-name thothii-7f901b48fe35 \ + --project-directory /srv/thothii-v2/source/ThothII \ + --env-file /srv/thothii-v2/operator/operator.env \ + -f /srv/thothii-v2/source/ThothII/compose.yaml \ + -f /srv/thothii-v2/source/ThothII/deploy/compose.server.yaml \ + -f /srv/thothii-v2/source/ThothII/deploy/compose.git-ssh.yaml \ + -f /srv/thothii-v2/operator/compose.portal-upstream.yaml \ + -f /srv/thothii-v2/source/ThothII/deploy/psd-server-v2/generated/compose.models.yaml "$@" +} +thoth_fix_compose exec -T core node /app/backend/dist/operator-command.js maintenance-activate +thoth_fix_compose exec -T core node /app/backend/dist/operator-command.js maintenance-status +sudo git -C /srv/thothii-v2/source/ThothII apply /tmp/thoth-session-memory-fix.patch +``` + +Dopo gestione delle sessioni e backup nella finestra, aggiornare esclusivamente +`THTII_RELEASE_IMAGE_TAG` in `/srv/thothii-v2/operator/operator.env` al valore +`49333a2d-session-memory-fix`, preservando gli altri valori e i permessi. +Quindi: + +```bash +thoth_fix_compose up -d --no-deps --no-build core frontend +sudo docker exec omics_portal-nginx-1 nginx -t +sudo docker exec omics_portal-nginx-1 nginx -s reload +sudo tht --installation /srv/thothii-v2/source/ThothII/deploy/psd-server-v2/thothii-installation.yaml status +sudo tht --installation /srv/thothii-v2/source/ThothII/deploy/psd-server-v2/thothii-installation.yaml doctor --json +``` + +Attendere il TTL di 30 secondi del manifest Django, verificare asset/config e +Memory; provare input/arresto nel portale prima della riapertura: + +```bash +thoth_fix_compose exec -T core node /app/backend/dist/operator-command.js maintenance-deactivate +``` + +Rollback applicativo: ripristinare l'operator.env protetto, ricreare solo +core/frontend con lo stesso launcher, verificare e ricaricare nginx. Per riallineare +anche i sorgenti usare `git apply --reverse --check` e poi `git apply --reverse` +sulla sola patch preparata, senza reset di altre modifiche. Non occorre ripristinare +PostgreSQL o indici per un rollback di queste due correzioni. diff --git a/frontend/e2e/ui-visual-review.spec.ts b/frontend/e2e/ui-visual-review.spec.ts index 853e36fe..27e9be69 100644 --- a/frontend/e2e/ui-visual-review.spec.ts +++ b/frontend/e2e/ui-visual-review.spec.ts @@ -579,3 +579,43 @@ test("host header and left rail leave the application a bounded working area", a expect(search!.width).toBeGreaterThan(180); expect(await inspect(page)).toEqual({ overflow: 0, serif: [], tiny: [] }); }); + +for (const width of [390, 1280]) { + test(`embedded active session keeps steering and stop inside the portal at ${width}px`, async ({ page }, testInfo) => { + await page.setViewportSize({ width, height: 822 }); + await fixtures(page, "embedded"); + await page.route("**/api/sessions", route => route.request().method() === "POST" + ? route.fulfill({ json: { id: "layout-session" } }) : route.fallback()); + await page.route("**/api/sessions/layout-session", route => route.fulfill({ json: { id: "layout-session", active: true, status: "open", workspace } })); + await page.goto("/"); + // The Omics template bounds #root below its topbar and clips overflow. + await page.addStyleTag({ content: ` + body { margin: 0; padding-top: 70px; } + #root { height: calc(100dvh - 110px); overflow: hidden; } + ` }); + const prompt = page.getByRole("textbox", { name: "New question", exact: true }); + await prompt.fill("Review the cohort"); + await page.getByRole("button", { name: "Send", exact: true }).click(); + const steering = page.getByRole("textbox", { name: "Steering", exact: true }); + await expect(steering).toBeVisible(); + // Drive the same store projection as streaming, without a real model. + await page.evaluate(async () => { + const path = "/src/store/sessionStore.ts"; + const { useSessionStore } = await import(/* @vite-ignore */ path); + useSessionStore.getState().applyEvent({ type: "text_delta", text: "Reviewing the cohort\n".repeat(80) }); + }); + await steering.fill("Please pause the analysis\n".repeat(15)); + const stop = page.getByRole("button", { name: "Stop and save session", exact: true }); + await page.locator("#root").evaluate(el => { el.scrollTop = 0; }); + const root = await page.locator("#root").boundingBox(); + for (const control of [steering, stop]) { + const box = await control.boundingBox(); + expect(box!.y).toBeGreaterThanOrEqual(root!.y); + expect(box!.y + box!.height).toBeLessThanOrEqual(root!.y + root!.height); + await expect(control).toBeInViewport(); + } + await page.screenshot({ path: testInfo.outputPath("embedded-session.png") }); + await stop.click(); + await expect(page.getByRole("dialog", { name: "Stop the session?" })).toBeVisible(); + }); +} diff --git a/frontend/src/index.css b/frontend/src/index.css index e4eaf6af..1908d587 100644 --- a/frontend/src/index.css +++ b/frontend/src/index.css @@ -222,6 +222,8 @@ body.thot-full-document { margin: 0; } background: oklch(var(--background)); color-scheme: light; } + /* Embedded hosts can reserve space for their own header/footer around #root. */ + .thot-host--embedded { max-height: 100%; } .thot-host[data-theme="dark"] { color-scheme: dark; } .thot-host--full { --thot-shell-gutter: max(20px, 1.5rem); } .thot-host--full > .thot-context-layout { diff --git a/harness/tests/memory/test_administration.py b/harness/tests/memory/test_administration.py index 9f4f12dd..5830f613 100644 --- a/harness/tests/memory/test_administration.py +++ b/harness/tests/memory/test_administration.py @@ -1067,3 +1067,36 @@ def test_hybrid_recall_with_configured_real_embedding_and_isolated_stores(databa scope=scope)} == {rule, general, parent} finally: s.close() + + +def test_empty_archive_rules_skip_unavailable_embedding_and_vectors(service, monkeypatch): + import json + from types import SimpleNamespace + + from typer.testing import CliRunner + + from tht.cli import app, memory_cmd, vector_cmd + from tht.ports.vector import VectorStoreError + from tht.vectorstore.embeddings import EmbeddingsError + + s, _ = service + snapshot = review_snapshot(s) + snapshot.decisions[:] = [d for d in snapshot.decisions + if not (d.type == "phase_approved" and int(d.subject.split(":")[1]) > 3)] + + def unavailable_embedding(_): + raise EmbeddingsError("Embedding unavailable") + + def unavailable_search(*args, **kwargs): + raise VectorStoreError("BM25 collection configuration mismatch") + + cfg = SimpleNamespace(database=SimpleNamespace(database="dwh", db_schema="sales"), embeddings=None) + monkeypatch.setattr(memory_cmd, "_load_config_or_exit", lambda _: cfg) + monkeypatch.setattr(memory_cmd, "memory_service", lambda _: s) + monkeypatch.setattr(memory_cmd, "load_snapshot_or_exit", lambda *_: snapshot) + monkeypatch.setattr(s, "close", lambda: None) + monkeypatch.setattr(vector_cmd, "open_searcher", lambda _: SimpleNamespace(search=unavailable_search)) + monkeypatch.setattr(vector_cmd, "make_embedder", lambda _: SimpleNamespace(embed_query=unavailable_embedding)) + result = CliRunner().invoke(app, ["memory", "rules", "Order", "--session", snapshot.manifest.id, "--json"]) + assert result.exit_code == 0, result.output + assert json.loads(result.output) == [] diff --git a/harness/tests/memory/test_recall.py b/harness/tests/memory/test_recall.py index 904668c0..dea55ef4 100644 --- a/harness/tests/memory/test_recall.py +++ b/harness/tests/memory/test_recall.py @@ -157,7 +157,7 @@ def test_recall_cli_reconstructs_applied_and_rejected_memory_from_persisted_f2_s cards = {r.id: Card(id=r.id, family="domain_clarification", subject=r.subject, detail=r.detail, scope="psd-clinical", workspace_id="psd-clinical", origin="workflow", created_at=r.ts, updated_at=r.ts, revision="r", indexed=True) for r in records} - archive = SimpleNamespace(list=lambda query: {}, get=lambda identity: cards[identity], + archive = SimpleNamespace(list=lambda query: {"total": len(cards)}, get=lambda identity: cards[identity], close=lambda: None) archive.operation = lambda: nullcontext(archive) service = MemoryService(archive, PrincipalContext(issuer="local", subject="reviewer"), @@ -179,3 +179,43 @@ def test_recall_cli_reconstructs_applied_and_rejected_memory_from_persisted_f2_s assert current_phase(repository.get(session_id)) == 2 assert embedder.questions == ["active patients"] assert searcher.calls == [([0.1, 0.2], 20, ["memory"])] + + +def test_empty_archive_recall_does_not_require_vector_projection(tmp_path, monkeypatch): + """A fresh installation can have an empty dense-only Memory collection.""" + from tht.ports.vector import VectorStoreError + + class UnavailableSearcher: + def search(self, *args, **kwargs): + raise VectorStoreError("BM25 collection configuration mismatch") + + archive = SimpleNamespace(list=lambda query: {"items": [], "total": 0}, close=lambda: None) + service = MemoryService(archive, PrincipalContext(issuer="local", subject="reviewer"), + store_factory=lambda: None, embedder_factory=Embedder) + embedder = Embedder() + monkeypatch.setattr("tht.cli.memory_cmd.memory_service", lambda cfg: service) + monkeypatch.setattr("tht.cli.vector_cmd.open_searcher", lambda cfg: UnavailableSearcher()) + monkeypatch.setattr("tht.cli.vector_cmd.make_embedder", lambda cfg: embedder) + for command in ("search", "solved-search"): + response = CliRunner().invoke(app, ["memory", command, "active patients", "--json", + "-c", str(_workspace_config(tmp_path))]) + assert response.exit_code == 0, response.output + assert json.loads(response.stdout) == [] + assert embedder.questions == [] + + +def test_recall_does_not_hide_an_unavailable_authoritative_archive(): + import pytest + + from tht.memory.models import MemoryUnavailable + + def unavailable(query): + raise MemoryUnavailable("Memory archive is unavailable") + + archive = SimpleNamespace(list=unavailable) + service = MemoryService(archive, PrincipalContext(issuer="local", subject="reviewer"), + store_factory=lambda: None, embedder_factory=Embedder) + embedder = Embedder() + with pytest.raises(MemoryUnavailable): + service.recall("active patients", searcher=Searcher([]), embedder=embedder) + assert embedder.questions == [] diff --git a/harness/tht/cli/memory_cmd.py b/harness/tht/cli/memory_cmd.py index af499152..62d14086 100644 --- a/harness/tht/cli/memory_cmd.py +++ b/harness/tht/cli/memory_cmd.py @@ -296,6 +296,7 @@ def rules_cmd(question: str, session: str = typer.Option(..., "--session"), filters: str = typer.Option("{}", "--filters"), json_out: bool = typer.Option(False, "--json"), config: Path = CONFIG_OPT): """Consult SQL rules and explained errors in schema linking and SQL construction.""" + from functools import cache from types import SimpleNamespace from tht.cli.vector_cmd import make_embedder, open_searcher @@ -307,8 +308,8 @@ def rules_cmd(question: str, session: str = typer.Option(..., "--session"), if current_phase(snapshot) not in {4, 6, 7}: raise ValueError("Memory rules are consulted in schema linking or SQL construction") scope = _recall_scope(cfg, filters) - vector = make_embedder(cfg.embeddings).embed_query(question) - embedder = SimpleNamespace(embed_query=lambda _: vector) + # Share one vector across both families, but only when the archive has cards. + embedder = SimpleNamespace(embed_query=cache(make_embedder(cfg.embeddings).embed_query)) searcher = open_searcher(cfg) candidates = [] for family in ("sql_rule", "explained_error"): diff --git a/harness/tht/memory/service.py b/harness/tht/memory/service.py index a5db3097..ad8a4cfa 100644 --- a/harness/tht/memory/service.py +++ b/harness/tht/memory/service.py @@ -129,7 +129,10 @@ class MemoryService: if not question.strip(): raise ValueError("Recall question must not be empty") scope = scope or RecallScope() - self.repository.list(CardQuery(page_size=1)) + # PostgreSQL is authoritative: an empty archive needs no search projection. + # Still read it first so archive failures cannot masquerade as zero results. + if self.repository.list(CardQuery(page_size=1))["total"] == 0: + return [] kinds = (["solved_question"] if family == "solved_question" else ["memory"] if family else ["memory", "solved_question"]) hits = searcher.search(embedder.embed_query(question),