diff --git a/docs/superpowers/plans/2026-07-16-user-owned-session-storage.md b/docs/superpowers/plans/2026-07-16-user-owned-session-storage.md new file mode 100644 index 00000000..54238d62 --- /dev/null +++ b/docs/superpowers/plans/2026-07-16-user-owned-session-storage.md @@ -0,0 +1,130 @@ +# User-owned sessions implementation plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Associate every ThothII session and preference with a stable logged-in or local OS principal. + +**Architecture:** The harness owns a repository contract. `FilesystemSessionRepository` stores readable phase documents under the local user home; `PostgresSessionRepository` stores the same logical snapshot in a private Supabase schema. The backend resolves a trusted principal before every action and passes it through to `tht`; the portal proxy supplies that identity. + +**Tech Stack:** Python 3.12, Typer, SQLAlchemy/psycopg2, PostgreSQL/Supabase RLS, Fastify/TypeScript, React/Vitest, Django/nginx. + +## Global Constraints + +- Server storage is PostgreSQL wire protocol with TLS verification, schema `thoth_sessions`; never PostgREST or browser DB access. +- Server identity is `(issuer, Django request.user.pk)`; clients never choose session owner. +- Admin authorization uses Authentik group `authentik Admins`; unauthorized and missing session both return HTTP 404. +- Server has no persistent session filesystem fallback or dual write; DB failure is HTTP 503 before Pi starts. +- Local mode uses `~/.thothii` or `THT_HOME`, runs loopback-only, and creates a UUID identity in `identity.json`. +- Persist current artifacts and append-only decisions only; no chat transcript, artifact revisions, session embeddings, or content in audit logs. +- New session IDs are UUIDv4. Existing `solved_question` behavior is unchanged. +- Follow TDD: each behavior test must fail before its implementation is written. + +--- + +### Task 1: Session repository contract and local implementation + +**Files:** +- Create: `harness/tht/session/repository.py`, `harness/tht/session/filesystem_repository.py` +- Modify: `harness/tht/config.py`, `harness/tht/session/store.py`, `harness/tht/session/models.py`, `harness/pyproject.toml` +- Test: `harness/tests/test_session_repository.py`, `harness/tests/test_local_identity.py` + +**Interfaces:** Produce `PrincipalContext`, `SessionSnapshot`, `SessionRepository`, and `build_session_repository(config, principal)`. The filesystem adapter must preserve the present documents under `/workspaces//sessions/` and read/write artifact keys, decisions, manifest fields, and per-principal preferences. + +- [ ] Write failing tests proving UUIDv4 creation, principal-scoped local roots, readable phase artifacts, decision append, `THT_HOME` override, and an identity UUID stable across restarts. +- [ ] Run `cd harness && .venv/bin/pytest tests/test_session_repository.py tests/test_local_identity.py -q`; confirm failure because repository and identity interfaces do not exist. +- [ ] Implement the smallest repository contract and filesystem adapter. Use `portalocker` for mutations and private local directory/file permissions where supported. +- [ ] Run the focused tests and `cd harness && .venv/bin/pytest -q && .venv/bin/ruff check .`. +- [ ] Commit `feat(harness): add local session repository`. + +### Task 2: PostgreSQL schema, migrations, and repository + +**Files:** +- Create: `harness/tht/session/postgres_repository.py`, `harness/tht/migrations/sessions/001_schema.sql`, `harness/tht/migrations/sessions/002_security.sql` +- Modify: `harness/tht/cli/session_cmd.py`, `harness/tht/config.py` +- Test: `harness/tests/test_postgres_session_repository.py`, `harness/tests/test_session_migrate_cmd.py` + +**Interfaces:** Add `tht session migrate --database-url URL [--status] --json`; `PostgresSessionRepository` implements the Task 1 contract, starts a transaction, sets actor/admin local settings, and maps `cte_sql:` to artifacts. + +- [ ] Write failing unit/integration tests for migration status, owner isolation, admin cross-user access, cascade delete with content-free audit tombstone, and no embedding invocation. +- [ ] Run the focused tests and confirm expected RED failures. +- [ ] Implement idempotent migration runner, private tables, forced RLS policies, role separation, advisory transaction locks, and repository methods using direct PostgreSQL TLS settings. +- [ ] Run focused tests, then full harness tests and Ruff. +- [ ] Commit `feat(harness): persist server sessions in postgres`. + +### Task 3: Migrate harness workflow and deterministic write commands + +**Files:** +- Modify: `harness/tht/session/store.py`, `harness/tht/decisions.py`, `harness/tht/phase.py`, `harness/tht/taskdoc.py`, `harness/tht/ctetest.py`, `harness/tht/solved.py`, `harness/tht/teardown.py`, `harness/tht/cli/{session,decision,cte,sql,phase,search,memory}_cmd.py`, `harness/.pi/skills/tht-sessione/SKILL.md`, `harness/.pi/extensions/tht-gate.js` +- Test: `harness/tests/test_session_repository_workflow.py`, `harness/gate/__tests__/session-repository-writes.test.js` + +**Interfaces:** Existing workflow commands operate through the configured repository. Add `tht cte save --session ID --name NAME --file -` and `tht sql set-final --session ID --file -`; Pi tools `write_cte_sql` and `write_final_sql` use them instead of direct session paths. + +- [ ] Write failing tests that run phase calculation and finalize against an in-memory/filesystem repository, and prove Pi write tools persist CTE/final SQL without direct `sessions/` writes. +- [ ] Run focused tests and record RED results. +- [ ] Refactor path-bound helpers into snapshot/ledger and repository calls; retain backwards-compatible readable local documents and pristine `--json` stdout. +- [ ] Make finalization atomically store report/evidence/status only after DWH verification; leave solved-question creation best-effort. +- [ ] Run harness and gate test suites, then commit `refactor(harness): route workflow persistence through repositories`. + +### Task 4: Trusted portal identity and proxy forwarding + +**Files:** +- Modify: `/home/chirone/omics_portal/kokoro/datamart_catalog_views.py`, `/home/chirone/omics_portal/nginx/nginx.conf` +- Create: `/home/chirone/omics_portal/kokoro/test_thothii_auth.py` + +**Interfaces:** The auth-request response supplies only `X-Thoth-Principal-Issuer`, `X-Thoth-Principal-Subject`, `X-Thoth-Principal-Display-Name`, and `X-Thoth-Is-Admin`. nginx removes client values for those headers and forwards subrequest values to ThothII. + +- [ ] Write Django tests for authenticated capability user, denied user, Django PK subject, admin group flag, and absent/spoofed client headers. +- [ ] Run the focused Docker test and confirm RED. +- [ ] Emit normalized headers from the capability endpoint and configure nginx auth-request header capture/injection. +- [ ] Run `docker compose exec -T web python manage.py test accounts.test_capabilities kokoro.test_thothii_auth -v 2`. +- [ ] Commit the portal changes in its own repository with `feat(thothii): forward trusted principal`. + +### Task 5: Backend principal enforcement and per-user settings + +**Files:** +- Create: `backend/src/auth/principal.ts` +- Modify: `backend/src/auth/auth.ts`, `backend/src/config.ts`, `backend/src/app.ts`, `backend/src/routes/{sessions,settings,sql,meta}.ts`, `backend/src/tht/tht-runner.ts`, `backend/src/pi/pi-process-manager.ts` +- Test: `backend/test/auth.test.ts`, `backend/test/routes-sessions.test.ts`, `backend/test/sse-route.test.ts`, `backend/test/routes-settings.test.ts` + +**Interfaces:** Add `GET /me`; require a `PrincipalContext` for all session, document, SQL, response, steer and SSE operations. `GET /sessions?scope=mine|all` permits `all` only for admins. Local mode creates `~/.thothii/identity.json`; upstream mode accepts only proxy-injected normalized headers. + +- [ ] Write failing route tests for missing identity (401), foreign session (404), admin all-scope, owner assignment server-side, SSE denial before Pi spawn, and preference isolation. +- [ ] Run `cd backend && npx vitest run test/auth.test.ts test/routes-sessions.test.ts test/sse-route.test.ts test/routes-settings.test.ts`; confirm RED. +- [ ] Implement principal parser, local identity, upstream validation, repository-aware `ThtRunner`, session authorization before Pi, `/me`, scope enforcement and async per-user settings. +- [ ] Run backend Vitest, typecheck, and build; commit `feat(backend): enforce user-owned sessions`. + +### Task 6: Frontend identity and administrator UX + +**Files:** +- Modify: `frontend/src/api/{client,sessions,settings,types}.ts`, `frontend/src/shell/{AppShell,NavSessions,SessionMenu}.tsx` +- Test: `frontend/src/api/sessions.test.ts`, `frontend/src/shell/AppShell.session-mgmt.test.tsx`, `frontend/src/shell/NavSessions.test.tsx` + +**Interfaces:** Fetch `/me`; default to `scope=mine`. Display explicit `All sessions` only to admins, show owner labels/admin banner, and require confirmation before cross-owner destructive actions. + +- [ ] Write failing component/API tests for regular-user scope, admin scope switch, owner label, admin banner, and cross-owner confirmation. +- [ ] Run focused Vitest tests and confirm RED. +- [ ] Implement typed client calls, session scope state, and explicit admin affordances without changing ordinary user flow. +- [ ] Run frontend Vitest, `npx tsc -b`, build and E2E; commit `feat(frontend): expose owned session scopes`. + +### Task 7: Deployment contract, migration and cutover tooling + +**Files:** +- Modify: `docker/`, deployment examples, `README.md`, `PROJECT_STATE.md` +- Test: `harness/tests/test_session_migrate_cmd.py`, `backend/test/config.test.ts` + +**Interfaces:** Define runtime/migrator DB secret names, `AUTH_MODE=upstream` server configuration, local loopback-only configuration, and health/readiness behavior returning 503 when repository storage is unavailable. + +- [ ] Write failing config tests for required server database/TLS inputs and rejected public/local combinations. +- [ ] Run focused tests and confirm RED. +- [ ] Document runtime/migrator roles, CA/secret injection, backup then deletion of the three legacy server sessions, maintenance-mode cutover, and no dual-write rollback policy. +- [ ] Run all three layer gates; commit `docs(deploy): document user-owned session cutover`. + +### Task 8: End-to-end authorization verification and final review + +**Files:** +- Modify: test fixtures only as required by earlier tasks. + +- [ ] Add cross-layer tests for A/B/admin isolation, spoofed-header rejection, concurrent session mutation, local two-home isolation, absent embeddings, and failed DB startup before Pi. +- [ ] Run harness, backend and frontend complete gates plus portal Docker tests. +- [ ] Run the final whole-branch review, fix all Critical and Important findings, and re-run the covering tests. +- [ ] Commit `test: cover user-owned session security boundaries` and prepare the branch for integration. diff --git a/docs/superpowers/specs/2026-07-16-user-owned-session-storage-design.md b/docs/superpowers/specs/2026-07-16-user-owned-session-storage-design.md new file mode 100644 index 00000000..29e3afdc --- /dev/null +++ b/docs/superpowers/specs/2026-07-16-user-owned-session-storage-design.md @@ -0,0 +1,25 @@ +# User-owned session storage design + +## Decisioni vincolanti + +- In server mode ThothII usa PostgreSQL diretto, nello schema Supabase privato + `thoth_sessions`; il browser non accede mai al database. +- In local mode ogni utente usa `~/.thothii` (override esplicito `THT_HOME`), + senza fallback o sincronizzazione con Supabase. +- Una sessione appartiene al principal `(issuer, subject)`. Nel portale il + subject è il PK Django; email e username non sono identificatori. +- Il proxy valida la sessione del portale e inietta identità normalizzata; il + backend richiede `AUTH_MODE=upstream` in produzione e non riceve token raw. +- Gli utenti nel gruppo Authentik `authentik Admins` possono gestire ogni + sessione. Un accesso non autorizzato restituisce 404. +- Lo schema contiene principals, principal_preferences, sessions, + session_artifacts, review_decisions e audit_log. Artefatti correnti e + decision ledger append-only; niente cronologia di artefatti né contenuto + nell'audit. +- La sicurezza server combina RLS forzata, un runtime role senza BYPASSRLS, + contesto attore transaction-local e filtri applicativi espliciti. +- Non vengono creati embeddings o vector columns per le sessioni. La memoria + solved_question esistente resta un flusso separato best-effort. +- Sessioni e preferenze server sono persistite solo nel DB; guasti DB sono + fail-closed (503). I file temporanei di export sono effimeri. +- Chat e SSE restano memoria runtime, non artefatti persistiti.