docs: add user-owned session storage plan
This commit is contained in:
User
@@ -0,0 +1,130 @@
|
||||
# User-owned sessions implementation plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Associate every ThothII session and preference with a stable logged-in or local OS principal.
|
||||
|
||||
**Architecture:** The harness owns a repository contract. `FilesystemSessionRepository` stores readable phase documents under the local user home; `PostgresSessionRepository` stores the same logical snapshot in a private Supabase schema. The backend resolves a trusted principal before every action and passes it through to `tht`; the portal proxy supplies that identity.
|
||||
|
||||
**Tech Stack:** Python 3.12, Typer, SQLAlchemy/psycopg2, PostgreSQL/Supabase RLS, Fastify/TypeScript, React/Vitest, Django/nginx.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Server storage is PostgreSQL wire protocol with TLS verification, schema `thoth_sessions`; never PostgREST or browser DB access.
|
||||
- Server identity is `(issuer, Django request.user.pk)`; clients never choose session owner.
|
||||
- Admin authorization uses Authentik group `authentik Admins`; unauthorized and missing session both return HTTP 404.
|
||||
- Server has no persistent session filesystem fallback or dual write; DB failure is HTTP 503 before Pi starts.
|
||||
- Local mode uses `~/.thothii` or `THT_HOME`, runs loopback-only, and creates a UUID identity in `identity.json`.
|
||||
- Persist current artifacts and append-only decisions only; no chat transcript, artifact revisions, session embeddings, or content in audit logs.
|
||||
- New session IDs are UUIDv4. Existing `solved_question` behavior is unchanged.
|
||||
- Follow TDD: each behavior test must fail before its implementation is written.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Session repository contract and local implementation
|
||||
|
||||
**Files:**
|
||||
- Create: `harness/tht/session/repository.py`, `harness/tht/session/filesystem_repository.py`
|
||||
- Modify: `harness/tht/config.py`, `harness/tht/session/store.py`, `harness/tht/session/models.py`, `harness/pyproject.toml`
|
||||
- Test: `harness/tests/test_session_repository.py`, `harness/tests/test_local_identity.py`
|
||||
|
||||
**Interfaces:** Produce `PrincipalContext`, `SessionSnapshot`, `SessionRepository`, and `build_session_repository(config, principal)`. The filesystem adapter must preserve the present documents under `<THT_HOME>/workspaces/<workspace>/sessions/<uuid4>` and read/write artifact keys, decisions, manifest fields, and per-principal preferences.
|
||||
|
||||
- [ ] Write failing tests proving UUIDv4 creation, principal-scoped local roots, readable phase artifacts, decision append, `THT_HOME` override, and an identity UUID stable across restarts.
|
||||
- [ ] Run `cd harness && .venv/bin/pytest tests/test_session_repository.py tests/test_local_identity.py -q`; confirm failure because repository and identity interfaces do not exist.
|
||||
- [ ] Implement the smallest repository contract and filesystem adapter. Use `portalocker` for mutations and private local directory/file permissions where supported.
|
||||
- [ ] Run the focused tests and `cd harness && .venv/bin/pytest -q && .venv/bin/ruff check .`.
|
||||
- [ ] Commit `feat(harness): add local session repository`.
|
||||
|
||||
### Task 2: PostgreSQL schema, migrations, and repository
|
||||
|
||||
**Files:**
|
||||
- Create: `harness/tht/session/postgres_repository.py`, `harness/tht/migrations/sessions/001_schema.sql`, `harness/tht/migrations/sessions/002_security.sql`
|
||||
- Modify: `harness/tht/cli/session_cmd.py`, `harness/tht/config.py`
|
||||
- Test: `harness/tests/test_postgres_session_repository.py`, `harness/tests/test_session_migrate_cmd.py`
|
||||
|
||||
**Interfaces:** Add `tht session migrate --database-url URL [--status] --json`; `PostgresSessionRepository` implements the Task 1 contract, starts a transaction, sets actor/admin local settings, and maps `cte_sql:<name>` to artifacts.
|
||||
|
||||
- [ ] Write failing unit/integration tests for migration status, owner isolation, admin cross-user access, cascade delete with content-free audit tombstone, and no embedding invocation.
|
||||
- [ ] Run the focused tests and confirm expected RED failures.
|
||||
- [ ] Implement idempotent migration runner, private tables, forced RLS policies, role separation, advisory transaction locks, and repository methods using direct PostgreSQL TLS settings.
|
||||
- [ ] Run focused tests, then full harness tests and Ruff.
|
||||
- [ ] Commit `feat(harness): persist server sessions in postgres`.
|
||||
|
||||
### Task 3: Migrate harness workflow and deterministic write commands
|
||||
|
||||
**Files:**
|
||||
- Modify: `harness/tht/session/store.py`, `harness/tht/decisions.py`, `harness/tht/phase.py`, `harness/tht/taskdoc.py`, `harness/tht/ctetest.py`, `harness/tht/solved.py`, `harness/tht/teardown.py`, `harness/tht/cli/{session,decision,cte,sql,phase,search,memory}_cmd.py`, `harness/.pi/skills/tht-sessione/SKILL.md`, `harness/.pi/extensions/tht-gate.js`
|
||||
- Test: `harness/tests/test_session_repository_workflow.py`, `harness/gate/__tests__/session-repository-writes.test.js`
|
||||
|
||||
**Interfaces:** Existing workflow commands operate through the configured repository. Add `tht cte save --session ID --name NAME --file -` and `tht sql set-final --session ID --file -`; Pi tools `write_cte_sql` and `write_final_sql` use them instead of direct session paths.
|
||||
|
||||
- [ ] Write failing tests that run phase calculation and finalize against an in-memory/filesystem repository, and prove Pi write tools persist CTE/final SQL without direct `sessions/<id>` writes.
|
||||
- [ ] Run focused tests and record RED results.
|
||||
- [ ] Refactor path-bound helpers into snapshot/ledger and repository calls; retain backwards-compatible readable local documents and pristine `--json` stdout.
|
||||
- [ ] Make finalization atomically store report/evidence/status only after DWH verification; leave solved-question creation best-effort.
|
||||
- [ ] Run harness and gate test suites, then commit `refactor(harness): route workflow persistence through repositories`.
|
||||
|
||||
### Task 4: Trusted portal identity and proxy forwarding
|
||||
|
||||
**Files:**
|
||||
- Modify: `/home/chirone/omics_portal/kokoro/datamart_catalog_views.py`, `/home/chirone/omics_portal/nginx/nginx.conf`
|
||||
- Create: `/home/chirone/omics_portal/kokoro/test_thothii_auth.py`
|
||||
|
||||
**Interfaces:** The auth-request response supplies only `X-Thoth-Principal-Issuer`, `X-Thoth-Principal-Subject`, `X-Thoth-Principal-Display-Name`, and `X-Thoth-Is-Admin`. nginx removes client values for those headers and forwards subrequest values to ThothII.
|
||||
|
||||
- [ ] Write Django tests for authenticated capability user, denied user, Django PK subject, admin group flag, and absent/spoofed client headers.
|
||||
- [ ] Run the focused Docker test and confirm RED.
|
||||
- [ ] Emit normalized headers from the capability endpoint and configure nginx auth-request header capture/injection.
|
||||
- [ ] Run `docker compose exec -T web python manage.py test accounts.test_capabilities kokoro.test_thothii_auth -v 2`.
|
||||
- [ ] Commit the portal changes in its own repository with `feat(thothii): forward trusted principal`.
|
||||
|
||||
### Task 5: Backend principal enforcement and per-user settings
|
||||
|
||||
**Files:**
|
||||
- Create: `backend/src/auth/principal.ts`
|
||||
- Modify: `backend/src/auth/auth.ts`, `backend/src/config.ts`, `backend/src/app.ts`, `backend/src/routes/{sessions,settings,sql,meta}.ts`, `backend/src/tht/tht-runner.ts`, `backend/src/pi/pi-process-manager.ts`
|
||||
- Test: `backend/test/auth.test.ts`, `backend/test/routes-sessions.test.ts`, `backend/test/sse-route.test.ts`, `backend/test/routes-settings.test.ts`
|
||||
|
||||
**Interfaces:** Add `GET /me`; require a `PrincipalContext` for all session, document, SQL, response, steer and SSE operations. `GET /sessions?scope=mine|all` permits `all` only for admins. Local mode creates `~/.thothii/identity.json`; upstream mode accepts only proxy-injected normalized headers.
|
||||
|
||||
- [ ] Write failing route tests for missing identity (401), foreign session (404), admin all-scope, owner assignment server-side, SSE denial before Pi spawn, and preference isolation.
|
||||
- [ ] Run `cd backend && npx vitest run test/auth.test.ts test/routes-sessions.test.ts test/sse-route.test.ts test/routes-settings.test.ts`; confirm RED.
|
||||
- [ ] Implement principal parser, local identity, upstream validation, repository-aware `ThtRunner`, session authorization before Pi, `/me`, scope enforcement and async per-user settings.
|
||||
- [ ] Run backend Vitest, typecheck, and build; commit `feat(backend): enforce user-owned sessions`.
|
||||
|
||||
### Task 6: Frontend identity and administrator UX
|
||||
|
||||
**Files:**
|
||||
- Modify: `frontend/src/api/{client,sessions,settings,types}.ts`, `frontend/src/shell/{AppShell,NavSessions,SessionMenu}.tsx`
|
||||
- Test: `frontend/src/api/sessions.test.ts`, `frontend/src/shell/AppShell.session-mgmt.test.tsx`, `frontend/src/shell/NavSessions.test.tsx`
|
||||
|
||||
**Interfaces:** Fetch `/me`; default to `scope=mine`. Display explicit `All sessions` only to admins, show owner labels/admin banner, and require confirmation before cross-owner destructive actions.
|
||||
|
||||
- [ ] Write failing component/API tests for regular-user scope, admin scope switch, owner label, admin banner, and cross-owner confirmation.
|
||||
- [ ] Run focused Vitest tests and confirm RED.
|
||||
- [ ] Implement typed client calls, session scope state, and explicit admin affordances without changing ordinary user flow.
|
||||
- [ ] Run frontend Vitest, `npx tsc -b`, build and E2E; commit `feat(frontend): expose owned session scopes`.
|
||||
|
||||
### Task 7: Deployment contract, migration and cutover tooling
|
||||
|
||||
**Files:**
|
||||
- Modify: `docker/`, deployment examples, `README.md`, `PROJECT_STATE.md`
|
||||
- Test: `harness/tests/test_session_migrate_cmd.py`, `backend/test/config.test.ts`
|
||||
|
||||
**Interfaces:** Define runtime/migrator DB secret names, `AUTH_MODE=upstream` server configuration, local loopback-only configuration, and health/readiness behavior returning 503 when repository storage is unavailable.
|
||||
|
||||
- [ ] Write failing config tests for required server database/TLS inputs and rejected public/local combinations.
|
||||
- [ ] Run focused tests and confirm RED.
|
||||
- [ ] Document runtime/migrator roles, CA/secret injection, backup then deletion of the three legacy server sessions, maintenance-mode cutover, and no dual-write rollback policy.
|
||||
- [ ] Run all three layer gates; commit `docs(deploy): document user-owned session cutover`.
|
||||
|
||||
### Task 8: End-to-end authorization verification and final review
|
||||
|
||||
**Files:**
|
||||
- Modify: test fixtures only as required by earlier tasks.
|
||||
|
||||
- [ ] Add cross-layer tests for A/B/admin isolation, spoofed-header rejection, concurrent session mutation, local two-home isolation, absent embeddings, and failed DB startup before Pi.
|
||||
- [ ] Run harness, backend and frontend complete gates plus portal Docker tests.
|
||||
- [ ] Run the final whole-branch review, fix all Critical and Important findings, and re-run the covering tests.
|
||||
- [ ] Commit `test: cover user-owned session security boundaries` and prepare the branch for integration.
|
||||
@@ -0,0 +1,25 @@
|
||||
# User-owned session storage design
|
||||
|
||||
## Decisioni vincolanti
|
||||
|
||||
- In server mode ThothII usa PostgreSQL diretto, nello schema Supabase privato
|
||||
`thoth_sessions`; il browser non accede mai al database.
|
||||
- In local mode ogni utente usa `~/.thothii` (override esplicito `THT_HOME`),
|
||||
senza fallback o sincronizzazione con Supabase.
|
||||
- Una sessione appartiene al principal `(issuer, subject)`. Nel portale il
|
||||
subject è il PK Django; email e username non sono identificatori.
|
||||
- Il proxy valida la sessione del portale e inietta identità normalizzata; il
|
||||
backend richiede `AUTH_MODE=upstream` in produzione e non riceve token raw.
|
||||
- Gli utenti nel gruppo Authentik `authentik Admins` possono gestire ogni
|
||||
sessione. Un accesso non autorizzato restituisce 404.
|
||||
- Lo schema contiene principals, principal_preferences, sessions,
|
||||
session_artifacts, review_decisions e audit_log. Artefatti correnti e
|
||||
decision ledger append-only; niente cronologia di artefatti né contenuto
|
||||
nell'audit.
|
||||
- La sicurezza server combina RLS forzata, un runtime role senza BYPASSRLS,
|
||||
contesto attore transaction-local e filtri applicativi espliciti.
|
||||
- Non vengono creati embeddings o vector columns per le sessioni. La memoria
|
||||
solved_question esistente resta un flusso separato best-effort.
|
||||
- Sessioni e preferenze server sono persistite solo nel DB; guasti DB sono
|
||||
fail-closed (503). I file temporanei di export sono effimeri.
|
||||
- Chat e SSE restano memoria runtime, non artefatti persistiti.
|
||||
Reference in New Issue
Block a user