diff --git a/.superpowers/sdd/task-2-report.md b/.superpowers/sdd/task-2-report.md index f9061d8c..404a87ab 100644 --- a/.superpowers/sdd/task-2-report.md +++ b/.superpowers/sdd/task-2-report.md @@ -28,3 +28,150 @@ git diff --check The local-vector and preprocess service secret declarations remain for Task 3, which converts those services to the same bundle helper. Documentation and smoke command migration is reserved for Task 4. + +--- + +# Task 2 report — PostgreSQL session repository + +## Scope delivered + +- Added `PostgresSessionRepository`, implementing the Task 1 repository contract with a + direct PostgreSQL SQLAlchemy connection, transaction-local RLS context, UUIDv4 validation, + current artifacts (including `cte_sql:`), append-only decisions, preferences, and + content-free deletion tombstones. +- Added `tht session migrate --database-url URL [--status] --json` and a checksum-protected, + advisory-transaction-locked migration runner. +- Added server session configuration selection. `session_storage.connection` uses direct + PostgreSQL TLS modes `verify-ca` or `verify-full`; it does not use PostgREST. +- Updated packaging and `.gitignore` so session migrations are present in the built wheel. +- Did not alter Task 3 workflow commands, Pi gate code, or backend code. + +## TDD evidence + +### RED + +Command: + +```sh +cd harness && .venv/bin/pytest tests/test_postgres_session_repository.py tests/test_session_migrate_cmd.py -q +``` + +Result before production implementation: `1 failed, 4 errors in 3.89s`. + +- Four setup errors were `ModuleNotFoundError: No module named + 'tht.session.postgres_repository'`. +- The migration CLI test failed because `tht session migrate` did not exist (`No such command + 'migrate'`). + +### GREEN + +Initial focused suite after implementation: `5 passed in 4.18s`. + +Final focused verification: + +```sh +cd harness && .venv/bin/pytest \ + tests/test_session_repository.py \ + tests/test_postgres_session_repository.py \ + tests/test_session_migrate_cmd.py \ + tests/test_vector_migration_packaging.py -q +``` + +Result: `12 passed in 5.80s`. + +Changed-file lint verification: + +```sh +cd harness && .venv/bin/ruff check \ + tht/session/postgres_repository.py tht/migrations/sessions tht/config.py \ + tht/session/repository.py tht/cli/session_cmd.py \ + tests/test_postgres_session_repository.py tests/test_session_migrate_cmd.py \ + tests/test_vector_migration_packaging.py +``` + +Result: `All checks passed!`. + +## Migration and role policy choices + +`001_schema.sql` creates only private `thoth_sessions` tables: + +- `principals` and `principal_preferences`; +- `sessions`, with `session_artifacts` and `review_decisions` cascading on session deletion; +- `audit_log`, which deliberately has no content/detail/metadata column and keeps only action, + session UUID, actor identity, owner identity, and timestamp. + +`002_security.sql` creates separate `thoth_sessions_runtime` and +`thoth_sessions_migrator` group roles, explicitly `NOLOGIN NOBYPASSRLS NOSUPERUSER`, revokes +public access, gives the runtime role only the operations required by the adapter, and enables +and forces RLS on every table. Owner/admin policies read only transaction-local settings: +`thoth_sessions.actor_issuer`, `thoth_sessions.actor_subject`, and +`thoth_sessions.is_admin`. The adapter starts every operation in a transaction, switches to the +restricted runtime role, sets those settings with `set_config(..., true)`, and uses advisory +transaction locks for migrations and per-session mutations. + +The runtime role remains a `NOLOGIN` group role by design. Deployment must provision a dedicated +non-superuser LOGIN role and grant it membership, for example: + +```sql +CREATE ROLE thoth_sessions_app LOGIN NOINHERIT PASSWORD ''; +GRANT thoth_sessions_runtime TO thoth_sessions_app; +``` + +This avoids embedding an environment-specific login name or credential in versioned SQL. The +new integration test proves that this non-superuser membership path can create and read a +session while the adapter executes as `thoth_sessions_runtime`. + +## Security/self-review + +- Owner isolation and admin cross-owner reads run against disposable PostgreSQL containers, + not Supabase. +- No table or column includes `embedding`; repository code imports no embedding/vector code; + the regression test writes a session artifact under a monkeypatched embedding sentinel. +- An unauthorized owner receives the same `SessionError` as an absent session, preserving the + future backend's 404 mapping boundary. +- The audit row is inserted before deleting the parent session, so cascades remove all artifact + and decision content while the tombstone survives. +- A security review found and this task fixed the initial `.gitignore` rule that would have + excluded `migrations/sessions/*.sql` from Git/wheels. The wheel test now asserts both session + migration files and checks both the existing vector CLI and the new session CLI. +- The review also highlighted runtime login provisioning. It is covered by a non-superuser + regression test and documented above; concrete credential/role deployment belongs to Task 7. + +## Remaining concerns + +- Full `harness/.venv/bin/pytest -q` could not complete in this execution environment: the + runner terminated the command after roughly 30 seconds. Captured output reached 44% with no + failures before termination; `pgrep` confirmed no pytest process remained. The Task 2 focused + suites above completed successfully. +- `harness/.venv/bin/ruff check .` currently reports 34 pre-existing violations in unrelated + test files (for example unused imports in `tests/l0/test_db_connection.py` and semicolon style + in `tests/test_phase_effective.py`). The changed-file Ruff command is clean. +- Task 7 must safely provision the dedicated runtime login/membership and inject its TLS + credentials/CA; this task intentionally does not create a deployment-specific LOGIN role or + password. + +## Review follow-up — unavailable migration database JSON contract + +### RED + +Command: + +```sh +cd harness && .venv/bin/pytest \ + tests/test_session_migrate_cmd.py::test_session_migrate_status_database_failure_is_pristine_json -q +``` + +Result: `1 failed in 0.46s`. The unreachable direct PostgreSQL URL exited with code 1 but left +stdout empty, so `json.loads(result.stdout)` raised `JSONDecodeError`. + +### GREEN + +The session migration CLI now catches `SQLAlchemyError` at the same command boundary as its +migration/domain errors and emits only `{"error": ...}` on stdout for `--json`. + +```sh +cd harness && .venv/bin/pytest tests/test_session_migrate_cmd.py -q +cd harness && .venv/bin/ruff check tht/cli/session_cmd.py tests/test_session_migrate_cmd.py +``` + +Result: `2 passed in 3.60s`; Ruff: `All checks passed!`. diff --git a/.superpowers/sdd/task-3-report.md b/.superpowers/sdd/task-3-report.md index 0228878d..5fb7ae6d 100644 --- a/.superpowers/sdd/task-3-report.md +++ b/.superpowers/sdd/task-3-report.md @@ -1,65 +1,56 @@ -# Task 3 report — reconnect SSE on same-session Resume +# Task 3 report — workflow repository migration -## Status +## RED -Complete. A successful Resume of the currently active session now replaces its existing -`EventSource` connection. Resuming a different session continues to reconnect through the -session ID change only, without a generation-driven second connection. +- `harness/tests/test_session_repository_workflow.py` initially failed at collection: + `persist_verified_finalization` did not exist. +- The new gate test initially failed because `write_cte_sql` and `write_final_sql` + were not registered. Its first run also exposed the worktree-local missing + Node dependency (`typebox`); `npm ci` installed the lockfile dependency. +- After the principal/legacy policy was clarified, the resolver tests initially + failed because `resolve_principal` did not exist. -## Implementation +## GREEN evidence -- `useSessionStream` accepts an optional `generation` argument (default `0`) and includes it - in the stream effect dependencies. A generation change therefore runs the existing cleanup, - closes the old source, and opens the same URL again. -- `AppShell` captures whether the requested Resume ID is already active before its existing - optimistic state updates. It increments the stream generation only after `resumeSession(id)` - succeeds and only for that same-ID case. -- The existing optimistic session switch, phase refresh, and failed-Resume rollback remain - unchanged. A failed POST cannot increment the generation. +- Focused Python regression set: `66 passed`: + `test_session_repository_workflow`, `test_session_repository`, session mutation/list/ + documents/schema-linking, CTE plan/next, decision phase gate, and phase requirement tests. +- Gate suite: `127 passed`, including + `session-repository-writes.test.js`. +- Changed-source Ruff checks pass. `git diff --check` passes. -## TDD evidence +## Implemented boundary -- RED command: - `cd frontend && npx vitest run src/stream/useSessionStream.test.tsx src/shell/AppShell.session-mgmt.test.tsx` -- RED result: 2 expected failures and 15 passes. The hook test observed - `first.closed === false`; the AppShell test observed one `FakeEventSource` instead of two - after the second same-ID Resume. -- GREEN focused result: the same command passed 2/2 files and 17/17 tests after the minimal - production wiring. +- Added `resolve_principal`: PostgreSQL session storage requires trusted + `THT_PRINCIPAL_ISSUER` and `THT_PRINCIPAL_SUBJECT`, optional display name, and + strict admin parsing (`1`/`true`). It fails closed and never substitutes a local + identity. Filesystem storage uses `local_principal()`. +- Filesystem repository creates UUIDv4 sessions only and permits safe historical + timestamp IDs (`YYYY-MM-DD-HHMMSS`) for read/mutate compatibility. PostgreSQL + remains UUIDv4 only. +- Phase helpers fold `SessionSnapshot` ledger/artifacts; decision, phase, CTE, + session mutation/list/document paths, retrieval-pack persistence, SQL promotion + lookup, and task-doc/CTE test helpers gained repository/snapshot paths. +- Finalization now publishes report, evidence, and finalized manifest through + `repository.finalize`: one PostgreSQL transaction; filesystem writes artifacts + before the finalized manifest commit marker. Solved-question indexing stays + best-effort after this durable write. +- Added `tht cte save --session --name --file -` and + `tht sql set-final --session --file -`; Pi tools and SKILL.md now use them. -## Full verification +## Outstanding in-scope migration work -- Baseline before edits: `cd frontend && npx vitest run` — 42/42 files and 251/251 tests passed. -- Focused tests: 2/2 files and 17/17 tests passed. -- Full frontend suite: `cd frontend && npx vitest run` — 42/42 files and 253/253 tests passed. -- Typecheck: `cd frontend && npx tsc -b` — exit 0. -- Production build: `cd frontend && npm run build` — exit 0; Vite transformed 4,835 modules - and completed the production bundle. -- `git diff --check` — passed. +Do not treat this task as complete yet. Remaining direct session path consumers are: -The suite and build retained the pre-existing MSW unhandled-request, React ref/`act`, Node type -stripping, and Vite chunk-size warnings. This task introduced no new warning category. +- `harness/tht/cli/memory_cmd.py`: lines 60, 93, 165, 400, 458. +- `harness/tht/cli/sql_cmd.py`: `_session_sql_file` at line 254 remains a legacy + Path-returning bridge for preview/save/export. +- `harness/tht/cli/session_cmd.py:session_dir` remains only as a compatibility + bridge for the out-of-scope datamart command and the still-unmigrated memory/ + SQL consumers; workflow mutations in session_cmd do not call it. -## Files - -- `frontend/src/stream/useSessionStream.ts` -- `frontend/src/stream/useSessionStream.test.tsx` -- `frontend/src/shell/AppShell.tsx` -- `frontend/src/shell/AppShell.session-mgmt.test.tsx` -- `.superpowers/sdd/task-3-report.md` - -## Self-review - -- Confirmed the old EventSource is closed before the replacement is retained by React's effect - lifecycle, and the replacement uses the identical session URL. -- Confirmed same-ID detection happens before the optimistic `setActiveSessionId(id)` call. -- Confirmed the generation increments only after a successful Resume POST; the catch/rollback - branch is unchanged. -- Confirmed a different ID leaves the generation unchanged, so the existing session-ID effect - change creates exactly one replacement connection. -- Confirmed the diff is frontend-only apart from this report and contains no backend, Docker, - configuration, or session changes. - -## Concerns - -None. +The full Python suite has not been conclusively re-run to completion after the +latest changes. An earlier root-directory invocation failed only because a +pre-existing test expects `workflow.yaml` relative to `harness/`. Full gate tests +are green. Full-repo Ruff currently fails on pre-existing test-file lint findings; +changed-source Ruff passes. diff --git a/.superpowers/sdd/task-5-report.md b/.superpowers/sdd/task-5-report.md new file mode 100644 index 00000000..cebde76c --- /dev/null +++ b/.superpowers/sdd/task-5-report.md @@ -0,0 +1,71 @@ +# Task 5 report — backend principal enforcement + +## RED + +Added backend route/auth tests before implementation. The initial focused run failed in +seven new assertions: `getPrincipal` did not exist, upstream requests still required the +legacy identity header, foreign session/SSE routes were not hidden, admin scope was not +enforced, new sessions had no trusted principal binding, and settings were global. + +## GREEN + +- Focused backend suite: `66 passed` across auth, sessions, SSE, and settings tests. +- Complete backend Vitest suite: `209 passed` across `22` files. +- `npx tsc --noEmit -p .`, `npm run build`, `git diff --check`, and changed Python + source Ruff all exit successfully. +- Harness targeted repository/local/migration tests and Python bytecode compilation exit + successfully. The new `tht session preferences get|set` commands are registered and + expose the expected Typer help. A direct local CLI preference smoke was not run because + the checked-in local workspace requires unavailable `THT_DB_HOST` configuration. + +## Route and child-process coverage + +- `GET /me` returns the request `PrincipalContext`; upstream accepts only the portal's + normalized `X-Thoth-*` identity tuple, with the legacy header ignored. Local mode uses + the same stable `THT_HOME`/`~/.thothii/identity.json` UUID contract as the harness. +- All session operations are principal-scoped: list (`mine` and admin-only `all`), show, + create, resume, close, delete, rename, group, archive, unarchive, documents, reviewer + response, steer, SQL preview/export, and SSE. Missing and foreign sessions are 404; + absent upstream identity is 401. SSE is authorized before response headers or hub + subscription, so a rejected request cannot attach to a live stream. +- New/resumed Pi runtimes and every route-spawned `tht` process receive + `THT_PRINCIPAL_ISSUER`, `THT_PRINCIPAL_SUBJECT`, optional display name, and admin flag. + The readiness `tht` child is also principal-bound. +- Settings use asynchronous repository-backed `tht session preferences get|set` in the + production runner, which isolates preferences by principal. The legacy settings file is + retained only as an injected-runner compatibility fallback for existing isolated tests. +- Repository/settings authorization failures map to 503 before model startup. SQL execution + errors remain 500 after authorization, preserving the prior API distinction. + +## Self-review and concerns + +- Confirmed the Task 4 portal emits lowercase `true`/`false` for the admin header; the + parser accepts that exact normalized form plus the repository's existing `1`/`0` + compatibility form, and rejects all other values. +- The harness principal resolver is the ownership authority; the backend never accepts an + owner supplied in request bodies. Its route guards use a repository-scoped `session show` + before every session resource operation. +- Existing dependency-injected route fakes without `sessionShow` retain a narrow test seam; + production `ThtRunner` always has that method, so deployed requests cannot bypass the + repository authorization check. + +## Review follow-up + +### RED + +Focused regressions initially failed exactly at the three review findings: stale ambient +display names survived into both `tht` and Pi child environments; mutation/document runner +methods dropped the selected workspace; and `expandLocalHome` did not exist. + +### GREEN + +- Child environments now remove all four `THT_PRINCIPAL_*` keys from their cloned base + environment before applying the exact request principal. Regression tests prove an absent + display name does not inherit a stale ambient value in either child path. +- `setName`, `setGroup`, `archive`, `unarchive`, and `documents` now take and retain an + optional workspace. The rename route regression proves `session show` authorization and + the mutation use the same non-default workspace. +- Local principal paths expand `~`/`~/...`; existing local home and identity file modes are + repaired to POSIX `0700`/`0600` when applicable, with Windows left unchanged. +- Focused suite: `74 passed`; full backend suite: `213 passed` across `22` files, followed by + TypeScript typecheck, production build, and diff check. diff --git a/.superpowers/sdd/task-6-report.md b/.superpowers/sdd/task-6-report.md new file mode 100644 index 00000000..88b7af75 --- /dev/null +++ b/.superpowers/sdd/task-6-report.md @@ -0,0 +1,65 @@ +# Task 6 — Frontend identity and administrator UX report + +## RED + +- Added API tests for the `/me` principal call and `mine`/`all` session-list scopes. +- Added component tests for regular-user scope, admin scope switching, owner labels, + administrator banner, foreign-owner delete confirmation, and foreign-owner archive + confirmation. +- Initial focused run: 7 expected failures (missing `getMe`, missing scope query, + missing owner label/admin controls, and missing foreign-action confirmation). +- The archive-confirmation regression was also run separately before its implementation + and failed because `window.confirm` was not called. + +## GREEN + +- `npx vitest run src/api/sessions.test.ts src/shell/NavSessions.test.tsx src/shell/AppShell.session-mgmt.test.tsx` + — passed (47 tests before the archive follow-up; the focused archive regression then passed). +- `npm test` — passed: 44 files / 305 tests. +- `npx tsc -b` — passed. +- `npm run build` — passed. +- `git diff --check` — passed. +- `npm run e2e` reached Playwright but could not run: the environment has no Chromium + executable at Playwright's configured cache path. No application test failure was reported. + +## Files changed + +- `frontend/src/api/types.ts`: typed principal and session scope contracts. +- `frontend/src/api/sessions.ts`: typed `/me` API call; scoped listing defaults to `mine`. +- `frontend/src/shell/AppShell.tsx`: identity query, admin-only session scope selector and + banner, owner-aware destructive action confirmations. +- `frontend/src/shell/NavSessions.tsx`: owner labels in the all-sessions view. +- `frontend/src/api/sessions.test.ts`, `frontend/src/shell/NavSessions.test.tsx`, and + `frontend/src/shell/AppShell.session-mgmt.test.tsx`: contract and UX coverage. + +## Self-review + +- Regular users remain fail-closed on `mine`; no administrator control renders without + `principal.isAdmin`. +- The all-sessions view includes owner labels (including `Unknown` for legacy records). +- Delete confirmation preserves the pre-existing select-all behavior and adds confirmation + for foreign/unknown owners. Foreign archive now also requires an explicit browser + confirmation; existing Stop & save already has its confirmation dialog. +- A read-only review found no critical, important, or minor issues. The archive guard was + added after that review in response to the requirement to cover every destructive rail + action, and has its own RED/GREEN regression plus the final full verification above. + +## Concerns + +- E2E remains environment-blocked until the Playwright Chromium browser is installed. +- Existing Vitest runs emit pre-existing MSW unmatched-request and dialog-ref warnings; all + assertions pass and this task does not modify those shared test/UI primitives. + +## Review remediation + +- A post-commit review correctly identified that matching `displayName` must never establish + ownership. The predicate now skips confirmation only when `session.author` exactly equals + `principal.subject`; all display-name matches and missing authors are conservative + cross-owner actions. +- Added RED/GREEN regressions where two principals share display name `Alice` but have distinct + subjects: both delete (with another session present, so select-all cannot mask the guard) and + archive require confirmation. +- Added `aria-pressed` to the My sessions / All sessions controls and asserts their selected state + before and after switching. +- Remediation verification: focused regressions passed; full frontend Vitest (44 files / 305 + tests), `npx tsc -b`, `npm run build`, and `git diff --check` all passed. diff --git a/.superpowers/sdd/task-7-report.md b/.superpowers/sdd/task-7-report.md new file mode 100644 index 00000000..ca384282 --- /dev/null +++ b/.superpowers/sdd/task-7-report.md @@ -0,0 +1,79 @@ +# Task 7 report — deployment contract and user-owned-session cutover + +## Scope + +Implemented the deployment contract only. No Supabase migration, portal change, live-stack +restart, session archive, or deletion was run. + +- `backend/src/config.ts` now makes the session-store deployment mode explicit. `local` is the + default and cannot be publicly exposed. `postgres` requires `AUTH_MODE=upstream`, direct DB + host/name/runtime user, an absolute runtime-password file, `verify-ca` or `verify-full`, and an + absolute CA path. +- `docker-compose.dev.yml` now publishes only loopback ports and explicitly selects local + session storage rooted at `/data/local-home`. +- `deploy/compose.session-server.yaml.example` separates the runtime and one-shot migrator + secrets. The core gets only `session_runtime_password` and the CA; the profile-gated + `session-migrate` service gets only `session_migrator_password` and the CA. +- `deploy/workspaces/server-sessions.yaml.example` binds the runtime repository to the + TLS-verified direct PostgreSQL configuration. The runtime password remains a file reference. +- `docker/cutover-legacy-sessions.sh` archives/checksums exactly three reviewed legacy sessions + and requires an explicit `--delete` rerun before deleting them. +- README, secret guidance, environment examples, and PROJECT_STATE describe the maintenance + sequence, Task 4+5 coordinated rollout, liveness vs storage 503 behavior, and the no-dual-write + rollback rule. + +## TDD evidence + +RED was established with: + +```sh +cd backend && npx vitest run test/config.test.ts +``` + +The new tests failed because `sessionStorage` did not exist and public/local and unauthenticated +server combinations were accepted. After implementing the minimal configuration contract, the +same focused suite passed (7 tests). Updating the existing upstream-health fixture to supply the +now-required server inputs confirmed that `/health` remains an unauthenticated `200` liveness +endpoint under the valid server contract. + +## Verification + +```text +cd harness && .venv/bin/pytest -q +826 passed, 5 deselected, 67 warnings in 63.01s + +cd backend && npx vitest run && npx tsc --noEmit -p . && npm run build +22 files / 215 tests passed; TypeScript check and production build passed + +cd frontend && npx vitest run && npx tsc -b && npm run build +full Vitest suite, TypeScript build, and Vite production build passed +``` + +The frontend gate retained its pre-existing React-ref/MSW/act warnings and Vite chunk-size warning; +none caused a test or build failure. + +Additional static validation passed: + +```text +docker compose config --quiet (base plus copied session-server overlay with temporary empty secrets) +bash -n docker/cutover-legacy-sessions.sh +git diff --check +``` + +## Manual gate remaining + +An operator must still choose the three reviewed legacy IDs, materialize real runtime/migrator/CA +secrets, deploy Task 4 and Task 5 together in a maintenance window, apply the one-shot migrator, +and run the documented authenticated smoke. The guarded helper has not been invoked with +`--delete`. + +## P1 correction — migrator TLS validation + +The original migrator Compose command interpolated `THT_SESSION_DB_SSLMODE` into its URL without +checking it. `docker/session-migrate.sh` now rejects every value except `verify-ca` and +`verify-full` before reading the password file or building that URL; the Compose service invokes +this helper. `docker/session-migrate.test.sh` first established RED because the helper did not +exist, then verified that `prefer` is rejected before `tht` can run and that `verify-full` reaches +a fake `tht` binary with the expected TLS URL. The helper and test pass `bash -n`; the focused +backend config/health suite remains green, and the base-plus-overlay Compose configuration renders +with temporary empty secret files. diff --git a/PROJECT_STATE.md b/PROJECT_STATE.md index f3a25690..3597c581 100644 --- a/PROJECT_STATE.md +++ b/PROJECT_STATE.md @@ -3,6 +3,28 @@ > Starting-point snapshot for new sessions. Last updated: 2026-07-15 (central live log and compact CTE density live). > Point a fresh session here ("read PROJECT_STATE.md") before substantial work. +## User-owned sessions cutover — prepared, manual gate pending (2026-07-16) + +- **Target contract:** the public server runs `AUTH_MODE=upstream` with Task 4 portal identity + forwarding and Task 5 principal enforcement deployed together. Its session source of truth is + direct TLS-verified PostgreSQL `thoth_sessions`; local development remains loopback-only with + filesystem sessions under `THT_HOME`. The core never receives the migrator credential. +- **Deployment material:** copy `deploy/compose.session-server.yaml.example` and + `deploy/workspaces/server-sessions.yaml.example` into reviewed, untracked operator files. The + runtime password, migrator password, and CA are three separate Docker secret mounts; server + startup rejects public/local storage and incomplete server DB/TLS configuration. +- **Readiness behavior:** `/health` remains the unauthenticated process liveness endpoint. Any + route requiring unavailable session/preferences storage returns fixed HTTP 503 before starting + Pi; this is intentional and must not be hidden by changing liveness to a database check. +- **Manual cutover only:** schedule maintenance, drain Pi work, run the one-shot migrator and + require `pending=[]` and `drifted=[]`, then replace core and perform an authenticated storage + smoke. Archive/checksum the three reviewed legacy filesystem session directories before deleting + exactly those three with `docker/cutover-legacy-sessions.sh --delete`; no deletion has been run + from this repository task. Do not import their untrusted ownership. +- **Rollback:** PostgreSQL remains the single source of truth. Revert only to a compatible fixed + release; never re-enable filesystem persistence, restore the archive into production, or + dual-write during rollback. + ## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12 ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale). diff --git a/README.md b/README.md index 8059d471..7d23e7fa 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,11 @@ The frontend depends on the core health check and proxies `/health` and `/api/*` application health endpoint intentionally checks process readiness only; external dependency diagnostics are exposed by `tht doctor` and do not prevent the UI from starting. +`docker-compose.dev.yml` is deliberately local: both published ports bind to `127.0.0.1`, +`THT_SESSION_STORAGE=local`, and `THT_HOME=/data/local-home`. Do not set +`THOTH_PUBLIC_EXPOSURE=true` for that profile; the backend rejects that public/local combination +at startup. + Run the end-to-end packaging check with: ```sh @@ -192,6 +197,78 @@ Compound providers are deliberately unsupported: `amazon-bedrock`, `azure-openai values. Selecting one fails before Pi starts; ambient AWS, Azure, and Cloudflare credentials are still scrubbed. Supporting them requires a future dedicated provider-specific configuration. +## User-owned session server cutover + +The server profile stores sessions and per-user preferences directly in PostgreSQL schema +`thoth_sessions`; it does not use PostgREST, browser storage, a shared session directory, or a +dual write. Start from [`deploy/compose.session-server.yaml.example`](deploy/compose.session-server.yaml.example) +and copy [`deploy/workspaces/server-sessions.yaml.example`](deploy/workspaces/server-sessions.yaml.example) +to the untracked `deploy/workspaces/server-sessions.yaml` mounted into the core container. + +The runtime login needs membership in the no-login database role `thoth_sessions_runtime` only. +The distinct, one-shot migrator login needs migration authority and uses +`thoth_sessions_migrator`; it must never be mounted into `core`. Set the non-secret endpoint and +role fields in the protected deployment environment: + +```dotenv +AUTH_MODE=upstream +THOTH_PUBLIC_EXPOSURE=true +THT_SESSION_STORAGE=postgres +THT_SESSION_DB_HOST=sessions-db.internal +THT_SESSION_DB_PORT=5432 +THT_SESSION_DB_NAME=thoth +THT_SESSION_RUNTIME_USER=thoth_sessions_app +THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate +THT_SESSION_DB_SSLMODE=verify-full +THT_SESSION_RUNTIME_PASSWORD_SOURCE=/secure/thoth/session-runtime-password +THT_SESSION_MIGRATOR_PASSWORD_SOURCE=/secure/thoth/session-migrator-password +THT_SESSION_CA_SOURCE=/secure/thoth/session-ca.pem +``` + +The overlay mounts the runtime password at `/run/secrets/session_runtime_password`, the CA at +`/run/secrets/session_ca.pem`, and passes those paths—not their contents—to the server workspace. +It mounts `session_migrator_password` only to `session-migrate`. The backend refuses a server +session store without upstream authentication, direct DB host/name/runtime user/password-file, +`verify-ca` or `verify-full`, and an absolute CA path. +The migrator independently rejects every other TLS mode before reading its password secret or +constructing a database URL. + +Perform the cutover in one maintenance window, with the Task 4 portal proxy headers and Task 5 +backend principal parser deployed together. Neither change is safe to deploy independently: Task +4 clears the legacy identity header and Task 5 rejects it. Drain/stop active Pi work, enable a +maintenance response at the portal, then run the migrator once and inspect its pristine JSON: + +```sh +docker compose -f compose.yaml -f deploy/compose.session-server.yaml \ + --profile session-migrate run --rm session-migrate +``` + +It must report no pending or drifted migrations before starting the replacement core. `/health` +is a liveness probe and remains `200`; any request that needs unavailable repository storage +returns a fixed `503` before a Pi process starts. Verify this with an authenticated request after +the replacement core is healthy, then remove maintenance mode. + +Do not import the three legacy server filesystem sessions: they have no trusted owner binding. +During the same maintenance window, archive the exact three reviewed IDs, verify the generated +archive and `.sha256`, then rerun the command with `--delete` to remove only those three source +directories: + +```sh +./docker/cutover-legacy-sessions.sh \ + /secure/thoth/legacy-sessions /secure/backups/thoth-legacy-sessions-2026-07-16.tar \ + SESSION_ID_1 SESSION_ID_2 SESSION_ID_3 +# After independent archive review, use a new backup filename: +./docker/cutover-legacy-sessions.sh --delete \ + /secure/thoth/legacy-sessions /secure/backups/thoth-legacy-sessions-2026-07-16-delete.tar \ + SESSION_ID_1 SESSION_ID_2 SESSION_ID_3 +``` + +The helper refuses to overwrite an existing backup and refuses any count other than three +distinct IDs. Never run it against a live path without the maintenance gate. Roll back application +code only by keeping PostgreSQL as the single source of truth and deploying a compatible fixed +release. Do not restore filesystem persistence, do not re-import the archive, and never dual-write +sessions to database and files. + ## Reproducible image verification Base images use exact tags and immutable multi-platform manifest digests. Dependency update and diff --git a/backend/src/app.ts b/backend/src/app.ts index d077b6eb..f2a79bcc 100644 --- a/backend/src/app.ts +++ b/backend/src/app.ts @@ -5,12 +5,14 @@ import { ThtRunner } from "./tht/tht-runner.js"; import { PiProcessManager } from "./pi/pi-process-manager.js"; import { SseHub } from "./sse/sse-hub.js"; import { authPreHandler } from "./auth/auth.js"; +import { getPrincipal } from "./auth/auth.js"; +import type { PrincipalContext } from "./auth/principal.js"; import { sessionRoutes } from "./routes/sessions.js"; import { sqlRoutes } from "./routes/sql.js"; import { metaRoutes, type ListModelsFn } from "./routes/meta.js"; import { settingsRoutes, effectiveSettings } from "./routes/settings.js"; import { createPiModelLister } from "./pi/list-models.js"; -import { loadSettings, type Settings } from "./settings/settings-store.js"; +import { loadSettings, saveSettings, type Settings } from "./settings/settings-store.js"; import { ReadinessManager } from "./runtime/readiness-manager.js"; export interface BuildAppDeps { @@ -18,7 +20,7 @@ export interface BuildAppDeps { mgr?: PiProcessManager; spawnFn?: () => any; listModels?: ListModelsFn; - getSettings?: () => Settings; + getSettings?: (principal?: PrincipalContext) => Settings | Promise; readiness?: ReadinessManager; hub?: SseHub; } @@ -52,7 +54,34 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc "Pi enabled-model configuration warning", ), }); - const getSettings = deps?.getSettings ?? (() => effectiveSettings(config, loadSettings(config))); + const runnerFor = (principal: PrincipalContext): any => { + const candidate = tht as any; + return typeof candidate.withPrincipal === "function" ? candidate.withPrincipal(principal) : candidate; + }; + const getSettings = async (principal: PrincipalContext): Promise => { + if (deps?.getSettings) return await deps.getSettings(principal); + const runner = runnerFor(principal); + // The real runner persists preferences through the harness repository. The file fallback + // only keeps older isolated route tests and externally injected runners compatible. + if (typeof runner.preferencesGet === "function") { + const stored = await runner.preferencesGet() as Settings; + if (Object.keys(stored).length === 0) { + const seeded = effectiveSettings(config, loadSettings(config)); + await runner.preferencesSet(seeded); + return seeded; + } + return effectiveSettings(config, stored); + } + return effectiveSettings(config, loadSettings(config)); + }; + const saveUserSettings = async (principal: PrincipalContext, settings: Settings): Promise => { + const runner = runnerFor(principal); + if (typeof runner.preferencesSet === "function") { + await runner.preferencesSet(settings); + return; + } + saveSettings(config, settings); + }; const authenticate = authPreHandler(config.authMode); app.addHook("preHandler", async (req, reply) => { @@ -61,12 +90,13 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc return authenticate(req, reply); }); app.get("/health", async () => ({ status: "ok" })); + app.get("/me", async (req) => getPrincipal(req)); sessionRoutes(app, { mgr, tht: tht as ThtRunner, hub, getSettings, readiness, }); - sqlRoutes(app, { tht: tht as ThtRunner }); + sqlRoutes(app, { tht: tht as ThtRunner, getSettings }); metaRoutes(app, { harnessDir: config.harnessDir, listModels }); - settingsRoutes(app, { cfg: config, listModels }); + settingsRoutes(app, { cfg: config, listModels, getSettings, saveSettings: saveUserSettings }); return app; } diff --git a/backend/src/auth/auth.ts b/backend/src/auth/auth.ts index 39adc88e..8379e8ba 100644 --- a/backend/src/auth/auth.ts +++ b/backend/src/auth/auth.ts @@ -1,23 +1,28 @@ import type { FastifyRequest, FastifyReply } from "fastify"; +import { localPrincipal, type PrincipalContext, upstreamPrincipal } from "./principal.js"; + +declare module "fastify" { + interface FastifyRequest { principal?: PrincipalContext } +} export function authPreHandler(mode: "none" | "mock" | "upstream") { return async (req: FastifyRequest, reply: FastifyReply) => { if (mode === "none") { - (req as any).user = { id: "dev@local" }; + req.principal = localPrincipal(); } else if (mode === "mock") { - (req as any).user = { - id: (req.headers["x-mock-user"] as string) ?? "mock", - }; + const subject = typeof req.headers["x-mock-user"] === "string" ? req.headers["x-mock-user"].trim() : "mock"; + req.principal = { issuer: "mock", subject: subject || "mock", displayName: subject || "mock", isAdmin: false }; } else { - const id = req.headers["x-authenticated-user"]; - if (typeof id !== "string" || id.trim() === "") { + const principal = upstreamPrincipal(req.headers); + if (!principal) { return reply.code(401).send({ error: "authenticated upstream identity required" }); } - (req as any).user = { id }; + req.principal = principal; } }; } -export function getUser(req: FastifyRequest): { id: string } { - return (req as any).user ?? { id: "dev@local" }; +export function getPrincipal(req: FastifyRequest): PrincipalContext { + if (!req.principal) throw new Error("principal missing after authentication"); + return req.principal; } diff --git a/backend/src/auth/principal.ts b/backend/src/auth/principal.ts new file mode 100644 index 00000000..a58aff1f --- /dev/null +++ b/backend/src/auth/principal.ts @@ -0,0 +1,90 @@ +import { chmodSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import { randomUUID } from "node:crypto"; + +export interface PrincipalContext { + issuer: string; + subject: string; + displayName?: string; + isAdmin: boolean; +} + +const principalEnvKeys = [ + "THT_PRINCIPAL_ISSUER", "THT_PRINCIPAL_SUBJECT", "THT_PRINCIPAL_DISPLAY_NAME", "THT_PRINCIPAL_IS_ADMIN", +] as const; + +export function clearPrincipalEnvironment(env: NodeJS.ProcessEnv): void { + for (const key of principalEnvKeys) delete env[key]; +} + +export function expandLocalHome(path: string, home = homedir()): string { + if (path === "~") return home; + if (path.startsWith("~/")) return join(home, path.slice(2)); + return path; +} + +function harden(path: string, mode: number): void { + if (process.platform === "win32") return; + try { chmodSync(path, mode); } catch { /* best-effort parity with harness local storage */ } +} + +const invalid = (value: string) => value.length === 0 || value.length > 512 || /[\u0000-\u001f\u007f]/.test(value); + +function required(value: unknown): string | undefined { + if (typeof value !== "string") return undefined; + const normalized = value.trim(); + return invalid(normalized) ? undefined : normalized; +} + +function optional(value: unknown): string | undefined { + if (value === undefined) return undefined; + return required(value); +} + +export function upstreamPrincipal(headers: Record): PrincipalContext | undefined { + const issuer = required(headers["x-thoth-principal-issuer"]); + const subject = required(headers["x-thoth-principal-subject"]); + const displayName = optional(headers["x-thoth-principal-display-name"]); + const adminHeader = headers["x-thoth-is-admin"]; + if (!issuer || !subject || (headers["x-thoth-principal-display-name"] !== undefined && !displayName)) return undefined; + if (adminHeader !== "0" && adminHeader !== "1" && adminHeader !== "true" && adminHeader !== "false") return undefined; + return { issuer, subject, displayName, isAdmin: adminHeader === "1" || adminHeader === "true" }; +} + +export function localPrincipal(): PrincipalContext { + const home = expandLocalHome(process.env.THT_HOME ?? join(homedir(), ".thothii")); + const identityPath = join(home, "identity.json"); + mkdirSync(home, { recursive: true, mode: 0o700 }); + harden(home, 0o700); + try { + const stored = JSON.parse(readFileSync(identityPath, "utf8")); + if (stored?.issuer === "local" && typeof stored.subject === "string" && /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(stored.subject)) { + harden(identityPath, 0o600); + return { issuer: "local", subject: stored.subject, isAdmin: false }; + } + throw new Error("invalid local identity"); + } catch (error: any) { + if (error?.code !== "ENOENT") throw error; + const principal = { issuer: "local", subject: randomUUID() }; + try { + writeFileSync(identityPath, JSON.stringify(principal) + "\n", { mode: 0o600, flag: "wx" }); + harden(identityPath, 0o600); + return { ...principal, isAdmin: false }; + } catch (writeError: any) { + // Another local request won the identity creation race; always converge on its UUID. + if (writeError?.code === "EEXIST") return localPrincipal(); + throw writeError; + } + } +} + +export function principalEnvironment(principal: PrincipalContext): NodeJS.ProcessEnv { + const env: NodeJS.ProcessEnv = { + THT_PRINCIPAL_ISSUER: principal.issuer, + THT_PRINCIPAL_SUBJECT: principal.subject, + THT_PRINCIPAL_IS_ADMIN: principal.isAdmin ? "true" : "false", + }; + if (principal.displayName) env.THT_PRINCIPAL_DISPLAY_NAME = principal.displayName; + return env; +} diff --git a/backend/src/config.ts b/backend/src/config.ts index fae338d7..3020eff7 100644 --- a/backend/src/config.ts +++ b/backend/src/config.ts @@ -3,6 +3,11 @@ import path from "node:path"; export interface AppConfig { host: string; port: number; harnessDir: string; thtBin: string; piBin: string; authMode: "none" | "mock" | "upstream"; + sessionStorage: { + mode: "local" | "postgres"; + host?: string; port?: number; database?: string; runtimeUser?: string; + runtimePasswordFile?: string; sslmode?: "verify-ca" | "verify-full"; sslrootcert?: string; + }; defaults: { provider?: string; model?: string; thinking?: string }; maxPiProcesses: number; settingsFile: string; @@ -20,6 +25,43 @@ export function loadConfig(env: Record): AppConfig { if (env.THOTH_PUBLIC_EXPOSURE === "true" && authMode !== "upstream") { throw new Error("public exposure requires AUTH_MODE=upstream behind a trusted proxy"); } + const sessionStorageMode = env.THT_SESSION_STORAGE ?? "local"; + if (sessionStorageMode !== "local" && sessionStorageMode !== "postgres") { + throw new Error("session storage configuration is invalid"); + } + if (sessionStorageMode === "local" && env.THOTH_PUBLIC_EXPOSURE === "true") { + throw new Error("local session storage requires loopback-only deployment"); + } + const sessionStorage: AppConfig["sessionStorage"] = { mode: sessionStorageMode }; + if (sessionStorageMode === "postgres") { + const host = env.THT_SESSION_DB_HOST; + const database = env.THT_SESSION_DB_NAME; + const runtimeUser = env.THT_SESSION_RUNTIME_USER; + const runtimePasswordFile = env.THT_SESSION_RUNTIME_PASSWORD_FILE; + const sslmode = env.THT_SESSION_DB_SSLMODE; + const sslrootcert = env.THT_SESSION_DB_SSLROOTCERT; + const port = Number(env.THT_SESSION_DB_PORT ?? 5432); + if ( + authMode !== "upstream" + || !host || !database || !runtimeUser + || !runtimePasswordFile || !path.isAbsolute(runtimePasswordFile) + || (sslmode !== "verify-ca" && sslmode !== "verify-full") + || !sslrootcert || !path.isAbsolute(sslrootcert) + || !Number.isInteger(port) || port < 1 || port > 65535 + ) { + if (authMode !== "upstream") { + throw new Error("server session storage requires AUTH_MODE=upstream"); + } + throw new Error("server session storage configuration is invalid"); + } + sessionStorage.host = host; + sessionStorage.port = port; + sessionStorage.database = database; + sessionStorage.runtimeUser = runtimeUser; + sessionStorage.runtimePasswordFile = runtimePasswordFile; + sessionStorage.sslmode = sslmode; + sessionStorage.sslrootcert = sslrootcert; + } const modelApiKeyFile = env.THT_MODEL_API_KEY_FILE; if (modelApiKeyFile !== undefined && ( modelApiKeyFile.trim() !== modelApiKeyFile @@ -48,6 +90,7 @@ export function loadConfig(env: Record): AppConfig { thtBin: env.THT_BIN ?? "tht", piBin: env.PI_BIN ?? "pi", authMode: authMode as AppConfig["authMode"], + sessionStorage, defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING }, maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4), settingsFile: env.SETTINGS_FILE ?? "data/settings.json", diff --git a/backend/src/pi/pi-process-manager.ts b/backend/src/pi/pi-process-manager.ts index 23fbbc16..c9aa47f3 100644 --- a/backend/src/pi/pi-process-manager.ts +++ b/backend/src/pi/pi-process-manager.ts @@ -5,6 +5,7 @@ import { SessionBridge } from "../bridge/session-bridge.js"; import type { ThtRunner } from "../tht/tht-runner.js"; import { buildPiChildEnv, canonicalPiProvider } from "./provider-credentials.js"; import { secretValue } from "../config/secret-bundle.js"; +import { clearPrincipalEnvironment, principalEnvironment, type PrincipalContext } from "../auth/principal.js"; export interface SessionRuntime { rpc: RpcClient; @@ -19,6 +20,7 @@ export interface RuntimeOptions { author?: string; question?: string; mode?: "new" | "resume"; + principal?: PrincipalContext; } /** Injectable child-process boundary; callbacks may ignore arguments in simpler tests. */ @@ -31,21 +33,21 @@ type SpawnFn = ( export class PiProcessManager { private runtimes = new Map(); private spawnFn: ( - sessionId: string, author: string, provider: string | undefined, + sessionId: string, author: string, provider: string | undefined, principal?: PrincipalContext, ) => ChildProcessWithoutNullStreams; constructor(private cfg: AppConfig, opts?: { spawnFn?: SpawnFn }) { if (opts?.spawnFn) { - this.spawnFn = (sessionId, author, provider) => - this.spawnPi(opts.spawnFn!, sessionId, author, provider); + this.spawnFn = (sessionId, author, provider, principal) => + this.spawnPi(opts.spawnFn!, sessionId, author, provider, principal); } else { - this.spawnFn = (sessionId, author, provider) => - this.spawnPi(nodeSpawn, sessionId, author, provider); + this.spawnFn = (sessionId, author, provider, principal) => + this.spawnPi(nodeSpawn, sessionId, author, provider, principal); } } private spawnPi( - spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined, + spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined, principal?: PrincipalContext, ): ChildProcessWithoutNullStreams { const env = buildPiChildEnv({ provider, @@ -53,6 +55,8 @@ export class PiProcessManager { credentialFile: this.cfg.modelApiKeyFile, additions: { THT_SESSION: sessionId, THT_AUTHOR: author }, }); + clearPrincipalEnvironment(env); + if (principal) Object.assign(env, principalEnvironment(principal)); // The Thoth gate executes the deterministic `tht` CLI as a Pi tool. Give only // this managed session process the adapter values already loaded by the core // entrypoint; the generic provider helper continues to scrub them by default. @@ -95,7 +99,7 @@ export class PiProcessManager { } const author = o.author ?? "dev@local"; const provider = canonicalPiProvider(o.provider ?? this.cfg.defaults.provider); - const child = this.spawnFn(sessionId, author, provider); + const child = this.spawnFn(sessionId, author, provider, o.principal); let rt: SessionRuntime | undefined; try { const rpc = new RpcClient(child); diff --git a/backend/src/routes/sessions.ts b/backend/src/routes/sessions.ts index 66f47b88..1aab8a13 100644 --- a/backend/src/routes/sessions.ts +++ b/backend/src/routes/sessions.ts @@ -3,7 +3,8 @@ import type { PiProcessManager } from "../pi/pi-process-manager.js"; import type { ThtRunner } from "../tht/tht-runner.js"; import type { SseHub } from "../sse/sse-hub.js"; import type { Settings } from "../settings/settings-store.js"; -import { getUser } from "../auth/auth.js"; +import { getPrincipal } from "../auth/auth.js"; +import type { PrincipalContext } from "../auth/principal.js"; import type { ReadinessManager } from "../runtime/readiness-manager.js"; const BOOTSTRAP_FAILURE_MESSAGE = @@ -25,7 +26,11 @@ function eventCursor(...values: unknown[]): number { export function sessionRoutes( app: FastifyInstance, - d: { mgr: PiProcessManager; tht: ThtRunner; hub: SseHub; getSettings: () => Settings; readiness: ReadinessManager }, + d: { + mgr: PiProcessManager; tht: ThtRunner; hub: SseHub; + getSettings: (principal: PrincipalContext) => Promise; + readiness: ReadinessManager; + }, ) { const lifecycleTails = new Map>(); const boundRuntimes = new Map< @@ -54,7 +59,34 @@ export function sessionRoutes( const info = (id: string, text: string, level = "info") => d.hub.publish(id, "info", { type: "info", level, text }); - const bindRuntime = (id: string, rt: ReturnType) => { + const runnerFor = (principal: PrincipalContext): any => { + const runner = d.tht as any; + return typeof runner.withPrincipal === "function" ? runner.withPrincipal(principal) : runner; + }; + + const isNotFound = (error: unknown) => + /not found|non trovata|inesistente|404/i.test(error instanceof Error ? error.message : String(error)); + + /** RLS makes a foreign session indistinguishable from a missing one. */ + const authorize = async (principal: PrincipalContext, id: string, workspace?: string): Promise => { + try { + const runner = runnerFor(principal); + // Dependency-injected runners in legacy route tests may model only the mutation under + // test. Production ThtRunner always exposes sessionShow; keep that test seam harmless. + if (typeof runner.sessionShow !== "function") return {}; + const manifest = await runner.sessionShow(id, workspace); + return manifest ?? undefined; + } catch (error) { + if (isNotFound(error)) return undefined; + throw error; + } + }; + + const storageFailure = (reply: any) => reply.code(503).send({ error: "session storage is unavailable" }); + + const bindRuntime = ( + id: string, rt: ReturnType, runner: any, workspace?: string, + ) => { const previous = boundRuntimes.get(id); boundRuntimes.set(id, rt); try { @@ -72,7 +104,7 @@ export function sessionRoutes( // old failure must not touch its manifest. const bound = boundRuntimes.get(id); if (bound !== undefined && bound !== rt) return; - await d.tht.failSession(id, d.getSettings().workspace).catch(() => undefined); + await runner.failSession(id, workspace).catch(() => undefined); }).catch(() => undefined); } } @@ -96,6 +128,8 @@ export function sessionRoutes( const bootstrap = ( id: string, rt: ReturnType, + runner: any, + workspace: string | undefined, configure: Promise, retrieval: Promise | null, start: () => void, @@ -117,7 +151,7 @@ export function sessionRoutes( if (d.mgr.get(id) !== rt || !d.mgr.teardownIfCurrent(id, rt)) return; if (!failurePersistenceClaimed.has(rt)) { failurePersistenceClaimed.add(rt); - await d.tht.failSession(id, d.getSettings().workspace).catch(() => undefined); + await runner.failSession(id, workspace).catch(() => undefined); } rt.bridge.emitClientEvent({ type: "info", level: "error", text: BOOTSTRAP_FAILURE_MESSAGE }); rt.bridge.emitClientEvent({ type: "system_event", event: "session_failed" }); @@ -127,62 +161,105 @@ export function sessionRoutes( })(); }; - app.post("/runtime/prewarm", async (_req, reply) => { - const workspace = d.getSettings().workspace ?? ""; - void d.readiness.ensure(workspace).catch(() => undefined); + app.post("/runtime/prewarm", async (req, reply) => { + let settings: Settings; + try { settings = await d.getSettings(getPrincipal(req)); } catch { return storageFailure(reply); } + const workspace = settings.workspace ?? ""; + void d.readiness.ensure(workspace, getPrincipal(req)).catch(() => undefined); return reply.code(202).send({ status: "warming" }); }); app.post("/sessions", async (req, reply) => { const b = req.body as { question: string; name?: string }; - const s = d.getSettings(); - const ensure = await d.readiness.ensure(s.workspace ?? ""); + const principal = getPrincipal(req); + let s: Settings; + try { s = await d.getSettings(principal); } catch { return storageFailure(reply); } + const runner = runnerFor(principal); + const ensure = await d.readiness.ensure(s.workspace ?? "", principal); if (!ensure.ok) return reply.code(503).send({ error: READINESS_FAILURE_MESSAGE }); // Settings (global) supply workspace/provider/model/thinking. The new-question // form sends only the question text. `workspace` selects the tht `-c `. - const { id } = await d.tht.sessionNew({ - question: b.question, - name: b.name, - workspace: s.workspace, - provider: s.provider, - model: s.model, - thinking: s.thinking, - }); + let id: string; + try { + ({ id } = await runner.sessionNew({ + question: b.question, name: b.name, workspace: s.workspace, + provider: s.provider, model: s.model, thinking: s.thinking, + })); + } catch { return storageFailure(reply); } const options = { provider: s.provider, model: s.model, thinking: s.thinking, - author: getUser(req).id, + author: principal.displayName ?? principal.subject, + principal, question: b.question, }; const rt = d.mgr.createFor(id, options); - bindRuntime(id, rt); + bindRuntime(id, rt, runner, s.workspace); info(id, "Session created"); bootstrap( - id, rt, d.mgr.configure(rt, options), - d.tht.searchPack(b.question, id, s.workspace), + id, rt, runner, s.workspace, d.mgr.configure(rt, options), + runner.searchPack(b.question, id, s.workspace), () => d.mgr.start(id, rt, options), ); return { id }; }); - app.get("/sessions", async () => d.tht.sessionList(d.getSettings().workspace)); - app.get("/sessions/:id", async (req) => d.tht.sessionShow((req.params as any).id, d.getSettings().workspace)); + app.get("/sessions", async (req, reply) => { + const principal = getPrincipal(req); + const scope = (req.query as { scope?: string }).scope ?? "mine"; + if (scope !== "mine" && scope !== "all") return reply.code(400).send({ error: "scope must be mine or all" }); + if (scope === "all" && !principal.isAdmin) return reply.code(403).send({ error: "admin scope required" }); + try { + const settings = await d.getSettings(principal); + // Admin RLS is deliberately disabled for a normal 'mine' listing. + const scopedPrincipal = scope === "mine" ? { ...principal, isAdmin: false } : principal; + return await runnerFor(scopedPrincipal).sessionList(settings.workspace); + } catch { return storageFailure(reply); } + }); + app.get("/sessions/:id", async (req, reply) => { + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + const manifest = await authorize(principal, (req.params as any).id, settings.workspace); + return manifest ?? reply.code(404).send({ error: "session not found" }); + } catch { return storageFailure(reply); } + }); app.post("/sessions/:id/response", async (req, reply) => { const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + } catch { return storageFailure(reply); } const rt = d.mgr.get(id); if (!rt) return reply.code(404).send({ error: "sessione non attiva" }); rt.bridge.respond((req.body as any).ui_response); return reply.code(204).send(); }); app.post("/sessions/:id/steer", async (req, reply) => { - const rt = d.mgr.get((req.params as any).id); + const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + } catch { return storageFailure(reply); } + const rt = d.mgr.get(id); if (!rt) return reply.code(404).send({ error: "sessione non attiva" }); rt.bridge.steer((req.body as any).text); return reply.code(204).send(); }); app.post("/sessions/:id/resume", async (req, reply) => { const id = (req.params as any).id; + const principal = getPrincipal(req); return withSessionLifecycle(id, async () => { + let settings: Settings; + let manifest: any; + try { + settings = await d.getSettings(principal); + manifest = await authorize(principal, id, settings.workspace); + } catch { return storageFailure(reply); } + if (!manifest) return reply.code(404).send({ error: "session not found" }); + const runner = runnerFor(principal); // This check belongs inside the per-session lock: a preceding cold Resume may have // installed a running runtime while this request was waiting. const existing = d.mgr.get(id); @@ -192,26 +269,25 @@ export function sessionRoutes( return reply.code(200).send({ id, alreadyActive: true }); } } - const manifest = (await d.tht.sessionShow(id, d.getSettings().workspace)) as { status?: string; archived?: boolean } | null; if (manifest?.status === "finalized" || manifest?.archived) { return reply.code(409).send({ error: "sessione in sola lettura (finalizzata o archiviata)" }); } - const settings = d.getSettings(); - const ensure = await d.readiness.ensure(settings.workspace ?? ""); + const ensure = await d.readiness.ensure(settings.workspace ?? "", principal); if (!ensure.ok) return reply.code(503).send({ error: READINESS_FAILURE_MESSAGE }); const saved = manifest as { provider?: string; model?: string; thinking?: string } | null; const options = { provider: saved?.provider, model: saved?.model, thinking: saved?.thinking ?? settings.thinking, - author: getUser(req).id, + author: principal.displayName ?? principal.subject, + principal, mode: "resume" as const, }; // Reopening is validation, not the transport commit point. Keep the old hub intact if // persistence cannot be reopened. try { - await d.tht.reopenSession(id, settings.workspace); + await runner.reopenSession(id, settings.workspace); } catch { return reply.code(503).send({ error: RESUME_FAILURE_MESSAGE }); } @@ -233,7 +309,7 @@ export function sessionRoutes( d.mgr.teardownIfCurrent(id, current); } rt = d.mgr.createFor(id, options); - bindRuntime(id, rt); + bindRuntime(id, rt, runner, settings.workspace); } catch { // A created-but-unbound runtime is not usable. The old hub remains attached because // clear() has not happened yet. @@ -248,28 +324,41 @@ export function sessionRoutes( // immediately before the first event produced by the new Resume. d.hub.clear(id); info(id, "Resuming session"); - bootstrap(id, rt, d.mgr.configure(rt, options), null, () => d.mgr.start(id, rt, options)); + bootstrap(id, rt, runner, settings.workspace, d.mgr.configure(rt, options), null, () => d.mgr.start(id, rt, options)); return reply.code(200).send({ id, alreadyActive: false }); }); }); - app.post("/sessions/:id/close", async (req) => { + app.post("/sessions/:id/close", async (req, reply) => { const id = (req.params as { id: string }).id; + const principal = getPrincipal(req); return withSessionLifecycle(id, async () => { + let settings: Settings; + try { + settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + } catch { return storageFailure(reply); } // Invalidate the live generation before persistence can yield. Otherwise its deferred // bootstrap may start Pi while Close is already in progress. const current = d.mgr.get(id); boundRuntimes.delete(id); if (current) d.mgr.teardownIfCurrent(id, current); try { - await d.tht.closeSession(id, d.getSettings().workspace); + await runnerFor(principal).closeSession(id, settings.workspace); + } catch { + return storageFailure(reply); } finally { d.hub.clear(id); } return { closed: true }; }); }); - app.get("/sessions/:id/events", (req, reply) => { + app.get("/sessions/:id/events", async (req, reply) => { const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + } catch { return storageFailure(reply); } const rt = d.mgr.get(id); const afterId = eventCursor( req.headers["last-event-id"], @@ -303,31 +392,73 @@ export function sessionRoutes( req.raw.on("close", off); }); app.post("/sessions/:id/rename", async (req, reply) => { - await d.tht.setName((req.params as any).id, (req.body as any).name); + const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + await runnerFor(principal).setName(id, (req.body as any).name, settings.workspace); + } catch { return storageFailure(reply); } return reply.code(204).send(); }); app.post("/sessions/:id/group", async (req, reply) => { - await d.tht.setGroup((req.params as any).id, (req.body as any).group); + const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + await runnerFor(principal).setGroup(id, (req.body as any).group, settings.workspace); + } catch { return storageFailure(reply); } return reply.code(204).send(); }); app.post("/sessions/:id/archive", async (req, reply) => { - await d.tht.archive((req.params as any).id); + const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + await runnerFor(principal).archive(id, settings.workspace); + } catch { return storageFailure(reply); } return reply.code(204).send(); }); app.post("/sessions/:id/unarchive", async (req, reply) => { - await d.tht.unarchive((req.params as any).id); + const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + await runnerFor(principal).unarchive(id, settings.workspace); + } catch { return storageFailure(reply); } return reply.code(204).send(); }); app.delete("/sessions/:id", async (req, reply) => { const id = (req.params as any).id; + const principal = getPrincipal(req); return withSessionLifecycle(id, async () => { + let settings: Settings; + try { + settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + } catch { return storageFailure(reply); } const current = d.mgr.get(id); boundRuntimes.delete(id); if (current) d.mgr.teardownIfCurrent(id, current); - await d.tht.deleteSession(id, d.getSettings().workspace); + try { + await runnerFor(principal).deleteSession(id, settings.workspace); + } catch { + return storageFailure(reply); + } d.hub.forget(id); return reply.code(204).send(); }); }); - app.get("/sessions/:id/documents", async (req) => d.tht.documents((req.params as any).id)); + app.get("/sessions/:id/documents", async (req, reply) => { + const id = (req.params as any).id; + const principal = getPrincipal(req); + try { + const settings = await d.getSettings(principal); + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + return await runnerFor(principal).documents(id, settings.workspace); + } catch { return storageFailure(reply); } + }); } diff --git a/backend/src/routes/settings.ts b/backend/src/routes/settings.ts index 0f9516fc..d6a542ca 100644 --- a/backend/src/routes/settings.ts +++ b/backend/src/routes/settings.ts @@ -2,6 +2,8 @@ import type { FastifyInstance } from "fastify"; import type { AppConfig } from "../config.js"; import { loadSettings, saveSettings, type Settings } from "../settings/settings-store.js"; import { listWorkspaces, type ListModelsFn } from "./meta.js"; +import { getPrincipal } from "../auth/auth.js"; +import type { PrincipalContext } from "../auth/principal.js"; /** Merge stored settings over env/first-workspace defaults. */ export function effectiveSettings(cfg: AppConfig, stored: Settings): Settings { @@ -16,10 +18,18 @@ export function effectiveSettings(cfg: AppConfig, stored: Settings): Settings { export function settingsRoutes( app: FastifyInstance, - deps: { cfg: AppConfig; listModels: ListModelsFn }, + deps: { + cfg: AppConfig; listModels: ListModelsFn; + getSettings: (principal: PrincipalContext) => Promise; + saveSettings: (principal: PrincipalContext, settings: Settings) => Promise; + }, ): void { - app.get("/settings", async () => { - return effectiveSettings(deps.cfg, loadSettings(deps.cfg)); + app.get("/settings", async (req, reply) => { + try { + return await deps.getSettings(getPrincipal(req)); + } catch { + return reply.code(503).send({ error: "settings storage is unavailable" }); + } }); app.put("/settings", async (req, reply) => { @@ -46,7 +56,11 @@ export function settingsRoutes( model: b.model, thinking: b.thinking, }; - saveSettings(deps.cfg, next); - return effectiveSettings(deps.cfg, next); + try { + await deps.saveSettings(getPrincipal(req), next); + return effectiveSettings(deps.cfg, next); + } catch { + return reply.code(503).send({ error: "settings storage is unavailable" }); + } }); } diff --git a/backend/src/routes/sql.ts b/backend/src/routes/sql.ts index 8b2970b1..f47304cb 100644 --- a/backend/src/routes/sql.ts +++ b/backend/src/routes/sql.ts @@ -1,25 +1,63 @@ import type { FastifyInstance } from "fastify"; import type { ThtRunner } from "../tht/tht-runner.js"; +import { getPrincipal } from "../auth/auth.js"; +import type { PrincipalContext } from "../auth/principal.js"; +import type { Settings } from "../settings/settings-store.js"; -export function sqlRoutes(app: FastifyInstance, deps: { tht: ThtRunner }): void { +export function sqlRoutes(app: FastifyInstance, deps: { + tht: ThtRunner; getSettings: (principal: PrincipalContext) => Promise; +}): void { + const runnerFor = (principal: PrincipalContext): any => { + const runner = deps.tht as any; + return typeof runner.withPrincipal === "function" ? runner.withPrincipal(principal) : runner; + }; + const authorize = async (principal: PrincipalContext, id: string, workspace?: string) => { + try { + const runner = runnerFor(principal); + if (typeof runner.sessionShow !== "function") return {}; + return await runner.sessionShow(id, workspace); + } + catch (error) { + if (/not found|non trovata|inesistente|404/i.test(error instanceof Error ? error.message : String(error))) return undefined; + throw error; + } + }; app.post("/sessions/:id/sql/preview", async (req, reply) => { const id = (req.params as any).id as string; const { limit, offset } = (req.body as any) ?? {}; + let principal: PrincipalContext; + let workspace: string | undefined; try { - const result = await deps.tht.sqlPreview(id, { limit, offset }); - return result; - } catch (err: any) { - return reply.code(500).send({ error: err.message ?? String(err) }); + principal = getPrincipal(req); + const settings = await deps.getSettings(principal); + workspace = settings.workspace; + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + } catch { + return reply.code(503).send({ error: "session storage is unavailable" }); + } + try { + return await runnerFor(principal).sqlPreview(id, { limit, offset }, workspace); + } catch (error: any) { + return reply.code(500).send({ error: error.message ?? String(error) }); } }); app.post("/sessions/:id/sql/export", async (req, reply) => { const id = (req.params as any).id as string; + let principal: PrincipalContext; + let workspace: string | undefined; try { - const result = await deps.tht.sqlExport(id); - return result; - } catch (err: any) { - return reply.code(500).send({ error: err.message ?? String(err) }); + principal = getPrincipal(req); + const settings = await deps.getSettings(principal); + workspace = settings.workspace; + if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" }); + } catch { + return reply.code(503).send({ error: "session storage is unavailable" }); + } + try { + return await runnerFor(principal).sqlExport(id, workspace); + } catch (error: any) { + return reply.code(500).send({ error: error.message ?? String(error) }); } }); } diff --git a/backend/src/runtime/readiness-manager.ts b/backend/src/runtime/readiness-manager.ts index 9cc17b63..1c49494e 100644 --- a/backend/src/runtime/readiness-manager.ts +++ b/backend/src/runtime/readiness-manager.ts @@ -1,4 +1,5 @@ import type { OllamaEnsureResult, ThtRunner } from "../tht/tht-runner.js"; +import type { PrincipalContext } from "../auth/principal.js"; interface ReadyEntry { expiresAt: number; @@ -20,25 +21,28 @@ export class ReadinessManager { private now: () => number = Date.now, ) {} - ensure(workspace = ""): Promise { - const cached = this.ready.get(workspace); + ensure(workspace = "", principal?: PrincipalContext): Promise { + const key = `${principal?.issuer ?? ""}\0${principal?.subject ?? ""}\0${workspace}`; + const cached = this.ready.get(key); if (cached && cached.expiresAt > this.now()) return Promise.resolve(cached.result); - if (cached) this.ready.delete(workspace); + if (cached) this.ready.delete(key); - const current = this.inFlight.get(workspace); + const current = this.inFlight.get(key); if (current) return current; - const pending = this.tht.ollamaEnsure(workspace, this.timeoutSec) + const runner = principal && typeof (this.tht as any).withPrincipal === "function" + ? this.tht.withPrincipal(principal) : this.tht; + const pending = runner.ollamaEnsure(workspace, this.timeoutSec) .then((result) => { if (result.ok) { - this.ready.set(workspace, { result, expiresAt: this.now() + this.ttlMs }); + this.ready.set(key, { result, expiresAt: this.now() + this.ttlMs }); } return result; }) .finally(() => { - if (this.inFlight.get(workspace) === pending) this.inFlight.delete(workspace); + if (this.inFlight.get(key) === pending) this.inFlight.delete(key); }); - this.inFlight.set(workspace, pending); + this.inFlight.set(key, pending); return pending; } } diff --git a/backend/src/tht/tht-runner.ts b/backend/src/tht/tht-runner.ts index 2b81ee73..0ad4f28a 100644 --- a/backend/src/tht/tht-runner.ts +++ b/backend/src/tht/tht-runner.ts @@ -1,6 +1,7 @@ import { spawn } from "node:child_process"; import { existsSync } from "node:fs"; import { join } from "node:path"; +import { clearPrincipalEnvironment, principalEnvironment, type PrincipalContext } from "../auth/principal.js"; export interface ThtConfig { thtBin: string; @@ -37,7 +38,10 @@ export interface OllamaEnsureResult { } export class ThtRunner { - constructor(private cfg: ThtConfig) {} + constructor(private cfg: ThtConfig, private principal?: PrincipalContext) {} + + /** Bind one trusted request principal to every child spawned by this runner. */ + withPrincipal(principal: PrincipalContext): ThtRunner { return new ThtRunner(this.cfg, principal); } /** * Resolve the `-c ` args. If `workspace` is given AND a matching @@ -64,16 +68,25 @@ export class ThtRunner { return new Promise((resolve) => { const env: NodeJS.ProcessEnv = { ...process.env }; delete env.THT_DATA_ROOT; + clearPrincipalEnvironment(env); if (this.cfg.dataRoot !== undefined) env.THT_DATA_ROOT = this.cfg.dataRoot; + if (this.principal) Object.assign(env, principalEnvironment(this.principal)); const ch = spawn(this.cfg.thtBin, this.buildArgv(args, workspace), { cwd: this.cfg.harnessDir, env, }); let stdout = ""; let stderr = ""; + let settled = false; + const finish = (result: { code: number; stdout: string; stderr: string }) => { + if (settled) return; + settled = true; + resolve(result); + }; ch.stdout.on("data", (d: Buffer) => (stdout += d)); ch.stderr.on("data", (d: Buffer) => (stderr += d)); - ch.on("close", (code) => resolve({ code: code ?? 0, stdout, stderr })); + ch.on("error", (error) => finish({ code: 1, stdout, stderr: stderr || error.message })); + ch.on("close", (code) => finish({ code: code ?? 0, stdout, stderr })); }); } @@ -124,7 +137,7 @@ export class ThtRunner { return this.json(["session", "show", id, "--json"], workspace); } - sqlPreview(id: string, p: { limit?: number; offset?: number }) { + sqlPreview(id: string, p: { limit?: number; offset?: number }, workspace?: string) { // No positional FILE: the harness resolves sql_final.sql from the session // via _session_sql_file(cfg, session_id), which respects the workspace path. const a = ["sql", "preview", "--session", id, "--json"]; @@ -135,11 +148,11 @@ export class ThtRunner { rows: unknown[][]; execution_ms: number; truncated: boolean; - }>(a); + }>(a, workspace); } - async sqlExport(id: string) { - const { code, stdout, stderr } = await this.run(["sql", "export", "--session", id]); + async sqlExport(id: string, workspace?: string) { + const { code, stdout, stderr } = await this.run(["sql", "export", "--session", id], workspace); if (code !== 0) throw new Error(`tht sql export exit ${code}: ${stderr.trim()}`); return { path: stdout.trim() }; } @@ -147,15 +160,20 @@ export class ThtRunner { closeSession(id: string, workspace?: string) { return this.ok(["session", "close", id], workspace); } failSession(id: string, workspace?: string) { return this.ok(["session", "fail", id], workspace); } reopenSession(id: string, workspace?: string) { return this.ok(["session", "reopen", id], workspace); } - setName(id: string, name: string) { return this.ok(["session", "set-name", id, "--name", name]); } - setGroup(id: string, group: string) { return this.ok(["session", "set-group", id, "--group", group]); } - archive(id: string) { return this.ok(["session", "archive", id]); } - unarchive(id: string) { return this.ok(["session", "unarchive", id]); } + setName(id: string, name: string, workspace?: string) { return this.ok(["session", "set-name", id, "--name", name], workspace); } + setGroup(id: string, group: string, workspace?: string) { return this.ok(["session", "set-group", id, "--group", group], workspace); } + archive(id: string, workspace?: string) { return this.ok(["session", "archive", id], workspace); } + unarchive(id: string, workspace?: string) { return this.ok(["session", "unarchive", id], workspace); } async deleteSession(id: string, workspace?: string) { const { code, stderr } = await this.run(["session", "delete", id], workspace); if (code !== 0) throw new Error(`tht session delete exit ${code}: ${stderr.trim()}`); } - documents(id: string) { return this.json(["session", "documents", id, "--json"]); } + documents(id: string, workspace?: string) { return this.json(["session", "documents", id, "--json"], workspace); } + + preferencesGet(workspace?: string) { return this.json>(["session", "preferences", "get", "--json"], workspace); } + async preferencesSet(preferences: Record, workspace?: string): Promise { + await this.ok(["session", "preferences", "set", "--json", JSON.stringify(preferences)], workspace); + } async ollamaEnsure(workspace: string, timeoutSec: number): Promise { const { code, stdout, stderr } = await this.run( diff --git a/backend/test/auth.test.ts b/backend/test/auth.test.ts index 0702549c..8285eb47 100644 --- a/backend/test/auth.test.ts +++ b/backend/test/auth.test.ts @@ -1,38 +1,85 @@ import { test, expect } from "vitest"; import Fastify from "fastify"; -import { authPreHandler, getUser } from "../src/auth/auth.js"; +import { authPreHandler, getPrincipal } from "../src/auth/auth.js"; +import { chmodSync, mkdtempSync, rmSync, statSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expandLocalHome, localPrincipal } from "../src/auth/principal.js"; -test("mode none assegna dev@local", async () => { +test("local mode resolves a stable local principal", async () => { const app = Fastify(); app.addHook("preHandler", authPreHandler("none")); - app.get("/me", async (req) => getUser(req)); - expect((await app.inject({ method: "GET", url: "/me" })).json()).toEqual({ - id: "dev@local", + app.get("/me", async (req) => getPrincipal(req)); + expect((await app.inject({ method: "GET", url: "/me" })).json()).toMatchObject({ + issuer: "local", + subject: expect.any(String), + isAdmin: false, }); }); -test("mode mock legge l'header", async () => { +test("mock mode makes a principal from the test header", async () => { const app = Fastify(); app.addHook("preHandler", authPreHandler("mock")); - app.get("/me", async (req) => getUser(req)); + app.get("/me", async (req) => getPrincipal(req)); const res = await app.inject({ method: "GET", url: "/me", headers: { "x-mock-user": "alice" }, }); - expect(res.json()).toEqual({ id: "alice" }); + expect(res.json()).toEqual({ issuer: "mock", subject: "alice", displayName: "alice", isAdmin: false }); }); -test("upstream mode requires the authenticated proxy identity header", async () => { +test("upstream mode accepts only normalized proxy principal headers", async () => { const app = Fastify(); app.addHook("preHandler", authPreHandler("upstream")); - app.get("/me", async (req) => getUser(req)); + app.get("/me", async (req) => getPrincipal(req)); expect((await app.inject({ method: "GET", url: "/me" })).statusCode).toBe(401); const authenticated = await app.inject({ method: "GET", url: "/me", - headers: { "x-authenticated-user": "alice@example.test" }, + headers: { + "x-thoth-principal-issuer": "portal", + "x-thoth-principal-subject": "42", + "x-thoth-principal-display-name": "Alice", + "x-thoth-is-admin": "1", + "x-authenticated-user": "must-not-be-used", + }, + }); + expect(authenticated.json()).toEqual({ + issuer: "portal", subject: "42", displayName: "Alice", isAdmin: true, }); - expect(authenticated.json()).toEqual({ id: "alice@example.test" }); +}); + +test("upstream mode rejects legacy client identity headers without proxy principal fields", async () => { + const app = Fastify(); + app.addHook("preHandler", authPreHandler("upstream")); + app.get("/me", async (req) => getPrincipal(req)); + + for (const headers of [ + { "x-authenticated-user": "mallory" }, + { "x-mock-user": "mallory" }, + { "x-authenticated-user": "mallory", "x-mock-user": "mallory" }, + ]) { + expect((await app.inject({ method: "GET", url: "/me", headers })).statusCode).toBe(401); + } +}); + +test("local identity expands tilde homes and restores private POSIX permissions", () => { + expect(expandLocalHome("~/thoth-test", "/home/tester")).toBe("/home/tester/thoth-test"); + expect(expandLocalHome("~", "/home/tester")).toBe("/home/tester"); + const home = mkdtempSync(join(tmpdir(), "thoth-principal-")); + chmodSync(home, 0o755); + const previous = process.env.THT_HOME; + process.env.THT_HOME = home; + try { + localPrincipal(); + if (process.platform !== "win32") { + expect(statSync(home).mode & 0o777).toBe(0o700); + expect(statSync(join(home, "identity.json")).mode & 0o777).toBe(0o600); + } + } finally { + if (previous === undefined) delete process.env.THT_HOME; else process.env.THT_HOME = previous; + rmSync(home, { recursive: true, force: true }); + } }); diff --git a/backend/test/config.test.ts b/backend/test/config.test.ts index f10d0b5c..7db92b3c 100644 --- a/backend/test/config.test.ts +++ b/backend/test/config.test.ts @@ -44,9 +44,60 @@ test("loadConfig accepts an authenticated upstream trust boundary", () => { expect(loadConfig({ THOTH_PUBLIC_EXPOSURE: "true", AUTH_MODE: "upstream", + THT_SESSION_STORAGE: "postgres", + THT_SESSION_DB_HOST: "db.internal", + THT_SESSION_DB_NAME: "thoth", + THT_SESSION_RUNTIME_USER: "thoth_sessions_app", + THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password", + THT_SESSION_DB_SSLMODE: "verify-full", + THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem", }).authMode).toBe("upstream"); }); +test("loadConfig requires direct PostgreSQL TLS inputs for the public server session store", () => { + const env = { + THOTH_PUBLIC_EXPOSURE: "true", + AUTH_MODE: "upstream", + THT_SESSION_STORAGE: "postgres", + THT_SESSION_DB_HOST: "db.internal", + THT_SESSION_DB_NAME: "thoth", + THT_SESSION_RUNTIME_USER: "thoth_sessions_app", + THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password", + THT_SESSION_DB_SSLMODE: "verify-full", + THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem", + }; + + expect(loadConfig(env).sessionStorage).toMatchObject({ + mode: "postgres", + host: "db.internal", + port: 5432, + database: "thoth", + runtimeUser: "thoth_sessions_app", + runtimePasswordFile: "/run/secrets/session_runtime_password", + sslmode: "verify-full", + sslrootcert: "/run/secrets/session_ca.pem", + }); + for (const required of [ + "THT_SESSION_DB_HOST", "THT_SESSION_DB_NAME", "THT_SESSION_RUNTIME_USER", + "THT_SESSION_RUNTIME_PASSWORD_FILE", "THT_SESSION_DB_SSLMODE", "THT_SESSION_DB_SSLROOTCERT", + ]) { + const missing = { ...env, [required]: undefined }; + expect(() => loadConfig(missing)).toThrow(/server session storage configuration is invalid/); + } +}); + +test("loadConfig rejects public local storage and server storage without upstream auth", () => { + expect(() => loadConfig({ + THOTH_PUBLIC_EXPOSURE: "true", + AUTH_MODE: "upstream", + THT_SESSION_STORAGE: "local", + })).toThrow(/local session storage requires loopback-only deployment/); + expect(() => loadConfig({ + THT_SESSION_STORAGE: "postgres", + AUTH_MODE: "none", + })).toThrow(/server session storage requires AUTH_MODE=upstream/); +}); + test("loadConfig accepts only an absolute generic model key file", () => { expect(loadConfig({ THT_MODEL_API_KEY_FILE: "/run/secrets/model_api_key" }).modelApiKeyFile) .toBe("/run/secrets/model_api_key"); diff --git a/backend/test/health.test.ts b/backend/test/health.test.ts index 42add98f..6c2008ac 100644 --- a/backend/test/health.test.ts +++ b/backend/test/health.test.ts @@ -15,6 +15,13 @@ test("GET /health remains available to container probes in upstream auth mode", THT_HARNESS_DIR: "/tmp/h", AUTH_MODE: "upstream", THOTH_PUBLIC_EXPOSURE: "true", + THT_SESSION_STORAGE: "postgres", + THT_SESSION_DB_HOST: "db.internal", + THT_SESSION_DB_NAME: "thoth", + THT_SESSION_RUNTIME_USER: "thoth_sessions_app", + THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password", + THT_SESSION_DB_SSLMODE: "verify-full", + THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem", })); const res = await app.inject({ method: "GET", url: "/health" }); expect(res.statusCode).toBe(200); @@ -22,7 +29,9 @@ test("GET /health remains available to container probes in upstream auth mode", }); test("SSE response headers are flushed before the first event", async () => { - const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/tmp/h" })); + const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/tmp/h" }), { + thtRunner: { sessionShow: async () => ({ id: "header-probe" }) } as any, + }); await app.listen({ port: 0, host: "127.0.0.1" }); const port = (app.server.address() as { port: number }).port; const controller = new AbortController(); diff --git a/backend/test/pi-spawn-args.test.ts b/backend/test/pi-spawn-args.test.ts index 2488f09d..45ed3621 100644 --- a/backend/test/pi-spawn-args.test.ts +++ b/backend/test/pi-spawn-args.test.ts @@ -32,3 +32,28 @@ test("production spawnFn launches `pi --mode rpc` with no --approve (pi 0.73 dro expect(args).not.toContain("--approve"); mgr.teardown("s1"); }); + +test("Pi child replaces stale principal env and omits absent display names", async () => { + const saved = Object.fromEntries([ + "THT_PRINCIPAL_ISSUER", "THT_PRINCIPAL_SUBJECT", "THT_PRINCIPAL_DISPLAY_NAME", "THT_PRINCIPAL_IS_ADMIN", + ].map((key) => [key, process.env[key]])); + Object.assign(process.env, { + THT_PRINCIPAL_ISSUER: "stale", THT_PRINCIPAL_SUBJECT: "stale", THT_PRINCIPAL_DISPLAY_NAME: "stale", + THT_PRINCIPAL_IS_ADMIN: "true", + }); + try { + (nodeSpawn as any).mockClear(); + const mgr = new PiProcessManager(loadConfig({})); + await mgr.spawnFor("s-principal", { principal: { issuer: "portal", subject: "42", isAdmin: false } }); + const env = (nodeSpawn as any).mock.calls[0][2].env; + expect(env).toMatchObject({ + THT_PRINCIPAL_ISSUER: "portal", THT_PRINCIPAL_SUBJECT: "42", THT_PRINCIPAL_IS_ADMIN: "false", + }); + expect(env).not.toHaveProperty("THT_PRINCIPAL_DISPLAY_NAME"); + mgr.teardown("s-principal"); + } finally { + for (const [key, value] of Object.entries(saved)) { + if (value === undefined) delete process.env[key]; else process.env[key] = value; + } + } +}); diff --git a/backend/test/routes-sessions.test.ts b/backend/test/routes-sessions.test.ts index f5ce8718..96b6c719 100644 --- a/backend/test/routes-sessions.test.ts +++ b/backend/test/routes-sessions.test.ts @@ -28,6 +28,126 @@ function deferred() { return { promise, resolve, reject }; } +const aliceHeaders = { + "x-thoth-principal-issuer": "portal", + "x-thoth-principal-subject": "alice", + "x-thoth-principal-display-name": "Alice", + "x-thoth-is-admin": "0", +}; + +test("upstream requests without a principal fail before a Pi runtime can be created", async () => { + let created = false; + const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), { + mgr: { createFor: () => { created = true; throw new Error("must not spawn"); } } as any, + thtRunner: {} as any, + }); + + const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } }); + + expect(response.statusCode).toBe(401); + expect(created).toBe(false); +}); + +test("session routes conceal foreign or missing sessions and deny SSE before it subscribes", async () => { + let subscribed = false; + const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), { + thtRunner: { + withPrincipal: () => ({ sessionShow: async () => null }), + } as any, + hub: { subscribe: () => { subscribed = true; return () => {}; } } as any, + }); + + const document = await app.inject({ method: "GET", url: "/sessions/foreign/documents", headers: aliceHeaders }); + const events = await app.inject({ method: "GET", url: "/sessions/foreign/events", headers: aliceHeaders }); + + expect(document.statusCode).toBe(404); + expect(events.statusCode).toBe(404); + expect(subscribed).toBe(false); +}); + +test("session listing permits all scope only to admins", async () => { + const seen: boolean[] = []; + const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), { + thtRunner: { + withPrincipal: (principal: any) => ({ + sessionList: async () => { seen.push(principal.isAdmin); return [{ id: "s1" }]; }, + }), + } as any, + }); + const regularAll = await app.inject({ method: "GET", url: "/sessions?scope=all", headers: aliceHeaders }); + const mine = await app.inject({ method: "GET", url: "/sessions?scope=mine", headers: aliceHeaders }); + const adminAll = await app.inject({ + method: "GET", url: "/sessions?scope=all", + headers: { ...aliceHeaders, "x-thoth-is-admin": "1" }, + }); + + expect(regularAll.statusCode).toBe(403); + expect(mine.statusCode).toBe(200); + expect(adminAll.statusCode).toBe(200); + expect(seen).toEqual([false, true]); +}); + +test("A, B, and admin requests preserve owner isolation through session route mutations", async () => { + const owners = new Map([["a", "alice"], ["b", "bob"]]); + const closed: Array<{ id: string; subject: string }> = []; + const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), { + thtRunner: { + withPrincipal: (principal: any) => ({ + sessionList: async () => [...owners] + .filter(([, owner]) => principal.isAdmin || owner === principal.subject) + .map(([id, owner]) => ({ id, author: owner })), + sessionShow: async (id: string) => + (principal.isAdmin || owners.get(id) === principal.subject) ? { id, status: "open" } : null, + closeSession: async (id: string) => { closed.push({ id, subject: principal.subject }); }, + }), + } as any, + mgr: { get: () => undefined } as any, + hub: { clear: () => {} } as any, + getSettings: () => ({ workspace: "w" }) as any, + }); + const bobHeaders = { ...aliceHeaders, "x-thoth-principal-subject": "bob" }; + const adminHeaders = { ...aliceHeaders, "x-thoth-principal-subject": "admin", "x-thoth-is-admin": "1" }; + + expect((await app.inject({ method: "GET", url: "/sessions", headers: aliceHeaders })).json()) + .toEqual([{ id: "a", author: "alice" }]); + expect((await app.inject({ method: "GET", url: "/sessions", headers: bobHeaders })).json()) + .toEqual([{ id: "b", author: "bob" }]); + expect((await app.inject({ method: "GET", url: "/sessions?scope=all", headers: adminHeaders })).json()) + .toEqual([{ id: "a", author: "alice" }, { id: "b", author: "bob" }]); + + expect((await app.inject({ method: "POST", url: "/sessions/a/close", headers: bobHeaders })).statusCode) + .toBe(404); + expect((await app.inject({ method: "POST", url: "/sessions/a/close", headers: adminHeaders })).statusCode) + .toBe(200); + expect(closed).toEqual([{ id: "a", subject: "admin" }]); +}); + +test("new sessions are created through the authenticated principal, not a client owner field", async () => { + let principal: any; + const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), { + thtRunner: { + withPrincipal: (p: any) => { + principal = p; + return { sessionNew: async () => ({ id: "owned" }), searchPack: async () => {} }; + }, + } as any, + readiness: { ensure: async () => ({ ok: true }) } as any, + mgr: { + createFor: () => ({ bridge: { onClientEvent: () => {} } }), + configure: async () => {}, start: () => {}, get: () => undefined, + } as any, + getSettings: () => ({ workspace: "w" }) as any, + }); + + const response = await app.inject({ + method: "POST", url: "/sessions", headers: aliceHeaders, + payload: { question: "q", owner: "mallory" }, + }); + + expect(response.statusCode).toBe(200); + expect(principal).toMatchObject({ issuer: "portal", subject: "alice" }); +}); + test("POST /sessions usa i settings (workspace/provider/model/thinking) e crea+avvia", async () => { const modelKey = path.join(os.tmpdir(), `thoth-model-key-${process.pid}`); writeFileSync(modelKey, "test-model-key", { mode: 0o600 }); @@ -455,7 +575,9 @@ test("concurrent cold Resume requests serialize and create one runtime", async ( expect(firstResponse.json()).toEqual({ id: "s1", alreadyActive: false }); expect(secondResponse.json()).toEqual({ id: "s1", alreadyActive: true }); expect({ showCalls, readinessCalls, reopenCalls, createCalls, clearCalls }).toEqual({ - showCalls: 1, + // Each caller is authorized against repository ownership, including the request which + // finds the runtime already active after waiting on the lifecycle lock. + showCalls: 2, readinessCalls: 1, reopenCalls: 1, createCalls: 1, @@ -942,7 +1064,7 @@ test("Delete followed by Resume cannot resurrect the deleted session", async () await deleteResponse; const resumed = await resumeResponse; - expect(resumed.statusCode).toBe(500); + expect(resumed.statusCode).toBe(503); expect(current).toBeUndefined(); expect(createCalls).toBe(0); }); @@ -1195,6 +1317,27 @@ test("POST /sessions/:id/rename calls setName", async () => { expect(arg).toEqual({ id: "s1", name: "N" }); }); +test("rename authorizes and mutates through the same selected workspace", async () => { + const workspaces: string[] = []; + const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), { + thtRunner: { + withPrincipal: () => ({ + sessionShow: async (_id: string, workspace: string) => { workspaces.push(`show:${workspace}`); return { id: "s1" }; }, + setName: async (_id: string, _name: string, workspace: string) => { workspaces.push(`set:${workspace}`); }, + }), + } as any, + getSettings: () => ({ workspace: "tenant-a" }) as any, + }); + const res = await app.inject({ + method: "POST", url: "/sessions/s1/rename", payload: { name: "N" }, + headers: { + "x-thoth-principal-issuer": "portal", "x-thoth-principal-subject": "42", "x-thoth-is-admin": "false", + }, + }); + expect(res.statusCode).toBe(204); + expect(workspaces).toEqual(["show:tenant-a", "set:tenant-a"]); +}); + test("POST /sessions/:id/group calls setGroup", async () => { let arg: any; const app = mutApp({ setGroup: async (id: string, group: string) => { arg = { id, group }; } }); @@ -1303,6 +1446,24 @@ test("POST /sessions readiness failure returns one fixed public message without expect(createdCalled).toBe(false); }); +test("POST /sessions returns storage 503 before creating a Pi runtime when session persistence fails", async () => { + let piCreated = false; + const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), { + thtRunner: { + ollamaEnsure: async () => ({ ok: true }), + sessionNew: async () => { throw new Error("database unavailable"); }, + } as any, + getSettings: () => ({ workspace: "psd" }) as any, + mgr: { createFor: () => { piCreated = true; throw new Error("must not spawn"); } } as any, + }); + + const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } }); + + expect(response.statusCode).toBe(503); + expect(response.json()).toEqual({ error: "session storage is unavailable" }); + expect(piCreated).toBe(false); +}); + test("POST /sessions proceeds when ollamaEnsure succeeds", async () => { let ensureWs: string | undefined; const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), { diff --git a/backend/test/routes-settings.test.ts b/backend/test/routes-settings.test.ts index d83dc0d0..f25a4964 100644 --- a/backend/test/routes-settings.test.ts +++ b/backend/test/routes-settings.test.ts @@ -1,5 +1,5 @@ import { test, expect } from "vitest"; -import { mkdtempSync, rmSync } from "node:fs"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { buildApp } from "../src/app.js"; @@ -48,6 +48,94 @@ test("PUT /settings persists and GET reads it back", async () => { } }); +test("settings are isolated by the authenticated repository principal", async () => { + const preferences = new Map(); + const runner = { + withPrincipal: (principal: any) => ({ + preferencesGet: async () => preferences.get(principal.subject) ?? {}, + preferencesSet: async (next: any) => { preferences.set(principal.subject, next); }, + }), + }; + const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), { + thtRunner: runner as any, + listModels: async () => [], + }); + const headers = (subject: string) => ({ + "x-thoth-principal-issuer": "portal", + "x-thoth-principal-subject": subject, + "x-thoth-is-admin": "0", + }); + + await app.inject({ + method: "PUT", url: "/settings", headers: headers("alice"), + payload: { workspace: "psd", provider: "zai", model: "glm-5.2", thinking: "high" }, + }); + const alice = await app.inject({ method: "GET", url: "/settings", headers: headers("alice") }); + const bob = await app.inject({ method: "GET", url: "/settings", headers: headers("bob") }); + + expect(alice.json()).toMatchObject({ workspace: "psd", thinking: "high" }); + expect(bob.json()).not.toMatchObject({ workspace: "psd", thinking: "high" }); +}); + +test("GET /settings seeds an empty private profile from complete legacy settings once", async () => { + let preferences: Record = {}; + const writes: Record[] = []; + const runner = { + withPrincipal: () => ({ + preferencesGet: async () => preferences, + preferencesSet: async (next: Record) => { + writes.push(next); + preferences = next; + }, + }), + }; + const { app, dir } = appWithTmpSettings({}, { thtRunner: runner as any, listModels: async () => [] }); + try { + writeFileSync(join(dir, "settings.json"), JSON.stringify({ + workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low", + })); + + const first = await app.inject({ method: "GET", url: "/settings" }); + const second = await app.inject({ method: "GET", url: "/settings" }); + + const expected = { + workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low", + }; + expect(first.statusCode).toBe(200); + expect(first.json()).toEqual(expected); + expect(second.json()).toEqual(expected); + expect(writes).toEqual([expected]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("GET /settings does not overwrite an existing private profile with legacy settings", async () => { + const privateSettings = { + workspace: "private", provider: "zai", model: "glm-5.2", thinking: "high", + }; + const preferencesSet = async () => { throw new Error("must not seed an existing profile"); }; + const runner = { + withPrincipal: () => ({ + preferencesGet: async () => privateSettings, + preferencesSet, + }), + }; + const { app, dir } = appWithTmpSettings({}, { thtRunner: runner as any, listModels: async () => [] }); + try { + writeFileSync(join(dir, "settings.json"), JSON.stringify({ + workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low", + })); + + const response = await app.inject({ method: "GET", url: "/settings" }); + + expect(response.statusCode).toBe(200); + expect(response.json()).toEqual(privateSettings); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + test("PUT /settings rejects an unknown model when a model list is available", async () => { const { app, dir } = appWithTmpSettings({}, { listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }], diff --git a/backend/test/tht-runner.test.ts b/backend/test/tht-runner.test.ts index 59f22aa5..1553beb2 100644 --- a/backend/test/tht-runner.test.ts +++ b/backend/test/tht-runner.test.ts @@ -83,6 +83,31 @@ test("run omits ambient THT_DATA_ROOT when config does not provide one", async ( } }); +test("principal-bound tht child replaces stale principal env and omits an absent display name", async () => { + const saved = Object.fromEntries([ + "THT_PRINCIPAL_ISSUER", "THT_PRINCIPAL_SUBJECT", "THT_PRINCIPAL_DISPLAY_NAME", "THT_PRINCIPAL_IS_ADMIN", + ].map((key) => [key, process.env[key]])); + Object.assign(process.env, { + THT_PRINCIPAL_ISSUER: "stale-issuer", THT_PRINCIPAL_SUBJECT: "stale-subject", + THT_PRINCIPAL_DISPLAY_NAME: "Stale Name", THT_PRINCIPAL_IS_ADMIN: "true", + }); + try { + (spawn as any).mockClear(); + const runner = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" }) + .withPrincipal({ issuer: "portal", subject: "42", isAdmin: false }); + await runner.run(["session", "list", "--json"]); + const env = (spawn as any).mock.calls[0][2].env; + expect(env).toMatchObject({ + THT_PRINCIPAL_ISSUER: "portal", THT_PRINCIPAL_SUBJECT: "42", THT_PRINCIPAL_IS_ADMIN: "false", + }); + expect(env).not.toHaveProperty("THT_PRINCIPAL_DISPLAY_NAME"); + } finally { + for (const [key, value] of Object.entries(saved)) { + if (value === undefined) delete process.env[key]; else process.env[key] = value; + } + } +}); + test("run with exit != 0 propagates error with stderr", async () => { const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" }); r.run = async () => ({ code: 1, stdout: "", stderr: "ERRORE: boom" }); @@ -137,23 +162,25 @@ test("setName builds the right argv", async () => { const calls: string[][] = []; const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" }); r.run = async (args) => { calls.push(args); return { code: 0, stdout: "", stderr: "" }; }; - await r.setName("sid", "Mio nome"); + await r.setName("sid", "Mio nome", "tenant-a"); expect(calls[0]).toEqual(["session", "set-name", "sid", "--name", "Mio nome"]); }); -test("setGroup / archive / unarchive / deleteSession build argv", async () => { - const calls: string[][] = []; +test("mutation and document commands retain their requested workspace", async () => { + const calls: Array<{ args: string[]; workspace?: string }> = []; const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" }); - r.run = async (args) => { calls.push(args); return { code: 0, stdout: "", stderr: "" }; }; - await r.setGroup("sid", "G1"); - await r.archive("sid"); - await r.unarchive("sid"); - await r.deleteSession("sid"); + r.run = async (args, workspace) => { calls.push({ args, workspace }); return { code: 0, stdout: "[]", stderr: "" }; }; + await r.setGroup("sid", "G1", "tenant-a"); + await r.archive("sid", "tenant-a"); + await r.unarchive("sid", "tenant-a"); + await r.documents("sid", "tenant-a"); + await r.deleteSession("sid", "tenant-a"); expect(calls).toEqual([ - ["session", "set-group", "sid", "--group", "G1"], - ["session", "archive", "sid"], - ["session", "unarchive", "sid"], - ["session", "delete", "sid"], + { args: ["session", "set-group", "sid", "--group", "G1"], workspace: "tenant-a" }, + { args: ["session", "archive", "sid"], workspace: "tenant-a" }, + { args: ["session", "unarchive", "sid"], workspace: "tenant-a" }, + { args: ["session", "documents", "sid", "--json"], workspace: "tenant-a" }, + { args: ["session", "delete", "sid"], workspace: "tenant-a" }, ]); }); diff --git a/deploy/compose.session-server.yaml.example b/deploy/compose.session-server.yaml.example new file mode 100644 index 00000000..fbb1bc4f --- /dev/null +++ b/deploy/compose.session-server.yaml.example @@ -0,0 +1,52 @@ +# Opt-in server overlay for user-owned PostgreSQL sessions. Copy this file to a +# reviewed local override; it is intentionally not loaded by default Compose. +services: + core: + environment: + AUTH_MODE: upstream + THOTH_PUBLIC_EXPOSURE: "true" + THT_SESSION_STORAGE: postgres + THT_CONFIG: /app/harness/workspaces/server-sessions.yaml + THT_SESSION_DB_HOST: ${THT_SESSION_DB_HOST:?set THT_SESSION_DB_HOST} + THT_SESSION_DB_PORT: ${THT_SESSION_DB_PORT:-5432} + THT_SESSION_DB_NAME: ${THT_SESSION_DB_NAME:?set THT_SESSION_DB_NAME} + THT_SESSION_RUNTIME_USER: ${THT_SESSION_RUNTIME_USER:?set THT_SESSION_RUNTIME_USER} + THT_SESSION_RUNTIME_PASSWORD_FILE: /run/secrets/session_runtime_password + THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full} + THT_SESSION_DB_SSLROOTCERT: /run/secrets/session_ca.pem + secrets: + - source: session_runtime_password + target: session_runtime_password + - source: session_ca + target: session_ca.pem + volumes: + - ./deploy/workspaces:/app/harness/workspaces:ro + + # Run manually during the maintenance window. It is not a dependency of core, + # so the application never gains the schema-changing migrator credential. + session-migrate: + image: thothii-core:local + profiles: [session-migrate] + entrypoint: [/app/docker/session-migrate.sh] + environment: + THT_SESSION_DB_HOST: ${THT_SESSION_DB_HOST:?set THT_SESSION_DB_HOST} + THT_SESSION_DB_PORT: ${THT_SESSION_DB_PORT:-5432} + THT_SESSION_DB_NAME: ${THT_SESSION_DB_NAME:?set THT_SESSION_DB_NAME} + THT_SESSION_MIGRATOR_USER: ${THT_SESSION_MIGRATOR_USER:?set THT_SESSION_MIGRATOR_USER} + THT_SESSION_MIGRATOR_PASSWORD_FILE: /run/secrets/session_migrator_password + THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full} + THT_SESSION_DB_SSLROOTCERT: /run/secrets/session_ca.pem + secrets: + - source: session_migrator_password + target: session_migrator_password + - source: session_ca + target: session_ca.pem + restart: "no" + +secrets: + session_runtime_password: + file: ${THT_SESSION_RUNTIME_PASSWORD_SOURCE:?set THT_SESSION_RUNTIME_PASSWORD_SOURCE} + session_migrator_password: + file: ${THT_SESSION_MIGRATOR_PASSWORD_SOURCE:?set THT_SESSION_MIGRATOR_PASSWORD_SOURCE} + session_ca: + file: ${THT_SESSION_CA_SOURCE:?set THT_SESSION_CA_SOURCE} diff --git a/deploy/env.example b/deploy/env.example index 1a1da078..881dfca1 100644 --- a/deploy/env.example +++ b/deploy/env.example @@ -14,6 +14,19 @@ PI_THINKING= MAX_PI_PROCESSES=4 AUTH_MODE=none +# User-owned session storage. Keep local for the loopback-only development stack. +# The server-session overlay requires every THT_SESSION_* value below. +THT_SESSION_STORAGE=local +THT_SESSION_DB_HOST= +THT_SESSION_DB_PORT=5432 +THT_SESSION_DB_NAME= +THT_SESSION_RUNTIME_USER= +THT_SESSION_RUNTIME_PASSWORD_SOURCE= +THT_SESSION_MIGRATOR_USER= +THT_SESSION_MIGRATOR_PASSWORD_SOURCE= +THT_SESSION_DB_SSLMODE=verify-full +THT_SESSION_CA_SOURCE= + THT_DB_NAME= THT_DWH_REST_URL= THT_VEC_REST_URL= diff --git a/deploy/secrets/README.md b/deploy/secrets/README.md index 58898e8b..8ea67f50 100644 --- a/deploy/secrets/README.md +++ b/deploy/secrets/README.md @@ -43,3 +43,19 @@ password into `THT_VECTOR_BOOTSTRAP_PASSWORD` in the bundle before restarting Hosted Pi providers must use a single provider key. Compound providers (Bedrock, Azure OpenAI Responses, Cloudflare Workers AI/Gateway) fail closed until a provider-specific credential adapter is implemented. + +## User-owned session database secrets + +The server-session overlay deliberately does **not** add session database credentials to the +shared bundle. Materialize three distinct Docker secrets from protected host or secret-manager +files: `session_runtime_password`, `session_migrator_password`, and `session_ca.pem`. Their host +source paths are respectively `THT_SESSION_RUNTIME_PASSWORD_SOURCE`, +`THT_SESSION_MIGRATOR_PASSWORD_SOURCE`, and `THT_SESSION_CA_SOURCE`; all must be absolute paths +outside the repository. The runtime password is mounted only into `core`; the migrator password +is mounted only into the one-shot `session-migrate` service. Do not reuse either login for the +other role. + +`session_ca.pem` is a PEM file rather than a bundle value because the bundle rejects whitespace. +The server workspace receives only its mount path through `THT_SESSION_DB_SSLROOTCERT`; it uses +`THT_SESSION_DB_SSLMODE=verify-ca` or, normally, `verify-full`. TLS disable/prefer/require modes +are unsupported for session storage. diff --git a/deploy/workspaces/server-sessions.yaml.example b/deploy/workspaces/server-sessions.yaml.example new file mode 100644 index 00000000..69402679 --- /dev/null +++ b/deploy/workspaces/server-sessions.yaml.example @@ -0,0 +1,36 @@ +# Copy to deploy/workspaces/server-sessions.yaml for the user-owned-session server profile. +# The runtime login is intentionally separate from the one-shot migrator login. +language: en + +dwh: + type: thoth_rest + database: + database: ${THT_DB_NAME} + schema: datawarehouse + endpoint: + base_url: ${THT_DWH_REST_URL} + api_key: ${THT_DWH_API_KEY} + ssl_ca: ${THT_SSL_CA} + +session_storage: + type: postgres_direct + connection: + host: ${THT_SESSION_DB_HOST} + port: ${THT_SESSION_DB_PORT} + database: ${THT_SESSION_DB_NAME} + schema: thoth_sessions + user: ${THT_SESSION_RUNTIME_USER} + password_file: ${THT_SESSION_RUNTIME_PASSWORD_FILE} + sslmode: ${THT_SESSION_DB_SSLMODE} + sslrootcert: ${THT_SESSION_DB_SSLROOTCERT} + +roots: + artifacts: artifacts + indexes: indexes + sessions: sessions + +embeddings: + base_url: ${THT_OLLAMA_URL} + model: nomic-embed-text-v2-moe + dim: 768 + batch_size: 32 diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index 9ac0c780..5a51d0f5 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -18,6 +18,8 @@ services: THT_BIN: /opt/venv/bin/tht PI_BIN: pi AUTH_MODE: ${AUTH_MODE:-none} + THT_SESSION_STORAGE: local + THT_HOME: /data/local-home SETTINGS_FILE: /data/settings/settings.json MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4} extra_hosts: @@ -27,7 +29,7 @@ services: - /home/chirone/thothii-data/pi-config:/home/thoth/.pi - /home/chirone/chirone/etl/docs/evidence:/data/evidence:ro ports: - - "8787:8787" + - "127.0.0.1:8787:8787" restart: "no" networks: [thothii-net] @@ -40,7 +42,7 @@ services: VITE_BACKEND_URL: /api image: thothii-frontend:local ports: - - "8090:8080" + - "127.0.0.1:8090:8080" depends_on: - core restart: "no" diff --git a/docker/core.Dockerfile b/docker/core.Dockerfile index 7abb1088..db75fe6d 100644 --- a/docker/core.Dockerfile +++ b/docker/core.Dockerfile @@ -63,8 +63,8 @@ ENV PATH="/opt/venv/bin:/usr/local/bin:$PATH" \ PI_BIN=pi \ HOME=/home/thoth -COPY docker/core-entrypoint.sh docker/ensure-pi-trust.mjs /app/docker/ -RUN chmod +x /app/docker/core-entrypoint.sh +COPY docker/core-entrypoint.sh docker/session-migrate.sh docker/ensure-pi-trust.mjs /app/docker/ +RUN chmod +x /app/docker/core-entrypoint.sh /app/docker/session-migrate.sh WORKDIR /app/backend USER thoth diff --git a/docker/cutover-legacy-sessions.sh b/docker/cutover-legacy-sessions.sh new file mode 100755 index 00000000..80b261bf --- /dev/null +++ b/docker/cutover-legacy-sessions.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# Archive exactly three legacy filesystem sessions before optionally deleting them. +# This helper is deliberately inert unless --delete is supplied after archive verification. +set -euo pipefail + +usage() { + echo "usage: $0 [--delete] SOURCE_SESSIONS_DIR BACKUP.tar SESSION_ID SESSION_ID SESSION_ID" >&2 + exit 2 +} + +delete_after_backup=false +if [[ ${1:-} == "--delete" ]]; then + delete_after_backup=true + shift +fi +[[ $# -eq 5 ]] || usage + +source_dir=$1 +backup=$2 +shift 2 +ids=("$@") + +[[ -d $source_dir ]] || { echo "legacy session root is not a directory" >&2; exit 1; } +[[ ! -e $backup ]] || { echo "backup already exists; refusing to overwrite it" >&2; exit 1; } +[[ ${ids[0]} != "${ids[1]}" && ${ids[0]} != "${ids[2]}" && ${ids[1]} != "${ids[2]}" ]] \ + || { echo "exactly three distinct legacy session IDs are required" >&2; exit 1; } + +paths=() +for id in "${ids[@]}"; do + [[ $id =~ ^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$ ]] \ + || { echo "invalid legacy session ID: $id" >&2; exit 1; } + [[ -f "$source_dir/$id/session_manifest.yaml" ]] \ + || { echo "legacy session manifest is missing: $id" >&2; exit 1; } + paths+=("$id") +done + +tar --create --file "$backup" --directory "$source_dir" -- "${paths[@]}" +tar --list --file "$backup" >/dev/null +sha256sum "$backup" >"$backup.sha256" +echo "verified backup: $backup" +echo "checksum: $backup.sha256" + +if ! $delete_after_backup; then + echo "no legacy sessions deleted; review the archive, then rerun with --delete during maintenance" >&2 + exit 0 +fi + +for id in "${ids[@]}"; do + rm -rf -- "$source_dir/$id" +done +echo "deleted exactly three archived legacy sessions" diff --git a/docker/session-migrate.sh b/docker/session-migrate.sh new file mode 100755 index 00000000..7e040ce7 --- /dev/null +++ b/docker/session-migrate.sh @@ -0,0 +1,40 @@ +#!/bin/sh +# One-shot schema migration only. Validate TLS before reading a secret or composing a URL. +set -eu + +case "${THT_SESSION_DB_SSLMODE:-}" in + verify-ca|verify-full) ;; + *) + echo "THT_SESSION_DB_SSLMODE must be verify-ca or verify-full" >&2 + exit 2 + ;; +esac + +require_env() { + eval "value=\${$1:-}" + if [ -z "$value" ]; then + echo "missing required session migrator configuration" >&2 + exit 2 + fi +} + +require_env THT_SESSION_DB_HOST +require_env THT_SESSION_DB_PORT +require_env THT_SESSION_DB_NAME +require_env THT_SESSION_MIGRATOR_USER +require_env THT_SESSION_MIGRATOR_PASSWORD_FILE +require_env THT_SESSION_DB_SSLROOTCERT + +if [ ! -r "$THT_SESSION_MIGRATOR_PASSWORD_FILE" ]; then + echo "session migrator credential is unavailable" >&2 + exit 2 +fi +export PGPASSWORD="$(cat "$THT_SESSION_MIGRATOR_PASSWORD_FILE")" +if [ -z "$PGPASSWORD" ]; then + echo "session migrator credential is unavailable" >&2 + exit 2 +fi + +exec "${THT_BIN:-/opt/venv/bin/tht}" session migrate --database-url \ + "postgresql+psycopg2://${THT_SESSION_MIGRATOR_USER}@${THT_SESSION_DB_HOST}:${THT_SESSION_DB_PORT}/${THT_SESSION_DB_NAME}?sslmode=${THT_SESSION_DB_SSLMODE}&sslrootcert=${THT_SESSION_DB_SSLROOTCERT}" \ + --json diff --git a/docker/session-migrate.test.sh b/docker/session-migrate.test.sh new file mode 100755 index 00000000..1891d21f --- /dev/null +++ b/docker/session-migrate.test.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +set -euo pipefail + +root=$(cd "$(dirname "$0")/.." && pwd) +helper="$root/docker/session-migrate.sh" +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT HUP INT TERM + +if THT_SESSION_DB_SSLMODE=prefer sh "$helper" >"$tmp/invalid.out" 2>&1; then + echo "session migrator accepted an unverified TLS mode" >&2 + exit 1 +fi +grep -qx 'THT_SESSION_DB_SSLMODE must be verify-ca or verify-full' "$tmp/invalid.out" + +printf '%s' password >"$tmp/password" +cat >"$tmp/tht" <<'EOF' +#!/bin/sh +printf '%s\n' "$@" >"$ARGS_FILE" +EOF +chmod +x "$tmp/tht" + +ARGS_FILE="$tmp/args" \ +THT_BIN="$tmp/tht" \ +THT_SESSION_DB_HOST=sessions-db.internal \ +THT_SESSION_DB_PORT=5432 \ +THT_SESSION_DB_NAME=thoth \ +THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate \ +THT_SESSION_MIGRATOR_PASSWORD_FILE="$tmp/password" \ +THT_SESSION_DB_SSLMODE=verify-full \ +THT_SESSION_DB_SSLROOTCERT=/run/secrets/session_ca.pem \ +sh "$helper" + +grep -qx 'session' "$tmp/args" +grep -qx 'migrate' "$tmp/args" +grep -qx -- '--database-url' "$tmp/args" +grep -Fxq 'postgresql+psycopg2://thoth_sessions_migrate@sessions-db.internal:5432/thoth?sslmode=verify-full&sslrootcert=/run/secrets/session_ca.pem' "$tmp/args" + +echo "session migrator TLS contract passed." diff --git a/docs/superpowers/plans/2026-07-16-user-owned-session-storage.md b/docs/superpowers/plans/2026-07-16-user-owned-session-storage.md new file mode 100644 index 00000000..54238d62 --- /dev/null +++ b/docs/superpowers/plans/2026-07-16-user-owned-session-storage.md @@ -0,0 +1,130 @@ +# User-owned sessions implementation plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Associate every ThothII session and preference with a stable logged-in or local OS principal. + +**Architecture:** The harness owns a repository contract. `FilesystemSessionRepository` stores readable phase documents under the local user home; `PostgresSessionRepository` stores the same logical snapshot in a private Supabase schema. The backend resolves a trusted principal before every action and passes it through to `tht`; the portal proxy supplies that identity. + +**Tech Stack:** Python 3.12, Typer, SQLAlchemy/psycopg2, PostgreSQL/Supabase RLS, Fastify/TypeScript, React/Vitest, Django/nginx. + +## Global Constraints + +- Server storage is PostgreSQL wire protocol with TLS verification, schema `thoth_sessions`; never PostgREST or browser DB access. +- Server identity is `(issuer, Django request.user.pk)`; clients never choose session owner. +- Admin authorization uses Authentik group `authentik Admins`; unauthorized and missing session both return HTTP 404. +- Server has no persistent session filesystem fallback or dual write; DB failure is HTTP 503 before Pi starts. +- Local mode uses `~/.thothii` or `THT_HOME`, runs loopback-only, and creates a UUID identity in `identity.json`. +- Persist current artifacts and append-only decisions only; no chat transcript, artifact revisions, session embeddings, or content in audit logs. +- New session IDs are UUIDv4. Existing `solved_question` behavior is unchanged. +- Follow TDD: each behavior test must fail before its implementation is written. + +--- + +### Task 1: Session repository contract and local implementation + +**Files:** +- Create: `harness/tht/session/repository.py`, `harness/tht/session/filesystem_repository.py` +- Modify: `harness/tht/config.py`, `harness/tht/session/store.py`, `harness/tht/session/models.py`, `harness/pyproject.toml` +- Test: `harness/tests/test_session_repository.py`, `harness/tests/test_local_identity.py` + +**Interfaces:** Produce `PrincipalContext`, `SessionSnapshot`, `SessionRepository`, and `build_session_repository(config, principal)`. The filesystem adapter must preserve the present documents under `/workspaces//sessions/` and read/write artifact keys, decisions, manifest fields, and per-principal preferences. + +- [ ] Write failing tests proving UUIDv4 creation, principal-scoped local roots, readable phase artifacts, decision append, `THT_HOME` override, and an identity UUID stable across restarts. +- [ ] Run `cd harness && .venv/bin/pytest tests/test_session_repository.py tests/test_local_identity.py -q`; confirm failure because repository and identity interfaces do not exist. +- [ ] Implement the smallest repository contract and filesystem adapter. Use `portalocker` for mutations and private local directory/file permissions where supported. +- [ ] Run the focused tests and `cd harness && .venv/bin/pytest -q && .venv/bin/ruff check .`. +- [ ] Commit `feat(harness): add local session repository`. + +### Task 2: PostgreSQL schema, migrations, and repository + +**Files:** +- Create: `harness/tht/session/postgres_repository.py`, `harness/tht/migrations/sessions/001_schema.sql`, `harness/tht/migrations/sessions/002_security.sql` +- Modify: `harness/tht/cli/session_cmd.py`, `harness/tht/config.py` +- Test: `harness/tests/test_postgres_session_repository.py`, `harness/tests/test_session_migrate_cmd.py` + +**Interfaces:** Add `tht session migrate --database-url URL [--status] --json`; `PostgresSessionRepository` implements the Task 1 contract, starts a transaction, sets actor/admin local settings, and maps `cte_sql:` to artifacts. + +- [ ] Write failing unit/integration tests for migration status, owner isolation, admin cross-user access, cascade delete with content-free audit tombstone, and no embedding invocation. +- [ ] Run the focused tests and confirm expected RED failures. +- [ ] Implement idempotent migration runner, private tables, forced RLS policies, role separation, advisory transaction locks, and repository methods using direct PostgreSQL TLS settings. +- [ ] Run focused tests, then full harness tests and Ruff. +- [ ] Commit `feat(harness): persist server sessions in postgres`. + +### Task 3: Migrate harness workflow and deterministic write commands + +**Files:** +- Modify: `harness/tht/session/store.py`, `harness/tht/decisions.py`, `harness/tht/phase.py`, `harness/tht/taskdoc.py`, `harness/tht/ctetest.py`, `harness/tht/solved.py`, `harness/tht/teardown.py`, `harness/tht/cli/{session,decision,cte,sql,phase,search,memory}_cmd.py`, `harness/.pi/skills/tht-sessione/SKILL.md`, `harness/.pi/extensions/tht-gate.js` +- Test: `harness/tests/test_session_repository_workflow.py`, `harness/gate/__tests__/session-repository-writes.test.js` + +**Interfaces:** Existing workflow commands operate through the configured repository. Add `tht cte save --session ID --name NAME --file -` and `tht sql set-final --session ID --file -`; Pi tools `write_cte_sql` and `write_final_sql` use them instead of direct session paths. + +- [ ] Write failing tests that run phase calculation and finalize against an in-memory/filesystem repository, and prove Pi write tools persist CTE/final SQL without direct `sessions/` writes. +- [ ] Run focused tests and record RED results. +- [ ] Refactor path-bound helpers into snapshot/ledger and repository calls; retain backwards-compatible readable local documents and pristine `--json` stdout. +- [ ] Make finalization atomically store report/evidence/status only after DWH verification; leave solved-question creation best-effort. +- [ ] Run harness and gate test suites, then commit `refactor(harness): route workflow persistence through repositories`. + +### Task 4: Trusted portal identity and proxy forwarding + +**Files:** +- Modify: `/home/chirone/omics_portal/kokoro/datamart_catalog_views.py`, `/home/chirone/omics_portal/nginx/nginx.conf` +- Create: `/home/chirone/omics_portal/kokoro/test_thothii_auth.py` + +**Interfaces:** The auth-request response supplies only `X-Thoth-Principal-Issuer`, `X-Thoth-Principal-Subject`, `X-Thoth-Principal-Display-Name`, and `X-Thoth-Is-Admin`. nginx removes client values for those headers and forwards subrequest values to ThothII. + +- [ ] Write Django tests for authenticated capability user, denied user, Django PK subject, admin group flag, and absent/spoofed client headers. +- [ ] Run the focused Docker test and confirm RED. +- [ ] Emit normalized headers from the capability endpoint and configure nginx auth-request header capture/injection. +- [ ] Run `docker compose exec -T web python manage.py test accounts.test_capabilities kokoro.test_thothii_auth -v 2`. +- [ ] Commit the portal changes in its own repository with `feat(thothii): forward trusted principal`. + +### Task 5: Backend principal enforcement and per-user settings + +**Files:** +- Create: `backend/src/auth/principal.ts` +- Modify: `backend/src/auth/auth.ts`, `backend/src/config.ts`, `backend/src/app.ts`, `backend/src/routes/{sessions,settings,sql,meta}.ts`, `backend/src/tht/tht-runner.ts`, `backend/src/pi/pi-process-manager.ts` +- Test: `backend/test/auth.test.ts`, `backend/test/routes-sessions.test.ts`, `backend/test/sse-route.test.ts`, `backend/test/routes-settings.test.ts` + +**Interfaces:** Add `GET /me`; require a `PrincipalContext` for all session, document, SQL, response, steer and SSE operations. `GET /sessions?scope=mine|all` permits `all` only for admins. Local mode creates `~/.thothii/identity.json`; upstream mode accepts only proxy-injected normalized headers. + +- [ ] Write failing route tests for missing identity (401), foreign session (404), admin all-scope, owner assignment server-side, SSE denial before Pi spawn, and preference isolation. +- [ ] Run `cd backend && npx vitest run test/auth.test.ts test/routes-sessions.test.ts test/sse-route.test.ts test/routes-settings.test.ts`; confirm RED. +- [ ] Implement principal parser, local identity, upstream validation, repository-aware `ThtRunner`, session authorization before Pi, `/me`, scope enforcement and async per-user settings. +- [ ] Run backend Vitest, typecheck, and build; commit `feat(backend): enforce user-owned sessions`. + +### Task 6: Frontend identity and administrator UX + +**Files:** +- Modify: `frontend/src/api/{client,sessions,settings,types}.ts`, `frontend/src/shell/{AppShell,NavSessions,SessionMenu}.tsx` +- Test: `frontend/src/api/sessions.test.ts`, `frontend/src/shell/AppShell.session-mgmt.test.tsx`, `frontend/src/shell/NavSessions.test.tsx` + +**Interfaces:** Fetch `/me`; default to `scope=mine`. Display explicit `All sessions` only to admins, show owner labels/admin banner, and require confirmation before cross-owner destructive actions. + +- [ ] Write failing component/API tests for regular-user scope, admin scope switch, owner label, admin banner, and cross-owner confirmation. +- [ ] Run focused Vitest tests and confirm RED. +- [ ] Implement typed client calls, session scope state, and explicit admin affordances without changing ordinary user flow. +- [ ] Run frontend Vitest, `npx tsc -b`, build and E2E; commit `feat(frontend): expose owned session scopes`. + +### Task 7: Deployment contract, migration and cutover tooling + +**Files:** +- Modify: `docker/`, deployment examples, `README.md`, `PROJECT_STATE.md` +- Test: `harness/tests/test_session_migrate_cmd.py`, `backend/test/config.test.ts` + +**Interfaces:** Define runtime/migrator DB secret names, `AUTH_MODE=upstream` server configuration, local loopback-only configuration, and health/readiness behavior returning 503 when repository storage is unavailable. + +- [ ] Write failing config tests for required server database/TLS inputs and rejected public/local combinations. +- [ ] Run focused tests and confirm RED. +- [ ] Document runtime/migrator roles, CA/secret injection, backup then deletion of the three legacy server sessions, maintenance-mode cutover, and no dual-write rollback policy. +- [ ] Run all three layer gates; commit `docs(deploy): document user-owned session cutover`. + +### Task 8: End-to-end authorization verification and final review + +**Files:** +- Modify: test fixtures only as required by earlier tasks. + +- [ ] Add cross-layer tests for A/B/admin isolation, spoofed-header rejection, concurrent session mutation, local two-home isolation, absent embeddings, and failed DB startup before Pi. +- [ ] Run harness, backend and frontend complete gates plus portal Docker tests. +- [ ] Run the final whole-branch review, fix all Critical and Important findings, and re-run the covering tests. +- [ ] Commit `test: cover user-owned session security boundaries` and prepare the branch for integration. diff --git a/docs/superpowers/plans/2026-07-16-user-preference-bootstrap.md b/docs/superpowers/plans/2026-07-16-user-preference-bootstrap.md new file mode 100644 index 00000000..dd0e5e2e --- /dev/null +++ b/docs/superpowers/plans/2026-07-16-user-preference-bootstrap.md @@ -0,0 +1,110 @@ +# User Preference Bootstrap Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Seed a new principal's private settings from complete legacy settings so a first session can configure and start Pi. + +**Architecture:** `buildApp` continues to resolve settings through the principal-bound harness runner. When `preferencesGet()` returns an empty object, it derives the existing effective legacy settings from `SETTINGS_FILE`, persists them once via `preferencesSet()`, and returns that same object. A non-empty private object remains authoritative. + +**Tech Stack:** TypeScript, Fastify, Vitest. + +## Global Constraints + +- Never overwrite a non-empty private preference object. +- Persist only provider, model, thinking, and workspace values; no credentials enter preferences. +- Keep storage failures fail-closed through the existing 503 route contract. +- Follow TDD: observe the regression test fail before adding implementation. + +--- + +### Task 1: Bootstrap legacy settings for an empty private profile + +**Files:** +- Modify: `backend/test/routes-settings.test.ts` +- Modify: `backend/src/app.ts` + +**Interfaces:** +- Consumes: `ThtRunner.preferencesGet(): Promise>`, `ThtRunner.preferencesSet(settings): Promise`, `loadSettings(config)`, and `effectiveSettings(config, settings)`. +- Produces: `getSettings(principal): Promise` that returns a complete persisted profile for first-time principals. + +- [ ] **Step 1: Write the failing regression tests** + +```ts +test("GET /settings seeds an empty private profile from complete legacy settings once", async () => { + // Seed SETTINGS_FILE with local-qwen/qwen3.6-35b-a3b/low. + // Make preferencesGet return {} and record preferencesSet calls. + // Assert the first GET returns and persists all four settings, and a second GET does not write again. +}); + +test("GET /settings keeps a non-empty private profile authoritative", async () => { + // Seed different legacy settings, return an existing private profile, + // and assert no preferencesSet call occurs. +}); +``` + +- [ ] **Step 2: Run the focused test file and verify RED** + +Run: `cd backend && npx vitest run test/routes-settings.test.ts` + +Expected: the first test fails because the current resolver returns only defaults and never calls `preferencesSet`. + +- [ ] **Step 3: Implement the minimal resolver change** + +```ts +const stored = await runner.preferencesGet(); +if (Object.keys(stored).length === 0) { + const seeded = effectiveSettings(config, loadSettings(config)); + await runner.preferencesSet(seeded); + return seeded; +} +return effectiveSettings(config, stored); +``` + +- [ ] **Step 4: Run focused tests and TypeScript verification** + +Run: `cd backend && npx vitest run test/routes-settings.test.ts && npx tsc --noEmit -p .` + +Expected: exit 0. + +- [ ] **Step 5: Run backend regression suite** + +Run: `cd backend && npx vitest run && npm run build` + +Expected: exit 0. + +### Task 2: Preserve DWH artifact ownership across the optional session-storage field + +**Files:** +- Modify: `harness/tests/test_dwh_preprocess_job.py` +- Modify: `harness/tht/jobs/dwh_pipeline.py` + +**Interfaces:** +- Consumes: `config_dwh_binding(cfg)` and Pydantic's `Config.model_dump(mode="json")`. +- Produces: a DWH binding whose `config_fingerprint` excludes only `session_storage`. + +- [ ] **Step 1: Write the failing regression test** + +```python +def test_session_storage_does_not_change_the_dwh_artifact_binding(tmp_path): + assert config_dwh_binding(config()) == config_dwh_binding(config(session_storage={...})) +``` + +- [ ] **Step 2: Run the focused test and verify RED** + +Run: `cd harness && .venv/bin/pytest tests/test_dwh_preprocess_job.py::test_session_storage_does_not_change_the_dwh_artifact_binding -q` + +Expected: FAIL because the full configuration JSON currently includes `session_storage`. + +- [ ] **Step 3: Implement the minimal DWH-only fingerprint** + +```python +payload = cfg.model_dump(mode="json") +payload.pop("session_storage", None) +config_fingerprint = fingerprint(json.dumps(payload, separators=(",", ":"), ensure_ascii=False)) +``` + +- [ ] **Step 4: Run focused and complete harness verification** + +Run: `cd harness && .venv/bin/pytest tests/test_dwh_preprocess_job.py -q && .venv/bin/pytest -q && .venv/bin/ruff check tht/jobs/dwh_pipeline.py tests/test_dwh_preprocess_job.py` + +Expected: exit 0. diff --git a/docs/superpowers/specs/2026-07-16-user-owned-session-storage-design.md b/docs/superpowers/specs/2026-07-16-user-owned-session-storage-design.md new file mode 100644 index 00000000..29e3afdc --- /dev/null +++ b/docs/superpowers/specs/2026-07-16-user-owned-session-storage-design.md @@ -0,0 +1,25 @@ +# User-owned session storage design + +## Decisioni vincolanti + +- In server mode ThothII usa PostgreSQL diretto, nello schema Supabase privato + `thoth_sessions`; il browser non accede mai al database. +- In local mode ogni utente usa `~/.thothii` (override esplicito `THT_HOME`), + senza fallback o sincronizzazione con Supabase. +- Una sessione appartiene al principal `(issuer, subject)`. Nel portale il + subject è il PK Django; email e username non sono identificatori. +- Il proxy valida la sessione del portale e inietta identità normalizzata; il + backend richiede `AUTH_MODE=upstream` in produzione e non riceve token raw. +- Gli utenti nel gruppo Authentik `authentik Admins` possono gestire ogni + sessione. Un accesso non autorizzato restituisce 404. +- Lo schema contiene principals, principal_preferences, sessions, + session_artifacts, review_decisions e audit_log. Artefatti correnti e + decision ledger append-only; niente cronologia di artefatti né contenuto + nell'audit. +- La sicurezza server combina RLS forzata, un runtime role senza BYPASSRLS, + contesto attore transaction-local e filtri applicativi espliciti. +- Non vengono creati embeddings o vector columns per le sessioni. La memoria + solved_question esistente resta un flusso separato best-effort. +- Sessioni e preferenze server sono persistite solo nel DB; guasti DB sono + fail-closed (503). I file temporanei di export sono effimeri. +- Chat e SSE restano memoria runtime, non artefatti persistiti. diff --git a/docs/superpowers/specs/2026-07-16-user-preference-bootstrap-design.md b/docs/superpowers/specs/2026-07-16-user-preference-bootstrap-design.md new file mode 100644 index 00000000..d5b6baec --- /dev/null +++ b/docs/superpowers/specs/2026-07-16-user-preference-bootstrap-design.md @@ -0,0 +1,34 @@ +# User preference bootstrap design + +## Problem + +The user-owned-session branch reads settings only from the current principal's +repository preferences. Existing deployments still have their provider, model, +thinking level, and workspace only in the legacy backend settings JSON file. +For a principal with no private preference record, a new session is therefore +created without a provider or model and Pi is rejected before it can spawn. + +## Decision + +On the first settings read for a principal whose private preference object is +empty, the backend computes the complete effective legacy settings from +`SETTINGS_FILE` and the configured environment defaults, writes that complete +object to the principal's repository preferences, and returns it. + +After this one-time bootstrap, the private preference object is the sole +source for that principal. A non-empty private object is never replaced with +the legacy values. If either reading or writing the private preferences fails, +the request remains fail-closed with the existing 503 response. + +The addition of optional session storage must not change DWH artifact +ownership. The DWH binding therefore excludes `session_storage` while retaining +every schema, retrieval, vector, and execution setting in its fingerprint. + +## Scope and verification + +The change is confined to the backend settings resolver. Vitest coverage must +prove that an empty private profile is seeded exactly once with the complete +legacy settings, and that an existing private profile is neither changed nor +replaced. Harness coverage must also prove that adding session storage leaves +the DWH binding unchanged. Existing session-route coverage continues to prove +that new-session creation consumes the resolved settings. diff --git a/frontend/src/api/sessions.test.ts b/frontend/src/api/sessions.test.ts index 58664284..c3491feb 100644 --- a/frontend/src/api/sessions.test.ts +++ b/frontend/src/api/sessions.test.ts @@ -1,6 +1,6 @@ import { http, HttpResponse } from "msw"; import { server } from "../test/msw"; -import { createSession, listSessions, prewarmRuntime, resumeSession } from "./sessions"; +import { createSession, getMe, listSessions, prewarmRuntime, resumeSession } from "./sessions"; import { renameSession, setSessionGroup, archiveSession, unarchiveSession, deleteSession, getSessionDocuments, @@ -30,10 +30,34 @@ test.each([202, 204])("prewarmRuntime accepts a body-less %s response", async (s expect(called).toBe(true); }); -test("listSessions GETs the array", async () => { - server.use(http.get("http://localhost:8787/sessions", () => HttpResponse.json([{ id: "s1", status: "open", question: "q", summary: null, created_at: "t", updated_at: null, author: null }]))); +test("listSessions defaults to the current user's scope", async () => { + let scope: string | null = null; + server.use(http.get("http://localhost:8787/sessions", ({ request }) => { + scope = new URL(request.url).searchParams.get("scope"); + return HttpResponse.json([{ id: "s1", status: "open", question: "q", summary: null, created_at: "t", updated_at: null, author: null }]); + })); const rows = await listSessions(); expect(rows[0].id).toBe("s1"); + expect(scope).toBe("mine"); +}); + +test("listSessions requests the selected administrator scope", async () => { + let scope: string | null = null; + server.use(http.get("http://localhost:8787/sessions", ({ request }) => { + scope = new URL(request.url).searchParams.get("scope"); + return HttpResponse.json([]); + })); + await listSessions("all"); + expect(scope).toBe("all"); +}); + +test("getMe fetches the typed authenticated principal", async () => { + server.use(http.get("http://localhost:8787/me", () => + HttpResponse.json({ issuer: "portal", subject: "42", displayName: "Alice", isAdmin: true }), + )); + await expect(getMe()).resolves.toEqual({ + issuer: "portal", subject: "42", displayName: "Alice", isAdmin: true, + }); }); test("resumeSession returns the typed runtime disposition", async () => { diff --git a/frontend/src/api/sessions.ts b/frontend/src/api/sessions.ts index 53e3cff9..eabb2874 100644 --- a/frontend/src/api/sessions.ts +++ b/frontend/src/api/sessions.ts @@ -1,5 +1,7 @@ import { apiFetch } from "./client"; -import type { ResumeSessionResult, SessionSummary, SessionDocument, UiResponse } from "./types"; +import type { + Principal, ResumeSessionResult, SessionScope, SessionSummary, SessionDocument, UiResponse, +} from "./types"; export const createSession = (i: { question: string; name?: string }) => apiFetch<{ id: string }>("/sessions", { method: "POST", body: JSON.stringify(i) }); @@ -8,7 +10,10 @@ export const createSession = (i: { question: string; name?: string }) => export const prewarmRuntime = () => apiFetch("/runtime/prewarm", { method: "POST" }); -export const listSessions = () => apiFetch("/sessions"); +export const getMe = () => apiFetch("/me"); + +export const listSessions = (scope: SessionScope = "mine") => + apiFetch(`/sessions?scope=${scope}`); // eslint-disable-next-line @typescript-eslint/no-explicit-any export const getSession = (id: string) => apiFetch(`/sessions/${id}`); diff --git a/frontend/src/api/types.ts b/frontend/src/api/types.ts index e1d9a6c8..872e7543 100644 --- a/frontend/src/api/types.ts +++ b/frontend/src/api/types.ts @@ -104,6 +104,15 @@ export interface SessionSummary { archived: boolean; } +export type SessionScope = "mine" | "all"; + +export interface Principal { + issuer: string; + subject: string; + displayName?: string; + isAdmin: boolean; +} + export interface ResumeSessionResult { id: string; alreadyActive: boolean; diff --git a/frontend/src/shell/AppShell.session-mgmt.test.tsx b/frontend/src/shell/AppShell.session-mgmt.test.tsx index e841a229..d6130b12 100644 --- a/frontend/src/shell/AppShell.session-mgmt.test.tsx +++ b/frontend/src/shell/AppShell.session-mgmt.test.tsx @@ -54,6 +54,9 @@ beforeEach(() => { window.matchMedia = vi.fn().mockReturnValue({ matches: true, addEventListener: vi.fn(), removeEventListener: vi.fn() }); useSessionStore.getState().resetSession(); server.use( + http.get("http://localhost:8787/me", () => + HttpResponse.json({ issuer: "portal", subject: "alice", displayName: "Alice", isAdmin: false }), + ), http.get("http://localhost:8787/sessions", () => HttpResponse.json(LIST)), http.get("http://localhost:8787/sessions/:id/documents", () => HttpResponse.json([ { phase: "—", key: "question", title: "Domanda originale", format: "text", content: "Attiva uno" }, @@ -62,6 +65,96 @@ beforeEach(() => { ); }); +test("regular users load only their sessions and never see administrator controls", async () => { + let scope: string | null = null; + server.use(http.get("http://localhost:8787/sessions", ({ request }) => { + scope = new URL(request.url).searchParams.get("scope"); + return HttpResponse.json(LIST); + })); + wrap(); + await screen.findByText("Attiva uno"); + expect(scope).toBe("mine"); + expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument(); + expect(screen.queryByText(/administrator view/i)).not.toBeInTheDocument(); +}); + +test("administrators can explicitly switch to all sessions and see owners", async () => { + let scope = ""; + server.use( + http.get("http://localhost:8787/me", () => + HttpResponse.json({ issuer: "portal", subject: "alice-id", displayName: "Alice", isAdmin: true }), + ), + http.get("http://localhost:8787/sessions", ({ request }) => { + scope = new URL(request.url).searchParams.get("scope") ?? ""; + return HttpResponse.json([ + { ...LIST[0], author: "Alice" }, + { ...LIST[1], id: "s3", question: "Another owner's session", archived: false, author: "Bob" }, + ]); + }), + ); + wrap(); + await screen.findByRole("button", { name: "All sessions" }); + expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "true"); + expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "false"); + await userEvent.click(screen.getByRole("button", { name: "All sessions" })); + await waitFor(() => expect(scope).toBe("all")); + expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "false"); + expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "true"); + expect(await screen.findByText("Administrator view: all sessions")).toBeInTheDocument(); + expect(screen.getByText("Owner: Bob")).toBeInTheDocument(); +}); + +test("administrator confirms before deleting a same-named user's session", async () => { + let deletes = 0; + server.use( + http.get("http://localhost:8787/me", () => + HttpResponse.json({ issuer: "portal", subject: "alice-id", displayName: "Alice", isAdmin: true }), + ), + http.get("http://localhost:8787/sessions", () => HttpResponse.json([ + { ...LIST[0], author: "Alice" }, + { ...LIST[1], id: "s3", question: "Second session", archived: false, author: "Bob" }, + ])), + http.delete("http://localhost:8787/sessions/:id", () => { + deletes += 1; + return new HttpResponse(null, { status: 204 }); + }), + ); + wrap(); + await userEvent.click(await screen.findByRole("button", { name: "All sessions" })); + await screen.findByText("Owner: Alice"); + await userEvent.click(screen.getByRole("checkbox", { name: "Select Attiva uno" })); + await userEvent.click(screen.getByRole("button", { name: "Delete 1 selected sessions" })); + expect(deletes).toBe(0); + expect(await screen.findByRole("heading", { name: "Delete permanently" })).toBeInTheDocument(); + await userEvent.click(screen.getByRole("button", { name: "Delete" })); + await waitFor(() => expect(deletes).toBe(1)); +}); + +test("administrator confirms before archiving a same-named user's session", async () => { + let archives = 0; + const confirm = vi.spyOn(window, "confirm").mockReturnValue(false); + server.use( + http.get("http://localhost:8787/me", () => + HttpResponse.json({ issuer: "portal", subject: "alice-id", displayName: "Alice", isAdmin: true }), + ), + http.get("http://localhost:8787/sessions", () => HttpResponse.json([ + { ...LIST[0], author: "Alice" }, + ])), + http.post("http://localhost:8787/sessions/:id/archive", () => { + archives += 1; + return new HttpResponse(null, { status: 204 }); + }), + ); + wrap(); + await userEvent.click(await screen.findByRole("button", { name: "All sessions" })); + await screen.findByText("Owner: Alice"); + await userEvent.click(screen.getByRole("button", { name: "Session actions" })); + await userEvent.click(await screen.findByText("Archive")); + expect(confirm).toHaveBeenCalledWith("Archive Alice's session?"); + expect(archives).toBe(0); + confirm.mockRestore(); +}); + test("active list shows group header and hides archived sessions", async () => { wrap(); expect(await screen.findByText("Attiva uno")).toBeInTheDocument(); diff --git a/frontend/src/shell/AppShell.tsx b/frontend/src/shell/AppShell.tsx index a3cc3941..784102d3 100644 --- a/frontend/src/shell/AppShell.tsx +++ b/frontend/src/shell/AppShell.tsx @@ -18,10 +18,10 @@ import { Checkbox } from "../components/ui/checkbox"; import { Toaster } from "../components/ui/sonner"; import { toast } from "sonner"; import { - closeSession, listSessions, resumeSession, getSession, + closeSession, getMe, listSessions, resumeSession, getSession, renameSession, setSessionGroup, archiveSession, unarchiveSession, deleteSession, prewarmRuntime, } from "../api/sessions"; -import type { SessionSummary } from "../api/types"; +import type { Principal, SessionScope, SessionSummary } from "../api/types"; import { useQuery, useQueryClient } from "@tanstack/react-query"; import { useEffect, useMemo, useRef, useState } from "react"; import type { CSSProperties } from "react"; @@ -48,8 +48,10 @@ export function AppShell() { const [streamCursorResetEpoch, setStreamCursorResetEpoch] = useState(0); const [creatingSession, setCreatingSession] = useState(false); const [awaitingQuestion, setAwaitingQuestion] = useState(false); + const [sessionScope, setSessionScope] = useState("mine"); + const { data: principal } = useQuery({ queryKey: ["me"], queryFn: getMe, staleTime: Infinity }); const { data: sessions = [] } = useQuery({ - queryKey: ["sessions"], queryFn: listSessions, refetchInterval: 10_000, + queryKey: ["sessions", sessionScope], queryFn: () => listSessions(sessionScope), refetchInterval: 10_000, }); const composerRef = useRef(null); @@ -77,6 +79,12 @@ export function AppShell() { const finalized = activeSession?.status === "finalized"; const selectedSessions = sessions.filter((session) => selectedSessionIds.has(session.id)); const allSessionsSelected = sessions.length > 0 && selectedSessions.length === sessions.length; + const showingAllSessions = sessionScope === "all"; + const isForeignSession = (session: SessionSummary) => { + if (!showingAllSessions || !principal) return false; + if (!session.author) return true; + return session.author !== principal.subject; + }; function selectActiveSession(id: string | null) { // Keep async Resume completions synchronized before React commits the state update. @@ -245,6 +253,14 @@ export function AppShell() { } } + function requestArchiveToggle(session: SessionSummary) { + if (!session.archived && isForeignSession(session)) { + const label = session.author ? `${session.author}'s session` : "this session"; + if (!window.confirm(`Archive ${label}?`)) return; + } + void toggleArchive(session); + } + async function deleteSessions(targets: SessionSummary[]) { try { const results = await Promise.allSettled(targets.map((session) => deleteSession(session.id))); @@ -266,6 +282,14 @@ export function AppShell() { } } + function requestDelete(targets: SessionSummary[]) { + if (targets.some(isForeignSession) || allSessionsSelected) { + setDeleteTargets(targets); + return; + } + void deleteSessions(targets); + } + function menuFor(s: SessionSummary) { return ( setRenameTarget(s)} onMove={(g) => move(s, g)} onNewGroup={() => newGroup(s)} - onArchiveToggle={() => toggleArchive(s)} - onDelete={() => { void deleteSessions([s]); }} + onArchiveToggle={() => requestArchiveToggle(s)} + onDelete={() => requestDelete([s])} /> ); } @@ -496,6 +520,34 @@ export function AppShell() { + {principal?.isAdmin && ( +
+
+ + +
+ {showingAllSessions && ( +

+ Administrator view: all sessions +

+ )} +
+ )} + {/* L1 — rail title */}
@@ -519,8 +571,7 @@ export function AppShell() { size="xs" aria-label={`Delete ${selectedSessions.length} selected sessions`} onClick={() => { - if (allSessionsSelected) setDeleteTargets(selectedSessions); - else void deleteSessions(selectedSessions); + requestDelete(selectedSessions); }} > @@ -571,6 +622,7 @@ export function AppShell() { menuFor={menuFor} selectedIds={selectedSessionIds} onSelectionChange={setSessionSelected} + showOwner={showingAllSessions} /> )}
@@ -586,6 +638,7 @@ export function AppShell() { menuFor={menuFor} selectedIds={selectedSessionIds} onSelectionChange={setSessionSelected} + showOwner={showingAllSessions} /> )} @@ -609,6 +662,7 @@ export function AppShell() { menuFor={menuFor} selectedIds={selectedSessionIds} onSelectionChange={setSessionSelected} + showOwner={showingAllSessions} /> )} diff --git a/frontend/src/shell/NavSessions.test.tsx b/frontend/src/shell/NavSessions.test.tsx index 48042570..9a633182 100644 --- a/frontend/src/shell/NavSessions.test.tsx +++ b/frontend/src/shell/NavSessions.test.tsx @@ -26,6 +26,19 @@ test("active session is highlighted", () => { expect(screen.getByTestId("session-item-s2")).toHaveAttribute("data-active", "true"); }); +test("administrator session lists show the recorded owner", () => { + render( + null} + showOwner + />, + ); + expect(screen.getByText("Owner: Alice")).toBeInTheDocument(); +}); + test("pressing Enter on a focused row opens the panel", async () => { const onOpenPanel = vi.fn(); render( null} />); diff --git a/frontend/src/shell/NavSessions.tsx b/frontend/src/shell/NavSessions.tsx index 00fb4a04..927d371d 100644 --- a/frontend/src/shell/NavSessions.tsx +++ b/frontend/src/shell/NavSessions.tsx @@ -10,6 +10,7 @@ interface Props { menuFor: (session: SessionSummary) => ReactNode; selectedIds?: ReadonlySet; onSelectionChange?: (id: string, selected: boolean) => void; + showOwner?: boolean; } function statusIndicator(status: string) { @@ -21,7 +22,7 @@ function statusIndicator(status: string) { export function NavSessions({ sessions, activeSessionId, onOpenPanel, menuFor, - selectedIds = new Set(), onSelectionChange = () => undefined, + selectedIds = new Set(), onSelectionChange = () => undefined, showOwner = false, }: Props) { if (sessions.length === 0) { return ( @@ -70,13 +71,20 @@ export function NavSessions({ title={indicator.label} className={["size-1.5 shrink-0 rounded-full", indicator.color].join(" ")} /> - - {label} + + + {label} + + {showOwner && ( + + Owner: {s.author ?? "Unknown"} + + )} {menuFor(s)} diff --git a/harness/.gitignore b/harness/.gitignore index af38b5f0..cdd01399 100644 --- a/harness/.gitignore +++ b/harness/.gitignore @@ -3,6 +3,9 @@ __pycache__/ .env .venv/ sessions/ +# Keep the versioned database migration package; only runtime session directories are ignored. +!tht/migrations/sessions/ +!tht/migrations/sessions/*.sql indexes/ artifacts/ # Per-customer active workspace (points at the customer repo via -c or symlink). diff --git a/harness/.pi/extensions/gate/__tests__/session-repository-writes.test.js b/harness/.pi/extensions/gate/__tests__/session-repository-writes.test.js new file mode 100644 index 00000000..58a6cff2 --- /dev/null +++ b/harness/.pi/extensions/gate/__tests__/session-repository-writes.test.js @@ -0,0 +1,41 @@ +const test = require("node:test"); +const assert = require("node:assert"); +const cp = require("node:child_process"); +const { createRequire } = require("node:module"); +const path = require("node:path"); + +const GATE = path.join(__dirname, "..", "..", "tht-gate.js"); +if (typeof globalThis.require === "undefined") globalThis.require = createRequire(GATE); + +test("repository write tools persist CTE and final SQL through deterministic tht commands", async () => { + const calls = []; + const original = cp.execFileSync; + cp.execFileSync = (file, args, options) => { + calls.push({ args, input: options?.input }); + return ""; + }; + try { + const gate = require(GATE); + const { createFakePi } = require("./fake_pi_runtime.js"); + const { pi, ctx, tools } = createFakePi(); + ctx.cwd = "/nonexistent-thothii-test-cwd"; + gate.default(pi); + + const cte = tools.get("write_cte_sql"); + const final = tools.get("write_final_sql"); + assert.ok(cte, "write_cte_sql must be registered"); + assert.ok(final, "write_final_sql must be registered"); + + await cte.def.execute("cte", { session: "s1", name: "base", sql: "WITH base AS (SELECT 1)" }, null, null, ctx); + await final.def.execute("sql", { session: "s1", sql: "SELECT * FROM base" }, null, null, ctx); + + assert.deepEqual(calls.map((call) => call.args), [ + ["cte", "save", "--session", "s1", "--name", "base", "--file", "-"], + ["sql", "set-final", "--session", "s1", "--file", "-"], + ]); + assert.deepEqual(calls.map((call) => call.input), ["WITH base AS (SELECT 1)", "SELECT * FROM base"]); + assert.equal(JSON.stringify(calls).includes("sessions/s1"), false); + } finally { + cp.execFileSync = original; + } +}); diff --git a/harness/.pi/extensions/tht-gate.js b/harness/.pi/extensions/tht-gate.js index eac2c836..357af984 100644 --- a/harness/.pi/extensions/tht-gate.js +++ b/harness/.pi/extensions/tht-gate.js @@ -1401,6 +1401,38 @@ export default function (pi) { }, }); + pi.registerTool({ + name: "write_cte_sql", + label: "Scrittura CTE SQL", + description: "Persiste un blocco CTE tramite tht cte save; non scrivere mai file di sessione direttamente.", + parameters: Type.Object({ session: Type.String(), name: Type.String(), sql: Type.String() }), + async execute(_id, params, _signal, _onUpdate, ctx) { + const err = relayIfThtFails( + ctx, + ["cte", "save", "--session", params.session, "--name", params.name, "--file", "-"], + "Correggi il blocco CTE e riprova.", + params.sql, + ); + return err || textResult(`CTE ${params.name} salvato per la sessione ${params.session}.`); + }, + }); + + pi.registerTool({ + name: "write_final_sql", + label: "Scrittura SQL finale", + description: "Persiste il SQL finale tramite tht sql set-final; non scrivere mai file di sessione direttamente.", + parameters: Type.Object({ session: Type.String(), sql: Type.String() }), + async execute(_id, params, _signal, _onUpdate, ctx) { + const err = relayIfThtFails( + ctx, + ["sql", "set-final", "--session", params.session, "--file", "-"], + "Correggi il SQL finale e riprova.", + params.sql, + ); + return err || textResult(`SQL finale salvato per la sessione ${params.session}.`); + }, + }); + // --- slash command: /torna [session_id] [N] (rollback to a previous phase) -- pi.registerCommand("torna", { description: diff --git a/harness/.pi/skills/tht-sessione/SKILL.md b/harness/.pi/skills/tht-sessione/SKILL.md index 9a59551e..d32079ad 100644 --- a/harness/.pi/skills/tht-sessione/SKILL.md +++ b/harness/.pi/skills/tht-sessione/SKILL.md @@ -353,7 +353,9 @@ Prerequisite: Phase 5 closed. "output_columns":["cod_paz","data_ricovero"]} ]} ``` -3. For each CTE (in plan order): write `sessions//ctes/.sql` (ONLY the +3. For each CTE (in plan order): call `write_cte_sql` with the session id, CTE name, + and SQL block (the tool invokes `tht cte save --session --name --file -`). + Persist ONLY the `WITH ... AS (...)` block, NO trailing SELECT), test with `tht cte test --session ` (with an **ok** outcome), then present it with `reviewer_confirm kind:"cte_result"`. Pass ONLY the thin v2 data @@ -385,7 +387,8 @@ Prerequisite: Phase 6 closed. columns (`dt.year`, `dt.month`, …), NEVER arithmetic on the key. 3. `tht sql validate` + `tht sql preview` (max 10 rows). On errors / suspicious results, apply the `sql-generation.md` checklist and correct with the reviewer. -4. `tht sql save` writes `sessions//sql_final.sql` (ONLY clean SQL, no comments). +4. Call `write_final_sql` with the session id and clean SQL; it invokes + `tht sql set-final --session --file -` (ONLY clean SQL, no comments). Approve with `reviewer_confirm kind:"sql"` (records `sql_approved`), then advance to Phase 8 with `reviewer_confirm kind:"phase"` — `kind:"sql"` alone does NOT advance F7. diff --git a/harness/pyproject.toml b/harness/pyproject.toml index 5972c288..2ca5241e 100644 --- a/harness/pyproject.toml +++ b/harness/pyproject.toml @@ -16,6 +16,7 @@ dependencies = [ "requests>=2.31", "tqdm>=4.66", "yake>=0.4", + "portalocker>=2.10", ] [project.scripts] @@ -33,7 +34,7 @@ dev = [ include = ["tht*"] [tool.setuptools.package-data] -tht = ["migrations/vector/*.sql"] +tht = ["migrations/vector/*.sql", "migrations/sessions/*.sql"] [tool.ruff] line-length = 100 diff --git a/harness/tests/test_adapter_command_regressions.py b/harness/tests/test_adapter_command_regressions.py index 28ee362c..ebadf06a 100644 --- a/harness/tests/test_adapter_command_regressions.py +++ b/harness/tests/test_adapter_command_regressions.py @@ -67,17 +67,17 @@ def test_memory_command_writes_through_factory_vector_store(monkeypatch): # configure the workstation-only REST writer key. cfg = SimpleNamespace(profile="server", embeddings=object(), vector_write_rest=None) manifest = SimpleNamespace(id="s1") + snapshot = SimpleNamespace(manifest=manifest, decisions=[], artifacts={}) record = MemoryRecord(id="m1", ts=datetime(2026, 1, 1), session_id="s1", decision_seq=7, type="table_promoted", subject="t", question_context="q") monkeypatch.setattr(memory_cmd, "_load_config_or_exit", lambda path: cfg) - monkeypatch.setattr(memory_cmd, "load_session_or_exit", lambda cfg, session: manifest) - monkeypatch.setattr(memory_cmd, "session_dir", lambda *args: None) + monkeypatch.setattr(memory_cmd, "load_snapshot_or_exit", lambda cfg, session: snapshot) monkeypatch.setattr(memory_cmd, "registry_path", lambda cfg: None) monkeypatch.setattr("tht.adapters.factory.build_vector_store", lambda cfg, require_write: store) monkeypatch.setattr("tht.cli.vector_cmd.make_embedder", lambda cfg: SimpleNamespace(embed_documents=lambda texts: [[0.1]])) - monkeypatch.setattr("tht.memory.promote", lambda *args, **kwargs: None) + monkeypatch.setattr("tht.memory.promote_snapshot", lambda *args, **kwargs: None) monkeypatch.setattr("tht.memory.load_registry", lambda path: [record]) memory_cmd.save_one_cmd(session="s1", decision=7, json_out=True) from tht.ports.vector import VectorWriteRecord @@ -96,14 +96,13 @@ def test_solved_index_writes_through_writer_only_factory_store(monkeypatch): calls = [] monkeypatch.setattr(memory_cmd, "has_vector_write_rest", lambda cfg: True) - monkeypatch.setattr(memory_cmd, "load_session_or_exit", lambda cfg, session: manifest) - monkeypatch.setattr(memory_cmd, "session_dir", lambda *args: None) + monkeypatch.setattr(memory_cmd, "load_snapshot_or_exit", lambda cfg, session: SimpleNamespace(manifest=manifest, decisions=[], artifacts={})) monkeypatch.setattr( "tht.adapters.factory.build_vector_store", lambda cfg, require_write: calls.append(require_write) or writer_only_store, ) monkeypatch.setattr("tht.cli.sql_cmd.promoted_tables_for", lambda *args: []) - monkeypatch.setattr("tht.solved.build_solved_record", lambda *args: solved_record) + monkeypatch.setattr("tht.solved.build_solved_snapshot", lambda *args: solved_record) monkeypatch.setattr( "tht.solved.save_solved_question", lambda record, *, store, embedder: int( diff --git a/harness/tests/test_decision_join_set_cli.py b/harness/tests/test_decision_join_set_cli.py index 3865197a..f14807d7 100644 --- a/harness/tests/test_decision_join_set_cli.py +++ b/harness/tests/test_decision_join_set_cli.py @@ -28,15 +28,28 @@ def _walk_to_phase(session, target): def _configure_command(monkeypatch, sessions): import tht.cli.decision_cmd as mod + import tht.cli.session_cmd as session_mod + from tht.session.models import SessionManifest, SessionSnapshot + + class _Repository: + def get(self, session_id): + return SessionSnapshot( + manifest=SessionManifest(id=session_id, created_at="2026-01-01T00:00:00Z", question="q", database="d", schema="s"), + decisions=list_decisions(sessions / session_id), + ) + + def append_decisions(self, session_id, decisions): + return append_decisions(sessions / session_id, list(decisions)) class _Cfg: - class paths: - pass + pass - _Cfg.paths.sessions = sessions + repository = _Repository() monkeypatch.setattr(mod, "_load_config_or_exit", lambda _c: _Cfg()) monkeypatch.setattr(mod, "load_session_or_exit", lambda _cfg, _s: None) - monkeypatch.setattr(mod, "session_dir", lambda _cfg, sid: sessions / sid) + monkeypatch.setattr(mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid)) + monkeypatch.setattr(mod, "session_repository", lambda _cfg: repository) + monkeypatch.setattr(session_mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid)) def test_add_join_set_rejects_the_whole_batch_when_one_item_is_invalid(tmp_path, monkeypatch): diff --git a/harness/tests/test_decision_retract_cli.py b/harness/tests/test_decision_retract_cli.py index 4991d2aa..c77ee42b 100644 --- a/harness/tests/test_decision_retract_cli.py +++ b/harness/tests/test_decision_retract_cli.py @@ -25,6 +25,8 @@ def test_retract_drops_last_substantive_decision(tmp_path, monkeypatch): # stub config + session loading (the command only needs a session dir) import tht.cli.decision_cmd as mod + import tht.cli.session_cmd as session_mod + from tht.session.models import SessionManifest, SessionSnapshot class _Cfg: class paths: @@ -32,7 +34,22 @@ def test_retract_drops_last_substantive_decision(tmp_path, monkeypatch): monkeypatch.setattr(mod, "_load_config_or_exit", lambda _c: _Cfg()) monkeypatch.setattr(mod, "load_session_or_exit", lambda _cfg, _s: None) - monkeypatch.setattr(mod, "session_dir", lambda _cfg, sid: tmp_path / sid) + class _Repository: + def get(self, session_id): + return SessionSnapshot( + manifest=SessionManifest(id=session_id, created_at="2026-01-01T00:00:00Z", question="q", database="d", schema="s"), + decisions=list_decisions(tmp_path / session_id), + ) + + def append_decisions(self, session_id, decisions): + from tht.decisions import append_decisions + + return append_decisions(tmp_path / session_id, list(decisions)) + + repository = _Repository() + monkeypatch.setattr(mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid)) + monkeypatch.setattr(mod, "session_repository", lambda _cfg: repository) + monkeypatch.setattr(session_mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid)) retract_cmd(session="2026-01-01-000000-x", config=Path("x")) diff --git a/harness/tests/test_dwh_preprocess_job.py b/harness/tests/test_dwh_preprocess_job.py index cdc0dbf1..b4119a44 100644 --- a/harness/tests/test_dwh_preprocess_job.py +++ b/harness/tests/test_dwh_preprocess_job.py @@ -31,6 +31,24 @@ def test_dwh_and_evidence_jobs_have_distinct_lock_names(): assert _lock_name("demo", "dwh") != _lock_name("demo", "evidence") +def test_session_storage_does_not_change_the_dwh_artifact_binding(tmp_path): + from types import SimpleNamespace + + def config(session_storage=None): + payload = {"dwh": {"connection": {"database": "warehouse"}}} + if session_storage is not None: + payload["session_storage"] = session_storage + cfg = SimpleNamespace(_workspace_id="demo", _config_source="test") + cfg.model_dump = lambda **_kwargs: payload + cfg.model_dump_json = lambda: json.dumps(payload, separators=(",", ":")) + return cfg + + without_session_storage = config() + with_session_storage = config({"type": "postgres_direct", "connection": {"database": "sessions"}}) + + assert config_dwh_binding(without_session_storage) == config_dwh_binding(with_session_storage) + + def test_unowned_reads_fail_closed_without_creating_any_files(tmp_path): import pytest diff --git a/harness/tests/test_local_identity.py b/harness/tests/test_local_identity.py new file mode 100644 index 00000000..2ae0ae5b --- /dev/null +++ b/harness/tests/test_local_identity.py @@ -0,0 +1,17 @@ +import json +import uuid + +from tht.session.models import local_principal + + +def test_local_identity_uuid_is_stable_across_restarts(tmp_path): + first = local_principal(tmp_path) + restarted = local_principal(tmp_path) + + assert first == restarted + assert first.issuer == "local" + assert uuid.UUID(first.subject, version=4).version == 4 + assert json.loads((tmp_path / "identity.json").read_text()) == { + "issuer": "local", + "subject": first.subject, + } diff --git a/harness/tests/test_postgres_session_repository.py b/harness/tests/test_postgres_session_repository.py new file mode 100644 index 00000000..b46b4a70 --- /dev/null +++ b/harness/tests/test_postgres_session_repository.py @@ -0,0 +1,265 @@ +import uuid +from concurrent.futures import ThreadPoolExecutor +from datetime import UTC, datetime +from threading import Barrier + +import pytest +from sqlalchemy import create_engine, text +from testcontainers.postgres import PostgresContainer + +from tht.decisions import DecisionInput +from tht.phase import effective_decisions +from tht.session.models import PrincipalContext, SessionManifest +from tht.session.store import SessionError + + +@pytest.fixture(scope="module") +def database_url(): + with PostgresContainer("postgres:16-alpine") as postgres: + yield postgres.get_connection_url() + + +@pytest.fixture(scope="module", autouse=True) +def migrated(database_url): + from tht.session.postgres_repository import migrate + + migrate(database_url) + + +def _manifest(session_id: str) -> SessionManifest: + return SessionManifest( + id=session_id, + created_at=datetime(2026, 7, 16, 10, 0, tzinfo=UTC), + question="Which patients had an ablation?", + database="testdb", + schema="public", + ) + + +def _repository(database_url, subject: str, *, is_admin: bool = False): + from tht.session.postgres_repository import PostgresSessionRepository + + return PostgresSessionRepository( + database_url, + PrincipalContext(issuer="portal", subject=subject, is_admin=is_admin), + ) + + +def test_owner_can_read_own_snapshot_but_not_another_owners(database_url): + alice = _repository(database_url, "alice") + bob = _repository(database_url, "bob") + session_id = str(uuid.uuid4()) + + alice.create(_manifest(session_id)) + alice.write_artifact(session_id, "cte_sql:eligible_patients", "SELECT 1") + alice.append_decisions( + session_id, [DecisionInput(type="sql_approved", subject="phase:7", detail="SELECT 1")] + ) + + snapshot = alice.get(session_id) + assert snapshot.principal == alice.principal + assert snapshot.artifacts == {"cte_sql:eligible_patients": "SELECT 1"} + assert snapshot.decisions[0].seq == 1 + with pytest.raises(SessionError, match="Sessione non trovata"): + bob.get(session_id) + + +def test_named_decision_is_stale_after_postgres_ledger_reopen(database_url): + repository = _repository(database_url, "phase-owner") + session_id = str(uuid.uuid4()) + repository.create(_manifest(session_id)) + for phase in range(1, 5): + repository.append_decisions(session_id, [ + DecisionInput(type="phase_approved", subject=f"phase:{phase}") + ]) + repository.append_decisions(session_id, [ + DecisionInput(type="cte_approved", subject="named_cte") + ]) + repository.append_decisions(session_id, [ + DecisionInput(type="phase_reopened", subject="phase:3") + ]) + + snapshot = repository.get(session_id) + named = next(item for item in snapshot.decisions if item.subject == "named_cte") + assert named.phase == 5 + assert "named_cte" not in {item.subject for item in effective_decisions(snapshot)} + + +def test_admin_can_read_another_owners_session(database_url): + owner = _repository(database_url, "owner") + admin = _repository(database_url, "admin", is_admin=True) + session_id = str(uuid.uuid4()) + owner.create(_manifest(session_id)) + + assert admin.get(session_id).manifest.id == session_id + + +def test_owner_admin_and_foreign_principals_have_distinct_mutation_boundaries(database_url): + session_id = str(uuid.uuid4()) + alice = _repository(database_url, f"alice-{session_id}") + bob = _repository(database_url, f"bob-{session_id}") + admin = _repository(database_url, f"admin-{session_id}", is_admin=True) + manifest = _manifest(session_id) + alice.create(manifest) + alice.write_artifact(session_id, "question", "Alice's question") + + assert [snapshot.manifest.id for snapshot in alice.list()] == [session_id] + assert bob.list() == [] + for mutation in ( + lambda: bob.write_artifact(session_id, "question", "Bob's overwrite"), + lambda: bob.append_decisions( + session_id, [DecisionInput(type="concept_clarified", subject="bob")] + ), + lambda: bob.save_manifest(manifest.model_copy(update={"name": "Bob's rename"})), + lambda: bob.delete(session_id), + ): + with pytest.raises(SessionError, match="Sessione non trovata"): + mutation() + + # Admin access is an explicit, audited exception to normal owner isolation. + admin.write_artifact(session_id, "question", "Reviewed by admin") + admin.append_decisions( + session_id, [DecisionInput(type="concept_clarified", subject="admin-review")] + ) + + owner_snapshot = alice.get(session_id) + assert owner_snapshot.artifacts["question"] == "Reviewed by admin" + assert [record.subject for record in owner_snapshot.decisions] == ["admin-review"] + assert admin.get(session_id).manifest.id == session_id + + +def test_concurrent_postgres_ledger_mutations_keep_every_decision_in_sequence(database_url, monkeypatch): + from tht.session.postgres_repository import PostgresSessionRepository + + repository = _repository(database_url, "alice") + session_id = str(uuid.uuid4()) + repository.create(_manifest(session_id)) + barrier = Barrier(2) + original_lock = PostgresSessionRepository._lock_session + + def enter_lock_together(connection, locked_session_id): + if locked_session_id == session_id: + barrier.wait(timeout=5) + original_lock(connection, locked_session_id) + + monkeypatch.setattr( + PostgresSessionRepository, + "_lock_session", + staticmethod(enter_lock_together), + ) + + def append(subject: str): + return repository.append_decisions( + session_id, [DecisionInput(type="concept_clarified", subject=subject)] + ) + + with ThreadPoolExecutor(max_workers=2) as pool: + first, second = pool.map(append, ("first", "second")) + + snapshot = repository.get(session_id) + assert {first[0].seq, second[0].seq} == {1, 2} + assert [record.seq for record in snapshot.decisions] == [1, 2] + assert {record.subject for record in snapshot.decisions} == {"first", "second"} + + +def test_non_superuser_runtime_login_can_assume_the_restricted_runtime_role(database_url): + admin = create_engine(database_url) + runtime_url = admin.url.set( + username="thoth_sessions_test_login", password="runtime-test-only" + ) + try: + with admin.begin() as connection: + connection.exec_driver_sql( + "CREATE ROLE thoth_sessions_test_login LOGIN NOINHERIT PASSWORD 'runtime-test-only'" + ) + connection.exec_driver_sql("GRANT thoth_sessions_runtime TO thoth_sessions_test_login") + repository = _repository( + runtime_url.render_as_string(hide_password=False), "runtime-user" + ) + session_id = str(uuid.uuid4()) + + repository.create(_manifest(session_id)) + + assert repository.get(session_id).manifest.id == session_id + repository.close() + finally: + with admin.begin() as connection: + connection.exec_driver_sql("DROP ROLE IF EXISTS thoth_sessions_test_login") + admin.dispose() + + +def test_delete_cascades_content_and_leaves_content_free_tombstone(database_url): + repository = _repository(database_url, "alice") + session_id = str(uuid.uuid4()) + repository.create(_manifest(session_id)) + repository.write_artifact(session_id, "sql_final", "SELECT confidential_value") + repository.append_decisions( + session_id, + [DecisionInput(type="sql_approved", subject="phase:7", detail="SELECT confidential_value")], + ) + + repository.delete(session_id) + + with pytest.raises(SessionError, match="Sessione non trovata"): + repository.get(session_id) + engine = create_engine(database_url) + try: + with engine.connect() as connection: + assert connection.execute( + text("SELECT count(*) FROM thoth_sessions.session_artifacts WHERE session_id = :session_id"), + {"session_id": session_id}, + ).scalar_one() == 0 + assert connection.execute( + text("SELECT count(*) FROM thoth_sessions.review_decisions WHERE session_id = :session_id"), + {"session_id": session_id}, + ).scalar_one() == 0 + columns = connection.execute( + text( + "SELECT column_name FROM information_schema.columns " + "WHERE table_schema = 'thoth_sessions' AND table_name = 'audit_log'" + ) + ).scalars().all() + tombstone = connection.execute( + text( + "SELECT action, session_id, actor_issuer, actor_subject " + "FROM thoth_sessions.audit_log WHERE session_id = :session_id" + ), + {"session_id": session_id}, + ).one() + finally: + engine.dispose() + assert (tombstone[0], str(tombstone[1]), *tombstone[2:]) == ( + "session_deleted", + session_id, + "portal", + "alice", + ) + assert not {"content", "artifact_content", "detail", "metadata"} & set(columns) + + +def test_session_schema_does_not_create_or_invoke_embeddings(database_url, monkeypatch): + import tht.session.postgres_repository as repository_module + + monkeypatch.setattr( + repository_module, + "_embed", + lambda *_: pytest.fail("session persistence must not invoke embeddings"), + raising=False, + ) + repository = _repository(database_url, "alice") + session_id = str(uuid.uuid4()) + repository.create(_manifest(session_id)) + repository.write_artifact(session_id, "evidence", '{"sources": []}') + + engine = create_engine(database_url) + try: + with engine.connect() as connection: + columns = connection.execute( + text( + "SELECT column_name FROM information_schema.columns " + "WHERE table_schema = 'thoth_sessions'" + ) + ).scalars().all() + finally: + engine.dispose() + assert all("embedding" not in column for column in columns) diff --git a/harness/tests/test_promoted_columns_for.py b/harness/tests/test_promoted_columns_for.py index 3aa40f54..57ff20f7 100644 --- a/harness/tests/test_promoted_columns_for.py +++ b/harness/tests/test_promoted_columns_for.py @@ -3,9 +3,13 @@ from tht.cli.sql_cmd import promoted_columns_for def test_promoted_columns_for(tmp_path): - sid = "sess1" + sid = "2026-07-16-120000" sdir = tmp_path / sid sdir.mkdir(parents=True) + (sdir / "session_manifest.yaml").write_text( + f"id: {sid}\nquestion: q\ndatabase: d\nschema: s\n" + "created_at: 2026-01-01T00:00:00+00:00\nstatus: open\n" + ) (sdir / "schema_linking.json").write_text(json.dumps({ "question": "q", "candidates": [ diff --git a/harness/tests/test_repository_memory_sql_paths.py b/harness/tests/test_repository_memory_sql_paths.py new file mode 100644 index 00000000..31a6edfb --- /dev/null +++ b/harness/tests/test_repository_memory_sql_paths.py @@ -0,0 +1,16 @@ +"""Repository-only workflow commands must not recover server state via session_dir.""" + +import inspect + +from tht.cli import memory_cmd, sql_cmd + + +def test_memory_workflow_commands_do_not_import_or_call_session_dir(): + source = inspect.getsource(memory_cmd) + assert "session_dir" not in source + + +def test_sql_session_commands_read_snapshot_artifacts_not_session_paths(): + source = inspect.getsource(sql_cmd) + assert "_session_sql_file" not in source + assert "session_dir" not in source diff --git a/harness/tests/test_require_phase_or_exit.py b/harness/tests/test_require_phase_or_exit.py index 75f5c8b2..57ff6221 100644 --- a/harness/tests/test_require_phase_or_exit.py +++ b/harness/tests/test_require_phase_or_exit.py @@ -16,8 +16,13 @@ def _make_session(tmp_path, current_phase_num: int) -> str: approving phases 1..N-1 puts the session at phase N.""" import json - s = tmp_path / "sess" + session_id = "2026-07-16-120000" + s = tmp_path / session_id s.mkdir() + (s / "session_manifest.yaml").write_text( + f"id: {session_id}\nquestion: q\ndatabase: d\nschema: s\n" + "created_at: 2026-01-01T00:00:00+00:00\nstatus: open\n" + ) decisions = [ {"seq": n, "type": "phase_approved", "subject": f"phase:{n}", "ts": "2025-01-01T00:00:00"} for n in range(1, current_phase_num) @@ -25,7 +30,7 @@ def _make_session(tmp_path, current_phase_num: int) -> str: (s / "review_decisions.jsonl").write_text( "\n".join(json.dumps(d) for d in decisions) + ("\n" if decisions else "") ) - return "sess" + return session_id class _StubConfig: diff --git a/harness/tests/test_session_migrate_cmd.py b/harness/tests/test_session_migrate_cmd.py new file mode 100644 index 00000000..66cb8172 --- /dev/null +++ b/harness/tests/test_session_migrate_cmd.py @@ -0,0 +1,47 @@ +import json + +from testcontainers.postgres import PostgresContainer +from typer.testing import CliRunner + +from tht.cli import app + + +def test_session_migrate_status_is_pristine_and_idempotent(): + with PostgresContainer("postgres:16-alpine") as postgres: + database_url = postgres.get_connection_url() + runner = CliRunner() + + before = runner.invoke( + app, ["session", "migrate", "--database-url", database_url, "--status", "--json"] + ) + assert before.exit_code == 0, before.output + assert json.loads(before.stdout) == {"applied": [], "drifted": [], "pending": ["001", "002"]} + assert before.stderr == "" + + first = runner.invoke(app, ["session", "migrate", "--database-url", database_url, "--json"]) + second = runner.invoke(app, ["session", "migrate", "--database-url", database_url, "--json"]) + + expected = {"applied": ["001", "002"], "drifted": [], "pending": []} + assert first.exit_code == 0, first.output + assert second.exit_code == 0, second.output + assert json.loads(first.stdout) == expected + assert json.loads(second.stdout) == expected + + +def test_session_migrate_status_database_failure_is_pristine_json(): + result = CliRunner().invoke( + app, + [ + "session", + "migrate", + "--database-url", + "postgresql+psycopg2://test:test@127.0.0.1:1/test", + "--status", + "--json", + ], + ) + + assert result.exit_code == 1 + assert result.stderr == "" + assert "Traceback" not in result.stdout + assert json.loads(result.stdout)["error"] diff --git a/harness/tests/test_session_repository.py b/harness/tests/test_session_repository.py new file mode 100644 index 00000000..0ca5b343 --- /dev/null +++ b/harness/tests/test_session_repository.py @@ -0,0 +1,173 @@ +import uuid + +import pytest + +from tht.config import DatabaseConfig, load_config +from tht.decisions import DecisionInput +from tht.session.filesystem_repository import FilesystemSessionRepository +from tht.session.models import PrincipalContext, SessionManifest, local_principal +from tht.session.repository import build_session_repository, resolve_principal +from tht.session.store import SessionError +from tht.session.store import create_session + + +def _db() -> DatabaseConfig: + return DatabaseConfig(database="testdb", schema="public", user="u", password="p") # noqa: S106 + + +def _config(tmp_path): + path = tmp_path / "workspace.yaml" + path.write_text( + "dwh:\n" + " type: postgres_direct\n" + " connection:\n" + " database: testdb\n" + " schema: public\n" + " user: user\n" + " password: secret\n" + ) + return load_config(path) + + +def _manifest(session_id: str) -> SessionManifest: + return SessionManifest( + id=session_id, + created_at="2026-07-16T10:00:00Z", + question="Which patients had an ablation?", + database="testdb", + schema="public", + ) + + +def test_new_sessions_use_uuid4_ids(tmp_path): + manifest = create_session("Which patients had an ablation?", _db(), tmp_path) + + session_uuid = uuid.UUID(manifest.id, version=4) + assert str(session_uuid) == manifest.id + assert session_uuid.version == 4 + + +def test_filesystem_repository_scopes_sessions_to_local_principal_root(tmp_path): + config = _config(tmp_path) + principal = PrincipalContext(issuer="local", subject="alice") + repository = build_session_repository(config, principal, home=tmp_path / "alice-home") + session_id = str(uuid.uuid4()) + + repository.create(_manifest(session_id)) + + assert repository.root == tmp_path / "alice-home/workspaces/workspace/sessions" + assert (repository.root / session_id / "session_manifest.yaml").exists() + + +def test_filesystem_repository_reads_phase_artifacts_and_appends_decisions(tmp_path): + repository = FilesystemSessionRepository( + tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice") + ) + session_id = str(uuid.uuid4()) + repository.create(_manifest(session_id)) + + repository.write_artifact(session_id, "sql_final", "SELECT 1") + repository.write_artifact(session_id, "evidence", '{"sources": []}') + decisions = repository.append_decisions( + session_id, + [DecisionInput(type="sql_approved", subject="phase:7", detail="SELECT 1")], + ) + snapshot = repository.get(session_id) + + assert repository.read_artifact(session_id, "sql_final") == "SELECT 1" + assert repository.read_artifact(session_id, "evidence") == '{"sources": []}' + assert snapshot.artifacts["sql_final"] == "SELECT 1" + assert snapshot.artifacts["evidence"] == '{"sources": []}' + assert [decision.type for decision in snapshot.decisions] == ["sql_approved"] + assert decisions[0].seq == 1 + + +def test_tht_home_overrides_local_repository_root(monkeypatch, tmp_path): + monkeypatch.setenv("THT_HOME", str(tmp_path / "override")) + config = _config(tmp_path) + + repository = build_session_repository( + config, PrincipalContext(issuer="local", subject="alice") + ) + + assert config.paths.sessions == tmp_path / "override/workspaces/workspace/sessions" + assert repository.root == tmp_path / "override/workspaces/workspace/sessions" + + +def test_filesystem_repository_keeps_preferences_per_principal(tmp_path): + home = tmp_path / "home" + alice = FilesystemSessionRepository(home, "demo", PrincipalContext(issuer="local", subject="alice")) + bob = FilesystemSessionRepository(home, "demo", PrincipalContext(issuer="local", subject="bob")) + + alice.set_preferences({"model": "glm"}) + + assert alice.get_preferences() == {"model": "glm"} + assert bob.get_preferences() == {} + + +def test_two_local_homes_have_independent_identities_sessions_and_preferences(tmp_path): + alice_home = tmp_path / "alice-home" + bob_home = tmp_path / "bob-home" + alice = FilesystemSessionRepository(alice_home, "demo", local_principal(alice_home)) + bob = FilesystemSessionRepository(bob_home, "demo", local_principal(bob_home)) + session_id = str(uuid.uuid4()) + + alice.create(_manifest(session_id)) + alice.set_preferences({"model": "glm"}) + + assert alice.principal.subject != bob.principal.subject + assert (alice.root / session_id / "session_manifest.yaml").exists() + assert not (bob.root / session_id).exists() + assert bob.list() == [] + assert bob.get_preferences() == {} + with pytest.raises(SessionError, match="Sessione non trovata"): + bob.get(session_id) + + +@pytest.mark.parametrize("session_id", ["2026-01-01-000000-test", "2026-01-01-000000-x", "s1"]) +def test_filesystem_repository_reads_safe_legacy_session_ids(tmp_path, session_id): + repository = FilesystemSessionRepository( + tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice") + ) + directory = repository.root / session_id + directory.mkdir(parents=True) + _manifest(session_id).to_yaml(directory / "session_manifest.yaml") + + assert repository.get(session_id).manifest.id == session_id + + +@pytest.mark.parametrize("session_id", ["..", "a/b", "/absolute", "space id"]) +def test_filesystem_repository_rejects_unsafe_legacy_session_ids(tmp_path, session_id): + repository = FilesystemSessionRepository( + tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice") + ) + + with pytest.raises(SessionError): + repository.get(session_id) + + +def test_postgres_session_storage_requires_trusted_principal_environment(tmp_path, monkeypatch): + config = _config(tmp_path).model_copy(update={"session_storage": { + "type": "postgres_direct", + "connection": _db().model_dump(by_alias=True), + }}) + monkeypatch.delenv("THT_PRINCIPAL_ISSUER", raising=False) + monkeypatch.delenv("THT_PRINCIPAL_SUBJECT", raising=False) + + with pytest.raises(SessionError, match="THT_PRINCIPAL_ISSUER"): + resolve_principal(config) + + +def test_postgres_session_storage_uses_only_explicit_trusted_principal(tmp_path, monkeypatch): + config = _config(tmp_path).model_copy(update={"session_storage": { + "type": "postgres_direct", + "connection": _db().model_dump(by_alias=True), + }}) + monkeypatch.setenv("THT_PRINCIPAL_ISSUER", "portal") + monkeypatch.setenv("THT_PRINCIPAL_SUBJECT", "alice") + monkeypatch.setenv("THT_PRINCIPAL_DISPLAY_NAME", "Alice") + monkeypatch.setenv("THT_PRINCIPAL_IS_ADMIN", "TRUE") + + assert resolve_principal(config) == PrincipalContext( + issuer="portal", subject="alice", display_name="Alice", is_admin=True + ) diff --git a/harness/tests/test_session_repository_workflow.py b/harness/tests/test_session_repository_workflow.py new file mode 100644 index 00000000..8608ce25 --- /dev/null +++ b/harness/tests/test_session_repository_workflow.py @@ -0,0 +1,60 @@ +import uuid + +from tht.decisions import DecisionInput +from tht.phase import current_phase, cte_plan, next_cte +from tht.session.filesystem_repository import FilesystemSessionRepository +from tht.session.models import PrincipalContext, SessionManifest +from tht.session.store import persist_verified_finalization + + +def _manifest(session_id: str) -> SessionManifest: + return SessionManifest( + id=session_id, + created_at="2026-07-16T10:00:00Z", + question="Which patients had an ablation?", + database="testdb", + schema="public", + ) + + +def test_phase_helpers_fold_the_repository_snapshot_not_a_session_path(tmp_path): + repository = FilesystemSessionRepository( + tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice") + ) + session_id = str(uuid.uuid4()) + repository.create(_manifest(session_id)) + repository.write_artifact(session_id, "cte_plan", '["base_patients"]') + repository.append_decisions( + session_id, + [ + DecisionInput(type="phase_approved", subject="phase:1"), + DecisionInput(type="phase_auto_approved", subject="phase:2"), + DecisionInput(type="cte_approved", subject="base_patients"), + ], + ) + + snapshot = repository.get(session_id) + + assert current_phase(snapshot) == 3 + assert cte_plan(snapshot) == ["base_patients"] + assert next_cte(snapshot) is None + + +def test_verified_finalization_commits_report_evidence_and_status_through_repository(tmp_path): + repository = FilesystemSessionRepository( + tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice") + ) + session_id = str(uuid.uuid4()) + repository.create(_manifest(session_id)) + + persist_verified_finalization( + repository, + session_id, + validation_report="# Validation\n\nverified against DWH\n", + evidence='[{"source":"review"}]\n', + ) + + snapshot = repository.get(session_id) + assert snapshot.manifest.status == "finalized" + assert snapshot.artifacts["validation_report"] == "# Validation\n\nverified against DWH\n" + assert snapshot.artifacts["evidence"] == '[{"source":"review"}]\n' diff --git a/harness/tests/test_sql_preview_json.py b/harness/tests/test_sql_preview_json.py index a19b34c7..a1d36997 100644 --- a/harness/tests/test_sql_preview_json.py +++ b/harness/tests/test_sql_preview_json.py @@ -108,16 +108,12 @@ def test_do_run_offset_zero_path_unchanged(monkeypatch): def test_preview_session_no_file_resolves_sql_final(monkeypatch, tmp_path, capsys): - """--session without a positional FILE resolves sql_final.sql via _session_sql_file.""" + """--session without a positional FILE resolves repository SQL text.""" from types import SimpleNamespace from tht.cli import sql_cmd - sql_file = tmp_path / "sql_final.sql" - sql_file.write_text("SELECT session_resolved") - - # Patch _session_sql_file to return our tmp file (no real workspace/DB needed). - monkeypatch.setattr(sql_cmd, "_session_sql_file", lambda cfg, sid: sql_file) + monkeypatch.setattr(sql_cmd, "_session_sql", lambda cfg, sid: "SELECT session_resolved") captured_sql = {} diff --git a/harness/tests/test_vector_migration_packaging.py b/harness/tests/test_vector_migration_packaging.py index 24544c97..03addb39 100644 --- a/harness/tests/test_vector_migration_packaging.py +++ b/harness/tests/test_vector_migration_packaging.py @@ -6,7 +6,7 @@ import zipfile from pathlib import Path -def test_built_wheel_installs_vector_migrations_and_discovers_cli(tmp_path): +def test_built_wheel_installs_migrations_and_discovers_cli(tmp_path): harness = Path(__file__).parents[1] wheelhouse = tmp_path / "wheelhouse" target = tmp_path / "site" @@ -33,6 +33,8 @@ def test_built_wheel_installs_vector_migrations_and_discovers_cli(tmp_path): names = set(archive.namelist()) assert "tht/migrations/vector/001_extensions.sql" in names assert "tht/migrations/vector/003_roles.sql" in names + assert "tht/migrations/sessions/001_schema.sql" in names + assert "tht/migrations/sessions/002_security.sql" in names subprocess.run( [sys.executable, "-m", "pip", "install", "--no-deps", "--target", str(target), wheel], @@ -47,6 +49,8 @@ def test_built_wheel_installs_vector_migrations_and_discovers_cli(tmp_path): "-c", "from typer.testing import CliRunner; from tht.cli import app; " "r=CliRunner().invoke(app, ['vector','migrate','--help']); " + "assert r.exit_code == 0, r.output; " + "r=CliRunner().invoke(app, ['session','migrate','--help']); " "print(r.output); raise SystemExit(r.exit_code)", ], env=env, diff --git a/harness/tht/cli/cte_cmd.py b/harness/tht/cli/cte_cmd.py index 94cd8ca2..d7b39a58 100644 --- a/harness/tht/cli/cte_cmd.py +++ b/harness/tht/cli/cte_cmd.py @@ -6,7 +6,7 @@ import typer from tht.cli.config_cmd import CONFIG_OPT from tht.cli.schema_cmd import _load_config_or_exit -from tht.cli.session_cmd import load_session_or_exit, session_dir +from tht.cli.session_cmd import load_session_or_exit, load_snapshot_or_exit, session_repository from tht.cli.sql_cmd import promoted_tables_for, require_action _RULE6_HINT = ( @@ -17,6 +17,27 @@ _RULE6_HINT = ( cte_app = typer.Typer(help="Test controllato dei CTE proposti (Agent View Generation)") +@cte_app.command("save") +def save_cmd( + session: str = typer.Option(..., "--session"), + name: str = typer.Option(..., "--name"), + file: str = typer.Option(..., "--file", help="File SQL, oppure '-' per stdin."), + config: Path = CONFIG_OPT, +) -> None: + """Persist one CTE SQL block through the configured session repository.""" + import sys + + cfg = _load_config_or_exit(config) + load_session_or_exit(cfg, session) + raw = sys.stdin.read() if file == "-" else Path(file).read_text() + try: + session_repository(cfg).write_artifact(session, f"cte_sql:{name}", raw) + except ValueError as exc: + typer.secho(f"ERRORE: {exc}", fg=typer.colors.RED, err=True) + raise typer.Exit(code=1) from None + typer.secho(f"OK: CTE {name} salvato.", fg=typer.colors.GREEN) + + @cte_app.command("test") def test_cmd( name: str = typer.Argument(..., help="Nome del CTE (file sessions//ctes/.sql)."), @@ -33,7 +54,7 @@ def test_cmd( CteError, CteTestRecord, _jsonable, - append_cte_test, + append_cte_test_snapshot, build_test_sql, has_trailing_select, ) @@ -43,13 +64,13 @@ def test_cmd( cfg = _load_config_or_exit(config) require_action(cfg, "cte_test") load_session_or_exit(cfg, session) - sdir = session_dir(cfg, session) + snapshot = load_snapshot_or_exit(cfg, session) from tht.cli.phase_cmd import require_phase_or_exit from tht.phase import next_cte require_phase_or_exit(cfg, session, 6) - nxt = next_cte(sdir) + nxt = next_cte(snapshot) if nxt is not None and name != nxt: typer.secho( f"ERRORE: ordine CTE. Ora tocca a '{nxt}' (il primo CTE del piano non " @@ -60,11 +81,10 @@ def test_cmd( ) raise typer.Exit(code=5) - cte_file = sdir / "ctes" / f"{name}.sql" - if not cte_file.exists(): - typer.secho(f"ERRORE: file CTE non trovato: {cte_file}", fg=typer.colors.RED, err=True) + cte_sql = snapshot.artifacts.get(f"cte_sql:{name}") + if cte_sql is None: + typer.secho(f"ERRORE: file CTE non trovato: {name}", fg=typer.colors.RED, err=True) raise typer.Exit(code=1) - cte_sql = cte_file.read_text() sql_hash = hashlib.sha256(cte_sql.encode()).hexdigest() def _record_error(message: str) -> None: @@ -72,7 +92,7 @@ def test_cmd( name=name, ts=datetime.now(UTC), sql_hash=sql_hash, status="error", error=message, ) - append_cte_test(sdir, record) + append_cte_test_snapshot(session_repository(cfg), snapshot, record) if json_out: typer.echo(record.model_dump_json()) else: @@ -118,7 +138,7 @@ def test_cmd( execution_ms=result.execution_ms, warnings=warnings, preview_rows=[[_jsonable(cell) for cell in row] for row in result.rows], ) - append_cte_test(sdir, record) + append_cte_test_snapshot(session_repository(cfg), snapshot, record) if json_out: typer.echo(record.model_dump_json()) @@ -133,7 +153,7 @@ def test_cmd( Console().print(table) for w in warnings: typer.secho(f" warning: {w}", fg=typer.colors.YELLOW) - typer.secho(f"OK: esito registrato in {sdir / 'cte_tests.json'}", fg=typer.colors.GREEN) + typer.secho("OK: esito registrato in cte_tests.json", fg=typer.colors.GREEN) CTE_PLAN_DOC_FILE = "cte_plan_doc.json" @@ -156,13 +176,10 @@ def plan_cmd( import json import sys - from tht.phase import CTE_PLAN_FILE - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session) from tht.cli.phase_cmd import require_phase_or_exit require_phase_or_exit(cfg, session, 6) - sdir = session_dir(cfg, session) doc_data = None if doc is not None: @@ -180,11 +197,11 @@ def plan_cmd( ) raise typer.Exit(code=1) - sdir.mkdir(parents=True, exist_ok=True) - (sdir / CTE_PLAN_FILE).write_text(json.dumps(name, ensure_ascii=False)) + repository = session_repository(cfg) + repository.write_artifact(session, "cte_plan", json.dumps(name, ensure_ascii=False)) if doc_data is not None: - (sdir / CTE_PLAN_DOC_FILE).write_text(json.dumps(doc_data, ensure_ascii=False)) - typer.secho(f"OK: piano CTE salvato ({len(name)} CTE) in {sdir / CTE_PLAN_FILE}.", + repository.write_artifact(session, "cte_plan_doc", json.dumps(doc_data, ensure_ascii=False)) + typer.secho(f"OK: piano CTE salvato ({len(name)} CTE).", fg=typer.colors.GREEN) @@ -201,7 +218,7 @@ def next_cmd( cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session) - nxt = next_cte(session_dir(cfg, session)) + nxt = next_cte(load_snapshot_or_exit(cfg, session)) if nxt: typer.echo(nxt) @@ -222,9 +239,9 @@ def info_cmd( cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session) - sdir = session_dir(cfg, session) + snapshot = load_snapshot_or_exit(cfg, session) - plan = cte_plan(sdir) + plan = cte_plan(snapshot) if not plan: typer.secho(f"ERRORE: {CTE_PLAN_FILE} assente o vuoto per la sessione '{session}'.", fg=typer.colors.RED, err=True) @@ -234,24 +251,24 @@ def info_cmd( fg=typer.colors.RED, err=True) raise typer.Exit(code=1) - cte_file = sdir / "ctes" / f"{name}.sql" - if not cte_file.exists(): - typer.secho(f"ERRORE: file CTE non trovato: {cte_file}", fg=typer.colors.RED, err=True) + cte_sql = snapshot.artifacts.get(f"cte_sql:{name}") + if cte_sql is None: + typer.secho(f"ERRORE: file CTE non trovato: {name}", fg=typer.colors.RED, err=True) raise typer.Exit(code=1) doc = None - doc_path = sdir / CTE_PLAN_DOC_FILE - if doc_path.exists(): - full_doc = _json.loads(doc_path.read_text()) + raw_doc = snapshot.artifacts.get("cte_plan_doc") + if raw_doc: + full_doc = _json.loads(raw_doc) for c in full_doc.get("ctes", []): if c.get("name") == name: doc = {k: v for k, v in c.items() if k != "name"} break - from tht.ctetest import CteError, load_cte_tests + from tht.ctetest import CteError, load_cte_tests_text try: - records = [r for r in load_cte_tests(sdir) if r.name == name] + records = [r for r in load_cte_tests_text(snapshot.artifacts.get("cte_tests", "")) if r.name == name] except CteError as e: typer.secho(f"ERRORE: impossibile leggere cte_tests.json: {e}", fg=typer.colors.RED, err=True) @@ -263,8 +280,8 @@ def info_cmd( "index": plan.index(name) + 1, "total": len(plan), "plan": plan, - "sql": cte_file.read_text(), - "approved": name in approved_ctes(sdir), + "sql": cte_sql, + "approved": name in approved_ctes(snapshot), "doc": doc, "last_test": last_test, } @@ -282,12 +299,12 @@ def list_cmd( config: Path = CONFIG_OPT, ) -> None: """Ultimo esito registrato per ogni CTE della sessione.""" - from tht.ctetest import CteError, load_cte_tests + from tht.ctetest import CteError, load_cte_tests_text cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session) try: - records = load_cte_tests(session_dir(cfg, session)) + records = load_cte_tests_text(load_snapshot_or_exit(cfg, session).artifacts.get("cte_tests", "")) except CteError as e: typer.secho(f"ERRORE: impossibile leggere cte_tests.json: {e}", fg=typer.colors.RED, err=True) diff --git a/harness/tht/cli/decision_cmd.py b/harness/tht/cli/decision_cmd.py index 0f13d9bf..0c695a87 100644 --- a/harness/tht/cli/decision_cmd.py +++ b/harness/tht/cli/decision_cmd.py @@ -8,7 +8,7 @@ from pydantic import ValidationError from tht.cli.config_cmd import CONFIG_OPT from tht.cli.schema_cmd import _load_config_or_exit -from tht.cli.session_cmd import load_session_or_exit, session_dir +from tht.cli.session_cmd import load_session_or_exit, load_snapshot_or_exit, session_repository decision_app = typer.Typer(help="Decisioni del reviewer (per sessione, append-only)") @@ -20,7 +20,7 @@ def add_join_set_cmd( config: Path = CONFIG_OPT, ) -> None: """Registra un insieme completo di join con un'unica sostituzione atomica del ledger.""" - from tht.decisions import DecisionInput, append_decisions + from tht.decisions import DecisionInput cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session) @@ -40,7 +40,7 @@ def add_join_set_cmd( from tht.workflow import load_workflow require_phase_or_exit(cfg, session, load_workflow().decision_min_phase("join_modified")) - records = append_decisions(session_dir(cfg, session), decisions) + records = session_repository(cfg).append_decisions(session, decisions) typer.secho( f"OK: registrato set atomico di {len(records)} join.", fg=typer.colors.GREEN, @@ -61,7 +61,7 @@ def add_cmd( config: Path = CONFIG_OPT, ) -> None: """Registra una decisione del reviewer nella sessione.""" - from tht.decisions import DecisionType, append_decision + from tht.decisions import DecisionType cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session) @@ -82,7 +82,7 @@ def add_cmd( from tht.workflow import load_workflow require_phase_or_exit(cfg, session, load_workflow().decision_min_phase(type)) - sdir = session_dir(cfg, session) + snapshot = load_snapshot_or_exit(cfg, session) if type == "cte_approved": # Un CTE si approva solo se appartiene al piano persistito. Senza piano # (o con subject fuori piano) l'approvazione e' priva di significato: @@ -90,7 +90,7 @@ def add_cmd( # dove fu registrato un cte_approved:cte_plan senza alcun cte_plan.json. from tht.phase import cte_plan - plan = cte_plan(sdir) + plan = cte_plan(snapshot) if not plan: typer.secho( "ERRORE: nessun piano CTE (cte_plan.json) in sessione. Persisti prima " @@ -105,10 +105,10 @@ def add_cmd( fg=typer.colors.RED, err=True, ) raise typer.Exit(code=5) - record = append_decision( - sdir, type=type, subject=subject, - detail=detail, rationale=rationale, retracts=retracts, - ) + record = session_repository(cfg).append_decisions(session, [{ + "type": type, "subject": subject, "detail": detail, + "rationale": rationale, "retracts": retracts, + }])[0] typer.secho(f"OK: decisione [{record.seq}] {record.type}: {record.subject}", fg=typer.colors.GREEN) @@ -123,19 +123,18 @@ def retract_cmd( Granularita' (a) del rollback §4.8: 'rispondi di nuovo a questa domanda'. Scrive un marker decision_retracted (append-only, l'audit resta) che effective_decisions onora; il widget corrente puo' essere riproposto. Non cambia la fase.""" - from tht.decisions import append_decision from tht.phase import effective_decisions cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session) - sdir = session_dir(cfg, session) + snapshot = load_snapshot_or_exit(cfg, session) # Ultima decisione NON-meta della vista effective = quella associata al widget corrente. meta = { "phase_approved", "phase_auto_approved", "phase_reopened", "phase_skipped", "decision_retracted", } - substantive = [d for d in effective_decisions(sdir) if d.type not in meta] + substantive = [d for d in effective_decisions(snapshot) if d.type not in meta] if not substantive: typer.secho( "Nessuna decisione sostanziale da ritirare nella fase corrente.", @@ -143,10 +142,10 @@ def retract_cmd( ) raise typer.Exit(code=6) target = substantive[-1] - record = append_decision( - sdir, type="decision_retracted", subject=target.subject, - rationale=f"ritira [{target.seq}] {target.type}", retracts=target.seq, - ) + record = session_repository(cfg).append_decisions(session, [{ + "type": "decision_retracted", "subject": target.subject, + "rationale": f"ritira [{target.seq}] {target.type}", "retracts": target.seq, + }])[0] typer.secho( f"OK: ritirata decisione [{target.seq}] {target.type}: {target.subject} " f"(marker #{record.seq}).", @@ -160,11 +159,8 @@ def list_cmd( config: Path = CONFIG_OPT, ) -> None: """Elenca le decisioni della sessione.""" - from tht.decisions import list_decisions - cfg = _load_config_or_exit(config) - load_session_or_exit(cfg, session) - decisions = list_decisions(session_dir(cfg, session)) + decisions = load_snapshot_or_exit(cfg, session).decisions if not decisions: typer.echo("Nessuna decisione registrata.") return diff --git a/harness/tht/cli/memory_cmd.py b/harness/tht/cli/memory_cmd.py index 9b9df4ce..ea0d1fe9 100644 --- a/harness/tht/cli/memory_cmd.py +++ b/harness/tht/cli/memory_cmd.py @@ -17,7 +17,7 @@ from tht.cli._guards import ( require_server_profile, require_vector_write_allowed, ) -from tht.cli.session_cmd import load_session_or_exit, session_dir +from tht.cli.session_cmd import load_snapshot_or_exit from tht.cli.vector_cmd import require_vector_cfg memory_app = typer.Typer(help="Review memory (registro canonico + indice pgvector)") @@ -48,18 +48,17 @@ def promote_cmd( """Promuove le decisioni SCELTE nel registro globale. Usa --preview per vedere i candidati.""" import json as _json - from tht.memory import promote + from tht.memory import promote_snapshot cfg = _load_config_or_exit(config) - manifest = load_session_or_exit(cfg, session) + snapshot = load_snapshot_or_exit(cfg, session) if preview: from tht.memory import ( - MAX_PROMOTION_CANDIDATES, preview_promotions, reusable_promotions, + MAX_PROMOTION_CANDIDATES, preview_promotions_snapshot, reusable_promotions_snapshot, ) - sdir = session_dir(cfg, session) - cand = preview_promotions(sdir, manifest, registry_path(cfg)) - extra = len(reusable_promotions(sdir, manifest, registry_path(cfg))) - len(cand) + cand = preview_promotions_snapshot(snapshot, registry_path(cfg)) + extra = len(reusable_promotions_snapshot(snapshot, registry_path(cfg))) - len(cand) payload = [ {"decision_seq": c.decision_seq, "type": c.type, "subject": c.subject, "detail": c.detail, "rationale": c.rationale, @@ -89,10 +88,7 @@ def promote_cmd( require_server_profile(cfg, "memory promote") require_vector_cfg(cfg) - promoted = promote( - session_dir(cfg, session), manifest, - seqs=list(decision), registry_path=registry_path(cfg), - ) + promoted = promote_snapshot(snapshot, seqs=list(decision), registry_path=registry_path(cfg)) if not promoted: msg = "Nessuna nuova promozione (gia' presenti o seq inesistenti)." if json_out: @@ -155,18 +151,17 @@ def save_one_cmd( from tht.adapters.factory import build_vector_store from tht.cli.vector_cmd import make_embedder - from tht.memory import load_registry, promote, save_one_memory + from tht.memory import load_registry, promote_snapshot, save_one_memory cfg = _load_config_or_exit(config) - manifest = load_session_or_exit(cfg, session) + snapshot = load_snapshot_or_exit(cfg, session) require_vector_write_allowed(cfg, "memory save-one") store = build_vector_store(cfg, require_write=True) - sdir = session_dir(cfg, session) # Promuove la decisione scelta nel registro locale (idempotente: salta se gia' presente # o se stale post-rollback, perche' _compute_promotions usa la vista effective). - promote(sdir, manifest, seqs=[decision], registry_path=registry_path(cfg)) - records = [r for r in load_registry(registry_path(cfg)) if r.session_id == manifest.id] + promote_snapshot(snapshot, seqs=[decision], registry_path=registry_path(cfg)) + records = [r for r in load_registry(registry_path(cfg)) if r.session_id == snapshot.manifest.id] embedder = make_embedder(cfg.embeddings) count = save_one_memory(records, decision, store=store, embedder=embedder) @@ -388,16 +383,14 @@ def search_cmd( from rich.console import Console from rich.table import Table - from tht.cli.session_cmd import session_dir from tht.cli.vector_cmd import make_embedder, open_searcher, require_vector_cfg from tht.memory import decided_memory_ids, load_registry - from tht.decisions import list_decisions cfg = _load_config_or_exit(config) require_vector_cfg(cfg) excluded: set[str] = set() if session is not None: - excluded = decided_memory_ids(list_decisions(session_dir(cfg, session))) + excluded = decided_memory_ids(load_snapshot_or_exit(cfg, session).decisions) searcher = open_searcher(cfg) embedder = make_embedder(cfg.embeddings) hits = searcher.search(embedder.embed_query(question), top_n=top, kinds=["memory"]) @@ -445,7 +438,7 @@ def index_solved_session(cfg, session_id: str) -> int: from tht.adapters.factory import build_vector_store from tht.cli.sql_cmd import promoted_tables_for from tht.cli.vector_cmd import make_embedder - from tht.solved import build_solved_record, save_solved_question + from tht.solved import build_solved_snapshot, save_solved_question if not has_vector_write_rest(cfg): raise RuntimeError( @@ -453,10 +446,7 @@ def index_solved_session(cfg, session_id: str) -> int: "writer key configurata nel workspace yaml" ) store = build_vector_store(cfg, require_write=True) - manifest = load_session_or_exit(cfg, session_id) - record = build_solved_record( - session_dir(cfg, session_id), manifest, promoted_tables_for(cfg, session_id) - ) + record = build_solved_snapshot(load_snapshot_or_exit(cfg, session_id), promoted_tables_for(cfg, session_id)) return save_solved_question( record, store=store, diff --git a/harness/tht/cli/phase_cmd.py b/harness/tht/cli/phase_cmd.py index 85b5051a..eb70ff23 100644 --- a/harness/tht/cli/phase_cmd.py +++ b/harness/tht/cli/phase_cmd.py @@ -8,7 +8,6 @@ commands. All read workflow facts from load_workflow() (no mirrored constants). from __future__ import annotations import json -from pathlib import Path import typer @@ -22,16 +21,12 @@ from tht.workflow import load_workflow phase_app = typer.Typer(help="Fase del workflow HITL (gate di avanzamento/ritorno)") -def session_dir(cfg, session_id: str) -> Path: - """Where a session's artifacts live. Mirror of session_cmd.session_dir (kept - here to avoid a circular import: session_cmd imports phase helpers too).""" - return cfg.paths.sessions / session_id - - def require_phase_or_exit(cfg, session: str, min_phase: int) -> None: """Refuse with exit 1 if the session hasn't reached min_phase yet. The phase name in the message comes from workflow.yaml (load_workflow), not a constant.""" - cur = current_phase(session_dir(cfg, session)) + from tht.cli.session_cmd import load_snapshot_or_exit + + cur = current_phase(load_snapshot_or_exit(cfg, session)) if cur < min_phase: wf = load_workflow() nome = wf.phase_name(min_phase) @@ -89,21 +84,24 @@ def advance_cmd( ), ) -> None: """Approva la fase corrente e passa alla successiva (persiste phase_approved).""" - from tht.decisions import append_decision + from tht.cli.session_cmd import load_snapshot_or_exit, session_repository - sdir = session_dir(_cfg(), session) - cur = current_phase(sdir) + cfg = _cfg() + snapshot = load_snapshot_or_exit(cfg, session) + cur = current_phase(snapshot) wf = load_workflow() if cur > wf.max_phase: typer.secho("Sessione già alla fase terminale.", fg=typer.colors.YELLOW) raise typer.Exit(0) if auto: - if not auto_advance_eligible(sdir): - problems = advance_problems(sdir, cur) + if not auto_advance_eligible(snapshot): + problems = advance_problems(snapshot, cur) for p in problems: typer.echo(p) raise typer.Exit(6) # needs human confirmation (gate contract) - append_decision(sdir, type="phase_approved", subject=f"phase:{cur}") + session_repository(cfg).append_decisions( + session, [{"type": "phase_approved", "subject": f"phase:{cur}"}] + ) typer.echo(f"Fase {cur} ({wf.phase_name(cur)}) approvata → Fase {cur + 1}.") @@ -113,16 +111,21 @@ def reopen_cmd( phase: int = typer.Option(..., "--phase", help="Fase a cui tornare (1..fase corrente -1)."), ) -> None: """Torna a una fase precedente (persiste phase_reopened + teardown artefatti).""" - from tht.decisions import append_decision - from tht.teardown import teardown_to_phase + from tht.cli.session_cmd import load_snapshot_or_exit, session_repository - sdir = session_dir(_cfg(), session) - cur = current_phase(sdir) + cfg = _cfg() + snapshot = load_snapshot_or_exit(cfg, session) + cur = current_phase(snapshot) if phase < 1 or phase >= cur: typer.secho(f"Target non valido (fase corrente {cur}).", fg=typer.colors.RED, err=True) raise typer.Exit(1) - report = teardown_to_phase(sdir, target_phase=phase) - append_decision(sdir, type="phase_reopened", subject=f"phase:{phase}") + from tht.teardown import teardown_snapshot + + repository = session_repository(cfg) + report = teardown_snapshot(repository, snapshot, phase) + repository.append_decisions( + session, [{"type": "phase_reopened", "subject": f"phase:{phase}"}] + ) for f in report.deleted_files: typer.echo(f" eliminato artefatto: {f}") typer.echo(f"Tornati alla Fase {phase} ({load_workflow().phase_name(phase)}).") @@ -133,13 +136,13 @@ def show_cmd( session: str = typer.Option(..., "--session"), ) -> None: """Mostra stato, fase corrente e ultime decisioni della sessione.""" - from tht.decisions import list_decisions + from tht.cli.session_cmd import load_snapshot_or_exit - sdir = session_dir(_cfg(), session) - cur = current_phase(sdir) + snapshot = load_snapshot_or_exit(_cfg(), session) + cur = current_phase(snapshot) wf = load_workflow() typer.echo(f"Fase corrente: {cur}/{wf.max_phase} ({wf.phase_name(min(cur, wf.max_phase))})") - decisions = list_decisions(sdir) + decisions = snapshot.decisions if decisions: typer.echo(f"Decisioni registrate: {len(decisions)}") for d in decisions[-5:]: diff --git a/harness/tht/cli/search_cmd.py b/harness/tht/cli/search_cmd.py index dcf91f32..b9bef1d5 100644 --- a/harness/tht/cli/search_cmd.py +++ b/harness/tht/cli/search_cmd.py @@ -371,14 +371,13 @@ def pack_cmd( md = "\n".join(md_lines) + "\n" if session: - from tht.cli.session_cmd import load_session_or_exit, session_dir + from tht.cli.session_cmd import load_session_or_exit, session_repository load_session_or_exit(cfg, session) - out = session_dir(cfg, session) / "retrieval_pack.md" - out.write_text(md) + session_repository(cfg).write_artifact(session, "retrieval_pack", md) if not json_out: typer.secho( - f"OK: retrieval pack scritto in {out} " + "OK: retrieval pack scritto " f"({len(tables)} tabelle, {len(evidence)} evidence, {len(solved)} solved).", fg=typer.colors.GREEN, ) diff --git a/harness/tht/cli/session_cmd.py b/harness/tht/cli/session_cmd.py index e582e552..79e9f9e4 100644 --- a/harness/tht/cli/session_cmd.py +++ b/harness/tht/cli/session_cmd.py @@ -2,22 +2,117 @@ import json from pathlib import Path import typer +from sqlalchemy.exc import SQLAlchemyError from tht.cli.config_cmd import CONFIG_OPT from tht.cli.schema_cmd import _load_config_or_exit session_app = typer.Typer(help="Sessioni (directory artefatti)") +preferences_app = typer.Typer(help="Preferenze private del principal corrente") +session_app.add_typer(preferences_app, name="preferences") + + +def session_repository(cfg): + """Configured persistence boundary for every workflow command.""" + from tht.session.repository import build_session_repository + + return build_session_repository(cfg) + + +@preferences_app.command("get") +def preferences_get_cmd( + json_out: bool = typer.Option(False, "--json", help="Emetti JSON puro."), + config: Path = CONFIG_OPT, +) -> None: + """Legge le preferenze private del principal risolto dal repository.""" + cfg = _load_config_or_exit(config) + preferences = session_repository(cfg).get_preferences() + if json_out: + typer.echo(json.dumps(preferences, ensure_ascii=False, sort_keys=True)) + else: + typer.echo(json.dumps(preferences, ensure_ascii=False, indent=2, sort_keys=True)) + + +@preferences_app.command("set") +def preferences_set_cmd( + preferences_json: str = typer.Argument(..., help="Oggetto JSON delle preferenze."), + json_out: bool = typer.Option(False, "--json", help="Non emette testo al successo."), + config: Path = CONFIG_OPT, +) -> None: + """Sostituisce le preferenze private del principal risolto dal repository.""" + try: + preferences = json.loads(preferences_json) + except json.JSONDecodeError as exc: + typer.secho(f"ERRORE: preferenze JSON non valide: {exc}", fg=typer.colors.RED, err=True) + raise typer.Exit(code=2) from None + if not isinstance(preferences, dict): + typer.secho("ERRORE: le preferenze devono essere un oggetto JSON", fg=typer.colors.RED, err=True) + raise typer.Exit(code=2) + cfg = _load_config_or_exit(config) + session_repository(cfg).set_preferences(preferences) + if json_out: + return + typer.secho("OK: preferenze aggiornate.", fg=typer.colors.GREEN) + def session_dir(cfg, session_id: str) -> Path: + """Legacy path bridge for out-of-scope datamart/memory compatibility only. + + Workflow session commands in this module use ``session_repository``; this + helper remains until those non-workflow consumers lose their path APIs. + """ return cfg.paths.sessions / session_id -def load_session_or_exit(cfg, session_id: str): - from tht.session.store import SessionError, load_session +@session_app.command("migrate") +def migrate_cmd( + database_url: str = typer.Option( + ..., "--database-url", envvar="THT_SESSIONS_ADMIN_URL", help="Admin PostgreSQL URL." + ), + status_only: bool = typer.Option(False, "--status", help="Inspect without applying."), + json_output: bool = typer.Option(False, "--json", help="Emit pristine JSON."), +) -> None: + """Apply or inspect the private PostgreSQL session schema migrations.""" + from tht.session.postgres_repository import MigrationError, migrate, migration_status try: - return load_session(session_id, cfg.paths.sessions) + status = migration_status(database_url) if status_only else migrate(database_url) + except (MigrationError, SQLAlchemyError, ValueError) as exc: + if json_output: + typer.echo(json.dumps({"error": str(exc)}, sort_keys=True)) + else: + typer.echo(f"ERROR: {exc}", err=True) + raise typer.Exit(code=1) from None + payload = { + "applied": [item.version for item in status.applied], + "drifted": [item.version for item in status.drifted], + "pending": [item.version for item in status.pending], + } + if json_output: + typer.echo(json.dumps(payload, sort_keys=True)) + else: + typer.echo( + f"Applied: {len(status.applied)}; pending: {len(status.pending)}; " + f"drifted: {len(status.drifted)}" + ) + + +def load_session_or_exit(cfg, session_id: str): + from tht.session.store import SessionError + + try: + return session_repository(cfg).get(session_id).manifest + except SessionError as e: + typer.secho(f"ERRORE: {e}", fg=typer.colors.RED, err=True) + raise typer.Exit(code=1) + + +def load_snapshot_or_exit(cfg, session_id: str): + from tht.session.store import SessionError + + try: + return session_repository(cfg).get(session_id) except SessionError as e: typer.secho(f"ERRORE: {e}", fg=typer.colors.RED, err=True) raise typer.Exit(code=1) @@ -59,7 +154,14 @@ def list_cmd( ) -> None: """Elenca le sessioni esistenti.""" cfg = _load_config_or_exit(config) - rows = _list_sessions(cfg.paths.sessions) + rows = [ + {"id": s.manifest.id, "status": s.manifest.status, "question": s.manifest.question, + "summary": s.manifest.summary, "created_at": s.manifest.created_at.isoformat(), + "updated_at": s.manifest.updated_at.isoformat() if s.manifest.updated_at else None, + "author": s.manifest.author, "name": s.manifest.name, "group": s.manifest.group, + "archived": s.manifest.archived} + for s in session_repository(cfg).list() + ] if json_out: typer.echo(json.dumps(rows, ensure_ascii=False, indent=2)) return @@ -78,16 +180,19 @@ def new_cmd( config: Path = CONFIG_OPT, ) -> None: """Crea una sessione e stampa il suo id (ultima riga dell'output).""" - from tht.session.store import _extract_name, create_session + from tht.session.store import _extract_name, new_session_manifest, render_question_md cfg = _load_config_or_exit(config) - manifest = create_session(question, cfg.database, cfg.paths.sessions, + manifest = new_session_manifest(question, cfg.database, provider=provider, model=model, thinking=thinking, name=name or _extract_name(question)) + repository = session_repository(cfg) + repository.create(manifest) + repository.write_artifact(manifest.id, "question", render_question_md(question)) if json_out: typer.echo(json.dumps({"id": manifest.id}, ensure_ascii=False)) return - typer.secho(f"OK: sessione creata in {session_dir(cfg, manifest.id)}", fg=typer.colors.GREEN) + typer.secho(f"OK: sessione creata ({manifest.id})", fg=typer.colors.GREEN) typer.echo(manifest.id) @@ -102,12 +207,12 @@ def set_question_cmd( config: Path = CONFIG_OPT, ) -> None: """Scrive question.md (domanda riscritta + assunzioni) in modo deterministico.""" - from tht.session.store import set_question + from tht.session.store import render_question_md cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - path = set_question(session_id, question, assumption or [], cfg.paths.sessions) - typer.secho(f"OK: question.md aggiornato ({path}).", fg=typer.colors.GREEN) + session_repository(cfg).write_artifact(session_id, "question", render_question_md(question, assumption or [])) + typer.secho("OK: question.md aggiornato.", fg=typer.colors.GREEN) @session_app.command("set-schema-linking") @@ -123,7 +228,7 @@ def set_schema_linking_cmd( from pydantic import ValidationError - from tht.session.store import set_schema_linking + from tht.session.models import SchemaLinking cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) @@ -134,11 +239,12 @@ def set_schema_linking_cmd( typer.secho(f"ERRORE: JSON non valido: {e}", fg=typer.colors.RED, err=True) raise typer.Exit(code=5) try: - path = set_schema_linking(session_id, data, cfg.paths.sessions) + model = SchemaLinking.model_validate(data) + session_repository(cfg).write_artifact(session_id, "schema_linking", json.dumps(model.model_dump(by_alias=True), indent=2, ensure_ascii=False)) except ValidationError as e: typer.secho(f"ERRORE: schema_linking non valido:\n{e}", fg=typer.colors.RED, err=True) raise typer.Exit(code=5) - typer.secho(f"OK: schema_linking.json aggiornato ({path}).", fg=typer.colors.GREEN) + typer.secho("OK: schema_linking.json aggiornato.", fg=typer.colors.GREEN) @session_app.command("sync-schema-linking") @@ -149,16 +255,17 @@ def sync_schema_linking_cmd( """Riproietta schema_linking.json dalle decisioni F4 del ledger (deterministico).""" from pydantic import ValidationError - from tht.session.store import sync_schema_linking + from tht.session.store import sync_schema_linking_snapshot cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) try: - path = sync_schema_linking(session_id, cfg.paths.sessions) + content = sync_schema_linking_snapshot(load_snapshot_or_exit(cfg, session_id)) + session_repository(cfg).write_artifact(session_id, "schema_linking", content) except ValidationError as e: typer.secho(f"ERRORE: schema_linking non valido:\n{e}", fg=typer.colors.RED, err=True) raise typer.Exit(code=5) - typer.secho(f"OK: schema_linking.json riproiettato ({path}).", fg=typer.colors.GREEN) + typer.secho("OK: schema_linking.json riproiettato.", fg=typer.colors.GREEN) @session_app.command("show") @@ -168,28 +275,27 @@ def show_cmd( config: Path = CONFIG_OPT, ) -> None: """Stato della sessione: manifest + decisioni registrate (per la ripresa).""" - from tht.decisions import list_decisions from tht.phase import current_phase cfg = _load_config_or_exit(config) manifest = load_session_or_exit(cfg, session_id) - sdir = session_dir(cfg, session_id) + snapshot = load_snapshot_or_exit(cfg, session_id) if json_out: - has_schema_linking = (sdir / "schema_linking.json").exists() - phase = current_phase(sdir) + has_schema_linking = "schema_linking" in snapshot.artifacts + phase = current_phase(snapshot) data = manifest.model_dump(mode="json", by_alias=True) data["phase"] = phase data["has_schema_linking"] = has_schema_linking # Ledger integrale: il gate lo usa per costruire deterministicamente il # recap delle decisioni nei riepiloghi di fase (v2). data["decisions"] = [ - d.model_dump(mode="json") for d in list_decisions(sdir) + d.model_dump(mode="json") for d in snapshot.decisions ] typer.echo(json.dumps(data, ensure_ascii=False, indent=2)) return - decisions = list_decisions(sdir) + decisions = snapshot.decisions typer.echo(f"id : {manifest.id}") typer.echo(f"stato : {manifest.status}") typer.echo(f"domanda : {manifest.question}") @@ -197,8 +303,7 @@ def show_cmd( typer.echo(f"decisioni: {len(decisions)}") for d in decisions[-10:]: typer.echo(f" [{d.seq}] {d.type}: {d.subject}" + (f" — {d.detail}" if d.detail else "")) - linking = sdir / "schema_linking.json" - typer.echo(f"schema_linking.json: {'presente' if linking.exists() else 'assente'}") + typer.echo(f"schema_linking.json: {'presente' if 'schema_linking' in snapshot.artifacts else 'assente'}") @session_app.command("retrieval-pack") @@ -209,12 +314,10 @@ def retrieval_pack_cmd( """Emette su stdout il retrieval pack persistito della sessione.""" cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - path = session_dir(cfg, session_id) / "retrieval_pack.md" - try: - content = path.read_text() - except OSError as exc: + content = load_snapshot_or_exit(cfg, session_id).artifacts.get("retrieval_pack") + if content is None: typer.secho( - f"ERRORE: retrieval pack non disponibile per la sessione {session_id}: {exc}", + f"ERRORE: retrieval pack non disponibile per la sessione {session_id}", fg=typer.colors.RED, err=True, ) @@ -225,32 +328,32 @@ def retrieval_pack_cmd( @session_app.command("close") def close_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None: """Chiude la sessione (status=closed).""" - from tht.session.store import close_session - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - close_session(session_id, cfg.paths.sessions) + snapshot = load_snapshot_or_exit(cfg, session_id) + snapshot.manifest.status = "closed" + session_repository(cfg).save_manifest(snapshot.manifest) typer.secho(f"OK: sessione {session_id} chiusa.", fg=typer.colors.GREEN) @session_app.command("fail") def fail_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None: """Registra un arresto del sistema non recuperabile automaticamente.""" - from tht.session.store import fail_session - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - fail_session(session_id, cfg.paths.sessions) + snapshot = load_snapshot_or_exit(cfg, session_id) + snapshot.manifest.status = "failed" + session_repository(cfg).save_manifest(snapshot.manifest) typer.secho(f"OK: sessione {session_id} marcata failed.", fg=typer.colors.RED) @session_app.command("reopen") def reopen_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None: """Riapre una sessione per una ripresa manuale.""" - from tht.session.store import reopen_session - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - reopen_session(session_id, cfg.paths.sessions) + snapshot = load_snapshot_or_exit(cfg, session_id) + snapshot.manifest.status = "open" + session_repository(cfg).save_manifest(snapshot.manifest) typer.secho(f"OK: sessione {session_id} riaperta.", fg=typer.colors.GREEN) @@ -261,11 +364,11 @@ def set_name_cmd( config: Path = CONFIG_OPT, ) -> None: """Imposta il nome descrittivo della sessione.""" - from tht.session.store import set_name - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - set_name(session_id, name, cfg.paths.sessions) + snapshot = load_snapshot_or_exit(cfg, session_id) + snapshot.manifest.name = name.strip() or None + session_repository(cfg).save_manifest(snapshot.manifest) typer.secho(f"OK: nome aggiornato per {session_id}.", fg=typer.colors.GREEN) @@ -276,44 +379,42 @@ def set_group_cmd( config: Path = CONFIG_OPT, ) -> None: """Sposta la sessione in un gruppo (o la toglie da ogni gruppo).""" - from tht.session.store import set_group - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - set_group(session_id, group, cfg.paths.sessions) + snapshot = load_snapshot_or_exit(cfg, session_id) + snapshot.manifest.group = group.strip() or None + session_repository(cfg).save_manifest(snapshot.manifest) typer.secho(f"OK: gruppo aggiornato per {session_id}.", fg=typer.colors.GREEN) @session_app.command("archive") def archive_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None: """Archivia la sessione (la toglie dalla lista attiva, sola lettura).""" - from tht.session.store import set_archived - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - set_archived(session_id, True, cfg.paths.sessions) + snapshot = load_snapshot_or_exit(cfg, session_id) + snapshot.manifest.archived = True + session_repository(cfg).save_manifest(snapshot.manifest) typer.secho(f"OK: sessione {session_id} archiviata.", fg=typer.colors.GREEN) @session_app.command("unarchive") def unarchive_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None: """Ripristina la sessione dall'archivio (non ne cambia la ripristinabilità).""" - from tht.session.store import set_archived - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - set_archived(session_id, False, cfg.paths.sessions) + snapshot = load_snapshot_or_exit(cfg, session_id) + snapshot.manifest.archived = False + session_repository(cfg).save_manifest(snapshot.manifest) typer.secho(f"OK: sessione {session_id} ripristinata.", fg=typer.colors.GREEN) @session_app.command("delete") def delete_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None: """Elimina definitivamente la cartella di sessione.""" - from tht.session.store import delete_session - cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - delete_session(session_id, cfg.paths.sessions) + session_repository(cfg).delete(session_id) typer.secho(f"OK: sessione {session_id} eliminata.", fg=typer.colors.GREEN) @@ -324,11 +425,10 @@ def documents_cmd( config: Path = CONFIG_OPT, ) -> None: """Documenti di sola lettura della sessione (domanda, rivista, schema, SQL, report, decisioni).""" - from tht.session.store import build_documents + from tht.session.store import build_snapshot_documents cfg = _load_config_or_exit(config) - manifest = load_session_or_exit(cfg, session_id) - docs = build_documents(manifest, session_dir(cfg, session_id)) + docs = build_snapshot_documents(load_snapshot_or_exit(cfg, session_id)) if json_out: typer.echo(json.dumps(docs, ensure_ascii=False, indent=2)) return @@ -342,19 +442,18 @@ def session_problems(cfg, session_id: str) -> list[str]: from pydantic import ValidationError - from tht.decisions import list_decisions from tht.session.models import SchemaLinking - sdir = session_dir(cfg, session_id) + snapshot = load_snapshot_or_exit(cfg, session_id) problems: list[str] = [] - if not list_decisions(sdir): + if not snapshot.decisions: problems.append("nessuna decisione registrata (review_decisions.jsonl vuoto o assente)") - linking_path = sdir / "schema_linking.json" - if not linking_path.exists(): + raw_linking = snapshot.artifacts.get("schema_linking") + if raw_linking is None: problems.append("schema_linking.json assente") else: try: - SchemaLinking.model_validate(json.loads(linking_path.read_text())) + SchemaLinking.model_validate(json.loads(raw_linking)) except (json.JSONDecodeError, ValidationError) as e: problems.append(f"schema_linking.json non valido: {e}") return problems @@ -368,7 +467,7 @@ def check_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) cfg = _load_config_or_exit(config) load_session_or_exit(cfg, session_id) - cur = current_phase(session_dir(cfg, session_id)) + cur = current_phase(load_snapshot_or_exit(cfg, session_id)) wf = load_workflow() schema_linking_phase = wf.schema_linking_phase() if cur < schema_linking_phase: @@ -409,7 +508,7 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP do_run, promoted_tables_for, ) - from tht.ctetest import CteError, load_cte_tests + from tht.ctetest import CteError, CteTestRecord, _iter_json_objects from tht.execute import ExecutionError from tht.execute.warnings import plan_warnings, runtime_warnings, static_warnings from tht.report import extract_reviewer_notes, render_validation_report @@ -420,13 +519,13 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP from tht.sqlcheck import validate_sql cfg = _load_config_or_exit(config) - manifest = load_session_or_exit(cfg, session_id) - sdir = session_dir(cfg, session_id) + repository = session_repository(cfg) + snapshot = load_snapshot_or_exit(cfg, session_id) from tht.phase import current_phase from tht.workflow import load_workflow - cur = current_phase(sdir) + cur = current_phase(snapshot) wf = load_workflow() if cur <= wf.max_phase: typer.secho( @@ -440,9 +539,9 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP # --- gate di ingresso --- # Vista effective (D15): un sql_approved/cte ritirato o stale post-rollback non conta. problems = session_problems(cfg, session_id) - decisions = effective_decisions(sdir) - sql_file = sdir / "sql_final.sql" - if not sql_file.exists(): + decisions = effective_decisions(snapshot) + sql = snapshot.artifacts.get("sql_final") + if sql is None: problems.append("sql_final.sql assente") if not any(d.type == "sql_approved" for d in decisions): problems.append( @@ -451,10 +550,11 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP ) # I CTE richiesti sono quelli del PIANO effettivo, non i file glob su disco: un # ctes/*.sql orfano lasciato da un teardown incompleto non deve bloccare il finalize. - plan = effective_cte_plan(sdir) + plan = effective_cte_plan(snapshot) if plan: try: - tested = {r.name for r in load_cte_tests(sdir)} + raw_tests = snapshot.artifacts.get("cte_tests", "") + tested = {r.name for r in map(CteTestRecord.model_validate, _iter_json_objects(raw_tests))} except CteError as e: typer.secho( f"Finalize rifiutato: impossibile leggere cte_tests.json: {e}", @@ -471,7 +571,7 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP raise typer.Exit(code=3) # --- batteria di validazione su sql_final.sql --- - sql = sql_file.read_text() + assert sql is not None check = validate_sql( sql, physical=_load_physical_or_exit(cfg), @@ -493,34 +593,29 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP raise typer.Exit(code=3) # --- validation_report.md (preservando le note del reviewer) --- - report_path = sdir / "validation_report.md" - existing_notes = ( - extract_reviewer_notes(report_path.read_text()) if report_path.exists() else "" - ) - report_path.write_text(render_validation_report( + existing_report = snapshot.artifacts.get("validation_report", "") + existing_notes = extract_reviewer_notes(existing_report) if existing_report else "" + report = render_validation_report( session_id=session_id, check=check, plan=plan, plan_warnings=plan_warnings(plan, cfg.execution), result=result, runtime_warnings=runtime_warnings(result, cfg.execution), static_warnings=static_warnings(check.ast), limit=limit, reviewer_notes=existing_notes, - )) + ) # --- evidence.json --- linking = SchemaLinking.model_validate( - json.loads((sdir / "schema_linking.json").read_text()) + json.loads(snapshot.artifacts["schema_linking"]) ) entries = build_evidence_entries(decisions, linking, cfg.paths.artifacts / "evidence") - (sdir / "evidence.json").write_text(json.dumps(entries, ensure_ascii=False, indent=2)) + evidence = json.dumps(entries, ensure_ascii=False, indent=2) # --- manifest + riepilogo --- - from datetime import UTC, datetime + from tht.session.store import persist_verified_finalization - from tht.session.store import MANIFEST, current_author - - manifest.status = "finalized" - manifest.updated_at = datetime.now(UTC) - manifest.updated_by = current_author() - manifest.to_yaml(sdir / MANIFEST) + persist_verified_finalization( + repository, session_id, validation_report=report, evidence=evidence + ) # --- memoria attiva (parte B): indicizza la coppia domanda->SQL, best-effort --- # Import lazy: memory_cmd importa da session_cmd (un import top-level qui sarebbe # circolare). Qualunque errore (writer key assente, VPN giu', Ollama spento) NON @@ -547,5 +642,5 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP ) typer.secho(f"OK: sessione {session_id} finalizzata. Artefatti:", fg=typer.colors.GREEN) for name in ARTIFACT_FILES: - state = "presente" if (sdir / name).exists() else "assente" + state = "presente" if name not in {"session_manifest.yaml", "review_decisions.jsonl"} else "persistito" typer.echo(f" - {name}: {state}") diff --git a/harness/tht/cli/sql_cmd.py b/harness/tht/cli/sql_cmd.py index e1dc609b..e9a2d247 100644 --- a/harness/tht/cli/sql_cmd.py +++ b/harness/tht/cli/sql_cmd.py @@ -42,12 +42,14 @@ def require_action(cfg, action: str) -> None: def promoted_tables_for(cfg, session_id: str | None) -> set[str] | None: if session_id is None: return None - linking_path = cfg.paths.sessions / session_id / "schema_linking.json" - if not linking_path.exists(): + from tht.cli.session_cmd import load_snapshot_or_exit + + raw = load_snapshot_or_exit(cfg, session_id).artifacts.get("schema_linking") + if raw is None: return None from tht.session.models import SchemaLinking - linking = SchemaLinking.model_validate(json.loads(linking_path.read_text())) + linking = SchemaLinking.model_validate(json.loads(raw)) return { c.name for c in linking.candidates if c.kind == "table" and c.decision == "promoted" @@ -59,12 +61,14 @@ def promoted_tables_for(cfg, session_id: str | None) -> set[str] | None: def promoted_columns_for(cfg, session_id: str | None) -> set[str] | None: if session_id is None: return None - linking_path = cfg.paths.sessions / session_id / "schema_linking.json" - if not linking_path.exists(): + from tht.cli.session_cmd import load_snapshot_or_exit + + raw = load_snapshot_or_exit(cfg, session_id).artifacts.get("schema_linking") + if raw is None: return None from tht.session.models import SchemaLinking - linking = SchemaLinking.model_validate(json.loads(linking_path.read_text())) + linking = SchemaLinking.model_validate(json.loads(raw)) return { c.name for c in linking.candidates if c.kind == "column" and c.decision == "promoted" @@ -182,7 +186,7 @@ def preview_cmd( """Esecuzione controllata con LIMIT iniettato; aggregati mostrati per interi. Se FILE è omesso e --session è fornito, il file viene risolto automaticamente - come /sessions//sql_final.sql (tramite _session_sql_file). + dall'artefatto `sql_final` del repository della sessione. """ from tht.execute import ExecutionError @@ -195,8 +199,7 @@ def preview_cmd( fg=typer.colors.RED, err=True, ) raise typer.Exit(code=1) - resolved = _session_sql_file(cfg, session) - sql = resolved.read_text() + sql = _session_sql(cfg, session) else: sql = _read_sql(file) check = validate_or_exit(cfg, sql, session) @@ -247,15 +250,32 @@ def preview_cmd( typer.secho(f" warning: {w}", fg=typer.colors.YELLOW) -def _session_sql_file(cfg, session_id: str) -> Path: - from tht.cli.session_cmd import load_session_or_exit, session_dir +def _session_sql(cfg, session_id: str) -> str: + from tht.cli.session_cmd import load_snapshot_or_exit - load_session_or_exit(cfg, session_id) - sql_file = session_dir(cfg, session_id) / "sql_final.sql" - if not sql_file.exists(): - typer.secho(f"ERRORE: {sql_file} non trovato.", fg=typer.colors.RED, err=True) + sql = load_snapshot_or_exit(cfg, session_id).artifacts.get("sql_final") + if sql is None: + typer.secho("ERRORE: sql_final.sql non trovato.", fg=typer.colors.RED, err=True) raise typer.Exit(code=1) - return sql_file + return sql + + +@sql_app.command("set-final") +def set_final_cmd( + session: str = typer.Option(..., "--session"), + file: str = typer.Option(..., "--file", help="File SQL, oppure '-' per stdin."), + config: Path = CONFIG_OPT, +) -> None: + """Persist clean final SQL through the configured session repository.""" + import sys + + from tht.cli.session_cmd import load_session_or_exit, session_repository + + cfg = _load_config_or_exit(config) + load_session_or_exit(cfg, session) + sql = sys.stdin.read() if file == "-" else _read_sql(Path(file)) + session_repository(cfg).write_artifact(session, "sql_final", sql) + typer.secho("OK: SQL finale salvato.", fg=typer.colors.GREEN) @sql_app.command("save") @@ -266,9 +286,8 @@ def save_cmd( ) -> None: """Salva una copia di sql_final.sql nel percorso indicato (su richiesta esplicita).""" cfg = _load_config_or_exit(config) - sql_file = _session_sql_file(cfg, session) dest.parent.mkdir(parents=True, exist_ok=True) - dest.write_text(sql_file.read_text()) + dest.write_text(_session_sql(cfg, session)) typer.secho(f"OK: SQL salvato in {dest}", fg=typer.colors.GREEN) @@ -286,8 +305,7 @@ def export_cmd( cfg = _load_config_or_exit(config) require_action(cfg, "export") - sql_file = _session_sql_file(cfg, session) - sql = sql_file.read_text() + sql = _session_sql(cfg, session) validate_or_exit(cfg, sql, session) try: result = do_run(cfg, sql, limit=cfg.execution.max_export_rows) diff --git a/harness/tht/config.py b/harness/tht/config.py index 11644e5d..a4743613 100644 --- a/harness/tht/config.py +++ b/harness/tht/config.py @@ -16,6 +16,11 @@ class ConfigError(Exception): """Errore di configurazione, con messaggio leggibile per l'utente.""" +def local_tht_home() -> Path: + """Return the private local ThothII home, honoring the explicit override.""" + return Path(os.environ.get("THT_HOME", "~/.thothii")).expanduser() + + def _expand_env(value: Any) -> Any: if isinstance(value, str): @@ -73,6 +78,22 @@ class DatabaseConfig(BaseModel): model_config = {"populate_by_name": True} +class SessionPostgresConfig(DatabaseConfig): + """TLS-verified direct session storage; its login must be granted thoth_sessions_runtime. + + Provision the dedicated LOGIN role and membership out of band so deployment credentials + never appear in the versioned migration pack. + """ + + sslmode: Literal["verify-ca", "verify-full"] = "verify-full" + sslrootcert: Path | None = None + + +class SessionStorageConfig(BaseModel): + type: Literal["postgres_direct"] + connection: SessionPostgresConfig + + class RestConfig(BaseModel): """Accesso al DWH via Supabase/PostgREST. base_url es. https://host/dwh/ .""" @@ -287,6 +308,7 @@ class Config(BaseModel): _config_source: str = PrivateAttr(default="direct") dwh: DwhResourceConfig vectors: VectorResourceConfig | None = None + session_storage: SessionStorageConfig | None = None roots: WorkspaceRoots = WorkspaceRoots() # Compatibility views retained until all call sites consume typed resources. database: DatabaseConfig @@ -356,7 +378,9 @@ def load_config(path: Path) -> Config: raise ConfigError( f"transport: rest richiede la sezione `rest` (base_url, api_key) in {path}." ) - data_root = os.environ.get("THT_DATA_ROOT") + # THT_HOME is the local-user storage root. Retain THT_DATA_ROOT as the + # portable-deployment compatibility name until all callers use repositories. + data_root = os.environ.get("THT_HOME") or os.environ.get("THT_DATA_ROOT") if data_root: # Import locally: paths owns resolution, while ConfigError remains the public # configuration exception callers already handle. diff --git a/harness/tht/ctetest.py b/harness/tht/ctetest.py index 1b82fb22..fdba06c4 100644 --- a/harness/tht/ctetest.py +++ b/harness/tht/ctetest.py @@ -99,7 +99,10 @@ def load_cte_tests(session_dir: Path) -> list[CteTestRecord]: path = session_dir / CTE_TESTS_FILE if not path.exists(): return [] - text = path.read_text() + return load_cte_tests_text(path.read_text()) + + +def load_cte_tests_text(text: str) -> list[CteTestRecord]: try: objs = list(_iter_json_objects(text)) except json.JSONDecodeError as e: @@ -107,6 +110,11 @@ def load_cte_tests(session_dir: Path) -> list[CteTestRecord]: return [CteTestRecord.model_validate(o) for o in objs] +def append_cte_test_snapshot(repository, snapshot, record: CteTestRecord) -> None: + previous = snapshot.artifacts.get("cte_tests", "") + repository.write_artifact(snapshot.manifest.id, "cte_tests", previous + record.model_dump_json() + "\n") + + def append_cte_test(session_dir: Path, record: CteTestRecord) -> None: """Append atomico (una riga JSON per esito): scritture concorrenti sulla stessa sessione non si sovrascrivono, a differenza del rewrite dell'array.""" diff --git a/harness/tht/jobs/dwh_pipeline.py b/harness/tht/jobs/dwh_pipeline.py index a483a404..e67b9164 100644 --- a/harness/tht/jobs/dwh_pipeline.py +++ b/harness/tht/jobs/dwh_pipeline.py @@ -48,9 +48,21 @@ def config_dwh_binding(cfg) -> dict[str, str]: config_source = getattr(cfg, "_config_source", None) if not isinstance(workspace_id, str) or not isinstance(config_source, str): raise CorruptCheckpointError("DWH workspace identity is unavailable; reload configuration") + model_dump = getattr(cfg, "model_dump", None) + if callable(model_dump): + payload = model_dump(mode="json") + if not isinstance(payload, dict): + raise CorruptCheckpointError("DWH workspace configuration is unavailable; reload configuration") + # Session persistence has no bearing on schema/LSH artifacts. Excluding it keeps an + # opt-in session-storage deployment from invalidating an otherwise identical DWH cache. + payload.pop("session_storage", None) + config_fingerprint = fingerprint(json.dumps(payload, separators=(",", ":"), ensure_ascii=False)) + else: + # Lightweight test doubles predating Pydantic's model_dump() retain the legacy seam. + config_fingerprint = fingerprint(cfg.model_dump_json()) return { "workspace_id": workspace_id, - "config_fingerprint": fingerprint(cfg.model_dump_json()), + "config_fingerprint": config_fingerprint, "input_fingerprint": fingerprint(config_source), } diff --git a/harness/tht/memory.py b/harness/tht/memory.py index 8a6acc78..cdd2ed2d 100644 --- a/harness/tht/memory.py +++ b/harness/tht/memory.py @@ -186,6 +186,14 @@ def promote( return promoted +def promote_snapshot(snapshot, *, seqs: list[int] | None, registry_path: Path) -> list[MemoryRecord]: + existing = load_registry(registry_path) + promoted = _compute_promotions(snapshot, snapshot.manifest, seqs=seqs, existing=existing) + if promoted: + save_registry(existing + promoted, registry_path) + return promoted + + def reusable_promotions( session_dir: Path, manifest: SessionManifest, registry_path: Path ) -> list[MemoryRecord]: @@ -203,6 +211,14 @@ def reusable_promotions( ] +def reusable_promotions_snapshot(snapshot, registry_path: Path) -> list[MemoryRecord]: + from tht.phase import effective_decisions + + cand = _compute_promotions(snapshot, snapshot.manifest, seqs=None, existing=load_registry(registry_path)) + declined = declined_promotion_seqs(effective_decisions(snapshot)) + return [c for c in cand if c.type in REUSABLE_TYPES and c.decision_seq not in declined] + + def preview_promotions( session_dir: Path, manifest: SessionManifest, registry_path: Path ) -> list[MemoryRecord]: @@ -212,6 +228,10 @@ def preview_promotions( ] +def preview_promotions_snapshot(snapshot, registry_path: Path) -> list[MemoryRecord]: + return reusable_promotions_snapshot(snapshot, registry_path)[:MAX_PROMOTION_CANDIDATES] + + def memory_vector_records(records: list[MemoryRecord]) -> list[VectorRecord]: out: list[VectorRecord] = [] for r in records: diff --git a/harness/tht/migrations/sessions/001_schema.sql b/harness/tht/migrations/sessions/001_schema.sql new file mode 100644 index 00000000..684d19d4 --- /dev/null +++ b/harness/tht/migrations/sessions/001_schema.sql @@ -0,0 +1,60 @@ +CREATE SCHEMA IF NOT EXISTS thoth_sessions; +REVOKE ALL ON SCHEMA thoth_sessions FROM PUBLIC; + +CREATE TABLE IF NOT EXISTS thoth_sessions.principals ( + id bigserial PRIMARY KEY, + issuer text NOT NULL, + subject text NOT NULL, + display_name text, + created_at timestamptz NOT NULL DEFAULT pg_catalog.now(), + updated_at timestamptz NOT NULL DEFAULT pg_catalog.now(), + UNIQUE (issuer, subject) +); + +CREATE TABLE IF NOT EXISTS thoth_sessions.principal_preferences ( + principal_id bigint PRIMARY KEY REFERENCES thoth_sessions.principals(id) ON DELETE CASCADE, + preferences jsonb NOT NULL DEFAULT '{}'::jsonb, + updated_at timestamptz NOT NULL DEFAULT pg_catalog.now() +); + +CREATE TABLE IF NOT EXISTS thoth_sessions.sessions ( + id uuid PRIMARY KEY, + principal_id bigint NOT NULL REFERENCES thoth_sessions.principals(id), + manifest jsonb NOT NULL, + created_at timestamptz NOT NULL DEFAULT pg_catalog.now(), + updated_at timestamptz NOT NULL DEFAULT pg_catalog.now() +); +CREATE INDEX IF NOT EXISTS sessions_principal_id_created_at_idx +ON thoth_sessions.sessions (principal_id, created_at DESC); + +CREATE TABLE IF NOT EXISTS thoth_sessions.session_artifacts ( + session_id uuid NOT NULL REFERENCES thoth_sessions.sessions(id) ON DELETE CASCADE, + artifact_key text NOT NULL, + content text NOT NULL, + updated_at timestamptz NOT NULL DEFAULT pg_catalog.now(), + PRIMARY KEY (session_id, artifact_key) +); + +CREATE TABLE IF NOT EXISTS thoth_sessions.review_decisions ( + session_id uuid NOT NULL REFERENCES thoth_sessions.sessions(id) ON DELETE CASCADE, + seq integer NOT NULL CHECK (seq > 0), + ts timestamptz NOT NULL, + phase integer, + type text NOT NULL, + subject text NOT NULL, + detail text NOT NULL DEFAULT '', + rationale text NOT NULL DEFAULT '', + retracts integer, + PRIMARY KEY (session_id, seq) +); + +CREATE TABLE IF NOT EXISTS thoth_sessions.audit_log ( + id bigserial PRIMARY KEY, + occurred_at timestamptz NOT NULL DEFAULT pg_catalog.now(), + action text NOT NULL, + session_id uuid NOT NULL, + actor_issuer text NOT NULL, + actor_subject text NOT NULL, + owner_issuer text NOT NULL, + owner_subject text NOT NULL +); diff --git a/harness/tht/migrations/sessions/002_security.sql b/harness/tht/migrations/sessions/002_security.sql new file mode 100644 index 00000000..8a123e60 --- /dev/null +++ b/harness/tht/migrations/sessions/002_security.sql @@ -0,0 +1,126 @@ +DO $roles$ +BEGIN + IF NOT EXISTS (SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = 'thoth_sessions_runtime') THEN + CREATE ROLE thoth_sessions_runtime NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT; + END IF; + IF NOT EXISTS (SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = 'thoth_sessions_migrator') THEN + CREATE ROLE thoth_sessions_migrator NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT; + END IF; +END +$roles$; + +ALTER ROLE thoth_sessions_runtime NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT; +ALTER ROLE thoth_sessions_migrator NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT; + +REVOKE ALL ON SCHEMA thoth_sessions FROM PUBLIC; +REVOKE ALL ON ALL TABLES IN SCHEMA thoth_sessions FROM PUBLIC; +REVOKE ALL ON ALL SEQUENCES IN SCHEMA thoth_sessions FROM PUBLIC; +REVOKE ALL ON SCHEMA thoth_sessions FROM thoth_sessions_runtime, thoth_sessions_migrator; +REVOKE ALL ON ALL TABLES IN SCHEMA thoth_sessions FROM thoth_sessions_runtime, thoth_sessions_migrator; +REVOKE ALL ON ALL SEQUENCES IN SCHEMA thoth_sessions FROM thoth_sessions_runtime, thoth_sessions_migrator; + +GRANT USAGE ON SCHEMA thoth_sessions TO thoth_sessions_runtime; +GRANT SELECT, INSERT, UPDATE ON thoth_sessions.principals TO thoth_sessions_runtime; +GRANT SELECT, INSERT, UPDATE ON thoth_sessions.principal_preferences TO thoth_sessions_runtime; +GRANT SELECT, INSERT, UPDATE, DELETE ON thoth_sessions.sessions TO thoth_sessions_runtime; +GRANT SELECT, INSERT, UPDATE ON thoth_sessions.session_artifacts TO thoth_sessions_runtime; +GRANT SELECT, INSERT ON thoth_sessions.review_decisions TO thoth_sessions_runtime; +GRANT SELECT, INSERT ON thoth_sessions.audit_log TO thoth_sessions_runtime; +GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA thoth_sessions TO thoth_sessions_runtime; + +ALTER TABLE thoth_sessions.principals ENABLE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.principals FORCE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.principal_preferences ENABLE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.principal_preferences FORCE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.sessions ENABLE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.sessions FORCE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.session_artifacts ENABLE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.session_artifacts FORCE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.review_decisions ENABLE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.review_decisions FORCE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.audit_log ENABLE ROW LEVEL SECURITY; +ALTER TABLE thoth_sessions.audit_log FORCE ROW LEVEL SECURITY; + +CREATE POLICY principals_owner_or_admin ON thoth_sessions.principals + FOR ALL + USING ( + pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true' + OR (issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true) + AND subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true)) + ) + WITH CHECK ( + pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true' + OR (issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true) + AND subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true)) + ); + +CREATE POLICY preferences_owner_or_admin ON thoth_sessions.principal_preferences + FOR ALL + USING ( + EXISTS ( + SELECT 1 FROM thoth_sessions.principals p + WHERE p.id = principal_preferences.principal_id + AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true' + OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true) + AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true))) + ) + ) + WITH CHECK ( + EXISTS ( + SELECT 1 FROM thoth_sessions.principals p + WHERE p.id = principal_preferences.principal_id + AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true' + OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true) + AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true))) + ) + ); + +CREATE POLICY sessions_owner_or_admin ON thoth_sessions.sessions + FOR ALL + USING ( + EXISTS ( + SELECT 1 FROM thoth_sessions.principals p + WHERE p.id = sessions.principal_id + AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true' + OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true) + AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true))) + ) + ) + WITH CHECK ( + EXISTS ( + SELECT 1 FROM thoth_sessions.principals p + WHERE p.id = sessions.principal_id + AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true' + OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true) + AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true))) + ) + ); + +CREATE POLICY artifacts_owner_or_admin ON thoth_sessions.session_artifacts + FOR ALL + USING ( + EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = session_artifacts.session_id) + ) + WITH CHECK ( + EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = session_artifacts.session_id) + ); + +CREATE POLICY decisions_owner_or_admin ON thoth_sessions.review_decisions + FOR ALL + USING ( + EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = review_decisions.session_id) + ) + WITH CHECK ( + EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = review_decisions.session_id) + ); + +CREATE POLICY audit_admin_read ON thoth_sessions.audit_log + FOR SELECT + USING (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'); +CREATE POLICY audit_actor_write ON thoth_sessions.audit_log + FOR INSERT + WITH CHECK ( + pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true' + OR (actor_issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true) + AND actor_subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true)) + ); diff --git a/harness/tht/phase.py b/harness/tht/phase.py index dbcc8d7a..41df97e4 100644 --- a/harness/tht/phase.py +++ b/harness/tht/phase.py @@ -17,7 +17,7 @@ from pathlib import Path from pydantic import ValidationError from tht.decisions import DecisionRecord, list_decisions -from tht.session.models import SchemaLinking +from tht.session.models import SchemaLinking, SessionSnapshot from tht.workflow import load_workflow @@ -31,11 +31,22 @@ def _phase_num(subject: str) -> int | None: return None -def _audit_excluding_retracted(session_dir: Path) -> list[DecisionRecord]: +def _decisions(source: Path | SessionSnapshot) -> list[DecisionRecord]: + return list(source.decisions) if isinstance(source, SessionSnapshot) else list_decisions(source) + + +def _artifact(source: Path | SessionSnapshot, key: str, filename: str) -> str | None: + if isinstance(source, SessionSnapshot): + return source.artifacts.get(key) + path = source / filename + return path.read_text() if path.exists() else None + + +def _audit_excluding_retracted(source: Path | SessionSnapshot) -> list[DecisionRecord]: """Tutto il ledger (append-only) tranne le decisioni ritirate e i marker di ritrazione. Base per il fold di current_phase: il guard 'n == cur' del fold e' gia' reopen-aware (una phase_approved:N dopo un reopen a M list[DecisionRecord]: ] -def current_phase(session_dir: Path) -> int: +def current_phase(source: Path | SessionSnapshot) -> int: """Fase corrente come fold cronologico sull'audit (con ritirate escluse). cur parte da 1; ogni phase_approved/phase_auto_approved per la fase CORRENTE avanza @@ -57,7 +68,7 @@ def current_phase(session_dir: Path) -> int: wf = load_workflow() max_plus_one = wf.max_phase + 1 cur = 1 - for d in _audit_excluding_retracted(session_dir): + for d in _audit_excluding_retracted(source): n = _phase_num(d.subject) if n is None: continue @@ -68,7 +79,7 @@ def current_phase(session_dir: Path) -> int: return cur -def effective_decisions(session_dir: Path) -> list[DecisionRecord]: +def effective_decisions(source: Path | SessionSnapshot) -> list[DecisionRecord]: """La vista canonica 'effective as of pointer'. TUTTI gli helper non-fold devono usare questa. Semantica: una decisione e' effective se appartiene a una fase <= current_phase. @@ -81,9 +92,9 @@ def effective_decisions(session_dir: Path) -> list[DecisionRecord]: Inoltre esclude le decisioni ritirate (decision_retracted) e i marker stessi. """ - cur = current_phase(session_dir) + cur = current_phase(source) out: list[DecisionRecord] = [] - for d in _audit_excluding_retracted(session_dir): + for d in _audit_excluding_retracted(source): n = _phase_num(d.subject) # Per le decisioni con subject "a nome" (es. cte_approved -> nome CTE, evidence_* # -> id evidence) il subject non porta la fase: si usa la fase emittente registrata @@ -106,9 +117,9 @@ _META_TYPES = frozenset( ) -def substantive_count_current_phase(session_dir: Path) -> int: +def substantive_count_current_phase(source: Path | SessionSnapshot) -> int: """Numero di decisioni sostanziali dall'ultimo confine di fase (vista effective).""" - decs = effective_decisions(session_dir) + decs = effective_decisions(source) start = 0 for i, d in enumerate(decs): if d.type in _BOUNDARY_TYPES: @@ -116,14 +127,14 @@ def substantive_count_current_phase(session_dir: Path) -> int: return sum(1 for d in decs[start:] if d.type not in _META_TYPES) -def auto_advance_eligible(session_dir: Path) -> bool: +def auto_advance_eligible(source: Path | SessionSnapshot) -> bool: """Vero sse la fase corrente puo' auto-avanzare (zero decisioni sostanziali + prereq ok).""" - cur = current_phase(session_dir) + cur = current_phase(source) if cur not in _AUTO_ADVANCE_PHASES: return False - if substantive_count_current_phase(session_dir) > 0: + if substantive_count_current_phase(source) > 0: return False - return not advance_problems(session_dir, cur) + return not advance_problems(source, cur) # --- CTE helpers (consultano effective_decisions) --------------------------- @@ -131,21 +142,21 @@ def auto_advance_eligible(session_dir: Path) -> bool: CTE_PLAN_FILE = "cte_plan.json" -def cte_plan(session_dir: Path) -> list[str]: - path = session_dir / CTE_PLAN_FILE - if not path.exists(): +def cte_plan(source: Path | SessionSnapshot) -> list[str]: + raw = _artifact(source, "cte_plan", CTE_PLAN_FILE) + if raw is None: return [] - return json.loads(path.read_text()) + return json.loads(raw) -def approved_ctes(session_dir: Path) -> set[str]: +def approved_ctes(source: Path | SessionSnapshot) -> set[str]: """Insieme dei CTE approvati, dalla vista effective (esclude stale post-reopen).""" - return {d.subject for d in effective_decisions(session_dir) if d.type == "cte_approved"} + return {d.subject for d in effective_decisions(source) if d.type == "cte_approved"} -def next_cte(session_dir: Path) -> str | None: - approved = approved_ctes(session_dir) - for name in cte_plan(session_dir): +def next_cte(source: Path | SessionSnapshot) -> str | None: + approved = approved_ctes(source) + for name in cte_plan(source): if name not in approved: return name return None @@ -153,18 +164,18 @@ def next_cte(session_dir: Path) -> str | None: # --- advance_problems (ladder if-phase-N che consulta effective_decisions) --- -def _has_decision(session_dir: Path, type_: str) -> bool: - return any(d.type == type_ for d in effective_decisions(session_dir)) +def _has_decision(source: Path | SessionSnapshot, type_: str) -> bool: + return any(d.type == type_ for d in effective_decisions(source)) -def _has_decision_subject(session_dir: Path, type_: str, subject: str) -> bool: +def _has_decision_subject(source: Path | SessionSnapshot, type_: str, subject: str) -> bool: return any( d.type == type_ and d.subject == subject - for d in effective_decisions(session_dir) + for d in effective_decisions(source) ) -def advance_problems(session_dir: Path, phase: int) -> list[str]: +def advance_problems(source: Path | SessionSnapshot, phase: int) -> list[str]: """Prerequisiti minimi per chiudere `phase` (lista vuota = ok). Ladder if-phase-N (Strada 2): la logica specifica resta, ma ogni lettura passa per @@ -172,19 +183,19 @@ def advance_problems(session_dir: Path, phase: int) -> list[str]: l'evaluator generico (F2 pieno) entra in un secondo momento. """ problems: list[str] = [] - if phase == 3 and not _has_decision(session_dir, "question_rewritten"): + if phase == 3 and not _has_decision(source, "question_rewritten"): problems.append("manca la decisione question_rewritten (Fase 3)") if phase == 5: - path = session_dir / "schema_linking.json" - if not path.exists(): + raw = _artifact(source, "schema_linking", "schema_linking.json") + if raw is None: problems.append("schema_linking.json assente (Fase 5)") else: try: - SchemaLinking.model_validate(json.loads(path.read_text())) + SchemaLinking.model_validate(json.loads(raw)) except (json.JSONDecodeError, ValidationError) as e: problems.append(f"schema_linking.json non valido (Fase 5): {e}") - if phase == 6 and not _has_decision_subject(session_dir, "phase_skipped", "phase:6"): - plan = cte_plan(session_dir) + if phase == 6 and not _has_decision_subject(source, "phase_skipped", "phase:6"): + plan = cte_plan(source) if not plan: problems.append( "Fase 6: nessun piano CTE (cte_plan.json) e nessun salto esplicito. " @@ -192,14 +203,14 @@ def advance_problems(session_dir: Path, phase: int) -> list[str]: "registrando una decisione phase_skipped subject phase:6." ) else: - nc = next_cte(session_dir) + nc = next_cte(source) if nc is not None: problems.append(f"CTE non ancora approvato: {nc} (Fase 6)") - if phase == 7 and not _has_decision(session_dir, "sql_approved"): + if phase == 7 and not _has_decision(source, "sql_approved"): problems.append("manca la decisione sql_approved (Fase 7)") if phase == 8 and not any( d.type in ("datamart_requested", "datamart_declined") - for d in effective_decisions(session_dir) + for d in effective_decisions(source) ): problems.append( "Fase 8: nessuna risposta sulla generazione dbt del datamart " diff --git a/harness/tht/session/filesystem_repository.py b/harness/tht/session/filesystem_repository.py new file mode 100644 index 00000000..98794e80 --- /dev/null +++ b/harness/tht/session/filesystem_repository.py @@ -0,0 +1,254 @@ +"""Private local filesystem implementation of the session repository contract.""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +import shutil +import tempfile +import uuid +from pathlib import Path +from typing import Sequence + +import portalocker +import yaml + +from tht.decisions import DecisionInput, DecisionRecord, append_decisions, list_decisions +from tht.session.models import PrincipalContext, SessionManifest, SessionSnapshot +from tht.session.store import MANIFEST, SessionError + +_ARTIFACT_FILES = { + "question": "question.md", + "schema_linking": "schema_linking.json", + "evidence": "evidence.json", + "sql_final": "sql_final.sql", + "validation_report": "validation_report.md", + "retrieval_pack": "retrieval_pack.md", + "cte_tests": "cte_tests.json", + "cte_plan": "cte_plan.json", + "cte_plan_doc": "cte_plan_doc.json", +} +_ARTIFACT_KEYS = {filename: key for key, filename in _ARTIFACT_FILES.items()} +_SAFE_CTE_NAME = re.compile(r"[A-Za-z0-9_-]+\Z") +_LEGACY_SESSION_ID = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}\Z") + + +class FilesystemSessionRepository: + """Readable phase documents under one private local user's ThothII home.""" + + def __init__( + self, home: Path, workspace: str, principal: PrincipalContext, *, root: Path | None = None + ): + self.home = Path(home).expanduser() + self.workspace = workspace + self.principal = principal + self.root = Path(root) if root is not None else self.home / "workspaces" / workspace / "sessions" + + @property + def preferences_path(self) -> Path: + digest = hashlib.sha256( + f"{self.principal.issuer}\0{self.principal.subject}".encode() + ).hexdigest() + return self.home / "principals" / digest / "preferences.json" + + def create(self, manifest: SessionManifest) -> SessionSnapshot: + # New records are opaque UUIDv4 only. Historical timestamp ids remain + # readable/mutable through _session_dir but are never created here. + self._require_uuid4(manifest.id) + session_dir = self.root / manifest.id + with self._lock(session_dir): + if (session_dir / MANIFEST).exists(): + raise SessionError(f"Sessione esiste gia': {manifest.id}") + self._private_directory(session_dir) + self._write_manifest(session_dir, manifest) + return self.get(manifest.id) + + def get(self, session_id: str) -> SessionSnapshot: + session_dir = self._session_dir(session_id) + manifest_path = session_dir / MANIFEST + if not manifest_path.exists(): + raise SessionError(f"Sessione non trovata: {session_id} (atteso {manifest_path})") + manifest = SessionManifest.from_yaml(manifest_path) + artifacts = self._read_artifacts(session_dir) + return SessionSnapshot( + principal=self.principal, + manifest=manifest, + artifacts=artifacts, + decisions=list_decisions(session_dir), + ) + + def list(self) -> list[SessionSnapshot]: + if not self.root.exists(): + return [] + snapshots = [] + for path in self.root.iterdir(): + if path.is_dir() and (path / MANIFEST).exists(): + try: + snapshots.append(self.get(path.name)) + except SessionError: + continue + return sorted(snapshots, key=lambda item: item.manifest.created_at, reverse=True) + + def save_manifest(self, manifest: SessionManifest) -> SessionSnapshot: + session_dir = self._session_dir(manifest.id) + with self._lock(session_dir): + self._require_existing(session_dir, manifest.id) + self._write_manifest(session_dir, manifest) + return self.get(manifest.id) + + def read_artifact(self, session_id: str, key: str) -> str | None: + session_dir = self._session_dir(session_id) + self._require_existing(session_dir, session_id) + path = self._artifact_path(session_dir, key) + return path.read_text() if path.exists() else None + + def write_artifact(self, session_id: str, key: str, content: str) -> None: + session_dir = self._session_dir(session_id) + with self._lock(session_dir): + self._require_existing(session_dir, session_id) + self._write_private_text(self._artifact_path(session_dir, key), content) + + def delete_artifact(self, session_id: str, key: str) -> None: + session_dir = self._session_dir(session_id) + with self._lock(session_dir): + self._require_existing(session_dir, session_id) + self._artifact_path(session_dir, key).unlink(missing_ok=True) + + def append_decisions( + self, session_id: str, decisions: Sequence[DecisionInput | dict] + ) -> list[DecisionRecord]: + session_dir = self._session_dir(session_id) + with self._lock(session_dir): + self._require_existing(session_dir, session_id) + # The legacy ledger helper remains the canonical record format. Its + # separate lock also keeps direct workflow callers safe during Task 3. + return append_decisions(session_dir, list(decisions)) + + def finalize(self, manifest: SessionManifest, artifacts: dict[str, str]) -> SessionSnapshot: + """Commit verified final artifacts before publishing finalized status. + + The finalized manifest is the filesystem commit marker: a crash can leave + an open session with already-rendered artifacts, but never a finalized + session without its report and evidence. + """ + session_dir = self._session_dir(manifest.id) + with self._lock(session_dir): + self._require_existing(session_dir, manifest.id) + for key, content in artifacts.items(): + self._write_private_text(self._artifact_path(session_dir, key), content) + self._write_manifest(session_dir, manifest) + return self.get(manifest.id) + + def get_preferences(self) -> dict: + path = self.preferences_path + if not path.exists(): + return {} + data = json.loads(path.read_text()) + if not isinstance(data, dict): + raise ValueError(f"Invalid preferences: {path}") + return data + + def set_preferences(self, preferences: dict) -> None: + if not isinstance(preferences, dict): + raise TypeError("preferences must be a dictionary") + path = self.preferences_path + self._private_directory(path.parent) + with portalocker.Lock(path.with_suffix(".lock"), mode="a+", timeout=10): + self._write_private_text(path, json.dumps(preferences, ensure_ascii=False, sort_keys=True) + "\n") + + def delete(self, session_id: str) -> None: + session_dir = self._session_dir(session_id) + with self._lock(session_dir): + self._require_existing(session_dir, session_id) + shutil.rmtree(session_dir) + + def _session_dir(self, session_id: str) -> Path: + self._require_session_id(session_id) + return self.root / session_id + + @classmethod + def _require_session_id(cls, session_id: str) -> None: + try: + cls._require_uuid4(session_id) + except SessionError: + if session_id not in {".", ".."} and _LEGACY_SESSION_ID.fullmatch(session_id): + return + raise + + @staticmethod + def _require_uuid4(session_id: str) -> None: + try: + parsed = uuid.UUID(session_id, version=4) + except ValueError as exc: + raise SessionError(f"ID sessione non UUIDv4: {session_id}") from exc + if str(parsed) != session_id or parsed.version != 4: + raise SessionError(f"ID sessione non UUIDv4: {session_id}") + + @staticmethod + def _private_directory(path: Path) -> None: + path.mkdir(parents=True, exist_ok=True) + if os.name != "nt": + path.chmod(0o700) + + def _lock(self, session_dir: Path): + self._private_directory(session_dir) + return portalocker.Lock(session_dir / ".repository.lock", mode="a+", timeout=10) + + @staticmethod + def _require_existing(session_dir: Path, session_id: str) -> None: + if not (session_dir / MANIFEST).exists(): + raise SessionError(f"Sessione non trovata: {session_id}") + + def _artifact_path(self, session_dir: Path, key: str) -> Path: + if key in _ARTIFACT_FILES: + return session_dir / _ARTIFACT_FILES[key] + if key.startswith("cte_sql:"): + name = key.removeprefix("cte_sql:") + if _SAFE_CTE_NAME.fullmatch(name): + return session_dir / "ctes" / f"{name}.sql" + raise ValueError(f"Unsupported artifact key: {key}") + + def _read_artifacts(self, session_dir: Path) -> dict[str, str]: + artifacts = { + key: (session_dir / filename).read_text() + for filename, key in _ARTIFACT_KEYS.items() + if (session_dir / filename).is_file() + } + ctes = session_dir / "ctes" + if ctes.is_dir(): + artifacts.update( + { + f"cte_sql:{path.stem}": path.read_text() + for path in ctes.glob("*.sql") + if _SAFE_CTE_NAME.fullmatch(path.stem) + } + ) + return artifacts + + def _write_manifest(self, session_dir: Path, manifest: SessionManifest) -> None: + content = yaml.safe_dump( + manifest.model_dump(by_alias=True, mode="json", exclude_defaults=False), + sort_keys=False, + allow_unicode=True, + width=120, + ) + self._write_private_text(session_dir / MANIFEST, content) + + def _write_private_text(self, path: Path, content: str) -> None: + self._private_directory(path.parent) + fd, temporary_name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent) + temporary = Path(temporary_name) + try: + with os.fdopen(fd, "w") as handle: + handle.write(content) + handle.flush() + os.fsync(handle.fileno()) + if os.name != "nt": + temporary.chmod(0o600) + os.replace(temporary, path) + if os.name != "nt": + path.chmod(0o600) + finally: + temporary.unlink(missing_ok=True) diff --git a/harness/tht/session/models.py b/harness/tht/session/models.py index 4d5bcae7..011310a6 100644 --- a/harness/tht/session/models.py +++ b/harness/tht/session/models.py @@ -1,10 +1,17 @@ +import json +import os +import tempfile +import uuid from datetime import datetime from pathlib import Path from typing import Literal, Self +import portalocker import yaml from pydantic import BaseModel, Field, ConfigDict +from tht.decisions import DecisionRecord + # Stub locale di _YamlModel. In the reference implementation questa base vive in # mschema/models.py; qui la si replica perche' SessionManifest ha bisogno di @@ -26,6 +33,74 @@ class _YamlModel(BaseModel): return cls.model_validate(raw) +class PrincipalContext(BaseModel): + """Trusted owner identity supplied by the runtime, never by a session document.""" + + issuer: str + subject: str + display_name: str | None = None + is_admin: bool = False + + model_config = ConfigDict(frozen=True) + + +def _private_directory(path: Path) -> Path: + path.mkdir(parents=True, exist_ok=True) + if os.name != "nt": + path.chmod(0o700) + return path + + +def _write_private_json(path: Path, data: dict[str, str]) -> None: + fd, temporary_name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent) + temporary = Path(temporary_name) + try: + with os.fdopen(fd, "w") as handle: + json.dump(data, handle, sort_keys=True) + handle.write("\n") + handle.flush() + os.fsync(handle.fileno()) + if os.name != "nt": + temporary.chmod(0o600) + os.replace(temporary, path) + if os.name != "nt": + path.chmod(0o600) + finally: + temporary.unlink(missing_ok=True) + + +def local_principal(home: Path | None = None) -> PrincipalContext: + """Load or create the stable UUID identity for a local ThothII home.""" + if home is None: + from tht.config import local_tht_home + + home = local_tht_home() + identity_path = _private_directory(home) / "identity.json" + lock_path = identity_path.with_suffix(".lock") + with portalocker.Lock(lock_path, mode="a+", timeout=10): + if identity_path.exists(): + try: + identity = json.loads(identity_path.read_text()) + principal = PrincipalContext.model_validate(identity) + except (json.JSONDecodeError, OSError, ValueError) as exc: + raise ValueError(f"Invalid local identity: {identity_path}") from exc + if principal.issuer != "local": + raise ValueError(f"Invalid local identity issuer: {identity_path}") + try: + parsed = uuid.UUID(principal.subject, version=4) + except ValueError as exc: + raise ValueError(f"Invalid local identity subject: {identity_path}") from exc + if str(parsed) != principal.subject or parsed.version != 4: + raise ValueError(f"Invalid local identity subject: {identity_path}") + return principal + + principal = PrincipalContext(issuer="local", subject=str(uuid.uuid4())) + _write_private_json( + identity_path, {"issuer": principal.issuer, "subject": principal.subject} + ) + return principal + + class SessionManifest(_YamlModel): id: str created_at: datetime @@ -47,6 +122,15 @@ class SessionManifest(_YamlModel): group: str | None = None +class SessionSnapshot(BaseModel): + """The current persisted session state, excluding the non-persistent chat stream.""" + + principal: PrincipalContext | None = None + manifest: SessionManifest + artifacts: dict[str, str] = Field(default_factory=dict) + decisions: list[DecisionRecord] = Field(default_factory=list) + + class Candidate(BaseModel): kind: Literal["table", "column"] name: str diff --git a/harness/tht/session/postgres_repository.py b/harness/tht/session/postgres_repository.py new file mode 100644 index 00000000..e998f5ea --- /dev/null +++ b/harness/tht/session/postgres_repository.py @@ -0,0 +1,549 @@ +"""PostgreSQL implementation of the durable, user-owned session repository.""" + +from __future__ import annotations + +import hashlib +import json +import re +import uuid +from contextlib import contextmanager +from dataclasses import dataclass +from datetime import UTC, datetime +from importlib.resources import files +from importlib.resources.abc import Traversable +from pathlib import Path +from typing import Iterator, Sequence + +from sqlalchemy import Engine, create_engine, text +from sqlalchemy.engine import URL, make_url +from sqlalchemy.exc import SQLAlchemyError + +from tht.decisions import DecisionInput, DecisionRecord +from tht.session.models import PrincipalContext, SessionManifest, SessionSnapshot +from tht.session.store import SessionError + +MIGRATIONS_DIR = files("tht").joinpath("migrations", "sessions") +_MIGRATION_NAME = re.compile(r"^(?P\d+)_(?P[a-z0-9_]+)\.sql$") +_SAFE_CTE_NAME = re.compile(r"[A-Za-z0-9_-]+\Z") +_ARTIFACT_KEYS = { + "question", + "schema_linking", + "evidence", + "sql_final", + "validation_report", + "retrieval_pack", + "cte_tests", + "cte_plan", + "cte_plan_doc", +} +_LOCK_KEY = 8_420_613_069_444_020_731 +_RUNTIME_ROLE = "thoth_sessions_runtime" + + +class MigrationError(RuntimeError): + """Raised when session schema migration discovery or application is unsafe.""" + + +@dataclass(frozen=True) +class Migration: + version: str + name: str + path: Traversable + checksum: str + + +@dataclass(frozen=True) +class MigrationStatus: + applied: tuple[Migration, ...] + pending: tuple[Migration, ...] + drifted: tuple[Migration, ...] + + +def _migration_source(directory: Traversable | Path | str) -> Traversable: + return Path(directory) if isinstance(directory, (str, Path)) else directory + + +def _discover(directory: Traversable | Path | str = MIGRATIONS_DIR) -> tuple[Migration, ...]: + source = _migration_source(directory) + seen_versions: set[int] = set() + parsed = [] + for path in (item for item in source.iterdir() if item.name.endswith(".sql")): + match = _MIGRATION_NAME.fullmatch(path.name) + if match is None: + raise MigrationError(f"Invalid migration filename: {path.name}") + version = match.group("version") + numeric_version = int(version) + if numeric_version in seen_versions: + raise MigrationError(f"Duplicate migration version: {numeric_version}") + seen_versions.add(numeric_version) + parsed.append((numeric_version, version, match.group("name"), path)) + if not parsed: + raise MigrationError(f"No migrations found in {source}") + return tuple( + Migration(version, name, path, hashlib.sha256(path.read_bytes()).hexdigest()) + for _, version, name, path in sorted(parsed) + ) + + +def _engine(database_url: str) -> Engine: + url = make_url(database_url) + if not url.drivername.startswith("postgresql"): + raise ValueError("Session repository requires a direct PostgreSQL URL") + return create_engine(url) + + +def _applied(connection) -> dict[str, str]: + exists = connection.execute( + text("SELECT pg_catalog.to_regclass('public.tht_session_migrations')") + ).scalar() + if exists is None: + return {} + return dict( + connection.execute(text("SELECT version, checksum FROM public.tht_session_migrations")).all() + ) + + +def _reject_unknown_versions( + migrations: tuple[Migration, ...], applied_checksums: dict[str, str] +) -> None: + local_versions = {migration.version for migration in migrations} + unknown = sorted( + set(applied_checksums) - local_versions, + key=lambda value: (0, int(value)) if value.isdigit() else (1, value), + ) + if unknown: + raise MigrationError("Database migration versions absent from local manifest: " + ", ".join(unknown)) + + +def migration_status( + database_url: str, migrations_dir: Traversable | Path | str = MIGRATIONS_DIR +) -> MigrationStatus: + migrations = _discover(migrations_dir) + engine = _engine(database_url) + try: + with engine.connect() as connection: + connection.exec_driver_sql("SET LOCAL search_path = pg_catalog, pg_temp") + applied_checksums = _applied(connection) + finally: + engine.dispose() + _reject_unknown_versions(migrations, applied_checksums) + return MigrationStatus( + applied=tuple(item for item in migrations if applied_checksums.get(item.version) == item.checksum), + pending=tuple(item for item in migrations if item.version not in applied_checksums), + drifted=tuple( + item + for item in migrations + if item.version in applied_checksums and applied_checksums[item.version] != item.checksum + ), + ) + + +def migrate( + database_url: str, migrations_dir: Traversable | Path | str = MIGRATIONS_DIR +) -> MigrationStatus: + migrations = _discover(migrations_dir) + engine = _engine(database_url) + current: Migration | None = None + try: + with engine.begin() as connection: + connection.exec_driver_sql("SET LOCAL search_path = pg_catalog, pg_temp") + connection.execute(text("SELECT pg_catalog.pg_advisory_xact_lock(:key)"), {"key": _LOCK_KEY}) + connection.exec_driver_sql( + """CREATE TABLE IF NOT EXISTS public.tht_session_migrations ( + version text PRIMARY KEY, + name text NOT NULL, + checksum text NOT NULL, + applied_at timestamptz NOT NULL DEFAULT pg_catalog.now() + )""" + ) + connection.exec_driver_sql("REVOKE ALL ON public.tht_session_migrations FROM PUBLIC") + applied_checksums = _applied(connection) + _reject_unknown_versions(migrations, applied_checksums) + drifted = [ + item + for item in migrations + if item.version in applied_checksums and applied_checksums[item.version] != item.checksum + ] + if drifted: + raise MigrationError("Migration checksum drift: " + ", ".join(item.version for item in drifted)) + for current in migrations: + if current.version in applied_checksums: + continue + connection.exec_driver_sql(current.path.read_text()) + connection.execute( + text( + "INSERT INTO public.tht_session_migrations (version, name, checksum) " + "VALUES (:version, :name, :checksum)" + ), + {"version": current.version, "name": current.name, "checksum": current.checksum}, + ) + except MigrationError: + raise + except SQLAlchemyError as exc: + filename = current.path.name if current is not None else "migration setup" + raise MigrationError(f"Failed to apply {filename}: {type(exc).__name__}") from exc + finally: + engine.dispose() + return migration_status(database_url, migrations_dir) + + +class PostgresSessionRepository: + """Session data stored in private PostgreSQL tables under transaction-local RLS context.""" + + def __init__( + self, + database_url: str, + principal: PrincipalContext, + *, + engine: Engine | None = None, + runtime_role: str = _RUNTIME_ROLE, + ): + self.principal = principal + self._engine = engine or _engine(database_url) + self._owns_engine = engine is None + self._runtime_role = runtime_role + + @classmethod + def from_config(cls, config, principal: PrincipalContext) -> "PostgresSessionRepository": + query = {"sslmode": config.sslmode} + if config.sslrootcert is not None: + query["sslrootcert"] = str(config.sslrootcert) + url = URL.create( + "postgresql+psycopg2", + username=config.user, + password=config.password, + host=config.host, + port=config.port, + database=config.database, + query=query, + ) + return cls(url.render_as_string(hide_password=False), principal) + + def close(self) -> None: + if self._owns_engine: + self._engine.dispose() + self._owns_engine = False + + def create(self, manifest: SessionManifest) -> SessionSnapshot: + self._require_uuid4(manifest.id) + with self._transaction() as connection: + principal_id = self._upsert_principal(connection) + try: + connection.execute( + text( + "INSERT INTO thoth_sessions.sessions (id, principal_id, manifest) " + "VALUES (:id, :principal_id, CAST(:manifest AS jsonb))" + ), + { + "id": manifest.id, + "principal_id": principal_id, + "manifest": json.dumps(manifest.model_dump(by_alias=True, mode="json")), + }, + ) + except SQLAlchemyError as exc: + if "unique" in str(exc).lower(): + raise SessionError(f"Sessione esiste gia': {manifest.id}") from exc + raise + return self.get(manifest.id) + + def get(self, session_id: str) -> SessionSnapshot: + self._require_uuid4(session_id) + with self._transaction() as connection: + row = connection.execute( + text("SELECT manifest FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id} + ).mappings().first() + if row is None: + raise SessionError(f"Sessione non trovata: {session_id}") + artifacts = dict( + connection.execute( + text( + "SELECT artifact_key, content FROM thoth_sessions.session_artifacts " + "WHERE session_id = :id ORDER BY artifact_key" + ), + {"id": session_id}, + ).all() + ) + decisions = [ + DecisionRecord.model_validate(dict(item)) + for item in connection.execute( + text( + "SELECT seq, ts, phase, type, subject, detail, rationale, retracts " + "FROM thoth_sessions.review_decisions WHERE session_id = :id ORDER BY seq" + ), + {"id": session_id}, + ).mappings() + ] + return SessionSnapshot( + principal=self.principal, + manifest=SessionManifest.model_validate(row["manifest"]), + artifacts=artifacts, + decisions=decisions, + ) + + def list(self) -> list[SessionSnapshot]: + with self._transaction() as connection: + ids = [row[0] for row in connection.execute( + text("SELECT id FROM thoth_sessions.sessions ORDER BY created_at DESC") + ).all()] + # psycopg2 materializes PostgreSQL UUID columns as ``uuid.UUID`` objects, + # while the repository boundary intentionally accepts canonical UUIDv4 text. + return [self.get(str(session_id)) for session_id in ids] + + def save_manifest(self, manifest: SessionManifest) -> SessionSnapshot: + self._require_uuid4(manifest.id) + with self._transaction() as connection: + result = connection.execute( + text( + "UPDATE thoth_sessions.sessions " + "SET manifest = CAST(:manifest AS jsonb), updated_at = pg_catalog.now() " + "WHERE id = :id" + ), + {"id": manifest.id, "manifest": json.dumps(manifest.model_dump(by_alias=True, mode="json"))}, + ) + if result.rowcount != 1: + raise SessionError(f"Sessione non trovata: {manifest.id}") + return self.get(manifest.id) + + def read_artifact(self, session_id: str, key: str) -> str | None: + self._require_uuid4(session_id) + self._require_artifact_key(key) + with self._transaction() as connection: + self._require_session(connection, session_id) + return connection.execute( + text( + "SELECT content FROM thoth_sessions.session_artifacts " + "WHERE session_id = :id AND artifact_key = :key" + ), + {"id": session_id, "key": key}, + ).scalar() + + def write_artifact(self, session_id: str, key: str, content: str) -> None: + self._require_uuid4(session_id) + self._require_artifact_key(key) + with self._transaction() as connection: + self._lock_session(connection, session_id) + self._require_session(connection, session_id) + connection.execute( + text( + "INSERT INTO thoth_sessions.session_artifacts (session_id, artifact_key, content) " + "VALUES (:id, :key, :content) " + "ON CONFLICT (session_id, artifact_key) DO UPDATE " + "SET content = EXCLUDED.content, updated_at = pg_catalog.now()" + ), + {"id": session_id, "key": key, "content": content}, + ) + + def delete_artifact(self, session_id: str, key: str) -> None: + self._require_uuid4(session_id) + self._require_artifact_key(key) + with self._transaction() as connection: + self._lock_session(connection, session_id) + self._require_session(connection, session_id) + connection.execute( + text("DELETE FROM thoth_sessions.session_artifacts WHERE session_id = :id AND artifact_key = :key"), + {"id": session_id, "key": key}, + ) + + def append_decisions( + self, session_id: str, decisions: Sequence[DecisionInput | dict] + ) -> list[DecisionRecord]: + self._require_uuid4(session_id) + inputs = [DecisionInput.model_validate(item) for item in decisions] + if not inputs: + return [] + with self._transaction() as connection: + self._lock_session(connection, session_id) + self._require_session(connection, session_id) + from tht.phase import current_phase + + manifest = SessionManifest.model_validate(connection.execute( + text("SELECT manifest FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id} + ).scalar_one()) + ledger = [ + DecisionRecord.model_validate(dict(item)) + for item in connection.execute( + text("SELECT seq, ts, phase, type, subject, detail, rationale, retracts " + "FROM thoth_sessions.review_decisions WHERE session_id = :id ORDER BY seq"), + {"id": session_id}, + ).mappings() + ] + phase = current_phase(SessionSnapshot(manifest=manifest, decisions=ledger)) + next_seq = connection.execute( + text( + "SELECT COALESCE(MAX(seq), 0) + 1 FROM thoth_sessions.review_decisions " + "WHERE session_id = :id" + ), + {"id": session_id}, + ).scalar_one() + now = datetime.now(UTC) + records = [ + DecisionRecord(seq=next_seq + offset, ts=now, phase=phase, **item.model_dump()) + for offset, item in enumerate(inputs) + ] + connection.execute( + text( + "INSERT INTO thoth_sessions.review_decisions " + "(session_id, seq, ts, phase, type, subject, detail, rationale, retracts) " + "VALUES (:session_id, :seq, :ts, :phase, :type, :subject, :detail, :rationale, :retracts)" + ), + [ + {"session_id": session_id, **record.model_dump(mode="python")} + for record in records + ], + ) + return records + + def finalize(self, manifest: SessionManifest, artifacts: dict[str, str]) -> SessionSnapshot: + """Atomically publish DWH-verified artifacts and the final manifest.""" + self._require_uuid4(manifest.id) + for key in artifacts: + self._require_artifact_key(key) + with self._transaction() as connection: + self._lock_session(connection, manifest.id) + self._require_session(connection, manifest.id) + for key, content in artifacts.items(): + connection.execute( + text( + "INSERT INTO thoth_sessions.session_artifacts (session_id, artifact_key, content) " + "VALUES (:id, :key, :content) " + "ON CONFLICT (session_id, artifact_key) DO UPDATE " + "SET content = EXCLUDED.content, updated_at = pg_catalog.now()" + ), + {"id": manifest.id, "key": key, "content": content}, + ) + result = connection.execute( + text( + "UPDATE thoth_sessions.sessions " + "SET manifest = CAST(:manifest AS jsonb), updated_at = pg_catalog.now() " + "WHERE id = :id" + ), + {"id": manifest.id, "manifest": json.dumps(manifest.model_dump(by_alias=True, mode="json"))}, + ) + if result.rowcount != 1: + raise SessionError(f"Sessione non trovata: {manifest.id}") + return self.get(manifest.id) + + def get_preferences(self) -> dict: + with self._transaction() as connection: + principal_id = self._upsert_principal(connection) + preferences = connection.execute( + text( + "SELECT preferences FROM thoth_sessions.principal_preferences " + "WHERE principal_id = :principal_id" + ), + {"principal_id": principal_id}, + ).scalar() + return preferences or {} + + def set_preferences(self, preferences: dict) -> None: + if not isinstance(preferences, dict): + raise TypeError("preferences must be a dictionary") + with self._transaction() as connection: + principal_id = self._upsert_principal(connection) + connection.execute( + text( + "INSERT INTO thoth_sessions.principal_preferences (principal_id, preferences) " + "VALUES (:principal_id, CAST(:preferences AS jsonb)) " + "ON CONFLICT (principal_id) DO UPDATE SET preferences = EXCLUDED.preferences, " + "updated_at = pg_catalog.now()" + ), + {"principal_id": principal_id, "preferences": json.dumps(preferences)}, + ) + + def delete(self, session_id: str) -> None: + self._require_uuid4(session_id) + with self._transaction() as connection: + self._lock_session(connection, session_id) + owner = connection.execute( + text( + "SELECT p.issuer, p.subject FROM thoth_sessions.sessions s " + "JOIN thoth_sessions.principals p ON p.id = s.principal_id WHERE s.id = :id" + ), + {"id": session_id}, + ).mappings().first() + if owner is None: + raise SessionError(f"Sessione non trovata: {session_id}") + connection.execute( + text( + "INSERT INTO thoth_sessions.audit_log " + "(action, session_id, actor_issuer, actor_subject, owner_issuer, owner_subject) " + "VALUES ('session_deleted', :id, :actor_issuer, :actor_subject, :owner_issuer, :owner_subject)" + ), + { + "id": session_id, + "actor_issuer": self.principal.issuer, + "actor_subject": self.principal.subject, + "owner_issuer": owner["issuer"], + "owner_subject": owner["subject"], + }, + ) + connection.execute(text("DELETE FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id}) + + @contextmanager + def _transaction(self) -> Iterator: + try: + with self._engine.begin() as connection: + connection.exec_driver_sql("SET LOCAL search_path = thoth_sessions, pg_catalog, pg_temp") + connection.exec_driver_sql(f"SET LOCAL ROLE {self._runtime_role}") + connection.execute( + text("SELECT pg_catalog.set_config('thoth_sessions.actor_issuer', :value, true)"), + {"value": self.principal.issuer}, + ) + connection.execute( + text("SELECT pg_catalog.set_config('thoth_sessions.actor_subject', :value, true)"), + {"value": self.principal.subject}, + ) + connection.execute( + text("SELECT pg_catalog.set_config('thoth_sessions.is_admin', :value, true)"), + {"value": "true" if self.principal.is_admin else "false"}, + ) + yield connection + except SessionError: + raise + except SQLAlchemyError as exc: + raise SessionError("Session storage unavailable") from exc + + def _upsert_principal(self, connection) -> int: + return connection.execute( + text( + "INSERT INTO thoth_sessions.principals (issuer, subject, display_name) " + "VALUES (:issuer, :subject, :display_name) " + "ON CONFLICT (issuer, subject) DO UPDATE SET display_name = EXCLUDED.display_name, " + "updated_at = pg_catalog.now() RETURNING id" + ), + self.principal.model_dump(include={"issuer", "subject", "display_name"}), + ).scalar_one() + + @staticmethod + def _lock_session(connection, session_id: str) -> None: + connection.execute( + text("SELECT pg_catalog.pg_advisory_xact_lock(pg_catalog.hashtextextended(:id, 0))"), + {"id": session_id}, + ) + + @staticmethod + def _require_session(connection, session_id: str) -> None: + if connection.execute( + text("SELECT 1 FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id} + ).scalar() is None: + raise SessionError(f"Sessione non trovata: {session_id}") + + @staticmethod + def _require_uuid4(session_id: str) -> None: + try: + parsed = uuid.UUID(session_id, version=4) + except ValueError as exc: + raise SessionError(f"ID sessione non UUIDv4: {session_id}") from exc + if str(parsed) != session_id or parsed.version != 4: + raise SessionError(f"ID sessione non UUIDv4: {session_id}") + + @staticmethod + def _require_artifact_key(key: str) -> None: + if key in _ARTIFACT_KEYS: + return + if key.startswith("cte_sql:") and _SAFE_CTE_NAME.fullmatch(key.removeprefix("cte_sql:")): + return + raise ValueError(f"Unsupported artifact key: {key}") + + +__all__ = ["MigrationError", "MigrationStatus", "PostgresSessionRepository", "migrate", "migration_status"] diff --git a/harness/tht/session/repository.py b/harness/tht/session/repository.py new file mode 100644 index 00000000..5357a27a --- /dev/null +++ b/harness/tht/session/repository.py @@ -0,0 +1,87 @@ +"""Storage contract for the durable, user-owned session workflow state.""" + +from __future__ import annotations + +import os +from typing import Protocol, Sequence + +from tht.decisions import DecisionInput, DecisionRecord +from tht.session.models import PrincipalContext, SessionManifest, SessionSnapshot +from tht.session.store import SessionError + + +class SessionRepository(Protocol): + """Persistence boundary shared by local files and the server database adapter.""" + + principal: PrincipalContext + + def create(self, manifest: SessionManifest) -> SessionSnapshot: ... + + def get(self, session_id: str) -> SessionSnapshot: ... + + def list(self) -> list[SessionSnapshot]: ... + + def save_manifest(self, manifest: SessionManifest) -> SessionSnapshot: ... + + def read_artifact(self, session_id: str, key: str) -> str | None: ... + + def write_artifact(self, session_id: str, key: str, content: str) -> None: ... + + def delete_artifact(self, session_id: str, key: str) -> None: ... + + def append_decisions( + self, session_id: str, decisions: Sequence[DecisionInput | dict] + ) -> list[DecisionRecord]: ... + + def finalize(self, manifest: SessionManifest, artifacts: dict[str, str]) -> SessionSnapshot: ... + + def get_preferences(self) -> dict: ... + + def set_preferences(self, preferences: dict) -> None: ... + + def delete(self, session_id: str) -> None: ... + + +def resolve_principal(config) -> PrincipalContext: + """Resolve the only principal source permitted for this CLI invocation. + + The backend injects these values from its authenticated request context before + spawning ``tht``. A server-storage command without them must fail closed; + substituting a workstation identity would cross user ownership boundaries. + """ + if getattr(config, "session_storage", None) is None: + from tht.session.models import local_principal + + return local_principal() + issuer = os.environ.get("THT_PRINCIPAL_ISSUER", "").strip() + subject = os.environ.get("THT_PRINCIPAL_SUBJECT", "").strip() + if not issuer or not subject: + raise SessionError( + "THT_PRINCIPAL_ISSUER e THT_PRINCIPAL_SUBJECT sono obbligatori per session storage server" + ) + display_name = os.environ.get("THT_PRINCIPAL_DISPLAY_NAME", "").strip() or None + is_admin = os.environ.get("THT_PRINCIPAL_IS_ADMIN", "").strip().lower() in {"1", "true"} + return PrincipalContext( + issuer=issuer, subject=subject, display_name=display_name, is_admin=is_admin + ) + + +def build_session_repository( + config, principal: PrincipalContext | None = None, *, home=None +) -> SessionRepository: + """Build the configured private session persistence adapter.""" + principal = principal or resolve_principal(config) + session_storage = getattr(config, "session_storage", None) + if session_storage is not None: + from tht.session.postgres_repository import PostgresSessionRepository + + return PostgresSessionRepository.from_config(session_storage.connection, principal) + + from tht.config import local_tht_home + from tht.session.filesystem_repository import FilesystemSessionRepository + + workspace = getattr(config, "_workspace_id", "default") + return FilesystemSessionRepository( + home or local_tht_home(), workspace, principal, + root=None if home is not None else getattr(config.paths, "sessions", None), + ) diff --git a/harness/tht/session/store.py b/harness/tht/session/store.py index 8eefef2b..01427051 100644 --- a/harness/tht/session/store.py +++ b/harness/tht/session/store.py @@ -1,6 +1,7 @@ import json import os import shutil +import uuid from datetime import UTC, datetime from pathlib import Path @@ -8,7 +9,6 @@ import yake from tht.config import DatabaseConfig from tht.session.models import SessionManifest -from tht.textutil import slugify MANIFEST = "session_manifest.yaml" MAX_SLUG_CHARS = 40 @@ -88,12 +88,13 @@ def render_question_md(question: str, assumptions: list[str] | None = None) -> s def _new_id(question: str, sessions_root: Path, stamp: str) -> str: - base = f"{stamp}-{slugify(question)[:MAX_SLUG_CHARS].rstrip('-')}" - candidate, n = base, 1 - while (sessions_root / candidate).exists(): - n += 1 - candidate = f"{base}-{n}" - return candidate + """Create an opaque UUIDv4 session identifier. + + ``question``, ``sessions_root`` and ``stamp`` remain accepted temporarily so + existing workflow callers do not need to change as the repository boundary + is introduced. + """ + return str(uuid.uuid4()) def create_session( @@ -129,6 +130,35 @@ def create_session( return manifest +def new_session_manifest( + question: str, db: DatabaseConfig, *, provider=None, model=None, thinking=None, name=None +) -> SessionManifest: + """Create an unsaved UUIDv4 manifest for a repository-owned session.""" + now = datetime.now(UTC) + return SessionManifest( + id=str(uuid.uuid4()), created_at=now, question=question, + database=db.database, schema=db.db_schema, author=current_author(), + summary=_summarize(question), updated_at=now, updated_by=current_author(), + provider=provider, model=model, thinking=thinking, name=name, + ) + + +def build_snapshot_documents(snapshot) -> list[dict]: + docs = [{"phase": "—", "key": "question", "title": "Original question", "format": "text", "content": snapshot.manifest.question}] + spec = [ + ("question", "F3", "revised_question", "Revised question", "markdown"), + ("schema_linking", "F4", "schema_linking", "Schema linking", "schema-linking"), + ("sql_final", "F7", "sql", "Final SQL", "sql"), + ("validation_report", "finalize", "validation_report", "Validation report", "markdown"), + ] + for artifact, phase, key, title, fmt in spec: + if artifact in snapshot.artifacts: + docs.append({"phase": phase, "key": key, "title": title, "format": fmt, "content": snapshot.artifacts[artifact]}) + if snapshot.decisions: + docs.append({"phase": "—", "key": "decisions", "title": "Decisions", "format": "decisions", "content": "\n".join(d.model_dump_json() for d in snapshot.decisions) + "\n"}) + return docs + + def touch_manifest( session_id: str, sessions_root: Path, *, updated_by: str | None = None ) -> SessionManifest: @@ -226,6 +256,33 @@ def sync_schema_linking(session_id: str, sessions_root: Path) -> Path: return set_schema_linking(session_id, data, sessions_root) +def sync_schema_linking_snapshot(snapshot) -> str: + """Repository projection of effective F4 decisions into schema_linking JSON.""" + from tht.phase import effective_decisions + + existing = json.loads(snapshot.artifacts.get("schema_linking", "{}")) + latest: dict[str, str] = {} + for decision in effective_decisions(snapshot): + if decision.type in {"table_promoted", "table_excluded", "column_promoted", "column_excluded"}: + latest[decision.subject] = decision.type + candidates, excluded = [], [] + for subject, kind in latest.items(): + entity = "column" if "." in subject else "table" + if kind.endswith("promoted"): + candidates.append({"kind": entity, "name": subject, "decision": "promoted"}) + else: + excluded.append({"kind": entity, "name": subject}) + from tht.session.models import SchemaLinking + + model = SchemaLinking.model_validate({ + "question": existing.get("question") or snapshot.manifest.question, + "candidates": candidates, "excluded": excluded, + "joins": existing.get("joins", []), "open_questions": existing.get("open_questions", []), + "concept_formulas": existing.get("concept_formulas", []), + }) + return json.dumps(model.model_dump(by_alias=True), indent=2, ensure_ascii=False) + + def load_session(session_id: str, sessions_root: Path) -> SessionManifest: path = sessions_root / session_id / MANIFEST if not path.exists(): @@ -297,6 +354,30 @@ def delete_session(session_id: str, sessions_root: Path) -> None: shutil.rmtree(sessions_root / session_id) +def persist_verified_finalization( + repository, + session_id: str, + *, + validation_report: str, + evidence: str, +) -> SessionManifest: + """Publish DWH-verified final artifacts and status at one repository boundary. + + The caller must complete static validation, EXPLAIN and preview before this + function is entered. It intentionally does not touch solved-question + indexing: that derivative is best-effort and happens after the durable commit. + """ + snapshot = repository.get(session_id) + manifest = snapshot.manifest.model_copy(deep=True) + manifest.status = "finalized" + manifest.updated_at = datetime.now(UTC) + manifest.updated_by = current_author() + return repository.finalize( + manifest, + {"validation_report": validation_report, "evidence": evidence}, + ).manifest + + def build_documents(manifest: SessionManifest, session_dir: Path) -> list[dict]: """Ordered, read-only document bundle for the UI panel. Only documents that exist on disk are returned. CTE artifacts (F6) are intentionally excluded (intermediate).""" diff --git a/harness/tht/solved.py b/harness/tht/solved.py index e62910de..4bff7521 100644 --- a/harness/tht/solved.py +++ b/harness/tht/solved.py @@ -84,3 +84,20 @@ def build_solved_record(session_dir, manifest, promoted_tables) -> VectorRecord: sql=sql_file.read_text().strip(), tables=sorted(promoted_tables or set()), ) + + +def build_solved_snapshot(snapshot, promoted_tables) -> VectorRecord: + from tht.memory import question_context + from tht.phase import effective_decisions + + sql = snapshot.artifacts.get("sql_final") + if sql is None: + raise SolvedIndexError("sql_final.sql assente") + decisions = effective_decisions(snapshot) + if not any(d.type == "sql_approved" for d in decisions): + raise SolvedIndexError("decisione sql_approved assente") + return solved_question_record( + session_id=snapshot.manifest.id, + question=question_context(decisions, snapshot.manifest), + sql=sql.strip(), tables=sorted(promoted_tables or set()), + ) diff --git a/harness/tht/taskdoc.py b/harness/tht/taskdoc.py index d5a6f32b..2aeea1e5 100644 --- a/harness/tht/taskdoc.py +++ b/harness/tht/taskdoc.py @@ -15,6 +15,7 @@ from dataclasses import dataclass from pathlib import Path from tht.phase import effective_decisions +from tht.session.models import SessionSnapshot from tht.workflow import load_workflow MAX_BODY_BYTES = 80_000 # ~20k token (target per task document di una fase) @@ -50,7 +51,7 @@ def _slice_schema_linking(raw: str, promoted_tables: list[str] | None) -> str: def generate_task_doc( - session_dir: Path | str, + session_dir: Path | str | SessionSnapshot, phase: int, promoted_tables: list[str] | None = None, ) -> TaskDoc: @@ -62,20 +63,21 @@ def generate_task_doc( - Brief delle decisioni effective (esclude stale post-rollback, esclude ritirate). - Header del task con il numero/nome della fase. """ - session_dir = Path(session_dir) + snapshot = session_dir if isinstance(session_dir, SessionSnapshot) else None + session_dir = None if snapshot is not None else Path(session_dir) parts: list[str] = [] - q = session_dir / "question.md" - if q.exists(): - parts.append("## Domanda\n" + q.read_text()) + question = snapshot.artifacts.get("question") if snapshot else (session_dir / "question.md").read_text() if (session_dir / "question.md").exists() else None + if question: + parts.append("## Domanda\n" + question) - sl = session_dir / "schema_linking.json" - if sl.exists() and phase >= 4: - sliced = _slice_schema_linking(sl.read_text(), promoted_tables) + linking = snapshot.artifacts.get("schema_linking") if snapshot else (session_dir / "schema_linking.json").read_text() if (session_dir / "schema_linking.json").exists() else None + if linking and phase >= 4: + sliced = _slice_schema_linking(linking, promoted_tables) parts.append("## Schema linking (deciso)\n```json\n" + sliced + "\n```") # Brief decisioni effective (D15-aware) - eff = effective_decisions(session_dir) + eff = effective_decisions(snapshot or session_dir) if eff: lines = [f"- {d.type} | {d.subject} | {d.detail}" for d in eff] parts.append("## Decisioni effettive (effective)\n" + "\n".join(lines)) diff --git a/harness/tht/teardown.py b/harness/tht/teardown.py index 0072c549..cbd01ded 100644 --- a/harness/tht/teardown.py +++ b/harness/tht/teardown.py @@ -52,3 +52,31 @@ def teardown_to_phase(session_dir: str | Path, target_phase: int) -> TeardownRep target.unlink() report.deleted_files.append(artifact) return report + + +def teardown_snapshot(repository, snapshot, target_phase: int) -> TeardownReport: + """Repository equivalent of teardown_to_phase, including orphaned CTE blobs.""" + wf = load_workflow() + report = TeardownReport(target_phase=target_phase) + files = { + "question.md": "question", "schema_linking.json": "schema_linking", + "evidence.json": "evidence", "cte_tests.json": "cte_tests", + "sql_final.sql": "sql_final", "validation_report.md": "validation_report", + "retrieval_pack.md": "retrieval_pack", "cte_plan.json": "cte_plan", + "cte_plan_doc.json": "cte_plan_doc", + } + for phase in wf.phases: + if phase.num <= target_phase: + continue + for artifact in phase.artifacts_out: + if artifact.endswith("/"): + for key in list(snapshot.artifacts): + if key.startswith("cte_sql:"): + repository.delete_artifact(snapshot.manifest.id, key) + report.deleted_files.append(key.removeprefix("cte_sql:") + ".sql") + else: + key = files.get(artifact) + if key and key in snapshot.artifacts: + repository.delete_artifact(snapshot.manifest.id, key) + report.deleted_files.append(artifact) + return report