mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Merge branch feat/user-owned-sessions

Browse Source
This commit is contained in:
User
2026-07-16 20:40:51 +02:00
parent 3abe69badc 5cacf70a0d
commit bd7fa6d2c1
85 changed files with 4682 additions and 497 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.superpowers/sdd/task-2-report.md
+147
View File
@@ -28,3 +28,150 @@ git diff --check
The local-vector and preprocess service secret declarations remain for Task 3, which converts
those services to the same bundle helper. Documentation and smoke command migration is reserved
for Task 4.
---
# Task 2 report — PostgreSQL session repository
## Scope delivered
- Added `PostgresSessionRepository`, implementing the Task 1 repository contract with a
direct PostgreSQL SQLAlchemy connection, transaction-local RLS context, UUIDv4 validation,
current artifacts (including `cte_sql:<name>`), append-only decisions, preferences, and
content-free deletion tombstones.
- Added `tht session migrate --database-url URL [--status] --json` and a checksum-protected,
advisory-transaction-locked migration runner.
- Added server session configuration selection. `session_storage.connection` uses direct
PostgreSQL TLS modes `verify-ca` or `verify-full`; it does not use PostgREST.
- Updated packaging and `.gitignore` so session migrations are present in the built wheel.
- Did not alter Task 3 workflow commands, Pi gate code, or backend code.
## TDD evidence
### RED
Command:
```sh
cd harness && .venv/bin/pytest tests/test_postgres_session_repository.py tests/test_session_migrate_cmd.py -q
```
Result before production implementation: `1 failed, 4 errors in 3.89s`.
- Four setup errors were `ModuleNotFoundError: No module named
'tht.session.postgres_repository'`.
- The migration CLI test failed because `tht session migrate` did not exist (`No such command
'migrate'`).
### GREEN
Initial focused suite after implementation: `5 passed in 4.18s`.
Final focused verification:
```sh
cd harness && .venv/bin/pytest \
tests/test_session_repository.py \
tests/test_postgres_session_repository.py \
tests/test_session_migrate_cmd.py \
tests/test_vector_migration_packaging.py -q
```
Result: `12 passed in 5.80s`.
Changed-file lint verification:
```sh
cd harness && .venv/bin/ruff check \
tht/session/postgres_repository.py tht/migrations/sessions tht/config.py \
tht/session/repository.py tht/cli/session_cmd.py \
tests/test_postgres_session_repository.py tests/test_session_migrate_cmd.py \
tests/test_vector_migration_packaging.py
```
Result: `All checks passed!`.
## Migration and role policy choices
`001_schema.sql` creates only private `thoth_sessions` tables:
- `principals` and `principal_preferences`;
- `sessions`, with `session_artifacts` and `review_decisions` cascading on session deletion;
- `audit_log`, which deliberately has no content/detail/metadata column and keeps only action,
session UUID, actor identity, owner identity, and timestamp.
`002_security.sql` creates separate `thoth_sessions_runtime` and
`thoth_sessions_migrator` group roles, explicitly `NOLOGIN NOBYPASSRLS NOSUPERUSER`, revokes
public access, gives the runtime role only the operations required by the adapter, and enables
and forces RLS on every table. Owner/admin policies read only transaction-local settings:
`thoth_sessions.actor_issuer`, `thoth_sessions.actor_subject`, and
`thoth_sessions.is_admin`. The adapter starts every operation in a transaction, switches to the
restricted runtime role, sets those settings with `set_config(..., true)`, and uses advisory
transaction locks for migrations and per-session mutations.
The runtime role remains a `NOLOGIN` group role by design. Deployment must provision a dedicated
non-superuser LOGIN role and grant it membership, for example:
```sql
CREATE ROLE thoth_sessions_app LOGIN NOINHERIT PASSWORD '<secret>';
GRANT thoth_sessions_runtime TO thoth_sessions_app;
```
This avoids embedding an environment-specific login name or credential in versioned SQL. The
new integration test proves that this non-superuser membership path can create and read a
session while the adapter executes as `thoth_sessions_runtime`.
## Security/self-review
- Owner isolation and admin cross-owner reads run against disposable PostgreSQL containers,
not Supabase.
- No table or column includes `embedding`; repository code imports no embedding/vector code;
the regression test writes a session artifact under a monkeypatched embedding sentinel.
- An unauthorized owner receives the same `SessionError` as an absent session, preserving the
future backend's 404 mapping boundary.
- The audit row is inserted before deleting the parent session, so cascades remove all artifact
and decision content while the tombstone survives.
- A security review found and this task fixed the initial `.gitignore` rule that would have
excluded `migrations/sessions/*.sql` from Git/wheels. The wheel test now asserts both session
migration files and checks both the existing vector CLI and the new session CLI.
- The review also highlighted runtime login provisioning. It is covered by a non-superuser
regression test and documented above; concrete credential/role deployment belongs to Task 7.
## Remaining concerns
- Full `harness/.venv/bin/pytest -q` could not complete in this execution environment: the
runner terminated the command after roughly 30 seconds. Captured output reached 44% with no
failures before termination; `pgrep` confirmed no pytest process remained. The Task 2 focused
suites above completed successfully.
- `harness/.venv/bin/ruff check .` currently reports 34 pre-existing violations in unrelated
test files (for example unused imports in `tests/l0/test_db_connection.py` and semicolon style
in `tests/test_phase_effective.py`). The changed-file Ruff command is clean.
- Task 7 must safely provision the dedicated runtime login/membership and inject its TLS
credentials/CA; this task intentionally does not create a deployment-specific LOGIN role or
password.
## Review follow-up — unavailable migration database JSON contract
### RED
Command:
```sh
cd harness && .venv/bin/pytest \
tests/test_session_migrate_cmd.py::test_session_migrate_status_database_failure_is_pristine_json -q
```
Result: `1 failed in 0.46s`. The unreachable direct PostgreSQL URL exited with code 1 but left
stdout empty, so `json.loads(result.stdout)` raised `JSONDecodeError`.
### GREEN
The session migration CLI now catches `SQLAlchemyError` at the same command boundary as its
migration/domain errors and emits only `{"error": ...}` on stdout for `--json`.
```sh
cd harness && .venv/bin/pytest tests/test_session_migrate_cmd.py -q
cd harness && .venv/bin/ruff check tht/cli/session_cmd.py tests/test_session_migrate_cmd.py
```
Result: `2 passed in 3.60s`; Ruff: `All checks passed!`.
.superpowers/sdd/task-3-report.md
+46 -55
View File
@@ -1,65 +1,56 @@
# Task 3 report — reconnect SSE on same-session Resume
# Task 3 report — workflow repository migration
## Status
## RED
Complete. A successful Resume of the currently active session now replaces its existing
`EventSource` connection. Resuming a different session continues to reconnect through the
session ID change only, without a generation-driven second connection.
- `harness/tests/test_session_repository_workflow.py` initially failed at collection:
`persist_verified_finalization` did not exist.
- The new gate test initially failed because `write_cte_sql` and `write_final_sql`
were not registered. Its first run also exposed the worktree-local missing
Node dependency (`typebox`); `npm ci` installed the lockfile dependency.
- After the principal/legacy policy was clarified, the resolver tests initially
failed because `resolve_principal` did not exist.
## Implementation
## GREEN evidence
- `useSessionStream` accepts an optional `generation` argument (default `0`) and includes it
in the stream effect dependencies. A generation change therefore runs the existing cleanup,
closes the old source, and opens the same URL again.
- `AppShell` captures whether the requested Resume ID is already active before its existing
optimistic state updates. It increments the stream generation only after `resumeSession(id)`
succeeds and only for that same-ID case.
- The existing optimistic session switch, phase refresh, and failed-Resume rollback remain
unchanged. A failed POST cannot increment the generation.
- Focused Python regression set: `66 passed`:
`test_session_repository_workflow`, `test_session_repository`, session mutation/list/
documents/schema-linking, CTE plan/next, decision phase gate, and phase requirement tests.
- Gate suite: `127 passed`, including
`session-repository-writes.test.js`.
- Changed-source Ruff checks pass. `git diff --check` passes.
## TDD evidence
## Implemented boundary
- RED command:
`cd frontend && npx vitest run src/stream/useSessionStream.test.tsx src/shell/AppShell.session-mgmt.test.tsx`
- RED result: 2 expected failures and 15 passes. The hook test observed
`first.closed === false`; the AppShell test observed one `FakeEventSource` instead of two
after the second same-ID Resume.
- GREEN focused result: the same command passed 2/2 files and 17/17 tests after the minimal
production wiring.
- Added `resolve_principal`: PostgreSQL session storage requires trusted
`THT_PRINCIPAL_ISSUER` and `THT_PRINCIPAL_SUBJECT`, optional display name, and
strict admin parsing (`1`/`true`). It fails closed and never substitutes a local
identity. Filesystem storage uses `local_principal()`.
- Filesystem repository creates UUIDv4 sessions only and permits safe historical
timestamp IDs (`YYYY-MM-DD-HHMMSS`) for read/mutate compatibility. PostgreSQL
remains UUIDv4 only.
- Phase helpers fold `SessionSnapshot` ledger/artifacts; decision, phase, CTE,
session mutation/list/document paths, retrieval-pack persistence, SQL promotion
lookup, and task-doc/CTE test helpers gained repository/snapshot paths.
- Finalization now publishes report, evidence, and finalized manifest through
`repository.finalize`: one PostgreSQL transaction; filesystem writes artifacts
before the finalized manifest commit marker. Solved-question indexing stays
best-effort after this durable write.
- Added `tht cte save --session --name --file -` and
`tht sql set-final --session --file -`; Pi tools and SKILL.md now use them.
## Full verification
## Outstanding in-scope migration work
- Baseline before edits: `cd frontend && npx vitest run` — 42/42 files and 251/251 tests passed.
- Focused tests: 2/2 files and 17/17 tests passed.
- Full frontend suite: `cd frontend && npx vitest run` — 42/42 files and 253/253 tests passed.
- Typecheck: `cd frontend && npx tsc -b` — exit 0.
- Production build: `cd frontend && npm run build` — exit 0; Vite transformed 4,835 modules
and completed the production bundle.
- `git diff --check` — passed.
Do not treat this task as complete yet. Remaining direct session path consumers are:
The suite and build retained the pre-existing MSW unhandled-request, React ref/`act`, Node type
stripping, and Vite chunk-size warnings. This task introduced no new warning category.
- `harness/tht/cli/memory_cmd.py`: lines 60, 93, 165, 400, 458.
- `harness/tht/cli/sql_cmd.py`: `_session_sql_file` at line 254 remains a legacy
Path-returning bridge for preview/save/export.
- `harness/tht/cli/session_cmd.py:session_dir` remains only as a compatibility
bridge for the out-of-scope datamart command and the still-unmigrated memory/
SQL consumers; workflow mutations in session_cmd do not call it.
## Files
- `frontend/src/stream/useSessionStream.ts`
- `frontend/src/stream/useSessionStream.test.tsx`
- `frontend/src/shell/AppShell.tsx`
- `frontend/src/shell/AppShell.session-mgmt.test.tsx`
- `.superpowers/sdd/task-3-report.md`
## Self-review
- Confirmed the old EventSource is closed before the replacement is retained by React's effect
lifecycle, and the replacement uses the identical session URL.
- Confirmed same-ID detection happens before the optimistic `setActiveSessionId(id)` call.
- Confirmed the generation increments only after a successful Resume POST; the catch/rollback
branch is unchanged.
- Confirmed a different ID leaves the generation unchanged, so the existing session-ID effect
change creates exactly one replacement connection.
- Confirmed the diff is frontend-only apart from this report and contains no backend, Docker,
configuration, or session changes.
## Concerns
None.
The full Python suite has not been conclusively re-run to completion after the
latest changes. An earlier root-directory invocation failed only because a
pre-existing test expects `workflow.yaml` relative to `harness/`. Full gate tests
are green. Full-repo Ruff currently fails on pre-existing test-file lint findings;
changed-source Ruff passes.
.superpowers/sdd/task-5-report.md
+71
View File
@@ -0,0 +1,71 @@
# Task 5 report — backend principal enforcement
## RED
Added backend route/auth tests before implementation. The initial focused run failed in
seven new assertions: `getPrincipal` did not exist, upstream requests still required the
legacy identity header, foreign session/SSE routes were not hidden, admin scope was not
enforced, new sessions had no trusted principal binding, and settings were global.
## GREEN
- Focused backend suite: `66 passed` across auth, sessions, SSE, and settings tests.
- Complete backend Vitest suite: `209 passed` across `22` files.
- `npx tsc --noEmit -p .`, `npm run build`, `git diff --check`, and changed Python
source Ruff all exit successfully.
- Harness targeted repository/local/migration tests and Python bytecode compilation exit
successfully. The new `tht session preferences get|set` commands are registered and
expose the expected Typer help. A direct local CLI preference smoke was not run because
the checked-in local workspace requires unavailable `THT_DB_HOST` configuration.
## Route and child-process coverage
- `GET /me` returns the request `PrincipalContext`; upstream accepts only the portal's
normalized `X-Thoth-*` identity tuple, with the legacy header ignored. Local mode uses
the same stable `THT_HOME`/`~/.thothii/identity.json` UUID contract as the harness.
- All session operations are principal-scoped: list (`mine` and admin-only `all`), show,
create, resume, close, delete, rename, group, archive, unarchive, documents, reviewer
response, steer, SQL preview/export, and SSE. Missing and foreign sessions are 404;
absent upstream identity is 401. SSE is authorized before response headers or hub
subscription, so a rejected request cannot attach to a live stream.
- New/resumed Pi runtimes and every route-spawned `tht` process receive
`THT_PRINCIPAL_ISSUER`, `THT_PRINCIPAL_SUBJECT`, optional display name, and admin flag.
The readiness `tht` child is also principal-bound.
- Settings use asynchronous repository-backed `tht session preferences get|set` in the
production runner, which isolates preferences by principal. The legacy settings file is
retained only as an injected-runner compatibility fallback for existing isolated tests.
- Repository/settings authorization failures map to 503 before model startup. SQL execution
errors remain 500 after authorization, preserving the prior API distinction.
## Self-review and concerns
- Confirmed the Task 4 portal emits lowercase `true`/`false` for the admin header; the
parser accepts that exact normalized form plus the repository's existing `1`/`0`
compatibility form, and rejects all other values.
- The harness principal resolver is the ownership authority; the backend never accepts an
owner supplied in request bodies. Its route guards use a repository-scoped `session show`
before every session resource operation.
- Existing dependency-injected route fakes without `sessionShow` retain a narrow test seam;
production `ThtRunner` always has that method, so deployed requests cannot bypass the
repository authorization check.
## Review follow-up
### RED
Focused regressions initially failed exactly at the three review findings: stale ambient
display names survived into both `tht` and Pi child environments; mutation/document runner
methods dropped the selected workspace; and `expandLocalHome` did not exist.
### GREEN
- Child environments now remove all four `THT_PRINCIPAL_*` keys from their cloned base
environment before applying the exact request principal. Regression tests prove an absent
display name does not inherit a stale ambient value in either child path.
- `setName`, `setGroup`, `archive`, `unarchive`, and `documents` now take and retain an
optional workspace. The rename route regression proves `session show` authorization and
the mutation use the same non-default workspace.
- Local principal paths expand `~`/`~/...`; existing local home and identity file modes are
repaired to POSIX `0700`/`0600` when applicable, with Windows left unchanged.
- Focused suite: `74 passed`; full backend suite: `213 passed` across `22` files, followed by
TypeScript typecheck, production build, and diff check.
.superpowers/sdd/task-6-report.md
+65
View File
@@ -0,0 +1,65 @@
# Task 6 — Frontend identity and administrator UX report
## RED
- Added API tests for the `/me` principal call and `mine`/`all` session-list scopes.
- Added component tests for regular-user scope, admin scope switching, owner labels,
administrator banner, foreign-owner delete confirmation, and foreign-owner archive
confirmation.
- Initial focused run: 7 expected failures (missing `getMe`, missing scope query,
missing owner label/admin controls, and missing foreign-action confirmation).
- The archive-confirmation regression was also run separately before its implementation
and failed because `window.confirm` was not called.
## GREEN
- `npx vitest run src/api/sessions.test.ts src/shell/NavSessions.test.tsx src/shell/AppShell.session-mgmt.test.tsx`
— passed (47 tests before the archive follow-up; the focused archive regression then passed).
- `npm test` — passed: 44 files / 305 tests.
- `npx tsc -b` — passed.
- `npm run build` — passed.
- `git diff --check` — passed.
- `npm run e2e` reached Playwright but could not run: the environment has no Chromium
executable at Playwright's configured cache path. No application test failure was reported.
## Files changed
- `frontend/src/api/types.ts`: typed principal and session scope contracts.
- `frontend/src/api/sessions.ts`: typed `/me` API call; scoped listing defaults to `mine`.
- `frontend/src/shell/AppShell.tsx`: identity query, admin-only session scope selector and
banner, owner-aware destructive action confirmations.
- `frontend/src/shell/NavSessions.tsx`: owner labels in the all-sessions view.
- `frontend/src/api/sessions.test.ts`, `frontend/src/shell/NavSessions.test.tsx`, and
`frontend/src/shell/AppShell.session-mgmt.test.tsx`: contract and UX coverage.
## Self-review
- Regular users remain fail-closed on `mine`; no administrator control renders without
`principal.isAdmin`.
- The all-sessions view includes owner labels (including `Unknown` for legacy records).
- Delete confirmation preserves the pre-existing select-all behavior and adds confirmation
for foreign/unknown owners. Foreign archive now also requires an explicit browser
confirmation; existing Stop & save already has its confirmation dialog.
- A read-only review found no critical, important, or minor issues. The archive guard was
added after that review in response to the requirement to cover every destructive rail
action, and has its own RED/GREEN regression plus the final full verification above.
## Concerns
- E2E remains environment-blocked until the Playwright Chromium browser is installed.
- Existing Vitest runs emit pre-existing MSW unmatched-request and dialog-ref warnings; all
assertions pass and this task does not modify those shared test/UI primitives.
## Review remediation
- A post-commit review correctly identified that matching `displayName` must never establish
ownership. The predicate now skips confirmation only when `session.author` exactly equals
`principal.subject`; all display-name matches and missing authors are conservative
cross-owner actions.
- Added RED/GREEN regressions where two principals share display name `Alice` but have distinct
subjects: both delete (with another session present, so select-all cannot mask the guard) and
archive require confirmation.
- Added `aria-pressed` to the My sessions / All sessions controls and asserts their selected state
before and after switching.
- Remediation verification: focused regressions passed; full frontend Vitest (44 files / 305
tests), `npx tsc -b`, `npm run build`, and `git diff --check` all passed.
.superpowers/sdd/task-7-report.md
+79
View File
@@ -0,0 +1,79 @@
# Task 7 report — deployment contract and user-owned-session cutover
## Scope
Implemented the deployment contract only. No Supabase migration, portal change, live-stack
restart, session archive, or deletion was run.
- `backend/src/config.ts` now makes the session-store deployment mode explicit. `local` is the
default and cannot be publicly exposed. `postgres` requires `AUTH_MODE=upstream`, direct DB
host/name/runtime user, an absolute runtime-password file, `verify-ca` or `verify-full`, and an
absolute CA path.
- `docker-compose.dev.yml` now publishes only loopback ports and explicitly selects local
session storage rooted at `/data/local-home`.
- `deploy/compose.session-server.yaml.example` separates the runtime and one-shot migrator
secrets. The core gets only `session_runtime_password` and the CA; the profile-gated
`session-migrate` service gets only `session_migrator_password` and the CA.
- `deploy/workspaces/server-sessions.yaml.example` binds the runtime repository to the
TLS-verified direct PostgreSQL configuration. The runtime password remains a file reference.
- `docker/cutover-legacy-sessions.sh` archives/checksums exactly three reviewed legacy sessions
and requires an explicit `--delete` rerun before deleting them.
- README, secret guidance, environment examples, and PROJECT_STATE describe the maintenance
sequence, Task 4+5 coordinated rollout, liveness vs storage 503 behavior, and the no-dual-write
rollback rule.
## TDD evidence
RED was established with:
```sh
cd backend && npx vitest run test/config.test.ts
```
The new tests failed because `sessionStorage` did not exist and public/local and unauthenticated
server combinations were accepted. After implementing the minimal configuration contract, the
same focused suite passed (7 tests). Updating the existing upstream-health fixture to supply the
now-required server inputs confirmed that `/health` remains an unauthenticated `200` liveness
endpoint under the valid server contract.
## Verification
```text
cd harness && .venv/bin/pytest -q
826 passed, 5 deselected, 67 warnings in 63.01s
cd backend && npx vitest run && npx tsc --noEmit -p . && npm run build
22 files / 215 tests passed; TypeScript check and production build passed
cd frontend && npx vitest run && npx tsc -b && npm run build
full Vitest suite, TypeScript build, and Vite production build passed
```
The frontend gate retained its pre-existing React-ref/MSW/act warnings and Vite chunk-size warning;
none caused a test or build failure.
Additional static validation passed:
```text
docker compose config --quiet (base plus copied session-server overlay with temporary empty secrets)
bash -n docker/cutover-legacy-sessions.sh
git diff --check
```
## Manual gate remaining
An operator must still choose the three reviewed legacy IDs, materialize real runtime/migrator/CA
secrets, deploy Task 4 and Task 5 together in a maintenance window, apply the one-shot migrator,
and run the documented authenticated smoke. The guarded helper has not been invoked with
`--delete`.
## P1 correction — migrator TLS validation
The original migrator Compose command interpolated `THT_SESSION_DB_SSLMODE` into its URL without
checking it. `docker/session-migrate.sh` now rejects every value except `verify-ca` and
`verify-full` before reading the password file or building that URL; the Compose service invokes
this helper. `docker/session-migrate.test.sh` first established RED because the helper did not
exist, then verified that `prefer` is rejected before `tht` can run and that `verify-full` reaches
a fake `tht` binary with the expected TLS URL. The helper and test pass `bash -n`; the focused
backend config/health suite remains green, and the base-plus-overlay Compose configuration renders
with temporary empty secret files.
PROJECT_STATE.md
+22
View File
@@ -3,6 +3,28 @@
> Starting-point snapshot for new sessions. Last updated: 2026-07-15 (central live log and compact CTE density live).
> Point a fresh session here ("read PROJECT_STATE.md") before substantial work.
## User-owned sessions cutover — prepared, manual gate pending (2026-07-16)
- **Target contract:** the public server runs `AUTH_MODE=upstream` with Task 4 portal identity
forwarding and Task 5 principal enforcement deployed together. Its session source of truth is
direct TLS-verified PostgreSQL `thoth_sessions`; local development remains loopback-only with
filesystem sessions under `THT_HOME`. The core never receives the migrator credential.
- **Deployment material:** copy `deploy/compose.session-server.yaml.example` and
`deploy/workspaces/server-sessions.yaml.example` into reviewed, untracked operator files. The
runtime password, migrator password, and CA are three separate Docker secret mounts; server
startup rejects public/local storage and incomplete server DB/TLS configuration.
- **Readiness behavior:** `/health` remains the unauthenticated process liveness endpoint. Any
route requiring unavailable session/preferences storage returns fixed HTTP 503 before starting
Pi; this is intentional and must not be hidden by changing liveness to a database check.
- **Manual cutover only:** schedule maintenance, drain Pi work, run the one-shot migrator and
require `pending=[]` and `drifted=[]`, then replace core and perform an authenticated storage
smoke. Archive/checksum the three reviewed legacy filesystem session directories before deleting
exactly those three with `docker/cutover-legacy-sessions.sh --delete`; no deletion has been run
from this repository task. Do not import their untrusted ownership.
- **Rollback:** PostgreSQL remains the single source of truth. Revert only to a compatible fixed
release; never re-enable filesystem persistence, restore the archive into production, or
dual-write during rollback.
## Deployment — Docker locale (Profile A, co-located) — LIVE 2026-07-12
ThothII gira in Docker sul server co-locato, **embedded nel portale omics_portal** a `https://aritmolab.policlinicosandonato.it/datamart-builder` (backend invisibile, tutto same-origin via nginx del portale).
README.md
+77
View File
@@ -60,6 +60,11 @@ The frontend depends on the core health check and proxies `/health` and `/api/*`
application health endpoint intentionally checks process readiness only; external dependency
diagnostics are exposed by `tht doctor` and do not prevent the UI from starting.
`docker-compose.dev.yml` is deliberately local: both published ports bind to `127.0.0.1`,
`THT_SESSION_STORAGE=local`, and `THT_HOME=/data/local-home`. Do not set
`THOTH_PUBLIC_EXPOSURE=true` for that profile; the backend rejects that public/local combination
at startup.
Run the end-to-end packaging check with:
```sh
@@ -192,6 +197,78 @@ Compound providers are deliberately unsupported: `amazon-bedrock`, `azure-openai
values. Selecting one fails before Pi starts; ambient AWS, Azure, and Cloudflare credentials are
still scrubbed. Supporting them requires a future dedicated provider-specific configuration.
## User-owned session server cutover
The server profile stores sessions and per-user preferences directly in PostgreSQL schema
`thoth_sessions`; it does not use PostgREST, browser storage, a shared session directory, or a
dual write. Start from [`deploy/compose.session-server.yaml.example`](deploy/compose.session-server.yaml.example)
and copy [`deploy/workspaces/server-sessions.yaml.example`](deploy/workspaces/server-sessions.yaml.example)
to the untracked `deploy/workspaces/server-sessions.yaml` mounted into the core container.
The runtime login needs membership in the no-login database role `thoth_sessions_runtime` only.
The distinct, one-shot migrator login needs migration authority and uses
`thoth_sessions_migrator`; it must never be mounted into `core`. Set the non-secret endpoint and
role fields in the protected deployment environment:
```dotenv
AUTH_MODE=upstream
THOTH_PUBLIC_EXPOSURE=true
THT_SESSION_STORAGE=postgres
THT_SESSION_DB_HOST=sessions-db.internal
THT_SESSION_DB_PORT=5432
THT_SESSION_DB_NAME=thoth
THT_SESSION_RUNTIME_USER=thoth_sessions_app
THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate
THT_SESSION_DB_SSLMODE=verify-full
THT_SESSION_RUNTIME_PASSWORD_SOURCE=/secure/thoth/session-runtime-password
THT_SESSION_MIGRATOR_PASSWORD_SOURCE=/secure/thoth/session-migrator-password
THT_SESSION_CA_SOURCE=/secure/thoth/session-ca.pem
```
The overlay mounts the runtime password at `/run/secrets/session_runtime_password`, the CA at
`/run/secrets/session_ca.pem`, and passes those paths—not their contents—to the server workspace.
It mounts `session_migrator_password` only to `session-migrate`. The backend refuses a server
session store without upstream authentication, direct DB host/name/runtime user/password-file,
`verify-ca` or `verify-full`, and an absolute CA path.
The migrator independently rejects every other TLS mode before reading its password secret or
constructing a database URL.
Perform the cutover in one maintenance window, with the Task 4 portal proxy headers and Task 5
backend principal parser deployed together. Neither change is safe to deploy independently: Task
4 clears the legacy identity header and Task 5 rejects it. Drain/stop active Pi work, enable a
maintenance response at the portal, then run the migrator once and inspect its pristine JSON:
```sh
docker compose -f compose.yaml -f deploy/compose.session-server.yaml \
--profile session-migrate run --rm session-migrate
```
It must report no pending or drifted migrations before starting the replacement core. `/health`
is a liveness probe and remains `200`; any request that needs unavailable repository storage
returns a fixed `503` before a Pi process starts. Verify this with an authenticated request after
the replacement core is healthy, then remove maintenance mode.
Do not import the three legacy server filesystem sessions: they have no trusted owner binding.
During the same maintenance window, archive the exact three reviewed IDs, verify the generated
archive and `.sha256`, then rerun the command with `--delete` to remove only those three source
directories:
```sh
./docker/cutover-legacy-sessions.sh \
/secure/thoth/legacy-sessions /secure/backups/thoth-legacy-sessions-2026-07-16.tar \
SESSION_ID_1 SESSION_ID_2 SESSION_ID_3
# After independent archive review, use a new backup filename:
./docker/cutover-legacy-sessions.sh --delete \
/secure/thoth/legacy-sessions /secure/backups/thoth-legacy-sessions-2026-07-16-delete.tar \
SESSION_ID_1 SESSION_ID_2 SESSION_ID_3
```
The helper refuses to overwrite an existing backup and refuses any count other than three
distinct IDs. Never run it against a live path without the maintenance gate. Roll back application
code only by keeping PostgreSQL as the single source of truth and deploying a compatible fixed
release. Do not restore filesystem persistence, do not re-import the archive, and never dual-write
sessions to database and files.
## Reproducible image verification
Base images use exact tags and immutable multi-platform manifest digests. Dependency update and
backend/src/app.ts
+35 -5
View File
@@ -5,12 +5,14 @@ import { ThtRunner } from "./tht/tht-runner.js";
import { PiProcessManager } from "./pi/pi-process-manager.js";
import { SseHub } from "./sse/sse-hub.js";
import { authPreHandler } from "./auth/auth.js";
import { getPrincipal } from "./auth/auth.js";
import type { PrincipalContext } from "./auth/principal.js";
import { sessionRoutes } from "./routes/sessions.js";
import { sqlRoutes } from "./routes/sql.js";
import { metaRoutes, type ListModelsFn } from "./routes/meta.js";
import { settingsRoutes, effectiveSettings } from "./routes/settings.js";
import { createPiModelLister } from "./pi/list-models.js";
import { loadSettings, type Settings } from "./settings/settings-store.js";
import { loadSettings, saveSettings, type Settings } from "./settings/settings-store.js";
import { ReadinessManager } from "./runtime/readiness-manager.js";
export interface BuildAppDeps {
@@ -18,7 +20,7 @@ export interface BuildAppDeps {
mgr?: PiProcessManager;
spawnFn?: () => any;
listModels?: ListModelsFn;
getSettings?: () => Settings;
getSettings?: (principal?: PrincipalContext) => Settings | Promise<Settings>;
readiness?: ReadinessManager;
hub?: SseHub;
}
@@ -52,7 +54,34 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
"Pi enabled-model configuration warning",
),
});
const getSettings = deps?.getSettings ?? (() => effectiveSettings(config, loadSettings(config)));
const runnerFor = (principal: PrincipalContext): any => {
const candidate = tht as any;
return typeof candidate.withPrincipal === "function" ? candidate.withPrincipal(principal) : candidate;
};
const getSettings = async (principal: PrincipalContext): Promise<Settings> => {
if (deps?.getSettings) return await deps.getSettings(principal);
const runner = runnerFor(principal);
// The real runner persists preferences through the harness repository. The file fallback
// only keeps older isolated route tests and externally injected runners compatible.
if (typeof runner.preferencesGet === "function") {
const stored = await runner.preferencesGet() as Settings;
if (Object.keys(stored).length === 0) {
const seeded = effectiveSettings(config, loadSettings(config));
await runner.preferencesSet(seeded);
return seeded;
}
return effectiveSettings(config, stored);
}
return effectiveSettings(config, loadSettings(config));
};
const saveUserSettings = async (principal: PrincipalContext, settings: Settings): Promise<void> => {
const runner = runnerFor(principal);
if (typeof runner.preferencesSet === "function") {
await runner.preferencesSet(settings);
return;
}
saveSettings(config, settings);
};
const authenticate = authPreHandler(config.authMode);
app.addHook("preHandler", async (req, reply) => {
@@ -61,12 +90,13 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
return authenticate(req, reply);
});
app.get("/health", async () => ({ status: "ok" }));
app.get("/me", async (req) => getPrincipal(req));
sessionRoutes(app, {
mgr, tht: tht as ThtRunner, hub, getSettings, readiness,
});
sqlRoutes(app, { tht: tht as ThtRunner });
sqlRoutes(app, { tht: tht as ThtRunner, getSettings });
metaRoutes(app, { harnessDir: config.harnessDir, listModels });
settingsRoutes(app, { cfg: config, listModels });
settingsRoutes(app, { cfg: config, listModels, getSettings, saveSettings: saveUserSettings });
return app;
}
backend/src/auth/auth.ts
+14 -9
View File
@@ -1,23 +1,28 @@
import type { FastifyRequest, FastifyReply } from "fastify";
import { localPrincipal, type PrincipalContext, upstreamPrincipal } from "./principal.js";
declare module "fastify" {
interface FastifyRequest { principal?: PrincipalContext }
}
export function authPreHandler(mode: "none" | "mock" | "upstream") {
return async (req: FastifyRequest, reply: FastifyReply) => {
if (mode === "none") {
(req as any).user = { id: "dev@local" };
req.principal = localPrincipal();
} else if (mode === "mock") {
(req as any).user = {
id: (req.headers["x-mock-user"] as string) ?? "mock",
};
const subject = typeof req.headers["x-mock-user"] === "string" ? req.headers["x-mock-user"].trim() : "mock";
req.principal = { issuer: "mock", subject: subject || "mock", displayName: subject || "mock", isAdmin: false };
} else {
const id = req.headers["x-authenticated-user"];
if (typeof id !== "string" || id.trim() === "") {
const principal = upstreamPrincipal(req.headers);
if (!principal) {
return reply.code(401).send({ error: "authenticated upstream identity required" });
}
(req as any).user = { id };
req.principal = principal;
}
};
}
export function getUser(req: FastifyRequest): { id: string } {
return (req as any).user ?? { id: "dev@local" };
export function getPrincipal(req: FastifyRequest): PrincipalContext {
if (!req.principal) throw new Error("principal missing after authentication");
return req.principal;
}
backend/src/auth/principal.ts
+90
View File
@@ -0,0 +1,90 @@
import { chmodSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { homedir } from "node:os";
import { join } from "node:path";
import { randomUUID } from "node:crypto";
export interface PrincipalContext {
issuer: string;
subject: string;
displayName?: string;
isAdmin: boolean;
}
const principalEnvKeys = [
"THT_PRINCIPAL_ISSUER", "THT_PRINCIPAL_SUBJECT", "THT_PRINCIPAL_DISPLAY_NAME", "THT_PRINCIPAL_IS_ADMIN",
] as const;
export function clearPrincipalEnvironment(env: NodeJS.ProcessEnv): void {
for (const key of principalEnvKeys) delete env[key];
}
export function expandLocalHome(path: string, home = homedir()): string {
if (path === "~") return home;
if (path.startsWith("~/")) return join(home, path.slice(2));
return path;
}
function harden(path: string, mode: number): void {
if (process.platform === "win32") return;
try { chmodSync(path, mode); } catch { /* best-effort parity with harness local storage */ }
}
const invalid = (value: string) => value.length === 0 || value.length > 512 || /[\u0000-\u001f\u007f]/.test(value);
function required(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const normalized = value.trim();
return invalid(normalized) ? undefined : normalized;
}
function optional(value: unknown): string | undefined {
if (value === undefined) return undefined;
return required(value);
}
export function upstreamPrincipal(headers: Record<string, unknown>): PrincipalContext | undefined {
const issuer = required(headers["x-thoth-principal-issuer"]);
const subject = required(headers["x-thoth-principal-subject"]);
const displayName = optional(headers["x-thoth-principal-display-name"]);
const adminHeader = headers["x-thoth-is-admin"];
if (!issuer || !subject || (headers["x-thoth-principal-display-name"] !== undefined && !displayName)) return undefined;
if (adminHeader !== "0" && adminHeader !== "1" && adminHeader !== "true" && adminHeader !== "false") return undefined;
return { issuer, subject, displayName, isAdmin: adminHeader === "1" || adminHeader === "true" };
}
export function localPrincipal(): PrincipalContext {
const home = expandLocalHome(process.env.THT_HOME ?? join(homedir(), ".thothii"));
const identityPath = join(home, "identity.json");
mkdirSync(home, { recursive: true, mode: 0o700 });
harden(home, 0o700);
try {
const stored = JSON.parse(readFileSync(identityPath, "utf8"));
if (stored?.issuer === "local" && typeof stored.subject === "string" && /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i.test(stored.subject)) {
harden(identityPath, 0o600);
return { issuer: "local", subject: stored.subject, isAdmin: false };
}
throw new Error("invalid local identity");
} catch (error: any) {
if (error?.code !== "ENOENT") throw error;
const principal = { issuer: "local", subject: randomUUID() };
try {
writeFileSync(identityPath, JSON.stringify(principal) + "\n", { mode: 0o600, flag: "wx" });
harden(identityPath, 0o600);
return { ...principal, isAdmin: false };
} catch (writeError: any) {
// Another local request won the identity creation race; always converge on its UUID.
if (writeError?.code === "EEXIST") return localPrincipal();
throw writeError;
}
}
}
export function principalEnvironment(principal: PrincipalContext): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = {
THT_PRINCIPAL_ISSUER: principal.issuer,
THT_PRINCIPAL_SUBJECT: principal.subject,
THT_PRINCIPAL_IS_ADMIN: principal.isAdmin ? "true" : "false",
};
if (principal.displayName) env.THT_PRINCIPAL_DISPLAY_NAME = principal.displayName;
return env;
}
backend/src/config.ts
+43
View File
@@ -3,6 +3,11 @@ import path from "node:path";
export interface AppConfig {
host: string; port: number; harnessDir: string; thtBin: string; piBin: string;
authMode: "none" | "mock" | "upstream";
sessionStorage: {
mode: "local" | "postgres";
host?: string; port?: number; database?: string; runtimeUser?: string;
runtimePasswordFile?: string; sslmode?: "verify-ca" | "verify-full"; sslrootcert?: string;
};
defaults: { provider?: string; model?: string; thinking?: string };
maxPiProcesses: number;
settingsFile: string;
@@ -20,6 +25,43 @@ export function loadConfig(env: Record<string, string | undefined>): AppConfig {
if (env.THOTH_PUBLIC_EXPOSURE === "true" && authMode !== "upstream") {
throw new Error("public exposure requires AUTH_MODE=upstream behind a trusted proxy");
}
const sessionStorageMode = env.THT_SESSION_STORAGE ?? "local";
if (sessionStorageMode !== "local" && sessionStorageMode !== "postgres") {
throw new Error("session storage configuration is invalid");
}
if (sessionStorageMode === "local" && env.THOTH_PUBLIC_EXPOSURE === "true") {
throw new Error("local session storage requires loopback-only deployment");
}
const sessionStorage: AppConfig["sessionStorage"] = { mode: sessionStorageMode };
if (sessionStorageMode === "postgres") {
const host = env.THT_SESSION_DB_HOST;
const database = env.THT_SESSION_DB_NAME;
const runtimeUser = env.THT_SESSION_RUNTIME_USER;
const runtimePasswordFile = env.THT_SESSION_RUNTIME_PASSWORD_FILE;
const sslmode = env.THT_SESSION_DB_SSLMODE;
const sslrootcert = env.THT_SESSION_DB_SSLROOTCERT;
const port = Number(env.THT_SESSION_DB_PORT ?? 5432);
if (
authMode !== "upstream"
|| !host || !database || !runtimeUser
|| !runtimePasswordFile || !path.isAbsolute(runtimePasswordFile)
|| (sslmode !== "verify-ca" && sslmode !== "verify-full")
|| !sslrootcert || !path.isAbsolute(sslrootcert)
|| !Number.isInteger(port) || port < 1 || port > 65535
) {
if (authMode !== "upstream") {
throw new Error("server session storage requires AUTH_MODE=upstream");
}
throw new Error("server session storage configuration is invalid");
}
sessionStorage.host = host;
sessionStorage.port = port;
sessionStorage.database = database;
sessionStorage.runtimeUser = runtimeUser;
sessionStorage.runtimePasswordFile = runtimePasswordFile;
sessionStorage.sslmode = sslmode;
sessionStorage.sslrootcert = sslrootcert;
}
const modelApiKeyFile = env.THT_MODEL_API_KEY_FILE;
if (modelApiKeyFile !== undefined && (
modelApiKeyFile.trim() !== modelApiKeyFile
@@ -48,6 +90,7 @@ export function loadConfig(env: Record<string, string | undefined>): AppConfig {
thtBin: env.THT_BIN ?? "tht",
piBin: env.PI_BIN ?? "pi",
authMode: authMode as AppConfig["authMode"],
sessionStorage,
defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING },
maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4),
settingsFile: env.SETTINGS_FILE ?? "data/settings.json",
backend/src/pi/pi-process-manager.ts
+11 -7
View File
@@ -5,6 +5,7 @@ import { SessionBridge } from "../bridge/session-bridge.js";
import type { ThtRunner } from "../tht/tht-runner.js";
import { buildPiChildEnv, canonicalPiProvider } from "./provider-credentials.js";
import { secretValue } from "../config/secret-bundle.js";
import { clearPrincipalEnvironment, principalEnvironment, type PrincipalContext } from "../auth/principal.js";
export interface SessionRuntime {
rpc: RpcClient;
@@ -19,6 +20,7 @@ export interface RuntimeOptions {
author?: string;
question?: string;
mode?: "new" | "resume";
principal?: PrincipalContext;
}
/** Injectable child-process boundary; callbacks may ignore arguments in simpler tests. */
@@ -31,21 +33,21 @@ type SpawnFn = (
export class PiProcessManager {
private runtimes = new Map<string, SessionRuntime>();
private spawnFn: (
sessionId: string, author: string, provider: string | undefined,
sessionId: string, author: string, provider: string | undefined, principal?: PrincipalContext,
) => ChildProcessWithoutNullStreams;
constructor(private cfg: AppConfig, opts?: { spawnFn?: SpawnFn }) {
if (opts?.spawnFn) {
this.spawnFn = (sessionId, author, provider) =>
this.spawnPi(opts.spawnFn!, sessionId, author, provider);
this.spawnFn = (sessionId, author, provider, principal) =>
this.spawnPi(opts.spawnFn!, sessionId, author, provider, principal);
} else {
this.spawnFn = (sessionId, author, provider) =>
this.spawnPi(nodeSpawn, sessionId, author, provider);
this.spawnFn = (sessionId, author, provider, principal) =>
this.spawnPi(nodeSpawn, sessionId, author, provider, principal);
}
}
private spawnPi(
spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined,
spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined, principal?: PrincipalContext,
): ChildProcessWithoutNullStreams {
const env = buildPiChildEnv({
provider,
@@ -53,6 +55,8 @@ export class PiProcessManager {
credentialFile: this.cfg.modelApiKeyFile,
additions: { THT_SESSION: sessionId, THT_AUTHOR: author },
});
clearPrincipalEnvironment(env);
if (principal) Object.assign(env, principalEnvironment(principal));
// The Thoth gate executes the deterministic `tht` CLI as a Pi tool. Give only
// this managed session process the adapter values already loaded by the core
// entrypoint; the generic provider helper continues to scrub them by default.
@@ -95,7 +99,7 @@ export class PiProcessManager {
}
const author = o.author ?? "dev@local";
const provider = canonicalPiProvider(o.provider ?? this.cfg.defaults.provider);
const child = this.spawnFn(sessionId, author, provider);
const child = this.spawnFn(sessionId, author, provider, o.principal);
let rt: SessionRuntime | undefined;
try {
const rpc = new RpcClient(child);
backend/src/routes/sessions.ts
+172 -41
View File
@@ -3,7 +3,8 @@ import type { PiProcessManager } from "../pi/pi-process-manager.js";
import type { ThtRunner } from "../tht/tht-runner.js";
import type { SseHub } from "../sse/sse-hub.js";
import type { Settings } from "../settings/settings-store.js";
import { getUser } from "../auth/auth.js";
import { getPrincipal } from "../auth/auth.js";
import type { PrincipalContext } from "../auth/principal.js";
import type { ReadinessManager } from "../runtime/readiness-manager.js";
const BOOTSTRAP_FAILURE_MESSAGE =
@@ -25,7 +26,11 @@ function eventCursor(...values: unknown[]): number {
export function sessionRoutes(
app: FastifyInstance,
d: { mgr: PiProcessManager; tht: ThtRunner; hub: SseHub; getSettings: () => Settings; readiness: ReadinessManager },
d: {
mgr: PiProcessManager; tht: ThtRunner; hub: SseHub;
getSettings: (principal: PrincipalContext) => Promise<Settings>;
readiness: ReadinessManager;
},
) {
const lifecycleTails = new Map<string, Promise<void>>();
const boundRuntimes = new Map<
@@ -54,7 +59,34 @@ export function sessionRoutes(
const info = (id: string, text: string, level = "info") =>
d.hub.publish(id, "info", { type: "info", level, text });
const bindRuntime = (id: string, rt: ReturnType<PiProcessManager["createFor"]>) => {
const runnerFor = (principal: PrincipalContext): any => {
const runner = d.tht as any;
return typeof runner.withPrincipal === "function" ? runner.withPrincipal(principal) : runner;
};
const isNotFound = (error: unknown) =>
/not found|non trovata|inesistente|404/i.test(error instanceof Error ? error.message : String(error));
/** RLS makes a foreign session indistinguishable from a missing one. */
const authorize = async (principal: PrincipalContext, id: string, workspace?: string): Promise<any | undefined> => {
try {
const runner = runnerFor(principal);
// Dependency-injected runners in legacy route tests may model only the mutation under
// test. Production ThtRunner always exposes sessionShow; keep that test seam harmless.
if (typeof runner.sessionShow !== "function") return {};
const manifest = await runner.sessionShow(id, workspace);
return manifest ?? undefined;
} catch (error) {
if (isNotFound(error)) return undefined;
throw error;
}
};
const storageFailure = (reply: any) => reply.code(503).send({ error: "session storage is unavailable" });
const bindRuntime = (
id: string, rt: ReturnType<PiProcessManager["createFor"]>, runner: any, workspace?: string,
) => {
const previous = boundRuntimes.get(id);
boundRuntimes.set(id, rt);
try {
@@ -72,7 +104,7 @@ export function sessionRoutes(
// old failure must not touch its manifest.
const bound = boundRuntimes.get(id);
if (bound !== undefined && bound !== rt) return;
await d.tht.failSession(id, d.getSettings().workspace).catch(() => undefined);
await runner.failSession(id, workspace).catch(() => undefined);
}).catch(() => undefined);
}
}
@@ -96,6 +128,8 @@ export function sessionRoutes(
const bootstrap = (
id: string,
rt: ReturnType<PiProcessManager["createFor"]>,
runner: any,
workspace: string | undefined,
configure: Promise<void>,
retrieval: Promise<void> | null,
start: () => void,
@@ -117,7 +151,7 @@ export function sessionRoutes(
if (d.mgr.get(id) !== rt || !d.mgr.teardownIfCurrent(id, rt)) return;
if (!failurePersistenceClaimed.has(rt)) {
failurePersistenceClaimed.add(rt);
await d.tht.failSession(id, d.getSettings().workspace).catch(() => undefined);
await runner.failSession(id, workspace).catch(() => undefined);
}
rt.bridge.emitClientEvent({ type: "info", level: "error", text: BOOTSTRAP_FAILURE_MESSAGE });
rt.bridge.emitClientEvent({ type: "system_event", event: "session_failed" });
@@ -127,62 +161,105 @@ export function sessionRoutes(
})();
};
app.post("/runtime/prewarm", async (_req, reply) => {
const workspace = d.getSettings().workspace ?? "";
void d.readiness.ensure(workspace).catch(() => undefined);
app.post("/runtime/prewarm", async (req, reply) => {
let settings: Settings;
try { settings = await d.getSettings(getPrincipal(req)); } catch { return storageFailure(reply); }
const workspace = settings.workspace ?? "";
void d.readiness.ensure(workspace, getPrincipal(req)).catch(() => undefined);
return reply.code(202).send({ status: "warming" });
});
app.post("/sessions", async (req, reply) => {
const b = req.body as { question: string; name?: string };
const s = d.getSettings();
const ensure = await d.readiness.ensure(s.workspace ?? "");
const principal = getPrincipal(req);
let s: Settings;
try { s = await d.getSettings(principal); } catch { return storageFailure(reply); }
const runner = runnerFor(principal);
const ensure = await d.readiness.ensure(s.workspace ?? "", principal);
if (!ensure.ok) return reply.code(503).send({ error: READINESS_FAILURE_MESSAGE });
// Settings (global) supply workspace/provider/model/thinking. The new-question
// form sends only the question text. `workspace` selects the tht `-c <config>`.
const { id } = await d.tht.sessionNew({
question: b.question,
name: b.name,
workspace: s.workspace,
provider: s.provider,
model: s.model,
thinking: s.thinking,
});
let id: string;
try {
({ id } = await runner.sessionNew({
question: b.question, name: b.name, workspace: s.workspace,
provider: s.provider, model: s.model, thinking: s.thinking,
}));
} catch { return storageFailure(reply); }
const options = {
provider: s.provider,
model: s.model,
thinking: s.thinking,
author: getUser(req).id,
author: principal.displayName ?? principal.subject,
principal,
question: b.question,
};
const rt = d.mgr.createFor(id, options);
bindRuntime(id, rt);
bindRuntime(id, rt, runner, s.workspace);
info(id, "Session created");
bootstrap(
id, rt, d.mgr.configure(rt, options),
d.tht.searchPack(b.question, id, s.workspace),
id, rt, runner, s.workspace, d.mgr.configure(rt, options),
runner.searchPack(b.question, id, s.workspace),
() => d.mgr.start(id, rt, options),
);
return { id };
});
app.get("/sessions", async () => d.tht.sessionList(d.getSettings().workspace));
app.get("/sessions/:id", async (req) => d.tht.sessionShow((req.params as any).id, d.getSettings().workspace));
app.get("/sessions", async (req, reply) => {
const principal = getPrincipal(req);
const scope = (req.query as { scope?: string }).scope ?? "mine";
if (scope !== "mine" && scope !== "all") return reply.code(400).send({ error: "scope must be mine or all" });
if (scope === "all" && !principal.isAdmin) return reply.code(403).send({ error: "admin scope required" });
try {
const settings = await d.getSettings(principal);
// Admin RLS is deliberately disabled for a normal 'mine' listing.
const scopedPrincipal = scope === "mine" ? { ...principal, isAdmin: false } : principal;
return await runnerFor(scopedPrincipal).sessionList(settings.workspace);
} catch { return storageFailure(reply); }
});
app.get("/sessions/:id", async (req, reply) => {
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
const manifest = await authorize(principal, (req.params as any).id, settings.workspace);
return manifest ?? reply.code(404).send({ error: "session not found" });
} catch { return storageFailure(reply); }
});
app.post("/sessions/:id/response", async (req, reply) => {
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
} catch { return storageFailure(reply); }
const rt = d.mgr.get(id);
if (!rt) return reply.code(404).send({ error: "sessione non attiva" });
rt.bridge.respond((req.body as any).ui_response);
return reply.code(204).send();
});
app.post("/sessions/:id/steer", async (req, reply) => {
const rt = d.mgr.get((req.params as any).id);
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
} catch { return storageFailure(reply); }
const rt = d.mgr.get(id);
if (!rt) return reply.code(404).send({ error: "sessione non attiva" });
rt.bridge.steer((req.body as any).text);
return reply.code(204).send();
});
app.post("/sessions/:id/resume", async (req, reply) => {
const id = (req.params as any).id;
const principal = getPrincipal(req);
return withSessionLifecycle(id, async () => {
let settings: Settings;
let manifest: any;
try {
settings = await d.getSettings(principal);
manifest = await authorize(principal, id, settings.workspace);
} catch { return storageFailure(reply); }
if (!manifest) return reply.code(404).send({ error: "session not found" });
const runner = runnerFor(principal);
// This check belongs inside the per-session lock: a preceding cold Resume may have
// installed a running runtime while this request was waiting.
const existing = d.mgr.get(id);
@@ -192,26 +269,25 @@ export function sessionRoutes(
return reply.code(200).send({ id, alreadyActive: true });
}
}
const manifest = (await d.tht.sessionShow(id, d.getSettings().workspace)) as { status?: string; archived?: boolean } | null;
if (manifest?.status === "finalized" || manifest?.archived) {
return reply.code(409).send({ error: "sessione in sola lettura (finalizzata o archiviata)" });
}
const settings = d.getSettings();
const ensure = await d.readiness.ensure(settings.workspace ?? "");
const ensure = await d.readiness.ensure(settings.workspace ?? "", principal);
if (!ensure.ok) return reply.code(503).send({ error: READINESS_FAILURE_MESSAGE });
const saved = manifest as { provider?: string; model?: string; thinking?: string } | null;
const options = {
provider: saved?.provider,
model: saved?.model,
thinking: saved?.thinking ?? settings.thinking,
author: getUser(req).id,
author: principal.displayName ?? principal.subject,
principal,
mode: "resume" as const,
};
// Reopening is validation, not the transport commit point. Keep the old hub intact if
// persistence cannot be reopened.
try {
await d.tht.reopenSession(id, settings.workspace);
await runner.reopenSession(id, settings.workspace);
} catch {
return reply.code(503).send({ error: RESUME_FAILURE_MESSAGE });
}
@@ -233,7 +309,7 @@ export function sessionRoutes(
d.mgr.teardownIfCurrent(id, current);
}
rt = d.mgr.createFor(id, options);
bindRuntime(id, rt);
bindRuntime(id, rt, runner, settings.workspace);
} catch {
// A created-but-unbound runtime is not usable. The old hub remains attached because
// clear() has not happened yet.
@@ -248,28 +324,41 @@ export function sessionRoutes(
// immediately before the first event produced by the new Resume.
d.hub.clear(id);
info(id, "Resuming session");
bootstrap(id, rt, d.mgr.configure(rt, options), null, () => d.mgr.start(id, rt, options));
bootstrap(id, rt, runner, settings.workspace, d.mgr.configure(rt, options), null, () => d.mgr.start(id, rt, options));
return reply.code(200).send({ id, alreadyActive: false });
});
});
app.post("/sessions/:id/close", async (req) => {
app.post("/sessions/:id/close", async (req, reply) => {
const id = (req.params as { id: string }).id;
const principal = getPrincipal(req);
return withSessionLifecycle(id, async () => {
let settings: Settings;
try {
settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
} catch { return storageFailure(reply); }
// Invalidate the live generation before persistence can yield. Otherwise its deferred
// bootstrap may start Pi while Close is already in progress.
const current = d.mgr.get(id);
boundRuntimes.delete(id);
if (current) d.mgr.teardownIfCurrent(id, current);
try {
await d.tht.closeSession(id, d.getSettings().workspace);
await runnerFor(principal).closeSession(id, settings.workspace);
} catch {
return storageFailure(reply);
} finally {
d.hub.clear(id);
}
return { closed: true };
});
});
app.get("/sessions/:id/events", (req, reply) => {
app.get("/sessions/:id/events", async (req, reply) => {
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
} catch { return storageFailure(reply); }
const rt = d.mgr.get(id);
const afterId = eventCursor(
req.headers["last-event-id"],
@@ -303,31 +392,73 @@ export function sessionRoutes(
req.raw.on("close", off);
});
app.post("/sessions/:id/rename", async (req, reply) => {
await d.tht.setName((req.params as any).id, (req.body as any).name);
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
await runnerFor(principal).setName(id, (req.body as any).name, settings.workspace);
} catch { return storageFailure(reply); }
return reply.code(204).send();
});
app.post("/sessions/:id/group", async (req, reply) => {
await d.tht.setGroup((req.params as any).id, (req.body as any).group);
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
await runnerFor(principal).setGroup(id, (req.body as any).group, settings.workspace);
} catch { return storageFailure(reply); }
return reply.code(204).send();
});
app.post("/sessions/:id/archive", async (req, reply) => {
await d.tht.archive((req.params as any).id);
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
await runnerFor(principal).archive(id, settings.workspace);
} catch { return storageFailure(reply); }
return reply.code(204).send();
});
app.post("/sessions/:id/unarchive", async (req, reply) => {
await d.tht.unarchive((req.params as any).id);
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
await runnerFor(principal).unarchive(id, settings.workspace);
} catch { return storageFailure(reply); }
return reply.code(204).send();
});
app.delete("/sessions/:id", async (req, reply) => {
const id = (req.params as any).id;
const principal = getPrincipal(req);
return withSessionLifecycle(id, async () => {
let settings: Settings;
try {
settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
} catch { return storageFailure(reply); }
const current = d.mgr.get(id);
boundRuntimes.delete(id);
if (current) d.mgr.teardownIfCurrent(id, current);
await d.tht.deleteSession(id, d.getSettings().workspace);
try {
await runnerFor(principal).deleteSession(id, settings.workspace);
} catch {
return storageFailure(reply);
}
d.hub.forget(id);
return reply.code(204).send();
});
});
app.get("/sessions/:id/documents", async (req) => d.tht.documents((req.params as any).id));
app.get("/sessions/:id/documents", async (req, reply) => {
const id = (req.params as any).id;
const principal = getPrincipal(req);
try {
const settings = await d.getSettings(principal);
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
return await runnerFor(principal).documents(id, settings.workspace);
} catch { return storageFailure(reply); }
});
}
backend/src/routes/settings.ts
+19 -5
View File
@@ -2,6 +2,8 @@ import type { FastifyInstance } from "fastify";
import type { AppConfig } from "../config.js";
import { loadSettings, saveSettings, type Settings } from "../settings/settings-store.js";
import { listWorkspaces, type ListModelsFn } from "./meta.js";
import { getPrincipal } from "../auth/auth.js";
import type { PrincipalContext } from "../auth/principal.js";
/** Merge stored settings over env/first-workspace defaults. */
export function effectiveSettings(cfg: AppConfig, stored: Settings): Settings {
@@ -16,10 +18,18 @@ export function effectiveSettings(cfg: AppConfig, stored: Settings): Settings {
export function settingsRoutes(
app: FastifyInstance,
deps: { cfg: AppConfig; listModels: ListModelsFn },
deps: {
cfg: AppConfig; listModels: ListModelsFn;
getSettings: (principal: PrincipalContext) => Promise<Settings>;
saveSettings: (principal: PrincipalContext, settings: Settings) => Promise<void>;
},
): void {
app.get("/settings", async () => {
return effectiveSettings(deps.cfg, loadSettings(deps.cfg));
app.get("/settings", async (req, reply) => {
try {
return await deps.getSettings(getPrincipal(req));
} catch {
return reply.code(503).send({ error: "settings storage is unavailable" });
}
});
app.put("/settings", async (req, reply) => {
@@ -46,7 +56,11 @@ export function settingsRoutes(
model: b.model,
thinking: b.thinking,
};
saveSettings(deps.cfg, next);
return effectiveSettings(deps.cfg, next);
try {
await deps.saveSettings(getPrincipal(req), next);
return effectiveSettings(deps.cfg, next);
} catch {
return reply.code(503).send({ error: "settings storage is unavailable" });
}
});
}
backend/src/routes/sql.ts
+47 -9
View File
@@ -1,25 +1,63 @@
import type { FastifyInstance } from "fastify";
import type { ThtRunner } from "../tht/tht-runner.js";
import { getPrincipal } from "../auth/auth.js";
import type { PrincipalContext } from "../auth/principal.js";
import type { Settings } from "../settings/settings-store.js";
export function sqlRoutes(app: FastifyInstance, deps: { tht: ThtRunner }): void {
export function sqlRoutes(app: FastifyInstance, deps: {
tht: ThtRunner; getSettings: (principal: PrincipalContext) => Promise<Settings>;
}): void {
const runnerFor = (principal: PrincipalContext): any => {
const runner = deps.tht as any;
return typeof runner.withPrincipal === "function" ? runner.withPrincipal(principal) : runner;
};
const authorize = async (principal: PrincipalContext, id: string, workspace?: string) => {
try {
const runner = runnerFor(principal);
if (typeof runner.sessionShow !== "function") return {};
return await runner.sessionShow(id, workspace);
}
catch (error) {
if (/not found|non trovata|inesistente|404/i.test(error instanceof Error ? error.message : String(error))) return undefined;
throw error;
}
};
app.post("/sessions/:id/sql/preview", async (req, reply) => {
const id = (req.params as any).id as string;
const { limit, offset } = (req.body as any) ?? {};
let principal: PrincipalContext;
let workspace: string | undefined;
try {
const result = await deps.tht.sqlPreview(id, { limit, offset });
return result;
} catch (err: any) {
return reply.code(500).send({ error: err.message ?? String(err) });
principal = getPrincipal(req);
const settings = await deps.getSettings(principal);
workspace = settings.workspace;
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
} catch {
return reply.code(503).send({ error: "session storage is unavailable" });
}
try {
return await runnerFor(principal).sqlPreview(id, { limit, offset }, workspace);
} catch (error: any) {
return reply.code(500).send({ error: error.message ?? String(error) });
}
});
app.post("/sessions/:id/sql/export", async (req, reply) => {
const id = (req.params as any).id as string;
let principal: PrincipalContext;
let workspace: string | undefined;
try {
const result = await deps.tht.sqlExport(id);
return result;
} catch (err: any) {
return reply.code(500).send({ error: err.message ?? String(err) });
principal = getPrincipal(req);
const settings = await deps.getSettings(principal);
workspace = settings.workspace;
if (!await authorize(principal, id, settings.workspace)) return reply.code(404).send({ error: "session not found" });
} catch {
return reply.code(503).send({ error: "session storage is unavailable" });
}
try {
return await runnerFor(principal).sqlExport(id, workspace);
} catch (error: any) {
return reply.code(500).send({ error: error.message ?? String(error) });
}
});
}
backend/src/runtime/readiness-manager.ts
+12 -8
View File
@@ -1,4 +1,5 @@
import type { OllamaEnsureResult, ThtRunner } from "../tht/tht-runner.js";
import type { PrincipalContext } from "../auth/principal.js";
interface ReadyEntry {
expiresAt: number;
@@ -20,25 +21,28 @@ export class ReadinessManager {
private now: () => number = Date.now,
) {}
ensure(workspace = ""): Promise<OllamaEnsureResult> {
const cached = this.ready.get(workspace);
ensure(workspace = "", principal?: PrincipalContext): Promise<OllamaEnsureResult> {
const key = `${principal?.issuer ?? ""}\0${principal?.subject ?? ""}\0${workspace}`;
const cached = this.ready.get(key);
if (cached && cached.expiresAt > this.now()) return Promise.resolve(cached.result);
if (cached) this.ready.delete(workspace);
if (cached) this.ready.delete(key);
const current = this.inFlight.get(workspace);
const current = this.inFlight.get(key);
if (current) return current;
const pending = this.tht.ollamaEnsure(workspace, this.timeoutSec)
const runner = principal && typeof (this.tht as any).withPrincipal === "function"
? this.tht.withPrincipal(principal) : this.tht;
const pending = runner.ollamaEnsure(workspace, this.timeoutSec)
.then((result) => {
if (result.ok) {
this.ready.set(workspace, { result, expiresAt: this.now() + this.ttlMs });
this.ready.set(key, { result, expiresAt: this.now() + this.ttlMs });
}
return result;
})
.finally(() => {
if (this.inFlight.get(workspace) === pending) this.inFlight.delete(workspace);
if (this.inFlight.get(key) === pending) this.inFlight.delete(key);
});
this.inFlight.set(workspace, pending);
this.inFlight.set(key, pending);
return pending;
}
}
backend/src/tht/tht-runner.ts
+29 -11
View File
@@ -1,6 +1,7 @@
import { spawn } from "node:child_process";
import { existsSync } from "node:fs";
import { join } from "node:path";
import { clearPrincipalEnvironment, principalEnvironment, type PrincipalContext } from "../auth/principal.js";
export interface ThtConfig {
thtBin: string;
@@ -37,7 +38,10 @@ export interface OllamaEnsureResult {
}
export class ThtRunner {
constructor(private cfg: ThtConfig) {}
constructor(private cfg: ThtConfig, private principal?: PrincipalContext) {}
/** Bind one trusted request principal to every child spawned by this runner. */
withPrincipal(principal: PrincipalContext): ThtRunner { return new ThtRunner(this.cfg, principal); }
/**
* Resolve the `-c <config>` args. If `workspace` is given AND a matching
@@ -64,16 +68,25 @@ export class ThtRunner {
return new Promise((resolve) => {
const env: NodeJS.ProcessEnv = { ...process.env };
delete env.THT_DATA_ROOT;
clearPrincipalEnvironment(env);
if (this.cfg.dataRoot !== undefined) env.THT_DATA_ROOT = this.cfg.dataRoot;
if (this.principal) Object.assign(env, principalEnvironment(this.principal));
const ch = spawn(this.cfg.thtBin, this.buildArgv(args, workspace), {
cwd: this.cfg.harnessDir,
env,
});
let stdout = "";
let stderr = "";
let settled = false;
const finish = (result: { code: number; stdout: string; stderr: string }) => {
if (settled) return;
settled = true;
resolve(result);
};
ch.stdout.on("data", (d: Buffer) => (stdout += d));
ch.stderr.on("data", (d: Buffer) => (stderr += d));
ch.on("close", (code) => resolve({ code: code ?? 0, stdout, stderr }));
ch.on("error", (error) => finish({ code: 1, stdout, stderr: stderr || error.message }));
ch.on("close", (code) => finish({ code: code ?? 0, stdout, stderr }));
});
}
@@ -124,7 +137,7 @@ export class ThtRunner {
return this.json<unknown>(["session", "show", id, "--json"], workspace);
}
sqlPreview(id: string, p: { limit?: number; offset?: number }) {
sqlPreview(id: string, p: { limit?: number; offset?: number }, workspace?: string) {
// No positional FILE: the harness resolves sql_final.sql from the session
// via _session_sql_file(cfg, session_id), which respects the workspace path.
const a = ["sql", "preview", "--session", id, "--json"];
@@ -135,11 +148,11 @@ export class ThtRunner {
rows: unknown[][];
execution_ms: number;
truncated: boolean;
}>(a);
}>(a, workspace);
}
async sqlExport(id: string) {
const { code, stdout, stderr } = await this.run(["sql", "export", "--session", id]);
async sqlExport(id: string, workspace?: string) {
const { code, stdout, stderr } = await this.run(["sql", "export", "--session", id], workspace);
if (code !== 0) throw new Error(`tht sql export exit ${code}: ${stderr.trim()}`);
return { path: stdout.trim() };
}
@@ -147,15 +160,20 @@ export class ThtRunner {
closeSession(id: string, workspace?: string) { return this.ok(["session", "close", id], workspace); }
failSession(id: string, workspace?: string) { return this.ok(["session", "fail", id], workspace); }
reopenSession(id: string, workspace?: string) { return this.ok(["session", "reopen", id], workspace); }
setName(id: string, name: string) { return this.ok(["session", "set-name", id, "--name", name]); }
setGroup(id: string, group: string) { return this.ok(["session", "set-group", id, "--group", group]); }
archive(id: string) { return this.ok(["session", "archive", id]); }
unarchive(id: string) { return this.ok(["session", "unarchive", id]); }
setName(id: string, name: string, workspace?: string) { return this.ok(["session", "set-name", id, "--name", name], workspace); }
setGroup(id: string, group: string, workspace?: string) { return this.ok(["session", "set-group", id, "--group", group], workspace); }
archive(id: string, workspace?: string) { return this.ok(["session", "archive", id], workspace); }
unarchive(id: string, workspace?: string) { return this.ok(["session", "unarchive", id], workspace); }
async deleteSession(id: string, workspace?: string) {
const { code, stderr } = await this.run(["session", "delete", id], workspace);
if (code !== 0) throw new Error(`tht session delete exit ${code}: ${stderr.trim()}`);
}
documents(id: string) { return this.json<SessionDocument[]>(["session", "documents", id, "--json"]); }
documents(id: string, workspace?: string) { return this.json<SessionDocument[]>(["session", "documents", id, "--json"], workspace); }
preferencesGet(workspace?: string) { return this.json<Record<string, unknown>>(["session", "preferences", "get", "--json"], workspace); }
async preferencesSet(preferences: Record<string, unknown>, workspace?: string): Promise<void> {
await this.ok(["session", "preferences", "set", "--json", JSON.stringify(preferences)], workspace);
}
async ollamaEnsure(workspace: string, timeoutSec: number): Promise<OllamaEnsureResult> {
const { code, stdout, stderr } = await this.run(
backend/test/auth.test.ts
+59 -12
View File
@@ -1,38 +1,85 @@
import { test, expect } from "vitest";
import Fastify from "fastify";
import { authPreHandler, getUser } from "../src/auth/auth.js";
import { authPreHandler, getPrincipal } from "../src/auth/auth.js";
import { chmodSync, mkdtempSync, rmSync, statSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { expandLocalHome, localPrincipal } from "../src/auth/principal.js";
test("mode none assegna dev@local", async () => {
test("local mode resolves a stable local principal", async () => {
const app = Fastify();
app.addHook("preHandler", authPreHandler("none"));
app.get("/me", async (req) => getUser(req));
expect((await app.inject({ method: "GET", url: "/me" })).json()).toEqual({
id: "dev@local",
app.get("/me", async (req) => getPrincipal(req));
expect((await app.inject({ method: "GET", url: "/me" })).json()).toMatchObject({
issuer: "local",
subject: expect.any(String),
isAdmin: false,
});
});
test("mode mock legge l'header", async () => {
test("mock mode makes a principal from the test header", async () => {
const app = Fastify();
app.addHook("preHandler", authPreHandler("mock"));
app.get("/me", async (req) => getUser(req));
app.get("/me", async (req) => getPrincipal(req));
const res = await app.inject({
method: "GET",
url: "/me",
headers: { "x-mock-user": "alice" },
});
expect(res.json()).toEqual({ id: "alice" });
expect(res.json()).toEqual({ issuer: "mock", subject: "alice", displayName: "alice", isAdmin: false });
});
test("upstream mode requires the authenticated proxy identity header", async () => {
test("upstream mode accepts only normalized proxy principal headers", async () => {
const app = Fastify();
app.addHook("preHandler", authPreHandler("upstream"));
app.get("/me", async (req) => getUser(req));
app.get("/me", async (req) => getPrincipal(req));
expect((await app.inject({ method: "GET", url: "/me" })).statusCode).toBe(401);
const authenticated = await app.inject({
method: "GET",
url: "/me",
headers: { "x-authenticated-user": "alice@example.test" },
headers: {
"x-thoth-principal-issuer": "portal",
"x-thoth-principal-subject": "42",
"x-thoth-principal-display-name": "Alice",
"x-thoth-is-admin": "1",
"x-authenticated-user": "must-not-be-used",
},
});
expect(authenticated.json()).toEqual({
issuer: "portal", subject: "42", displayName: "Alice", isAdmin: true,
});
expect(authenticated.json()).toEqual({ id: "alice@example.test" });
});
test("upstream mode rejects legacy client identity headers without proxy principal fields", async () => {
const app = Fastify();
app.addHook("preHandler", authPreHandler("upstream"));
app.get("/me", async (req) => getPrincipal(req));
for (const headers of [
{ "x-authenticated-user": "mallory" },
{ "x-mock-user": "mallory" },
{ "x-authenticated-user": "mallory", "x-mock-user": "mallory" },
]) {
expect((await app.inject({ method: "GET", url: "/me", headers })).statusCode).toBe(401);
}
});
test("local identity expands tilde homes and restores private POSIX permissions", () => {
expect(expandLocalHome("~/thoth-test", "/home/tester")).toBe("/home/tester/thoth-test");
expect(expandLocalHome("~", "/home/tester")).toBe("/home/tester");
const home = mkdtempSync(join(tmpdir(), "thoth-principal-"));
chmodSync(home, 0o755);
const previous = process.env.THT_HOME;
process.env.THT_HOME = home;
try {
localPrincipal();
if (process.platform !== "win32") {
expect(statSync(home).mode & 0o777).toBe(0o700);
expect(statSync(join(home, "identity.json")).mode & 0o777).toBe(0o600);
}
} finally {
if (previous === undefined) delete process.env.THT_HOME; else process.env.THT_HOME = previous;
rmSync(home, { recursive: true, force: true });
}
});
backend/test/config.test.ts
+51
View File
@@ -44,9 +44,60 @@ test("loadConfig accepts an authenticated upstream trust boundary", () => {
expect(loadConfig({
THOTH_PUBLIC_EXPOSURE: "true",
AUTH_MODE: "upstream",
THT_SESSION_STORAGE: "postgres",
THT_SESSION_DB_HOST: "db.internal",
THT_SESSION_DB_NAME: "thoth",
THT_SESSION_RUNTIME_USER: "thoth_sessions_app",
THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password",
THT_SESSION_DB_SSLMODE: "verify-full",
THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem",
}).authMode).toBe("upstream");
});
test("loadConfig requires direct PostgreSQL TLS inputs for the public server session store", () => {
const env = {
THOTH_PUBLIC_EXPOSURE: "true",
AUTH_MODE: "upstream",
THT_SESSION_STORAGE: "postgres",
THT_SESSION_DB_HOST: "db.internal",
THT_SESSION_DB_NAME: "thoth",
THT_SESSION_RUNTIME_USER: "thoth_sessions_app",
THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password",
THT_SESSION_DB_SSLMODE: "verify-full",
THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem",
};
expect(loadConfig(env).sessionStorage).toMatchObject({
mode: "postgres",
host: "db.internal",
port: 5432,
database: "thoth",
runtimeUser: "thoth_sessions_app",
runtimePasswordFile: "/run/secrets/session_runtime_password",
sslmode: "verify-full",
sslrootcert: "/run/secrets/session_ca.pem",
});
for (const required of [
"THT_SESSION_DB_HOST", "THT_SESSION_DB_NAME", "THT_SESSION_RUNTIME_USER",
"THT_SESSION_RUNTIME_PASSWORD_FILE", "THT_SESSION_DB_SSLMODE", "THT_SESSION_DB_SSLROOTCERT",
]) {
const missing = { ...env, [required]: undefined };
expect(() => loadConfig(missing)).toThrow(/server session storage configuration is invalid/);
}
});
test("loadConfig rejects public local storage and server storage without upstream auth", () => {
expect(() => loadConfig({
THOTH_PUBLIC_EXPOSURE: "true",
AUTH_MODE: "upstream",
THT_SESSION_STORAGE: "local",
})).toThrow(/local session storage requires loopback-only deployment/);
expect(() => loadConfig({
THT_SESSION_STORAGE: "postgres",
AUTH_MODE: "none",
})).toThrow(/server session storage requires AUTH_MODE=upstream/);
});
test("loadConfig accepts only an absolute generic model key file", () => {
expect(loadConfig({ THT_MODEL_API_KEY_FILE: "/run/secrets/model_api_key" }).modelApiKeyFile)
.toBe("/run/secrets/model_api_key");
backend/test/health.test.ts
+10 -1
View File
@@ -15,6 +15,13 @@ test("GET /health remains available to container probes in upstream auth mode",
THT_HARNESS_DIR: "/tmp/h",
AUTH_MODE: "upstream",
THOTH_PUBLIC_EXPOSURE: "true",
THT_SESSION_STORAGE: "postgres",
THT_SESSION_DB_HOST: "db.internal",
THT_SESSION_DB_NAME: "thoth",
THT_SESSION_RUNTIME_USER: "thoth_sessions_app",
THT_SESSION_RUNTIME_PASSWORD_FILE: "/run/secrets/session_runtime_password",
THT_SESSION_DB_SSLMODE: "verify-full",
THT_SESSION_DB_SSLROOTCERT: "/run/secrets/session_ca.pem",
}));
const res = await app.inject({ method: "GET", url: "/health" });
expect(res.statusCode).toBe(200);
@@ -22,7 +29,9 @@ test("GET /health remains available to container probes in upstream auth mode",
});
test("SSE response headers are flushed before the first event", async () => {
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/tmp/h" }));
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "/tmp/h" }), {
thtRunner: { sessionShow: async () => ({ id: "header-probe" }) } as any,
});
await app.listen({ port: 0, host: "127.0.0.1" });
const port = (app.server.address() as { port: number }).port;
const controller = new AbortController();
backend/test/pi-spawn-args.test.ts
+25
View File
@@ -32,3 +32,28 @@ test("production spawnFn launches `pi --mode rpc` with no --approve (pi 0.73 dro
expect(args).not.toContain("--approve");
mgr.teardown("s1");
});
test("Pi child replaces stale principal env and omits absent display names", async () => {
const saved = Object.fromEntries([
"THT_PRINCIPAL_ISSUER", "THT_PRINCIPAL_SUBJECT", "THT_PRINCIPAL_DISPLAY_NAME", "THT_PRINCIPAL_IS_ADMIN",
].map((key) => [key, process.env[key]]));
Object.assign(process.env, {
THT_PRINCIPAL_ISSUER: "stale", THT_PRINCIPAL_SUBJECT: "stale", THT_PRINCIPAL_DISPLAY_NAME: "stale",
THT_PRINCIPAL_IS_ADMIN: "true",
});
try {
(nodeSpawn as any).mockClear();
const mgr = new PiProcessManager(loadConfig({}));
await mgr.spawnFor("s-principal", { principal: { issuer: "portal", subject: "42", isAdmin: false } });
const env = (nodeSpawn as any).mock.calls[0][2].env;
expect(env).toMatchObject({
THT_PRINCIPAL_ISSUER: "portal", THT_PRINCIPAL_SUBJECT: "42", THT_PRINCIPAL_IS_ADMIN: "false",
});
expect(env).not.toHaveProperty("THT_PRINCIPAL_DISPLAY_NAME");
mgr.teardown("s-principal");
} finally {
for (const [key, value] of Object.entries(saved)) {
if (value === undefined) delete process.env[key]; else process.env[key] = value;
}
}
});
backend/test/routes-sessions.test.ts
+163 -2
View File
@@ -28,6 +28,126 @@ function deferred<T = void>() {
return { promise, resolve, reject };
}
const aliceHeaders = {
"x-thoth-principal-issuer": "portal",
"x-thoth-principal-subject": "alice",
"x-thoth-principal-display-name": "Alice",
"x-thoth-is-admin": "0",
};
test("upstream requests without a principal fail before a Pi runtime can be created", async () => {
let created = false;
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
mgr: { createFor: () => { created = true; throw new Error("must not spawn"); } } as any,
thtRunner: {} as any,
});
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
expect(response.statusCode).toBe(401);
expect(created).toBe(false);
});
test("session routes conceal foreign or missing sessions and deny SSE before it subscribes", async () => {
let subscribed = false;
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
withPrincipal: () => ({ sessionShow: async () => null }),
} as any,
hub: { subscribe: () => { subscribed = true; return () => {}; } } as any,
});
const document = await app.inject({ method: "GET", url: "/sessions/foreign/documents", headers: aliceHeaders });
const events = await app.inject({ method: "GET", url: "/sessions/foreign/events", headers: aliceHeaders });
expect(document.statusCode).toBe(404);
expect(events.statusCode).toBe(404);
expect(subscribed).toBe(false);
});
test("session listing permits all scope only to admins", async () => {
const seen: boolean[] = [];
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
withPrincipal: (principal: any) => ({
sessionList: async () => { seen.push(principal.isAdmin); return [{ id: "s1" }]; },
}),
} as any,
});
const regularAll = await app.inject({ method: "GET", url: "/sessions?scope=all", headers: aliceHeaders });
const mine = await app.inject({ method: "GET", url: "/sessions?scope=mine", headers: aliceHeaders });
const adminAll = await app.inject({
method: "GET", url: "/sessions?scope=all",
headers: { ...aliceHeaders, "x-thoth-is-admin": "1" },
});
expect(regularAll.statusCode).toBe(403);
expect(mine.statusCode).toBe(200);
expect(adminAll.statusCode).toBe(200);
expect(seen).toEqual([false, true]);
});
test("A, B, and admin requests preserve owner isolation through session route mutations", async () => {
const owners = new Map([["a", "alice"], ["b", "bob"]]);
const closed: Array<{ id: string; subject: string }> = [];
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
withPrincipal: (principal: any) => ({
sessionList: async () => [...owners]
.filter(([, owner]) => principal.isAdmin || owner === principal.subject)
.map(([id, owner]) => ({ id, author: owner })),
sessionShow: async (id: string) =>
(principal.isAdmin || owners.get(id) === principal.subject) ? { id, status: "open" } : null,
closeSession: async (id: string) => { closed.push({ id, subject: principal.subject }); },
}),
} as any,
mgr: { get: () => undefined } as any,
hub: { clear: () => {} } as any,
getSettings: () => ({ workspace: "w" }) as any,
});
const bobHeaders = { ...aliceHeaders, "x-thoth-principal-subject": "bob" };
const adminHeaders = { ...aliceHeaders, "x-thoth-principal-subject": "admin", "x-thoth-is-admin": "1" };
expect((await app.inject({ method: "GET", url: "/sessions", headers: aliceHeaders })).json())
.toEqual([{ id: "a", author: "alice" }]);
expect((await app.inject({ method: "GET", url: "/sessions", headers: bobHeaders })).json())
.toEqual([{ id: "b", author: "bob" }]);
expect((await app.inject({ method: "GET", url: "/sessions?scope=all", headers: adminHeaders })).json())
.toEqual([{ id: "a", author: "alice" }, { id: "b", author: "bob" }]);
expect((await app.inject({ method: "POST", url: "/sessions/a/close", headers: bobHeaders })).statusCode)
.toBe(404);
expect((await app.inject({ method: "POST", url: "/sessions/a/close", headers: adminHeaders })).statusCode)
.toBe(200);
expect(closed).toEqual([{ id: "a", subject: "admin" }]);
});
test("new sessions are created through the authenticated principal, not a client owner field", async () => {
let principal: any;
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
withPrincipal: (p: any) => {
principal = p;
return { sessionNew: async () => ({ id: "owned" }), searchPack: async () => {} };
},
} as any,
readiness: { ensure: async () => ({ ok: true }) } as any,
mgr: {
createFor: () => ({ bridge: { onClientEvent: () => {} } }),
configure: async () => {}, start: () => {}, get: () => undefined,
} as any,
getSettings: () => ({ workspace: "w" }) as any,
});
const response = await app.inject({
method: "POST", url: "/sessions", headers: aliceHeaders,
payload: { question: "q", owner: "mallory" },
});
expect(response.statusCode).toBe(200);
expect(principal).toMatchObject({ issuer: "portal", subject: "alice" });
});
test("POST /sessions usa i settings (workspace/provider/model/thinking) e crea+avvia", async () => {
const modelKey = path.join(os.tmpdir(), `thoth-model-key-${process.pid}`);
writeFileSync(modelKey, "test-model-key", { mode: 0o600 });
@@ -455,7 +575,9 @@ test("concurrent cold Resume requests serialize and create one runtime", async (
expect(firstResponse.json()).toEqual({ id: "s1", alreadyActive: false });
expect(secondResponse.json()).toEqual({ id: "s1", alreadyActive: true });
expect({ showCalls, readinessCalls, reopenCalls, createCalls, clearCalls }).toEqual({
showCalls: 1,
// Each caller is authorized against repository ownership, including the request which
// finds the runtime already active after waiting on the lifecycle lock.
showCalls: 2,
readinessCalls: 1,
reopenCalls: 1,
createCalls: 1,
@@ -942,7 +1064,7 @@ test("Delete followed by Resume cannot resurrect the deleted session", async ()
await deleteResponse;
const resumed = await resumeResponse;
expect(resumed.statusCode).toBe(500);
expect(resumed.statusCode).toBe(503);
expect(current).toBeUndefined();
expect(createCalls).toBe(0);
});
@@ -1195,6 +1317,27 @@ test("POST /sessions/:id/rename calls setName", async () => {
expect(arg).toEqual({ id: "s1", name: "N" });
});
test("rename authorizes and mutates through the same selected workspace", async () => {
const workspaces: string[] = [];
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
withPrincipal: () => ({
sessionShow: async (_id: string, workspace: string) => { workspaces.push(`show:${workspace}`); return { id: "s1" }; },
setName: async (_id: string, _name: string, workspace: string) => { workspaces.push(`set:${workspace}`); },
}),
} as any,
getSettings: () => ({ workspace: "tenant-a" }) as any,
});
const res = await app.inject({
method: "POST", url: "/sessions/s1/rename", payload: { name: "N" },
headers: {
"x-thoth-principal-issuer": "portal", "x-thoth-principal-subject": "42", "x-thoth-is-admin": "false",
},
});
expect(res.statusCode).toBe(204);
expect(workspaces).toEqual(["show:tenant-a", "set:tenant-a"]);
});
test("POST /sessions/:id/group calls setGroup", async () => {
let arg: any;
const app = mutApp({ setGroup: async (id: string, group: string) => { arg = { id, group }; } });
@@ -1303,6 +1446,24 @@ test("POST /sessions readiness failure returns one fixed public message without
expect(createdCalled).toBe(false);
});
test("POST /sessions returns storage 503 before creating a Pi runtime when session persistence fails", async () => {
let piCreated = false;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
thtRunner: {
ollamaEnsure: async () => ({ ok: true }),
sessionNew: async () => { throw new Error("database unavailable"); },
} as any,
getSettings: () => ({ workspace: "psd" }) as any,
mgr: { createFor: () => { piCreated = true; throw new Error("must not spawn"); } } as any,
});
const response = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
expect(response.statusCode).toBe(503);
expect(response.json()).toEqual({ error: "session storage is unavailable" });
expect(piCreated).toBe(false);
});
test("POST /sessions proceeds when ollamaEnsure succeeds", async () => {
let ensureWs: string | undefined;
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
backend/test/routes-settings.test.ts
+89 -1
View File
@@ -1,5 +1,5 @@
import { test, expect } from "vitest";
import { mkdtempSync, rmSync } from "node:fs";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { buildApp } from "../src/app.js";
@@ -48,6 +48,94 @@ test("PUT /settings persists and GET reads it back", async () => {
}
});
test("settings are isolated by the authenticated repository principal", async () => {
const preferences = new Map<string, any>();
const runner = {
withPrincipal: (principal: any) => ({
preferencesGet: async () => preferences.get(principal.subject) ?? {},
preferencesSet: async (next: any) => { preferences.set(principal.subject, next); },
}),
};
const app = buildApp(loadConfig({ AUTH_MODE: "upstream", THT_HARNESS_DIR: "../harness" }), {
thtRunner: runner as any,
listModels: async () => [],
});
const headers = (subject: string) => ({
"x-thoth-principal-issuer": "portal",
"x-thoth-principal-subject": subject,
"x-thoth-is-admin": "0",
});
await app.inject({
method: "PUT", url: "/settings", headers: headers("alice"),
payload: { workspace: "psd", provider: "zai", model: "glm-5.2", thinking: "high" },
});
const alice = await app.inject({ method: "GET", url: "/settings", headers: headers("alice") });
const bob = await app.inject({ method: "GET", url: "/settings", headers: headers("bob") });
expect(alice.json()).toMatchObject({ workspace: "psd", thinking: "high" });
expect(bob.json()).not.toMatchObject({ workspace: "psd", thinking: "high" });
});
test("GET /settings seeds an empty private profile from complete legacy settings once", async () => {
let preferences: Record<string, unknown> = {};
const writes: Record<string, unknown>[] = [];
const runner = {
withPrincipal: () => ({
preferencesGet: async () => preferences,
preferencesSet: async (next: Record<string, unknown>) => {
writes.push(next);
preferences = next;
},
}),
};
const { app, dir } = appWithTmpSettings({}, { thtRunner: runner as any, listModels: async () => [] });
try {
writeFileSync(join(dir, "settings.json"), JSON.stringify({
workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low",
}));
const first = await app.inject({ method: "GET", url: "/settings" });
const second = await app.inject({ method: "GET", url: "/settings" });
const expected = {
workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low",
};
expect(first.statusCode).toBe(200);
expect(first.json()).toEqual(expected);
expect(second.json()).toEqual(expected);
expect(writes).toEqual([expected]);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
test("GET /settings does not overwrite an existing private profile with legacy settings", async () => {
const privateSettings = {
workspace: "private", provider: "zai", model: "glm-5.2", thinking: "high",
};
const preferencesSet = async () => { throw new Error("must not seed an existing profile"); };
const runner = {
withPrincipal: () => ({
preferencesGet: async () => privateSettings,
preferencesSet,
}),
};
const { app, dir } = appWithTmpSettings({}, { thtRunner: runner as any, listModels: async () => [] });
try {
writeFileSync(join(dir, "settings.json"), JSON.stringify({
workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low",
}));
const response = await app.inject({ method: "GET", url: "/settings" });
expect(response.statusCode).toBe(200);
expect(response.json()).toEqual(privateSettings);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
test("PUT /settings rejects an unknown model when a model list is available", async () => {
const { app, dir } = appWithTmpSettings({}, {
listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }],
backend/test/tht-runner.test.ts
+39 -12
View File
@@ -83,6 +83,31 @@ test("run omits ambient THT_DATA_ROOT when config does not provide one", async (
}
});
test("principal-bound tht child replaces stale principal env and omits an absent display name", async () => {
const saved = Object.fromEntries([
"THT_PRINCIPAL_ISSUER", "THT_PRINCIPAL_SUBJECT", "THT_PRINCIPAL_DISPLAY_NAME", "THT_PRINCIPAL_IS_ADMIN",
].map((key) => [key, process.env[key]]));
Object.assign(process.env, {
THT_PRINCIPAL_ISSUER: "stale-issuer", THT_PRINCIPAL_SUBJECT: "stale-subject",
THT_PRINCIPAL_DISPLAY_NAME: "Stale Name", THT_PRINCIPAL_IS_ADMIN: "true",
});
try {
(spawn as any).mockClear();
const runner = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" })
.withPrincipal({ issuer: "portal", subject: "42", isAdmin: false });
await runner.run(["session", "list", "--json"]);
const env = (spawn as any).mock.calls[0][2].env;
expect(env).toMatchObject({
THT_PRINCIPAL_ISSUER: "portal", THT_PRINCIPAL_SUBJECT: "42", THT_PRINCIPAL_IS_ADMIN: "false",
});
expect(env).not.toHaveProperty("THT_PRINCIPAL_DISPLAY_NAME");
} finally {
for (const [key, value] of Object.entries(saved)) {
if (value === undefined) delete process.env[key]; else process.env[key] = value;
}
}
});
test("run with exit != 0 propagates error with stderr", async () => {
const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" });
r.run = async () => ({ code: 1, stdout: "", stderr: "ERRORE: boom" });
@@ -137,23 +162,25 @@ test("setName builds the right argv", async () => {
const calls: string[][] = [];
const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" });
r.run = async (args) => { calls.push(args); return { code: 0, stdout: "", stderr: "" }; };
await r.setName("sid", "Mio nome");
await r.setName("sid", "Mio nome", "tenant-a");
expect(calls[0]).toEqual(["session", "set-name", "sid", "--name", "Mio nome"]);
});
test("setGroup / archive / unarchive / deleteSession build argv", async () => {
const calls: string[][] = [];
test("mutation and document commands retain their requested workspace", async () => {
const calls: Array<{ args: string[]; workspace?: string }> = [];
const r = new ThtRunner({ thtBin: "tht", harnessDir: "/h", configPath: "config/tht.yaml" });
r.run = async (args) => { calls.push(args); return { code: 0, stdout: "", stderr: "" }; };
await r.setGroup("sid", "G1");
await r.archive("sid");
await r.unarchive("sid");
await r.deleteSession("sid");
r.run = async (args, workspace) => { calls.push({ args, workspace }); return { code: 0, stdout: "[]", stderr: "" }; };
await r.setGroup("sid", "G1", "tenant-a");
await r.archive("sid", "tenant-a");
await r.unarchive("sid", "tenant-a");
await r.documents("sid", "tenant-a");
await r.deleteSession("sid", "tenant-a");
expect(calls).toEqual([
["session", "set-group", "sid", "--group", "G1"],
["session", "archive", "sid"],
["session", "unarchive", "sid"],
["session", "delete", "sid"],
{ args: ["session", "set-group", "sid", "--group", "G1"], workspace: "tenant-a" },
{ args: ["session", "archive", "sid"], workspace: "tenant-a" },
{ args: ["session", "unarchive", "sid"], workspace: "tenant-a" },
{ args: ["session", "documents", "sid", "--json"], workspace: "tenant-a" },
{ args: ["session", "delete", "sid"], workspace: "tenant-a" },
]);
});
deploy/compose.session-server.yaml.example
+52
View File
@@ -0,0 +1,52 @@
# Opt-in server overlay for user-owned PostgreSQL sessions. Copy this file to a
# reviewed local override; it is intentionally not loaded by default Compose.
services:
core:
environment:
AUTH_MODE: upstream
THOTH_PUBLIC_EXPOSURE: "true"
THT_SESSION_STORAGE: postgres
THT_CONFIG: /app/harness/workspaces/server-sessions.yaml
THT_SESSION_DB_HOST: ${THT_SESSION_DB_HOST:?set THT_SESSION_DB_HOST}
THT_SESSION_DB_PORT: ${THT_SESSION_DB_PORT:-5432}
THT_SESSION_DB_NAME: ${THT_SESSION_DB_NAME:?set THT_SESSION_DB_NAME}
THT_SESSION_RUNTIME_USER: ${THT_SESSION_RUNTIME_USER:?set THT_SESSION_RUNTIME_USER}
THT_SESSION_RUNTIME_PASSWORD_FILE: /run/secrets/session_runtime_password
THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full}
THT_SESSION_DB_SSLROOTCERT: /run/secrets/session_ca.pem
secrets:
- source: session_runtime_password
target: session_runtime_password
- source: session_ca
target: session_ca.pem
volumes:
- ./deploy/workspaces:/app/harness/workspaces:ro
# Run manually during the maintenance window. It is not a dependency of core,
# so the application never gains the schema-changing migrator credential.
session-migrate:
image: thothii-core:local
profiles: [session-migrate]
entrypoint: [/app/docker/session-migrate.sh]
environment:
THT_SESSION_DB_HOST: ${THT_SESSION_DB_HOST:?set THT_SESSION_DB_HOST}
THT_SESSION_DB_PORT: ${THT_SESSION_DB_PORT:-5432}
THT_SESSION_DB_NAME: ${THT_SESSION_DB_NAME:?set THT_SESSION_DB_NAME}
THT_SESSION_MIGRATOR_USER: ${THT_SESSION_MIGRATOR_USER:?set THT_SESSION_MIGRATOR_USER}
THT_SESSION_MIGRATOR_PASSWORD_FILE: /run/secrets/session_migrator_password
THT_SESSION_DB_SSLMODE: ${THT_SESSION_DB_SSLMODE:-verify-full}
THT_SESSION_DB_SSLROOTCERT: /run/secrets/session_ca.pem
secrets:
- source: session_migrator_password
target: session_migrator_password
- source: session_ca
target: session_ca.pem
restart: "no"
secrets:
session_runtime_password:
file: ${THT_SESSION_RUNTIME_PASSWORD_SOURCE:?set THT_SESSION_RUNTIME_PASSWORD_SOURCE}
session_migrator_password:
file: ${THT_SESSION_MIGRATOR_PASSWORD_SOURCE:?set THT_SESSION_MIGRATOR_PASSWORD_SOURCE}
session_ca:
file: ${THT_SESSION_CA_SOURCE:?set THT_SESSION_CA_SOURCE}
deploy/env.example
+13
View File
@@ -14,6 +14,19 @@ PI_THINKING=
MAX_PI_PROCESSES=4
AUTH_MODE=none
# User-owned session storage. Keep local for the loopback-only development stack.
# The server-session overlay requires every THT_SESSION_* value below.
THT_SESSION_STORAGE=local
THT_SESSION_DB_HOST=
THT_SESSION_DB_PORT=5432
THT_SESSION_DB_NAME=
THT_SESSION_RUNTIME_USER=
THT_SESSION_RUNTIME_PASSWORD_SOURCE=
THT_SESSION_MIGRATOR_USER=
THT_SESSION_MIGRATOR_PASSWORD_SOURCE=
THT_SESSION_DB_SSLMODE=verify-full
THT_SESSION_CA_SOURCE=
THT_DB_NAME=
THT_DWH_REST_URL=
THT_VEC_REST_URL=
deploy/secrets/README.md
+16
View File
@@ -43,3 +43,19 @@ password into `THT_VECTOR_BOOTSTRAP_PASSWORD` in the bundle before restarting
Hosted Pi providers must use a single provider key. Compound providers (Bedrock, Azure OpenAI
Responses, Cloudflare Workers AI/Gateway) fail closed until a provider-specific credential
adapter is implemented.
## User-owned session database secrets
The server-session overlay deliberately does **not** add session database credentials to the
shared bundle. Materialize three distinct Docker secrets from protected host or secret-manager
files: `session_runtime_password`, `session_migrator_password`, and `session_ca.pem`. Their host
source paths are respectively `THT_SESSION_RUNTIME_PASSWORD_SOURCE`,
`THT_SESSION_MIGRATOR_PASSWORD_SOURCE`, and `THT_SESSION_CA_SOURCE`; all must be absolute paths
outside the repository. The runtime password is mounted only into `core`; the migrator password
is mounted only into the one-shot `session-migrate` service. Do not reuse either login for the
other role.
`session_ca.pem` is a PEM file rather than a bundle value because the bundle rejects whitespace.
The server workspace receives only its mount path through `THT_SESSION_DB_SSLROOTCERT`; it uses
`THT_SESSION_DB_SSLMODE=verify-ca` or, normally, `verify-full`. TLS disable/prefer/require modes
are unsupported for session storage.
deploy/workspaces/server-sessions.yaml.example
+36
View File
@@ -0,0 +1,36 @@
# Copy to deploy/workspaces/server-sessions.yaml for the user-owned-session server profile.
# The runtime login is intentionally separate from the one-shot migrator login.
language: en
dwh:
type: thoth_rest
database:
database: ${THT_DB_NAME}
schema: datawarehouse
endpoint:
base_url: ${THT_DWH_REST_URL}
api_key: ${THT_DWH_API_KEY}
ssl_ca: ${THT_SSL_CA}
session_storage:
type: postgres_direct
connection:
host: ${THT_SESSION_DB_HOST}
port: ${THT_SESSION_DB_PORT}
database: ${THT_SESSION_DB_NAME}
schema: thoth_sessions
user: ${THT_SESSION_RUNTIME_USER}
password_file: ${THT_SESSION_RUNTIME_PASSWORD_FILE}
sslmode: ${THT_SESSION_DB_SSLMODE}
sslrootcert: ${THT_SESSION_DB_SSLROOTCERT}
roots:
artifacts: artifacts
indexes: indexes
sessions: sessions
embeddings:
base_url: ${THT_OLLAMA_URL}
model: nomic-embed-text-v2-moe
dim: 768
batch_size: 32
docker-compose.dev.yml
+4 -2
View File
@@ -18,6 +18,8 @@ services:
THT_BIN: /opt/venv/bin/tht
PI_BIN: pi
AUTH_MODE: ${AUTH_MODE:-none}
THT_SESSION_STORAGE: local
THT_HOME: /data/local-home
SETTINGS_FILE: /data/settings/settings.json
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
extra_hosts:
@@ -27,7 +29,7 @@ services:
- /home/chirone/thothii-data/pi-config:/home/thoth/.pi
- /home/chirone/chirone/etl/docs/evidence:/data/evidence:ro
ports:
- "8787:8787"
- "127.0.0.1:8787:8787"
restart: "no"
networks: [thothii-net]
@@ -40,7 +42,7 @@ services:
VITE_BACKEND_URL: /api
image: thothii-frontend:local
ports:
- "8090:8080"
- "127.0.0.1:8090:8080"
depends_on:
- core
restart: "no"
docker/core.Dockerfile
+2 -2
View File
@@ -63,8 +63,8 @@ ENV PATH="/opt/venv/bin:/usr/local/bin:$PATH" \
PI_BIN=pi \
HOME=/home/thoth
COPY docker/core-entrypoint.sh docker/ensure-pi-trust.mjs /app/docker/
RUN chmod +x /app/docker/core-entrypoint.sh
COPY docker/core-entrypoint.sh docker/session-migrate.sh docker/ensure-pi-trust.mjs /app/docker/
RUN chmod +x /app/docker/core-entrypoint.sh /app/docker/session-migrate.sh
WORKDIR /app/backend
USER thoth
docker/cutover-legacy-sessions.sh
Executable
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Archive exactly three legacy filesystem sessions before optionally deleting them.
# This helper is deliberately inert unless --delete is supplied after archive verification.
set -euo pipefail
usage() {
echo "usage: $0 [--delete] SOURCE_SESSIONS_DIR BACKUP.tar SESSION_ID SESSION_ID SESSION_ID" >&2
exit 2
}
delete_after_backup=false
if [[ ${1:-} == "--delete" ]]; then
delete_after_backup=true
shift
fi
[[ $# -eq 5 ]] || usage
source_dir=$1
backup=$2
shift 2
ids=("$@")
[[ -d $source_dir ]] || { echo "legacy session root is not a directory" >&2; exit 1; }
[[ ! -e $backup ]] || { echo "backup already exists; refusing to overwrite it" >&2; exit 1; }
[[ ${ids[0]} != "${ids[1]}" && ${ids[0]} != "${ids[2]}" && ${ids[1]} != "${ids[2]}" ]] \
|| { echo "exactly three distinct legacy session IDs are required" >&2; exit 1; }
paths=()
for id in "${ids[@]}"; do
[[ $id =~ ^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$ ]] \
|| { echo "invalid legacy session ID: $id" >&2; exit 1; }
[[ -f "$source_dir/$id/session_manifest.yaml" ]] \
|| { echo "legacy session manifest is missing: $id" >&2; exit 1; }
paths+=("$id")
done
tar --create --file "$backup" --directory "$source_dir" -- "${paths[@]}"
tar --list --file "$backup" >/dev/null
sha256sum "$backup" >"$backup.sha256"
echo "verified backup: $backup"
echo "checksum: $backup.sha256"
if ! $delete_after_backup; then
echo "no legacy sessions deleted; review the archive, then rerun with --delete during maintenance" >&2
exit 0
fi
for id in "${ids[@]}"; do
rm -rf -- "$source_dir/$id"
done
echo "deleted exactly three archived legacy sessions"
docker/session-migrate.sh
Executable
+40
View File
@@ -0,0 +1,40 @@
#!/bin/sh
# One-shot schema migration only. Validate TLS before reading a secret or composing a URL.
set -eu
case "${THT_SESSION_DB_SSLMODE:-}" in
verify-ca|verify-full) ;;
*)
echo "THT_SESSION_DB_SSLMODE must be verify-ca or verify-full" >&2
exit 2
;;
esac
require_env() {
eval "value=\${$1:-}"
if [ -z "$value" ]; then
echo "missing required session migrator configuration" >&2
exit 2
fi
}
require_env THT_SESSION_DB_HOST
require_env THT_SESSION_DB_PORT
require_env THT_SESSION_DB_NAME
require_env THT_SESSION_MIGRATOR_USER
require_env THT_SESSION_MIGRATOR_PASSWORD_FILE
require_env THT_SESSION_DB_SSLROOTCERT
if [ ! -r "$THT_SESSION_MIGRATOR_PASSWORD_FILE" ]; then
echo "session migrator credential is unavailable" >&2
exit 2
fi
export PGPASSWORD="$(cat "$THT_SESSION_MIGRATOR_PASSWORD_FILE")"
if [ -z "$PGPASSWORD" ]; then
echo "session migrator credential is unavailable" >&2
exit 2
fi
exec "${THT_BIN:-/opt/venv/bin/tht}" session migrate --database-url \
"postgresql+psycopg2://${THT_SESSION_MIGRATOR_USER}@${THT_SESSION_DB_HOST}:${THT_SESSION_DB_PORT}/${THT_SESSION_DB_NAME}?sslmode=${THT_SESSION_DB_SSLMODE}&sslrootcert=${THT_SESSION_DB_SSLROOTCERT}" \
--json
docker/session-migrate.test.sh
Executable
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
set -euo pipefail
root=$(cd "$(dirname "$0")/.." && pwd)
helper="$root/docker/session-migrate.sh"
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT HUP INT TERM
if THT_SESSION_DB_SSLMODE=prefer sh "$helper" >"$tmp/invalid.out" 2>&1; then
echo "session migrator accepted an unverified TLS mode" >&2
exit 1
fi
grep -qx 'THT_SESSION_DB_SSLMODE must be verify-ca or verify-full' "$tmp/invalid.out"
printf '%s' password >"$tmp/password"
cat >"$tmp/tht" <<'EOF'
#!/bin/sh
printf '%s\n' "$@" >"$ARGS_FILE"
EOF
chmod +x "$tmp/tht"
ARGS_FILE="$tmp/args" \
THT_BIN="$tmp/tht" \
THT_SESSION_DB_HOST=sessions-db.internal \
THT_SESSION_DB_PORT=5432 \
THT_SESSION_DB_NAME=thoth \
THT_SESSION_MIGRATOR_USER=thoth_sessions_migrate \
THT_SESSION_MIGRATOR_PASSWORD_FILE="$tmp/password" \
THT_SESSION_DB_SSLMODE=verify-full \
THT_SESSION_DB_SSLROOTCERT=/run/secrets/session_ca.pem \
sh "$helper"
grep -qx 'session' "$tmp/args"
grep -qx 'migrate' "$tmp/args"
grep -qx -- '--database-url' "$tmp/args"
grep -Fxq 'postgresql+psycopg2://thoth_sessions_migrate@sessions-db.internal:5432/thoth?sslmode=verify-full&sslrootcert=/run/secrets/session_ca.pem' "$tmp/args"
echo "session migrator TLS contract passed."
docs/superpowers/plans/2026-07-16-user-owned-session-storage.md
+130
View File
@@ -0,0 +1,130 @@
# User-owned sessions implementation plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Associate every ThothII session and preference with a stable logged-in or local OS principal.
**Architecture:** The harness owns a repository contract. `FilesystemSessionRepository` stores readable phase documents under the local user home; `PostgresSessionRepository` stores the same logical snapshot in a private Supabase schema. The backend resolves a trusted principal before every action and passes it through to `tht`; the portal proxy supplies that identity.
**Tech Stack:** Python 3.12, Typer, SQLAlchemy/psycopg2, PostgreSQL/Supabase RLS, Fastify/TypeScript, React/Vitest, Django/nginx.
## Global Constraints
- Server storage is PostgreSQL wire protocol with TLS verification, schema `thoth_sessions`; never PostgREST or browser DB access.
- Server identity is `(issuer, Django request.user.pk)`; clients never choose session owner.
- Admin authorization uses Authentik group `authentik Admins`; unauthorized and missing session both return HTTP 404.
- Server has no persistent session filesystem fallback or dual write; DB failure is HTTP 503 before Pi starts.
- Local mode uses `~/.thothii` or `THT_HOME`, runs loopback-only, and creates a UUID identity in `identity.json`.
- Persist current artifacts and append-only decisions only; no chat transcript, artifact revisions, session embeddings, or content in audit logs.
- New session IDs are UUIDv4. Existing `solved_question` behavior is unchanged.
- Follow TDD: each behavior test must fail before its implementation is written.
---
### Task 1: Session repository contract and local implementation
**Files:**
- Create: `harness/tht/session/repository.py`, `harness/tht/session/filesystem_repository.py`
- Modify: `harness/tht/config.py`, `harness/tht/session/store.py`, `harness/tht/session/models.py`, `harness/pyproject.toml`
- Test: `harness/tests/test_session_repository.py`, `harness/tests/test_local_identity.py`
**Interfaces:** Produce `PrincipalContext`, `SessionSnapshot`, `SessionRepository`, and `build_session_repository(config, principal)`. The filesystem adapter must preserve the present documents under `<THT_HOME>/workspaces/<workspace>/sessions/<uuid4>` and read/write artifact keys, decisions, manifest fields, and per-principal preferences.
- [ ] Write failing tests proving UUIDv4 creation, principal-scoped local roots, readable phase artifacts, decision append, `THT_HOME` override, and an identity UUID stable across restarts.
- [ ] Run `cd harness && .venv/bin/pytest tests/test_session_repository.py tests/test_local_identity.py -q`; confirm failure because repository and identity interfaces do not exist.
- [ ] Implement the smallest repository contract and filesystem adapter. Use `portalocker` for mutations and private local directory/file permissions where supported.
- [ ] Run the focused tests and `cd harness && .venv/bin/pytest -q && .venv/bin/ruff check .`.
- [ ] Commit `feat(harness): add local session repository`.
### Task 2: PostgreSQL schema, migrations, and repository
**Files:**
- Create: `harness/tht/session/postgres_repository.py`, `harness/tht/migrations/sessions/001_schema.sql`, `harness/tht/migrations/sessions/002_security.sql`
- Modify: `harness/tht/cli/session_cmd.py`, `harness/tht/config.py`
- Test: `harness/tests/test_postgres_session_repository.py`, `harness/tests/test_session_migrate_cmd.py`
**Interfaces:** Add `tht session migrate --database-url URL [--status] --json`; `PostgresSessionRepository` implements the Task 1 contract, starts a transaction, sets actor/admin local settings, and maps `cte_sql:<name>` to artifacts.
- [ ] Write failing unit/integration tests for migration status, owner isolation, admin cross-user access, cascade delete with content-free audit tombstone, and no embedding invocation.
- [ ] Run the focused tests and confirm expected RED failures.
- [ ] Implement idempotent migration runner, private tables, forced RLS policies, role separation, advisory transaction locks, and repository methods using direct PostgreSQL TLS settings.
- [ ] Run focused tests, then full harness tests and Ruff.
- [ ] Commit `feat(harness): persist server sessions in postgres`.
### Task 3: Migrate harness workflow and deterministic write commands
**Files:**
- Modify: `harness/tht/session/store.py`, `harness/tht/decisions.py`, `harness/tht/phase.py`, `harness/tht/taskdoc.py`, `harness/tht/ctetest.py`, `harness/tht/solved.py`, `harness/tht/teardown.py`, `harness/tht/cli/{session,decision,cte,sql,phase,search,memory}_cmd.py`, `harness/.pi/skills/tht-sessione/SKILL.md`, `harness/.pi/extensions/tht-gate.js`
- Test: `harness/tests/test_session_repository_workflow.py`, `harness/gate/__tests__/session-repository-writes.test.js`
**Interfaces:** Existing workflow commands operate through the configured repository. Add `tht cte save --session ID --name NAME --file -` and `tht sql set-final --session ID --file -`; Pi tools `write_cte_sql` and `write_final_sql` use them instead of direct session paths.
- [ ] Write failing tests that run phase calculation and finalize against an in-memory/filesystem repository, and prove Pi write tools persist CTE/final SQL without direct `sessions/<id>` writes.
- [ ] Run focused tests and record RED results.
- [ ] Refactor path-bound helpers into snapshot/ledger and repository calls; retain backwards-compatible readable local documents and pristine `--json` stdout.
- [ ] Make finalization atomically store report/evidence/status only after DWH verification; leave solved-question creation best-effort.
- [ ] Run harness and gate test suites, then commit `refactor(harness): route workflow persistence through repositories`.
### Task 4: Trusted portal identity and proxy forwarding
**Files:**
- Modify: `/home/chirone/omics_portal/kokoro/datamart_catalog_views.py`, `/home/chirone/omics_portal/nginx/nginx.conf`
- Create: `/home/chirone/omics_portal/kokoro/test_thothii_auth.py`
**Interfaces:** The auth-request response supplies only `X-Thoth-Principal-Issuer`, `X-Thoth-Principal-Subject`, `X-Thoth-Principal-Display-Name`, and `X-Thoth-Is-Admin`. nginx removes client values for those headers and forwards subrequest values to ThothII.
- [ ] Write Django tests for authenticated capability user, denied user, Django PK subject, admin group flag, and absent/spoofed client headers.
- [ ] Run the focused Docker test and confirm RED.
- [ ] Emit normalized headers from the capability endpoint and configure nginx auth-request header capture/injection.
- [ ] Run `docker compose exec -T web python manage.py test accounts.test_capabilities kokoro.test_thothii_auth -v 2`.
- [ ] Commit the portal changes in its own repository with `feat(thothii): forward trusted principal`.
### Task 5: Backend principal enforcement and per-user settings
**Files:**
- Create: `backend/src/auth/principal.ts`
- Modify: `backend/src/auth/auth.ts`, `backend/src/config.ts`, `backend/src/app.ts`, `backend/src/routes/{sessions,settings,sql,meta}.ts`, `backend/src/tht/tht-runner.ts`, `backend/src/pi/pi-process-manager.ts`
- Test: `backend/test/auth.test.ts`, `backend/test/routes-sessions.test.ts`, `backend/test/sse-route.test.ts`, `backend/test/routes-settings.test.ts`
**Interfaces:** Add `GET /me`; require a `PrincipalContext` for all session, document, SQL, response, steer and SSE operations. `GET /sessions?scope=mine|all` permits `all` only for admins. Local mode creates `~/.thothii/identity.json`; upstream mode accepts only proxy-injected normalized headers.
- [ ] Write failing route tests for missing identity (401), foreign session (404), admin all-scope, owner assignment server-side, SSE denial before Pi spawn, and preference isolation.
- [ ] Run `cd backend && npx vitest run test/auth.test.ts test/routes-sessions.test.ts test/sse-route.test.ts test/routes-settings.test.ts`; confirm RED.
- [ ] Implement principal parser, local identity, upstream validation, repository-aware `ThtRunner`, session authorization before Pi, `/me`, scope enforcement and async per-user settings.
- [ ] Run backend Vitest, typecheck, and build; commit `feat(backend): enforce user-owned sessions`.
### Task 6: Frontend identity and administrator UX
**Files:**
- Modify: `frontend/src/api/{client,sessions,settings,types}.ts`, `frontend/src/shell/{AppShell,NavSessions,SessionMenu}.tsx`
- Test: `frontend/src/api/sessions.test.ts`, `frontend/src/shell/AppShell.session-mgmt.test.tsx`, `frontend/src/shell/NavSessions.test.tsx`
**Interfaces:** Fetch `/me`; default to `scope=mine`. Display explicit `All sessions` only to admins, show owner labels/admin banner, and require confirmation before cross-owner destructive actions.
- [ ] Write failing component/API tests for regular-user scope, admin scope switch, owner label, admin banner, and cross-owner confirmation.
- [ ] Run focused Vitest tests and confirm RED.
- [ ] Implement typed client calls, session scope state, and explicit admin affordances without changing ordinary user flow.
- [ ] Run frontend Vitest, `npx tsc -b`, build and E2E; commit `feat(frontend): expose owned session scopes`.
### Task 7: Deployment contract, migration and cutover tooling
**Files:**
- Modify: `docker/`, deployment examples, `README.md`, `PROJECT_STATE.md`
- Test: `harness/tests/test_session_migrate_cmd.py`, `backend/test/config.test.ts`
**Interfaces:** Define runtime/migrator DB secret names, `AUTH_MODE=upstream` server configuration, local loopback-only configuration, and health/readiness behavior returning 503 when repository storage is unavailable.
- [ ] Write failing config tests for required server database/TLS inputs and rejected public/local combinations.
- [ ] Run focused tests and confirm RED.
- [ ] Document runtime/migrator roles, CA/secret injection, backup then deletion of the three legacy server sessions, maintenance-mode cutover, and no dual-write rollback policy.
- [ ] Run all three layer gates; commit `docs(deploy): document user-owned session cutover`.
### Task 8: End-to-end authorization verification and final review
**Files:**
- Modify: test fixtures only as required by earlier tasks.
- [ ] Add cross-layer tests for A/B/admin isolation, spoofed-header rejection, concurrent session mutation, local two-home isolation, absent embeddings, and failed DB startup before Pi.
- [ ] Run harness, backend and frontend complete gates plus portal Docker tests.
- [ ] Run the final whole-branch review, fix all Critical and Important findings, and re-run the covering tests.
- [ ] Commit `test: cover user-owned session security boundaries` and prepare the branch for integration.
docs/superpowers/plans/2026-07-16-user-preference-bootstrap.md
+110
View File
@@ -0,0 +1,110 @@
# User Preference Bootstrap Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Seed a new principal's private settings from complete legacy settings so a first session can configure and start Pi.
**Architecture:** `buildApp` continues to resolve settings through the principal-bound harness runner. When `preferencesGet()` returns an empty object, it derives the existing effective legacy settings from `SETTINGS_FILE`, persists them once via `preferencesSet()`, and returns that same object. A non-empty private object remains authoritative.
**Tech Stack:** TypeScript, Fastify, Vitest.
## Global Constraints
- Never overwrite a non-empty private preference object.
- Persist only provider, model, thinking, and workspace values; no credentials enter preferences.
- Keep storage failures fail-closed through the existing 503 route contract.
- Follow TDD: observe the regression test fail before adding implementation.
---
### Task 1: Bootstrap legacy settings for an empty private profile
**Files:**
- Modify: `backend/test/routes-settings.test.ts`
- Modify: `backend/src/app.ts`
**Interfaces:**
- Consumes: `ThtRunner.preferencesGet(): Promise<Record<string, unknown>>`, `ThtRunner.preferencesSet(settings): Promise<void>`, `loadSettings(config)`, and `effectiveSettings(config, settings)`.
- Produces: `getSettings(principal): Promise<Settings>` that returns a complete persisted profile for first-time principals.
- [ ] **Step 1: Write the failing regression tests**
```ts
test("GET /settings seeds an empty private profile from complete legacy settings once", async () => {
// Seed SETTINGS_FILE with local-qwen/qwen3.6-35b-a3b/low.
// Make preferencesGet return {} and record preferencesSet calls.
// Assert the first GET returns and persists all four settings, and a second GET does not write again.
});
test("GET /settings keeps a non-empty private profile authoritative", async () => {
// Seed different legacy settings, return an existing private profile,
// and assert no preferencesSet call occurs.
});
```
- [ ] **Step 2: Run the focused test file and verify RED**
Run: `cd backend && npx vitest run test/routes-settings.test.ts`
Expected: the first test fails because the current resolver returns only defaults and never calls `preferencesSet`.
- [ ] **Step 3: Implement the minimal resolver change**
```ts
const stored = await runner.preferencesGet();
if (Object.keys(stored).length === 0) {
const seeded = effectiveSettings(config, loadSettings(config));
await runner.preferencesSet(seeded);
return seeded;
}
return effectiveSettings(config, stored);
```
- [ ] **Step 4: Run focused tests and TypeScript verification**
Run: `cd backend && npx vitest run test/routes-settings.test.ts && npx tsc --noEmit -p .`
Expected: exit 0.
- [ ] **Step 5: Run backend regression suite**
Run: `cd backend && npx vitest run && npm run build`
Expected: exit 0.
### Task 2: Preserve DWH artifact ownership across the optional session-storage field
**Files:**
- Modify: `harness/tests/test_dwh_preprocess_job.py`
- Modify: `harness/tht/jobs/dwh_pipeline.py`
**Interfaces:**
- Consumes: `config_dwh_binding(cfg)` and Pydantic's `Config.model_dump(mode="json")`.
- Produces: a DWH binding whose `config_fingerprint` excludes only `session_storage`.
- [ ] **Step 1: Write the failing regression test**
```python
def test_session_storage_does_not_change_the_dwh_artifact_binding(tmp_path):
assert config_dwh_binding(config()) == config_dwh_binding(config(session_storage={...}))
```
- [ ] **Step 2: Run the focused test and verify RED**
Run: `cd harness && .venv/bin/pytest tests/test_dwh_preprocess_job.py::test_session_storage_does_not_change_the_dwh_artifact_binding -q`
Expected: FAIL because the full configuration JSON currently includes `session_storage`.
- [ ] **Step 3: Implement the minimal DWH-only fingerprint**
```python
payload = cfg.model_dump(mode="json")
payload.pop("session_storage", None)
config_fingerprint = fingerprint(json.dumps(payload, separators=(",", ":"), ensure_ascii=False))
```
- [ ] **Step 4: Run focused and complete harness verification**
Run: `cd harness && .venv/bin/pytest tests/test_dwh_preprocess_job.py -q && .venv/bin/pytest -q && .venv/bin/ruff check tht/jobs/dwh_pipeline.py tests/test_dwh_preprocess_job.py`
Expected: exit 0.
docs/superpowers/specs/2026-07-16-user-owned-session-storage-design.md
+25
View File
@@ -0,0 +1,25 @@
# User-owned session storage design
## Decisioni vincolanti
- In server mode ThothII usa PostgreSQL diretto, nello schema Supabase privato
`thoth_sessions`; il browser non accede mai al database.
- In local mode ogni utente usa `~/.thothii` (override esplicito `THT_HOME`),
senza fallback o sincronizzazione con Supabase.
- Una sessione appartiene al principal `(issuer, subject)`. Nel portale il
subject è il PK Django; email e username non sono identificatori.
- Il proxy valida la sessione del portale e inietta identità normalizzata; il
backend richiede `AUTH_MODE=upstream` in produzione e non riceve token raw.
- Gli utenti nel gruppo Authentik `authentik Admins` possono gestire ogni
sessione. Un accesso non autorizzato restituisce 404.
- Lo schema contiene principals, principal_preferences, sessions,
session_artifacts, review_decisions e audit_log. Artefatti correnti e
decision ledger append-only; niente cronologia di artefatti né contenuto
nell'audit.
- La sicurezza server combina RLS forzata, un runtime role senza BYPASSRLS,
contesto attore transaction-local e filtri applicativi espliciti.
- Non vengono creati embeddings o vector columns per le sessioni. La memoria
solved_question esistente resta un flusso separato best-effort.
- Sessioni e preferenze server sono persistite solo nel DB; guasti DB sono
fail-closed (503). I file temporanei di export sono effimeri.
- Chat e SSE restano memoria runtime, non artefatti persistiti.
docs/superpowers/specs/2026-07-16-user-preference-bootstrap-design.md
+34
View File
@@ -0,0 +1,34 @@
# User preference bootstrap design
## Problem
The user-owned-session branch reads settings only from the current principal's
repository preferences. Existing deployments still have their provider, model,
thinking level, and workspace only in the legacy backend settings JSON file.
For a principal with no private preference record, a new session is therefore
created without a provider or model and Pi is rejected before it can spawn.
## Decision
On the first settings read for a principal whose private preference object is
empty, the backend computes the complete effective legacy settings from
`SETTINGS_FILE` and the configured environment defaults, writes that complete
object to the principal's repository preferences, and returns it.
After this one-time bootstrap, the private preference object is the sole
source for that principal. A non-empty private object is never replaced with
the legacy values. If either reading or writing the private preferences fails,
the request remains fail-closed with the existing 503 response.
The addition of optional session storage must not change DWH artifact
ownership. The DWH binding therefore excludes `session_storage` while retaining
every schema, retrieval, vector, and execution setting in its fingerprint.
## Scope and verification
The change is confined to the backend settings resolver. Vitest coverage must
prove that an empty private profile is seeded exactly once with the complete
legacy settings, and that an existing private profile is neither changed nor
replaced. Harness coverage must also prove that adding session storage leaves
the DWH binding unchanged. Existing session-route coverage continues to prove
that new-session creation consumes the resolved settings.
frontend/src/api/sessions.test.ts
+27 -3
View File
@@ -1,6 +1,6 @@
import { http, HttpResponse } from "msw";
import { server } from "../test/msw";
import { createSession, listSessions, prewarmRuntime, resumeSession } from "./sessions";
import { createSession, getMe, listSessions, prewarmRuntime, resumeSession } from "./sessions";
import {
renameSession, setSessionGroup, archiveSession, unarchiveSession,
deleteSession, getSessionDocuments,
@@ -30,10 +30,34 @@ test.each([202, 204])("prewarmRuntime accepts a body-less %s response", async (s
expect(called).toBe(true);
});
test("listSessions GETs the array", async () => {
server.use(http.get("http://localhost:8787/sessions", () => HttpResponse.json([{ id: "s1", status: "open", question: "q", summary: null, created_at: "t", updated_at: null, author: null }])));
test("listSessions defaults to the current user's scope", async () => {
let scope: string | null = null;
server.use(http.get("http://localhost:8787/sessions", ({ request }) => {
scope = new URL(request.url).searchParams.get("scope");
return HttpResponse.json([{ id: "s1", status: "open", question: "q", summary: null, created_at: "t", updated_at: null, author: null }]);
}));
const rows = await listSessions();
expect(rows[0].id).toBe("s1");
expect(scope).toBe("mine");
});
test("listSessions requests the selected administrator scope", async () => {
let scope: string | null = null;
server.use(http.get("http://localhost:8787/sessions", ({ request }) => {
scope = new URL(request.url).searchParams.get("scope");
return HttpResponse.json([]);
}));
await listSessions("all");
expect(scope).toBe("all");
});
test("getMe fetches the typed authenticated principal", async () => {
server.use(http.get("http://localhost:8787/me", () =>
HttpResponse.json({ issuer: "portal", subject: "42", displayName: "Alice", isAdmin: true }),
));
await expect(getMe()).resolves.toEqual({
issuer: "portal", subject: "42", displayName: "Alice", isAdmin: true,
});
});
test("resumeSession returns the typed runtime disposition", async () => {
frontend/src/api/sessions.ts
+7 -2
View File
@@ -1,5 +1,7 @@
import { apiFetch } from "./client";
import type { ResumeSessionResult, SessionSummary, SessionDocument, UiResponse } from "./types";
import type {
Principal, ResumeSessionResult, SessionScope, SessionSummary, SessionDocument, UiResponse,
} from "./types";
export const createSession = (i: { question: string; name?: string }) =>
apiFetch<{ id: string }>("/sessions", { method: "POST", body: JSON.stringify(i) });
@@ -8,7 +10,10 @@ export const createSession = (i: { question: string; name?: string }) =>
export const prewarmRuntime = () =>
apiFetch<void>("/runtime/prewarm", { method: "POST" });
export const listSessions = () => apiFetch<SessionSummary[]>("/sessions");
export const getMe = () => apiFetch<Principal>("/me");
export const listSessions = (scope: SessionScope = "mine") =>
apiFetch<SessionSummary[]>(`/sessions?scope=${scope}`);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export const getSession = (id: string) => apiFetch<any>(`/sessions/${id}`);
frontend/src/api/types.ts
+9
View File
@@ -104,6 +104,15 @@ export interface SessionSummary {
archived: boolean;
}
export type SessionScope = "mine" | "all";
export interface Principal {
issuer: string;
subject: string;
displayName?: string;
isAdmin: boolean;
}
export interface ResumeSessionResult {
id: string;
alreadyActive: boolean;
frontend/src/shell/AppShell.session-mgmt.test.tsx
+93
View File
@@ -54,6 +54,9 @@ beforeEach(() => {
window.matchMedia = vi.fn().mockReturnValue({ matches: true, addEventListener: vi.fn(), removeEventListener: vi.fn() });
useSessionStore.getState().resetSession();
server.use(
http.get("http://localhost:8787/me", () =>
HttpResponse.json({ issuer: "portal", subject: "alice", displayName: "Alice", isAdmin: false }),
),
http.get("http://localhost:8787/sessions", () => HttpResponse.json(LIST)),
http.get("http://localhost:8787/sessions/:id/documents", () => HttpResponse.json([
{ phase: "—", key: "question", title: "Domanda originale", format: "text", content: "Attiva uno" },
@@ -62,6 +65,96 @@ beforeEach(() => {
);
});
test("regular users load only their sessions and never see administrator controls", async () => {
let scope: string | null = null;
server.use(http.get("http://localhost:8787/sessions", ({ request }) => {
scope = new URL(request.url).searchParams.get("scope");
return HttpResponse.json(LIST);
}));
wrap();
await screen.findByText("Attiva uno");
expect(scope).toBe("mine");
expect(screen.queryByRole("button", { name: "All sessions" })).not.toBeInTheDocument();
expect(screen.queryByText(/administrator view/i)).not.toBeInTheDocument();
});
test("administrators can explicitly switch to all sessions and see owners", async () => {
let scope = "";
server.use(
http.get("http://localhost:8787/me", () =>
HttpResponse.json({ issuer: "portal", subject: "alice-id", displayName: "Alice", isAdmin: true }),
),
http.get("http://localhost:8787/sessions", ({ request }) => {
scope = new URL(request.url).searchParams.get("scope") ?? "";
return HttpResponse.json([
{ ...LIST[0], author: "Alice" },
{ ...LIST[1], id: "s3", question: "Another owner's session", archived: false, author: "Bob" },
]);
}),
);
wrap();
await screen.findByRole("button", { name: "All sessions" });
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "true");
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "false");
await userEvent.click(screen.getByRole("button", { name: "All sessions" }));
await waitFor(() => expect(scope).toBe("all"));
expect(screen.getByRole("button", { name: "My sessions" })).toHaveAttribute("aria-pressed", "false");
expect(screen.getByRole("button", { name: "All sessions" })).toHaveAttribute("aria-pressed", "true");
expect(await screen.findByText("Administrator view: all sessions")).toBeInTheDocument();
expect(screen.getByText("Owner: Bob")).toBeInTheDocument();
});
test("administrator confirms before deleting a same-named user's session", async () => {
let deletes = 0;
server.use(
http.get("http://localhost:8787/me", () =>
HttpResponse.json({ issuer: "portal", subject: "alice-id", displayName: "Alice", isAdmin: true }),
),
http.get("http://localhost:8787/sessions", () => HttpResponse.json([
{ ...LIST[0], author: "Alice" },
{ ...LIST[1], id: "s3", question: "Second session", archived: false, author: "Bob" },
])),
http.delete("http://localhost:8787/sessions/:id", () => {
deletes += 1;
return new HttpResponse(null, { status: 204 });
}),
);
wrap();
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
await screen.findByText("Owner: Alice");
await userEvent.click(screen.getByRole("checkbox", { name: "Select Attiva uno" }));
await userEvent.click(screen.getByRole("button", { name: "Delete 1 selected sessions" }));
expect(deletes).toBe(0);
expect(await screen.findByRole("heading", { name: "Delete permanently" })).toBeInTheDocument();
await userEvent.click(screen.getByRole("button", { name: "Delete" }));
await waitFor(() => expect(deletes).toBe(1));
});
test("administrator confirms before archiving a same-named user's session", async () => {
let archives = 0;
const confirm = vi.spyOn(window, "confirm").mockReturnValue(false);
server.use(
http.get("http://localhost:8787/me", () =>
HttpResponse.json({ issuer: "portal", subject: "alice-id", displayName: "Alice", isAdmin: true }),
),
http.get("http://localhost:8787/sessions", () => HttpResponse.json([
{ ...LIST[0], author: "Alice" },
])),
http.post("http://localhost:8787/sessions/:id/archive", () => {
archives += 1;
return new HttpResponse(null, { status: 204 });
}),
);
wrap();
await userEvent.click(await screen.findByRole("button", { name: "All sessions" }));
await screen.findByText("Owner: Alice");
await userEvent.click(screen.getByRole("button", { name: "Session actions" }));
await userEvent.click(await screen.findByText("Archive"));
expect(confirm).toHaveBeenCalledWith("Archive Alice's session?");
expect(archives).toBe(0);
confirm.mockRestore();
});
test("active list shows group header and hides archived sessions", async () => {
wrap();
expect(await screen.findByText("Attiva uno")).toBeInTheDocument();
frontend/src/shell/AppShell.tsx
+61 -7
View File
@@ -18,10 +18,10 @@ import { Checkbox } from "../components/ui/checkbox";
import { Toaster } from "../components/ui/sonner";
import { toast } from "sonner";
import {
closeSession, listSessions, resumeSession, getSession,
closeSession, getMe, listSessions, resumeSession, getSession,
renameSession, setSessionGroup, archiveSession, unarchiveSession, deleteSession, prewarmRuntime,
} from "../api/sessions";
import type { SessionSummary } from "../api/types";
import type { Principal, SessionScope, SessionSummary } from "../api/types";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { useEffect, useMemo, useRef, useState } from "react";
import type { CSSProperties } from "react";
@@ -48,8 +48,10 @@ export function AppShell() {
const [streamCursorResetEpoch, setStreamCursorResetEpoch] = useState(0);
const [creatingSession, setCreatingSession] = useState(false);
const [awaitingQuestion, setAwaitingQuestion] = useState(false);
const [sessionScope, setSessionScope] = useState<SessionScope>("mine");
const { data: principal } = useQuery<Principal>({ queryKey: ["me"], queryFn: getMe, staleTime: Infinity });
const { data: sessions = [] } = useQuery<SessionSummary[]>({
queryKey: ["sessions"], queryFn: listSessions, refetchInterval: 10_000,
queryKey: ["sessions", sessionScope], queryFn: () => listSessions(sessionScope), refetchInterval: 10_000,
});
const composerRef = useRef<HTMLTextAreaElement>(null);
@@ -77,6 +79,12 @@ export function AppShell() {
const finalized = activeSession?.status === "finalized";
const selectedSessions = sessions.filter((session) => selectedSessionIds.has(session.id));
const allSessionsSelected = sessions.length > 0 && selectedSessions.length === sessions.length;
const showingAllSessions = sessionScope === "all";
const isForeignSession = (session: SessionSummary) => {
if (!showingAllSessions || !principal) return false;
if (!session.author) return true;
return session.author !== principal.subject;
};
function selectActiveSession(id: string | null) {
// Keep async Resume completions synchronized before React commits the state update.
@@ -245,6 +253,14 @@ export function AppShell() {
}
}
function requestArchiveToggle(session: SessionSummary) {
if (!session.archived && isForeignSession(session)) {
const label = session.author ? `${session.author}'s session` : "this session";
if (!window.confirm(`Archive ${label}?`)) return;
}
void toggleArchive(session);
}
async function deleteSessions(targets: SessionSummary[]) {
try {
const results = await Promise.allSettled(targets.map((session) => deleteSession(session.id)));
@@ -266,6 +282,14 @@ export function AppShell() {
}
}
function requestDelete(targets: SessionSummary[]) {
if (targets.some(isForeignSession) || allSessionsSelected) {
setDeleteTargets(targets);
return;
}
void deleteSessions(targets);
}
function menuFor(s: SessionSummary) {
return (
<SessionMenu
@@ -276,8 +300,8 @@ export function AppShell() {
onRename={() => setRenameTarget(s)}
onMove={(g) => move(s, g)}
onNewGroup={() => newGroup(s)}
onArchiveToggle={() => toggleArchive(s)}
onDelete={() => { void deleteSessions([s]); }}
onArchiveToggle={() => requestArchiveToggle(s)}
onDelete={() => requestDelete([s])}
/>
);
}
@@ -496,6 +520,34 @@ export function AppShell() {
</Button>
</div>
{principal?.isAdmin && (
<div className="px-4 pb-3">
<div className="grid grid-cols-2 gap-1 rounded-lg bg-muted p-1" aria-label="Session scope">
<Button
variant={sessionScope === "mine" ? "secondary" : "ghost"}
size="xs"
aria-pressed={sessionScope === "mine"}
onClick={() => setSessionScope("mine")}
>
My sessions
</Button>
<Button
variant={showingAllSessions ? "secondary" : "ghost"}
size="xs"
aria-pressed={showingAllSessions}
onClick={() => setSessionScope("all")}
>
All sessions
</Button>
</div>
{showingAllSessions && (
<p className="mt-2 text-xs font-medium text-amber-700 dark:text-amber-400">
Administrator view: all sessions
</p>
)}
</div>
)}
{/* L1 — rail title */}
<div className="px-4 pb-1.5 pt-1">
<span className="thot-label text-[0.8rem] font-bold tracking-[0.18em] text-primary">
@@ -519,8 +571,7 @@ export function AppShell() {
size="xs"
aria-label={`Delete ${selectedSessions.length} selected sessions`}
onClick={() => {
if (allSessionsSelected) setDeleteTargets(selectedSessions);
else void deleteSessions(selectedSessions);
requestDelete(selectedSessions);
}}
>
<Trash2 />
@@ -571,6 +622,7 @@ export function AppShell() {
menuFor={menuFor}
selectedIds={selectedSessionIds}
onSelectionChange={setSessionSelected}
showOwner={showingAllSessions}
/>
)}
</div>
@@ -586,6 +638,7 @@ export function AppShell() {
menuFor={menuFor}
selectedIds={selectedSessionIds}
onSelectionChange={setSessionSelected}
showOwner={showingAllSessions}
/>
)}
</div>
@@ -609,6 +662,7 @@ export function AppShell() {
menuFor={menuFor}
selectedIds={selectedSessionIds}
onSelectionChange={setSessionSelected}
showOwner={showingAllSessions}
/>
)}
</div>
frontend/src/shell/NavSessions.test.tsx
+13
View File
@@ -26,6 +26,19 @@ test("active session is highlighted", () => {
expect(screen.getByTestId("session-item-s2")).toHaveAttribute("data-active", "true");
});
test("administrator session lists show the recorded owner", () => {
render(
<NavSessions
sessions={[{ ...SESSIONS[0], author: "Alice" }]}
activeSessionId={null}
onOpenPanel={vi.fn()}
menuFor={() => null}
showOwner
/>,
);
expect(screen.getByText("Owner: Alice")).toBeInTheDocument();
});
test("pressing Enter on a focused row opens the panel", async () => {
const onOpenPanel = vi.fn();
render(<NavSessions sessions={SESSIONS} activeSessionId={null} onOpenPanel={onOpenPanel} menuFor={() => null} />);
frontend/src/shell/NavSessions.tsx
+16 -8
View File
@@ -10,6 +10,7 @@ interface Props {
menuFor: (session: SessionSummary) => ReactNode;
selectedIds?: ReadonlySet<string>;
onSelectionChange?: (id: string, selected: boolean) => void;
showOwner?: boolean;
}
function statusIndicator(status: string) {
@@ -21,7 +22,7 @@ function statusIndicator(status: string) {
export function NavSessions({
sessions, activeSessionId, onOpenPanel, menuFor,
selectedIds = new Set<string>(), onSelectionChange = () => undefined,
selectedIds = new Set<string>(), onSelectionChange = () => undefined, showOwner = false,
}: Props) {
if (sessions.length === 0) {
return (
@@ -70,13 +71,20 @@ export function NavSessions({
title={indicator.label}
className={["size-1.5 shrink-0 rounded-full", indicator.color].join(" ")}
/>
<span
className={[
"min-w-0 flex-1 truncate text-[0.8rem] leading-snug",
active ? "font-semibold text-foreground" : "font-medium text-foreground/90",
].join(" ")}
>
{label}
<span className="min-w-0 flex-1">
<span
className={[
"block truncate text-[0.8rem] leading-snug",
active ? "font-semibold text-foreground" : "font-medium text-foreground/90",
].join(" ")}
>
{label}
</span>
{showOwner && (
<span className="block truncate text-[0.65rem] text-muted-foreground">
Owner: {s.author ?? "Unknown"}
</span>
)}
</span>
{menuFor(s)}
</div>
harness/.gitignore
+3
View File
@@ -3,6 +3,9 @@ __pycache__/
.env
.venv/
sessions/
# Keep the versioned database migration package; only runtime session directories are ignored.
!tht/migrations/sessions/
!tht/migrations/sessions/*.sql
indexes/
artifacts/
# Per-customer active workspace (points at the customer repo via -c or symlink).
harness/.pi/extensions/gate/__tests__/session-repository-writes.test.js
+41
View File
@@ -0,0 +1,41 @@
const test = require("node:test");
const assert = require("node:assert");
const cp = require("node:child_process");
const { createRequire } = require("node:module");
const path = require("node:path");
const GATE = path.join(__dirname, "..", "..", "tht-gate.js");
if (typeof globalThis.require === "undefined") globalThis.require = createRequire(GATE);
test("repository write tools persist CTE and final SQL through deterministic tht commands", async () => {
const calls = [];
const original = cp.execFileSync;
cp.execFileSync = (file, args, options) => {
calls.push({ args, input: options?.input });
return "";
};
try {
const gate = require(GATE);
const { createFakePi } = require("./fake_pi_runtime.js");
const { pi, ctx, tools } = createFakePi();
ctx.cwd = "/nonexistent-thothii-test-cwd";
gate.default(pi);
const cte = tools.get("write_cte_sql");
const final = tools.get("write_final_sql");
assert.ok(cte, "write_cte_sql must be registered");
assert.ok(final, "write_final_sql must be registered");
await cte.def.execute("cte", { session: "s1", name: "base", sql: "WITH base AS (SELECT 1)" }, null, null, ctx);
await final.def.execute("sql", { session: "s1", sql: "SELECT * FROM base" }, null, null, ctx);
assert.deepEqual(calls.map((call) => call.args), [
["cte", "save", "--session", "s1", "--name", "base", "--file", "-"],
["sql", "set-final", "--session", "s1", "--file", "-"],
]);
assert.deepEqual(calls.map((call) => call.input), ["WITH base AS (SELECT 1)", "SELECT * FROM base"]);
assert.equal(JSON.stringify(calls).includes("sessions/s1"), false);
} finally {
cp.execFileSync = original;
}
});
harness/.pi/extensions/tht-gate.js
+32
View File
@@ -1401,6 +1401,38 @@ export default function (pi) {
},
});
pi.registerTool({
name: "write_cte_sql",
label: "Scrittura CTE SQL",
description: "Persiste un blocco CTE tramite tht cte save; non scrivere mai file di sessione direttamente.",
parameters: Type.Object({ session: Type.String(), name: Type.String(), sql: Type.String() }),
async execute(_id, params, _signal, _onUpdate, ctx) {
const err = relayIfThtFails(
ctx,
["cte", "save", "--session", params.session, "--name", params.name, "--file", "-"],
"Correggi il blocco CTE e riprova.",
params.sql,
);
return err || textResult(`CTE ${params.name} salvato per la sessione ${params.session}.`);
},
});
pi.registerTool({
name: "write_final_sql",
label: "Scrittura SQL finale",
description: "Persiste il SQL finale tramite tht sql set-final; non scrivere mai file di sessione direttamente.",
parameters: Type.Object({ session: Type.String(), sql: Type.String() }),
async execute(_id, params, _signal, _onUpdate, ctx) {
const err = relayIfThtFails(
ctx,
["sql", "set-final", "--session", params.session, "--file", "-"],
"Correggi il SQL finale e riprova.",
params.sql,
);
return err || textResult(`SQL finale salvato per la sessione ${params.session}.`);
},
});
// --- slash command: /torna [session_id] [N] (rollback to a previous phase) --
pi.registerCommand("torna", {
description:
harness/.pi/skills/tht-sessione/SKILL.md
+5 -2
View File
@@ -353,7 +353,9 @@ Prerequisite: Phase 5 closed.
"output_columns":["cod_paz","data_ricovero"]}
]}
```
3. For each CTE (in plan order): write `sessions/<id>/ctes/<name>.sql` (ONLY the
3. For each CTE (in plan order): call `write_cte_sql` with the session id, CTE name,
and SQL block (the tool invokes `tht cte save --session <id> --name <name> --file -`).
Persist ONLY the
`WITH ... AS (...)` block, NO trailing SELECT), test with `tht cte test --session
<id> <name>` (with an **ok** outcome), then present it with
`reviewer_confirm kind:"cte_result"`. Pass ONLY the thin v2 data
@@ -385,7 +387,8 @@ Prerequisite: Phase 6 closed.
columns (`dt.year`, `dt.month`, …), NEVER arithmetic on the key.
3. `tht sql validate` + `tht sql preview` (max 10 rows). On errors / suspicious
results, apply the `sql-generation.md` checklist and correct with the reviewer.
4. `tht sql save` writes `sessions/<id>/sql_final.sql` (ONLY clean SQL, no comments).
4. Call `write_final_sql` with the session id and clean SQL; it invokes
`tht sql set-final --session <id> --file -` (ONLY clean SQL, no comments).
Approve with `reviewer_confirm kind:"sql"` (records `sql_approved`), then advance to
Phase 8 with `reviewer_confirm kind:"phase"` — `kind:"sql"` alone does NOT advance F7.
harness/pyproject.toml
+2 -1
View File
@@ -16,6 +16,7 @@ dependencies = [
"requests>=2.31",
"tqdm>=4.66",
"yake>=0.4",
"portalocker>=2.10",
]
[project.scripts]
@@ -33,7 +34,7 @@ dev = [
include = ["tht*"]
[tool.setuptools.package-data]
tht = ["migrations/vector/*.sql"]
tht = ["migrations/vector/*.sql", "migrations/sessions/*.sql"]
[tool.ruff]
line-length = 100
harness/tests/test_adapter_command_regressions.py
+5 -6
View File
@@ -67,17 +67,17 @@ def test_memory_command_writes_through_factory_vector_store(monkeypatch):
# configure the workstation-only REST writer key.
cfg = SimpleNamespace(profile="server", embeddings=object(), vector_write_rest=None)
manifest = SimpleNamespace(id="s1")
snapshot = SimpleNamespace(manifest=manifest, decisions=[], artifacts={})
record = MemoryRecord(id="m1", ts=datetime(2026, 1, 1), session_id="s1",
decision_seq=7, type="table_promoted", subject="t",
question_context="q")
monkeypatch.setattr(memory_cmd, "_load_config_or_exit", lambda path: cfg)
monkeypatch.setattr(memory_cmd, "load_session_or_exit", lambda cfg, session: manifest)
monkeypatch.setattr(memory_cmd, "session_dir", lambda *args: None)
monkeypatch.setattr(memory_cmd, "load_snapshot_or_exit", lambda cfg, session: snapshot)
monkeypatch.setattr(memory_cmd, "registry_path", lambda cfg: None)
monkeypatch.setattr("tht.adapters.factory.build_vector_store", lambda cfg, require_write: store)
monkeypatch.setattr("tht.cli.vector_cmd.make_embedder",
lambda cfg: SimpleNamespace(embed_documents=lambda texts: [[0.1]]))
monkeypatch.setattr("tht.memory.promote", lambda *args, **kwargs: None)
monkeypatch.setattr("tht.memory.promote_snapshot", lambda *args, **kwargs: None)
monkeypatch.setattr("tht.memory.load_registry", lambda path: [record])
memory_cmd.save_one_cmd(session="s1", decision=7, json_out=True)
from tht.ports.vector import VectorWriteRecord
@@ -96,14 +96,13 @@ def test_solved_index_writes_through_writer_only_factory_store(monkeypatch):
calls = []
monkeypatch.setattr(memory_cmd, "has_vector_write_rest", lambda cfg: True)
monkeypatch.setattr(memory_cmd, "load_session_or_exit", lambda cfg, session: manifest)
monkeypatch.setattr(memory_cmd, "session_dir", lambda *args: None)
monkeypatch.setattr(memory_cmd, "load_snapshot_or_exit", lambda cfg, session: SimpleNamespace(manifest=manifest, decisions=[], artifacts={}))
monkeypatch.setattr(
"tht.adapters.factory.build_vector_store",
lambda cfg, require_write: calls.append(require_write) or writer_only_store,
)
monkeypatch.setattr("tht.cli.sql_cmd.promoted_tables_for", lambda *args: [])
monkeypatch.setattr("tht.solved.build_solved_record", lambda *args: solved_record)
monkeypatch.setattr("tht.solved.build_solved_snapshot", lambda *args: solved_record)
monkeypatch.setattr(
"tht.solved.save_solved_question",
lambda record, *, store, embedder: int(
harness/tests/test_decision_join_set_cli.py
+17 -4
View File
@@ -28,15 +28,28 @@ def _walk_to_phase(session, target):
def _configure_command(monkeypatch, sessions):
import tht.cli.decision_cmd as mod
import tht.cli.session_cmd as session_mod
from tht.session.models import SessionManifest, SessionSnapshot
class _Repository:
def get(self, session_id):
return SessionSnapshot(
manifest=SessionManifest(id=session_id, created_at="2026-01-01T00:00:00Z", question="q", database="d", schema="s"),
decisions=list_decisions(sessions / session_id),
)
def append_decisions(self, session_id, decisions):
return append_decisions(sessions / session_id, list(decisions))
class _Cfg:
class paths:
pass
pass
_Cfg.paths.sessions = sessions
repository = _Repository()
monkeypatch.setattr(mod, "_load_config_or_exit", lambda _c: _Cfg())
monkeypatch.setattr(mod, "load_session_or_exit", lambda _cfg, _s: None)
monkeypatch.setattr(mod, "session_dir", lambda _cfg, sid: sessions / sid)
monkeypatch.setattr(mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid))
monkeypatch.setattr(mod, "session_repository", lambda _cfg: repository)
monkeypatch.setattr(session_mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid))
def test_add_join_set_rejects_the_whole_batch_when_one_item_is_invalid(tmp_path, monkeypatch):
harness/tests/test_decision_retract_cli.py
+18 -1
View File
@@ -25,6 +25,8 @@ def test_retract_drops_last_substantive_decision(tmp_path, monkeypatch):
# stub config + session loading (the command only needs a session dir)
import tht.cli.decision_cmd as mod
import tht.cli.session_cmd as session_mod
from tht.session.models import SessionManifest, SessionSnapshot
class _Cfg:
class paths:
@@ -32,7 +34,22 @@ def test_retract_drops_last_substantive_decision(tmp_path, monkeypatch):
monkeypatch.setattr(mod, "_load_config_or_exit", lambda _c: _Cfg())
monkeypatch.setattr(mod, "load_session_or_exit", lambda _cfg, _s: None)
monkeypatch.setattr(mod, "session_dir", lambda _cfg, sid: tmp_path / sid)
class _Repository:
def get(self, session_id):
return SessionSnapshot(
manifest=SessionManifest(id=session_id, created_at="2026-01-01T00:00:00Z", question="q", database="d", schema="s"),
decisions=list_decisions(tmp_path / session_id),
)
def append_decisions(self, session_id, decisions):
from tht.decisions import append_decisions
return append_decisions(tmp_path / session_id, list(decisions))
repository = _Repository()
monkeypatch.setattr(mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid))
monkeypatch.setattr(mod, "session_repository", lambda _cfg: repository)
monkeypatch.setattr(session_mod, "load_snapshot_or_exit", lambda _cfg, sid: repository.get(sid))
retract_cmd(session="2026-01-01-000000-x", config=Path("x"))
harness/tests/test_dwh_preprocess_job.py
+18
View File
@@ -31,6 +31,24 @@ def test_dwh_and_evidence_jobs_have_distinct_lock_names():
assert _lock_name("demo", "dwh") != _lock_name("demo", "evidence")
def test_session_storage_does_not_change_the_dwh_artifact_binding(tmp_path):
from types import SimpleNamespace
def config(session_storage=None):
payload = {"dwh": {"connection": {"database": "warehouse"}}}
if session_storage is not None:
payload["session_storage"] = session_storage
cfg = SimpleNamespace(_workspace_id="demo", _config_source="test")
cfg.model_dump = lambda **_kwargs: payload
cfg.model_dump_json = lambda: json.dumps(payload, separators=(",", ":"))
return cfg
without_session_storage = config()
with_session_storage = config({"type": "postgres_direct", "connection": {"database": "sessions"}})
assert config_dwh_binding(without_session_storage) == config_dwh_binding(with_session_storage)
def test_unowned_reads_fail_closed_without_creating_any_files(tmp_path):
import pytest
harness/tests/test_local_identity.py
+17
View File
@@ -0,0 +1,17 @@
import json
import uuid
from tht.session.models import local_principal
def test_local_identity_uuid_is_stable_across_restarts(tmp_path):
first = local_principal(tmp_path)
restarted = local_principal(tmp_path)
assert first == restarted
assert first.issuer == "local"
assert uuid.UUID(first.subject, version=4).version == 4
assert json.loads((tmp_path / "identity.json").read_text()) == {
"issuer": "local",
"subject": first.subject,
}
harness/tests/test_postgres_session_repository.py
+265
View File
@@ -0,0 +1,265 @@
import uuid
from concurrent.futures import ThreadPoolExecutor
from datetime import UTC, datetime
from threading import Barrier
import pytest
from sqlalchemy import create_engine, text
from testcontainers.postgres import PostgresContainer
from tht.decisions import DecisionInput
from tht.phase import effective_decisions
from tht.session.models import PrincipalContext, SessionManifest
from tht.session.store import SessionError
@pytest.fixture(scope="module")
def database_url():
with PostgresContainer("postgres:16-alpine") as postgres:
yield postgres.get_connection_url()
@pytest.fixture(scope="module", autouse=True)
def migrated(database_url):
from tht.session.postgres_repository import migrate
migrate(database_url)
def _manifest(session_id: str) -> SessionManifest:
return SessionManifest(
id=session_id,
created_at=datetime(2026, 7, 16, 10, 0, tzinfo=UTC),
question="Which patients had an ablation?",
database="testdb",
schema="public",
)
def _repository(database_url, subject: str, *, is_admin: bool = False):
from tht.session.postgres_repository import PostgresSessionRepository
return PostgresSessionRepository(
database_url,
PrincipalContext(issuer="portal", subject=subject, is_admin=is_admin),
)
def test_owner_can_read_own_snapshot_but_not_another_owners(database_url):
alice = _repository(database_url, "alice")
bob = _repository(database_url, "bob")
session_id = str(uuid.uuid4())
alice.create(_manifest(session_id))
alice.write_artifact(session_id, "cte_sql:eligible_patients", "SELECT 1")
alice.append_decisions(
session_id, [DecisionInput(type="sql_approved", subject="phase:7", detail="SELECT 1")]
)
snapshot = alice.get(session_id)
assert snapshot.principal == alice.principal
assert snapshot.artifacts == {"cte_sql:eligible_patients": "SELECT 1"}
assert snapshot.decisions[0].seq == 1
with pytest.raises(SessionError, match="Sessione non trovata"):
bob.get(session_id)
def test_named_decision_is_stale_after_postgres_ledger_reopen(database_url):
repository = _repository(database_url, "phase-owner")
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
for phase in range(1, 5):
repository.append_decisions(session_id, [
DecisionInput(type="phase_approved", subject=f"phase:{phase}")
])
repository.append_decisions(session_id, [
DecisionInput(type="cte_approved", subject="named_cte")
])
repository.append_decisions(session_id, [
DecisionInput(type="phase_reopened", subject="phase:3")
])
snapshot = repository.get(session_id)
named = next(item for item in snapshot.decisions if item.subject == "named_cte")
assert named.phase == 5
assert "named_cte" not in {item.subject for item in effective_decisions(snapshot)}
def test_admin_can_read_another_owners_session(database_url):
owner = _repository(database_url, "owner")
admin = _repository(database_url, "admin", is_admin=True)
session_id = str(uuid.uuid4())
owner.create(_manifest(session_id))
assert admin.get(session_id).manifest.id == session_id
def test_owner_admin_and_foreign_principals_have_distinct_mutation_boundaries(database_url):
session_id = str(uuid.uuid4())
alice = _repository(database_url, f"alice-{session_id}")
bob = _repository(database_url, f"bob-{session_id}")
admin = _repository(database_url, f"admin-{session_id}", is_admin=True)
manifest = _manifest(session_id)
alice.create(manifest)
alice.write_artifact(session_id, "question", "Alice's question")
assert [snapshot.manifest.id for snapshot in alice.list()] == [session_id]
assert bob.list() == []
for mutation in (
lambda: bob.write_artifact(session_id, "question", "Bob's overwrite"),
lambda: bob.append_decisions(
session_id, [DecisionInput(type="concept_clarified", subject="bob")]
),
lambda: bob.save_manifest(manifest.model_copy(update={"name": "Bob's rename"})),
lambda: bob.delete(session_id),
):
with pytest.raises(SessionError, match="Sessione non trovata"):
mutation()
# Admin access is an explicit, audited exception to normal owner isolation.
admin.write_artifact(session_id, "question", "Reviewed by admin")
admin.append_decisions(
session_id, [DecisionInput(type="concept_clarified", subject="admin-review")]
)
owner_snapshot = alice.get(session_id)
assert owner_snapshot.artifacts["question"] == "Reviewed by admin"
assert [record.subject for record in owner_snapshot.decisions] == ["admin-review"]
assert admin.get(session_id).manifest.id == session_id
def test_concurrent_postgres_ledger_mutations_keep_every_decision_in_sequence(database_url, monkeypatch):
from tht.session.postgres_repository import PostgresSessionRepository
repository = _repository(database_url, "alice")
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
barrier = Barrier(2)
original_lock = PostgresSessionRepository._lock_session
def enter_lock_together(connection, locked_session_id):
if locked_session_id == session_id:
barrier.wait(timeout=5)
original_lock(connection, locked_session_id)
monkeypatch.setattr(
PostgresSessionRepository,
"_lock_session",
staticmethod(enter_lock_together),
)
def append(subject: str):
return repository.append_decisions(
session_id, [DecisionInput(type="concept_clarified", subject=subject)]
)
with ThreadPoolExecutor(max_workers=2) as pool:
first, second = pool.map(append, ("first", "second"))
snapshot = repository.get(session_id)
assert {first[0].seq, second[0].seq} == {1, 2}
assert [record.seq for record in snapshot.decisions] == [1, 2]
assert {record.subject for record in snapshot.decisions} == {"first", "second"}
def test_non_superuser_runtime_login_can_assume_the_restricted_runtime_role(database_url):
admin = create_engine(database_url)
runtime_url = admin.url.set(
username="thoth_sessions_test_login", password="runtime-test-only"
)
try:
with admin.begin() as connection:
connection.exec_driver_sql(
"CREATE ROLE thoth_sessions_test_login LOGIN NOINHERIT PASSWORD 'runtime-test-only'"
)
connection.exec_driver_sql("GRANT thoth_sessions_runtime TO thoth_sessions_test_login")
repository = _repository(
runtime_url.render_as_string(hide_password=False), "runtime-user"
)
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
assert repository.get(session_id).manifest.id == session_id
repository.close()
finally:
with admin.begin() as connection:
connection.exec_driver_sql("DROP ROLE IF EXISTS thoth_sessions_test_login")
admin.dispose()
def test_delete_cascades_content_and_leaves_content_free_tombstone(database_url):
repository = _repository(database_url, "alice")
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
repository.write_artifact(session_id, "sql_final", "SELECT confidential_value")
repository.append_decisions(
session_id,
[DecisionInput(type="sql_approved", subject="phase:7", detail="SELECT confidential_value")],
)
repository.delete(session_id)
with pytest.raises(SessionError, match="Sessione non trovata"):
repository.get(session_id)
engine = create_engine(database_url)
try:
with engine.connect() as connection:
assert connection.execute(
text("SELECT count(*) FROM thoth_sessions.session_artifacts WHERE session_id = :session_id"),
{"session_id": session_id},
).scalar_one() == 0
assert connection.execute(
text("SELECT count(*) FROM thoth_sessions.review_decisions WHERE session_id = :session_id"),
{"session_id": session_id},
).scalar_one() == 0
columns = connection.execute(
text(
"SELECT column_name FROM information_schema.columns "
"WHERE table_schema = 'thoth_sessions' AND table_name = 'audit_log'"
)
).scalars().all()
tombstone = connection.execute(
text(
"SELECT action, session_id, actor_issuer, actor_subject "
"FROM thoth_sessions.audit_log WHERE session_id = :session_id"
),
{"session_id": session_id},
).one()
finally:
engine.dispose()
assert (tombstone[0], str(tombstone[1]), *tombstone[2:]) == (
"session_deleted",
session_id,
"portal",
"alice",
)
assert not {"content", "artifact_content", "detail", "metadata"} & set(columns)
def test_session_schema_does_not_create_or_invoke_embeddings(database_url, monkeypatch):
import tht.session.postgres_repository as repository_module
monkeypatch.setattr(
repository_module,
"_embed",
lambda *_: pytest.fail("session persistence must not invoke embeddings"),
raising=False,
)
repository = _repository(database_url, "alice")
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
repository.write_artifact(session_id, "evidence", '{"sources": []}')
engine = create_engine(database_url)
try:
with engine.connect() as connection:
columns = connection.execute(
text(
"SELECT column_name FROM information_schema.columns "
"WHERE table_schema = 'thoth_sessions'"
)
).scalars().all()
finally:
engine.dispose()
assert all("embedding" not in column for column in columns)
harness/tests/test_promoted_columns_for.py
+5 -1
View File
@@ -3,9 +3,13 @@ from tht.cli.sql_cmd import promoted_columns_for
def test_promoted_columns_for(tmp_path):
sid = "sess1"
sid = "2026-07-16-120000"
sdir = tmp_path / sid
sdir.mkdir(parents=True)
(sdir / "session_manifest.yaml").write_text(
f"id: {sid}\nquestion: q\ndatabase: d\nschema: s\n"
"created_at: 2026-01-01T00:00:00+00:00\nstatus: open\n"
)
(sdir / "schema_linking.json").write_text(json.dumps({
"question": "q",
"candidates": [
harness/tests/test_repository_memory_sql_paths.py
+16
View File
@@ -0,0 +1,16 @@
"""Repository-only workflow commands must not recover server state via session_dir."""
import inspect
from tht.cli import memory_cmd, sql_cmd
def test_memory_workflow_commands_do_not_import_or_call_session_dir():
source = inspect.getsource(memory_cmd)
assert "session_dir" not in source
def test_sql_session_commands_read_snapshot_artifacts_not_session_paths():
source = inspect.getsource(sql_cmd)
assert "_session_sql_file" not in source
assert "session_dir" not in source
harness/tests/test_require_phase_or_exit.py
+7 -2
View File
@@ -16,8 +16,13 @@ def _make_session(tmp_path, current_phase_num: int) -> str:
approving phases 1..N-1 puts the session at phase N."""
import json
s = tmp_path / "sess"
session_id = "2026-07-16-120000"
s = tmp_path / session_id
s.mkdir()
(s / "session_manifest.yaml").write_text(
f"id: {session_id}\nquestion: q\ndatabase: d\nschema: s\n"
"created_at: 2026-01-01T00:00:00+00:00\nstatus: open\n"
)
decisions = [
{"seq": n, "type": "phase_approved", "subject": f"phase:{n}", "ts": "2025-01-01T00:00:00"}
for n in range(1, current_phase_num)
@@ -25,7 +30,7 @@ def _make_session(tmp_path, current_phase_num: int) -> str:
(s / "review_decisions.jsonl").write_text(
"\n".join(json.dumps(d) for d in decisions) + ("\n" if decisions else "")
)
return "sess"
return session_id
class _StubConfig:
harness/tests/test_session_migrate_cmd.py
+47
View File
@@ -0,0 +1,47 @@
import json
from testcontainers.postgres import PostgresContainer
from typer.testing import CliRunner
from tht.cli import app
def test_session_migrate_status_is_pristine_and_idempotent():
with PostgresContainer("postgres:16-alpine") as postgres:
database_url = postgres.get_connection_url()
runner = CliRunner()
before = runner.invoke(
app, ["session", "migrate", "--database-url", database_url, "--status", "--json"]
)
assert before.exit_code == 0, before.output
assert json.loads(before.stdout) == {"applied": [], "drifted": [], "pending": ["001", "002"]}
assert before.stderr == ""
first = runner.invoke(app, ["session", "migrate", "--database-url", database_url, "--json"])
second = runner.invoke(app, ["session", "migrate", "--database-url", database_url, "--json"])
expected = {"applied": ["001", "002"], "drifted": [], "pending": []}
assert first.exit_code == 0, first.output
assert second.exit_code == 0, second.output
assert json.loads(first.stdout) == expected
assert json.loads(second.stdout) == expected
def test_session_migrate_status_database_failure_is_pristine_json():
result = CliRunner().invoke(
app,
[
"session",
"migrate",
"--database-url",
"postgresql+psycopg2://test:test@127.0.0.1:1/test",
"--status",
"--json",
],
)
assert result.exit_code == 1
assert result.stderr == ""
assert "Traceback" not in result.stdout
assert json.loads(result.stdout)["error"]
harness/tests/test_session_repository.py
+173
View File
@@ -0,0 +1,173 @@
import uuid
import pytest
from tht.config import DatabaseConfig, load_config
from tht.decisions import DecisionInput
from tht.session.filesystem_repository import FilesystemSessionRepository
from tht.session.models import PrincipalContext, SessionManifest, local_principal
from tht.session.repository import build_session_repository, resolve_principal
from tht.session.store import SessionError
from tht.session.store import create_session
def _db() -> DatabaseConfig:
return DatabaseConfig(database="testdb", schema="public", user="u", password="p") # noqa: S106
def _config(tmp_path):
path = tmp_path / "workspace.yaml"
path.write_text(
"dwh:\n"
" type: postgres_direct\n"
" connection:\n"
" database: testdb\n"
" schema: public\n"
" user: user\n"
" password: secret\n"
)
return load_config(path)
def _manifest(session_id: str) -> SessionManifest:
return SessionManifest(
id=session_id,
created_at="2026-07-16T10:00:00Z",
question="Which patients had an ablation?",
database="testdb",
schema="public",
)
def test_new_sessions_use_uuid4_ids(tmp_path):
manifest = create_session("Which patients had an ablation?", _db(), tmp_path)
session_uuid = uuid.UUID(manifest.id, version=4)
assert str(session_uuid) == manifest.id
assert session_uuid.version == 4
def test_filesystem_repository_scopes_sessions_to_local_principal_root(tmp_path):
config = _config(tmp_path)
principal = PrincipalContext(issuer="local", subject="alice")
repository = build_session_repository(config, principal, home=tmp_path / "alice-home")
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
assert repository.root == tmp_path / "alice-home/workspaces/workspace/sessions"
assert (repository.root / session_id / "session_manifest.yaml").exists()
def test_filesystem_repository_reads_phase_artifacts_and_appends_decisions(tmp_path):
repository = FilesystemSessionRepository(
tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice")
)
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
repository.write_artifact(session_id, "sql_final", "SELECT 1")
repository.write_artifact(session_id, "evidence", '{"sources": []}')
decisions = repository.append_decisions(
session_id,
[DecisionInput(type="sql_approved", subject="phase:7", detail="SELECT 1")],
)
snapshot = repository.get(session_id)
assert repository.read_artifact(session_id, "sql_final") == "SELECT 1"
assert repository.read_artifact(session_id, "evidence") == '{"sources": []}'
assert snapshot.artifacts["sql_final"] == "SELECT 1"
assert snapshot.artifacts["evidence"] == '{"sources": []}'
assert [decision.type for decision in snapshot.decisions] == ["sql_approved"]
assert decisions[0].seq == 1
def test_tht_home_overrides_local_repository_root(monkeypatch, tmp_path):
monkeypatch.setenv("THT_HOME", str(tmp_path / "override"))
config = _config(tmp_path)
repository = build_session_repository(
config, PrincipalContext(issuer="local", subject="alice")
)
assert config.paths.sessions == tmp_path / "override/workspaces/workspace/sessions"
assert repository.root == tmp_path / "override/workspaces/workspace/sessions"
def test_filesystem_repository_keeps_preferences_per_principal(tmp_path):
home = tmp_path / "home"
alice = FilesystemSessionRepository(home, "demo", PrincipalContext(issuer="local", subject="alice"))
bob = FilesystemSessionRepository(home, "demo", PrincipalContext(issuer="local", subject="bob"))
alice.set_preferences({"model": "glm"})
assert alice.get_preferences() == {"model": "glm"}
assert bob.get_preferences() == {}
def test_two_local_homes_have_independent_identities_sessions_and_preferences(tmp_path):
alice_home = tmp_path / "alice-home"
bob_home = tmp_path / "bob-home"
alice = FilesystemSessionRepository(alice_home, "demo", local_principal(alice_home))
bob = FilesystemSessionRepository(bob_home, "demo", local_principal(bob_home))
session_id = str(uuid.uuid4())
alice.create(_manifest(session_id))
alice.set_preferences({"model": "glm"})
assert alice.principal.subject != bob.principal.subject
assert (alice.root / session_id / "session_manifest.yaml").exists()
assert not (bob.root / session_id).exists()
assert bob.list() == []
assert bob.get_preferences() == {}
with pytest.raises(SessionError, match="Sessione non trovata"):
bob.get(session_id)
@pytest.mark.parametrize("session_id", ["2026-01-01-000000-test", "2026-01-01-000000-x", "s1"])
def test_filesystem_repository_reads_safe_legacy_session_ids(tmp_path, session_id):
repository = FilesystemSessionRepository(
tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice")
)
directory = repository.root / session_id
directory.mkdir(parents=True)
_manifest(session_id).to_yaml(directory / "session_manifest.yaml")
assert repository.get(session_id).manifest.id == session_id
@pytest.mark.parametrize("session_id", ["..", "a/b", "/absolute", "space id"])
def test_filesystem_repository_rejects_unsafe_legacy_session_ids(tmp_path, session_id):
repository = FilesystemSessionRepository(
tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice")
)
with pytest.raises(SessionError):
repository.get(session_id)
def test_postgres_session_storage_requires_trusted_principal_environment(tmp_path, monkeypatch):
config = _config(tmp_path).model_copy(update={"session_storage": {
"type": "postgres_direct",
"connection": _db().model_dump(by_alias=True),
}})
monkeypatch.delenv("THT_PRINCIPAL_ISSUER", raising=False)
monkeypatch.delenv("THT_PRINCIPAL_SUBJECT", raising=False)
with pytest.raises(SessionError, match="THT_PRINCIPAL_ISSUER"):
resolve_principal(config)
def test_postgres_session_storage_uses_only_explicit_trusted_principal(tmp_path, monkeypatch):
config = _config(tmp_path).model_copy(update={"session_storage": {
"type": "postgres_direct",
"connection": _db().model_dump(by_alias=True),
}})
monkeypatch.setenv("THT_PRINCIPAL_ISSUER", "portal")
monkeypatch.setenv("THT_PRINCIPAL_SUBJECT", "alice")
monkeypatch.setenv("THT_PRINCIPAL_DISPLAY_NAME", "Alice")
monkeypatch.setenv("THT_PRINCIPAL_IS_ADMIN", "TRUE")
assert resolve_principal(config) == PrincipalContext(
issuer="portal", subject="alice", display_name="Alice", is_admin=True
)
harness/tests/test_session_repository_workflow.py
+60
View File
@@ -0,0 +1,60 @@
import uuid
from tht.decisions import DecisionInput
from tht.phase import current_phase, cte_plan, next_cte
from tht.session.filesystem_repository import FilesystemSessionRepository
from tht.session.models import PrincipalContext, SessionManifest
from tht.session.store import persist_verified_finalization
def _manifest(session_id: str) -> SessionManifest:
return SessionManifest(
id=session_id,
created_at="2026-07-16T10:00:00Z",
question="Which patients had an ablation?",
database="testdb",
schema="public",
)
def test_phase_helpers_fold_the_repository_snapshot_not_a_session_path(tmp_path):
repository = FilesystemSessionRepository(
tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice")
)
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
repository.write_artifact(session_id, "cte_plan", '["base_patients"]')
repository.append_decisions(
session_id,
[
DecisionInput(type="phase_approved", subject="phase:1"),
DecisionInput(type="phase_auto_approved", subject="phase:2"),
DecisionInput(type="cte_approved", subject="base_patients"),
],
)
snapshot = repository.get(session_id)
assert current_phase(snapshot) == 3
assert cte_plan(snapshot) == ["base_patients"]
assert next_cte(snapshot) is None
def test_verified_finalization_commits_report_evidence_and_status_through_repository(tmp_path):
repository = FilesystemSessionRepository(
tmp_path / "home", "demo", PrincipalContext(issuer="local", subject="alice")
)
session_id = str(uuid.uuid4())
repository.create(_manifest(session_id))
persist_verified_finalization(
repository,
session_id,
validation_report="# Validation\n\nverified against DWH\n",
evidence='[{"source":"review"}]\n',
)
snapshot = repository.get(session_id)
assert snapshot.manifest.status == "finalized"
assert snapshot.artifacts["validation_report"] == "# Validation\n\nverified against DWH\n"
assert snapshot.artifacts["evidence"] == '[{"source":"review"}]\n'
harness/tests/test_sql_preview_json.py
+2 -6
View File
@@ -108,16 +108,12 @@ def test_do_run_offset_zero_path_unchanged(monkeypatch):
def test_preview_session_no_file_resolves_sql_final(monkeypatch, tmp_path, capsys):
"""--session without a positional FILE resolves sql_final.sql via _session_sql_file."""
"""--session without a positional FILE resolves repository SQL text."""
from types import SimpleNamespace
from tht.cli import sql_cmd
sql_file = tmp_path / "sql_final.sql"
sql_file.write_text("SELECT session_resolved")
# Patch _session_sql_file to return our tmp file (no real workspace/DB needed).
monkeypatch.setattr(sql_cmd, "_session_sql_file", lambda cfg, sid: sql_file)
monkeypatch.setattr(sql_cmd, "_session_sql", lambda cfg, sid: "SELECT session_resolved")
captured_sql = {}
harness/tests/test_vector_migration_packaging.py
+5 -1
View File
@@ -6,7 +6,7 @@ import zipfile
from pathlib import Path
def test_built_wheel_installs_vector_migrations_and_discovers_cli(tmp_path):
def test_built_wheel_installs_migrations_and_discovers_cli(tmp_path):
harness = Path(__file__).parents[1]
wheelhouse = tmp_path / "wheelhouse"
target = tmp_path / "site"
@@ -33,6 +33,8 @@ def test_built_wheel_installs_vector_migrations_and_discovers_cli(tmp_path):
names = set(archive.namelist())
assert "tht/migrations/vector/001_extensions.sql" in names
assert "tht/migrations/vector/003_roles.sql" in names
assert "tht/migrations/sessions/001_schema.sql" in names
assert "tht/migrations/sessions/002_security.sql" in names
subprocess.run(
[sys.executable, "-m", "pip", "install", "--no-deps", "--target", str(target), wheel],
@@ -47,6 +49,8 @@ def test_built_wheel_installs_vector_migrations_and_discovers_cli(tmp_path):
"-c",
"from typer.testing import CliRunner; from tht.cli import app; "
"r=CliRunner().invoke(app, ['vector','migrate','--help']); "
"assert r.exit_code == 0, r.output; "
"r=CliRunner().invoke(app, ['session','migrate','--help']); "
"print(r.output); raise SystemExit(r.exit_code)",
],
env=env,
harness/tht/cli/cte_cmd.py
+50 -33
View File
@@ -6,7 +6,7 @@ import typer
from tht.cli.config_cmd import CONFIG_OPT
from tht.cli.schema_cmd import _load_config_or_exit
from tht.cli.session_cmd import load_session_or_exit, session_dir
from tht.cli.session_cmd import load_session_or_exit, load_snapshot_or_exit, session_repository
from tht.cli.sql_cmd import promoted_tables_for, require_action
_RULE6_HINT = (
@@ -17,6 +17,27 @@ _RULE6_HINT = (
cte_app = typer.Typer(help="Test controllato dei CTE proposti (Agent View Generation)")
@cte_app.command("save")
def save_cmd(
session: str = typer.Option(..., "--session"),
name: str = typer.Option(..., "--name"),
file: str = typer.Option(..., "--file", help="File SQL, oppure '-' per stdin."),
config: Path = CONFIG_OPT,
) -> None:
"""Persist one CTE SQL block through the configured session repository."""
import sys
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
raw = sys.stdin.read() if file == "-" else Path(file).read_text()
try:
session_repository(cfg).write_artifact(session, f"cte_sql:{name}", raw)
except ValueError as exc:
typer.secho(f"ERRORE: {exc}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1) from None
typer.secho(f"OK: CTE {name} salvato.", fg=typer.colors.GREEN)
@cte_app.command("test")
def test_cmd(
name: str = typer.Argument(..., help="Nome del CTE (file sessions/<id>/ctes/<nome>.sql)."),
@@ -33,7 +54,7 @@ def test_cmd(
CteError,
CteTestRecord,
_jsonable,
append_cte_test,
append_cte_test_snapshot,
build_test_sql,
has_trailing_select,
)
@@ -43,13 +64,13 @@ def test_cmd(
cfg = _load_config_or_exit(config)
require_action(cfg, "cte_test")
load_session_or_exit(cfg, session)
sdir = session_dir(cfg, session)
snapshot = load_snapshot_or_exit(cfg, session)
from tht.cli.phase_cmd import require_phase_or_exit
from tht.phase import next_cte
require_phase_or_exit(cfg, session, 6)
nxt = next_cte(sdir)
nxt = next_cte(snapshot)
if nxt is not None and name != nxt:
typer.secho(
f"ERRORE: ordine CTE. Ora tocca a '{nxt}' (il primo CTE del piano non "
@@ -60,11 +81,10 @@ def test_cmd(
)
raise typer.Exit(code=5)
cte_file = sdir / "ctes" / f"{name}.sql"
if not cte_file.exists():
typer.secho(f"ERRORE: file CTE non trovato: {cte_file}", fg=typer.colors.RED, err=True)
cte_sql = snapshot.artifacts.get(f"cte_sql:{name}")
if cte_sql is None:
typer.secho(f"ERRORE: file CTE non trovato: {name}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
cte_sql = cte_file.read_text()
sql_hash = hashlib.sha256(cte_sql.encode()).hexdigest()
def _record_error(message: str) -> None:
@@ -72,7 +92,7 @@ def test_cmd(
name=name, ts=datetime.now(UTC), sql_hash=sql_hash,
status="error", error=message,
)
append_cte_test(sdir, record)
append_cte_test_snapshot(session_repository(cfg), snapshot, record)
if json_out:
typer.echo(record.model_dump_json())
else:
@@ -118,7 +138,7 @@ def test_cmd(
execution_ms=result.execution_ms, warnings=warnings,
preview_rows=[[_jsonable(cell) for cell in row] for row in result.rows],
)
append_cte_test(sdir, record)
append_cte_test_snapshot(session_repository(cfg), snapshot, record)
if json_out:
typer.echo(record.model_dump_json())
@@ -133,7 +153,7 @@ def test_cmd(
Console().print(table)
for w in warnings:
typer.secho(f" warning: {w}", fg=typer.colors.YELLOW)
typer.secho(f"OK: esito registrato in {sdir / 'cte_tests.json'}", fg=typer.colors.GREEN)
typer.secho("OK: esito registrato in cte_tests.json", fg=typer.colors.GREEN)
CTE_PLAN_DOC_FILE = "cte_plan_doc.json"
@@ -156,13 +176,10 @@ def plan_cmd(
import json
import sys
from tht.phase import CTE_PLAN_FILE
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
from tht.cli.phase_cmd import require_phase_or_exit
require_phase_or_exit(cfg, session, 6)
sdir = session_dir(cfg, session)
doc_data = None
if doc is not None:
@@ -180,11 +197,11 @@ def plan_cmd(
)
raise typer.Exit(code=1)
sdir.mkdir(parents=True, exist_ok=True)
(sdir / CTE_PLAN_FILE).write_text(json.dumps(name, ensure_ascii=False))
repository = session_repository(cfg)
repository.write_artifact(session, "cte_plan", json.dumps(name, ensure_ascii=False))
if doc_data is not None:
(sdir / CTE_PLAN_DOC_FILE).write_text(json.dumps(doc_data, ensure_ascii=False))
typer.secho(f"OK: piano CTE salvato ({len(name)} CTE) in {sdir / CTE_PLAN_FILE}.",
repository.write_artifact(session, "cte_plan_doc", json.dumps(doc_data, ensure_ascii=False))
typer.secho(f"OK: piano CTE salvato ({len(name)} CTE).",
fg=typer.colors.GREEN)
@@ -201,7 +218,7 @@ def next_cmd(
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
nxt = next_cte(session_dir(cfg, session))
nxt = next_cte(load_snapshot_or_exit(cfg, session))
if nxt:
typer.echo(nxt)
@@ -222,9 +239,9 @@ def info_cmd(
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
sdir = session_dir(cfg, session)
snapshot = load_snapshot_or_exit(cfg, session)
plan = cte_plan(sdir)
plan = cte_plan(snapshot)
if not plan:
typer.secho(f"ERRORE: {CTE_PLAN_FILE} assente o vuoto per la sessione '{session}'.",
fg=typer.colors.RED, err=True)
@@ -234,24 +251,24 @@ def info_cmd(
fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
cte_file = sdir / "ctes" / f"{name}.sql"
if not cte_file.exists():
typer.secho(f"ERRORE: file CTE non trovato: {cte_file}", fg=typer.colors.RED, err=True)
cte_sql = snapshot.artifacts.get(f"cte_sql:{name}")
if cte_sql is None:
typer.secho(f"ERRORE: file CTE non trovato: {name}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
doc = None
doc_path = sdir / CTE_PLAN_DOC_FILE
if doc_path.exists():
full_doc = _json.loads(doc_path.read_text())
raw_doc = snapshot.artifacts.get("cte_plan_doc")
if raw_doc:
full_doc = _json.loads(raw_doc)
for c in full_doc.get("ctes", []):
if c.get("name") == name:
doc = {k: v for k, v in c.items() if k != "name"}
break
from tht.ctetest import CteError, load_cte_tests
from tht.ctetest import CteError, load_cte_tests_text
try:
records = [r for r in load_cte_tests(sdir) if r.name == name]
records = [r for r in load_cte_tests_text(snapshot.artifacts.get("cte_tests", "")) if r.name == name]
except CteError as e:
typer.secho(f"ERRORE: impossibile leggere cte_tests.json: {e}",
fg=typer.colors.RED, err=True)
@@ -263,8 +280,8 @@ def info_cmd(
"index": plan.index(name) + 1,
"total": len(plan),
"plan": plan,
"sql": cte_file.read_text(),
"approved": name in approved_ctes(sdir),
"sql": cte_sql,
"approved": name in approved_ctes(snapshot),
"doc": doc,
"last_test": last_test,
}
@@ -282,12 +299,12 @@ def list_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Ultimo esito registrato per ogni CTE della sessione."""
from tht.ctetest import CteError, load_cte_tests
from tht.ctetest import CteError, load_cte_tests_text
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
try:
records = load_cte_tests(session_dir(cfg, session))
records = load_cte_tests_text(load_snapshot_or_exit(cfg, session).artifacts.get("cte_tests", ""))
except CteError as e:
typer.secho(f"ERRORE: impossibile leggere cte_tests.json: {e}",
fg=typer.colors.RED, err=True)
harness/tht/cli/decision_cmd.py
+17 -21
View File
@@ -8,7 +8,7 @@ from pydantic import ValidationError
from tht.cli.config_cmd import CONFIG_OPT
from tht.cli.schema_cmd import _load_config_or_exit
from tht.cli.session_cmd import load_session_or_exit, session_dir
from tht.cli.session_cmd import load_session_or_exit, load_snapshot_or_exit, session_repository
decision_app = typer.Typer(help="Decisioni del reviewer (per sessione, append-only)")
@@ -20,7 +20,7 @@ def add_join_set_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Registra un insieme completo di join con un'unica sostituzione atomica del ledger."""
from tht.decisions import DecisionInput, append_decisions
from tht.decisions import DecisionInput
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
@@ -40,7 +40,7 @@ def add_join_set_cmd(
from tht.workflow import load_workflow
require_phase_or_exit(cfg, session, load_workflow().decision_min_phase("join_modified"))
records = append_decisions(session_dir(cfg, session), decisions)
records = session_repository(cfg).append_decisions(session, decisions)
typer.secho(
f"OK: registrato set atomico di {len(records)} join.",
fg=typer.colors.GREEN,
@@ -61,7 +61,7 @@ def add_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Registra una decisione del reviewer nella sessione."""
from tht.decisions import DecisionType, append_decision
from tht.decisions import DecisionType
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
@@ -82,7 +82,7 @@ def add_cmd(
from tht.workflow import load_workflow
require_phase_or_exit(cfg, session, load_workflow().decision_min_phase(type))
sdir = session_dir(cfg, session)
snapshot = load_snapshot_or_exit(cfg, session)
if type == "cte_approved":
# Un CTE si approva solo se appartiene al piano persistito. Senza piano
# (o con subject fuori piano) l'approvazione e' priva di significato:
@@ -90,7 +90,7 @@ def add_cmd(
# dove fu registrato un cte_approved:cte_plan senza alcun cte_plan.json.
from tht.phase import cte_plan
plan = cte_plan(sdir)
plan = cte_plan(snapshot)
if not plan:
typer.secho(
"ERRORE: nessun piano CTE (cte_plan.json) in sessione. Persisti prima "
@@ -105,10 +105,10 @@ def add_cmd(
fg=typer.colors.RED, err=True,
)
raise typer.Exit(code=5)
record = append_decision(
sdir, type=type, subject=subject,
detail=detail, rationale=rationale, retracts=retracts,
)
record = session_repository(cfg).append_decisions(session, [{
"type": type, "subject": subject, "detail": detail,
"rationale": rationale, "retracts": retracts,
}])[0]
typer.secho(f"OK: decisione [{record.seq}] {record.type}: {record.subject}",
fg=typer.colors.GREEN)
@@ -123,19 +123,18 @@ def retract_cmd(
Granularita' (a) del rollback §4.8: 'rispondi di nuovo a questa domanda'. Scrive un
marker decision_retracted (append-only, l'audit resta) che effective_decisions onora;
il widget corrente puo' essere riproposto. Non cambia la fase."""
from tht.decisions import append_decision
from tht.phase import effective_decisions
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
sdir = session_dir(cfg, session)
snapshot = load_snapshot_or_exit(cfg, session)
# Ultima decisione NON-meta della vista effective = quella associata al widget corrente.
meta = {
"phase_approved", "phase_auto_approved", "phase_reopened",
"phase_skipped", "decision_retracted",
}
substantive = [d for d in effective_decisions(sdir) if d.type not in meta]
substantive = [d for d in effective_decisions(snapshot) if d.type not in meta]
if not substantive:
typer.secho(
"Nessuna decisione sostanziale da ritirare nella fase corrente.",
@@ -143,10 +142,10 @@ def retract_cmd(
)
raise typer.Exit(code=6)
target = substantive[-1]
record = append_decision(
sdir, type="decision_retracted", subject=target.subject,
rationale=f"ritira [{target.seq}] {target.type}", retracts=target.seq,
)
record = session_repository(cfg).append_decisions(session, [{
"type": "decision_retracted", "subject": target.subject,
"rationale": f"ritira [{target.seq}] {target.type}", "retracts": target.seq,
}])[0]
typer.secho(
f"OK: ritirata decisione [{target.seq}] {target.type}: {target.subject} "
f"(marker #{record.seq}).",
@@ -160,11 +159,8 @@ def list_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Elenca le decisioni della sessione."""
from tht.decisions import list_decisions
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
decisions = list_decisions(session_dir(cfg, session))
decisions = load_snapshot_or_exit(cfg, session).decisions
if not decisions:
typer.echo("Nessuna decisione registrata.")
return
harness/tht/cli/memory_cmd.py
+14 -24
View File
@@ -17,7 +17,7 @@ from tht.cli._guards import (
require_server_profile,
require_vector_write_allowed,
)
from tht.cli.session_cmd import load_session_or_exit, session_dir
from tht.cli.session_cmd import load_snapshot_or_exit
from tht.cli.vector_cmd import require_vector_cfg
memory_app = typer.Typer(help="Review memory (registro canonico + indice pgvector)")
@@ -48,18 +48,17 @@ def promote_cmd(
"""Promuove le decisioni SCELTE nel registro globale. Usa --preview per vedere i candidati."""
import json as _json
from tht.memory import promote
from tht.memory import promote_snapshot
cfg = _load_config_or_exit(config)
manifest = load_session_or_exit(cfg, session)
snapshot = load_snapshot_or_exit(cfg, session)
if preview:
from tht.memory import (
MAX_PROMOTION_CANDIDATES, preview_promotions, reusable_promotions,
MAX_PROMOTION_CANDIDATES, preview_promotions_snapshot, reusable_promotions_snapshot,
)
sdir = session_dir(cfg, session)
cand = preview_promotions(sdir, manifest, registry_path(cfg))
extra = len(reusable_promotions(sdir, manifest, registry_path(cfg))) - len(cand)
cand = preview_promotions_snapshot(snapshot, registry_path(cfg))
extra = len(reusable_promotions_snapshot(snapshot, registry_path(cfg))) - len(cand)
payload = [
{"decision_seq": c.decision_seq, "type": c.type, "subject": c.subject,
"detail": c.detail, "rationale": c.rationale,
@@ -89,10 +88,7 @@ def promote_cmd(
require_server_profile(cfg, "memory promote")
require_vector_cfg(cfg)
promoted = promote(
session_dir(cfg, session), manifest,
seqs=list(decision), registry_path=registry_path(cfg),
)
promoted = promote_snapshot(snapshot, seqs=list(decision), registry_path=registry_path(cfg))
if not promoted:
msg = "Nessuna nuova promozione (gia' presenti o seq inesistenti)."
if json_out:
@@ -155,18 +151,17 @@ def save_one_cmd(
from tht.adapters.factory import build_vector_store
from tht.cli.vector_cmd import make_embedder
from tht.memory import load_registry, promote, save_one_memory
from tht.memory import load_registry, promote_snapshot, save_one_memory
cfg = _load_config_or_exit(config)
manifest = load_session_or_exit(cfg, session)
snapshot = load_snapshot_or_exit(cfg, session)
require_vector_write_allowed(cfg, "memory save-one")
store = build_vector_store(cfg, require_write=True)
sdir = session_dir(cfg, session)
# Promuove la decisione scelta nel registro locale (idempotente: salta se gia' presente
# o se stale post-rollback, perche' _compute_promotions usa la vista effective).
promote(sdir, manifest, seqs=[decision], registry_path=registry_path(cfg))
records = [r for r in load_registry(registry_path(cfg)) if r.session_id == manifest.id]
promote_snapshot(snapshot, seqs=[decision], registry_path=registry_path(cfg))
records = [r for r in load_registry(registry_path(cfg)) if r.session_id == snapshot.manifest.id]
embedder = make_embedder(cfg.embeddings)
count = save_one_memory(records, decision, store=store, embedder=embedder)
@@ -388,16 +383,14 @@ def search_cmd(
from rich.console import Console
from rich.table import Table
from tht.cli.session_cmd import session_dir
from tht.cli.vector_cmd import make_embedder, open_searcher, require_vector_cfg
from tht.memory import decided_memory_ids, load_registry
from tht.decisions import list_decisions
cfg = _load_config_or_exit(config)
require_vector_cfg(cfg)
excluded: set[str] = set()
if session is not None:
excluded = decided_memory_ids(list_decisions(session_dir(cfg, session)))
excluded = decided_memory_ids(load_snapshot_or_exit(cfg, session).decisions)
searcher = open_searcher(cfg)
embedder = make_embedder(cfg.embeddings)
hits = searcher.search(embedder.embed_query(question), top_n=top, kinds=["memory"])
@@ -445,7 +438,7 @@ def index_solved_session(cfg, session_id: str) -> int:
from tht.adapters.factory import build_vector_store
from tht.cli.sql_cmd import promoted_tables_for
from tht.cli.vector_cmd import make_embedder
from tht.solved import build_solved_record, save_solved_question
from tht.solved import build_solved_snapshot, save_solved_question
if not has_vector_write_rest(cfg):
raise RuntimeError(
@@ -453,10 +446,7 @@ def index_solved_session(cfg, session_id: str) -> int:
"writer key configurata nel workspace yaml"
)
store = build_vector_store(cfg, require_write=True)
manifest = load_session_or_exit(cfg, session_id)
record = build_solved_record(
session_dir(cfg, session_id), manifest, promoted_tables_for(cfg, session_id)
)
record = build_solved_snapshot(load_snapshot_or_exit(cfg, session_id), promoted_tables_for(cfg, session_id))
return save_solved_question(
record,
store=store,
harness/tht/cli/phase_cmd.py
+27 -24
View File
@@ -8,7 +8,6 @@ commands. All read workflow facts from load_workflow() (no mirrored constants).
from __future__ import annotations
import json
from pathlib import Path
import typer
@@ -22,16 +21,12 @@ from tht.workflow import load_workflow
phase_app = typer.Typer(help="Fase del workflow HITL (gate di avanzamento/ritorno)")
def session_dir(cfg, session_id: str) -> Path:
"""Where a session's artifacts live. Mirror of session_cmd.session_dir (kept
here to avoid a circular import: session_cmd imports phase helpers too)."""
return cfg.paths.sessions / session_id
def require_phase_or_exit(cfg, session: str, min_phase: int) -> None:
"""Refuse with exit 1 if the session hasn't reached min_phase yet. The phase
name in the message comes from workflow.yaml (load_workflow), not a constant."""
cur = current_phase(session_dir(cfg, session))
from tht.cli.session_cmd import load_snapshot_or_exit
cur = current_phase(load_snapshot_or_exit(cfg, session))
if cur < min_phase:
wf = load_workflow()
nome = wf.phase_name(min_phase)
@@ -89,21 +84,24 @@ def advance_cmd(
),
) -> None:
"""Approva la fase corrente e passa alla successiva (persiste phase_approved)."""
from tht.decisions import append_decision
from tht.cli.session_cmd import load_snapshot_or_exit, session_repository
sdir = session_dir(_cfg(), session)
cur = current_phase(sdir)
cfg = _cfg()
snapshot = load_snapshot_or_exit(cfg, session)
cur = current_phase(snapshot)
wf = load_workflow()
if cur > wf.max_phase:
typer.secho("Sessione già alla fase terminale.", fg=typer.colors.YELLOW)
raise typer.Exit(0)
if auto:
if not auto_advance_eligible(sdir):
problems = advance_problems(sdir, cur)
if not auto_advance_eligible(snapshot):
problems = advance_problems(snapshot, cur)
for p in problems:
typer.echo(p)
raise typer.Exit(6) # needs human confirmation (gate contract)
append_decision(sdir, type="phase_approved", subject=f"phase:{cur}")
session_repository(cfg).append_decisions(
session, [{"type": "phase_approved", "subject": f"phase:{cur}"}]
)
typer.echo(f"Fase {cur} ({wf.phase_name(cur)}) approvata → Fase {cur + 1}.")
@@ -113,16 +111,21 @@ def reopen_cmd(
phase: int = typer.Option(..., "--phase", help="Fase a cui tornare (1..fase corrente -1)."),
) -> None:
"""Torna a una fase precedente (persiste phase_reopened + teardown artefatti)."""
from tht.decisions import append_decision
from tht.teardown import teardown_to_phase
from tht.cli.session_cmd import load_snapshot_or_exit, session_repository
sdir = session_dir(_cfg(), session)
cur = current_phase(sdir)
cfg = _cfg()
snapshot = load_snapshot_or_exit(cfg, session)
cur = current_phase(snapshot)
if phase < 1 or phase >= cur:
typer.secho(f"Target non valido (fase corrente {cur}).", fg=typer.colors.RED, err=True)
raise typer.Exit(1)
report = teardown_to_phase(sdir, target_phase=phase)
append_decision(sdir, type="phase_reopened", subject=f"phase:{phase}")
from tht.teardown import teardown_snapshot
repository = session_repository(cfg)
report = teardown_snapshot(repository, snapshot, phase)
repository.append_decisions(
session, [{"type": "phase_reopened", "subject": f"phase:{phase}"}]
)
for f in report.deleted_files:
typer.echo(f" eliminato artefatto: {f}")
typer.echo(f"Tornati alla Fase {phase} ({load_workflow().phase_name(phase)}).")
@@ -133,13 +136,13 @@ def show_cmd(
session: str = typer.Option(..., "--session"),
) -> None:
"""Mostra stato, fase corrente e ultime decisioni della sessione."""
from tht.decisions import list_decisions
from tht.cli.session_cmd import load_snapshot_or_exit
sdir = session_dir(_cfg(), session)
cur = current_phase(sdir)
snapshot = load_snapshot_or_exit(_cfg(), session)
cur = current_phase(snapshot)
wf = load_workflow()
typer.echo(f"Fase corrente: {cur}/{wf.max_phase} ({wf.phase_name(min(cur, wf.max_phase))})")
decisions = list_decisions(sdir)
decisions = snapshot.decisions
if decisions:
typer.echo(f"Decisioni registrate: {len(decisions)}")
for d in decisions[-5:]:
harness/tht/cli/search_cmd.py
+3 -4
View File
@@ -371,14 +371,13 @@ def pack_cmd(
md = "\n".join(md_lines) + "\n"
if session:
from tht.cli.session_cmd import load_session_or_exit, session_dir
from tht.cli.session_cmd import load_session_or_exit, session_repository
load_session_or_exit(cfg, session)
out = session_dir(cfg, session) / "retrieval_pack.md"
out.write_text(md)
session_repository(cfg).write_artifact(session, "retrieval_pack", md)
if not json_out:
typer.secho(
f"OK: retrieval pack scritto in {out} "
"OK: retrieval pack scritto "
f"({len(tables)} tabelle, {len(evidence)} evidence, {len(solved)} solved).",
fg=typer.colors.GREEN,
)
harness/tht/cli/session_cmd.py
+184 -89
View File
@@ -2,22 +2,117 @@ import json
from pathlib import Path
import typer
from sqlalchemy.exc import SQLAlchemyError
from tht.cli.config_cmd import CONFIG_OPT
from tht.cli.schema_cmd import _load_config_or_exit
session_app = typer.Typer(help="Sessioni (directory artefatti)")
preferences_app = typer.Typer(help="Preferenze private del principal corrente")
session_app.add_typer(preferences_app, name="preferences")
def session_repository(cfg):
"""Configured persistence boundary for every workflow command."""
from tht.session.repository import build_session_repository
return build_session_repository(cfg)
@preferences_app.command("get")
def preferences_get_cmd(
json_out: bool = typer.Option(False, "--json", help="Emetti JSON puro."),
config: Path = CONFIG_OPT,
) -> None:
"""Legge le preferenze private del principal risolto dal repository."""
cfg = _load_config_or_exit(config)
preferences = session_repository(cfg).get_preferences()
if json_out:
typer.echo(json.dumps(preferences, ensure_ascii=False, sort_keys=True))
else:
typer.echo(json.dumps(preferences, ensure_ascii=False, indent=2, sort_keys=True))
@preferences_app.command("set")
def preferences_set_cmd(
preferences_json: str = typer.Argument(..., help="Oggetto JSON delle preferenze."),
json_out: bool = typer.Option(False, "--json", help="Non emette testo al successo."),
config: Path = CONFIG_OPT,
) -> None:
"""Sostituisce le preferenze private del principal risolto dal repository."""
try:
preferences = json.loads(preferences_json)
except json.JSONDecodeError as exc:
typer.secho(f"ERRORE: preferenze JSON non valide: {exc}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=2) from None
if not isinstance(preferences, dict):
typer.secho("ERRORE: le preferenze devono essere un oggetto JSON", fg=typer.colors.RED, err=True)
raise typer.Exit(code=2)
cfg = _load_config_or_exit(config)
session_repository(cfg).set_preferences(preferences)
if json_out:
return
typer.secho("OK: preferenze aggiornate.", fg=typer.colors.GREEN)
def session_dir(cfg, session_id: str) -> Path:
"""Legacy path bridge for out-of-scope datamart/memory compatibility only.
Workflow session commands in this module use ``session_repository``; this
helper remains until those non-workflow consumers lose their path APIs.
"""
return cfg.paths.sessions / session_id
def load_session_or_exit(cfg, session_id: str):
from tht.session.store import SessionError, load_session
@session_app.command("migrate")
def migrate_cmd(
database_url: str = typer.Option(
..., "--database-url", envvar="THT_SESSIONS_ADMIN_URL", help="Admin PostgreSQL URL."
),
status_only: bool = typer.Option(False, "--status", help="Inspect without applying."),
json_output: bool = typer.Option(False, "--json", help="Emit pristine JSON."),
) -> None:
"""Apply or inspect the private PostgreSQL session schema migrations."""
from tht.session.postgres_repository import MigrationError, migrate, migration_status
try:
return load_session(session_id, cfg.paths.sessions)
status = migration_status(database_url) if status_only else migrate(database_url)
except (MigrationError, SQLAlchemyError, ValueError) as exc:
if json_output:
typer.echo(json.dumps({"error": str(exc)}, sort_keys=True))
else:
typer.echo(f"ERROR: {exc}", err=True)
raise typer.Exit(code=1) from None
payload = {
"applied": [item.version for item in status.applied],
"drifted": [item.version for item in status.drifted],
"pending": [item.version for item in status.pending],
}
if json_output:
typer.echo(json.dumps(payload, sort_keys=True))
else:
typer.echo(
f"Applied: {len(status.applied)}; pending: {len(status.pending)}; "
f"drifted: {len(status.drifted)}"
)
def load_session_or_exit(cfg, session_id: str):
from tht.session.store import SessionError
try:
return session_repository(cfg).get(session_id).manifest
except SessionError as e:
typer.secho(f"ERRORE: {e}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
def load_snapshot_or_exit(cfg, session_id: str):
from tht.session.store import SessionError
try:
return session_repository(cfg).get(session_id)
except SessionError as e:
typer.secho(f"ERRORE: {e}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
@@ -59,7 +154,14 @@ def list_cmd(
) -> None:
"""Elenca le sessioni esistenti."""
cfg = _load_config_or_exit(config)
rows = _list_sessions(cfg.paths.sessions)
rows = [
{"id": s.manifest.id, "status": s.manifest.status, "question": s.manifest.question,
"summary": s.manifest.summary, "created_at": s.manifest.created_at.isoformat(),
"updated_at": s.manifest.updated_at.isoformat() if s.manifest.updated_at else None,
"author": s.manifest.author, "name": s.manifest.name, "group": s.manifest.group,
"archived": s.manifest.archived}
for s in session_repository(cfg).list()
]
if json_out:
typer.echo(json.dumps(rows, ensure_ascii=False, indent=2))
return
@@ -78,16 +180,19 @@ def new_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Crea una sessione e stampa il suo id (ultima riga dell'output)."""
from tht.session.store import _extract_name, create_session
from tht.session.store import _extract_name, new_session_manifest, render_question_md
cfg = _load_config_or_exit(config)
manifest = create_session(question, cfg.database, cfg.paths.sessions,
manifest = new_session_manifest(question, cfg.database,
provider=provider, model=model, thinking=thinking,
name=name or _extract_name(question))
repository = session_repository(cfg)
repository.create(manifest)
repository.write_artifact(manifest.id, "question", render_question_md(question))
if json_out:
typer.echo(json.dumps({"id": manifest.id}, ensure_ascii=False))
return
typer.secho(f"OK: sessione creata in {session_dir(cfg, manifest.id)}", fg=typer.colors.GREEN)
typer.secho(f"OK: sessione creata ({manifest.id})", fg=typer.colors.GREEN)
typer.echo(manifest.id)
@@ -102,12 +207,12 @@ def set_question_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Scrive question.md (domanda riscritta + assunzioni) in modo deterministico."""
from tht.session.store import set_question
from tht.session.store import render_question_md
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
path = set_question(session_id, question, assumption or [], cfg.paths.sessions)
typer.secho(f"OK: question.md aggiornato ({path}).", fg=typer.colors.GREEN)
session_repository(cfg).write_artifact(session_id, "question", render_question_md(question, assumption or []))
typer.secho("OK: question.md aggiornato.", fg=typer.colors.GREEN)
@session_app.command("set-schema-linking")
@@ -123,7 +228,7 @@ def set_schema_linking_cmd(
from pydantic import ValidationError
from tht.session.store import set_schema_linking
from tht.session.models import SchemaLinking
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
@@ -134,11 +239,12 @@ def set_schema_linking_cmd(
typer.secho(f"ERRORE: JSON non valido: {e}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=5)
try:
path = set_schema_linking(session_id, data, cfg.paths.sessions)
model = SchemaLinking.model_validate(data)
session_repository(cfg).write_artifact(session_id, "schema_linking", json.dumps(model.model_dump(by_alias=True), indent=2, ensure_ascii=False))
except ValidationError as e:
typer.secho(f"ERRORE: schema_linking non valido:\n{e}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=5)
typer.secho(f"OK: schema_linking.json aggiornato ({path}).", fg=typer.colors.GREEN)
typer.secho("OK: schema_linking.json aggiornato.", fg=typer.colors.GREEN)
@session_app.command("sync-schema-linking")
@@ -149,16 +255,17 @@ def sync_schema_linking_cmd(
"""Riproietta schema_linking.json dalle decisioni F4 del ledger (deterministico)."""
from pydantic import ValidationError
from tht.session.store import sync_schema_linking
from tht.session.store import sync_schema_linking_snapshot
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
try:
path = sync_schema_linking(session_id, cfg.paths.sessions)
content = sync_schema_linking_snapshot(load_snapshot_or_exit(cfg, session_id))
session_repository(cfg).write_artifact(session_id, "schema_linking", content)
except ValidationError as e:
typer.secho(f"ERRORE: schema_linking non valido:\n{e}", fg=typer.colors.RED, err=True)
raise typer.Exit(code=5)
typer.secho(f"OK: schema_linking.json riproiettato ({path}).", fg=typer.colors.GREEN)
typer.secho("OK: schema_linking.json riproiettato.", fg=typer.colors.GREEN)
@session_app.command("show")
@@ -168,28 +275,27 @@ def show_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Stato della sessione: manifest + decisioni registrate (per la ripresa)."""
from tht.decisions import list_decisions
from tht.phase import current_phase
cfg = _load_config_or_exit(config)
manifest = load_session_or_exit(cfg, session_id)
sdir = session_dir(cfg, session_id)
snapshot = load_snapshot_or_exit(cfg, session_id)
if json_out:
has_schema_linking = (sdir / "schema_linking.json").exists()
phase = current_phase(sdir)
has_schema_linking = "schema_linking" in snapshot.artifacts
phase = current_phase(snapshot)
data = manifest.model_dump(mode="json", by_alias=True)
data["phase"] = phase
data["has_schema_linking"] = has_schema_linking
# Ledger integrale: il gate lo usa per costruire deterministicamente il
# recap delle decisioni nei riepiloghi di fase (v2).
data["decisions"] = [
d.model_dump(mode="json") for d in list_decisions(sdir)
d.model_dump(mode="json") for d in snapshot.decisions
]
typer.echo(json.dumps(data, ensure_ascii=False, indent=2))
return
decisions = list_decisions(sdir)
decisions = snapshot.decisions
typer.echo(f"id : {manifest.id}")
typer.echo(f"stato : {manifest.status}")
typer.echo(f"domanda : {manifest.question}")
@@ -197,8 +303,7 @@ def show_cmd(
typer.echo(f"decisioni: {len(decisions)}")
for d in decisions[-10:]:
typer.echo(f" [{d.seq}] {d.type}: {d.subject}" + (f" — {d.detail}" if d.detail else ""))
linking = sdir / "schema_linking.json"
typer.echo(f"schema_linking.json: {'presente' if linking.exists() else 'assente'}")
typer.echo(f"schema_linking.json: {'presente' if 'schema_linking' in snapshot.artifacts else 'assente'}")
@session_app.command("retrieval-pack")
@@ -209,12 +314,10 @@ def retrieval_pack_cmd(
"""Emette su stdout il retrieval pack persistito della sessione."""
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
path = session_dir(cfg, session_id) / "retrieval_pack.md"
try:
content = path.read_text()
except OSError as exc:
content = load_snapshot_or_exit(cfg, session_id).artifacts.get("retrieval_pack")
if content is None:
typer.secho(
f"ERRORE: retrieval pack non disponibile per la sessione {session_id}: {exc}",
f"ERRORE: retrieval pack non disponibile per la sessione {session_id}",
fg=typer.colors.RED,
err=True,
)
@@ -225,32 +328,32 @@ def retrieval_pack_cmd(
@session_app.command("close")
def close_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None:
"""Chiude la sessione (status=closed)."""
from tht.session.store import close_session
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
close_session(session_id, cfg.paths.sessions)
snapshot = load_snapshot_or_exit(cfg, session_id)
snapshot.manifest.status = "closed"
session_repository(cfg).save_manifest(snapshot.manifest)
typer.secho(f"OK: sessione {session_id} chiusa.", fg=typer.colors.GREEN)
@session_app.command("fail")
def fail_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None:
"""Registra un arresto del sistema non recuperabile automaticamente."""
from tht.session.store import fail_session
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
fail_session(session_id, cfg.paths.sessions)
snapshot = load_snapshot_or_exit(cfg, session_id)
snapshot.manifest.status = "failed"
session_repository(cfg).save_manifest(snapshot.manifest)
typer.secho(f"OK: sessione {session_id} marcata failed.", fg=typer.colors.RED)
@session_app.command("reopen")
def reopen_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None:
"""Riapre una sessione per una ripresa manuale."""
from tht.session.store import reopen_session
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
reopen_session(session_id, cfg.paths.sessions)
snapshot = load_snapshot_or_exit(cfg, session_id)
snapshot.manifest.status = "open"
session_repository(cfg).save_manifest(snapshot.manifest)
typer.secho(f"OK: sessione {session_id} riaperta.", fg=typer.colors.GREEN)
@@ -261,11 +364,11 @@ def set_name_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Imposta il nome descrittivo della sessione."""
from tht.session.store import set_name
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
set_name(session_id, name, cfg.paths.sessions)
snapshot = load_snapshot_or_exit(cfg, session_id)
snapshot.manifest.name = name.strip() or None
session_repository(cfg).save_manifest(snapshot.manifest)
typer.secho(f"OK: nome aggiornato per {session_id}.", fg=typer.colors.GREEN)
@@ -276,44 +379,42 @@ def set_group_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Sposta la sessione in un gruppo (o la toglie da ogni gruppo)."""
from tht.session.store import set_group
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
set_group(session_id, group, cfg.paths.sessions)
snapshot = load_snapshot_or_exit(cfg, session_id)
snapshot.manifest.group = group.strip() or None
session_repository(cfg).save_manifest(snapshot.manifest)
typer.secho(f"OK: gruppo aggiornato per {session_id}.", fg=typer.colors.GREEN)
@session_app.command("archive")
def archive_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None:
"""Archivia la sessione (la toglie dalla lista attiva, sola lettura)."""
from tht.session.store import set_archived
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
set_archived(session_id, True, cfg.paths.sessions)
snapshot = load_snapshot_or_exit(cfg, session_id)
snapshot.manifest.archived = True
session_repository(cfg).save_manifest(snapshot.manifest)
typer.secho(f"OK: sessione {session_id} archiviata.", fg=typer.colors.GREEN)
@session_app.command("unarchive")
def unarchive_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None:
"""Ripristina la sessione dall'archivio (non ne cambia la ripristinabilità)."""
from tht.session.store import set_archived
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
set_archived(session_id, False, cfg.paths.sessions)
snapshot = load_snapshot_or_exit(cfg, session_id)
snapshot.manifest.archived = False
session_repository(cfg).save_manifest(snapshot.manifest)
typer.secho(f"OK: sessione {session_id} ripristinata.", fg=typer.colors.GREEN)
@session_app.command("delete")
def delete_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT) -> None:
"""Elimina definitivamente la cartella di sessione."""
from tht.session.store import delete_session
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
delete_session(session_id, cfg.paths.sessions)
session_repository(cfg).delete(session_id)
typer.secho(f"OK: sessione {session_id} eliminata.", fg=typer.colors.GREEN)
@@ -324,11 +425,10 @@ def documents_cmd(
config: Path = CONFIG_OPT,
) -> None:
"""Documenti di sola lettura della sessione (domanda, rivista, schema, SQL, report, decisioni)."""
from tht.session.store import build_documents
from tht.session.store import build_snapshot_documents
cfg = _load_config_or_exit(config)
manifest = load_session_or_exit(cfg, session_id)
docs = build_documents(manifest, session_dir(cfg, session_id))
docs = build_snapshot_documents(load_snapshot_or_exit(cfg, session_id))
if json_out:
typer.echo(json.dumps(docs, ensure_ascii=False, indent=2))
return
@@ -342,19 +442,18 @@ def session_problems(cfg, session_id: str) -> list[str]:
from pydantic import ValidationError
from tht.decisions import list_decisions
from tht.session.models import SchemaLinking
sdir = session_dir(cfg, session_id)
snapshot = load_snapshot_or_exit(cfg, session_id)
problems: list[str] = []
if not list_decisions(sdir):
if not snapshot.decisions:
problems.append("nessuna decisione registrata (review_decisions.jsonl vuoto o assente)")
linking_path = sdir / "schema_linking.json"
if not linking_path.exists():
raw_linking = snapshot.artifacts.get("schema_linking")
if raw_linking is None:
problems.append("schema_linking.json assente")
else:
try:
SchemaLinking.model_validate(json.loads(linking_path.read_text()))
SchemaLinking.model_validate(json.loads(raw_linking))
except (json.JSONDecodeError, ValidationError) as e:
problems.append(f"schema_linking.json non valido: {e}")
return problems
@@ -368,7 +467,7 @@ def check_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OPT)
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session_id)
cur = current_phase(session_dir(cfg, session_id))
cur = current_phase(load_snapshot_or_exit(cfg, session_id))
wf = load_workflow()
schema_linking_phase = wf.schema_linking_phase()
if cur < schema_linking_phase:
@@ -409,7 +508,7 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP
do_run,
promoted_tables_for,
)
from tht.ctetest import CteError, load_cte_tests
from tht.ctetest import CteError, CteTestRecord, _iter_json_objects
from tht.execute import ExecutionError
from tht.execute.warnings import plan_warnings, runtime_warnings, static_warnings
from tht.report import extract_reviewer_notes, render_validation_report
@@ -420,13 +519,13 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP
from tht.sqlcheck import validate_sql
cfg = _load_config_or_exit(config)
manifest = load_session_or_exit(cfg, session_id)
sdir = session_dir(cfg, session_id)
repository = session_repository(cfg)
snapshot = load_snapshot_or_exit(cfg, session_id)
from tht.phase import current_phase
from tht.workflow import load_workflow
cur = current_phase(sdir)
cur = current_phase(snapshot)
wf = load_workflow()
if cur <= wf.max_phase:
typer.secho(
@@ -440,9 +539,9 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP
# --- gate di ingresso ---
# Vista effective (D15): un sql_approved/cte ritirato o stale post-rollback non conta.
problems = session_problems(cfg, session_id)
decisions = effective_decisions(sdir)
sql_file = sdir / "sql_final.sql"
if not sql_file.exists():
decisions = effective_decisions(snapshot)
sql = snapshot.artifacts.get("sql_final")
if sql is None:
problems.append("sql_final.sql assente")
if not any(d.type == "sql_approved" for d in decisions):
problems.append(
@@ -451,10 +550,11 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP
)
# I CTE richiesti sono quelli del PIANO effettivo, non i file glob su disco: un
# ctes/*.sql orfano lasciato da un teardown incompleto non deve bloccare il finalize.
plan = effective_cte_plan(sdir)
plan = effective_cte_plan(snapshot)
if plan:
try:
tested = {r.name for r in load_cte_tests(sdir)}
raw_tests = snapshot.artifacts.get("cte_tests", "")
tested = {r.name for r in map(CteTestRecord.model_validate, _iter_json_objects(raw_tests))}
except CteError as e:
typer.secho(
f"Finalize rifiutato: impossibile leggere cte_tests.json: {e}",
@@ -471,7 +571,7 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP
raise typer.Exit(code=3)
# --- batteria di validazione su sql_final.sql ---
sql = sql_file.read_text()
assert sql is not None
check = validate_sql(
sql,
physical=_load_physical_or_exit(cfg),
@@ -493,34 +593,29 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP
raise typer.Exit(code=3)
# --- validation_report.md (preservando le note del reviewer) ---
report_path = sdir / "validation_report.md"
existing_notes = (
extract_reviewer_notes(report_path.read_text()) if report_path.exists() else ""
)
report_path.write_text(render_validation_report(
existing_report = snapshot.artifacts.get("validation_report", "")
existing_notes = extract_reviewer_notes(existing_report) if existing_report else ""
report = render_validation_report(
session_id=session_id, check=check, plan=plan,
plan_warnings=plan_warnings(plan, cfg.execution),
result=result, runtime_warnings=runtime_warnings(result, cfg.execution),
static_warnings=static_warnings(check.ast), limit=limit,
reviewer_notes=existing_notes,
))
)
# --- evidence.json ---
linking = SchemaLinking.model_validate(
json.loads((sdir / "schema_linking.json").read_text())
json.loads(snapshot.artifacts["schema_linking"])
)
entries = build_evidence_entries(decisions, linking, cfg.paths.artifacts / "evidence")
(sdir / "evidence.json").write_text(json.dumps(entries, ensure_ascii=False, indent=2))
evidence = json.dumps(entries, ensure_ascii=False, indent=2)
# --- manifest + riepilogo ---
from datetime import UTC, datetime
from tht.session.store import persist_verified_finalization
from tht.session.store import MANIFEST, current_author
manifest.status = "finalized"
manifest.updated_at = datetime.now(UTC)
manifest.updated_by = current_author()
manifest.to_yaml(sdir / MANIFEST)
persist_verified_finalization(
repository, session_id, validation_report=report, evidence=evidence
)
# --- memoria attiva (parte B): indicizza la coppia domanda->SQL, best-effort ---
# Import lazy: memory_cmd importa da session_cmd (un import top-level qui sarebbe
# circolare). Qualunque errore (writer key assente, VPN giu', Ollama spento) NON
@@ -547,5 +642,5 @@ def finalize_cmd(session_id: str = typer.Argument(...), config: Path = CONFIG_OP
)
typer.secho(f"OK: sessione {session_id} finalizzata. Artefatti:", fg=typer.colors.GREEN)
for name in ARTIFACT_FILES:
state = "presente" if (sdir / name).exists() else "assente"
state = "presente" if name not in {"session_manifest.yaml", "review_decisions.jsonl"} else "persistito"
typer.echo(f" - {name}: {state}")
harness/tht/cli/sql_cmd.py
+38 -20
View File
@@ -42,12 +42,14 @@ def require_action(cfg, action: str) -> None:
def promoted_tables_for(cfg, session_id: str | None) -> set[str] | None:
if session_id is None:
return None
linking_path = cfg.paths.sessions / session_id / "schema_linking.json"
if not linking_path.exists():
from tht.cli.session_cmd import load_snapshot_or_exit
raw = load_snapshot_or_exit(cfg, session_id).artifacts.get("schema_linking")
if raw is None:
return None
from tht.session.models import SchemaLinking
linking = SchemaLinking.model_validate(json.loads(linking_path.read_text()))
linking = SchemaLinking.model_validate(json.loads(raw))
return {
c.name for c in linking.candidates
if c.kind == "table" and c.decision == "promoted"
@@ -59,12 +61,14 @@ def promoted_tables_for(cfg, session_id: str | None) -> set[str] | None:
def promoted_columns_for(cfg, session_id: str | None) -> set[str] | None:
if session_id is None:
return None
linking_path = cfg.paths.sessions / session_id / "schema_linking.json"
if not linking_path.exists():
from tht.cli.session_cmd import load_snapshot_or_exit
raw = load_snapshot_or_exit(cfg, session_id).artifacts.get("schema_linking")
if raw is None:
return None
from tht.session.models import SchemaLinking
linking = SchemaLinking.model_validate(json.loads(linking_path.read_text()))
linking = SchemaLinking.model_validate(json.loads(raw))
return {
c.name for c in linking.candidates
if c.kind == "column" and c.decision == "promoted"
@@ -182,7 +186,7 @@ def preview_cmd(
"""Esecuzione controllata con LIMIT iniettato; aggregati mostrati per interi.
Se FILE è omesso e --session è fornito, il file viene risolto automaticamente
come <workspace>/sessions/<session>/sql_final.sql (tramite _session_sql_file).
dall'artefatto `sql_final` del repository della sessione.
"""
from tht.execute import ExecutionError
@@ -195,8 +199,7 @@ def preview_cmd(
fg=typer.colors.RED, err=True,
)
raise typer.Exit(code=1)
resolved = _session_sql_file(cfg, session)
sql = resolved.read_text()
sql = _session_sql(cfg, session)
else:
sql = _read_sql(file)
check = validate_or_exit(cfg, sql, session)
@@ -247,15 +250,32 @@ def preview_cmd(
typer.secho(f" warning: {w}", fg=typer.colors.YELLOW)
def _session_sql_file(cfg, session_id: str) -> Path:
from tht.cli.session_cmd import load_session_or_exit, session_dir
def _session_sql(cfg, session_id: str) -> str:
from tht.cli.session_cmd import load_snapshot_or_exit
load_session_or_exit(cfg, session_id)
sql_file = session_dir(cfg, session_id) / "sql_final.sql"
if not sql_file.exists():
typer.secho(f"ERRORE: {sql_file} non trovato.", fg=typer.colors.RED, err=True)
sql = load_snapshot_or_exit(cfg, session_id).artifacts.get("sql_final")
if sql is None:
typer.secho("ERRORE: sql_final.sql non trovato.", fg=typer.colors.RED, err=True)
raise typer.Exit(code=1)
return sql_file
return sql
@sql_app.command("set-final")
def set_final_cmd(
session: str = typer.Option(..., "--session"),
file: str = typer.Option(..., "--file", help="File SQL, oppure '-' per stdin."),
config: Path = CONFIG_OPT,
) -> None:
"""Persist clean final SQL through the configured session repository."""
import sys
from tht.cli.session_cmd import load_session_or_exit, session_repository
cfg = _load_config_or_exit(config)
load_session_or_exit(cfg, session)
sql = sys.stdin.read() if file == "-" else _read_sql(Path(file))
session_repository(cfg).write_artifact(session, "sql_final", sql)
typer.secho("OK: SQL finale salvato.", fg=typer.colors.GREEN)
@sql_app.command("save")
@@ -266,9 +286,8 @@ def save_cmd(
) -> None:
"""Salva una copia di sql_final.sql nel percorso indicato (su richiesta esplicita)."""
cfg = _load_config_or_exit(config)
sql_file = _session_sql_file(cfg, session)
dest.parent.mkdir(parents=True, exist_ok=True)
dest.write_text(sql_file.read_text())
dest.write_text(_session_sql(cfg, session))
typer.secho(f"OK: SQL salvato in {dest}", fg=typer.colors.GREEN)
@@ -286,8 +305,7 @@ def export_cmd(
cfg = _load_config_or_exit(config)
require_action(cfg, "export")
sql_file = _session_sql_file(cfg, session)
sql = sql_file.read_text()
sql = _session_sql(cfg, session)
validate_or_exit(cfg, sql, session)
try:
result = do_run(cfg, sql, limit=cfg.execution.max_export_rows)
harness/tht/config.py
+25 -1
View File
@@ -16,6 +16,11 @@ class ConfigError(Exception):
"""Errore di configurazione, con messaggio leggibile per l'utente."""
def local_tht_home() -> Path:
"""Return the private local ThothII home, honoring the explicit override."""
return Path(os.environ.get("THT_HOME", "~/.thothii")).expanduser()
def _expand_env(value: Any) -> Any:
if isinstance(value, str):
@@ -73,6 +78,22 @@ class DatabaseConfig(BaseModel):
model_config = {"populate_by_name": True}
class SessionPostgresConfig(DatabaseConfig):
"""TLS-verified direct session storage; its login must be granted thoth_sessions_runtime.
Provision the dedicated LOGIN role and membership out of band so deployment credentials
never appear in the versioned migration pack.
"""
sslmode: Literal["verify-ca", "verify-full"] = "verify-full"
sslrootcert: Path | None = None
class SessionStorageConfig(BaseModel):
type: Literal["postgres_direct"]
connection: SessionPostgresConfig
class RestConfig(BaseModel):
"""Accesso al DWH via Supabase/PostgREST. base_url es. https://host/dwh/ ."""
@@ -287,6 +308,7 @@ class Config(BaseModel):
_config_source: str = PrivateAttr(default="direct")
dwh: DwhResourceConfig
vectors: VectorResourceConfig | None = None
session_storage: SessionStorageConfig | None = None
roots: WorkspaceRoots = WorkspaceRoots()
# Compatibility views retained until all call sites consume typed resources.
database: DatabaseConfig
@@ -356,7 +378,9 @@ def load_config(path: Path) -> Config:
raise ConfigError(
f"transport: rest richiede la sezione `rest` (base_url, api_key) in {path}."
)
data_root = os.environ.get("THT_DATA_ROOT")
# THT_HOME is the local-user storage root. Retain THT_DATA_ROOT as the
# portable-deployment compatibility name until all callers use repositories.
data_root = os.environ.get("THT_HOME") or os.environ.get("THT_DATA_ROOT")
if data_root:
# Import locally: paths owns resolution, while ConfigError remains the public
# configuration exception callers already handle.
harness/tht/ctetest.py
+9 -1
View File
@@ -99,7 +99,10 @@ def load_cte_tests(session_dir: Path) -> list[CteTestRecord]:
path = session_dir / CTE_TESTS_FILE
if not path.exists():
return []
text = path.read_text()
return load_cte_tests_text(path.read_text())
def load_cte_tests_text(text: str) -> list[CteTestRecord]:
try:
objs = list(_iter_json_objects(text))
except json.JSONDecodeError as e:
@@ -107,6 +110,11 @@ def load_cte_tests(session_dir: Path) -> list[CteTestRecord]:
return [CteTestRecord.model_validate(o) for o in objs]
def append_cte_test_snapshot(repository, snapshot, record: CteTestRecord) -> None:
previous = snapshot.artifacts.get("cte_tests", "")
repository.write_artifact(snapshot.manifest.id, "cte_tests", previous + record.model_dump_json() + "\n")
def append_cte_test(session_dir: Path, record: CteTestRecord) -> None:
"""Append atomico (una riga JSON per esito): scritture concorrenti sulla
stessa sessione non si sovrascrivono, a differenza del rewrite dell'array."""
harness/tht/jobs/dwh_pipeline.py
+13 -1
View File
@@ -48,9 +48,21 @@ def config_dwh_binding(cfg) -> dict[str, str]:
config_source = getattr(cfg, "_config_source", None)
if not isinstance(workspace_id, str) or not isinstance(config_source, str):
raise CorruptCheckpointError("DWH workspace identity is unavailable; reload configuration")
model_dump = getattr(cfg, "model_dump", None)
if callable(model_dump):
payload = model_dump(mode="json")
if not isinstance(payload, dict):
raise CorruptCheckpointError("DWH workspace configuration is unavailable; reload configuration")
# Session persistence has no bearing on schema/LSH artifacts. Excluding it keeps an
# opt-in session-storage deployment from invalidating an otherwise identical DWH cache.
payload.pop("session_storage", None)
config_fingerprint = fingerprint(json.dumps(payload, separators=(",", ":"), ensure_ascii=False))
else:
# Lightweight test doubles predating Pydantic's model_dump() retain the legacy seam.
config_fingerprint = fingerprint(cfg.model_dump_json())
return {
"workspace_id": workspace_id,
"config_fingerprint": fingerprint(cfg.model_dump_json()),
"config_fingerprint": config_fingerprint,
"input_fingerprint": fingerprint(config_source),
}
harness/tht/memory.py
+20
View File
@@ -186,6 +186,14 @@ def promote(
return promoted
def promote_snapshot(snapshot, *, seqs: list[int] | None, registry_path: Path) -> list[MemoryRecord]:
existing = load_registry(registry_path)
promoted = _compute_promotions(snapshot, snapshot.manifest, seqs=seqs, existing=existing)
if promoted:
save_registry(existing + promoted, registry_path)
return promoted
def reusable_promotions(
session_dir: Path, manifest: SessionManifest, registry_path: Path
) -> list[MemoryRecord]:
@@ -203,6 +211,14 @@ def reusable_promotions(
]
def reusable_promotions_snapshot(snapshot, registry_path: Path) -> list[MemoryRecord]:
from tht.phase import effective_decisions
cand = _compute_promotions(snapshot, snapshot.manifest, seqs=None, existing=load_registry(registry_path))
declined = declined_promotion_seqs(effective_decisions(snapshot))
return [c for c in cand if c.type in REUSABLE_TYPES and c.decision_seq not in declined]
def preview_promotions(
session_dir: Path, manifest: SessionManifest, registry_path: Path
) -> list[MemoryRecord]:
@@ -212,6 +228,10 @@ def preview_promotions(
]
def preview_promotions_snapshot(snapshot, registry_path: Path) -> list[MemoryRecord]:
return reusable_promotions_snapshot(snapshot, registry_path)[:MAX_PROMOTION_CANDIDATES]
def memory_vector_records(records: list[MemoryRecord]) -> list[VectorRecord]:
out: list[VectorRecord] = []
for r in records:
harness/tht/migrations/sessions/001_schema.sql
+60
View File
@@ -0,0 +1,60 @@
CREATE SCHEMA IF NOT EXISTS thoth_sessions;
REVOKE ALL ON SCHEMA thoth_sessions FROM PUBLIC;
CREATE TABLE IF NOT EXISTS thoth_sessions.principals (
id bigserial PRIMARY KEY,
issuer text NOT NULL,
subject text NOT NULL,
display_name text,
created_at timestamptz NOT NULL DEFAULT pg_catalog.now(),
updated_at timestamptz NOT NULL DEFAULT pg_catalog.now(),
UNIQUE (issuer, subject)
);
CREATE TABLE IF NOT EXISTS thoth_sessions.principal_preferences (
principal_id bigint PRIMARY KEY REFERENCES thoth_sessions.principals(id) ON DELETE CASCADE,
preferences jsonb NOT NULL DEFAULT '{}'::jsonb,
updated_at timestamptz NOT NULL DEFAULT pg_catalog.now()
);
CREATE TABLE IF NOT EXISTS thoth_sessions.sessions (
id uuid PRIMARY KEY,
principal_id bigint NOT NULL REFERENCES thoth_sessions.principals(id),
manifest jsonb NOT NULL,
created_at timestamptz NOT NULL DEFAULT pg_catalog.now(),
updated_at timestamptz NOT NULL DEFAULT pg_catalog.now()
);
CREATE INDEX IF NOT EXISTS sessions_principal_id_created_at_idx
ON thoth_sessions.sessions (principal_id, created_at DESC);
CREATE TABLE IF NOT EXISTS thoth_sessions.session_artifacts (
session_id uuid NOT NULL REFERENCES thoth_sessions.sessions(id) ON DELETE CASCADE,
artifact_key text NOT NULL,
content text NOT NULL,
updated_at timestamptz NOT NULL DEFAULT pg_catalog.now(),
PRIMARY KEY (session_id, artifact_key)
);
CREATE TABLE IF NOT EXISTS thoth_sessions.review_decisions (
session_id uuid NOT NULL REFERENCES thoth_sessions.sessions(id) ON DELETE CASCADE,
seq integer NOT NULL CHECK (seq > 0),
ts timestamptz NOT NULL,
phase integer,
type text NOT NULL,
subject text NOT NULL,
detail text NOT NULL DEFAULT '',
rationale text NOT NULL DEFAULT '',
retracts integer,
PRIMARY KEY (session_id, seq)
);
CREATE TABLE IF NOT EXISTS thoth_sessions.audit_log (
id bigserial PRIMARY KEY,
occurred_at timestamptz NOT NULL DEFAULT pg_catalog.now(),
action text NOT NULL,
session_id uuid NOT NULL,
actor_issuer text NOT NULL,
actor_subject text NOT NULL,
owner_issuer text NOT NULL,
owner_subject text NOT NULL
);
harness/tht/migrations/sessions/002_security.sql
+126
View File
@@ -0,0 +1,126 @@
DO $roles$
BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = 'thoth_sessions_runtime') THEN
CREATE ROLE thoth_sessions_runtime NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT;
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = 'thoth_sessions_migrator') THEN
CREATE ROLE thoth_sessions_migrator NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT;
END IF;
END
$roles$;
ALTER ROLE thoth_sessions_runtime NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT;
ALTER ROLE thoth_sessions_migrator NOLOGIN NOBYPASSRLS NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT;
REVOKE ALL ON SCHEMA thoth_sessions FROM PUBLIC;
REVOKE ALL ON ALL TABLES IN SCHEMA thoth_sessions FROM PUBLIC;
REVOKE ALL ON ALL SEQUENCES IN SCHEMA thoth_sessions FROM PUBLIC;
REVOKE ALL ON SCHEMA thoth_sessions FROM thoth_sessions_runtime, thoth_sessions_migrator;
REVOKE ALL ON ALL TABLES IN SCHEMA thoth_sessions FROM thoth_sessions_runtime, thoth_sessions_migrator;
REVOKE ALL ON ALL SEQUENCES IN SCHEMA thoth_sessions FROM thoth_sessions_runtime, thoth_sessions_migrator;
GRANT USAGE ON SCHEMA thoth_sessions TO thoth_sessions_runtime;
GRANT SELECT, INSERT, UPDATE ON thoth_sessions.principals TO thoth_sessions_runtime;
GRANT SELECT, INSERT, UPDATE ON thoth_sessions.principal_preferences TO thoth_sessions_runtime;
GRANT SELECT, INSERT, UPDATE, DELETE ON thoth_sessions.sessions TO thoth_sessions_runtime;
GRANT SELECT, INSERT, UPDATE ON thoth_sessions.session_artifacts TO thoth_sessions_runtime;
GRANT SELECT, INSERT ON thoth_sessions.review_decisions TO thoth_sessions_runtime;
GRANT SELECT, INSERT ON thoth_sessions.audit_log TO thoth_sessions_runtime;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA thoth_sessions TO thoth_sessions_runtime;
ALTER TABLE thoth_sessions.principals ENABLE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.principals FORCE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.principal_preferences ENABLE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.principal_preferences FORCE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.sessions ENABLE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.sessions FORCE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.session_artifacts ENABLE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.session_artifacts FORCE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.review_decisions ENABLE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.review_decisions FORCE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.audit_log ENABLE ROW LEVEL SECURITY;
ALTER TABLE thoth_sessions.audit_log FORCE ROW LEVEL SECURITY;
CREATE POLICY principals_owner_or_admin ON thoth_sessions.principals
FOR ALL
USING (
pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'
OR (issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true)
AND subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true))
)
WITH CHECK (
pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'
OR (issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true)
AND subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true))
);
CREATE POLICY preferences_owner_or_admin ON thoth_sessions.principal_preferences
FOR ALL
USING (
EXISTS (
SELECT 1 FROM thoth_sessions.principals p
WHERE p.id = principal_preferences.principal_id
AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'
OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true)
AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true)))
)
)
WITH CHECK (
EXISTS (
SELECT 1 FROM thoth_sessions.principals p
WHERE p.id = principal_preferences.principal_id
AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'
OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true)
AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true)))
)
);
CREATE POLICY sessions_owner_or_admin ON thoth_sessions.sessions
FOR ALL
USING (
EXISTS (
SELECT 1 FROM thoth_sessions.principals p
WHERE p.id = sessions.principal_id
AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'
OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true)
AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true)))
)
)
WITH CHECK (
EXISTS (
SELECT 1 FROM thoth_sessions.principals p
WHERE p.id = sessions.principal_id
AND (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'
OR (p.issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true)
AND p.subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true)))
)
);
CREATE POLICY artifacts_owner_or_admin ON thoth_sessions.session_artifacts
FOR ALL
USING (
EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = session_artifacts.session_id)
)
WITH CHECK (
EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = session_artifacts.session_id)
);
CREATE POLICY decisions_owner_or_admin ON thoth_sessions.review_decisions
FOR ALL
USING (
EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = review_decisions.session_id)
)
WITH CHECK (
EXISTS (SELECT 1 FROM thoth_sessions.sessions s WHERE s.id = review_decisions.session_id)
);
CREATE POLICY audit_admin_read ON thoth_sessions.audit_log
FOR SELECT
USING (pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true');
CREATE POLICY audit_actor_write ON thoth_sessions.audit_log
FOR INSERT
WITH CHECK (
pg_catalog.current_setting('thoth_sessions.is_admin', true) = 'true'
OR (actor_issuer = pg_catalog.current_setting('thoth_sessions.actor_issuer', true)
AND actor_subject = pg_catalog.current_setting('thoth_sessions.actor_subject', true))
);
harness/tht/phase.py
+48 -37
View File
@@ -17,7 +17,7 @@ from pathlib import Path
from pydantic import ValidationError
from tht.decisions import DecisionRecord, list_decisions
from tht.session.models import SchemaLinking
from tht.session.models import SchemaLinking, SessionSnapshot
from tht.workflow import load_workflow
@@ -31,11 +31,22 @@ def _phase_num(subject: str) -> int | None:
return None
def _audit_excluding_retracted(session_dir: Path) -> list[DecisionRecord]:
def _decisions(source: Path | SessionSnapshot) -> list[DecisionRecord]:
return list(source.decisions) if isinstance(source, SessionSnapshot) else list_decisions(source)
def _artifact(source: Path | SessionSnapshot, key: str, filename: str) -> str | None:
if isinstance(source, SessionSnapshot):
return source.artifacts.get(key)
path = source / filename
return path.read_text() if path.exists() else None
def _audit_excluding_retracted(source: Path | SessionSnapshot) -> list[DecisionRecord]:
"""Tutto il ledger (append-only) tranne le decisioni ritirate e i marker di ritrazione.
Base per il fold di current_phase: il guard 'n == cur' del fold e' gia' reopen-aware
(una phase_approved:N dopo un reopen a M<N non fa avanzare perche' cur!=N)."""
all_d = list_decisions(session_dir)
all_d = _decisions(source)
retracted_seqs = {
d.retracts for d in all_d if d.type == "decision_retracted" and d.retracts is not None
}
@@ -46,7 +57,7 @@ def _audit_excluding_retracted(session_dir: Path) -> list[DecisionRecord]:
]
def current_phase(session_dir: Path) -> int:
def current_phase(source: Path | SessionSnapshot) -> int:
"""Fase corrente come fold cronologico sull'audit (con ritirate escluse).
cur parte da 1; ogni phase_approved/phase_auto_approved per la fase CORRENTE avanza
@@ -57,7 +68,7 @@ def current_phase(session_dir: Path) -> int:
wf = load_workflow()
max_plus_one = wf.max_phase + 1
cur = 1
for d in _audit_excluding_retracted(session_dir):
for d in _audit_excluding_retracted(source):
n = _phase_num(d.subject)
if n is None:
continue
@@ -68,7 +79,7 @@ def current_phase(session_dir: Path) -> int:
return cur
def effective_decisions(session_dir: Path) -> list[DecisionRecord]:
def effective_decisions(source: Path | SessionSnapshot) -> list[DecisionRecord]:
"""La vista canonica 'effective as of pointer'. TUTTI gli helper non-fold devono usare questa.
Semantica: una decisione e' effective se appartiene a una fase <= current_phase.
@@ -81,9 +92,9 @@ def effective_decisions(session_dir: Path) -> list[DecisionRecord]:
Inoltre esclude le decisioni ritirate (decision_retracted) e i marker stessi.
"""
cur = current_phase(session_dir)
cur = current_phase(source)
out: list[DecisionRecord] = []
for d in _audit_excluding_retracted(session_dir):
for d in _audit_excluding_retracted(source):
n = _phase_num(d.subject)
# Per le decisioni con subject "a nome" (es. cte_approved -> nome CTE, evidence_*
# -> id evidence) il subject non porta la fase: si usa la fase emittente registrata
@@ -106,9 +117,9 @@ _META_TYPES = frozenset(
)
def substantive_count_current_phase(session_dir: Path) -> int:
def substantive_count_current_phase(source: Path | SessionSnapshot) -> int:
"""Numero di decisioni sostanziali dall'ultimo confine di fase (vista effective)."""
decs = effective_decisions(session_dir)
decs = effective_decisions(source)
start = 0
for i, d in enumerate(decs):
if d.type in _BOUNDARY_TYPES:
@@ -116,14 +127,14 @@ def substantive_count_current_phase(session_dir: Path) -> int:
return sum(1 for d in decs[start:] if d.type not in _META_TYPES)
def auto_advance_eligible(session_dir: Path) -> bool:
def auto_advance_eligible(source: Path | SessionSnapshot) -> bool:
"""Vero sse la fase corrente puo' auto-avanzare (zero decisioni sostanziali + prereq ok)."""
cur = current_phase(session_dir)
cur = current_phase(source)
if cur not in _AUTO_ADVANCE_PHASES:
return False
if substantive_count_current_phase(session_dir) > 0:
if substantive_count_current_phase(source) > 0:
return False
return not advance_problems(session_dir, cur)
return not advance_problems(source, cur)
# --- CTE helpers (consultano effective_decisions) ---------------------------
@@ -131,21 +142,21 @@ def auto_advance_eligible(session_dir: Path) -> bool:
CTE_PLAN_FILE = "cte_plan.json"
def cte_plan(session_dir: Path) -> list[str]:
path = session_dir / CTE_PLAN_FILE
if not path.exists():
def cte_plan(source: Path | SessionSnapshot) -> list[str]:
raw = _artifact(source, "cte_plan", CTE_PLAN_FILE)
if raw is None:
return []
return json.loads(path.read_text())
return json.loads(raw)
def approved_ctes(session_dir: Path) -> set[str]:
def approved_ctes(source: Path | SessionSnapshot) -> set[str]:
"""Insieme dei CTE approvati, dalla vista effective (esclude stale post-reopen)."""
return {d.subject for d in effective_decisions(session_dir) if d.type == "cte_approved"}
return {d.subject for d in effective_decisions(source) if d.type == "cte_approved"}
def next_cte(session_dir: Path) -> str | None:
approved = approved_ctes(session_dir)
for name in cte_plan(session_dir):
def next_cte(source: Path | SessionSnapshot) -> str | None:
approved = approved_ctes(source)
for name in cte_plan(source):
if name not in approved:
return name
return None
@@ -153,18 +164,18 @@ def next_cte(session_dir: Path) -> str | None:
# --- advance_problems (ladder if-phase-N che consulta effective_decisions) ---
def _has_decision(session_dir: Path, type_: str) -> bool:
return any(d.type == type_ for d in effective_decisions(session_dir))
def _has_decision(source: Path | SessionSnapshot, type_: str) -> bool:
return any(d.type == type_ for d in effective_decisions(source))
def _has_decision_subject(session_dir: Path, type_: str, subject: str) -> bool:
def _has_decision_subject(source: Path | SessionSnapshot, type_: str, subject: str) -> bool:
return any(
d.type == type_ and d.subject == subject
for d in effective_decisions(session_dir)
for d in effective_decisions(source)
)
def advance_problems(session_dir: Path, phase: int) -> list[str]:
def advance_problems(source: Path | SessionSnapshot, phase: int) -> list[str]:
"""Prerequisiti minimi per chiudere `phase` (lista vuota = ok).
Ladder if-phase-N (Strada 2): la logica specifica resta, ma ogni lettura passa per
@@ -172,19 +183,19 @@ def advance_problems(session_dir: Path, phase: int) -> list[str]:
l'evaluator generico (F2 pieno) entra in un secondo momento.
"""
problems: list[str] = []
if phase == 3 and not _has_decision(session_dir, "question_rewritten"):
if phase == 3 and not _has_decision(source, "question_rewritten"):
problems.append("manca la decisione question_rewritten (Fase 3)")
if phase == 5:
path = session_dir / "schema_linking.json"
if not path.exists():
raw = _artifact(source, "schema_linking", "schema_linking.json")
if raw is None:
problems.append("schema_linking.json assente (Fase 5)")
else:
try:
SchemaLinking.model_validate(json.loads(path.read_text()))
SchemaLinking.model_validate(json.loads(raw))
except (json.JSONDecodeError, ValidationError) as e:
problems.append(f"schema_linking.json non valido (Fase 5): {e}")
if phase == 6 and not _has_decision_subject(session_dir, "phase_skipped", "phase:6"):
plan = cte_plan(session_dir)
if phase == 6 and not _has_decision_subject(source, "phase_skipped", "phase:6"):
plan = cte_plan(source)
if not plan:
problems.append(
"Fase 6: nessun piano CTE (cte_plan.json) e nessun salto esplicito. "
@@ -192,14 +203,14 @@ def advance_problems(session_dir: Path, phase: int) -> list[str]:
"registrando una decisione phase_skipped subject phase:6."
)
else:
nc = next_cte(session_dir)
nc = next_cte(source)
if nc is not None:
problems.append(f"CTE non ancora approvato: {nc} (Fase 6)")
if phase == 7 and not _has_decision(session_dir, "sql_approved"):
if phase == 7 and not _has_decision(source, "sql_approved"):
problems.append("manca la decisione sql_approved (Fase 7)")
if phase == 8 and not any(
d.type in ("datamart_requested", "datamart_declined")
for d in effective_decisions(session_dir)
for d in effective_decisions(source)
):
problems.append(
"Fase 8: nessuna risposta sulla generazione dbt del datamart "
harness/tht/session/filesystem_repository.py
+254
View File
@@ -0,0 +1,254 @@
"""Private local filesystem implementation of the session repository contract."""
from __future__ import annotations
import hashlib
import json
import os
import re
import shutil
import tempfile
import uuid
from pathlib import Path
from typing import Sequence
import portalocker
import yaml
from tht.decisions import DecisionInput, DecisionRecord, append_decisions, list_decisions
from tht.session.models import PrincipalContext, SessionManifest, SessionSnapshot
from tht.session.store import MANIFEST, SessionError
_ARTIFACT_FILES = {
"question": "question.md",
"schema_linking": "schema_linking.json",
"evidence": "evidence.json",
"sql_final": "sql_final.sql",
"validation_report": "validation_report.md",
"retrieval_pack": "retrieval_pack.md",
"cte_tests": "cte_tests.json",
"cte_plan": "cte_plan.json",
"cte_plan_doc": "cte_plan_doc.json",
}
_ARTIFACT_KEYS = {filename: key for key, filename in _ARTIFACT_FILES.items()}
_SAFE_CTE_NAME = re.compile(r"[A-Za-z0-9_-]+\Z")
_LEGACY_SESSION_ID = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,127}\Z")
class FilesystemSessionRepository:
"""Readable phase documents under one private local user's ThothII home."""
def __init__(
self, home: Path, workspace: str, principal: PrincipalContext, *, root: Path | None = None
):
self.home = Path(home).expanduser()
self.workspace = workspace
self.principal = principal
self.root = Path(root) if root is not None else self.home / "workspaces" / workspace / "sessions"
@property
def preferences_path(self) -> Path:
digest = hashlib.sha256(
f"{self.principal.issuer}\0{self.principal.subject}".encode()
).hexdigest()
return self.home / "principals" / digest / "preferences.json"
def create(self, manifest: SessionManifest) -> SessionSnapshot:
# New records are opaque UUIDv4 only. Historical timestamp ids remain
# readable/mutable through _session_dir but are never created here.
self._require_uuid4(manifest.id)
session_dir = self.root / manifest.id
with self._lock(session_dir):
if (session_dir / MANIFEST).exists():
raise SessionError(f"Sessione esiste gia': {manifest.id}")
self._private_directory(session_dir)
self._write_manifest(session_dir, manifest)
return self.get(manifest.id)
def get(self, session_id: str) -> SessionSnapshot:
session_dir = self._session_dir(session_id)
manifest_path = session_dir / MANIFEST
if not manifest_path.exists():
raise SessionError(f"Sessione non trovata: {session_id} (atteso {manifest_path})")
manifest = SessionManifest.from_yaml(manifest_path)
artifacts = self._read_artifacts(session_dir)
return SessionSnapshot(
principal=self.principal,
manifest=manifest,
artifacts=artifacts,
decisions=list_decisions(session_dir),
)
def list(self) -> list[SessionSnapshot]:
if not self.root.exists():
return []
snapshots = []
for path in self.root.iterdir():
if path.is_dir() and (path / MANIFEST).exists():
try:
snapshots.append(self.get(path.name))
except SessionError:
continue
return sorted(snapshots, key=lambda item: item.manifest.created_at, reverse=True)
def save_manifest(self, manifest: SessionManifest) -> SessionSnapshot:
session_dir = self._session_dir(manifest.id)
with self._lock(session_dir):
self._require_existing(session_dir, manifest.id)
self._write_manifest(session_dir, manifest)
return self.get(manifest.id)
def read_artifact(self, session_id: str, key: str) -> str | None:
session_dir = self._session_dir(session_id)
self._require_existing(session_dir, session_id)
path = self._artifact_path(session_dir, key)
return path.read_text() if path.exists() else None
def write_artifact(self, session_id: str, key: str, content: str) -> None:
session_dir = self._session_dir(session_id)
with self._lock(session_dir):
self._require_existing(session_dir, session_id)
self._write_private_text(self._artifact_path(session_dir, key), content)
def delete_artifact(self, session_id: str, key: str) -> None:
session_dir = self._session_dir(session_id)
with self._lock(session_dir):
self._require_existing(session_dir, session_id)
self._artifact_path(session_dir, key).unlink(missing_ok=True)
def append_decisions(
self, session_id: str, decisions: Sequence[DecisionInput | dict]
) -> list[DecisionRecord]:
session_dir = self._session_dir(session_id)
with self._lock(session_dir):
self._require_existing(session_dir, session_id)
# The legacy ledger helper remains the canonical record format. Its
# separate lock also keeps direct workflow callers safe during Task 3.
return append_decisions(session_dir, list(decisions))
def finalize(self, manifest: SessionManifest, artifacts: dict[str, str]) -> SessionSnapshot:
"""Commit verified final artifacts before publishing finalized status.
The finalized manifest is the filesystem commit marker: a crash can leave
an open session with already-rendered artifacts, but never a finalized
session without its report and evidence.
"""
session_dir = self._session_dir(manifest.id)
with self._lock(session_dir):
self._require_existing(session_dir, manifest.id)
for key, content in artifacts.items():
self._write_private_text(self._artifact_path(session_dir, key), content)
self._write_manifest(session_dir, manifest)
return self.get(manifest.id)
def get_preferences(self) -> dict:
path = self.preferences_path
if not path.exists():
return {}
data = json.loads(path.read_text())
if not isinstance(data, dict):
raise ValueError(f"Invalid preferences: {path}")
return data
def set_preferences(self, preferences: dict) -> None:
if not isinstance(preferences, dict):
raise TypeError("preferences must be a dictionary")
path = self.preferences_path
self._private_directory(path.parent)
with portalocker.Lock(path.with_suffix(".lock"), mode="a+", timeout=10):
self._write_private_text(path, json.dumps(preferences, ensure_ascii=False, sort_keys=True) + "\n")
def delete(self, session_id: str) -> None:
session_dir = self._session_dir(session_id)
with self._lock(session_dir):
self._require_existing(session_dir, session_id)
shutil.rmtree(session_dir)
def _session_dir(self, session_id: str) -> Path:
self._require_session_id(session_id)
return self.root / session_id
@classmethod
def _require_session_id(cls, session_id: str) -> None:
try:
cls._require_uuid4(session_id)
except SessionError:
if session_id not in {".", ".."} and _LEGACY_SESSION_ID.fullmatch(session_id):
return
raise
@staticmethod
def _require_uuid4(session_id: str) -> None:
try:
parsed = uuid.UUID(session_id, version=4)
except ValueError as exc:
raise SessionError(f"ID sessione non UUIDv4: {session_id}") from exc
if str(parsed) != session_id or parsed.version != 4:
raise SessionError(f"ID sessione non UUIDv4: {session_id}")
@staticmethod
def _private_directory(path: Path) -> None:
path.mkdir(parents=True, exist_ok=True)
if os.name != "nt":
path.chmod(0o700)
def _lock(self, session_dir: Path):
self._private_directory(session_dir)
return portalocker.Lock(session_dir / ".repository.lock", mode="a+", timeout=10)
@staticmethod
def _require_existing(session_dir: Path, session_id: str) -> None:
if not (session_dir / MANIFEST).exists():
raise SessionError(f"Sessione non trovata: {session_id}")
def _artifact_path(self, session_dir: Path, key: str) -> Path:
if key in _ARTIFACT_FILES:
return session_dir / _ARTIFACT_FILES[key]
if key.startswith("cte_sql:"):
name = key.removeprefix("cte_sql:")
if _SAFE_CTE_NAME.fullmatch(name):
return session_dir / "ctes" / f"{name}.sql"
raise ValueError(f"Unsupported artifact key: {key}")
def _read_artifacts(self, session_dir: Path) -> dict[str, str]:
artifacts = {
key: (session_dir / filename).read_text()
for filename, key in _ARTIFACT_KEYS.items()
if (session_dir / filename).is_file()
}
ctes = session_dir / "ctes"
if ctes.is_dir():
artifacts.update(
{
f"cte_sql:{path.stem}": path.read_text()
for path in ctes.glob("*.sql")
if _SAFE_CTE_NAME.fullmatch(path.stem)
}
)
return artifacts
def _write_manifest(self, session_dir: Path, manifest: SessionManifest) -> None:
content = yaml.safe_dump(
manifest.model_dump(by_alias=True, mode="json", exclude_defaults=False),
sort_keys=False,
allow_unicode=True,
width=120,
)
self._write_private_text(session_dir / MANIFEST, content)
def _write_private_text(self, path: Path, content: str) -> None:
self._private_directory(path.parent)
fd, temporary_name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
temporary = Path(temporary_name)
try:
with os.fdopen(fd, "w") as handle:
handle.write(content)
handle.flush()
os.fsync(handle.fileno())
if os.name != "nt":
temporary.chmod(0o600)
os.replace(temporary, path)
if os.name != "nt":
path.chmod(0o600)
finally:
temporary.unlink(missing_ok=True)
harness/tht/session/models.py
+84
View File
@@ -1,10 +1,17 @@
import json
import os
import tempfile
import uuid
from datetime import datetime
from pathlib import Path
from typing import Literal, Self
import portalocker
import yaml
from pydantic import BaseModel, Field, ConfigDict
from tht.decisions import DecisionRecord
# Stub locale di _YamlModel. In the reference implementation questa base vive in
# mschema/models.py; qui la si replica perche' SessionManifest ha bisogno di
@@ -26,6 +33,74 @@ class _YamlModel(BaseModel):
return cls.model_validate(raw)
class PrincipalContext(BaseModel):
"""Trusted owner identity supplied by the runtime, never by a session document."""
issuer: str
subject: str
display_name: str | None = None
is_admin: bool = False
model_config = ConfigDict(frozen=True)
def _private_directory(path: Path) -> Path:
path.mkdir(parents=True, exist_ok=True)
if os.name != "nt":
path.chmod(0o700)
return path
def _write_private_json(path: Path, data: dict[str, str]) -> None:
fd, temporary_name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
temporary = Path(temporary_name)
try:
with os.fdopen(fd, "w") as handle:
json.dump(data, handle, sort_keys=True)
handle.write("\n")
handle.flush()
os.fsync(handle.fileno())
if os.name != "nt":
temporary.chmod(0o600)
os.replace(temporary, path)
if os.name != "nt":
path.chmod(0o600)
finally:
temporary.unlink(missing_ok=True)
def local_principal(home: Path | None = None) -> PrincipalContext:
"""Load or create the stable UUID identity for a local ThothII home."""
if home is None:
from tht.config import local_tht_home
home = local_tht_home()
identity_path = _private_directory(home) / "identity.json"
lock_path = identity_path.with_suffix(".lock")
with portalocker.Lock(lock_path, mode="a+", timeout=10):
if identity_path.exists():
try:
identity = json.loads(identity_path.read_text())
principal = PrincipalContext.model_validate(identity)
except (json.JSONDecodeError, OSError, ValueError) as exc:
raise ValueError(f"Invalid local identity: {identity_path}") from exc
if principal.issuer != "local":
raise ValueError(f"Invalid local identity issuer: {identity_path}")
try:
parsed = uuid.UUID(principal.subject, version=4)
except ValueError as exc:
raise ValueError(f"Invalid local identity subject: {identity_path}") from exc
if str(parsed) != principal.subject or parsed.version != 4:
raise ValueError(f"Invalid local identity subject: {identity_path}")
return principal
principal = PrincipalContext(issuer="local", subject=str(uuid.uuid4()))
_write_private_json(
identity_path, {"issuer": principal.issuer, "subject": principal.subject}
)
return principal
class SessionManifest(_YamlModel):
id: str
created_at: datetime
@@ -47,6 +122,15 @@ class SessionManifest(_YamlModel):
group: str | None = None
class SessionSnapshot(BaseModel):
"""The current persisted session state, excluding the non-persistent chat stream."""
principal: PrincipalContext | None = None
manifest: SessionManifest
artifacts: dict[str, str] = Field(default_factory=dict)
decisions: list[DecisionRecord] = Field(default_factory=list)
class Candidate(BaseModel):
kind: Literal["table", "column"]
name: str
harness/tht/session/postgres_repository.py
+549
View File
@@ -0,0 +1,549 @@
"""PostgreSQL implementation of the durable, user-owned session repository."""
from __future__ import annotations
import hashlib
import json
import re
import uuid
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import UTC, datetime
from importlib.resources import files
from importlib.resources.abc import Traversable
from pathlib import Path
from typing import Iterator, Sequence
from sqlalchemy import Engine, create_engine, text
from sqlalchemy.engine import URL, make_url
from sqlalchemy.exc import SQLAlchemyError
from tht.decisions import DecisionInput, DecisionRecord
from tht.session.models import PrincipalContext, SessionManifest, SessionSnapshot
from tht.session.store import SessionError
MIGRATIONS_DIR = files("tht").joinpath("migrations", "sessions")
_MIGRATION_NAME = re.compile(r"^(?P<version>\d+)_(?P<name>[a-z0-9_]+)\.sql$")
_SAFE_CTE_NAME = re.compile(r"[A-Za-z0-9_-]+\Z")
_ARTIFACT_KEYS = {
"question",
"schema_linking",
"evidence",
"sql_final",
"validation_report",
"retrieval_pack",
"cte_tests",
"cte_plan",
"cte_plan_doc",
}
_LOCK_KEY = 8_420_613_069_444_020_731
_RUNTIME_ROLE = "thoth_sessions_runtime"
class MigrationError(RuntimeError):
"""Raised when session schema migration discovery or application is unsafe."""
@dataclass(frozen=True)
class Migration:
version: str
name: str
path: Traversable
checksum: str
@dataclass(frozen=True)
class MigrationStatus:
applied: tuple[Migration, ...]
pending: tuple[Migration, ...]
drifted: tuple[Migration, ...]
def _migration_source(directory: Traversable | Path | str) -> Traversable:
return Path(directory) if isinstance(directory, (str, Path)) else directory
def _discover(directory: Traversable | Path | str = MIGRATIONS_DIR) -> tuple[Migration, ...]:
source = _migration_source(directory)
seen_versions: set[int] = set()
parsed = []
for path in (item for item in source.iterdir() if item.name.endswith(".sql")):
match = _MIGRATION_NAME.fullmatch(path.name)
if match is None:
raise MigrationError(f"Invalid migration filename: {path.name}")
version = match.group("version")
numeric_version = int(version)
if numeric_version in seen_versions:
raise MigrationError(f"Duplicate migration version: {numeric_version}")
seen_versions.add(numeric_version)
parsed.append((numeric_version, version, match.group("name"), path))
if not parsed:
raise MigrationError(f"No migrations found in {source}")
return tuple(
Migration(version, name, path, hashlib.sha256(path.read_bytes()).hexdigest())
for _, version, name, path in sorted(parsed)
)
def _engine(database_url: str) -> Engine:
url = make_url(database_url)
if not url.drivername.startswith("postgresql"):
raise ValueError("Session repository requires a direct PostgreSQL URL")
return create_engine(url)
def _applied(connection) -> dict[str, str]:
exists = connection.execute(
text("SELECT pg_catalog.to_regclass('public.tht_session_migrations')")
).scalar()
if exists is None:
return {}
return dict(
connection.execute(text("SELECT version, checksum FROM public.tht_session_migrations")).all()
)
def _reject_unknown_versions(
migrations: tuple[Migration, ...], applied_checksums: dict[str, str]
) -> None:
local_versions = {migration.version for migration in migrations}
unknown = sorted(
set(applied_checksums) - local_versions,
key=lambda value: (0, int(value)) if value.isdigit() else (1, value),
)
if unknown:
raise MigrationError("Database migration versions absent from local manifest: " + ", ".join(unknown))
def migration_status(
database_url: str, migrations_dir: Traversable | Path | str = MIGRATIONS_DIR
) -> MigrationStatus:
migrations = _discover(migrations_dir)
engine = _engine(database_url)
try:
with engine.connect() as connection:
connection.exec_driver_sql("SET LOCAL search_path = pg_catalog, pg_temp")
applied_checksums = _applied(connection)
finally:
engine.dispose()
_reject_unknown_versions(migrations, applied_checksums)
return MigrationStatus(
applied=tuple(item for item in migrations if applied_checksums.get(item.version) == item.checksum),
pending=tuple(item for item in migrations if item.version not in applied_checksums),
drifted=tuple(
item
for item in migrations
if item.version in applied_checksums and applied_checksums[item.version] != item.checksum
),
)
def migrate(
database_url: str, migrations_dir: Traversable | Path | str = MIGRATIONS_DIR
) -> MigrationStatus:
migrations = _discover(migrations_dir)
engine = _engine(database_url)
current: Migration | None = None
try:
with engine.begin() as connection:
connection.exec_driver_sql("SET LOCAL search_path = pg_catalog, pg_temp")
connection.execute(text("SELECT pg_catalog.pg_advisory_xact_lock(:key)"), {"key": _LOCK_KEY})
connection.exec_driver_sql(
"""CREATE TABLE IF NOT EXISTS public.tht_session_migrations (
version text PRIMARY KEY,
name text NOT NULL,
checksum text NOT NULL,
applied_at timestamptz NOT NULL DEFAULT pg_catalog.now()
)"""
)
connection.exec_driver_sql("REVOKE ALL ON public.tht_session_migrations FROM PUBLIC")
applied_checksums = _applied(connection)
_reject_unknown_versions(migrations, applied_checksums)
drifted = [
item
for item in migrations
if item.version in applied_checksums and applied_checksums[item.version] != item.checksum
]
if drifted:
raise MigrationError("Migration checksum drift: " + ", ".join(item.version for item in drifted))
for current in migrations:
if current.version in applied_checksums:
continue
connection.exec_driver_sql(current.path.read_text())
connection.execute(
text(
"INSERT INTO public.tht_session_migrations (version, name, checksum) "
"VALUES (:version, :name, :checksum)"
),
{"version": current.version, "name": current.name, "checksum": current.checksum},
)
except MigrationError:
raise
except SQLAlchemyError as exc:
filename = current.path.name if current is not None else "migration setup"
raise MigrationError(f"Failed to apply {filename}: {type(exc).__name__}") from exc
finally:
engine.dispose()
return migration_status(database_url, migrations_dir)
class PostgresSessionRepository:
"""Session data stored in private PostgreSQL tables under transaction-local RLS context."""
def __init__(
self,
database_url: str,
principal: PrincipalContext,
*,
engine: Engine | None = None,
runtime_role: str = _RUNTIME_ROLE,
):
self.principal = principal
self._engine = engine or _engine(database_url)
self._owns_engine = engine is None
self._runtime_role = runtime_role
@classmethod
def from_config(cls, config, principal: PrincipalContext) -> "PostgresSessionRepository":
query = {"sslmode": config.sslmode}
if config.sslrootcert is not None:
query["sslrootcert"] = str(config.sslrootcert)
url = URL.create(
"postgresql+psycopg2",
username=config.user,
password=config.password,
host=config.host,
port=config.port,
database=config.database,
query=query,
)
return cls(url.render_as_string(hide_password=False), principal)
def close(self) -> None:
if self._owns_engine:
self._engine.dispose()
self._owns_engine = False
def create(self, manifest: SessionManifest) -> SessionSnapshot:
self._require_uuid4(manifest.id)
with self._transaction() as connection:
principal_id = self._upsert_principal(connection)
try:
connection.execute(
text(
"INSERT INTO thoth_sessions.sessions (id, principal_id, manifest) "
"VALUES (:id, :principal_id, CAST(:manifest AS jsonb))"
),
{
"id": manifest.id,
"principal_id": principal_id,
"manifest": json.dumps(manifest.model_dump(by_alias=True, mode="json")),
},
)
except SQLAlchemyError as exc:
if "unique" in str(exc).lower():
raise SessionError(f"Sessione esiste gia': {manifest.id}") from exc
raise
return self.get(manifest.id)
def get(self, session_id: str) -> SessionSnapshot:
self._require_uuid4(session_id)
with self._transaction() as connection:
row = connection.execute(
text("SELECT manifest FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id}
).mappings().first()
if row is None:
raise SessionError(f"Sessione non trovata: {session_id}")
artifacts = dict(
connection.execute(
text(
"SELECT artifact_key, content FROM thoth_sessions.session_artifacts "
"WHERE session_id = :id ORDER BY artifact_key"
),
{"id": session_id},
).all()
)
decisions = [
DecisionRecord.model_validate(dict(item))
for item in connection.execute(
text(
"SELECT seq, ts, phase, type, subject, detail, rationale, retracts "
"FROM thoth_sessions.review_decisions WHERE session_id = :id ORDER BY seq"
),
{"id": session_id},
).mappings()
]
return SessionSnapshot(
principal=self.principal,
manifest=SessionManifest.model_validate(row["manifest"]),
artifacts=artifacts,
decisions=decisions,
)
def list(self) -> list[SessionSnapshot]:
with self._transaction() as connection:
ids = [row[0] for row in connection.execute(
text("SELECT id FROM thoth_sessions.sessions ORDER BY created_at DESC")
).all()]
# psycopg2 materializes PostgreSQL UUID columns as ``uuid.UUID`` objects,
# while the repository boundary intentionally accepts canonical UUIDv4 text.
return [self.get(str(session_id)) for session_id in ids]
def save_manifest(self, manifest: SessionManifest) -> SessionSnapshot:
self._require_uuid4(manifest.id)
with self._transaction() as connection:
result = connection.execute(
text(
"UPDATE thoth_sessions.sessions "
"SET manifest = CAST(:manifest AS jsonb), updated_at = pg_catalog.now() "
"WHERE id = :id"
),
{"id": manifest.id, "manifest": json.dumps(manifest.model_dump(by_alias=True, mode="json"))},
)
if result.rowcount != 1:
raise SessionError(f"Sessione non trovata: {manifest.id}")
return self.get(manifest.id)
def read_artifact(self, session_id: str, key: str) -> str | None:
self._require_uuid4(session_id)
self._require_artifact_key(key)
with self._transaction() as connection:
self._require_session(connection, session_id)
return connection.execute(
text(
"SELECT content FROM thoth_sessions.session_artifacts "
"WHERE session_id = :id AND artifact_key = :key"
),
{"id": session_id, "key": key},
).scalar()
def write_artifact(self, session_id: str, key: str, content: str) -> None:
self._require_uuid4(session_id)
self._require_artifact_key(key)
with self._transaction() as connection:
self._lock_session(connection, session_id)
self._require_session(connection, session_id)
connection.execute(
text(
"INSERT INTO thoth_sessions.session_artifacts (session_id, artifact_key, content) "
"VALUES (:id, :key, :content) "
"ON CONFLICT (session_id, artifact_key) DO UPDATE "
"SET content = EXCLUDED.content, updated_at = pg_catalog.now()"
),
{"id": session_id, "key": key, "content": content},
)
def delete_artifact(self, session_id: str, key: str) -> None:
self._require_uuid4(session_id)
self._require_artifact_key(key)
with self._transaction() as connection:
self._lock_session(connection, session_id)
self._require_session(connection, session_id)
connection.execute(
text("DELETE FROM thoth_sessions.session_artifacts WHERE session_id = :id AND artifact_key = :key"),
{"id": session_id, "key": key},
)
def append_decisions(
self, session_id: str, decisions: Sequence[DecisionInput | dict]
) -> list[DecisionRecord]:
self._require_uuid4(session_id)
inputs = [DecisionInput.model_validate(item) for item in decisions]
if not inputs:
return []
with self._transaction() as connection:
self._lock_session(connection, session_id)
self._require_session(connection, session_id)
from tht.phase import current_phase
manifest = SessionManifest.model_validate(connection.execute(
text("SELECT manifest FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id}
).scalar_one())
ledger = [
DecisionRecord.model_validate(dict(item))
for item in connection.execute(
text("SELECT seq, ts, phase, type, subject, detail, rationale, retracts "
"FROM thoth_sessions.review_decisions WHERE session_id = :id ORDER BY seq"),
{"id": session_id},
).mappings()
]
phase = current_phase(SessionSnapshot(manifest=manifest, decisions=ledger))
next_seq = connection.execute(
text(
"SELECT COALESCE(MAX(seq), 0) + 1 FROM thoth_sessions.review_decisions "
"WHERE session_id = :id"
),
{"id": session_id},
).scalar_one()
now = datetime.now(UTC)
records = [
DecisionRecord(seq=next_seq + offset, ts=now, phase=phase, **item.model_dump())
for offset, item in enumerate(inputs)
]
connection.execute(
text(
"INSERT INTO thoth_sessions.review_decisions "
"(session_id, seq, ts, phase, type, subject, detail, rationale, retracts) "
"VALUES (:session_id, :seq, :ts, :phase, :type, :subject, :detail, :rationale, :retracts)"
),
[
{"session_id": session_id, **record.model_dump(mode="python")}
for record in records
],
)
return records
def finalize(self, manifest: SessionManifest, artifacts: dict[str, str]) -> SessionSnapshot:
"""Atomically publish DWH-verified artifacts and the final manifest."""
self._require_uuid4(manifest.id)
for key in artifacts:
self._require_artifact_key(key)
with self._transaction() as connection:
self._lock_session(connection, manifest.id)
self._require_session(connection, manifest.id)
for key, content in artifacts.items():
connection.execute(
text(
"INSERT INTO thoth_sessions.session_artifacts (session_id, artifact_key, content) "
"VALUES (:id, :key, :content) "
"ON CONFLICT (session_id, artifact_key) DO UPDATE "
"SET content = EXCLUDED.content, updated_at = pg_catalog.now()"
),
{"id": manifest.id, "key": key, "content": content},
)
result = connection.execute(
text(
"UPDATE thoth_sessions.sessions "
"SET manifest = CAST(:manifest AS jsonb), updated_at = pg_catalog.now() "
"WHERE id = :id"
),
{"id": manifest.id, "manifest": json.dumps(manifest.model_dump(by_alias=True, mode="json"))},
)
if result.rowcount != 1:
raise SessionError(f"Sessione non trovata: {manifest.id}")
return self.get(manifest.id)
def get_preferences(self) -> dict:
with self._transaction() as connection:
principal_id = self._upsert_principal(connection)
preferences = connection.execute(
text(
"SELECT preferences FROM thoth_sessions.principal_preferences "
"WHERE principal_id = :principal_id"
),
{"principal_id": principal_id},
).scalar()
return preferences or {}
def set_preferences(self, preferences: dict) -> None:
if not isinstance(preferences, dict):
raise TypeError("preferences must be a dictionary")
with self._transaction() as connection:
principal_id = self._upsert_principal(connection)
connection.execute(
text(
"INSERT INTO thoth_sessions.principal_preferences (principal_id, preferences) "
"VALUES (:principal_id, CAST(:preferences AS jsonb)) "
"ON CONFLICT (principal_id) DO UPDATE SET preferences = EXCLUDED.preferences, "
"updated_at = pg_catalog.now()"
),
{"principal_id": principal_id, "preferences": json.dumps(preferences)},
)
def delete(self, session_id: str) -> None:
self._require_uuid4(session_id)
with self._transaction() as connection:
self._lock_session(connection, session_id)
owner = connection.execute(
text(
"SELECT p.issuer, p.subject FROM thoth_sessions.sessions s "
"JOIN thoth_sessions.principals p ON p.id = s.principal_id WHERE s.id = :id"
),
{"id": session_id},
).mappings().first()
if owner is None:
raise SessionError(f"Sessione non trovata: {session_id}")
connection.execute(
text(
"INSERT INTO thoth_sessions.audit_log "
"(action, session_id, actor_issuer, actor_subject, owner_issuer, owner_subject) "
"VALUES ('session_deleted', :id, :actor_issuer, :actor_subject, :owner_issuer, :owner_subject)"
),
{
"id": session_id,
"actor_issuer": self.principal.issuer,
"actor_subject": self.principal.subject,
"owner_issuer": owner["issuer"],
"owner_subject": owner["subject"],
},
)
connection.execute(text("DELETE FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id})
@contextmanager
def _transaction(self) -> Iterator:
try:
with self._engine.begin() as connection:
connection.exec_driver_sql("SET LOCAL search_path = thoth_sessions, pg_catalog, pg_temp")
connection.exec_driver_sql(f"SET LOCAL ROLE {self._runtime_role}")
connection.execute(
text("SELECT pg_catalog.set_config('thoth_sessions.actor_issuer', :value, true)"),
{"value": self.principal.issuer},
)
connection.execute(
text("SELECT pg_catalog.set_config('thoth_sessions.actor_subject', :value, true)"),
{"value": self.principal.subject},
)
connection.execute(
text("SELECT pg_catalog.set_config('thoth_sessions.is_admin', :value, true)"),
{"value": "true" if self.principal.is_admin else "false"},
)
yield connection
except SessionError:
raise
except SQLAlchemyError as exc:
raise SessionError("Session storage unavailable") from exc
def _upsert_principal(self, connection) -> int:
return connection.execute(
text(
"INSERT INTO thoth_sessions.principals (issuer, subject, display_name) "
"VALUES (:issuer, :subject, :display_name) "
"ON CONFLICT (issuer, subject) DO UPDATE SET display_name = EXCLUDED.display_name, "
"updated_at = pg_catalog.now() RETURNING id"
),
self.principal.model_dump(include={"issuer", "subject", "display_name"}),
).scalar_one()
@staticmethod
def _lock_session(connection, session_id: str) -> None:
connection.execute(
text("SELECT pg_catalog.pg_advisory_xact_lock(pg_catalog.hashtextextended(:id, 0))"),
{"id": session_id},
)
@staticmethod
def _require_session(connection, session_id: str) -> None:
if connection.execute(
text("SELECT 1 FROM thoth_sessions.sessions WHERE id = :id"), {"id": session_id}
).scalar() is None:
raise SessionError(f"Sessione non trovata: {session_id}")
@staticmethod
def _require_uuid4(session_id: str) -> None:
try:
parsed = uuid.UUID(session_id, version=4)
except ValueError as exc:
raise SessionError(f"ID sessione non UUIDv4: {session_id}") from exc
if str(parsed) != session_id or parsed.version != 4:
raise SessionError(f"ID sessione non UUIDv4: {session_id}")
@staticmethod
def _require_artifact_key(key: str) -> None:
if key in _ARTIFACT_KEYS:
return
if key.startswith("cte_sql:") and _SAFE_CTE_NAME.fullmatch(key.removeprefix("cte_sql:")):
return
raise ValueError(f"Unsupported artifact key: {key}")
__all__ = ["MigrationError", "MigrationStatus", "PostgresSessionRepository", "migrate", "migration_status"]
harness/tht/session/repository.py
+87
View File
@@ -0,0 +1,87 @@
"""Storage contract for the durable, user-owned session workflow state."""
from __future__ import annotations
import os
from typing import Protocol, Sequence
from tht.decisions import DecisionInput, DecisionRecord
from tht.session.models import PrincipalContext, SessionManifest, SessionSnapshot
from tht.session.store import SessionError
class SessionRepository(Protocol):
"""Persistence boundary shared by local files and the server database adapter."""
principal: PrincipalContext
def create(self, manifest: SessionManifest) -> SessionSnapshot: ...
def get(self, session_id: str) -> SessionSnapshot: ...
def list(self) -> list[SessionSnapshot]: ...
def save_manifest(self, manifest: SessionManifest) -> SessionSnapshot: ...
def read_artifact(self, session_id: str, key: str) -> str | None: ...
def write_artifact(self, session_id: str, key: str, content: str) -> None: ...
def delete_artifact(self, session_id: str, key: str) -> None: ...
def append_decisions(
self, session_id: str, decisions: Sequence[DecisionInput | dict]
) -> list[DecisionRecord]: ...
def finalize(self, manifest: SessionManifest, artifacts: dict[str, str]) -> SessionSnapshot: ...
def get_preferences(self) -> dict: ...
def set_preferences(self, preferences: dict) -> None: ...
def delete(self, session_id: str) -> None: ...
def resolve_principal(config) -> PrincipalContext:
"""Resolve the only principal source permitted for this CLI invocation.
The backend injects these values from its authenticated request context before
spawning ``tht``. A server-storage command without them must fail closed;
substituting a workstation identity would cross user ownership boundaries.
"""
if getattr(config, "session_storage", None) is None:
from tht.session.models import local_principal
return local_principal()
issuer = os.environ.get("THT_PRINCIPAL_ISSUER", "").strip()
subject = os.environ.get("THT_PRINCIPAL_SUBJECT", "").strip()
if not issuer or not subject:
raise SessionError(
"THT_PRINCIPAL_ISSUER e THT_PRINCIPAL_SUBJECT sono obbligatori per session storage server"
)
display_name = os.environ.get("THT_PRINCIPAL_DISPLAY_NAME", "").strip() or None
is_admin = os.environ.get("THT_PRINCIPAL_IS_ADMIN", "").strip().lower() in {"1", "true"}
return PrincipalContext(
issuer=issuer, subject=subject, display_name=display_name, is_admin=is_admin
)
def build_session_repository(
config, principal: PrincipalContext | None = None, *, home=None
) -> SessionRepository:
"""Build the configured private session persistence adapter."""
principal = principal or resolve_principal(config)
session_storage = getattr(config, "session_storage", None)
if session_storage is not None:
from tht.session.postgres_repository import PostgresSessionRepository
return PostgresSessionRepository.from_config(session_storage.connection, principal)
from tht.config import local_tht_home
from tht.session.filesystem_repository import FilesystemSessionRepository
workspace = getattr(config, "_workspace_id", "default")
return FilesystemSessionRepository(
home or local_tht_home(), workspace, principal,
root=None if home is not None else getattr(config.paths, "sessions", None),
)
harness/tht/session/store.py
+88 -7
View File
@@ -1,6 +1,7 @@
import json
import os
import shutil
import uuid
from datetime import UTC, datetime
from pathlib import Path
@@ -8,7 +9,6 @@ import yake
from tht.config import DatabaseConfig
from tht.session.models import SessionManifest
from tht.textutil import slugify
MANIFEST = "session_manifest.yaml"
MAX_SLUG_CHARS = 40
@@ -88,12 +88,13 @@ def render_question_md(question: str, assumptions: list[str] | None = None) -> s
def _new_id(question: str, sessions_root: Path, stamp: str) -> str:
base = f"{stamp}-{slugify(question)[:MAX_SLUG_CHARS].rstrip('-')}"
candidate, n = base, 1
while (sessions_root / candidate).exists():
n += 1
candidate = f"{base}-{n}"
return candidate
"""Create an opaque UUIDv4 session identifier.
``question``, ``sessions_root`` and ``stamp`` remain accepted temporarily so
existing workflow callers do not need to change as the repository boundary
is introduced.
"""
return str(uuid.uuid4())
def create_session(
@@ -129,6 +130,35 @@ def create_session(
return manifest
def new_session_manifest(
question: str, db: DatabaseConfig, *, provider=None, model=None, thinking=None, name=None
) -> SessionManifest:
"""Create an unsaved UUIDv4 manifest for a repository-owned session."""
now = datetime.now(UTC)
return SessionManifest(
id=str(uuid.uuid4()), created_at=now, question=question,
database=db.database, schema=db.db_schema, author=current_author(),
summary=_summarize(question), updated_at=now, updated_by=current_author(),
provider=provider, model=model, thinking=thinking, name=name,
)
def build_snapshot_documents(snapshot) -> list[dict]:
docs = [{"phase": "—", "key": "question", "title": "Original question", "format": "text", "content": snapshot.manifest.question}]
spec = [
("question", "F3", "revised_question", "Revised question", "markdown"),
("schema_linking", "F4", "schema_linking", "Schema linking", "schema-linking"),
("sql_final", "F7", "sql", "Final SQL", "sql"),
("validation_report", "finalize", "validation_report", "Validation report", "markdown"),
]
for artifact, phase, key, title, fmt in spec:
if artifact in snapshot.artifacts:
docs.append({"phase": phase, "key": key, "title": title, "format": fmt, "content": snapshot.artifacts[artifact]})
if snapshot.decisions:
docs.append({"phase": "—", "key": "decisions", "title": "Decisions", "format": "decisions", "content": "\n".join(d.model_dump_json() for d in snapshot.decisions) + "\n"})
return docs
def touch_manifest(
session_id: str, sessions_root: Path, *, updated_by: str | None = None
) -> SessionManifest:
@@ -226,6 +256,33 @@ def sync_schema_linking(session_id: str, sessions_root: Path) -> Path:
return set_schema_linking(session_id, data, sessions_root)
def sync_schema_linking_snapshot(snapshot) -> str:
"""Repository projection of effective F4 decisions into schema_linking JSON."""
from tht.phase import effective_decisions
existing = json.loads(snapshot.artifacts.get("schema_linking", "{}"))
latest: dict[str, str] = {}
for decision in effective_decisions(snapshot):
if decision.type in {"table_promoted", "table_excluded", "column_promoted", "column_excluded"}:
latest[decision.subject] = decision.type
candidates, excluded = [], []
for subject, kind in latest.items():
entity = "column" if "." in subject else "table"
if kind.endswith("promoted"):
candidates.append({"kind": entity, "name": subject, "decision": "promoted"})
else:
excluded.append({"kind": entity, "name": subject})
from tht.session.models import SchemaLinking
model = SchemaLinking.model_validate({
"question": existing.get("question") or snapshot.manifest.question,
"candidates": candidates, "excluded": excluded,
"joins": existing.get("joins", []), "open_questions": existing.get("open_questions", []),
"concept_formulas": existing.get("concept_formulas", []),
})
return json.dumps(model.model_dump(by_alias=True), indent=2, ensure_ascii=False)
def load_session(session_id: str, sessions_root: Path) -> SessionManifest:
path = sessions_root / session_id / MANIFEST
if not path.exists():
@@ -297,6 +354,30 @@ def delete_session(session_id: str, sessions_root: Path) -> None:
shutil.rmtree(sessions_root / session_id)
def persist_verified_finalization(
repository,
session_id: str,
*,
validation_report: str,
evidence: str,
) -> SessionManifest:
"""Publish DWH-verified final artifacts and status at one repository boundary.
The caller must complete static validation, EXPLAIN and preview before this
function is entered. It intentionally does not touch solved-question
indexing: that derivative is best-effort and happens after the durable commit.
"""
snapshot = repository.get(session_id)
manifest = snapshot.manifest.model_copy(deep=True)
manifest.status = "finalized"
manifest.updated_at = datetime.now(UTC)
manifest.updated_by = current_author()
return repository.finalize(
manifest,
{"validation_report": validation_report, "evidence": evidence},
).manifest
def build_documents(manifest: SessionManifest, session_dir: Path) -> list[dict]:
"""Ordered, read-only document bundle for the UI panel. Only documents that exist
on disk are returned. CTE artifacts (F6) are intentionally excluded (intermediate)."""
harness/tht/solved.py
+17
View File
@@ -84,3 +84,20 @@ def build_solved_record(session_dir, manifest, promoted_tables) -> VectorRecord:
sql=sql_file.read_text().strip(),
tables=sorted(promoted_tables or set()),
)
def build_solved_snapshot(snapshot, promoted_tables) -> VectorRecord:
from tht.memory import question_context
from tht.phase import effective_decisions
sql = snapshot.artifacts.get("sql_final")
if sql is None:
raise SolvedIndexError("sql_final.sql assente")
decisions = effective_decisions(snapshot)
if not any(d.type == "sql_approved" for d in decisions):
raise SolvedIndexError("decisione sql_approved assente")
return solved_question_record(
session_id=snapshot.manifest.id,
question=question_context(decisions, snapshot.manifest),
sql=sql.strip(), tables=sorted(promoted_tables or set()),
)
harness/tht/taskdoc.py
+11 -9
View File
@@ -15,6 +15,7 @@ from dataclasses import dataclass
from pathlib import Path
from tht.phase import effective_decisions
from tht.session.models import SessionSnapshot
from tht.workflow import load_workflow
MAX_BODY_BYTES = 80_000 # ~20k token (target per task document di una fase)
@@ -50,7 +51,7 @@ def _slice_schema_linking(raw: str, promoted_tables: list[str] | None) -> str:
def generate_task_doc(
session_dir: Path | str,
session_dir: Path | str | SessionSnapshot,
phase: int,
promoted_tables: list[str] | None = None,
) -> TaskDoc:
@@ -62,20 +63,21 @@ def generate_task_doc(
- Brief delle decisioni effective (esclude stale post-rollback, esclude ritirate).
- Header del task con il numero/nome della fase.
"""
session_dir = Path(session_dir)
snapshot = session_dir if isinstance(session_dir, SessionSnapshot) else None
session_dir = None if snapshot is not None else Path(session_dir)
parts: list[str] = []
q = session_dir / "question.md"
if q.exists():
parts.append("## Domanda\n" + q.read_text())
question = snapshot.artifacts.get("question") if snapshot else (session_dir / "question.md").read_text() if (session_dir / "question.md").exists() else None
if question:
parts.append("## Domanda\n" + question)
sl = session_dir / "schema_linking.json"
if sl.exists() and phase >= 4:
sliced = _slice_schema_linking(sl.read_text(), promoted_tables)
linking = snapshot.artifacts.get("schema_linking") if snapshot else (session_dir / "schema_linking.json").read_text() if (session_dir / "schema_linking.json").exists() else None
if linking and phase >= 4:
sliced = _slice_schema_linking(linking, promoted_tables)
parts.append("## Schema linking (deciso)\n```json\n" + sliced + "\n```")
# Brief decisioni effective (D15-aware)
eff = effective_decisions(session_dir)
eff = effective_decisions(snapshot or session_dir)
if eff:
lines = [f"- {d.type} | {d.subject} | {d.detail}" for d in eff]
parts.append("## Decisioni effettive (effective)\n" + "\n".join(lines))
harness/tht/teardown.py
+28
View File
@@ -52,3 +52,31 @@ def teardown_to_phase(session_dir: str | Path, target_phase: int) -> TeardownRep
target.unlink()
report.deleted_files.append(artifact)
return report
def teardown_snapshot(repository, snapshot, target_phase: int) -> TeardownReport:
"""Repository equivalent of teardown_to_phase, including orphaned CTE blobs."""
wf = load_workflow()
report = TeardownReport(target_phase=target_phase)
files = {
"question.md": "question", "schema_linking.json": "schema_linking",
"evidence.json": "evidence", "cte_tests.json": "cte_tests",
"sql_final.sql": "sql_final", "validation_report.md": "validation_report",
"retrieval_pack.md": "retrieval_pack", "cte_plan.json": "cte_plan",
"cte_plan_doc.json": "cte_plan_doc",
}
for phase in wf.phases:
if phase.num <= target_phase:
continue
for artifact in phase.artifacts_out:
if artifact.endswith("/"):
for key in list(snapshot.artifacts):
if key.startswith("cte_sql:"):
repository.delete_artifact(snapshot.manifest.id, key)
report.deleted_files.append(key.removeprefix("cte_sql:") + ".sql")
else:
key = files.get(artifact)
if key and key in snapshot.artifacts:
repository.delete_artifact(snapshot.manifest.id, key)
report.deleted_files.append(artifact)
return report