ThothII/AGENTS.md

# AGENTS.md

## Agent skills

### Issue tracker

Issues for this repository live in the self-hosted Gitea repository at `https://git.tylconsulting.it/mptyl/ThothII`; use its web UI or authenticated Gitea API. See `docs/agents/issue-tracker.md`.

### Triage labels

Use the canonical labels `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, and `wontfix`. See `docs/agents/triage-labels.md`.

### Domain docs

This is a single-context repository with root `CONTEXT.md` and `docs/adr/`. See `docs/agents/domain.md`.

This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.

## Start here

Read [PROJECT_STATE.md](PROJECT_STATE.md) for the current-state snapshot: what was last
built, pending manual gates, workspace/secret layout, and design-doc locations. This file
holds the stable commands + architecture mental model; PROJECT_STATE.md holds the evolving
detail. Current architecture and contracts live in `docs/architecture/`, `docs/contracts/`,
and `docs/evidence.md`; durable design decisions live in `docs/adr/`. Git history is the source
for superseded designs and implementation plans.

## Commands

The repo has three independently-built layers. Run the local Docker stack with `./scripts/run-stack.sh` after creating `deploy/env/local.env`; it starts the base+local Compose profile with `frontend`, `core`, `qdrant`, `embedding`, and the one-shot `embedding-model-init`. The core image contains Pi. Qdrant and Ollama are internal Compose services; DWH and LLM remain external configuration endpoints.

**Native host CLI `tht`** (`tools/tht/`)
- Operator surface: `setup`, `start`, `stop`, `status`, `doctor`, `auth`, `workspace`, and `pi`.
- Use `tht --installation <absolute-path>/thothii-installation.yaml <command>` for installation,
  authentication, diagnostics, lifecycle, and workspace operations.

**harness/** (Python workflow `tht` CLI + Pi gate extension)
- Install: `cd harness && python -m venv .venv && pip install -e ".[dev]"` (puts `tht` on PATH)
- Test: `.venv/bin/pytest -q` — `l2` (real GLM + remote DB) is opt-in via `addopts = -m 'not l2'`; `l0` (testcontainers) needs Docker
- Single test: `.venv/bin/pytest tests/test_session_mutations.py::test_set_name -v` (or `-k <pattern>`); include e2e with `-m l2`
- Lint: `.venv/bin/ruff check .` (line-length 100)

**backend/** (Fastify + TypeScript, vitest)
- Dev: `npm run dev` (tsx watch `src/server.ts`) · Build: `npm run build` (tsc → `dist/`)
- Test: `npx vitest run` · Single: `npx vitest run test/routes-sessions.test.ts -t "rename"`
- Typecheck: `npx tsc --noEmit -p .` (vitest does NOT type-check — run this before committing)

**frontend/** (React 18 + Vite + vitest)
- Dev: `npm run dev` (Vite; set `VITE_BACKEND_URL`) · Build: `npm run build`
- Test: `npx vitest run` · Single: `npx vitest run src/shell/NavSessions.test.tsx`
- Typecheck: `npx tsc -b` · E2E: `npm run e2e` (Playwright)

**Documentation** (MkDocs, repository-locked Python dependencies)
- Strict build: `./scripts/build-docs.sh`
- Refresh lock: `./scripts/update-docs-lock.sh`

No ESLint on the TS layers — `tsc` is the gate. Tests use vitest + MSW (no network).

## Architecture (the parts that need multiple files to see)

```
frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness → DWH (read-only)
```

- **The harness owns the workflow and all persistence.** The Python workflow CLI `tht` inside
  `core` is deterministic; `harness/.pi/extensions/tht-gate.js` is a Pi extension that drives an **8-phase
  NL→SQL workflow**. The single source of workflow truth is `harness/workflow.yaml`; the
  orchestration rules the model must follow are `harness/.pi/skills/tht-sessione/SKILL.md`.
  "Current phase" is computed by folding the decision ledger (`harness/tht/phase.py`), not
  stored — read it before reasoning about phase logic.

- **Persistence = phase documents, NOT chat.** A session is a directory under the workspace's
  `sessions/` path: `session_manifest.yaml` + per-phase artifacts (`question.md`,
  `schema_linking.json`, `sql_final.sql`, …) + `review_decisions.jsonl`. The contract
  (SKILL.md): *"the persisted state is the truth — what is not recorded did not happen."*
  There is no verbatim transcript store. A resumed Pi process rebuilds context from
  `tht session show <id>` + the on-disk artifacts.

- **The backend bridges sessions and owns the installation-local metadata catalog.** `ThtRunner`
  shells the Python workflow `tht` subcommands inside `core`;
  `PiProcessManager` runs one Pi child per session and bridges its RPC stream;
  `SessionBridge` maps Pi RPC events → client events (`ui_request`/`text_delta`/`info`);
  `SseHub` fans them out over SSE to the browser. The separate PostgreSQL catalog stores database
  metadata and sequential AI description-generation runs. Description generation samples the DWH
  through read-only connectors and calls a short-lived Python LiteLLM helper; it does not use Pi or
  expose a public CLI command. Sessions, metadata generation, and embedding resolve models from the
  generated Installation Model Catalog; `thothii-installation.yaml` is its only authored source.

- **Human-in-the-loop gate contract.** The model proposes; a human reviewer decides at gates
  via widgets (`reviewer_select` = single pick — a chosen option carrying a `decision` payload
  auto-confirms/persists directly, an option without one only asks; `reviewer_decide` = multiselect,
  each choice IS a decision; `reviewer_confirm` = artifact/phase gate). The frontend renders these
  widget-descriptors (`src/widgets/` registry) and the live transcript is rebuilt in-memory
  from the SSE stream (`src/store/sessionStore.ts`) — it is not persisted.

## Project-specific gotchas

- **`tht`'s `-c`/`--config` is a PER-COMMAND option** — it must follow the subcommand, never
  precede it (`ThtRunner.buildArgv` enforces this; prepending caused live 500s).
- **`--json` output must be pristine** (only valid JSON on stdout) — used as a machine contract.
- **Localization:** deterministic UI uses the EN/IT catalogs with English fallback;
  model interaction uses the session manifest's immutable `interaction_language`.
  Workspace content, SQL, and identifiers remain unchanged. For shell modes, portal
  integration, or translations, read `docs/operations/shell-and-localization.md`.
- **Server deployment:** for the coordinated ThothII/Omics upgrade, follow
  `docs/operations/server-codex-handoff.md`; it supersedes earlier Omics delivery
  instructions. Omics source integration uses GitHub with no repository relay prerequisite.
- **Server identity:** for portal login/logout, proxy headers, or Omics deploy, read
  `docs/install/authentication-upstream.md` before changing authentication. Omics
  uses embedded/upstream, not a second ThothII OIDC login. Full/embedded rendering
  is documented in `docs/architecture/application-shell.md`; release acceptance
  is in `docs/testing/authentication-manual-acceptance.md`.
- **Workspace schema v4** defines workspace identity and optional Evidence only. PostgreSQL Metadata
  Catalog owns database identity, binding, schema, descriptions, sensitivity, and relationships;
  embedding/model facts come from the installation catalog. The legacy `harness/workspaces/*.yaml` runtime snapshots still use
  absolute session/artifact/index paths; secrets stay in `harness/.env` (gitignored).
- **Settings are global** (`backend/data/settings.json`: workspace/thinking). Provider/model choices
  are ephemeral canonical catalog selections pinned into the session manifest.
- **Resume**: a resumable session re-enters at its last incomplete phase. The backend refuses
  resume with 409 when `finalized` or `archived`, and `PiProcessManager.spawnFor` must send
  `/riprendi-sessione <id>` (resume mode) vs `/nuova-domanda` (new) — sending the wrong prompt
  silently turns a resume into a new question.