ThothII/docs/superpowers/specs/2026-07-01-workflow-contract-hardening-design.md

# Workflow contract hardening — design

> Date: 2026-07-01 · Status: approved (design) · Scope: harness only (Pi gate + `tht` CLI + SKILL.md)

## Problem

Analysis of the last live session (`2026-06-30-165708`, GLM 5.2 — Pi transcript
`~/.pi/agent/sessions/--Users-mp-projects-ThothII-harness--/2026-06-30T16-57-09-634Z_*.jsonl`)
showed the model spending **63 of 77 tool calls** on exploration / reverse-engineering the
harness (reading `tht-gate.js` 3×, `phase.py`, `decision_cmd.py`, `session/models.py`; ~10
`--help` probes) instead of executing the workflow. Three concrete defects drive this, all
verified against source:

1. **`SKILL.md` misdescribes phase advancement.** Ground truth: `tht phase advance` (no
   `--auto`) writes `phase_approved` unconditionally (`phase_cmd.py:99-105`); `reviewer_decide(advance:true)`
   calls `advance --auto`, which advances only when `auto_advance_eligible` — phase ∈ `{2,6}`
   **and zero substantive decisions** (`phase.py:_AUTO_ADVANCE_PHASES`, `auto_advance_eligible`).
   So every phase that records real decisions (F1, F3, F4, F5, F6-with-CTEs, F7, F8) closes
   **only** via `reviewer_confirm kind:"phase"`; `advance:true` effectively auto-advances just
   F2-empty / F6-skipped. But `SKILL.md` Phase 3 (lines 170-171) says *"the reviewer_decide
   already advances"* and tells the model NOT to add a phase gate — false. This is the exact
   cause of the model's thrash at transcript turns 30-38 (question_rewritten recorded, phase
   stayed at 3).

2. **F6 CTE approval is broken (dead-ends the workflow).** The gate's `reviewer_confirm
   kind:"cte_result"` registers `cte_approved --subject phase:6` (`tht-gate.js:623-640`), but
   `decision_cmd.py:49-70` rejects `cte_approved` unless `subject` is a CTE name in the
   persisted plan (`approved_ctes`/`next_cte` key on the CTE name) → **exit 5 every time**.
   The last session died here (transcript turn 63).

3. **`schema_linking.json` has no writer.** In F4 the model hand-writes the artifact against
   the internal `SchemaLinking` pydantic model and validates it with ad-hoc `python -c`
   one-liners (transcript turns 45-52, including a venv-vs-system-python misfire) because
   there is no `tht` command to persist/validate it — unlike `question.md` (`set-question`).

## Goals

- Make the F6 CTE-approval path work end-to-end.
- Give the model a `tht`-mediated way to persist+validate `schema_linking.json`.
- Correct `SKILL.md` so it matches the real advance contract, and give the model a per-phase
  cheat-sheet so it stops discovering the CLI at runtime.

## Non-goals

- No rewrite of the working 8-phase workflow, `phase.py`, or `workflow.yaml` semantics.
- No change to `_AUTO_ADVANCE_PHASES` or the auto-advance eligibility rule (it is intentional:
  only trivially-empty phases auto-advance).
- Not touching the F7 `sql` path (`sql_approved --subject phase:7` is correct — no in-plan check).

## Part 2 — Fix F6 CTE approval (chosen approach: A2, gate derives `next_cte`)

The CTE being approved by `reviewer_confirm kind:"cte_result"` is always `next_cte` (the
first unapproved CTE in plan order) — `tht cte test <name>` already enforces `name == next_cte`
(`cte_cmd.py:44-54`). So the gate can derive the name authoritatively instead of trusting a
model-supplied value.

**Changes**
- `harness/tht/cli/cte_cmd.py`: add `tht cte next --session <id>` → prints the bare `next_cte`
  name on stdout (empty output when all CTEs are approved / no plan). Pristine stdout (machine
  contract). Reuses `tht.phase.next_cte`.
- `harness/.pi/extensions/tht-gate.js`, `reviewer_confirm` `kind:"cte_result"` branch: replace
  `--subject phase:${currentPhase}` with the name from `tht cte next --session <id>`; register
  `decision add --type cte_approved --subject <name>`. If `cte next` is empty, return a
  textResult explaining there is no CTE awaiting approval (no-op, should not occur in the normal
  plan-order flow). `kind:"sql"` branch unchanged.

**Tests**
- pytest `tests/test_cte_next.py` (new): `cte next` returns the first unapproved CTE, then the
  next after an approval, then empty when all approved. RED→GREEN.
- pytest contract test (extend existing decision/cte test): `decision add --type cte_approved
  --subject <valid CTE name>` succeeds and `next_cte` advances; `--subject phase:6` exits 5
  (documents the regression the gate previously hit).
- The gate `execute()` glue is L2/live-verified by repo convention (`tht-gate.js:21-24`); the
  behavioral guarantee is carried by the two pytest tests above + a deferred live F6 check.

## Part 3 — `schema_linking.json` writer/validator

Mirror the `question.md` pattern (`rewrite_question` gate tool → `tht session set-question` →
`store.set_question`).

`SchemaLinking` shape to document + validate (`session/models.py`, `extra=forbid`):
`question:str`, `candidates:[{kind:"table"|"column", name, evidence?:[str], decision?:"promoted"|"excluded"|"pending", signals?:{}, grounded_values?:[{}]}]`,
`joins:[{from, to, source?, decision?}]` (JSON key is `from`, aliased),
`excluded:[{kind, name}]`, `open_questions:[str]`, `concept_formulas:[{}]`.

**Changes**
- `harness/tht/session/store.py`: add `set_schema_linking(session_id, data: dict, sessions_path)`
  → `SchemaLinking.model_validate(data)` (raises on invalid), then writes `schema_linking.json`
  deterministically (`json.dumps(model.model_dump(by_alias=True), indent=2, ensure_ascii=False)`).
- `harness/tht/cli/session_cmd.py`: add `tht session set-schema-linking <id> --file <path>`
  (accepts `-` for stdin). Reads JSON, calls `set_schema_linking`; on `ValidationError` prints
  the pydantic error to stderr and exits 5 (so the model gets a precise, actionable message);
  on success prints `OK: schema_linking.json aggiornato (...)`.
- `harness/.pi/extensions/tht-gate.js`: add a `write_schema_linking` tool (parallel to
  `rewrite_question`): params `{session, schema_linking: object}`; passes the JSON to
  `tht session set-schema-linking <id> --file -` via `execFileSync` `input` (stdin, no temp
  file); relays validation errors back to the model via `relayIfThtFails`.

**Tests**
- pytest `tests/test_set_schema_linking.py` (new): valid dict → file written + re-validates;
  invalid (extra key / bad `kind`) → `ValidationError` / CLI exit 5, file not written; `from`
  alias round-trips. RED→GREEN.

## Part 1 — `SKILL.md` surgical corrections (done last, reflects Parts 2-3)

Targeted edits, not a rewrite:
- **Phase 3:** close with `reviewer_confirm kind:"phase"`; delete the "the reviewer_decide
  already advances" claim; set the rewriting `reviewer_decide` to `advance:false`. Keep the
  order (decide records `question_rewritten` → `rewrite_question` writes `question.md`).
- **Discipline 2:** restate the advance rule accurately — every substantive phase closes with
  `reviewer_confirm kind:"phase"` (F7: `kind:"sql"`); `advance:true` auto-advances ONLY F2 when
  memory recorded nothing and F6 when skipped/empty; never rely on it elsewhere.
- **Phase 1 step 4 vs Phase 3:** disambiguate where `rewrite_question` is invoked (it belongs to
  the Phase 3 flow after `question_rewritten`; the Phase 1 mention is removed/redirected).
- **Phase 4:** replace "write `schema_linking.json`" with "call `write_schema_linking`"; add the
  documented `SchemaLinking` shape so the model does not read `session/models.py`.
- **Phase 6:** keep "approve each CTE in plan order via `reviewer_confirm kind:"cte_result"`";
  no signature change (A2 derives the name), close F6 with `reviewer_confirm kind:"phase"`.
- **Add a top cheat-sheet table:** one row per phase → {artifact out, closing gate}. This is the
  state-machine at a glance that the model currently lacks.

## Build order & testing

Order: **Part 2 → Part 3 → Part 1** (SKILL documents the final contract). TDD on Parts 2-3
(pytest RED→GREEN; run `cd harness && .venv/bin/pytest -q` and the gate suite
`node --test` under `.pi/extensions/gate/__tests__` where applicable). Part 1 is documentation;
verify by re-reading against the corrected contract. Full live F6/F4 verification through the
real stack is deferred (needs VPN — see PROJECT_STATE open items).

## Risks

- **Gate glue is not unit-tested** (repo convention): the F6 and `write_schema_linking` wiring
  in `tht-gate.js` rely on pytest CLI contracts + a deferred live check. Mitigation: keep the
  gate changes minimal and push all logic behind the CLI (which IS tested).
- **`--file -` / stdin handling** in the CLI must be covered by a test (empty stdin, invalid
  JSON) so the gate path can't hang or write a partial file.