docs: plan P7 PSD migration to the workspace registry
This commit is contained in:
@@ -0,0 +1,122 @@
|
|||||||
|
# P7 — PSD migration to the workspace registry — Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to apply this plan task-by-task.
|
||||||
|
|
||||||
|
**Goal:** Move the existing Policlinico San Donato workspace from the legacy flat layout
|
||||||
|
(`tht-workspace-psd/psd.yaml` + root `evidence/` + `artifacts/mschema/annotations.yaml`) into the
|
||||||
|
P1.1 workspace registry repository, then re-embed/re-index it on the new internal semantic stack so
|
||||||
|
ThothII can run live against the real PSD DWH.
|
||||||
|
|
||||||
|
**Source of truth:** PRD D7 (`docs/prd/2026-08-09-workspace-preprocessing-prd.md`), the P1.1 layout
|
||||||
|
(`docs/superpowers/plans/2026-08-11-p1-1-workspace-directory-registry.md`), and
|
||||||
|
`docs/workspace-diagnostic-protocol.md`.
|
||||||
|
|
||||||
|
**Architecture:** The PSD content becomes an ordinary Git workspace repository consumed by ThothII
|
||||||
|
via `THT_WORKSPACE_GIT_REMOTE`. DWH access stays REST (PostgREST); Qdrant and Ollama embedding are
|
||||||
|
the internal Compose services. The legacy pgvector (768-dim, nomic) is superseded and must be
|
||||||
|
re-embedded with `qwen3-embedding:0.6b` (1024-dim).
|
||||||
|
|
||||||
|
**Tech Stack:** Git + YAML (descriptor/catalog), the existing `thothctl`/registry for validation.
|
||||||
|
No new code is required unless a validator gap is proven by a test.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current-state findings recorded by this plan
|
||||||
|
|
||||||
|
- `/Users/mp/projects/tht-workspace-psd` is already a Git repo on `main` (38 tracked files:
|
||||||
|
`.gitignore`, legacy `psd.yaml`, and 36 curated `evidence/*.md`) with **no remote**.
|
||||||
|
- The legacy `psd.yaml` declares REST DWH (`database: postgres`, `schema: datawarehouse`), X-API-Key
|
||||||
|
auth, internal CA, and the old external pgvector + Ollama; it has **no** `llm_policy`.
|
||||||
|
- Curated FK annotations exist at `artifacts/mschema/annotations.yaml` (367 FK lines) in the new
|
||||||
|
canonical `Annotations` shape and can be copied directly to `<id>/schema/annotations.yaml`.
|
||||||
|
- `physical.yaml` is absent, so DWH introspection must be re-run (requires VPN + DWH access).
|
||||||
|
- The legacy runtime dirs (`.tht-dwh/`, `.tht-jobs/`, `config/`, `corpus/`, `runtime-v2/`,
|
||||||
|
`.legacy-artifacts-backup-premerge/`) are untracked runtime state and must not enter the curated repo.
|
||||||
|
|
||||||
|
## Explicit decisions frozen by this plan
|
||||||
|
|
||||||
|
1. **Repository identity.** Reuse `/Users/mp/projects/tht-workspace-psd` as the author/curator clone;
|
||||||
|
publish it to a GitHub remote the owner creates. The ThothII installation clones that remote, not
|
||||||
|
any path inside the ThothII repo.
|
||||||
|
2. **Workspace id:** `psd-clinical` (catalog, descriptor, Qdrant collection, bindings namespace
|
||||||
|
`PSD_CLINICAL`).
|
||||||
|
3. **Descriptor (schema v3):** `dwh` = `postgres` / database `postgres` / schema `datawarehouse` /
|
||||||
|
`supported_transports: [rest_api]`; `semantic_index` = Qdrant `psd-clinical` 1024/cosine +
|
||||||
|
`qwen3-embedding:0.6b`; `language: it`; `llm_policy.allowed` starts from the historically active
|
||||||
|
PSD models (`zai/glm-5.2`, `deepseek/deepseek-v4-flash`, `deepseek/deepseek-v4-pro`,
|
||||||
|
`aritmolab/qwen3.6-35b-a3b`) and is owner-adjustable.
|
||||||
|
4. **DWH diagnostic:** `diagnostics.dwh_rest` = `POST /rpc/ping`, `auth: x-api-key`, response
|
||||||
|
`{ database: postgres, schema: datawarehouse }`. The exact ping RPC path/auth is verified by the
|
||||||
|
owner during the first live smoke and adjusted only in the Git descriptor.
|
||||||
|
5. **Curated content only:** the repo contains `thoth-workspaces.yaml`, `psd-clinical/workspace.yaml`,
|
||||||
|
`psd-clinical/evidence/`, `psd-clinical/schema/annotations.yaml`, plus API-generated
|
||||||
|
`workspace-docs/`. Legacy runtime dirs stay untracked (gitignore).
|
||||||
|
6. **Re-embedding:** the legacy pgvector is not reused. DWH introspection + schema/Evidence indexing
|
||||||
|
run afresh on the new stack (or, if the owner prefers, pgvector is exported and re-embedded);
|
||||||
|
this task is gated on VPN + DWH credentials + a running stack.
|
||||||
|
7. **Secrets stay out of Git:** DWH X-API-Key and CA are written as local secret files referenced by
|
||||||
|
`THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE` / `_TLS_CA_FILE`.
|
||||||
|
|
||||||
|
## Completion contract
|
||||||
|
|
||||||
|
1. `tht-workspace-psd` is restructured to the P1.1 layout and commits cleanly (no legacy runtime dirs).
|
||||||
|
2. A local registry bootstrap against that repository activates `psd-clinical` (schema-v3 valid,
|
||||||
|
Evidence materialized, annotations parsed and synced, docs generated).
|
||||||
|
3. `thothctl … workspace inspect --workspace psd-clinical --json` succeeds once the installation
|
||||||
|
bindings + VPN are present.
|
||||||
|
4. `thothctl … workspace preprocess run --workspace psd-clinical --json` completes DWH → FK → schema →
|
||||||
|
Evidence against the real DWH and indexes into the internal Qdrant.
|
||||||
|
5. A live session on `psd-clinical` reaches the first reviewer gate (P8 L2 smoke).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Restructure the repository (autonomous)
|
||||||
|
|
||||||
|
**Repo:** `/Users/mp/projects/tht-workspace-psd` (separate checkout).
|
||||||
|
|
||||||
|
1. Write `thoth-workspaces.yaml` (catalog with `psd-clinical`).
|
||||||
|
2. Write `psd-clinical/workspace.yaml` (schema v3, decisions 2–4).
|
||||||
|
3. `git mv` the 36 `evidence/*.md` files to `psd-clinical/evidence/`.
|
||||||
|
4. Copy the curated FK to `psd-clinical/schema/annotations.yaml`.
|
||||||
|
5. Add `.gitignore` for legacy runtime dirs; remove/leave legacy `psd.yaml` as a non-contract
|
||||||
|
historical note (do not commit the old flat paths as canonical).
|
||||||
|
6. Commit on `main` (no remote yet).
|
||||||
|
|
||||||
|
### Task 2: Local registry validation (autonomous, no DWH/secret)
|
||||||
|
|
||||||
|
1. From a scratch bare remote of the restructured repo, bootstrap a `WorkspaceRegistry` and assert
|
||||||
|
`psd-clinical` activates: descriptor valid, catalog matches, Evidence materialized with manifest,
|
||||||
|
annotations parsed/synced, `workspace-docs/psd-clinical` generated.
|
||||||
|
2. Verify `thothctl workspace inspect` fails only on the missing DWH bindings (not on the descriptor).
|
||||||
|
|
||||||
|
### Task 3: Installation bindings + secrets (owner)
|
||||||
|
|
||||||
|
1. Owner creates the GitHub remote and provides its URL + push credentials.
|
||||||
|
2. Owner provides (or confirms reuse of) the DWH X-API-Key and CA; write them as local secret files.
|
||||||
|
3. Fill `THT_WS_PSD_CLINICAL_DWH_*` bindings + `THT_WORKSPACE_GIT_REMOTE` + LLM provider in the
|
||||||
|
installation env (VPN active).
|
||||||
|
|
||||||
|
### Task 4: Re-embedding/indexing (owner + stack)
|
||||||
|
|
||||||
|
1. Start the stack; `embedding-model-init` pulls `qwen3-embedding:0.6b`.
|
||||||
|
2. `thothctl … workspace preprocess run --workspace psd-clinical --json` (DWH → FK → schema → Evidence)
|
||||||
|
against the real DWH; verify the Qdrant collection is populated and revision-scoped.
|
||||||
|
|
||||||
|
### Task 5: Live smoke + manual acceptance (owner, P8 L2)
|
||||||
|
|
||||||
|
1. New session on `psd-clinical` reaches the first reviewer gate; finalize one real query.
|
||||||
|
2. Record the P7/P8 manual acceptance in `docs/testing/p2-p6-manual-verification.md` and
|
||||||
|
PROJECT_STATE.md.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Owner checkpoint
|
||||||
|
|
||||||
|
Tasks 1–2 are executed now by the agent. Tasks 3–5 are blocked on owner-provided secrets/access
|
||||||
|
(GitHub remote, VPN, DWH key/CA, LLM provider) and on the live stack.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No P8/P9/P10 work (end-to-end CI, retention policy changes, ssh_tunnel runtime).
|
||||||
|
- No change to the accepted P1.1–P6 contracts.
|
||||||
|
- No secret value, certificate, or response body is committed or printed.
|
||||||
Reference in New Issue
Block a user