mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
aa22183ac7cde5e08c4ec121d3bb9a6b0c3a4b2f
ThothII/docs/superpowers/plans/2026-08-13-p7-psd-migration.md
T

6.9 KiB
Raw Blame History

P7 — PSD migration to the workspace registry — Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:executing-plans to apply this plan task-by-task.

Goal: Move the existing Policlinico San Donato workspace from the legacy flat layout (tht-workspace-psd/psd.yaml + root evidence/ + artifacts/mschema/annotations.yaml) into the P1.1 workspace registry repository, then re-embed/re-index it on the new internal semantic stack so ThothII can run live against the real PSD DWH.

Source of truth: PRD D7 (docs/prd/2026-08-09-workspace-preprocessing-prd.md), the P1.1 layout (docs/superpowers/plans/2026-08-11-p1-1-workspace-directory-registry.md), and docs/workspace-diagnostic-protocol.md.

Architecture: The PSD content becomes an ordinary Git workspace repository consumed by ThothII via THT_WORKSPACE_GIT_REMOTE. DWH access stays REST (PostgREST); Qdrant and Ollama embedding are the internal Compose services. The legacy pgvector (768-dim, nomic) is superseded and must be re-embedded with qwen3-embedding:0.6b (1024-dim).

Tech Stack: Git + YAML (descriptor/catalog), the existing thothctl/registry for validation. No new code is required unless a validator gap is proven by a test.

Current-state findings recorded by this plan

  • /Users/mp/projects/tht-workspace-psd is already a Git repo on main (38 tracked files: .gitignore, legacy psd.yaml, and 36 curated evidence/*.md) with no remote.
  • The legacy psd.yaml declares REST DWH (database: postgres, schema: datawarehouse), X-API-Key auth, internal CA, and the old external pgvector + Ollama; it has no llm_policy.
  • Curated FK annotations exist at artifacts/mschema/annotations.yaml (367 FK lines) in the new canonical Annotations shape and can be copied directly to <id>/schema/annotations.yaml.
  • physical.yaml is absent, so DWH introspection must be re-run (requires VPN + DWH access).
  • The legacy runtime dirs (.tht-dwh/, .tht-jobs/, config/, corpus/, runtime-v2/, .legacy-artifacts-backup-premerge/) are untracked runtime state and must not enter the curated repo.

Explicit decisions frozen by this plan

  1. Repository identity. Reuse /Users/mp/projects/tht-workspace-psd as the author/curator clone; publish it to a GitHub remote the owner creates. The ThothII installation clones that remote, not any path inside the ThothII repo.
  2. Workspace id: psd-clinical (catalog, descriptor, Qdrant collection, bindings namespace PSD_CLINICAL).
  3. Descriptor (schema v3): dwh = postgres / database postgres / schema datawarehouse / supported_transports: [rest_api]; semantic_index = Qdrant psd-clinical 1024/cosine + qwen3-embedding:0.6b; language: it; llm_policy.allowed starts from the historically active PSD models (zai/glm-5.2, deepseek/deepseek-v4-flash, deepseek/deepseek-v4-pro, aritmolab/qwen3.6-35b-a3b) and is owner-adjustable.
  4. DWH diagnostic: diagnostics.dwh_rest = POST /rpc/ping, auth: x-api-key, response { database: postgres, schema: datawarehouse }. The exact ping RPC path/auth is verified by the owner during the first live smoke and adjusted only in the Git descriptor.
  5. Curated content only: the repo contains thoth-workspaces.yaml, psd-clinical/workspace.yaml, psd-clinical/evidence/, psd-clinical/schema/annotations.yaml, plus API-generated workspace-docs/. Legacy runtime dirs stay untracked (gitignore).
  6. Re-embedding: the legacy pgvector is not reused. DWH introspection + schema/Evidence indexing run afresh on the new stack (or, if the owner prefers, pgvector is exported and re-embedded); this task is gated on VPN + DWH credentials + a running stack.
  7. Secrets stay out of Git: DWH X-API-Key and CA are written as local secret files referenced by THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE / _TLS_CA_FILE.

Completion contract

  1. tht-workspace-psd is restructured to the P1.1 layout and commits cleanly (no legacy runtime dirs).
  2. A local registry bootstrap against that repository activates psd-clinical (schema-v3 valid, Evidence materialized, annotations parsed and synced, docs generated).
  3. thothctl … workspace inspect --workspace psd-clinical --json succeeds once the installation bindings + VPN are present.
  4. thothctl … workspace preprocess run --workspace psd-clinical --json completes DWH → FK → schema → Evidence against the real DWH and indexes into the internal Qdrant.
  5. A live session on psd-clinical reaches the first reviewer gate (P8 L2 smoke).

Task 1: Restructure the repository (autonomous)

Repo: /Users/mp/projects/tht-workspace-psd (separate checkout).

  1. Write thoth-workspaces.yaml (catalog with psd-clinical).
  2. Write psd-clinical/workspace.yaml (schema v3, decisions 2–4).
  3. git mv the 36 evidence/*.md files to psd-clinical/evidence/.
  4. Copy the curated FK to psd-clinical/schema/annotations.yaml.
  5. Add .gitignore for legacy runtime dirs; remove/leave legacy psd.yaml as a non-contract historical note (do not commit the old flat paths as canonical).
  6. Commit on main (no remote yet).

Task 2: Local registry validation (autonomous, no DWH/secret)

  1. From a scratch bare remote of the restructured repo, bootstrap a WorkspaceRegistry and assert psd-clinical activates: descriptor valid, catalog matches, Evidence materialized with manifest, annotations parsed/synced, workspace-docs/psd-clinical generated.
  2. Verify thothctl workspace inspect fails only on the missing DWH bindings (not on the descriptor).

Task 3: Installation bindings + secrets (owner)

  1. Owner creates the GitHub remote and provides its URL + push credentials.
  2. Owner provides (or confirms reuse of) the DWH X-API-Key and CA; write them as local secret files.
  3. Fill THT_WS_PSD_CLINICAL_DWH_* bindings + THT_WORKSPACE_GIT_REMOTE + LLM provider in the installation env (VPN active).

Task 4: Re-embedding/indexing (owner + stack)

  1. Start the stack; embedding-model-init pulls qwen3-embedding:0.6b.
  2. thothctl … workspace preprocess run --workspace psd-clinical --json (DWH → FK → schema → Evidence) against the real DWH; verify the Qdrant collection is populated and revision-scoped.

Task 5: Live smoke + manual acceptance (owner, P8 L2)

  1. New session on psd-clinical reaches the first reviewer gate; finalize one real query.
  2. Record the P7/P8 manual acceptance in docs/testing/p2-p6-manual-verification.md and PROJECT_STATE.md.

Owner checkpoint

Tasks 1–2 are executed now by the agent. Tasks 3–5 are blocked on owner-provided secrets/access (GitHub remote, VPN, DWH key/CA, LLM provider) and on the live stack.

Non-goals

  • No P8/P9/P10 work (end-to-end CI, retention policy changes, ssh_tunnel runtime).
  • No change to the accepted P1.1–P6 contracts.
  • No secret value, certificate, or response body is committed or printed.
Reference in New Issue View Git Blame Copy Permalink