docs: add portable deployment implementation plans

2026-07-11 19:49:26 +02:00
parent 586180eaa1
commit ef46a4ca09
5 changed files with 1231 additions and 0 deletions
@@ -0,0 +1,305 @@
 # Adapter Foundations Implementation Plan
 > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
 **Goal:** Route all DWH and vector operations through stable typed contracts while preserving current direct/REST behavior.
 **Architecture:** Define small Python protocols and capability records, implement adapters around existing modules, and centralize construction in one factory. Introduce a discriminated workspace schema with a compatibility translator for current YAML.
 **Tech Stack:** Python 3.11+, `typing.Protocol`, Pydantic 2, SQLAlchemy, psycopg2, requests, pytest, Typer.
 ## Global Constraints
 - Do not change workflow phases or CLI output formats.
 - PostgreSQL direct and current Thoth/PostgREST are the only supported DWH transports in this plan.
 - Preserve reader/writer vector credential separation.
 - All migrations must accept existing `database.transport`, `rest`, `vector_db`, `vector_rest`, and `vector_write_rest` fields.
 - JSON stdout stays pristine; diagnostics go to stderr unless part of the JSON result.
 ---
 ### Task 1: Define DWH ports and capabilities
 **Files:**
 - Create: `harness/tht/ports/__init__.py`
 - Create: `harness/tht/ports/dwh.py`
 - Test: `harness/tests/test_dwh_port_contract.py`
 **Interfaces:**
 - Produces: `DwhCapabilities`, `DwhAdapter`, `DwhHealth`, and `UnsupportedCapability`.
 - Consumes: existing catalog models from `tht.db.introspect` and execution result types from `tht.db.execute`.
 - [ ] **Step 1: Write the failing protocol-shape test**
 ```python
 def test_fake_adapter_satisfies_runtime_protocol():
    adapter = FakeDwhAdapter()
    assert isinstance(adapter, DwhAdapter)
    assert adapter.capabilities.explain is True
    assert adapter.health().ok is True
 ```
 - [ ] **Step 2: Run the focused test and confirm failure**
 Run: `cd harness && .venv/bin/pytest tests/test_dwh_port_contract.py -q`  
 Expected: FAIL because `tht.ports.dwh` does not exist.
 - [ ] **Step 3: Add the minimal public contract**
 ```python
@dataclass(frozen=True)
 class DwhCapabilities:
    introspection: bool = True
    explain: bool = True
    sampling: bool = True
    distinct_values: bool = True
@runtime_checkable
 class DwhAdapter(Protocol):
    @property
    def capabilities(self) -> DwhCapabilities: ...
    def health(self) -> DwhHealth: ...
    def introspect(self) -> DatabaseCatalog: ...
    def run_query(self, sql: str, *, limit: int | None = None) -> QueryResult: ...
    def explain(self, sql: str) -> PlanSummary: ...
    def sample_column(self, table: str, column: str, *, limit: int) -> list[object]: ...
    def distinct_values(self, table: str, column: str) -> list[object]: ...
 ```
 - [ ] **Step 4: Run contract test and type-oriented import smoke test**
 Run: `cd harness && .venv/bin/pytest tests/test_dwh_port_contract.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/ports harness/tests/test_dwh_port_contract.py
 git commit -m "refactor(dwh): define adapter contract"
 ```
 ### Task 2: Wrap direct and REST DWH implementations
 **Files:**
 - Create: `harness/tht/adapters/dwh/__init__.py`
 - Create: `harness/tht/adapters/dwh/postgres.py`
 - Create: `harness/tht/adapters/dwh/thoth_rest.py`
 - Test: `harness/tests/test_dwh_adapters.py`
 - Modify: `harness/tht/db/execute.py`
 - Modify: `harness/tht/rest/execute.py`
 **Interfaces:**
 - Consumes: `DwhAdapter` from Task 1; existing `DatabaseConfig`, `RestConfig`, catalog, sampling, execute, and explain functions.
 - Produces: `PostgresDwhAdapter(config)` and `ThothRestDwhAdapter(database, rest)`.
 - [ ] **Step 1: Add parametrized contract tests for both wrappers**
 ```python
@pytest.mark.parametrize("factory", [postgres_factory, rest_factory])
 def test_adapter_rejects_write_sql(factory):
    with pytest.raises(ReadOnlyViolation):
        factory().run_query("delete from fact_sales")
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_dwh_adapters.py -q`  
 Expected: FAIL because the adapter classes are absent.
 - [ ] **Step 3: Implement thin wrappers, without duplicating transport logic**
 ```python
 class PostgresDwhAdapter:
    capabilities = DwhCapabilities()
    def __init__(self, config: DatabaseConfig): self._config = config
    def run_query(self, sql: str, *, limit: int | None = None) -> QueryResult:
        return run_query(self._config, sql, limit=limit)
 ```
 Implement the analogous REST wrapper by delegating to `tht.rest.*`; translate transport-specific errors only at the adapter boundary.
 - [ ] **Step 4: Run adapter, read-only, sampling, and REST tests**
 Run: `cd harness && .venv/bin/pytest tests/test_dwh_adapters.py tests/test_readonly_guard.py tests/test_rest_client.py tests/l0/test_db_sampling.py -q`  
 Expected: PASS; L0 may deselect when Docker is unavailable.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/adapters harness/tht/db/execute.py harness/tht/rest/execute.py harness/tests/test_dwh_adapters.py
 git commit -m "refactor(dwh): adapt direct and REST transports"
 ```
 ### Task 3: Define vector port and wrappers
 **Files:**
 - Create: `harness/tht/ports/vector.py`
 - Create: `harness/tht/adapters/vector/__init__.py`
 - Create: `harness/tht/adapters/vector/thoth_http.py`
 - Create: `harness/tht/adapters/vector/legacy_direct.py`
 - Test: `harness/tests/test_vector_port_contract.py`
 - Modify: `harness/tht/vectorstore/reader.py`
 **Interfaces:**
 - Produces: `VectorStore`, `VectorCapabilities`, `VectorHealth`, `VectorRecord`, `VectorHit`, `ThothHttpVectorStore`.
 - Preserves: current `VectorRestClient`, `DirectSearcher`, and `RestSearcher` behavior behind wrappers.
 - [ ] **Step 1: Write read/write capability and dual-credential tests**
 ```python
 def test_http_store_reports_reader_without_writer():
    store = ThothHttpVectorStore(reader=reader, writer=None)
    assert store.capabilities.search is True
    assert store.capabilities.upsert is False
    with pytest.raises(VectorWriteUnavailable):
        store.upsert("memory", [])
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_vector_port_contract.py -q`  
 Expected: FAIL because the vector port is absent.
 - [ ] **Step 3: Implement the vector contract and wrappers**
 ```python
@runtime_checkable
 class VectorStore(Protocol):
    @property
    def capabilities(self) -> VectorCapabilities: ...
    def health(self) -> VectorHealth: ...
    def search(self, collections: list[str], embedding: list[float], *, limit: int,
               kinds: list[str] | None = None) -> list[VectorHit]: ...
    def existing_hashes(self, collection: str, kinds: list[str]) -> dict[str, str]: ...
    def upsert(self, collection: str, records: list[VectorRecord]) -> int: ...
 ```
 - [ ] **Step 4: Run vector regression tests**
 Run: `cd harness && .venv/bin/pytest tests/test_vector_port_contract.py tests/test_vector_dual_key.py tests/test_search_similar_kinds.py tests/test_memory_save_one.py tests/test_solved_question.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/ports/vector.py harness/tht/adapters/vector harness/tht/vectorstore/reader.py harness/tests/test_vector_port_contract.py
 git commit -m "refactor(vector): define store contract"
 ```
 ### Task 4: Introduce discriminated resource configuration with legacy translation
 **Files:**
 - Create: `harness/tht/config_compat.py`
 - Modify: `harness/tht/config.py`
 - Modify: `harness/workspaces/tht.example.yaml`
 - Test: `harness/tests/test_config_resources.py`
 - Test: `harness/tests/test_config_legacy_compat.py`
 **Interfaces:**
 - Produces: `DwhResourceConfig`, `VectorResourceConfig`, `WorkspaceRoots` and `translate_legacy_config(raw)`.
 - Consumes: existing YAML environment expansion and `ConfigError` behavior.
 - [ ] **Step 1: Write new-schema and legacy-equivalence tests**
 ```python
 def test_legacy_rest_workspace_equals_new_resource_schema(tmp_path):
    old = load_config(write_old_workspace(tmp_path))
    new = load_config(write_new_workspace(tmp_path))
    assert old.dwh.model_dump() == new.dwh.model_dump()
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_config_resources.py tests/test_config_legacy_compat.py -q`  
 Expected: FAIL because `dwh` and discriminated vector resources are absent.
 - [ ] **Step 3: Add discriminated models and an isolated translator**
 ```python
 class PostgresDwhConfig(BaseModel):
    type: Literal["postgres_direct"]
    connection: DatabaseConfig
 class ThothRestDwhConfig(BaseModel):
    type: Literal["thoth_rest"]
    database: DatabaseIdentityConfig
    endpoint: RestConfig
 DwhResourceConfig = Annotated[
    PostgresDwhConfig | ThothRestDwhConfig,
    Field(discriminator="type"),
 ]
 ```
 Translate legacy keys before Pydantic validation and emit one deprecation warning to stderr, never stdout.
 - [ ] **Step 4: Run all config and workspace tests**
 Run: `cd harness && .venv/bin/pytest tests/test_config_resources.py tests/test_config_legacy_compat.py tests/test_workspace.py tests/test_vector_dual_key.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/config.py harness/tht/config_compat.py harness/workspaces/tht.example.yaml harness/tests/test_config_resources.py harness/tests/test_config_legacy_compat.py
 git commit -m "feat(config): add typed resource schema"
 ```
 ### Task 5: Centralize construction and migrate command call sites
 **Files:**
 - Create: `harness/tht/adapters/factory.py`
 - Modify: `harness/tht/cli/db_cmd.py`
 - Modify: `harness/tht/cli/schema_cmd.py`
 - Modify: `harness/tht/cli/sql_cmd.py`
 - Modify: `harness/tht/cli/lsh_cmd.py`
 - Modify: `harness/tht/cli/search_cmd.py`
 - Modify: `harness/tht/cli/vector_cmd.py`
 - Modify: `harness/tht/cli/memory_cmd.py`
 - Test: `harness/tests/test_adapter_factory.py`
 **Interfaces:**
 - Produces: `build_dwh(cfg: Config) -> DwhAdapter` and `build_vector_store(cfg: Config, *, require_write: bool = False) -> VectorStore`.
 - Consumes: resource configs from Task 4 and wrappers from Tasks 2-3.
 - [ ] **Step 1: Write exact factory selection and missing-writer tests**
 ```python
 def test_factory_selects_http_vector_and_requires_writer(config):
    assert isinstance(build_vector_store(config), ThothHttpVectorStore)
    with pytest.raises(ConfigError, match="writer"):
        build_vector_store(config, require_write=True)
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_adapter_factory.py -q`  
 Expected: FAIL because the factory does not exist.
 - [ ] **Step 3: Implement factory and replace per-command branching**
 ```python
 def build_dwh(cfg: Config) -> DwhAdapter:
    match cfg.dwh.type:
        case "postgres_direct": return PostgresDwhAdapter(cfg.dwh.connection)
        case "thoth_rest": return ThothRestDwhAdapter(cfg.dwh.database, cfg.dwh.endpoint)
        case other: raise ConfigError(f"Adapter DWH non supportato: {other}")
 ```
 Delete transport checks from migrated commands; keep CLI wording and exit codes stable.
 - [ ] **Step 4: Run focused command suites, full harness suite, and ruff**
 Run: `cd harness && .venv/bin/pytest tests/test_adapter_factory.py tests/test_schema_introspect_guard.py tests/test_sql_preview_json.py tests/test_search_pack.py tests/integration/test_gate_cli_signatures.py -q`  
 Expected: PASS.  
 Run: `cd harness && .venv/bin/pytest -q`  
 Expected: all non-L2 tests PASS.  
 Run: `cd harness && .venv/bin/ruff check tht tests/test_adapter_factory.py tests/test_dwh_port_contract.py tests/test_dwh_adapters.py tests/test_vector_port_contract.py tests/test_config_resources.py tests/test_config_legacy_compat.py`  
 Expected: no errors.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht harness/tests harness/workspaces/tht.example.yaml
 git commit -m "refactor(core): route integrations through adapter factory"
 ```
@@ -0,0 +1,295 @@
 # Container Packaging and Portable Storage Implementation Plan
 > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
 **Goal:** Run ThothII from exactly two application images with runtime configuration and host-independent persistent roots.
 **Architecture:** Build a multi-runtime core image for Fastify, Pi, and `tht`, plus a static frontend image. Resolve workspace paths beneath mounted logical roots and provide a Compose base for external dependencies.
 **Tech Stack:** Docker BuildKit, Docker Compose v2, Node 22, Python 3.11, Fastify, React/Vite, nginx or Caddy.
 ## Global Constraints
 - Plan 1 is complete and adapter factories are the only integration construction path.
 - Do not bake secrets or customer workspace content into images.
 - Containers run as non-root and write only beneath mounted data roots.
 - `pi`, `tht`, CA certificates, and native dependencies must work on every advertised architecture.
 - Frontend backend URL is runtime-configurable.
 ---
 ### Task 1: Add logical workspace roots and migration diagnostics
 **Files:**
 - Create: `harness/tht/paths.py`
 - Create: `harness/tht/cli/doctor_cmd.py`
 - Modify: `harness/tht/config.py`
 - Modify: `harness/tht/cli/__init__.py`
 - Test: `harness/tests/test_portable_paths.py`
 - Test: `harness/tests/test_doctor_cli.py`
 **Interfaces:**
 - Produces: `resolve_workspace_paths(config_path, cfg, data_root) -> ResolvedPaths` and `tht doctor --json`.
 - [ ] **Step 1: Test relative, absolute-legacy, and escape rejection**
 ```python
 def test_relative_paths_resolve_under_workspace_root(tmp_path):
    resolved = resolve_workspace_paths(cfg_path, cfg, tmp_path / "data")
    assert resolved.sessions == tmp_path / "data/workspaces/demo/sessions"
 def test_path_escape_is_rejected(tmp_path):
    with pytest.raises(ConfigError, match="outside workspace root"):
        resolve_workspace_paths(cfg_path, config_with_sessions("../../private"), tmp_path)
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_portable_paths.py tests/test_doctor_cli.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Implement resolution and JSON diagnostics**
 ```python
@dataclass(frozen=True)
 class ResolvedPaths:
    workspace: Path
    sessions: Path
    artifacts: Path
    indexes: Path
    corpus: Path
 ```
 `doctor --json` returns component statuses without printing warnings to stdout.
 - [ ] **Step 4: Run tests**
 Run: `cd harness && .venv/bin/pytest tests/test_portable_paths.py tests/test_doctor_cli.py tests/test_workspace.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/paths.py harness/tht/cli/doctor_cmd.py harness/tht/config.py harness/tht/cli/__init__.py harness/tests/test_portable_paths.py harness/tests/test_doctor_cli.py
 git commit -m "feat(storage): resolve portable workspace roots"
 ```
 ### Task 2: Make backend process paths and listening address container-safe
 **Files:**
 - Modify: `backend/src/config.ts`
 - Modify: `backend/src/server.ts`
 - Modify: `backend/src/pi/pi-process-manager.ts`
 - Modify: `backend/src/tht/tht-runner.ts`
 - Test: `backend/test/config.test.ts`
 - Test: `backend/test/pi-process-manager.test.ts`
 **Interfaces:**
 - Produces env contract: `HOST`, `PORT`, `THT_HARNESS_DIR`, `THT_BIN`, `PI_BIN`, `SETTINGS_FILE`, `THT_DATA_ROOT`.
 - [ ] **Step 1: Add tests for explicit binaries and `0.0.0.0` listening**
 ```typescript
 expect(loadConfig({ HOST: "0.0.0.0", THT_BIN: "/opt/venv/bin/tht" })).toMatchObject({
  host: "0.0.0.0", thtBin: "/opt/venv/bin/tht"
 });
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd backend && npx vitest run test/config.test.ts test/pi-process-manager.test.ts`  
 Expected: FAIL for missing host/data-root behavior.
 - [ ] **Step 3: Remove the `.venv/bin` PATH assumption**
 ```typescript
 const env = { ...process.env, THT_DATA_ROOT: cfg.dataRoot };
 const child = spawn(cfg.piBin, ["--mode", "rpc"], { cwd: cfg.harnessDir, env });
 ```
 - [ ] **Step 4: Run backend tests and typecheck**
 Run: `cd backend && npx vitest run && npx tsc --noEmit -p .`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add backend/src backend/test
 git commit -m "feat(backend): support container runtime paths"
 ```
 ### Task 3: Build the core image
 **Files:**
 - Create: `docker/core.Dockerfile`
 - Create: `docker/core-entrypoint.sh`
 - Create: `.dockerignore`
 - Create: `docker/smoke/core-smoke.sh`
 **Interfaces:**
 - Produces image entrypoints: `server`, `doctor`, `preprocess`, and arbitrary `tht ...`.
 - [ ] **Step 1: Add a smoke script that asserts binaries and health**
 ```sh
 test "$(id -u)" != "0"
 node --version
 python --version
 tht --help >/dev/null
 pi --version >/dev/null
 curl --fail http://127.0.0.1:8787/health
 ```
 - [ ] **Step 2: Build and observe the initial failure**
 Run: `docker build -f docker/core.Dockerfile -t thothii-core:test .`  
 Expected: FAIL because the Dockerfile is not present before implementation.
 - [ ] **Step 3: Implement a multi-stage core build**
 ```dockerfile
 FROM node:22-bookworm AS backend-build
 WORKDIR /src/backend
 COPY backend/package*.json ./
 RUN npm ci
 COPY backend/ ./
 RUN npm run build
 FROM python:3.11-slim-bookworm AS runtime
 RUN useradd --create-home --uid 10001 thoth
 WORKDIR /app
 COPY harness/ /app/harness/
 RUN python -m venv /opt/venv && /opt/venv/bin/pip install --no-cache-dir /app/harness
 COPY --from=backend-build /src/backend/dist /app/backend/dist
 COPY --from=backend-build /src/backend/node_modules /app/backend/node_modules
 ENV PATH="/opt/venv/bin:$PATH" HOST=0.0.0.0 PORT=8787
 USER thoth
 ENTRYPOINT ["/app/docker/core-entrypoint.sh"]
 CMD ["server"]
 ```
 Add Pi installation only from its pinned, redistributable source after the Phase 0 license/runtime gate; fail the build if `pi --version` is unavailable.
 - [ ] **Step 4: Build and run smoke test**
 Run: `docker build -f docker/core.Dockerfile -t thothii-core:test .`  
 Expected: success.  
 Run: `docker run --rm thothii-core:test doctor`  
 Expected: process starts and reports missing external configuration without traceback.
 - [ ] **Step 5: Commit**
 ```bash
 git add docker/core.Dockerfile docker/core-entrypoint.sh docker/smoke/core-smoke.sh .dockerignore
 git commit -m "build(docker): add core application image"
 ```
 ### Task 4: Add runtime frontend configuration and image
 **Files:**
 - Create: `frontend/public/config.js`
 - Create: `frontend/src/api/runtime-config.ts`
 - Modify: `frontend/src/api/client.ts`
 - Modify: `frontend/index.html`
 - Create: `docker/frontend.Dockerfile`
 - Create: `docker/frontend-entrypoint.sh`
 - Create: `docker/nginx.conf.template`
 - Test: `frontend/src/api/runtime-config.test.ts`
 **Interfaces:**
 - Produces browser contract: `window.__THOTHII_CONFIG__.backendBaseUrl`.
 - [ ] **Step 1: Write fallback and injected-config tests**
 ```typescript
 expect(resolveBackendUrl({ backendBaseUrl: "/api" })).toBe("/api");
 expect(resolveBackendUrl(undefined)).toBe(import.meta.env.VITE_BACKEND_URL ?? "");
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd frontend && npx vitest run src/api/runtime-config.test.ts`  
 Expected: FAIL.
 - [ ] **Step 3: Implement runtime config generation and reverse proxy**
 ```sh
 sed "s|__BACKEND_BASE_URL__|${BACKEND_BASE_URL:-/api}|g" \
  /usr/share/nginx/html/config.template.js > /usr/share/nginx/html/config.js
 exec nginx -g 'daemon off;'
 ```
 - [ ] **Step 4: Test and build**
 Run: `cd frontend && npx vitest run && npx tsc -b && npm run build`  
 Expected: PASS.  
 Run: `docker build -f docker/frontend.Dockerfile -t thothii-frontend:test .`  
 Expected: success.
 - [ ] **Step 5: Commit**
 ```bash
 git add frontend docker/frontend.Dockerfile docker/frontend-entrypoint.sh docker/nginx.conf.template
 git commit -m "build(docker): add runtime-configured frontend image"
 ```
 ### Task 5: Compose external profile and end-to-end smoke gate
 **Files:**
 - Create: `compose.yaml`
 - Create: `deploy/env.example`
 - Create: `deploy/workspaces/example.yaml`
 - Create: `scripts/docker-smoke.sh`
 - Modify: `README.md`
 - Test: `backend/test/health.test.ts`
 **Interfaces:**
 - Produces services `core` and `frontend`; persistent volume/mount contract beneath `/data`.
 - [ ] **Step 1: Add a smoke script for Compose config and HTTP health**
 ```sh
 docker compose config --quiet
 docker compose up --build --wait core frontend
 curl --fail http://localhost:8080/health
 docker compose down
 ```
 - [ ] **Step 2: Verify Compose is initially absent**
 Run: `docker compose config --quiet`  
 Expected: FAIL before `compose.yaml` is implemented.
 - [ ] **Step 3: Define the base deployment**
 ```yaml
 services:
  core:
    build: { context: ., dockerfile: docker/core.Dockerfile }
    environment:
      THT_DATA_ROOT: /data
      SETTINGS_FILE: /data/settings/settings.json
    volumes: ["./deploy/workspaces:/data/workspaces:ro", "thoth_data:/data"]
  frontend:
    build: { context: ., dockerfile: docker/frontend.Dockerfile }
    environment: { BACKEND_BASE_URL: /api }
    ports: ["8080:8080"]
 volumes: { thoth_data: {} }
 ```
 - [ ] **Step 4: Run Compose smoke and full layer gates**
 Run: `./scripts/docker-smoke.sh`  
 Expected: both health checks PASS.  
 Run: `cd harness && .venv/bin/pytest -q`  
 Run: `cd backend && npx vitest run && npx tsc --noEmit -p .`  
 Run: `cd frontend && npx vitest run && npx tsc -b`  
 Expected: all PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add compose.yaml deploy scripts/docker-smoke.sh README.md
 git commit -m "feat(deploy): add portable external-service stack"
 ```
@@ -0,0 +1,373 @@
 # Evidence Sources and Preprocessing Implementation Plan
 > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
 **Goal:** Materialize heterogeneous Evidence sources into a versioned canonical corpus and index it through resumable, idempotent jobs.
 **Architecture:** Source adapters only discover and acquire. Pure normalization/chunking stages create immutable version artifacts; vector indexing writes a staging generation; publish atomically switches the active manifest. DWH preprocessing uses the same job envelope but a separate pipeline.
 **Tech Stack:** Python 3.11+, Pydantic 2, requests, optional `fsspec`/S3 client, existing embeddings/vector ports, Typer, pytest.
 ## Global Constraints
 - Plans 1-3 are complete.
 - Runtime search reads only the active canonical corpus and vector generation.
 - Filesystem and HTTP sources are MVP; S3-compatible follows on the same port.
 - Source credentials never enter corpus metadata or logs.
 - Failed runs never replace the last valid published generation.
 - Document and DWH preprocessing are separate jobs with shared lock/report infrastructure.
 ---
 ### Task 1: Define Evidence source port and canonical records
 **Files:**
 - Create: `harness/tht/ports/evidence.py`
 - Create: `harness/tht/corpus/models.py`
 - Test: `harness/tests/test_evidence_port_contract.py`
 - Test: `harness/tests/test_corpus_models.py`
 **Interfaces:**
 - Produces: `EvidenceSource`, `SourceObject`, `AcquiredDocument`, `CanonicalDocument`, `CanonicalChunk`, `CorpusManifest`.
 - [ ] **Step 1: Write serialization and secret-exclusion tests**
 ```python
 def test_manifest_contains_provenance_without_credentials():
    manifest = CorpusManifest(documents=[document(source_uri="https://host/a.md")])
    payload = manifest.model_dump_json()
    assert "https://host/a.md" in payload
    assert "api_key" not in payload
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_evidence_port_contract.py tests/test_corpus_models.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Define immutable records and source protocol**
 ```python
@runtime_checkable
 class EvidenceSource(Protocol):
    def discover(self) -> Iterable[SourceObject]: ...
    def acquire(self, item: SourceObject) -> AcquiredDocument: ...
 class SourceObject(BaseModel, frozen=True):
    source_id: str
    uri: str
    fingerprint: str
    modified_at: datetime | None = None
    metadata: dict[str, JsonValue] = {}
 ```
 - [ ] **Step 4: Run model and protocol tests**
 Run: `cd harness && .venv/bin/pytest tests/test_evidence_port_contract.py tests/test_corpus_models.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/ports/evidence.py harness/tht/corpus/models.py harness/tests/test_evidence_port_contract.py harness/tests/test_corpus_models.py
 git commit -m "feat(evidence): define source and corpus contracts"
 ```
 ### Task 2: Implement filesystem and HTTP source adapters
 **Files:**
 - Create: `harness/tht/adapters/evidence/__init__.py`
 - Create: `harness/tht/adapters/evidence/filesystem.py`
 - Create: `harness/tht/adapters/evidence/http.py`
 - Modify: `harness/tht/config.py`
 - Modify: `harness/tht/adapters/factory.py`
 - Test: `harness/tests/test_filesystem_evidence_source.py`
 - Test: `harness/tests/test_http_evidence_source.py`
 **Interfaces:**
 - Produces: `FilesystemEvidenceSource`, `HttpManifestEvidenceSource`, `build_evidence_sources(cfg)`.
 - [ ] **Step 1: Test deterministic discovery and conditional HTTP acquisition**
 ```python
 def test_filesystem_discovery_is_stable(source):
    assert [x.uri for x in source.discover()] == sorted(x.uri for x in source.discover())
 def test_http_uses_etag_for_fingerprint(http_source):
    assert next(http_source.discover()).fingerprint == 'etag:"abc"'
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_filesystem_evidence_source.py tests/test_http_evidence_source.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Implement adapters with bounded reads**
 ```python
 class FilesystemEvidenceSource:
    def discover(self):
        for path in sorted(self.root.rglob("*.md")):
            yield SourceObject(source_id=stable_id(path), uri=path.as_uri(),
                               fingerprint=sha256_file(path))
 ```
 HTTP uses a declared manifest of URLs, connect/read timeouts, maximum bytes, ETag/Last-Modified where available, and content hashing as fallback.
 - [ ] **Step 4: Run adapter tests**
 Run: `cd harness && .venv/bin/pytest tests/test_filesystem_evidence_source.py tests/test_http_evidence_source.py tests/test_config_resources.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/adapters/evidence harness/tht/config.py harness/tht/adapters/factory.py harness/tests/test_filesystem_evidence_source.py harness/tests/test_http_evidence_source.py
 git commit -m "feat(evidence): add filesystem and HTTP sources"
 ```
 ### Task 3: Build deterministic normalization and chunking
 **Files:**
 - Create: `harness/tht/corpus/normalize.py`
 - Create: `harness/tht/corpus/chunk.py`
 - Test: `harness/tests/test_corpus_normalize.py`
 - Test: `harness/tests/test_corpus_chunk.py`
 **Interfaces:**
 - Produces: `normalize(acquired, pipeline_version) -> CanonicalDocument` and `chunk(document, policy) -> list[CanonicalChunk]`.
 - [ ] **Step 1: Pin UTF-8, frontmatter, line-ending, and stable chunk-id behavior**
 ```python
 def test_chunk_ids_are_stable_for_same_content():
    first = chunk(document("A\n\nB"), policy(max_chars=8))
    second = chunk(document("A\r\n\r\nB"), policy(max_chars=8))
    assert [x.chunk_id for x in first] == [x.chunk_id for x in second]
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_corpus_normalize.py tests/test_corpus_chunk.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Implement pure deterministic transforms**
 ```python
 chunk_id = sha256(f"{document.content_hash}:{ordinal}:{policy.version}".encode()).hexdigest()
 ```
 Reject undecodable or oversized content with a typed permanent error; never silently truncate source documents.
 - [ ] **Step 4: Run normalization/chunk tests**
 Run: `cd harness && .venv/bin/pytest tests/test_corpus_normalize.py tests/test_corpus_chunk.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/corpus/normalize.py harness/tht/corpus/chunk.py harness/tests/test_corpus_normalize.py harness/tests/test_corpus_chunk.py
 git commit -m "feat(corpus): add deterministic normalization and chunking"
 ```
 ### Task 4: Add shared job envelope, locking, and reports
 **Files:**
 - Create: `harness/tht/jobs/models.py`
 - Create: `harness/tht/jobs/runner.py`
 - Create: `harness/tht/jobs/locking.py`
 - Test: `harness/tests/test_job_runner.py`
 - Test: `harness/tests/test_job_locking.py`
 **Interfaces:**
 - Produces: `JobSpec`, `JobRun`, `JobReport`, `WorkspaceJobLock`, `run_job(spec, stages)`.
 - [ ] **Step 1: Test lock exclusion, resume, and JSON report schema**
 ```python
 def test_failed_stage_is_resumable(tmp_path):
    first = run_job(spec, [ok_stage, failing_stage])
    second = run_job(spec.with_resume(first.run_id), [ok_stage, recovered_stage])
    assert second.resumed_from == first.run_id
    assert second.status == "succeeded"
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_job_runner.py tests/test_job_locking.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Implement atomic report writes and workspace-scoped locks**
 ```python
 tmp = report_path.with_suffix(".tmp")
 tmp.write_text(report.model_dump_json(indent=2))
 tmp.replace(report_path)
 ```
 Persist checkpoints after each stage; ensure a crashed process leaves the active corpus untouched.
 - [ ] **Step 4: Run job infrastructure tests**
 Run: `cd harness && .venv/bin/pytest tests/test_job_runner.py tests/test_job_locking.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/jobs harness/tests/test_job_runner.py harness/tests/test_job_locking.py
 git commit -m "feat(jobs): add resumable preprocessing envelope"
 ```
 ### Task 5: Implement incremental document pipeline and atomic publish
 **Files:**
 - Create: `harness/tht/corpus/pipeline.py`
 - Create: `harness/tht/corpus/store.py`
 - Create: `harness/tht/cli/preprocess_cmd.py`
 - Modify: `harness/tht/cli/__init__.py`
 - Create: `harness/tht/search/evidence.py`
 - Test: `harness/tests/test_corpus_pipeline.py`
 - Test: `harness/tests/test_corpus_publish.py`
 - Test: `harness/tests/test_preprocess_cli.py`
 **Interfaces:**
 - Produces: `tht preprocess evidence [--dry-run] [--resume RUN_ID] [--json]`; active pointer `corpus/<workspace>/ACTIVE`.
 - Consumes: Evidence sources, canonical transforms, embedder, and `VectorStore`.
 - [ ] **Step 1: Test incremental skip and failed-run isolation**
 ```python
 def test_failed_generation_does_not_replace_active(corpus_store, pipeline):
    old = corpus_store.publish(valid_generation())
    with pytest.raises(StageError): pipeline.run(source_with_failure())
    assert corpus_store.active_generation() == old
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_corpus_pipeline.py tests/test_corpus_publish.py tests/test_preprocess_cli.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Implement staged generations and compare fingerprints**
 ```python
 changed = {
    item.source_id for item in discovered
    if previous.fingerprints.get(item.source_id) != item.fingerprint
 }
 ```
 Index changed chunks, mark removed documents, validate counts and embedding dimensions, then atomically replace `ACTIVE`. Make runtime Evidence lookup resolve files through the active manifest instead of `rglob` on the source directory.
 - [ ] **Step 4: Run pipeline and existing search/session tests**
 Run: `cd harness && .venv/bin/pytest tests/test_corpus_pipeline.py tests/test_corpus_publish.py tests/test_preprocess_cli.py tests/test_search_pack.py tests/test_session_documents.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/corpus harness/tht/cli/preprocess_cmd.py harness/tht/cli/__init__.py harness/tht/search/evidence.py harness/tests/test_corpus_pipeline.py harness/tests/test_corpus_publish.py harness/tests/test_preprocess_cli.py
 git commit -m "feat(preprocess): publish incremental Evidence corpus"
 ```
 ### Task 6: Move DWH introspection and LSH into separate jobs
 **Files:**
 - Create: `harness/tht/jobs/dwh_pipeline.py`
 - Modify: `harness/tht/cli/schema_cmd.py`
 - Modify: `harness/tht/cli/lsh_cmd.py`
 - Modify: `harness/tht/cli/preprocess_cmd.py`
 - Test: `harness/tests/test_dwh_preprocess_job.py`
 - Test: `harness/tests/test_lsh_job_resume.py`
 **Interfaces:**
 - Produces: `tht preprocess dwh --steps introspect,lsh [--resume RUN_ID] [--json]`.
 - [ ] **Step 1: Test independent document/DWH locks and LSH resume**
 ```python
 def test_dwh_and_evidence_jobs_have_distinct_lock_names():
    assert job_lock_name("demo", "dwh") != job_lock_name("demo", "evidence")
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_dwh_preprocess_job.py tests/test_lsh_job_resume.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Wrap existing commands as stages without changing their core algorithms**
 ```python
 stages = {
    "introspect": lambda ctx: refresh_catalog(ctx.dwh, ctx.paths.artifacts),
    "lsh": lambda ctx: build_lsh(ctx.dwh, ctx.paths.indexes, ctx.checkpoint),
 }
 ```
 - [ ] **Step 4: Run DWH/LSH regression and full harness suite**
 Run: `cd harness && .venv/bin/pytest tests/test_dwh_preprocess_job.py tests/test_lsh_job_resume.py tests/test_schema_introspect_guard.py tests/l0/test_db_sampling.py -q`  
 Run: `cd harness && .venv/bin/pytest -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/jobs/dwh_pipeline.py harness/tht/cli/schema_cmd.py harness/tht/cli/lsh_cmd.py harness/tht/cli/preprocess_cmd.py harness/tests/test_dwh_preprocess_job.py harness/tests/test_lsh_job_resume.py
 git commit -m "feat(preprocess): add resumable DWH jobs"
 ```
 ### Task 7: Add Compose job profiles, S3 follow-up adapter, and operational gates
 **Files:**
 - Modify: `compose.yaml`
 - Create: `harness/tht/adapters/evidence/s3.py`
 - Modify: `harness/pyproject.toml`
 - Test: `harness/tests/test_s3_evidence_source.py`
 - Create: `scripts/preprocess-smoke.sh`
 - Modify: `README.md`
 **Interfaces:**
 - Produces Compose profile `preprocess`; optional `s3` source type using endpoint URL, bucket, prefix, and secret references.
 - [ ] **Step 1: Add S3 contract tests against a fake endpoint and a Compose job smoke**
 ```python
 def test_s3_uri_and_fingerprint(source):
    item = next(source.discover())
    assert item.uri == "s3://evidence/clinical/a.md"
    assert item.fingerprint.startswith("etag:")
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/test_s3_evidence_source.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Implement S3 on the established port and Compose one-shot services**
 ```yaml
 preprocess-evidence:
  image: thothii-core:${THOTHII_TAG:-latest}
  profiles: ["preprocess"]
  command: ["preprocess", "evidence", "--json"]
  volumes: ["thoth_data:/data"]
 ```
 - [ ] **Step 4: Run all operational gates**
 Run: `./scripts/preprocess-smoke.sh`  
 Expected: second run reports all documents unchanged; a modified file creates and publishes one new generation.  
 Run: `cd harness && .venv/bin/pytest -q`  
 Run: `cd harness && .venv/bin/ruff check tht tests/test_*evidence* tests/test_corpus* tests/test_*job*`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add compose.yaml harness/tht/adapters/evidence/s3.py harness/pyproject.toml harness/tests/test_s3_evidence_source.py scripts/preprocess-smoke.sh README.md
 git commit -m "feat(preprocess): add deployment jobs and S3 source"
 ```
@@ -0,0 +1,211 @@
 # Optional Local pgvector Implementation Plan
 > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
 **Goal:** Let server and desktop deployments run an optional persistent pgvector service with behavior equivalent to the HTTP vector adapter.
 **Architecture:** Implement the final direct `VectorStore`, version vector schema migrations, add a standard pgvector image to Compose, and provide operational backup/restore commands.
 **Tech Stack:** PostgreSQL 16, pgvector, psycopg2/SQLAlchemy, Alembic or ordered SQL migrations, Docker Compose, pytest/testcontainers.
 ## Global Constraints
 - Plans 1 and 2 are complete.
 - pgvector is an infrastructure image, not a ThothII-owned application image.
 - Reader and writer roles are distinct even for local deployments.
 - Existing remote HTTP vector behavior remains supported.
 - Persistent data must survive application image replacement.
 ---
 ### Task 1: Implement direct pgvector store behind `VectorStore`
 **Files:**
 - Create: `harness/tht/adapters/vector/pgvector.py`
 - Test: `harness/tests/l0/test_pgvector_store.py`
 - Modify: `harness/tht/adapters/factory.py`
 - Modify: `harness/tht/config.py`
 **Interfaces:**
 - Produces: `PgVectorStore(read_config, write_config=None)` implementing the Plan 1 port.
 - [ ] **Step 1: Add L0 contract tests for search, kind filters, hashes, and upsert**
 ```python
 def test_pgvector_round_trip(store):
    assert store.upsert("memory", [record("a", [1.0, 0.0])]) == 1
    hits = store.search(["memory"], [1.0, 0.0], limit=5, kinds=["memory"])
    assert hits[0].metadata["content_hash"] == "a"
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/l0/test_pgvector_store.py -q`  
 Expected: FAIL because `PgVectorStore` is absent.
 - [ ] **Step 3: Implement parameterized SQL with allowlisted collection names**
 ```python
 def _collection(name: str) -> sql.Identifier:
    if name not in ALLOWED_COLLECTIONS:
        raise VectorStoreError(f"Collection not allowed: {name}")
    return sql.Identifier("vectors", name)
 ```
 Do not interpolate untrusted identifiers; reuse current metadata shapes and cosine distance ordering.
 - [ ] **Step 4: Run direct and HTTP parity tests**
 Run: `cd harness && .venv/bin/pytest tests/l0/test_pgvector_store.py tests/test_vector_port_contract.py tests/test_search_similar_kinds.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/tht/adapters/vector/pgvector.py harness/tht/adapters/factory.py harness/tht/config.py harness/tests/l0/test_pgvector_store.py
 git commit -m "feat(vector): add direct pgvector adapter"
 ```
 ### Task 2: Version schema and roles
 **Files:**
 - Create: `harness/migrations/vector/001_extensions.sql`
 - Create: `harness/migrations/vector/002_schema_tables.sql`
 - Create: `harness/migrations/vector/003_roles.sql`
 - Create: `harness/tht/cli/vector_migrate_cmd.py`
 - Test: `harness/tests/l0/test_vector_migrations.py`
 **Interfaces:**
 - Produces: `tht vector migrate`, `tht vector migrate --status --json`.
 - [ ] **Step 1: Test clean install and idempotent rerun**
 ```python
 def test_migrations_are_idempotent(database_url):
    migrate(database_url)
    migrate(database_url)
    assert migration_status(database_url).pending == []
 ```
 - [ ] **Step 2: Verify failure**
 Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_migrations.py -q`  
 Expected: FAIL.
 - [ ] **Step 3: Add ordered migrations and least-privilege roles**
 ```sql
 CREATE SCHEMA IF NOT EXISTS vectors;
 CREATE TABLE IF NOT EXISTS vectors.schema (..., embedding vector(768) NOT NULL);
 CREATE TABLE IF NOT EXISTS vectors.memory (..., embedding vector(768) NOT NULL);
 REVOKE ALL ON SCHEMA vectors FROM PUBLIC;
 ```
 Create reader grants for SELECT/search and writer grants for controlled insert/update; credentials are injected at deployment, not stored in SQL files.
 - [ ] **Step 4: Run migration and existing vector tests**
 Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_migrations.py tests/l0/test_pgvector_store.py -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add harness/migrations/vector harness/tht/cli/vector_migrate_cmd.py harness/tests/l0/test_vector_migrations.py
 git commit -m "feat(vector): version pgvector schema"
 ```
 ### Task 3: Add `local-vector` Compose profile
 **Files:**
 - Modify: `compose.yaml`
 - Create: `deploy/vector/init/00-bootstrap.sh`
 - Modify: `deploy/env.example`
 - Create: `scripts/local-vector-smoke.sh`
 **Interfaces:**
 - Produces service `vector-db`, volume `vector_data`, health-gated core dependency in the profile.
 - [ ] **Step 1: Add failing profile smoke command**
 Run: `docker compose --profile local-vector config --services`  
 Expected: output does not yet contain `vector-db`.
 - [ ] **Step 2: Define the standard infrastructure service**
 ```yaml
 vector-db:
  image: pgvector/pgvector:pg16
  profiles: ["local-vector"]
  volumes: ["vector_data:/var/lib/postgresql/data"]
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]
    interval: 5s
    timeout: 3s
    retries: 20
 ```
 - [ ] **Step 3: Wire the local resource config and migration job**
 Add a one-shot `vector-migrate` service using `thothii-core`; it must complete successfully before preprocessing writes.
 - [ ] **Step 4: Run local vector smoke**
 Run: `./scripts/local-vector-smoke.sh`  
 Expected: migration succeeds, one record is indexed, and search returns it after restarting `core`.
 - [ ] **Step 5: Commit**
 ```bash
 git add compose.yaml deploy/vector deploy/env.example scripts/local-vector-smoke.sh
 git commit -m "feat(deploy): add optional local pgvector profile"
 ```
 ### Task 4: Add backup, restore, and parity gates
 **Files:**
 - Create: `scripts/vector-backup.sh`
 - Create: `scripts/vector-restore.sh`
 - Create: `harness/tests/l0/test_vector_adapter_parity.py`
 - Modify: `README.md`
 **Interfaces:**
 - Produces versioned custom-format dumps and explicit restore into an empty target.
 - [ ] **Step 1: Add adapter parity scenarios**
 ```python
@pytest.mark.parametrize("store_fixture", ["direct_store", "http_store"])
 def test_kind_filtered_search_parity(request, store_fixture):
    store = request.getfixturevalue(store_fixture)
    assert normalize(store.search(...)) == EXPECTED_HITS
 ```
 - [ ] **Step 2: Verify parity test exposes any semantic differences**
 Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_adapter_parity.py -q`  
 Expected: FAIL until result ordering/error mapping is aligned.
 - [ ] **Step 3: Implement backup/restore safety checks**
 ```sh
 pg_dump --format=custom --schema=vectors --file="$OUTPUT" "$DATABASE_URL"
 pg_restore --exit-on-error --clean --if-exists --dbname="$TARGET_DATABASE_URL" "$INPUT"
 ```
 Require explicit target and refuse restore when it equals the active source URL.
 - [ ] **Step 4: Run parity, backup/restore, and full harness gates**
 Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_adapter_parity.py tests/l0/test_pgvector_store.py -q`  
 Run: `./scripts/local-vector-smoke.sh --backup-restore`  
 Run: `cd harness && .venv/bin/pytest -q`  
 Expected: PASS.
 - [ ] **Step 5: Commit**
 ```bash
 git add scripts/vector-backup.sh scripts/vector-restore.sh harness/tests/l0/test_vector_adapter_parity.py README.md
 git commit -m "docs(vector): add local backup restore and parity gate"
 ```
@@ -0,0 +1,47 @@
 # Portable Deployment Program
 > **For agentic workers:** execute the linked plans in order. Each plan ends with a compatibility gate and can be released independently.
 **Goal:** Deliver the approved portable ThothII architecture through four independently reviewable implementation plans.
 **Architecture:** Preserve the current frontend → backend → Pi → tht workflow while moving DWH, vector, and Evidence access behind typed adapters. Build two application images and compose optional pgvector and preprocessing services through deployment profiles.
 **Tech Stack:** Python 3.11+, Pydantic 2, SQLAlchemy/PostgreSQL, Fastify/TypeScript, React/Vite, Docker BuildKit, Docker Compose, PostgreSQL+pgvector.
 ## Global Constraints
 - Keep `harness/workflow.yaml` and phase semantics unchanged.
 - Keep session documents and `review_decisions.jsonl` as the persistence truth.
 - Preserve pristine JSON stdout for every `tht --json` command.
 - Keep DWH access read-only by credentials and client-side guards.
 - Keep vector reader and writer credentials separate.
 - Produce exactly two ThothII-owned application images; infrastructure images are optional dependencies.
 - Preserve current workspace behavior through an explicit migration window.
 - UI strings remain English; workspace content retains its configured language.
 - Run harness pytest, gate JS tests, backend vitest+tsc, and frontend vitest+tsc before release.
 ---
 ## Ordered plans
 1. [Adapter Foundations](2026-07-11-adapter-foundations.md)  
   Establishes protocols, capability reporting, factories, and backwards-compatible configuration.
 2. [Container Packaging and Portable Storage](2026-07-11-container-packaging-portable-storage.md)  
   Builds `thothii-core` and `thothii-frontend`, runtime configuration, logical roots, Compose base, and diagnostics.
 3. [Optional Local pgvector](2026-07-11-local-pgvector-profile.md)  
   Adds the direct vector adapter, schema migration tooling, persistent service profile, backup/restore, and parity tests.
 4. [Evidence Sources and Preprocessing](2026-07-11-evidence-preprocessing.md)  
   Adds source adapters, canonical corpus, incremental manifests, atomic publish, and separate document/DWH jobs.
 ## Program gates
 - [ ] After Plan 1, current server and workstation workspaces behave identically through the new factories.
 - [ ] After Plan 2, the current external-service installation runs from the two images.
 - [ ] After Plan 3, profiles B and C run with an optional local pgvector volume.
 - [ ] After Plan 4, runtime retrieval no longer requires live access to original Evidence sources.
 - [ ] Complete one L2 session for each deployed DWH transport and vector transport pairing in scope.
 - [ ] Update `PROJECT_STATE.md` only after each plan's verification evidence is available.