# Optional Local pgvector Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Let server and desktop deployments run an optional persistent pgvector service with behavior equivalent to the HTTP vector adapter.

**Architecture:** Implement the final direct `VectorStore`, version vector schema migrations, add a standard pgvector image to Compose, and provide operational backup/restore commands.

**Tech Stack:** PostgreSQL 16, pgvector, psycopg2/SQLAlchemy, Alembic or ordered SQL migrations, Docker Compose, pytest/testcontainers.

## Global Constraints

- Plans 1 and 2 are complete.
- pgvector is an infrastructure image, not a ThothII-owned application image.
- Reader and writer roles are distinct even for local deployments.
- Existing remote HTTP vector behavior remains supported.
- Persistent data must survive application image replacement.

---

### Task 1: Implement direct pgvector store behind `VectorStore`

**Files:**
- Create: `harness/tht/adapters/vector/pgvector.py`
- Test: `harness/tests/l0/test_pgvector_store.py`
- Modify: `harness/tht/adapters/factory.py`
- Modify: `harness/tht/config.py`

**Interfaces:**
- Produces: `PgVectorStore(read_config, write_config=None)` implementing the Plan 1 port.

- [ ] **Step 1: Add L0 contract tests for search, kind filters, hashes, and upsert**

```python
def test_pgvector_round_trip(store):
    assert store.upsert("memory", [record("a", [1.0, 0.0])]) == 1
    hits = store.search(["memory"], [1.0, 0.0], limit=5, kinds=["memory"])
    assert hits[0].metadata["content_hash"] == "a"
```

- [ ] **Step 2: Verify failure**

Run: `cd harness && .venv/bin/pytest tests/l0/test_pgvector_store.py -q`  
Expected: FAIL because `PgVectorStore` is absent.

- [ ] **Step 3: Implement parameterized SQL with allowlisted collection names**

```python
def _collection(name: str) -> sql.Identifier:
    if name not in ALLOWED_COLLECTIONS:
        raise VectorStoreError(f"Collection not allowed: {name}")
    return sql.Identifier("vectors", name)
```

Do not interpolate untrusted identifiers; reuse current metadata shapes and cosine distance ordering.

- [ ] **Step 4: Run direct and HTTP parity tests**

Run: `cd harness && .venv/bin/pytest tests/l0/test_pgvector_store.py tests/test_vector_port_contract.py tests/test_search_similar_kinds.py -q`  
Expected: PASS.

- [ ] **Step 5: Commit**

```bash
git add harness/tht/adapters/vector/pgvector.py harness/tht/adapters/factory.py harness/tht/config.py harness/tests/l0/test_pgvector_store.py
git commit -m "feat(vector): add direct pgvector adapter"
```

### Task 2: Version schema and roles

**Files:**
- Create: `harness/migrations/vector/001_extensions.sql`
- Create: `harness/migrations/vector/002_schema_tables.sql`
- Create: `harness/migrations/vector/003_roles.sql`
- Create: `harness/tht/cli/vector_migrate_cmd.py`
- Test: `harness/tests/l0/test_vector_migrations.py`

**Interfaces:**
- Produces: `tht vector migrate`, `tht vector migrate --status --json`.

- [ ] **Step 1: Test clean install and idempotent rerun**

```python
def test_migrations_are_idempotent(database_url):
    migrate(database_url)
    migrate(database_url)
    assert migration_status(database_url).pending == []
```

- [ ] **Step 2: Verify failure**

Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_migrations.py -q`  
Expected: FAIL.

- [ ] **Step 3: Add ordered migrations and least-privilege roles**

```sql
CREATE SCHEMA IF NOT EXISTS vectors;
CREATE TABLE IF NOT EXISTS vectors.schema (..., embedding vector(768) NOT NULL);
CREATE TABLE IF NOT EXISTS vectors.memory (..., embedding vector(768) NOT NULL);
REVOKE ALL ON SCHEMA vectors FROM PUBLIC;
```

Create reader grants for SELECT/search and writer grants for controlled insert/update; credentials are injected at deployment, not stored in SQL files.

- [ ] **Step 4: Run migration and existing vector tests**

Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_migrations.py tests/l0/test_pgvector_store.py -q`  
Expected: PASS.

- [ ] **Step 5: Commit**

```bash
git add harness/migrations/vector harness/tht/cli/vector_migrate_cmd.py harness/tests/l0/test_vector_migrations.py
git commit -m "feat(vector): version pgvector schema"
```

### Task 3: Add `local-vector` Compose profile

**Files:**
- Modify: `compose.yaml`
- Create: `deploy/vector/init/00-bootstrap.sh`
- Modify: `deploy/env.example`
- Create: `scripts/local-vector-smoke.sh`

**Interfaces:**
- Produces service `vector-db`, volume `vector_data`, health-gated core dependency in the profile.

- [ ] **Step 1: Add failing profile smoke command**

Run: `docker compose --profile local-vector config --services`  
Expected: output does not yet contain `vector-db`.

- [ ] **Step 2: Define the standard infrastructure service**

```yaml
vector-db:
  image: pgvector/pgvector:pg16
  profiles: ["local-vector"]
  volumes: ["vector_data:/var/lib/postgresql/data"]
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]
    interval: 5s
    timeout: 3s
    retries: 20
```

- [ ] **Step 3: Wire the local resource config and migration job**

Add a one-shot `vector-migrate` service using `thothii-core`; it must complete successfully before preprocessing writes.

- [ ] **Step 4: Run local vector smoke**

Run: `./scripts/local-vector-smoke.sh`  
Expected: migration succeeds, one record is indexed, and search returns it after restarting `core`.

- [ ] **Step 5: Commit**

```bash
git add compose.yaml deploy/vector deploy/env.example scripts/local-vector-smoke.sh
git commit -m "feat(deploy): add optional local pgvector profile"
```

### Task 4: Add backup, restore, and parity gates

**Files:**
- Create: `scripts/vector-backup.sh`
- Create: `scripts/vector-restore.sh`
- Create: `harness/tests/l0/test_vector_adapter_parity.py`
- Modify: `README.md`

**Interfaces:**
- Produces versioned custom-format dumps and explicit restore into an empty target.

- [ ] **Step 1: Add adapter parity scenarios**

```python
@pytest.mark.parametrize("store_fixture", ["direct_store", "http_store"])
def test_kind_filtered_search_parity(request, store_fixture):
    store = request.getfixturevalue(store_fixture)
    assert normalize(store.search(...)) == EXPECTED_HITS
```

- [ ] **Step 2: Verify parity test exposes any semantic differences**

Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_adapter_parity.py -q`  
Expected: FAIL until result ordering/error mapping is aligned.

- [ ] **Step 3: Implement backup/restore safety checks**

```sh
pg_dump --format=custom --schema=vectors --file="$OUTPUT" "$DATABASE_URL"
pg_restore --exit-on-error --clean --if-exists --dbname="$TARGET_DATABASE_URL" "$INPUT"
```

Require explicit target and refuse restore when it equals the active source URL.

- [ ] **Step 4: Run parity, backup/restore, and full harness gates**

Run: `cd harness && .venv/bin/pytest tests/l0/test_vector_adapter_parity.py tests/l0/test_pgvector_store.py -q`  
Run: `./scripts/local-vector-smoke.sh --backup-restore`  
Run: `cd harness && .venv/bin/pytest -q`  
Expected: PASS.

- [ ] **Step 5: Commit**

```bash
git add scripts/vector-backup.sh scripts/vector-restore.sh harness/tests/l0/test_vector_adapter_parity.py README.md
git commit -m "docs(vector): add local backup restore and parity gate"
```