150 lines
4.2 KiB
Markdown
150 lines
4.2 KiB
Markdown
# Task 4 Report — Narrow harness embedding configuration to internal Ollama
|
|
|
|
## Status
|
|
|
|
Implemented on 2026-08-08 in `/Users/mp/projects/ThothII/.worktrees/git-workspace-registry`.
|
|
|
|
## RED evidence
|
|
|
|
Command:
|
|
|
|
```bash
|
|
cd harness
|
|
./.venv/bin/pytest tests/test_internal_embeddings.py tests/test_config_resources.py -q
|
|
```
|
|
|
|
Observed before implementation:
|
|
|
|
- exit code `1`
|
|
- `10 failed, 10 passed`
|
|
- failures proved the missing `OllamaInternalEmbeddings` client and missing internal-only config validation
|
|
|
|
Representative failures:
|
|
|
|
- `ImportError: cannot import name 'OllamaInternalEmbeddings'`
|
|
- `AttributeError: 'EmbeddingsConfig' object has no attribute 'provider'`
|
|
- config tests `DID NOT RAISE ConfigError` for external provider, API key, and non-private base URL
|
|
|
|
## GREEN evidence
|
|
|
|
Focused behavior suite:
|
|
|
|
```bash
|
|
cd harness
|
|
./.venv/bin/pytest tests/test_internal_embeddings.py tests/test_config_resources.py -q
|
|
```
|
|
|
|
- exit code `0`
|
|
- `20 passed`
|
|
|
|
Relevant harness verification:
|
|
|
|
```bash
|
|
cd harness
|
|
./.venv/bin/pytest tests/test_internal_embeddings.py tests/test_config_resources.py tests/test_ollama_ensure.py -q
|
|
```
|
|
|
|
- exit code `0`
|
|
- `36 passed, 2 warnings`
|
|
|
|
Changed-file lint:
|
|
|
|
```bash
|
|
cd harness
|
|
./.venv/bin/ruff check tht/config.py tht/config_compat.py tht/vectorstore/embeddings.py tht/cli/ollama_cmd.py tests/test_config_resources.py tests/test_internal_embeddings.py
|
|
```
|
|
|
|
- exit code `0`
|
|
- `All checks passed!`
|
|
|
|
Patch hygiene:
|
|
|
|
```bash
|
|
git diff --check
|
|
```
|
|
|
|
- exit code `0`
|
|
|
|
## What changed
|
|
|
|
- translated schema-v3 `resources.embeddings` into the harness-compatible embedding config view
|
|
- validated the internal embedding contract only for that runtime-owned `resources.embeddings` path:
|
|
- provider must be `ollama_internal`
|
|
- model must be `qwen3-embedding:0.6b`
|
|
- dimensions must be `1024`
|
|
- base URL must be `http://embedding:11434` or loopback HTTP on port `11434`
|
|
- extra fields like `api_key` are rejected
|
|
- replaced the active embed client with `OllamaInternalEmbeddings`, using one bounded `/api/embed` request per batch
|
|
- removed task/query prefix rewriting from the active embedding path
|
|
- validated response count, vector dimension, and finite numeric values before returning embeddings
|
|
- kept `tht ollama ensure --json` stdout pristine while warming through the internal client
|
|
|
|
## Self-review
|
|
|
|
- kept changes inside the brief-listed files
|
|
- preserved DWH and session-persistence behavior
|
|
- preserved the legacy `OllamaEmbeddings` import path as an alias to avoid unrelated call-site churn
|
|
|
|
## Concerns
|
|
|
|
- the focused harness verification still emits two pre-existing warnings:
|
|
- `DeprecationWarning` from `testcontainers.postgres`
|
|
- `FutureWarning` because `resources` currently flows through the legacy config translation path
|
|
|
|
## Fix round 1 — 2026-08-08
|
|
|
|
### Findings addressed
|
|
|
|
- HIGH: external top-level `embeddings` remained an operational fallback and could still load
|
|
- MEDIUM: non-object embed JSON payloads escaped as raw `AttributeError`
|
|
|
|
### RED evidence
|
|
|
|
Command:
|
|
|
|
```bash
|
|
cd harness
|
|
./.venv/bin/pytest tests/test_internal_embeddings.py tests/test_config_resources.py tests/test_ollama_ensure.py -q
|
|
```
|
|
|
|
Observed before the fix:
|
|
|
|
- exit code `1`
|
|
- `2 failed, 36 passed, 2 warnings`
|
|
|
|
Representative failures:
|
|
|
|
- `AttributeError: 'list' object has no attribute 'get'` from `response.json()` returning a JSON array
|
|
- `Failed: DID NOT RAISE ConfigError` for top-level external `embeddings.provider=openai_compatible`
|
|
|
|
### GREEN evidence
|
|
|
|
Command:
|
|
|
|
```bash
|
|
cd harness
|
|
./.venv/bin/pytest tests/test_internal_embeddings.py tests/test_config_resources.py tests/test_ollama_ensure.py -q
|
|
```
|
|
|
|
Observed after the fix:
|
|
|
|
- exit code `0`
|
|
- `38 passed, 2 warnings`
|
|
|
|
Touched-file lint:
|
|
|
|
```bash
|
|
cd harness
|
|
./.venv/bin/ruff check tht/config.py tht/vectorstore/embeddings.py tests/test_internal_embeddings.py tests/test_config_resources.py
|
|
```
|
|
|
|
- exit code `0`
|
|
- `All checks passed!`
|
|
|
|
### Minimal fix
|
|
|
|
- validated the final active `cfg.embeddings` contract after config loading, so legacy top-level
|
|
embedding inputs now fail explicitly unless they exactly match the internal Ollama contract
|
|
- converted non-mapping embed JSON payloads into controlled `EmbeddingsError` failures with
|
|
sanitized diagnostics instead of raw attribute errors
|