feat: unify installation model catalog
This commit is contained in:
@@ -83,7 +83,8 @@ frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness →
|
|||||||
`SseHub` fans them out over SSE to the browser. The separate PostgreSQL catalog stores database
|
`SseHub` fans them out over SSE to the browser. The separate PostgreSQL catalog stores database
|
||||||
metadata and sequential AI description-generation runs. Description generation samples the DWH
|
metadata and sequential AI description-generation runs. Description generation samples the DWH
|
||||||
through read-only connectors and calls a short-lived Python LiteLLM helper; it does not use Pi or
|
through read-only connectors and calls a short-lived Python LiteLLM helper; it does not use Pi or
|
||||||
expose a public CLI command. App settings still live in `backend/data/settings.json`.
|
expose a public CLI command. Sessions, metadata generation, and embedding resolve models from the
|
||||||
|
generated Installation Model Catalog; `thothii-installation.yaml` is its only authored source.
|
||||||
|
|
||||||
- **Human-in-the-loop gate contract.** The model proposes; a human reviewer decides at gates
|
- **Human-in-the-loop gate contract.** The model proposes; a human reviewer decides at gates
|
||||||
via widgets (`reviewer_select` = single pick — a chosen option carrying a `decision` payload
|
via widgets (`reviewer_select` = single pick — a chosen option carrying a `decision` payload
|
||||||
@@ -99,11 +100,11 @@ frontend (React/SSE) → backend (Fastify) → pi --mode rpc → tht/harness →
|
|||||||
- **`--json` output must be pristine** (only valid JSON on stdout) — used as a machine contract.
|
- **`--json` output must be pristine** (only valid JSON on stdout) — used as a machine contract.
|
||||||
- **UI strings are English; document *content* stays the workspace language** (Italian for
|
- **UI strings are English; document *content* stays the workspace language** (Italian for
|
||||||
`psd`) because it's the real data. Only chrome/labels are English.
|
`psd`) because it's the real data. Only chrome/labels are English.
|
||||||
- **Workspaces** (`harness/workspaces/*.yaml`) set the DB target and **absolute**
|
- **Workspace schema v4** defines database and Evidence concerns only. Embedding/model facts come
|
||||||
`paths.sessions/artifacts/indexes` — for `psd` these point at a *separate, uncommitted* repo
|
from the installation catalog. The legacy `harness/workspaces/*.yaml` runtime snapshots still use
|
||||||
(`tht-workspace-psd/`). Secrets live ONLY in `harness/.env` (gitignored).
|
absolute session/artifact/index paths; secrets stay in `harness/.env` (gitignored).
|
||||||
- **Settings are global** (`backend/data/settings.json`: workspace/provider/model/thinking);
|
- **Settings are global** (`backend/data/settings.json`: workspace/thinking). Provider/model choices
|
||||||
the New-session form is question-only.
|
are ephemeral canonical catalog selections pinned into the session manifest.
|
||||||
- **Resume**: a resumable session re-enters at its last incomplete phase. The backend refuses
|
- **Resume**: a resumable session re-enters at its last incomplete phase. The backend refuses
|
||||||
resume with 409 when `finalized` or `archived`, and `PiProcessManager.spawnFor` must send
|
resume with 409 when `finalized` or `archived`, and `PiProcessManager.spawnFor` must send
|
||||||
`/riprendi-sessione <id>` (resume mode) vs `/nuova-domanda` (new) — sending the wrong prompt
|
`/riprendi-sessione <id>` (resume mode) vs `/nuova-domanda` (new) — sending the wrong prompt
|
||||||
|
|||||||
+22
-3
@@ -1,6 +1,6 @@
|
|||||||
# ThothII — Project State
|
# ThothII — Project State
|
||||||
|
|
||||||
Last updated: 2026-08-31.
|
Last updated: 2026-09-02.
|
||||||
|
|
||||||
This file is the short operational snapshot. Stable commands and the architecture mental model
|
This file is the short operational snapshot. Stable commands and the architecture mental model
|
||||||
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
|
live in `AGENTS.md`; current design and runtime contracts live under `docs/architecture/`,
|
||||||
@@ -59,10 +59,28 @@ tht --installation /absolute/path/thothii-installation.yaml workspace preprocess
|
|||||||
These commands use the profile-gated `workspace-maintenance` service. The former standalone
|
These commands use the profile-gated `workspace-maintenance` service. The former standalone
|
||||||
preprocessing Compose fixtures are retired.
|
preprocessing Compose fixtures are retired.
|
||||||
|
|
||||||
Workspace descriptors use schema v3. For PSD, workspace content and runtime roots point to the
|
Workspace descriptors use schema v4 and contain only database, Evidence, diagnostics, and binding
|
||||||
|
concerns; model, provider, embedding, and vector-store configuration is installation-owned. For
|
||||||
|
PSD, workspace content and runtime roots point to the
|
||||||
separate uncommitted repository `/Users/mp/projects/tht-workspace-psd`. Secrets remain outside
|
separate uncommitted repository `/Users/mp/projects/tht-workspace-psd`. Secrets remain outside
|
||||||
Git and are supplied only through installation-local protected files.
|
Git and are supplied only through installation-local protected files.
|
||||||
|
|
||||||
|
## Installation Model Catalog
|
||||||
|
|
||||||
|
`thothii-installation.yaml` schema version 2 is the only operator-authored source for session,
|
||||||
|
metadata-generation, and embedding models. The host `tht` lifecycle validates `modelCatalog` and
|
||||||
|
regenerates the backend catalog, Pi `models.json`/`settings.json`, and Compose override under the
|
||||||
|
installation-local `generated/` directory. Those projections are replaceable runtime adapters:
|
||||||
|
they are not edited, backed up, or treated as configuration.
|
||||||
|
|
||||||
|
Session and metadata defaults use canonical `provider/model` IDs. Provider authentication declares
|
||||||
|
one explicit mode (`secret_env`, `pi_auth`, or `none`); `secret_env` names a protected bundle key.
|
||||||
|
The backend settings store now owns only the selected workspace and thinking level. Existing v1
|
||||||
|
installations use the explicit catalog migration command; schema-v3 workspace descriptors are
|
||||||
|
converted deterministically in their curator-owned repository before commit. Strict runtime loading
|
||||||
|
does not silently infer or merge legacy sources. ADR 0013 and
|
||||||
|
`docs/plans/2026-09-02-installation-model-catalog.md` record the decision and implementation.
|
||||||
|
|
||||||
## Database management
|
## Database management
|
||||||
|
|
||||||
The database, table, and authoritative physical-schema catalog slices are implemented. Database
|
The database, table, and authoritative physical-schema catalog slices are implemented. Database
|
||||||
@@ -164,7 +182,8 @@ available only when no local start, worker, or helper is live. Runs remain inspe
|
|||||||
live SSE log with ordered polling fallback; there is no automatic resume or user-facing generation
|
live SSE log with ordered polling fallback; there is no automatic resume or user-facing generation
|
||||||
CLI. ADRs 0009–0010 record the runtime and source-sampling decisions.
|
CLI. ADRs 0009–0010 record the runtime and source-sampling decisions.
|
||||||
|
|
||||||
Metadata-generation setup accepts the protected `DEEPSEEK_API_KEY` and `ZAI_API_KEY` references.
|
The Installation Model Catalog accepts the protected `DEEPSEEK_API_KEY` and `ZAI_API_KEY`
|
||||||
|
references for metadata-generation providers.
|
||||||
It also accepts a model with no secret reference only when its OpenAI-compatible endpoint is
|
It also accepts a model with no secret reference only when its OpenAI-compatible endpoint is
|
||||||
explicit; this covers the VPN-only AritmoLab Qwen 3.6 server without creating a fake operator
|
explicit; this covers the VPN-only AritmoLab Qwen 3.6 server without creating a fake operator
|
||||||
credential. The Python client supplies only its fixed non-secret compatibility placeholder.
|
credential. The Python client supplies only its fixed non-secret compatibility placeholder.
|
||||||
|
|||||||
@@ -106,15 +106,18 @@ a remote user's partial list. The isolated deployment exercise is
|
|||||||
`./scripts/verify-workspace-install-docs.sh --profile local` or `--profile server`.
|
`./scripts/verify-workspace-install-docs.sh --profile local` or `--profile server`.
|
||||||
|
|
||||||
<!-- workspace-descriptor-contract:start -->
|
<!-- workspace-descriptor-contract:start -->
|
||||||
Schema v3 is the only accepted workspace descriptor. Schema v1 and v2 workspace descriptors are
|
Schema v4 is the only accepted workspace descriptor. Schema v1, v2, and v3 workspace descriptors
|
||||||
rejected before activation. Candidate snapshot validation therefore makes activation or a pull fail
|
are rejected before activation. Candidate snapshot validation therefore makes activation or a pull
|
||||||
atomically while the prior valid snapshot remains active. There is no in-product migrator or
|
fail atomically while the prior valid snapshot remains active. One workspace owns one Qdrant collection;
|
||||||
automatic conversion. A repository must already contain reviewed v3 descriptors. One workspace
|
|
||||||
owns one Qdrant collection;
|
|
||||||
schema, Evidence, and Memory records share that collection and stay separated by indexed payload
|
schema, Evidence, and Memory records share that collection and stay separated by indexed payload
|
||||||
`kind`.
|
`kind`.
|
||||||
<!-- workspace-descriptor-contract:end -->
|
<!-- workspace-descriptor-contract:end -->
|
||||||
|
|
||||||
|
<!-- non-workspace-migration:start -->
|
||||||
|
Convert a v3 descriptor before publication by setting `workspace.schema_version` to `4` and
|
||||||
|
removing `llm_policy` and `semantic_index`; no database or Evidence field changes.
|
||||||
|
<!-- non-workspace-migration:end -->
|
||||||
|
|
||||||
For NL→SQL runtime sessions, connector `ssh_tunnel` bindings remain diagnostic-only: their bounded
|
For NL→SQL runtime sessions, connector `ssh_tunnel` bindings remain diagnostic-only: their bounded
|
||||||
probe cleans up the loopback forward and returns `workspace_not_activatable`; session creation is
|
probe cleans up the loopback forward and returns `workspace_not_activatable`; session creation is
|
||||||
rejected before persistence. Database management is a separate boundary and supports a strict
|
rejected before persistence. Database management is a separate boundary and supports a strict
|
||||||
@@ -176,10 +179,10 @@ secret files, upstream-auth checks, and a fail-closed `503` assertion for its de
|
|||||||
unavailable disposable session endpoint. No real provider, database credential, or repository
|
unavailable disposable session endpoint. No real provider, database credential, or repository
|
||||||
secret is required.
|
secret is required.
|
||||||
|
|
||||||
For a clean server bind, `scripts/prepare-server-pi-state.sh` creates the hidden regular
|
For a clean server bind, `scripts/prepare-server-pi-state.sh` creates the hidden regular Pi agent
|
||||||
`agent/auth.json`, `agent/models.json`, and `agent/settings.json` mount targets atomically before
|
mount targets atomically before Compose. The auth target receives the protected credential bind;
|
||||||
Compose. The server smoke starts from an empty Pi-state root and applies this same preflight; the
|
the model and settings targets receive generated read-only projections. The server smoke starts
|
||||||
real protected/tracked sources remain separate read-only mounts. Deterministic fixture tests render
|
from an empty Pi-state root and applies this same preflight. Deterministic fixture tests render
|
||||||
both profiles, verify that bindings stay on `core`, check mount readability, and run the production
|
both profiles, verify that bindings stay on `core`, check mount readability, and run the production
|
||||||
workspace resolver. Wrong-service, wrong-value, and broken-secret-mount mutations must fail.
|
workspace resolver. Wrong-service, wrong-value, and broken-secret-mount mutations must fail.
|
||||||
|
|
||||||
@@ -189,7 +192,7 @@ an independent 32-minute outer timeout and does not retry a failed command.
|
|||||||
Current release status (2026-08-05): clean-root render/setup and the production runtime-binding
|
Current release status (2026-08-05): clean-root render/setup and the production runtime-binding
|
||||||
resolver contracts are green. The server fixture supplies all four private trusted claims,
|
resolver contracts are green. The server fixture supplies all four private trusted claims,
|
||||||
including exact non-admin value `0`, and a focused test proves nginx normalization produces the
|
including exact non-admin value `0`, and a focused test proves nginx normalization produces the
|
||||||
accepted non-admin backend principal. Canonical schema-v3 registry descriptors now pass through
|
accepted non-admin backend principal. Canonical schema-v4 registry descriptors now pass through
|
||||||
one backend-owned, secret-safe runtime handoff for inventory and session execution; canonical
|
one backend-owned, secret-safe runtime handoff for inventory and session execution; canonical
|
||||||
identity and durable session/artifact/index roots are retained. The fresh update-only smoke passed
|
identity and durable session/artifact/index roots are retained. The fresh update-only smoke passed
|
||||||
bad-candidate mutation, automatic `rolled_back` compensation, exact prior-image restoration,
|
bad-candidate mutation, automatic `rolled_back` compensation, exact prior-image restoration,
|
||||||
@@ -295,14 +298,12 @@ Copy `deploy/secrets/thothii.secrets.example` to a protected host file, include
|
|||||||
keys, and set its absolute path as `THT_SECRETS_FILE` in the operator env. Keep Pi's native
|
keys, and set its absolute path as `THT_SECRETS_FILE` in the operator env. Keep Pi's native
|
||||||
provider auth in the separate protected file named by `PI_AUTH_FILE`.
|
provider auth in the separate protected file named by `PI_AUTH_FILE`.
|
||||||
|
|
||||||
Description Generation is configured independently in the protected installation descriptor under
|
Interactive sessions, Description Generation, and embedding share the protected installation
|
||||||
`metadataGeneration`. Set `THT_INSTALLATION_CONFIG_SOURCE` to that exact host file; Compose mounts
|
descriptor's `modelCatalog`. Set `THT_INSTALLATION_CONFIG_SOURCE` to that exact host file; `tht`
|
||||||
it read-only into `core` and supplies the fixed runtime `THT_INSTALLATION_CONFIG_FILE` path. Each
|
validates it and generates the runtime catalog, Pi adapters, and Compose override before startup.
|
||||||
keyed model stores only an audited `apiKeyEnv` reference. The referenced value stays in the secret
|
Each authenticated provider stores only an audited `apiKeyEnv` reference; the referenced value stays
|
||||||
bundle; a model may omit `apiKeyEnv` only when it declares an explicit endpoint that accepts
|
in the secret bundle. A provider may use `authentication.mode: none` only with an explicit keyless
|
||||||
unauthenticated requests. The browser receives only model IDs, labels, and the configured default.
|
endpoint. The browser receives only eligible model IDs, labels, and the catalog default.
|
||||||
Configuration changes take effect after restart and do not use Pi settings or workspace
|
|
||||||
`llm_policy`.
|
|
||||||
|
|
||||||
Before enabling Description Generation, approve the selected model provider for bounded source-data
|
Before enabling Description Generation, approve the selected model provider for bounded source-data
|
||||||
disclosure. Every catalog column has a **Sensitive** flag that defaults to `false`. Administrators can
|
disclosure. Every catalog column has a **Sensitive** flag that defaults to `false`. Administrators can
|
||||||
@@ -333,22 +334,11 @@ the host/secret-manager materialization and add a reviewed Compose override that
|
|||||||
does not create that mount. The frontend remains on loopback; the authenticated host proxy is the
|
does not create that mount. The frontend remains on loopback; the authenticated host proxy is the
|
||||||
only public listener.
|
only public listener.
|
||||||
|
|
||||||
Set the selected model provider in application settings (or `PI_PROVIDER`). For each Pi spawn the
|
For each Pi spawn, the backend resolves the selected canonical provider/model in the runtime catalog,
|
||||||
backend validates and reads `THT_MODEL_API_KEY` from the bundle, then exposes its value only as the provider's
|
reads exactly that provider's declared `apiKeyEnv` value from the bundle, and exposes only that key
|
||||||
recognized child variable (for example `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`, or
|
to the child. Ambient provider credentials and secret-bundle paths are scrubbed. Providers needing a
|
||||||
`ZAI_API_KEY`). Neither the generic file path nor deprecated `PI_PROVIDER_API_KEY` is inherited by
|
compound credential bundle remain unsupported until the catalog gains an explicit generic contract
|
||||||
Pi. Local providers such as Ollama require no model key.
|
for them.
|
||||||
|
|
||||||
`THT_MODEL_API_KEY` supports Pi providers whose authentication is exactly one key:
|
|
||||||
`ant-ling`, `anthropic`, `cerebras`, `deepseek`, `fireworks`, `github-copilot`, `google`
|
|
||||||
(including the `gemini` alias), `google-vertex` when using its API-key mode, `groq`,
|
|
||||||
`huggingface`, `kimi-coding`, `minimax`, `minimax-cn`, `mistral`, `moonshotai`,
|
|
||||||
`moonshotai-cn`, `nvidia`, `openai`, `opencode`, `opencode-go`, `openrouter`, `together`,
|
|
||||||
`vercel-ai-gateway`, `xai`, the four `xiaomi*` providers, `zai`, and `zai-coding-cn`.
|
|
||||||
Compound providers are deliberately unsupported: `amazon-bedrock`, `azure-openai-responses`,
|
|
||||||
`cloudflare-workers-ai`, and `cloudflare-ai-gateway` require multiple credential/configuration
|
|
||||||
values. Selecting one fails before Pi starts; ambient AWS, Azure, and Cloudflare credentials are
|
|
||||||
still scrubbed. Supporting them requires a future dedicated provider-specific configuration.
|
|
||||||
|
|
||||||
## User-owned session server cutover
|
## User-owned session server cutover
|
||||||
|
|
||||||
|
|||||||
@@ -9,7 +9,8 @@
|
|||||||
"catalog:migrate": "node dist/catalog/migrate.js",
|
"catalog:migrate": "node dist/catalog/migrate.js",
|
||||||
"test": "vitest run",
|
"test": "vitest run",
|
||||||
"start": "node dist/server.js",
|
"start": "node dist/server.js",
|
||||||
"test:schema-v3-verifier": "python3 -I -B scripts/test_revision_state_policy.py && node --test scripts/verify-workspace-descriptor-files.test.mjs scripts/revision-state-policy.test.mjs"
|
"test:schema-v4-verifier": "python3 -I -B scripts/test_revision_state_policy.py && node --test scripts/verify-workspace-descriptor-files.test.mjs scripts/revision-state-policy.test.mjs",
|
||||||
|
"test:schema-v3-verifier": "npm run test:schema-v4-verifier"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@fastify/cookie": "11.1.2",
|
"@fastify/cookie": "11.1.2",
|
||||||
|
|||||||
@@ -1195,13 +1195,8 @@ export async function executeChecks({ checks, failAt, recorder } = {}) {
|
|||||||
|
|
||||||
function baseWorkspace(id, evidenceSource) {
|
function baseWorkspace(id, evidenceSource) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P1 ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P1 ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "postgres", schema: "public", supported_transports: ["postgres_direct"] },
|
dwh: { engine: "postgres", database: "postgres", schema: "public", supported_transports: ["postgres_direct"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } },
|
evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } },
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -109,7 +109,7 @@ async function validateDistFiles(repo,files){const dist=join(repo,"backend","dis
|
|||||||
|
|
||||||
export async function readManualOwnership({repositoryRoot=defaultRepositoryRoot}={}){const repo=realpathSync(repositoryRoot),root=fixedManualRoot(repo);noSymlinkExisting(repo,root);let rootEntry,ownershipEntry;try{rootEntry=await lstat(root);ownershipEntry=await lstat(join(root,"ownership.json"));}catch{throw new Error("manual ownership is missing");}if(!rootEntry.isDirectory()||rootEntry.isSymbolicLink()||await realpath(root)!==root||!ownershipEntry.isFile()||ownershipEntry.isSymbolicLink())throw new Error("manual ownership is unsafe");let value;try{value=JSON.parse(await readFile(join(root,"ownership.json"),"utf8"));}catch{throw new Error("manual ownership is malformed");}const baseValid=value.schemaVersion===1&&value.kind==="p1-manual-acceptance"&&HEX64.test(value.nonce??"")&&value.repositoryRoot===repo&&value.root===root&&value.status==="PENDING"&&["PREPARING","READY"].includes(value.stage)&&value.listener?.host===HOST&&value.listener?.port===PORT&&value.listener?.state==="stopped"&&typeof value.createdAt==="string"&&validEntrypoint(value.entrypoint,repo)&&validDistManifest(value.distManifest,root)&&JSON.stringify(value.resources)===JSON.stringify([root,{kind:"fastify",host:HOST,port:PORT}]);const readyLog=value.backendLog?.path===join(root,"logs/backend.log")&&Number.isSafeInteger(value.backendLog?.dev)&&Number.isSafeInteger(value.backendLog?.ino);if(!baseValid||(value.stage==="READY"?!readyLog:value.backendLog!==null))throw new Error("manual ownership identity mismatch");return value;}
|
export async function readManualOwnership({repositoryRoot=defaultRepositoryRoot}={}){const repo=realpathSync(repositoryRoot),root=fixedManualRoot(repo);noSymlinkExisting(repo,root);let rootEntry,ownershipEntry;try{rootEntry=await lstat(root);ownershipEntry=await lstat(join(root,"ownership.json"));}catch{throw new Error("manual ownership is missing");}if(!rootEntry.isDirectory()||rootEntry.isSymbolicLink()||await realpath(root)!==root||!ownershipEntry.isFile()||ownershipEntry.isSymbolicLink())throw new Error("manual ownership is unsafe");let value;try{value=JSON.parse(await readFile(join(root,"ownership.json"),"utf8"));}catch{throw new Error("manual ownership is malformed");}const baseValid=value.schemaVersion===1&&value.kind==="p1-manual-acceptance"&&HEX64.test(value.nonce??"")&&value.repositoryRoot===repo&&value.root===root&&value.status==="PENDING"&&["PREPARING","READY"].includes(value.stage)&&value.listener?.host===HOST&&value.listener?.port===PORT&&value.listener?.state==="stopped"&&typeof value.createdAt==="string"&&validEntrypoint(value.entrypoint,repo)&&validDistManifest(value.distManifest,root)&&JSON.stringify(value.resources)===JSON.stringify([root,{kind:"fastify",host:HOST,port:PORT}]);const readyLog=value.backendLog?.path===join(root,"logs/backend.log")&&Number.isSafeInteger(value.backendLog?.dev)&&Number.isSafeInteger(value.backendLog?.ino);if(!baseValid||(value.stage==="READY"?!readyLog:value.backendLog!==null))throw new Error("manual ownership identity mismatch");return value;}
|
||||||
async function run(executable,argv,options={}){return await exec(executable,argv,{...options,maxBuffer:2*1024*1024,encoding:"utf8"});}
|
async function run(executable,argv,options={}){return await exec(executable,argv,{...options,maxBuffer:2*1024*1024,encoding:"utf8"});}
|
||||||
function descriptor(id,source){return{workspace:{schema_version:3,id,name:`P1 ${id}`,language:"en"},dwh:{engine:"postgres",database:"postgres",schema:"public",supported_transports:["postgres_direct"]},semantic_index:{vector_store:{engine:"qdrant",collection:id,dimensions:1024,distance:"cosine"},embedding:{provider:"ollama_internal",model:"qwen3-embedding:0.6b",dimensions:1024}},llm_policy:{allowed:["zai/glm-5.2"]},evidence:{source,policy:{max_chunk_chars:4000,retain_published_generations:3}}};}
|
function descriptor(id,source){return{workspace:{schema_version:4,id,name:`P1 ${id}`,language:"en"},dwh:{engine:"postgres",database:"postgres",schema:"public",supported_transports:["postgres_direct"]},evidence:{source,policy:{max_chunk_chars:4000,retain_published_generations:3}}};}
|
||||||
function descriptors(){return[descriptor("p1-filesystem",{type:"filesystem",uri:"workspace-content/p1-filesystem/evidence",patterns:["**/*.md"],max_bytes:10485760}),descriptor("p1-http",{type:"http",uris:["https://evidence.example.test/guide.md"],authentication:"signed_urls_file",connect_timeout_ms:1250,read_timeout_ms:30001,max_bytes:12345,max_redirects:2,allow_private_hosts:false,max_cache_bytes:67890}),descriptor("p1-s3",{type:"s3",uri:"s3://p1-evidence/published/",endpoint_url:"https://s3.example.test/",region:"eu-west-1",credentials:"static_files",trusted_endpoint:true,allow_private_endpoint:false,allow_insecure_endpoint:false,max_bytes:12345,max_objects:33,max_pages:4,page_size:5})];}
|
function descriptors(){return[descriptor("p1-filesystem",{type:"filesystem",uri:"workspace-content/p1-filesystem/evidence",patterns:["**/*.md"],max_bytes:10485760}),descriptor("p1-http",{type:"http",uris:["https://evidence.example.test/guide.md"],authentication:"signed_urls_file",connect_timeout_ms:1250,read_timeout_ms:30001,max_bytes:12345,max_redirects:2,allow_private_hosts:false,max_cache_bytes:67890}),descriptor("p1-s3",{type:"s3",uri:"s3://p1-evidence/published/",endpoint_url:"https://s3.example.test/",region:"eu-west-1",credentials:"static_files",trusted_endpoint:true,allow_private_endpoint:false,allow_insecure_endpoint:false,max_bytes:12345,max_objects:33,max_pages:4,page_size:5})];}
|
||||||
function quote(value){return `'${String(value).replaceAll("'",`'"'"'`)}'`;}
|
function quote(value){return `'${String(value).replaceAll("'",`'"'"'`)}'`;}
|
||||||
async function checkPrerequisites(repo){for(const path of ["scripts/p1-acceptance.sh","scripts/test-p1-acceptance.sh","backend/scripts/p1-acceptance.mjs","backend/dist/server.js"]){try{await access(join(repo,path));}catch{throw new Error(`Task 8 prerequisite is missing: ${path}`);}}for(const command of ["node","npm","git","curl","unzip","zipinfo","lsof","python3"]){try{await run(command,[command==="unzip"||command==="lsof"?"-v":command==="zipinfo"?"-h":"--version"]);}catch{throw new Error(`missing prerequisite: ${command}`);}}const tht=join(repo,"harness",".venv","bin","tht");try{await access(tht,constants.X_OK);}catch{throw new Error("missing prerequisite: harness/.venv/bin/tht");}}
|
async function checkPrerequisites(repo){for(const path of ["scripts/p1-acceptance.sh","scripts/test-p1-acceptance.sh","backend/scripts/p1-acceptance.mjs","backend/dist/server.js"]){try{await access(join(repo,path));}catch{throw new Error(`Task 8 prerequisite is missing: ${path}`);}}for(const command of ["node","npm","git","curl","unzip","zipinfo","lsof","python3"]){try{await run(command,[command==="unzip"||command==="lsof"?"-v":command==="zipinfo"?"-h":"--version"]);}catch{throw new Error(`missing prerequisite: ${command}`);}}const tht=join(repo,"harness",".venv","bin","tht");try{await access(tht,constants.X_OK);}catch{throw new Error("missing prerequisite: harness/.venv/bin/tht");}}
|
||||||
|
|||||||
@@ -403,7 +403,7 @@ test("generated render command validates saved responses and owned snapshot befo
|
|||||||
});
|
});
|
||||||
|
|
||||||
const renderSnapshotYaml=`workspace:
|
const renderSnapshotYaml=`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: p1-filesystem
|
id: p1-filesystem
|
||||||
name: P1 filesystem
|
name: P1 filesystem
|
||||||
language: en
|
language: en
|
||||||
@@ -412,11 +412,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: public
|
schema: public
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store: {engine: qdrant, collection: p1-filesystem, dimensions: 1024, distance: cosine}
|
|
||||||
embedding: {provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024}
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
evidence:
|
evidence:
|
||||||
source: {type: filesystem, uri: workspace-content/p1-filesystem/evidence, patterns: ["**/*.md"], max_bytes: 10485760}
|
source: {type: filesystem, uri: workspace-content/p1-filesystem/evidence, patterns: ["**/*.md"], max_bytes: 10485760}
|
||||||
policy: {max_chunk_chars: 4000, retain_published_generations: 3}
|
policy: {max_chunk_chars: 4000, retain_published_generations: 3}
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ async function fixture() {
|
|||||||
await writeFile(join(root,"installation/base.yaml"),"{}\n");
|
await writeFile(join(root,"installation/base.yaml"),"{}\n");
|
||||||
const secret=join(root,"fixture-secrets/dwh-password"); await writeFile(secret,"not-inspected",{mode:0o600});
|
const secret=join(root,"fixture-secrets/dwh-password"); await writeFile(secret,"not-inspected",{mode:0o600});
|
||||||
await writeFile(snapshot,`workspace:
|
await writeFile(snapshot,`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: p1-filesystem
|
id: p1-filesystem
|
||||||
name: P1 filesystem
|
name: P1 filesystem
|
||||||
language: en
|
language: en
|
||||||
@@ -25,11 +25,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: public
|
schema: public
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store: {engine: qdrant, collection: p1-filesystem, dimensions: 1024, distance: cosine}
|
|
||||||
embedding: {provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024}
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
evidence:
|
evidence:
|
||||||
source: {type: filesystem, uri: workspace-content/p1-filesystem/evidence, patterns: ["**/*.md"], max_bytes: 10485760}
|
source: {type: filesystem, uri: workspace-content/p1-filesystem/evidence, patterns: ["**/*.md"], max_bytes: 10485760}
|
||||||
policy: {max_chunk_chars: 4000, retain_published_generations: 3}
|
policy: {max_chunk_chars: 4000, retain_published_generations: 3}
|
||||||
@@ -61,7 +56,7 @@ test("renderer refuses snapshot manifest head, digest, and expected-digest tampe
|
|||||||
|
|
||||||
test("renderer refuses a missing or malformed snapshot manifest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/nomanifest.yaml"); await rm(f.manifestPath); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest.*(missing|unbounded|unsafe)/); await writeFile(f.manifestPath,"{not json"); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest.*malformed/); await assert.rejects(lstat(output)); assert.deepEqual(await runtimeLeases(f),[]); });
|
test("renderer refuses a missing or malformed snapshot manifest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/nomanifest.yaml"); await rm(f.manifestPath); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest.*(missing|unbounded|unsafe)/); await writeFile(f.manifestPath,"{not json"); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest.*malformed/); await assert.rejects(lstat(output)); assert.deepEqual(await runtimeLeases(f),[]); });
|
||||||
|
|
||||||
test("renderer rejects a regular snapshot replacement against its manifest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/replaced.yaml"); await assert.rejects(call(f,{outputPath:output,beforePublish:async()=>{await writeFile(f.snapshot,"workspace:\n schema_version: 3\n id: p1-filesystem\n name: replaced\n")}}),/snapshot content changed/); await assert.rejects(lstat(output)); });
|
test("renderer rejects a regular snapshot replacement against its manifest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/replaced.yaml"); await assert.rejects(call(f,{outputPath:output,beforePublish:async()=>{await writeFile(f.snapshot,"workspace:\n schema_version: 4\n id: p1-filesystem\n name: replaced\n")}}),/snapshot content changed/); await assert.rejects(lstat(output)); });
|
||||||
|
|
||||||
test("renderer anchors publication when rendered parent is concurrently swapped", async()=>{
|
test("renderer anchors publication when rendered parent is concurrently swapped", async()=>{
|
||||||
const f=await fixture(),output=join(f.root,"rendered/raced.yaml"),moved=join(f.root,"rendered-moved"),outside=join(f.repo,"outside-rendered"); await mkdir(outside);
|
const f=await fixture(),output=join(f.root,"rendered/raced.yaml"),moved=join(f.root,"rendered-moved"),outside=join(f.repo,"outside-rendered"); await mkdir(outside);
|
||||||
|
|||||||
@@ -328,13 +328,8 @@ async function tht(ctx, argv, options = {}) {
|
|||||||
function namespace(id) { return id.toUpperCase().replaceAll("-", "_"); }
|
function namespace(id) { return id.toUpperCase().replaceAll("-", "_"); }
|
||||||
function baseWorkspace(id, evidenceSource) {
|
function baseWorkspace(id, evidenceSource) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P1.1 ${id}`, description: `Catalog entry for ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P1.1 ${id}`, description: `Catalog entry for ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "postgres", schema: "public", supported_transports: ["postgres_direct"] },
|
dwh: { engine: "postgres", database: "postgres", schema: "public", supported_transports: ["postgres_direct"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } },
|
evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } },
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -81,13 +81,8 @@ async function git(executable, argv, options = {}) {
|
|||||||
function namespace(id) { return id.toUpperCase().replaceAll("-", "_"); }
|
function namespace(id) { return id.toUpperCase().replaceAll("-", "_"); }
|
||||||
function baseWorkspace(id, evidenceSource) {
|
function baseWorkspace(id, evidenceSource) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P1.1 ${id}`, description: `Catalog entry for ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P1.1 ${id}`, description: `Catalog entry for ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "postgres", schema: "public", supported_transports: ["postgres_direct"] },
|
dwh: { engine: "postgres", database: "postgres", schema: "public", supported_transports: ["postgres_direct"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } },
|
evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } },
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -375,16 +375,11 @@ function installationProjectName(installationPath) {
|
|||||||
|
|
||||||
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P2 ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P2 ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
diagnostics: {
|
diagnostics: {
|
||||||
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
||||||
},
|
},
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -376,16 +376,11 @@ function installationProjectName(installationPath) {
|
|||||||
|
|
||||||
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P2 ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P2 ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
diagnostics: {
|
diagnostics: {
|
||||||
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
||||||
},
|
},
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -379,16 +379,11 @@ function installationProjectName(installationPath) {
|
|||||||
|
|
||||||
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P2 ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P2 ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
diagnostics: {
|
diagnostics: {
|
||||||
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
||||||
},
|
},
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -374,16 +374,11 @@ function installationProjectName(installationPath) {
|
|||||||
|
|
||||||
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P2 ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P2 ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
diagnostics: {
|
diagnostics: {
|
||||||
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
||||||
},
|
},
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -374,16 +374,11 @@ function installationProjectName(installationPath) {
|
|||||||
|
|
||||||
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
function baseWorkspace(id, { dwhBaseUrl, evidenceSource }) {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: `P2 ${id}`, language: "en" },
|
workspace: { schema_version: 4, id, name: `P2 ${id}`, language: "en" },
|
||||||
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
dwh: { engine: "postgres", database: "warehouse", schema: "dw", supported_transports: ["rest_api"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
diagnostics: {
|
diagnostics: {
|
||||||
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
dwh_rest: { method: "POST", path: "/rpc/ping", auth: "x-api-key", response: { database: "database", schema: "schema" } },
|
||||||
},
|
},
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
...(evidenceSource ? { evidence: { source: evidenceSource, policy: { max_chunk_chars: 4000, retain_published_generations: 3 } } } : {}),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -36,11 +36,10 @@ const reviewedExpandableBlocks = new Map([
|
|||||||
{ sha256: "b903e5dae953ae1372f1a5276f12a92ed3dd632b897f3afe5e00c646d90a1b42", rationale: "Same reviewed block in the repository-required CRLF checkout representation." },
|
{ sha256: "b903e5dae953ae1372f1a5276f12a92ed3dd632b897f3afe5e00c646d90a1b42", rationale: "Same reviewed block in the repository-required CRLF checkout representation." },
|
||||||
]],
|
]],
|
||||||
["scripts/unified-deployment-smoke.sh", [
|
["scripts/unified-deployment-smoke.sh", [
|
||||||
{ sha256: "1d60bf140165a8fabfa0c3729e776136904717e67becf3e0ab68c70d8e37847e", rationale: "Generates reviewed Task 13 runtime configuration." },
|
{ sha256: "b6c0826151b2c8b955399d1abf5b691cc8fe6b6454b17da000dde7ba3bc55d2d", rationale: "Generates the reviewed local Task 13 Compose override with normalized catalog mounts." },
|
||||||
{ sha256: "36d3d8a2362dbdc4fad90948d6c227586d749f56b9a4bc5b6b5a91bcbec6407b", rationale: "Generates the reviewed local Task 13 Compose override." },
|
{ sha256: "24f69d12b8554aa2bebba455be99fde3e60743eef5a40fa2ef5b29397a477c03", rationale: "Generates the reviewed local Task 13 installation descriptor with its model catalog." },
|
||||||
{ sha256: "c556f7d910d0788e219b042957e6b307cb9925b43920c680535d0d3a6dcbdb25", rationale: "Generates the reviewed local Task 13 installation descriptor." },
|
|
||||||
{ sha256: "526006fa6d48a8080b3834723630c64de5005a67243e944ebf1da15212b4d654", rationale: "Generates the reviewed server Task 13 Compose override." },
|
{ sha256: "526006fa6d48a8080b3834723630c64de5005a67243e944ebf1da15212b4d654", rationale: "Generates the reviewed server Task 13 Compose override." },
|
||||||
{ sha256: "c57ae2205c21ead0c2015a353aaabb948fa4ddd9b78a2cdcdb71f48cf2db742d", rationale: "Generates the reviewed projected-auth server Task 13 installation descriptor." },
|
{ sha256: "406ccead1967f642225c946fc4a23fe5b019c9764cc5153e1125876ade16ec90", rationale: "Generates the reviewed projected-auth server Task 13 installation descriptor with its model catalog." },
|
||||||
]],
|
]],
|
||||||
["scripts/vector-backup.sh", [
|
["scripts/vector-backup.sh", [
|
||||||
{ sha256: "571899db49dfdcec8107fbe1e0a86a61e7581979d3c4c248c20546843e275bcf", rationale: "Generates the reviewed backup manifest inside the helper command." },
|
{ sha256: "571899db49dfdcec8107fbe1e0a86a61e7581979d3c4c248c20546843e275bcf", rationale: "Generates the reviewed backup manifest inside the helper command." },
|
||||||
@@ -103,6 +102,7 @@ function isPolicyImplementationException(label, category) {
|
|||||||
]);
|
]);
|
||||||
if (implementations.has(label)) return true;
|
if (implementations.has(label)) return true;
|
||||||
if (category === "migration-marker" && new Set([
|
if (category === "migration-marker" && new Set([
|
||||||
|
"backend/src/workspaces/schema.ts",
|
||||||
"scripts/workspace_descriptor_doc_contract.py",
|
"scripts/workspace_descriptor_doc_contract.py",
|
||||||
"scripts/test_workspace_descriptor_doc_contract.py",
|
"scripts/test_workspace_descriptor_doc_contract.py",
|
||||||
"backend/scripts/clean-dist.test.mjs",
|
"backend/scripts/clean-dist.test.mjs",
|
||||||
@@ -173,7 +173,7 @@ function validateWorkspaceSource(source, label, { requireWorkspace, expandable =
|
|||||||
try {
|
try {
|
||||||
parseWorkspaceYaml(source);
|
parseWorkspaceYaml(source);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
throw new Error(`${label}: workspace descriptor is not valid schema v3: ${error instanceof Error ? error.message : String(error)}`);
|
throw new Error(`${label}: workspace descriptor is not valid schema v4: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
}
|
}
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -33,20 +33,20 @@ function bashN(root, path) {
|
|||||||
function replaceWorkspaceKeys(source, workspaceKey, schemaLine) {
|
function replaceWorkspaceKeys(source, workspaceKey, schemaLine) {
|
||||||
return source
|
return source
|
||||||
.replace(/^workspace:$/m, workspaceKey)
|
.replace(/^workspace:$/m, workspaceKey)
|
||||||
.replace(/^ schema_version: 3$/m, schemaLine);
|
.replace(/^ schema_version: 4$/m, schemaLine);
|
||||||
}
|
}
|
||||||
|
|
||||||
test("production parser accepts semantic v3 with quoted Unicode/tagged keys and spacing", async (t) => {
|
test("production parser accepts semantic v4 with quoted Unicode/tagged keys and spacing", async (t) => {
|
||||||
const root = await fixture(t);
|
const root = await fixture(t);
|
||||||
const unicode = replaceWorkspaceKeys(
|
const unicode = replaceWorkspaceKeys(
|
||||||
canonicalDescriptor,
|
canonicalDescriptor,
|
||||||
'"\\u0077orkspace" :',
|
'"\\u0077orkspace" :',
|
||||||
' "\\u0073chema_version" : 3',
|
' "\\u0073chema_version" : 4',
|
||||||
);
|
);
|
||||||
const tagged = replaceWorkspaceKeys(
|
const tagged = replaceWorkspaceKeys(
|
||||||
canonicalDescriptor,
|
canonicalDescriptor,
|
||||||
"!!str workspace :",
|
"!!str workspace :",
|
||||||
" !!str schema_version : 3",
|
" !!str schema_version : 4",
|
||||||
);
|
);
|
||||||
await put(root, "deploy/workspaces/unicode.yaml", unicode);
|
await put(root, "deploy/workspaces/unicode.yaml", unicode);
|
||||||
await put(root, "deploy/workspaces/tagged.yaml", tagged);
|
await put(root, "deploy/workspaces/tagged.yaml", tagged);
|
||||||
@@ -59,14 +59,15 @@ test("production parser accepts semantic v3 with quoted Unicode/tagged keys and
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("production parser rejects fancy keys with every non-v3 or ambiguous value", async (t) => {
|
test("production parser rejects fancy keys with every non-v4 or ambiguous value", async (t) => {
|
||||||
const invalid = [
|
const invalid = [
|
||||||
["unicode-v2", '"\\u0077orkspace" :', ' "\\u0073chema_version" : 2'],
|
["unicode-v2", '"\\u0077orkspace" :', ' "\\u0073chema_version" : 2'],
|
||||||
["tagged-leading-zero", "!!str workspace :", " !!str schema_version : 02"],
|
["unicode-v3", '"\\u0077orkspace" :', ' "\\u0073chema_version" : 3'],
|
||||||
["hexadecimal", "workspace :", " schema_version : 0x2"],
|
["tagged-leading-zero", "!!str workspace :", " !!str schema_version : 03"],
|
||||||
["multiline", "workspace :", " schema_version : >\n 3"],
|
["hexadecimal", "workspace :", " schema_version : 0x3"],
|
||||||
["duplicate", "workspace :", " schema_version : 3\n schema_version: 3"],
|
["multiline", "workspace :", " schema_version : >\n 4"],
|
||||||
["inline", "workspace: { schema_version: 3 }", " schema_version: 3"],
|
["duplicate", "workspace :", " schema_version : 4\n schema_version: 4"],
|
||||||
|
["inline", "workspace: { schema_version: 4 }", " schema_version: 4"],
|
||||||
];
|
];
|
||||||
for (const [name, workspaceKey, schemaLine] of invalid) {
|
for (const [name, workspaceKey, schemaLine] of invalid) {
|
||||||
await t.test(name, async () => {
|
await t.test(name, async () => {
|
||||||
@@ -120,7 +121,7 @@ test("PowerShell embedded workspace mappings are rejected while bundle-only stri
|
|||||||
const root = await fixture(t);
|
const root = await fixture(t);
|
||||||
const source = [
|
const source = [
|
||||||
"$workspace = @'",
|
"$workspace = @'",
|
||||||
canonicalDescriptor.replace(" schema_version: 3", " schema_version: 0x2").trimEnd(),
|
canonicalDescriptor.replace(" schema_version: 4", " schema_version: 0x2").trimEnd(),
|
||||||
"'@",
|
"'@",
|
||||||
'$bundle = @"',
|
'$bundle = @"',
|
||||||
"bundle:",
|
"bundle:",
|
||||||
@@ -137,7 +138,7 @@ test("PowerShell embedded workspace mappings are rejected while bundle-only stri
|
|||||||
|
|
||||||
test("workspace descriptor family entries require a top-level workspace", async (t) => {
|
test("workspace descriptor family entries require a top-level workspace", async (t) => {
|
||||||
const root = await fixture(t);
|
const root = await fixture(t);
|
||||||
await put(root, "scripts/fixtures/workspace-registry-future.yaml", "bundle:\n schema_version: 3\n");
|
await put(root, "scripts/fixtures/workspace-registry-future.yaml", "bundle:\n schema_version: 4\n");
|
||||||
await assert.rejects(
|
await assert.rejects(
|
||||||
verifyEntries({
|
verifyEntries({
|
||||||
root,
|
root,
|
||||||
@@ -179,7 +180,7 @@ test("script scalar workspace remains a bundle even with descriptor-like sibling
|
|||||||
test("standalone descriptor files require workspace to be a mapping", async (t) => {
|
test("standalone descriptor files require workspace to be a mapping", async (t) => {
|
||||||
const root = await fixture(t);
|
const root = await fixture(t);
|
||||||
const path = "scripts/fixtures/workspace-registry-scalar.yaml";
|
const path = "scripts/fixtures/workspace-registry-scalar.yaml";
|
||||||
await put(root, path, "workspace: analytics\nschema_version: 3\n");
|
await put(root, path, "workspace: analytics\nschema_version: 4\n");
|
||||||
await assert.rejects(
|
await assert.rejects(
|
||||||
verifyEntries({ root, entries: [entry("workspace_descriptor", path)] }),
|
verifyEntries({ root, entries: [entry("workspace_descriptor", path)] }),
|
||||||
/workspace.*mapping/i,
|
/workspace.*mapping/i,
|
||||||
@@ -193,11 +194,11 @@ test("Bash extractor supports hyphen, digit, escaped delimiters, and tab strippi
|
|||||||
name: "hyphen-v2",
|
name: "hyphen-v2",
|
||||||
opener: "cat <<'WORKSPACE-YAML'",
|
opener: "cat <<'WORKSPACE-YAML'",
|
||||||
delimiter: "WORKSPACE-YAML",
|
delimiter: "WORKSPACE-YAML",
|
||||||
descriptor: canonicalDescriptor.replace(" schema_version: 3", " schema_version: 2"),
|
descriptor: canonicalDescriptor.replace(" schema_version: 4", " schema_version: 2"),
|
||||||
rejected: true,
|
rejected: true,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "digit-v3",
|
name: "digit-v4",
|
||||||
opener: "cat <<2YAML",
|
opener: "cat <<2YAML",
|
||||||
delimiter: "2YAML",
|
delimiter: "2YAML",
|
||||||
descriptor: canonicalDescriptor,
|
descriptor: canonicalDescriptor,
|
||||||
@@ -207,11 +208,11 @@ test("Bash extractor supports hyphen, digit, escaped delimiters, and tab strippi
|
|||||||
name: "escaped-v2",
|
name: "escaped-v2",
|
||||||
opener: "cat <<WORKSPACE\\-YAML",
|
opener: "cat <<WORKSPACE\\-YAML",
|
||||||
delimiter: "WORKSPACE-YAML",
|
delimiter: "WORKSPACE-YAML",
|
||||||
descriptor: canonicalDescriptor.replace(" schema_version: 3", " schema_version: 2"),
|
descriptor: canonicalDescriptor.replace(" schema_version: 4", " schema_version: 2"),
|
||||||
rejected: true,
|
rejected: true,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "tab-strip-v3",
|
name: "tab-strip-v4",
|
||||||
opener: "cat <<-'TAB-YAML'",
|
opener: "cat <<-'TAB-YAML'",
|
||||||
delimiter: "\tTAB-YAML",
|
delimiter: "\tTAB-YAML",
|
||||||
descriptor: canonicalDescriptor.split("\n").map((line) => `\t${line}`).join("\n"),
|
descriptor: canonicalDescriptor.split("\n").map((line) => `\t${line}`).join("\n"),
|
||||||
@@ -272,7 +273,7 @@ test("non-stripping heredoc close requires an exact physical delimiter line", as
|
|||||||
"#!/usr/bin/env bash",
|
"#!/usr/bin/env bash",
|
||||||
"cat <<'---'",
|
"cat <<'---'",
|
||||||
"--- ",
|
"--- ",
|
||||||
canonicalDescriptor.replace(" schema_version: 3", " schema_version: 2").trimEnd(),
|
canonicalDescriptor.replace(" schema_version: 4", " schema_version: 2").trimEnd(),
|
||||||
"---",
|
"---",
|
||||||
"",
|
"",
|
||||||
].join("\n");
|
].join("\n");
|
||||||
@@ -313,7 +314,7 @@ test("double-quoted non-special backslash is preserved in the delimiter", async
|
|||||||
"#!/usr/bin/env bash",
|
"#!/usr/bin/env bash",
|
||||||
'cat <<"\\---"',
|
'cat <<"\\---"',
|
||||||
"---",
|
"---",
|
||||||
canonicalDescriptor.replace(" schema_version: 3", " schema_version: 2").trimEnd(),
|
canonicalDescriptor.replace(" schema_version: 4", " schema_version: 2").trimEnd(),
|
||||||
"\\---",
|
"\\---",
|
||||||
"",
|
"",
|
||||||
].join("\n");
|
].join("\n");
|
||||||
@@ -355,7 +356,7 @@ test("split heredoc operator continuation cannot bypass v2 validation", async (t
|
|||||||
"#!/usr/bin/env bash",
|
"#!/usr/bin/env bash",
|
||||||
"cat <\\",
|
"cat <\\",
|
||||||
"<'YAML'",
|
"<'YAML'",
|
||||||
canonicalDescriptor.replace(" schema_version: 3", " schema_version: 2").trimEnd(),
|
canonicalDescriptor.replace(" schema_version: 4", " schema_version: 2").trimEnd(),
|
||||||
"YAML",
|
"YAML",
|
||||||
"",
|
"",
|
||||||
].join("\n");
|
].join("\n");
|
||||||
@@ -424,7 +425,7 @@ test("PowerShell comment backslash cannot hide a following v2 here-string", asyn
|
|||||||
const source = [
|
const source = [
|
||||||
"# harmless PowerShell comment \\",
|
"# harmless PowerShell comment \\",
|
||||||
"$workspace = @'",
|
"$workspace = @'",
|
||||||
canonicalDescriptor.replace(" schema_version: 3", " schema_version: 2").trimEnd(),
|
canonicalDescriptor.replace(" schema_version: 4", " schema_version: 2").trimEnd(),
|
||||||
"'@",
|
"'@",
|
||||||
"",
|
"",
|
||||||
].join("\n");
|
].join("\n");
|
||||||
@@ -435,7 +436,7 @@ test("PowerShell comment backslash cannot hide a following v2 here-string", asyn
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("PowerShell dialect accepts normal v3 and non-workspace bundle here-strings", async (t) => {
|
test("PowerShell dialect accepts normal v4 and non-workspace bundle here-strings", async (t) => {
|
||||||
const root = await fixture(t);
|
const root = await fixture(t);
|
||||||
const path = "scripts/powershell-valid-smoke.ps1";
|
const path = "scripts/powershell-valid-smoke.ps1";
|
||||||
const source = [
|
const source = [
|
||||||
@@ -494,10 +495,10 @@ test("PowerShell cast and concatenation openers cannot hide embedded descriptors
|
|||||||
test("expandable YAML interpolation that can hide a workspace descriptor fails closed", async (t) => {
|
test("expandable YAML interpolation that can hide a workspace descriptor fails closed", async (t) => {
|
||||||
const root = await fixture(t);
|
const root = await fixture(t);
|
||||||
const cases = [
|
const cases = [
|
||||||
["braced-key", "${key}:\n schema_version: 3"],
|
["braced-key", "${key}:\n schema_version: 4"],
|
||||||
["plain-key", "$key:\n schema_version: 3"],
|
["plain-key", "$key:\n schema_version: 4"],
|
||||||
["quoted-key", '"$key" :\n schema_version: 3'],
|
["quoted-key", '"$key" :\n schema_version: 4'],
|
||||||
["subexpression-key", "$($key):\n schema_version: 3"],
|
["subexpression-key", "$($key):\n schema_version: 4"],
|
||||||
["version", "workspace:\n schema_version: $version"],
|
["version", "workspace:\n schema_version: $version"],
|
||||||
];
|
];
|
||||||
for (const [name, body] of cases) {
|
for (const [name, body] of cases) {
|
||||||
@@ -564,7 +565,7 @@ test("unmarked expandable Bash YAML cannot generate descriptor keys or values at
|
|||||||
"key=workspace",
|
"key=workspace",
|
||||||
"cat <<YAML",
|
"cat <<YAML",
|
||||||
generatedKey,
|
generatedKey,
|
||||||
" schema_version: 3",
|
" schema_version: 4",
|
||||||
"YAML",
|
"YAML",
|
||||||
"",
|
"",
|
||||||
].join("\n");
|
].join("\n");
|
||||||
@@ -600,14 +601,14 @@ test("an in-band marker cannot authorize expandable content", async (t) => {
|
|||||||
for (const [path, source] of [
|
for (const [path, source] of [
|
||||||
["scripts/fake-marker.sh", [
|
["scripts/fake-marker.sh", [
|
||||||
"#!/usr/bin/env bash",
|
"#!/usr/bin/env bash",
|
||||||
"# schema-v3-only: expandable-nonworkspace",
|
"# schema-v4-only: expandable-nonworkspace",
|
||||||
"cat <<YAML",
|
"cat <<YAML",
|
||||||
"${DESCRIPTOR}",
|
"${DESCRIPTOR}",
|
||||||
"YAML",
|
"YAML",
|
||||||
"",
|
"",
|
||||||
].join("\n")],
|
].join("\n")],
|
||||||
["scripts/fake-marker.ps1", [
|
["scripts/fake-marker.ps1", [
|
||||||
"# schema-v3-only: expandable-nonworkspace",
|
"# schema-v4-only: expandable-nonworkspace",
|
||||||
'$yaml = @"',
|
'$yaml = @"',
|
||||||
"$descriptor",
|
"$descriptor",
|
||||||
'"@',
|
'"@',
|
||||||
|
|||||||
+19
-7
@@ -22,7 +22,8 @@ import { isUsableAuthenticationSecret } from "./auth/secret-policy.js";
|
|||||||
import { secretValue } from "./config/secret-bundle.js";
|
import { secretValue } from "./config/secret-bundle.js";
|
||||||
import { sessionRoutes } from "./routes/sessions.js";
|
import { sessionRoutes } from "./routes/sessions.js";
|
||||||
import { sqlRoutes } from "./routes/sql.js";
|
import { sqlRoutes } from "./routes/sql.js";
|
||||||
import { metaRoutes, type ListModelsFn } from "./routes/meta.js";
|
import { metaRoutes } from "./routes/meta.js";
|
||||||
|
import type { ListModelsFn } from "./pi/list-models.js";
|
||||||
import { settingsRoutes, effectiveSettings } from "./routes/settings.js";
|
import { settingsRoutes, effectiveSettings } from "./routes/settings.js";
|
||||||
import { createPiModelLister } from "./pi/list-models.js";
|
import { createPiModelLister } from "./pi/list-models.js";
|
||||||
import { createPiManagement, type PiManagementService } from "./pi/management.js";
|
import { createPiManagement, type PiManagementService } from "./pi/management.js";
|
||||||
@@ -66,6 +67,7 @@ import { catalogDescriptionGenerationRoutes } from "./routes/catalog-description
|
|||||||
import { CatalogLogicalRelationshipService } from "./catalog/logical-relationship-service.js";
|
import { CatalogLogicalRelationshipService } from "./catalog/logical-relationship-service.js";
|
||||||
import { catalogLogicalRelationshipRoutes } from "./routes/catalog-logical-relationships.js";
|
import { catalogLogicalRelationshipRoutes } from "./routes/catalog-logical-relationships.js";
|
||||||
import { EffectiveRelationshipSnapshotProvider } from "./catalog/effective-relationship-snapshot.js";
|
import { EffectiveRelationshipSnapshotProvider } from "./catalog/effective-relationship-snapshot.js";
|
||||||
|
import { loadRuntimeModelCatalog, type RuntimeModelCatalog } from "./models/runtime-model-catalog.js";
|
||||||
|
|
||||||
export interface BuildAppDeps {
|
export interface BuildAppDeps {
|
||||||
thtRunner?: ThtRunner;
|
thtRunner?: ThtRunner;
|
||||||
@@ -88,6 +90,7 @@ export interface BuildAppDeps {
|
|||||||
catalogSyncWorker?: CatalogSyncWorker;
|
catalogSyncWorker?: CatalogSyncWorker;
|
||||||
catalogOperationCoordinator?: CatalogOperationCoordinator;
|
catalogOperationCoordinator?: CatalogOperationCoordinator;
|
||||||
metadataGenerationModels?: MetadataGenerationModels;
|
metadataGenerationModels?: MetadataGenerationModels;
|
||||||
|
runtimeModelCatalog?: RuntimeModelCatalog;
|
||||||
modelCompleter?: ModelCompleter;
|
modelCompleter?: ModelCompleter;
|
||||||
descriptionSourceSampler?: DescriptionSourceSampler;
|
descriptionSourceSampler?: DescriptionSourceSampler;
|
||||||
workspaceRuntimeSupport?: (workspace: WorkspaceDescriptor) => boolean;
|
workspaceRuntimeSupport?: (workspace: WorkspaceDescriptor) => boolean;
|
||||||
@@ -155,17 +158,22 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
|||||||
semanticRuntime: {
|
semanticRuntime: {
|
||||||
internalQdrantUrl: config.internalQdrantUrl,
|
internalQdrantUrl: config.internalQdrantUrl,
|
||||||
internalEmbeddingUrl: config.internalEmbeddingUrl,
|
internalEmbeddingUrl: config.internalEmbeddingUrl,
|
||||||
|
internalEmbeddingId: config.internalEmbeddingId,
|
||||||
internalEmbeddingModel: config.internalEmbeddingModel,
|
internalEmbeddingModel: config.internalEmbeddingModel,
|
||||||
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
|
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
const mgr = deps?.mgr ?? new PiProcessManager(config, deps?.spawnFn ? { spawnFn: deps.spawnFn } : undefined);
|
|
||||||
const hub = deps?.hub ?? new SseHub();
|
const hub = deps?.hub ?? new SseHub();
|
||||||
const workspaceRegistry = deps?.workspaceRegistry ?? new WorkspaceRegistry(config.workspaceRegistry);
|
const workspaceRegistry = deps?.workspaceRegistry ?? new WorkspaceRegistry(config.workspaceRegistry);
|
||||||
const catalogRepository = deps?.catalogRepository ?? createCatalogRepository(config.catalogDatabase);
|
const catalogRepository = deps?.catalogRepository ?? createCatalogRepository(config.catalogDatabase);
|
||||||
const catalogOperationCoordinator = deps?.catalogOperationCoordinator ?? new CatalogOperationCoordinator();
|
const catalogOperationCoordinator = deps?.catalogOperationCoordinator ?? new CatalogOperationCoordinator();
|
||||||
|
const runtimeModelCatalog = deps?.runtimeModelCatalog ?? loadRuntimeModelCatalog(config.modelCatalogFile);
|
||||||
|
const mgr = deps?.mgr ?? new PiProcessManager(config, {
|
||||||
|
...(deps?.spawnFn ? { spawnFn: deps.spawnFn } : {}),
|
||||||
|
modelCatalog: runtimeModelCatalog,
|
||||||
|
});
|
||||||
const metadataGenerationModels = deps?.metadataGenerationModels ?? loadMetadataGenerationModels({
|
const metadataGenerationModels = deps?.metadataGenerationModels ?? loadMetadataGenerationModels({
|
||||||
installationFile: config.installationConfigFile,
|
catalogFile: config.modelCatalogFile,
|
||||||
secretsFile: config.secretsFile,
|
secretsFile: config.secretsFile,
|
||||||
});
|
});
|
||||||
const modelCompleter = deps?.modelCompleter ?? new PythonModelCompleter({
|
const modelCompleter = deps?.modelCompleter ?? new PythonModelCompleter({
|
||||||
@@ -237,6 +245,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
|||||||
?? createProductionWorkspaceDiagnoser(config.workspaceDiagnosticTimeoutMs, undefined, {
|
?? createProductionWorkspaceDiagnoser(config.workspaceDiagnosticTimeoutMs, undefined, {
|
||||||
internalQdrantUrl: config.internalQdrantUrl,
|
internalQdrantUrl: config.internalQdrantUrl,
|
||||||
internalEmbeddingUrl: config.internalEmbeddingUrl,
|
internalEmbeddingUrl: config.internalEmbeddingUrl,
|
||||||
|
internalEmbeddingId: config.internalEmbeddingId,
|
||||||
internalEmbeddingModel: config.internalEmbeddingModel,
|
internalEmbeddingModel: config.internalEmbeddingModel,
|
||||||
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
|
internalEmbeddingDimensions: config.internalEmbeddingDimensions,
|
||||||
});
|
});
|
||||||
@@ -271,7 +280,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
|||||||
const getSettings = async (principal: PrincipalContext): Promise<Settings> => {
|
const getSettings = async (principal: PrincipalContext): Promise<Settings> => {
|
||||||
if (deps?.getSettings) return await deps.getSettings(principal);
|
if (deps?.getSettings) return await deps.getSettings(principal);
|
||||||
const stored = loadSettings(config);
|
const stored = loadSettings(config);
|
||||||
const effective = effectiveSettings(config, stored);
|
const effective = effectiveSettings(config, stored, runtimeModelCatalog);
|
||||||
// In the registry system the legacy `harness/workspaces/*.yaml` default is obsolete: when no
|
// In the registry system the legacy `harness/workspaces/*.yaml` default is obsolete: when no
|
||||||
// installation workspace is pinned, default to the first active registry workspace.
|
// installation workspace is pinned, default to the first active registry workspace.
|
||||||
if (!stored.workspace) {
|
if (!stored.workspace) {
|
||||||
@@ -284,7 +293,9 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
|||||||
}
|
}
|
||||||
return effective;
|
return effective;
|
||||||
};
|
};
|
||||||
const piManagement = deps?.piManagement ?? createPiManagement(config, { listModels });
|
const piManagement = deps?.piManagement ?? createPiManagement(config, {
|
||||||
|
modelCatalog: runtimeModelCatalog,
|
||||||
|
});
|
||||||
|
|
||||||
const maintenanceBarrier = deps?.maintenanceBarrier ?? new MaintenanceBarrier(config.maintenanceFile);
|
const maintenanceBarrier = deps?.maintenanceBarrier ?? new MaintenanceBarrier(config.maintenanceFile);
|
||||||
const localRegistryResolver = deps?.localUserRegistry === undefined
|
const localRegistryResolver = deps?.localUserRegistry === undefined
|
||||||
@@ -408,6 +419,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
|||||||
dwhPrecheck: config.dwhPrecheck,
|
dwhPrecheck: config.dwhPrecheck,
|
||||||
legacyWorkspaceMode: config.legacyWorkspaceMode,
|
legacyWorkspaceMode: config.legacyWorkspaceMode,
|
||||||
workspaceRuntimeSupport,
|
workspaceRuntimeSupport,
|
||||||
|
modelCatalog: runtimeModelCatalog,
|
||||||
maintenanceBarrier,
|
maintenanceBarrier,
|
||||||
effectiveRelationships,
|
effectiveRelationships,
|
||||||
});
|
});
|
||||||
@@ -439,7 +451,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
|||||||
return maintenanceBarrier.status();
|
return maintenanceBarrier.status();
|
||||||
});
|
});
|
||||||
sqlRoutes(app, { tht: tht as ThtRunner, getSettings, workspaceRegistry });
|
sqlRoutes(app, { tht: tht as ThtRunner, getSettings, workspaceRegistry });
|
||||||
metaRoutes(app, { harnessDir: config.harnessDir, listModels });
|
metaRoutes(app, { harnessDir: config.harnessDir, modelCatalog: runtimeModelCatalog });
|
||||||
workspaceRoutes(app, {
|
workspaceRoutes(app, {
|
||||||
registry: workspaceRegistry,
|
registry: workspaceRegistry,
|
||||||
config: config.workspaceRegistry,
|
config: config.workspaceRegistry,
|
||||||
@@ -472,7 +484,7 @@ export function buildApp(config: AppConfig, deps?: BuildAppDeps): FastifyInstanc
|
|||||||
worker: descriptionGenerationWorker,
|
worker: descriptionGenerationWorker,
|
||||||
sensitiveDataSuggestionRunner,
|
sensitiveDataSuggestionRunner,
|
||||||
});
|
});
|
||||||
settingsRoutes(app, { cfg: config, listModels, getSettings });
|
settingsRoutes(app, { cfg: config, getSettings });
|
||||||
piManagementRoutes(app, { service: piManagement });
|
piManagementRoutes(app, { service: piManagement });
|
||||||
|
|
||||||
return app;
|
return app;
|
||||||
|
|||||||
@@ -1,63 +1,5 @@
|
|||||||
import {
|
import { loadSecretBundle } from "../config/secret-bundle.js";
|
||||||
closeSync, constants, fstatSync, lstatSync, openSync, readFileSync,
|
import { loadRuntimeModelCatalog } from "../models/runtime-model-catalog.js";
|
||||||
type Stats,
|
|
||||||
} from "node:fs";
|
|
||||||
import { parseAllDocuments } from "yaml";
|
|
||||||
import { z } from "zod";
|
|
||||||
import {
|
|
||||||
loadSecretBundle,
|
|
||||||
METADATA_GENERATION_SECRET_KEYS,
|
|
||||||
} from "../config/secret-bundle.js";
|
|
||||||
|
|
||||||
const MAX_INSTALLATION_BYTES = 1024 * 1024;
|
|
||||||
const RUNTIME_INSTALLATION_FILE = "/run/thothii-installation/thothii-installation.yaml";
|
|
||||||
const modelId = z.string().regex(/^[a-z][a-z0-9._-]{0,63}$/);
|
|
||||||
const apiKeyEnvironment = z.enum(METADATA_GENERATION_SECRET_KEYS);
|
|
||||||
const endpointSchema = z.object({
|
|
||||||
baseUrl: z.string().min(1).max(2048).refine((value) => {
|
|
||||||
try {
|
|
||||||
const url = new URL(value);
|
|
||||||
return (url.protocol === "http:" || url.protocol === "https:")
|
|
||||||
&& url.username === "" && url.password === "" && url.search === "" && url.hash === "";
|
|
||||||
} catch {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}),
|
|
||||||
apiVersion: z.string().regex(/^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/).optional(),
|
|
||||||
}).strict();
|
|
||||||
const configuredModelSchema = z.object({
|
|
||||||
id: modelId,
|
|
||||||
label: z.string().min(1).max(128).refine((value) => value.trim() === value && !/\p{Cc}/u.test(value)),
|
|
||||||
litellm: z.object({
|
|
||||||
provider: z.string().regex(/^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/),
|
|
||||||
model: z.string().regex(/^[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}$/),
|
|
||||||
disableThinking: z.literal(true).optional(),
|
|
||||||
endpoint: endpointSchema.optional(),
|
|
||||||
}).strict(),
|
|
||||||
apiKeyEnv: apiKeyEnvironment.optional(),
|
|
||||||
}).strict().superRefine((value, context) => {
|
|
||||||
if (value.apiKeyEnv === undefined && value.litellm.endpoint === undefined) {
|
|
||||||
context.addIssue({
|
|
||||||
code: z.ZodIssueCode.custom,
|
|
||||||
path: ["apiKeyEnv"],
|
|
||||||
message: "keyless models require an explicit endpoint",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
if (value.litellm.disableThinking === true && value.litellm.endpoint === undefined) {
|
|
||||||
context.addIssue({
|
|
||||||
code: z.ZodIssueCode.custom,
|
|
||||||
path: ["litellm", "disableThinking"],
|
|
||||||
message: "thinking may be disabled only for an explicit endpoint",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
const metadataGenerationSchema = z.object({
|
|
||||||
default: modelId.optional(),
|
|
||||||
models: z.array(configuredModelSchema).max(64).default([]),
|
|
||||||
}).strict();
|
|
||||||
const installationSchema = z.object({
|
|
||||||
metadataGeneration: metadataGenerationSchema.optional(),
|
|
||||||
}).passthrough();
|
|
||||||
|
|
||||||
export interface MetadataGenerationModelChoice {
|
export interface MetadataGenerationModelChoice {
|
||||||
id: string;
|
id: string;
|
||||||
@@ -86,7 +28,6 @@ export class MetadataGenerationModelUnavailableError extends Error {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The complete interface callers need: safe discovery plus fail-closed runtime resolution. */
|
|
||||||
export interface MetadataGenerationModels {
|
export interface MetadataGenerationModels {
|
||||||
catalog(): MetadataGenerationModelCatalog;
|
catalog(): MetadataGenerationModelCatalog;
|
||||||
resolve(selection: string): ResolvedMetadataGenerationModel;
|
resolve(selection: string): ResolvedMetadataGenerationModel;
|
||||||
@@ -96,140 +37,78 @@ class RestartLoadedMetadataGenerationModels implements MetadataGenerationModels
|
|||||||
readonly #models: ReadonlyMap<string, ResolvedMetadataGenerationModel>;
|
readonly #models: ReadonlyMap<string, ResolvedMetadataGenerationModel>;
|
||||||
readonly #catalog: MetadataGenerationModelCatalog;
|
readonly #catalog: MetadataGenerationModelCatalog;
|
||||||
|
|
||||||
constructor(
|
constructor(models: ReadonlyMap<string, ResolvedMetadataGenerationModel>, defaultModel: string | null) {
|
||||||
models: ReadonlyMap<string, ResolvedMetadataGenerationModel> = new Map(),
|
|
||||||
defaultModel: string | null = null,
|
|
||||||
choices: MetadataGenerationModelChoice[] = [],
|
|
||||||
) {
|
|
||||||
this.#models = models;
|
this.#models = models;
|
||||||
this.#catalog = {
|
this.#catalog = {
|
||||||
models: choices.map((choice) => ({ ...choice })),
|
models: [...models.values()].map(({ id }) => ({ id, label: id })),
|
||||||
default: defaultModel,
|
default: defaultModel,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
catalog(): MetadataGenerationModelCatalog {
|
catalog(): MetadataGenerationModelCatalog {
|
||||||
return {
|
return { models: this.#catalog.models.map((choice) => ({ ...choice })), default: this.#catalog.default };
|
||||||
models: this.#catalog.models.map((choice) => ({ ...choice })),
|
|
||||||
default: this.#catalog.default,
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
resolve(selection: string): ResolvedMetadataGenerationModel {
|
resolve(selection: string): ResolvedMetadataGenerationModel {
|
||||||
const model = typeof selection === "string" ? this.#models.get(selection) : undefined;
|
const model = this.#models.get(selection);
|
||||||
if (!model) throw new MetadataGenerationModelUnavailableError();
|
if (!model) throw new MetadataGenerationModelUnavailableError();
|
||||||
return model;
|
return model;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function invalid(message = "metadata-generation configuration is invalid"): Error {
|
function invalid(message = "metadata-generation runtime catalog is invalid"): Error {
|
||||||
return new Error(message);
|
return new Error(message);
|
||||||
}
|
}
|
||||||
|
|
||||||
function protectedInstallationStat(file: string, info: Stats): boolean {
|
|
||||||
const mode = info.mode & 0o777;
|
|
||||||
if (!info.isFile() || info.isSymbolicLink() || info.nlink !== 1
|
|
||||||
|| info.size < 1 || info.size > MAX_INSTALLATION_BYTES) return false;
|
|
||||||
if (file === RUNTIME_INSTALLATION_FILE && info.uid === 0 && mode === 0o444) return true;
|
|
||||||
return info.uid === (process.getuid?.() ?? info.uid) && (mode === 0o400 || mode === 0o600);
|
|
||||||
}
|
|
||||||
|
|
||||||
function readProtectedInstallation(file: string): string {
|
|
||||||
let descriptor: number | undefined;
|
|
||||||
try {
|
|
||||||
const before = lstatSync(file);
|
|
||||||
if (!protectedInstallationStat(file, before)) throw new Error("unavailable");
|
|
||||||
descriptor = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW);
|
|
||||||
const opened = fstatSync(descriptor);
|
|
||||||
if (!protectedInstallationStat(file, opened)
|
|
||||||
|| before.dev !== opened.dev || before.ino !== opened.ino) throw new Error("unavailable");
|
|
||||||
const source = readFileSync(descriptor, "utf8");
|
|
||||||
const after = fstatSync(descriptor);
|
|
||||||
const current = lstatSync(file);
|
|
||||||
if (!protectedInstallationStat(file, after) || !protectedInstallationStat(file, current)
|
|
||||||
|| opened.dev !== after.dev || opened.ino !== after.ino
|
|
||||||
|| opened.dev !== current.dev || opened.ino !== current.ino) throw new Error("unavailable");
|
|
||||||
return source;
|
|
||||||
} finally {
|
|
||||||
if (descriptor !== undefined) try { closeSync(descriptor); } catch { /* sanitized below */ }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function readInstallation(file: string): unknown {
|
|
||||||
try {
|
|
||||||
const documents = parseAllDocuments(readProtectedInstallation(file), { uniqueKeys: true });
|
|
||||||
if (documents.length !== 1) throw invalid("metadata-generation installation must contain one YAML document");
|
|
||||||
const document = documents[0];
|
|
||||||
if (document.errors.length > 0 || document.warnings.length > 0) {
|
|
||||||
throw invalid("metadata-generation installation contains invalid YAML");
|
|
||||||
}
|
|
||||||
return document.toJSON();
|
|
||||||
} catch (error) {
|
|
||||||
if (error instanceof Error && error.message.startsWith("metadata-generation")) throw error;
|
|
||||||
throw invalid("metadata-generation installation is unavailable");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export function loadMetadataGenerationModels(options: {
|
export function loadMetadataGenerationModels(options: {
|
||||||
installationFile?: string;
|
catalogFile?: string;
|
||||||
secretsFile?: string;
|
secretsFile?: string;
|
||||||
}): MetadataGenerationModels {
|
}): MetadataGenerationModels {
|
||||||
if (!options.installationFile) return new RestartLoadedMetadataGenerationModels();
|
const catalog = loadRuntimeModelCatalog(options.catalogFile);
|
||||||
const installation = installationSchema.safeParse(readInstallation(options.installationFile));
|
const configured = catalog.metadataModels();
|
||||||
if (!installation.success) throw invalid();
|
if (configured.length === 0) return new RestartLoadedMetadataGenerationModels(new Map(), null);
|
||||||
const configured = installation.data.metadataGeneration;
|
|
||||||
if (!configured || configured.models.length === 0) {
|
|
||||||
if (configured?.default !== undefined) throw invalid("metadata-generation default does not identify a configured model");
|
|
||||||
return new RestartLoadedMetadataGenerationModels();
|
|
||||||
}
|
|
||||||
if (!configured.default) throw invalid("metadata-generation default is required when models are configured");
|
|
||||||
|
|
||||||
const seen = new Set<string>();
|
const requiresSecrets = configured.some((model) => model.authentication.mode === "secret_env");
|
||||||
for (const model of configured.models) {
|
|
||||||
if (seen.has(model.id)) throw invalid(`metadata-generation model id "${model.id}" is duplicated`);
|
|
||||||
seen.add(model.id);
|
|
||||||
}
|
|
||||||
if (!seen.has(configured.default)) {
|
|
||||||
throw invalid(`metadata-generation default "${configured.default}" is not configured`);
|
|
||||||
}
|
|
||||||
const requiresSecrets = configured.models.some((model) => model.apiKeyEnv !== undefined);
|
|
||||||
let secrets: ReadonlyMap<string, string> = new Map();
|
let secrets: ReadonlyMap<string, string> = new Map();
|
||||||
if (requiresSecrets) {
|
if (requiresSecrets) {
|
||||||
if (!options.secretsFile) throw invalid("metadata-generation keyed models require THT_SECRETS_FILE");
|
if (!options.secretsFile) throw invalid("metadata-generation keyed models require THT_SECRETS_FILE");
|
||||||
try {
|
try { secrets = loadSecretBundle(options.secretsFile); }
|
||||||
secrets = loadSecretBundle(options.secretsFile);
|
catch { throw invalid("metadata-generation secrets are unavailable"); }
|
||||||
} catch {
|
|
||||||
throw invalid("metadata-generation secrets are unavailable");
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const models = new Map<string, ResolvedMetadataGenerationModel>();
|
const models = new Map<string, ResolvedMetadataGenerationModel>();
|
||||||
for (const configuredModel of configured.models) {
|
const labels = new Map<string, string>();
|
||||||
|
for (const configuredModel of configured) {
|
||||||
|
const adapter = configuredModel.metadataAdapter;
|
||||||
|
if (!adapter || configuredModel.authentication.mode === "pi_auth") throw invalid();
|
||||||
|
const apiKeyEnv = configuredModel.authentication.apiKeyEnv;
|
||||||
let apiKey: string | undefined;
|
let apiKey: string | undefined;
|
||||||
if (configuredModel.apiKeyEnv !== undefined) {
|
if (configuredModel.authentication.mode === "secret_env") {
|
||||||
apiKey = secrets.get(configuredModel.apiKeyEnv);
|
if (!apiKeyEnv) throw invalid();
|
||||||
if (!apiKey) {
|
apiKey = secrets.get(apiKeyEnv);
|
||||||
throw invalid(`metadata-generation model "${configuredModel.id}" secret "${configuredModel.apiKeyEnv}" is missing`);
|
if (!apiKey) throw invalid(`metadata-generation model "${configuredModel.id}" secret "${apiKeyEnv}" is missing`);
|
||||||
}
|
|
||||||
if (apiKey.length > 16 * 1024 || /\s/u.test(apiKey)) {
|
if (apiKey.length > 16 * 1024 || /\s/u.test(apiKey)) {
|
||||||
throw invalid(`metadata-generation model "${configuredModel.id}" secret "${configuredModel.apiKeyEnv}" is unusable`);
|
throw invalid(`metadata-generation model "${configuredModel.id}" secret "${apiKeyEnv}" is unusable`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
labels.set(configuredModel.id, configuredModel.label);
|
||||||
models.set(configuredModel.id, Object.freeze({
|
models.set(configuredModel.id, Object.freeze({
|
||||||
id: configuredModel.id,
|
id: configuredModel.id,
|
||||||
provider: configuredModel.litellm.provider,
|
provider: adapter.litellmProvider,
|
||||||
model: configuredModel.litellm.model,
|
model: configuredModel.upstreamModel,
|
||||||
...(configuredModel.litellm.disableThinking === true ? { disableThinking: true as const } : {}),
|
...(configuredModel.metadataGeneration?.disableThinking === true
|
||||||
...(configuredModel.litellm.endpoint === undefined
|
? { disableThinking: true as const } : {}),
|
||||||
? {}
|
...(configuredModel.endpoint ? { endpoint: Object.freeze({ ...configuredModel.endpoint }) } : {}),
|
||||||
: { endpoint: Object.freeze({ ...configuredModel.litellm.endpoint }) }),
|
...(apiKeyEnv ? { apiKeyEnv, apiKey } : {}),
|
||||||
...(configuredModel.apiKeyEnv === undefined
|
|
||||||
? {}
|
|
||||||
: { apiKeyEnv: configuredModel.apiKeyEnv, apiKey }),
|
|
||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
return new RestartLoadedMetadataGenerationModels(
|
const result = new RestartLoadedMetadataGenerationModels(models, catalog.defaultMetadataGeneration);
|
||||||
models,
|
const safe = result.catalog();
|
||||||
configured.default,
|
return {
|
||||||
configured.models.map(({ id, label }) => ({ id, label })),
|
catalog: () => ({
|
||||||
);
|
default: safe.default,
|
||||||
|
models: safe.models.map((choice) => ({ ...choice, label: labels.get(choice.id) ?? choice.id })),
|
||||||
|
}),
|
||||||
|
resolve: (selection) => result.resolve(selection),
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import * as sensitiveDataFlagMigration from "./migrations/006_sensitive_data_fla
|
|||||||
import * as sensitiveDataSuggestionRunsMigration from "./migrations/007_sensitive_data_suggestion_runs.js";
|
import * as sensitiveDataSuggestionRunsMigration from "./migrations/007_sensitive_data_suggestion_runs.js";
|
||||||
import * as catalogLogicalRelationshipsMigration from "./migrations/008_catalog_logical_relationships.js";
|
import * as catalogLogicalRelationshipsMigration from "./migrations/008_catalog_logical_relationships.js";
|
||||||
import * as aiTokenUsageMigration from "./migrations/009_ai_token_usage.js";
|
import * as aiTokenUsageMigration from "./migrations/009_ai_token_usage.js";
|
||||||
|
import * as canonicalModelIdsMigration from "./migrations/010_canonical_model_ids.js";
|
||||||
|
|
||||||
const connectionString = process.env.THT_CATALOG_MIGRATOR_DATABASE_URL;
|
const connectionString = process.env.THT_CATALOG_MIGRATOR_DATABASE_URL;
|
||||||
const host = process.env.THT_CATALOG_DB_HOST;
|
const host = process.env.THT_CATALOG_DB_HOST;
|
||||||
@@ -46,6 +47,7 @@ const provider: MigrationProvider = {
|
|||||||
"007_sensitive_data_suggestion_runs": sensitiveDataSuggestionRunsMigration,
|
"007_sensitive_data_suggestion_runs": sensitiveDataSuggestionRunsMigration,
|
||||||
"008_catalog_logical_relationships": catalogLogicalRelationshipsMigration,
|
"008_catalog_logical_relationships": catalogLogicalRelationshipsMigration,
|
||||||
"009_ai_token_usage": aiTokenUsageMigration,
|
"009_ai_token_usage": aiTokenUsageMigration,
|
||||||
|
"010_canonical_model_ids": canonicalModelIdsMigration,
|
||||||
};
|
};
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
import { sql, type Kysely } from "kysely";
|
||||||
|
import type { CatalogDatabase } from "../repository.js";
|
||||||
|
|
||||||
|
const canonicalModelPattern = "^[a-z][a-z0-9._-]{0,63}/[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$";
|
||||||
|
const legacyModelPattern = "^[a-z][a-z0-9._-]{0,63}$";
|
||||||
|
|
||||||
|
export async function up(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||||
|
await sql.raw(`alter table description_generation_runs
|
||||||
|
drop constraint description_generation_runs_model_id_check,
|
||||||
|
add constraint description_generation_runs_model_id_check
|
||||||
|
check (model_id ~ '${canonicalModelPattern}')`).execute(db);
|
||||||
|
await sql.raw(`alter table sensitive_data_suggestion_runs
|
||||||
|
drop constraint sensitive_data_suggestion_runs_model_id_check,
|
||||||
|
add constraint sensitive_data_suggestion_runs_model_id_check
|
||||||
|
check (model_id ~ '${canonicalModelPattern}')`).execute(db);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function down(db: Kysely<CatalogDatabase>): Promise<void> {
|
||||||
|
await sql.raw(`alter table sensitive_data_suggestion_runs
|
||||||
|
drop constraint sensitive_data_suggestion_runs_model_id_check,
|
||||||
|
add constraint sensitive_data_suggestion_runs_model_id_check
|
||||||
|
check (model_id ~ '${legacyModelPattern}')`).execute(db);
|
||||||
|
await sql.raw(`alter table description_generation_runs
|
||||||
|
drop constraint description_generation_runs_model_id_check,
|
||||||
|
add constraint description_generation_runs_model_id_check
|
||||||
|
check (model_id ~ '${legacyModelPattern}')`).execute(db);
|
||||||
|
}
|
||||||
+32
-2
@@ -32,6 +32,7 @@ export interface AppConfig {
|
|||||||
piManagementTimeoutMs: number;
|
piManagementTimeoutMs: number;
|
||||||
secretsFile?: string;
|
secretsFile?: string;
|
||||||
installationConfigFile?: string;
|
installationConfigFile?: string;
|
||||||
|
modelCatalogFile?: string;
|
||||||
piAuthFile?: string;
|
piAuthFile?: string;
|
||||||
secretFiles: Readonly<Record<string, string | undefined>>;
|
secretFiles: Readonly<Record<string, string | undefined>>;
|
||||||
modelApiKeyFile?: string;
|
modelApiKeyFile?: string;
|
||||||
@@ -50,6 +51,7 @@ export interface AppConfig {
|
|||||||
workspaceSecretRuntimeRoot: string;
|
workspaceSecretRuntimeRoot: string;
|
||||||
internalQdrantUrl: string;
|
internalQdrantUrl: string;
|
||||||
internalEmbeddingUrl: string;
|
internalEmbeddingUrl: string;
|
||||||
|
internalEmbeddingId: string;
|
||||||
internalEmbeddingModel: string;
|
internalEmbeddingModel: string;
|
||||||
internalEmbeddingDimensions: number;
|
internalEmbeddingDimensions: number;
|
||||||
}
|
}
|
||||||
@@ -189,6 +191,21 @@ function positiveDimension(value: string | undefined, fallback: number): number
|
|||||||
return parsed;
|
return parsed;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function internalEmbeddingIdentity(
|
||||||
|
identityValue: string | undefined,
|
||||||
|
modelValue: string | undefined,
|
||||||
|
): { id: string; model: string } {
|
||||||
|
const id = identityValue ?? "ollama/qwen3-embedding:0.6b";
|
||||||
|
if (!/^ollama\/[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$/.test(id)) {
|
||||||
|
throw new Error("internal embedding identity configuration is invalid");
|
||||||
|
}
|
||||||
|
const model = id.slice(id.indexOf("/") + 1);
|
||||||
|
if (modelValue !== undefined && modelValue !== model) {
|
||||||
|
throw new Error("internal embedding model does not match its canonical identity");
|
||||||
|
}
|
||||||
|
return { id, model };
|
||||||
|
}
|
||||||
|
|
||||||
function catalogDatabase(env: Record<string, string | undefined>): CatalogConnectionConfig | undefined {
|
function catalogDatabase(env: Record<string, string | undefined>): CatalogConnectionConfig | undefined {
|
||||||
const value = env.THT_CATALOG_DATABASE_URL;
|
const value = env.THT_CATALOG_DATABASE_URL;
|
||||||
if (value !== undefined) {
|
if (value !== undefined) {
|
||||||
@@ -330,6 +347,13 @@ export function loadConfig(
|
|||||||
|| installationConfigFile.includes("\0")
|
|| installationConfigFile.includes("\0")
|
||||||
|| !path.isAbsolute(installationConfigFile)
|
|| !path.isAbsolute(installationConfigFile)
|
||||||
)) throw new Error("installation configuration is invalid");
|
)) throw new Error("installation configuration is invalid");
|
||||||
|
const modelCatalogFile = env.THT_MODEL_CATALOG_FILE;
|
||||||
|
if (modelCatalogFile !== undefined && (
|
||||||
|
modelCatalogFile.trim() !== modelCatalogFile
|
||||||
|
|| modelCatalogFile.length === 0
|
||||||
|
|| modelCatalogFile.includes("\0")
|
||||||
|
|| !path.isAbsolute(modelCatalogFile)
|
||||||
|
)) throw new Error("runtime model catalog configuration is invalid");
|
||||||
const piAuthFile = env.THT_PI_AUTH_FILE;
|
const piAuthFile = env.THT_PI_AUTH_FILE;
|
||||||
if (piAuthFile !== undefined && (
|
if (piAuthFile !== undefined && (
|
||||||
piAuthFile.trim() !== piAuthFile || piAuthFile.length === 0 || piAuthFile.includes("\0")
|
piAuthFile.trim() !== piAuthFile || piAuthFile.length === 0 || piAuthFile.includes("\0")
|
||||||
@@ -391,6 +415,10 @@ export function loadConfig(
|
|||||||
"internal embedding URL",
|
"internal embedding URL",
|
||||||
["embedding", "localhost"],
|
["embedding", "localhost"],
|
||||||
);
|
);
|
||||||
|
const internalEmbedding = internalEmbeddingIdentity(
|
||||||
|
env.THT_INTERNAL_EMBEDDING_ID,
|
||||||
|
env.THT_INTERNAL_EMBEDDING_MODEL,
|
||||||
|
);
|
||||||
return {
|
return {
|
||||||
host: env.HOST ?? "127.0.0.1",
|
host: env.HOST ?? "127.0.0.1",
|
||||||
port: Number(env.PORT ?? 8787),
|
port: Number(env.PORT ?? 8787),
|
||||||
@@ -404,7 +432,7 @@ export function loadConfig(
|
|||||||
publicExposure,
|
publicExposure,
|
||||||
sessionStorage,
|
sessionStorage,
|
||||||
catalogDatabase: catalogDatabase(env),
|
catalogDatabase: catalogDatabase(env),
|
||||||
defaults: { provider: env.PI_PROVIDER, model: env.PI_MODEL, thinking: env.PI_THINKING },
|
defaults: { thinking: env.PI_THINKING },
|
||||||
maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4),
|
maxPiProcesses: Number(env.MAX_PI_PROCESSES ?? 4),
|
||||||
settingsFile,
|
settingsFile,
|
||||||
maintenanceFile: env.THT_MAINTENANCE_FILE ?? path.join(path.dirname(settingsFile), "maintenance.json"),
|
maintenanceFile: env.THT_MAINTENANCE_FILE ?? path.join(path.dirname(settingsFile), "maintenance.json"),
|
||||||
@@ -413,6 +441,7 @@ export function loadConfig(
|
|||||||
piManagementTimeoutMs: piManagementTimeout(env.PI_MANAGEMENT_TIMEOUT_MS),
|
piManagementTimeoutMs: piManagementTimeout(env.PI_MANAGEMENT_TIMEOUT_MS),
|
||||||
secretsFile,
|
secretsFile,
|
||||||
installationConfigFile,
|
installationConfigFile,
|
||||||
|
modelCatalogFile,
|
||||||
piAuthFile,
|
piAuthFile,
|
||||||
secretFiles,
|
secretFiles,
|
||||||
modelApiKeyFile,
|
modelApiKeyFile,
|
||||||
@@ -425,7 +454,8 @@ export function loadConfig(
|
|||||||
workspaceSecretRuntimeRoot,
|
workspaceSecretRuntimeRoot,
|
||||||
internalQdrantUrl,
|
internalQdrantUrl,
|
||||||
internalEmbeddingUrl,
|
internalEmbeddingUrl,
|
||||||
internalEmbeddingModel: env.THT_INTERNAL_EMBEDDING_MODEL ?? "qwen3-embedding:0.6b",
|
internalEmbeddingId: internalEmbedding.id,
|
||||||
|
internalEmbeddingModel: internalEmbedding.model,
|
||||||
internalEmbeddingDimensions: positiveDimension(env.THT_INTERNAL_EMBEDDING_DIMENSIONS, 1024),
|
internalEmbeddingDimensions: positiveDimension(env.THT_INTERNAL_EMBEDDING_DIMENSIONS, 1024),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ import {
|
|||||||
isUsableAuthenticationSecret,
|
isUsableAuthenticationSecret,
|
||||||
} from "../auth/secret-policy.js";
|
} from "../auth/secret-policy.js";
|
||||||
|
|
||||||
/** Credential names that metadata-generation model entries may reference. */
|
/** Credential names that Installation Model Catalog providers may reference. */
|
||||||
export const METADATA_GENERATION_SECRET_KEYS = Object.freeze([
|
export const METADATA_GENERATION_SECRET_KEYS = Object.freeze([
|
||||||
"THT_METADATA_API_KEY", "ANTHROPIC_API_KEY", "AZURE_API_KEY", "GEMINI_API_KEY",
|
"THT_METADATA_API_KEY", "ANTHROPIC_API_KEY", "AZURE_API_KEY", "GEMINI_API_KEY",
|
||||||
"DEEPSEEK_API_KEY", "OPENAI_API_KEY", "OPENROUTER_API_KEY", "ZAI_API_KEY",
|
"DEEPSEEK_API_KEY", "OPENAI_API_KEY", "OPENROUTER_API_KEY", "ZAI_API_KEY",
|
||||||
|
|||||||
@@ -0,0 +1,161 @@
|
|||||||
|
import {
|
||||||
|
closeSync, constants, fstatSync, lstatSync, openSync, readFileSync, type Stats,
|
||||||
|
} from "node:fs";
|
||||||
|
import { z } from "zod";
|
||||||
|
|
||||||
|
const MAX_CATALOG_BYTES = 1024 * 1024;
|
||||||
|
const RUNTIME_CATALOG_FILE = "/run/thothii-model-catalog/catalog.json";
|
||||||
|
const canonicalId = z.string().regex(/^[a-z][a-z0-9._-]{0,63}\/[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$/);
|
||||||
|
const secretBundleKey = /^[A-Z][A-Z0-9_]{0,63}$/;
|
||||||
|
|
||||||
|
const endpointSchema = z.object({
|
||||||
|
baseUrl: z.string().url(),
|
||||||
|
apiVersion: z.string().optional(),
|
||||||
|
}).strict();
|
||||||
|
|
||||||
|
const authenticationSchema = z.object({
|
||||||
|
mode: z.enum(["secret_env", "pi_auth", "none"]),
|
||||||
|
apiKeyEnv: z.string().optional(),
|
||||||
|
}).strict();
|
||||||
|
|
||||||
|
const runtimeModelSchema = z.object({
|
||||||
|
id: canonicalId,
|
||||||
|
provider: z.string().min(1),
|
||||||
|
model: z.string().min(1),
|
||||||
|
label: z.string().min(1),
|
||||||
|
upstreamModel: z.string().min(1),
|
||||||
|
endpoint: endpointSchema.optional(),
|
||||||
|
authentication: authenticationSchema,
|
||||||
|
sessionAdapter: z.object({ mode: z.enum(["pi_builtin", "openai_compatible"]) }).strict().optional(),
|
||||||
|
metadataAdapter: z.object({ litellmProvider: z.string().min(1) }).strict().optional(),
|
||||||
|
session: z.object({
|
||||||
|
reasoning: z.boolean(),
|
||||||
|
input: z.array(z.string()).optional(),
|
||||||
|
cost: z.object({
|
||||||
|
input: z.number(), output: z.number(), cacheRead: z.number(), cacheWrite: z.number(),
|
||||||
|
}).strict().optional(),
|
||||||
|
contextWindow: z.number().int().positive().optional(),
|
||||||
|
maxTokens: z.number().int().positive().optional(),
|
||||||
|
compatibility: z.object({
|
||||||
|
supportsDeveloperRole: z.boolean(),
|
||||||
|
supportsReasoningEffort: z.boolean(),
|
||||||
|
supportsStore: z.boolean(),
|
||||||
|
maxTokensField: z.string().optional(),
|
||||||
|
}).strict().optional(),
|
||||||
|
}).strict().optional(),
|
||||||
|
metadataGeneration: z.object({ disableThinking: z.boolean() }).strict().optional(),
|
||||||
|
}).strict();
|
||||||
|
|
||||||
|
const catalogSchema = z.object({
|
||||||
|
schemaVersion: z.literal(1),
|
||||||
|
defaultSession: canonicalId,
|
||||||
|
defaultMetadataGeneration: canonicalId.optional(),
|
||||||
|
embedding: z.object({ id: canonicalId, dimensions: z.number().int().positive() }).strict(),
|
||||||
|
models: z.array(runtimeModelSchema).max(64),
|
||||||
|
}).strict();
|
||||||
|
|
||||||
|
export type RuntimeModel = z.infer<typeof runtimeModelSchema>;
|
||||||
|
|
||||||
|
export interface RuntimeModelCatalog {
|
||||||
|
readonly defaultSession: string | null;
|
||||||
|
readonly defaultMetadataGeneration: string | null;
|
||||||
|
readonly embedding: Readonly<{ id: string; dimensions: number }> | null;
|
||||||
|
sessionModels(): readonly RuntimeModel[];
|
||||||
|
metadataModels(): readonly RuntimeModel[];
|
||||||
|
hasSession(id: string): boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
class RestartLoadedRuntimeModelCatalog implements RuntimeModelCatalog {
|
||||||
|
readonly defaultSession: string | null;
|
||||||
|
readonly defaultMetadataGeneration: string | null;
|
||||||
|
readonly embedding: Readonly<{ id: string; dimensions: number }> | null;
|
||||||
|
readonly #sessions: readonly RuntimeModel[];
|
||||||
|
readonly #metadata: readonly RuntimeModel[];
|
||||||
|
readonly #sessionIds: ReadonlySet<string>;
|
||||||
|
|
||||||
|
constructor(catalog?: z.infer<typeof catalogSchema>) {
|
||||||
|
this.defaultSession = catalog?.defaultSession ?? null;
|
||||||
|
this.defaultMetadataGeneration = catalog?.defaultMetadataGeneration ?? null;
|
||||||
|
this.embedding = catalog ? Object.freeze({ ...catalog.embedding }) : null;
|
||||||
|
this.#sessions = Object.freeze((catalog?.models ?? []).filter((model) => model.session !== undefined));
|
||||||
|
this.#metadata = Object.freeze((catalog?.models ?? []).filter((model) => model.metadataGeneration !== undefined));
|
||||||
|
this.#sessionIds = new Set(this.#sessions.map((model) => model.id));
|
||||||
|
}
|
||||||
|
|
||||||
|
sessionModels(): readonly RuntimeModel[] { return this.#sessions.map((model) => ({ ...model })); }
|
||||||
|
metadataModels(): readonly RuntimeModel[] { return this.#metadata.map((model) => ({ ...model })); }
|
||||||
|
hasSession(id: string): boolean { return this.#sessionIds.has(id); }
|
||||||
|
}
|
||||||
|
|
||||||
|
function protectedCatalogStat(file: string, info: Stats): boolean {
|
||||||
|
const mode = info.mode & 0o777;
|
||||||
|
if (!info.isFile() || info.isSymbolicLink() || info.nlink !== 1
|
||||||
|
|| info.size < 1 || info.size > MAX_CATALOG_BYTES) return false;
|
||||||
|
if (file === RUNTIME_CATALOG_FILE && info.uid === 0 && (mode === 0o444 || mode === 0o644)) return true;
|
||||||
|
return info.uid === (process.getuid?.() ?? info.uid) && (mode === 0o400 || mode === 0o600 || mode === 0o644);
|
||||||
|
}
|
||||||
|
|
||||||
|
function readProtectedCatalog(file: string): unknown {
|
||||||
|
let descriptor: number | undefined;
|
||||||
|
try {
|
||||||
|
const before = lstatSync(file);
|
||||||
|
if (!protectedCatalogStat(file, before)) throw new Error("runtime model catalog is unavailable");
|
||||||
|
descriptor = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW);
|
||||||
|
const opened = fstatSync(descriptor);
|
||||||
|
if (!protectedCatalogStat(file, opened)
|
||||||
|
|| before.dev !== opened.dev || before.ino !== opened.ino) throw new Error("runtime model catalog is unavailable");
|
||||||
|
const source = readFileSync(descriptor, "utf8");
|
||||||
|
const after = fstatSync(descriptor);
|
||||||
|
const current = lstatSync(file);
|
||||||
|
if (!protectedCatalogStat(file, after) || !protectedCatalogStat(file, current)
|
||||||
|
|| opened.dev !== after.dev || opened.ino !== after.ino
|
||||||
|
|| opened.dev !== current.dev || opened.ino !== current.ino) throw new Error("runtime model catalog is unavailable");
|
||||||
|
return JSON.parse(source);
|
||||||
|
} catch {
|
||||||
|
throw new Error("runtime model catalog is unavailable");
|
||||||
|
} finally {
|
||||||
|
if (descriptor !== undefined) try { closeSync(descriptor); } catch { /* sanitized above */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadRuntimeModelCatalog(file?: string): RuntimeModelCatalog {
|
||||||
|
if (!file) return new RestartLoadedRuntimeModelCatalog();
|
||||||
|
const parsed = catalogSchema.safeParse(readProtectedCatalog(file));
|
||||||
|
if (!parsed.success) throw new Error("runtime model catalog is invalid");
|
||||||
|
if (parsed.data.models.some((model) => !validRuntimeModel(model))) {
|
||||||
|
throw new Error("runtime model catalog is invalid");
|
||||||
|
}
|
||||||
|
const ids = new Set(parsed.data.models.map((model) => model.id));
|
||||||
|
if (ids.size !== parsed.data.models.length) throw new Error("runtime model catalog contains duplicate models");
|
||||||
|
const sessions = parsed.data.models.filter((model) => model.session !== undefined).map((model) => model.id);
|
||||||
|
const metadata = parsed.data.models.filter((model) => model.metadataGeneration !== undefined).map((model) => model.id);
|
||||||
|
if (!sessions.includes(parsed.data.defaultSession)) throw new Error("runtime model catalog session default is invalid");
|
||||||
|
if ((metadata.length > 0) !== (parsed.data.defaultMetadataGeneration !== undefined)
|
||||||
|
|| (parsed.data.defaultMetadataGeneration !== undefined
|
||||||
|
&& !metadata.includes(parsed.data.defaultMetadataGeneration))) {
|
||||||
|
throw new Error("runtime model catalog metadata default is invalid");
|
||||||
|
}
|
||||||
|
return new RestartLoadedRuntimeModelCatalog(parsed.data);
|
||||||
|
}
|
||||||
|
|
||||||
|
function validRuntimeModel(model: RuntimeModel): boolean {
|
||||||
|
if ((model.session !== undefined) !== (model.sessionAdapter !== undefined)) return false;
|
||||||
|
if ((model.metadataGeneration !== undefined) !== (model.metadataAdapter !== undefined)) return false;
|
||||||
|
switch (model.authentication.mode) {
|
||||||
|
case "secret_env":
|
||||||
|
return model.authentication.apiKeyEnv !== undefined
|
||||||
|
&& secretBundleKey.test(model.authentication.apiKeyEnv);
|
||||||
|
case "pi_auth":
|
||||||
|
return model.authentication.apiKeyEnv === undefined
|
||||||
|
&& model.metadataGeneration === undefined
|
||||||
|
&& model.sessionAdapter?.mode === "pi_builtin";
|
||||||
|
case "none":
|
||||||
|
return model.authentication.apiKeyEnv === undefined && model.endpoint !== undefined;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function splitCanonicalModelId(id: string): { provider: string; model: string } {
|
||||||
|
const slash = id.indexOf("/");
|
||||||
|
if (slash <= 0 || slash === id.length - 1) throw new Error("model identity is invalid");
|
||||||
|
return { provider: id.slice(0, slash), model: id.slice(slash + 1) };
|
||||||
|
}
|
||||||
@@ -8,8 +8,8 @@ import { join } from "node:path";
|
|||||||
import { loadConfig, type AppConfig } from "./config.js";
|
import { loadConfig, type AppConfig } from "./config.js";
|
||||||
import { rolesToPermissions } from "./auth/config.js";
|
import { rolesToPermissions } from "./auth/config.js";
|
||||||
import type { PrincipalContext } from "./auth/principal.js";
|
import type { PrincipalContext } from "./auth/principal.js";
|
||||||
import { createPiModelLister } from "./pi/list-models.js";
|
|
||||||
import { createPiManagement } from "./pi/management.js";
|
import { createPiManagement } from "./pi/management.js";
|
||||||
|
import { loadRuntimeModelCatalog } from "./models/runtime-model-catalog.js";
|
||||||
import { effectiveSettings } from "./routes/settings.js";
|
import { effectiveSettings } from "./routes/settings.js";
|
||||||
import { MaintenanceBarrier } from "./runtime/maintenance-gate.js";
|
import { MaintenanceBarrier } from "./runtime/maintenance-gate.js";
|
||||||
import { loadSettings } from "./settings/settings-store.js";
|
import { loadSettings } from "./settings/settings-store.js";
|
||||||
@@ -19,7 +19,7 @@ import { WorkspaceSecretStore } from "./workspaces/secret-store.js";
|
|||||||
|
|
||||||
type OperatorAction = "maintenance-activate" | "maintenance-deactivate" | "maintenance-status"
|
type OperatorAction = "maintenance-activate" | "maintenance-deactivate" | "maintenance-status"
|
||||||
| "session-inventory" | "workflow-doctor" | "workspace-integrity"
|
| "session-inventory" | "workflow-doctor" | "workspace-integrity"
|
||||||
| "pi-options" | "pi-test" | "effective-settings";
|
| "pi-test" | "effective-settings";
|
||||||
|
|
||||||
const lifecyclePrincipal: PrincipalContext = {
|
const lifecyclePrincipal: PrincipalContext = {
|
||||||
issuer: "tht-operator-command",
|
issuer: "tht-operator-command",
|
||||||
@@ -110,9 +110,11 @@ export async function runOperatorAction(
|
|||||||
if (action === "session-inventory") return await sessionInventory(config);
|
if (action === "session-inventory") return await sessionInventory(config);
|
||||||
if (action === "workflow-doctor") return await workflowDiagnostics(config);
|
if (action === "workflow-doctor") return await workflowDiagnostics(config);
|
||||||
if (action === "workspace-integrity") return await workspaceIntegrity(config);
|
if (action === "workspace-integrity") return await workspaceIntegrity(config);
|
||||||
if (action === "effective-settings") return effectiveSettings(config, loadSettings(config));
|
const modelCatalog = loadRuntimeModelCatalog(config.modelCatalogFile);
|
||||||
const service = createPiManagement(config, { listModels: createPiModelLister(config) });
|
if (action === "effective-settings") {
|
||||||
if (action === "pi-options") return await service.options();
|
return effectiveSettings(config, loadSettings(config), modelCatalog);
|
||||||
|
}
|
||||||
|
const service = createPiManagement(config, { modelCatalog });
|
||||||
if (action === "pi-test") return await service.test();
|
if (action === "pi-test") return await service.test();
|
||||||
throw new Error("unsupported operator action");
|
throw new Error("unsupported operator action");
|
||||||
}
|
}
|
||||||
@@ -121,7 +123,7 @@ async function main(): Promise<void> {
|
|||||||
const action = process.argv[2] as OperatorAction | undefined;
|
const action = process.argv[2] as OperatorAction | undefined;
|
||||||
if (!action || ![
|
if (!action || ![
|
||||||
"maintenance-activate", "maintenance-deactivate", "maintenance-status", "session-inventory",
|
"maintenance-activate", "maintenance-deactivate", "maintenance-status", "session-inventory",
|
||||||
"workflow-doctor", "workspace-integrity", "pi-options", "pi-test", "effective-settings",
|
"workflow-doctor", "workspace-integrity", "pi-test", "effective-settings",
|
||||||
].includes(action)) throw new Error("invalid operator action");
|
].includes(action)) throw new Error("invalid operator action");
|
||||||
const result = await runOperatorAction(action, loadConfig(process.env));
|
const result = await runOperatorAction(action, loadConfig(process.env));
|
||||||
process.stdout.write(`${JSON.stringify(result)}\n`);
|
process.stdout.write(`${JSON.stringify(result)}\n`);
|
||||||
|
|||||||
@@ -18,6 +18,8 @@ export interface PiModel {
|
|||||||
reasoning: boolean;
|
reasoning: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type ListModelsFn = () => Promise<PiModel[]>;
|
||||||
|
|
||||||
interface Opts {
|
interface Opts {
|
||||||
spawnFn?: (
|
spawnFn?: (
|
||||||
command: string,
|
command: string,
|
||||||
@@ -36,7 +38,7 @@ interface Opts {
|
|||||||
* configured) via an ephemeral `pi --mode rpc` process. Result is cached for
|
* configured) via an ephemeral `pi --mode rpc` process. Result is cached for
|
||||||
* `ttlMs`. The returned function rejects on timeout/error; callers degrade.
|
* `ttlMs`. The returned function rejects on timeout/error; callers degrade.
|
||||||
*/
|
*/
|
||||||
export function createPiModelLister(cfg: AppConfig, opts: Opts = {}): () => Promise<PiModel[]> {
|
export function createPiModelLister(cfg: AppConfig, opts: Opts = {}): ListModelsFn {
|
||||||
const ttlMs = opts.ttlMs ?? 60_000;
|
const ttlMs = opts.ttlMs ?? 60_000;
|
||||||
const now = opts.nowMs ?? (() => Date.now());
|
const now = opts.nowMs ?? (() => Date.now());
|
||||||
const spawnFn = opts.spawnFn ?? nodeSpawn;
|
const spawnFn = opts.spawnFn ?? nodeSpawn;
|
||||||
|
|||||||
@@ -2,12 +2,11 @@ import { execFile as nodeExecFile } from "node:child_process";
|
|||||||
import { promisify } from "node:util";
|
import { promisify } from "node:util";
|
||||||
import type { AppConfig } from "../config.js";
|
import type { AppConfig } from "../config.js";
|
||||||
import { secretValue } from "../config/secret-bundle.js";
|
import { secretValue } from "../config/secret-bundle.js";
|
||||||
|
import { loadSettings, type Settings } from "../settings/settings-store.js";
|
||||||
import {
|
import {
|
||||||
loadSettings,
|
splitCanonicalModelId,
|
||||||
saveSettings,
|
type RuntimeModelCatalog,
|
||||||
type Settings,
|
} from "../models/runtime-model-catalog.js";
|
||||||
} from "../settings/settings-store.js";
|
|
||||||
import type { PiModel } from "./list-models.js";
|
|
||||||
import {
|
import {
|
||||||
configuredPiProviderApiKey,
|
configuredPiProviderApiKey,
|
||||||
PI_MANAGED_CONFIG_ERROR_MESSAGE,
|
PI_MANAGED_CONFIG_ERROR_MESSAGE,
|
||||||
@@ -45,13 +44,6 @@ export interface PiStatus {
|
|||||||
message?: string;
|
message?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface PiOptions {
|
|
||||||
providers: string[];
|
|
||||||
models: Array<{ provider: string; id: string }>;
|
|
||||||
reasoning: PiReasoning[];
|
|
||||||
checkedAt: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface PiTestResult {
|
export interface PiTestResult {
|
||||||
ready: boolean;
|
ready: boolean;
|
||||||
checkedAt: string;
|
checkedAt: string;
|
||||||
@@ -76,15 +68,13 @@ export type PiExecFile = (
|
|||||||
|
|
||||||
export interface PiManagementService {
|
export interface PiManagementService {
|
||||||
status(): Promise<PiStatus>;
|
status(): Promise<PiStatus>;
|
||||||
options(): Promise<PiOptions>;
|
|
||||||
configure(value: PiInstallationConfig): Promise<PiInstallationConfig & { updatedAt: string }>;
|
|
||||||
test(): Promise<PiTestResult>;
|
test(): Promise<PiTestResult>;
|
||||||
logs(): Promise<PiLogs>;
|
logs(): Promise<PiLogs>;
|
||||||
}
|
}
|
||||||
|
|
||||||
export class PiManagementError extends Error {
|
export class PiManagementError extends Error {
|
||||||
constructor(
|
constructor(
|
||||||
public readonly code: "pi_management_invalid_config" | "pi_management_unavailable" | "pi_management_write_failed",
|
public readonly code: "pi_management_unavailable",
|
||||||
message: string,
|
message: string,
|
||||||
) {
|
) {
|
||||||
super(message);
|
super(message);
|
||||||
@@ -93,10 +83,9 @@ export class PiManagementError extends Error {
|
|||||||
|
|
||||||
interface PiManagementDeps {
|
interface PiManagementDeps {
|
||||||
execute?: PiExecFile;
|
execute?: PiExecFile;
|
||||||
listModels: () => Promise<PiModel[]>;
|
modelCatalog: RuntimeModelCatalog;
|
||||||
smokeProvider?: PiProviderSmoke;
|
smokeProvider?: PiProviderSmoke;
|
||||||
readSettings?: () => Settings;
|
readSettings?: () => Settings;
|
||||||
saveSettings?: (settings: Settings) => Settings;
|
|
||||||
readLogs?: () => string | Promise<string>;
|
readLogs?: () => string | Promise<string>;
|
||||||
credentialStatus?: (provider: string | undefined) => PiCredentialStatus;
|
credentialStatus?: (provider: string | undefined) => PiCredentialStatus;
|
||||||
now?: () => Date;
|
now?: () => Date;
|
||||||
@@ -111,54 +100,36 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
|
|||||||
};
|
};
|
||||||
const execute = deps.execute ?? defaultExecFile;
|
const execute = deps.execute ?? defaultExecFile;
|
||||||
const readSettings = deps.readSettings ?? (() => loadSettings(config));
|
const readSettings = deps.readSettings ?? (() => loadSettings(config));
|
||||||
const persistSettings = deps.saveSettings ?? ((settings) => saveSettings(config, settings));
|
|
||||||
const readLogs = deps.readLogs ?? (() => diagnostics.join("\n"));
|
const readLogs = deps.readLogs ?? (() => diagnostics.join("\n"));
|
||||||
const smokeProvider = deps.smokeProvider ?? createPiProviderSmoke(config);
|
const smokeProvider = deps.smokeProvider ?? createPiProviderSmoke(config, {
|
||||||
|
modelCatalog: deps.modelCatalog,
|
||||||
|
});
|
||||||
const credentialStatus = deps.credentialStatus ?? ((provider: string | undefined) => {
|
const credentialStatus = deps.credentialStatus ?? ((provider: string | undefined) => {
|
||||||
try {
|
try {
|
||||||
|
const model = deps.modelCatalog.defaultSession
|
||||||
|
? deps.modelCatalog.sessionModels().find((entry) => entry.id === deps.modelCatalog.defaultSession)
|
||||||
|
: undefined;
|
||||||
|
const credentialName = model?.authentication.mode === "secret_env"
|
||||||
|
? model.authentication.apiKeyEnv
|
||||||
|
: undefined;
|
||||||
|
const configuredApiKey = configuredPiProviderApiKey(
|
||||||
|
readConfiguredPiAgentFile("models.json", true),
|
||||||
|
provider,
|
||||||
|
) ?? (credentialName ? `$${credentialName}` : undefined);
|
||||||
return piProviderCredentialStatus({
|
return piProviderCredentialStatus({
|
||||||
provider,
|
provider,
|
||||||
authProviders: loadPiAuthProviders(),
|
authProviders: loadPiAuthProviders(),
|
||||||
resolveCredentialValue: () => secretValue(config, "THT_MODEL_API_KEY"),
|
resolveCredentialValue: () => credentialName
|
||||||
|
? secretValue(config, credentialName)
|
||||||
|
: config.modelCatalogFile ? undefined : secretValue(config, "THT_MODEL_API_KEY"),
|
||||||
credentialFile: config.modelApiKeyFile,
|
credentialFile: config.modelApiKeyFile,
|
||||||
configuredApiKey: configuredPiProviderApiKey(
|
configuredApiKey,
|
||||||
readConfiguredPiAgentFile("models.json", true),
|
|
||||||
provider,
|
|
||||||
),
|
|
||||||
});
|
});
|
||||||
} catch {
|
} catch {
|
||||||
return "missing";
|
return "missing";
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
const closedOptions = async (): Promise<Omit<PiOptions, "checkedAt">> => {
|
|
||||||
let listed: PiModel[];
|
|
||||||
try {
|
|
||||||
listed = await deps.listModels();
|
|
||||||
} catch (error) {
|
|
||||||
if (isPiManagedConfigError(error)) {
|
|
||||||
throw new PiManagementError("pi_management_unavailable", PI_MANAGED_CONFIG_ERROR_MESSAGE);
|
|
||||||
}
|
|
||||||
throw new PiManagementError("pi_management_unavailable", "Pi model choices are unavailable");
|
|
||||||
}
|
|
||||||
const models: Array<{ provider: string; id: string }> = [];
|
|
||||||
const providers: string[] = [];
|
|
||||||
const seenModels = new Set<string>();
|
|
||||||
const seenProviders = new Set<string>();
|
|
||||||
for (const model of listed) {
|
|
||||||
if (!isChoice(model?.provider) || !isChoice(model?.id)) continue;
|
|
||||||
const key = `${model.provider}\u0000${model.id}`;
|
|
||||||
if (seenModels.has(key)) continue;
|
|
||||||
seenModels.add(key);
|
|
||||||
models.push({ provider: model.provider, id: model.id });
|
|
||||||
if (!seenProviders.has(model.provider)) {
|
|
||||||
seenProviders.add(model.provider);
|
|
||||||
providers.push(model.provider);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return { providers, models, reasoning: [...REASONING_CHOICES] };
|
|
||||||
};
|
|
||||||
|
|
||||||
const version = async (timeoutMs = config.piManagementTimeoutMs): Promise<string> => {
|
const version = async (timeoutMs = config.piManagementTimeoutMs): Promise<string> => {
|
||||||
let output: { stdout: string; stderr: string };
|
let output: { stdout: string; stderr: string };
|
||||||
try {
|
try {
|
||||||
@@ -180,12 +151,12 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
|
|||||||
|
|
||||||
const installationConfig = (): PiInstallationConfig => {
|
const installationConfig = (): PiInstallationConfig => {
|
||||||
const settings = readSettings();
|
const settings = readSettings();
|
||||||
const provider = config.defaults.provider ?? settings.provider;
|
|
||||||
const model = config.defaults.model ?? settings.model;
|
|
||||||
const reasoning = config.defaults.thinking ?? settings.thinking;
|
const reasoning = config.defaults.thinking ?? settings.thinking;
|
||||||
|
const selected = deps.modelCatalog.defaultSession
|
||||||
|
? splitCanonicalModelId(deps.modelCatalog.defaultSession)
|
||||||
|
: undefined;
|
||||||
return {
|
return {
|
||||||
...(isChoice(provider) ? { provider } : {}),
|
...(selected ? selected : {}),
|
||||||
...(isChoice(model) ? { model } : {}),
|
|
||||||
...(isReasoning(reasoning) ? { reasoning } : {}),
|
...(isReasoning(reasoning) ? { reasoning } : {}),
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
@@ -206,28 +177,6 @@ export function createPiManagement(config: AppConfig, deps: PiManagementDeps): P
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
async options(): Promise<PiOptions> {
|
|
||||||
const choices = await closedOptions();
|
|
||||||
return { ...choices, checkedAt: now().toISOString() };
|
|
||||||
},
|
|
||||||
|
|
||||||
async configure(value: PiInstallationConfig): Promise<PiInstallationConfig & { updatedAt: string }> {
|
|
||||||
if (!isInstallationConfig(value)) {
|
|
||||||
throw new PiManagementError("pi_management_invalid_config", "Pi installation configuration is invalid");
|
|
||||||
}
|
|
||||||
const choices = await closedOptions();
|
|
||||||
if (!choices.models.some((model) => model.provider === value.provider && model.id === value.model)) {
|
|
||||||
throw new PiManagementError("pi_management_invalid_config", "Pi provider and model must be selected from available choices");
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
persistSettings({ ...readSettings(), provider: value.provider, model: value.model, thinking: value.reasoning });
|
|
||||||
} catch {
|
|
||||||
throw new PiManagementError("pi_management_write_failed", "Pi installation configuration could not be saved");
|
|
||||||
}
|
|
||||||
addDiagnostic("Pi installation defaults updated");
|
|
||||||
return { ...value, updatedAt: now().toISOString() };
|
|
||||||
},
|
|
||||||
|
|
||||||
async test(): Promise<PiTestResult> {
|
async test(): Promise<PiTestResult> {
|
||||||
const checkedAt = now().toISOString();
|
const checkedAt = now().toISOString();
|
||||||
const deadline = Date.now() + config.piManagementTimeoutMs;
|
const deadline = Date.now() + config.piManagementTimeoutMs;
|
||||||
@@ -293,24 +242,10 @@ async function defaultExecFile(command: string, args: string[], options: PiExecF
|
|||||||
return { stdout: String(result.stdout), stderr: String(result.stderr) };
|
return { stdout: String(result.stdout), stderr: String(result.stderr) };
|
||||||
}
|
}
|
||||||
|
|
||||||
function isChoice(value: unknown): value is string {
|
|
||||||
return typeof value === "string" && value.length > 0 && value.length <= 128 && value.trim() === value
|
|
||||||
&& /^[A-Za-z0-9][A-Za-z0-9._/-]*$/u.test(value);
|
|
||||||
}
|
|
||||||
|
|
||||||
function isReasoning(value: unknown): value is PiReasoning {
|
function isReasoning(value: unknown): value is PiReasoning {
|
||||||
return typeof value === "string" && (REASONING_CHOICES as readonly string[]).includes(value);
|
return typeof value === "string" && (REASONING_CHOICES as readonly string[]).includes(value);
|
||||||
}
|
}
|
||||||
|
|
||||||
function isInstallationConfig(value: unknown): value is Required<PiInstallationConfig> {
|
|
||||||
if (!value || typeof value !== "object" || Array.isArray(value)) return false;
|
|
||||||
const candidate = value as Record<string, unknown>;
|
|
||||||
if (Object.keys(candidate).length !== 3 || Object.keys(candidate).some((key) => !["provider", "model", "reasoning"].includes(key))) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
return isChoice(candidate.provider) && isChoice(candidate.model) && isReasoning(candidate.reasoning);
|
|
||||||
}
|
|
||||||
|
|
||||||
function isTimeout(error: unknown): boolean {
|
function isTimeout(error: unknown): boolean {
|
||||||
return Boolean(
|
return Boolean(
|
||||||
error && typeof error === "object" && (
|
error && typeof error === "object" && (
|
||||||
|
|||||||
@@ -11,6 +11,10 @@ import {
|
|||||||
configuredPiProviderApiKey,
|
configuredPiProviderApiKey,
|
||||||
createPiRuntimeAgentSnapshot,
|
createPiRuntimeAgentSnapshot,
|
||||||
} from "./managed-config.js";
|
} from "./managed-config.js";
|
||||||
|
import {
|
||||||
|
loadRuntimeModelCatalog,
|
||||||
|
type RuntimeModelCatalog,
|
||||||
|
} from "../models/runtime-model-catalog.js";
|
||||||
|
|
||||||
export interface SessionRuntime {
|
export interface SessionRuntime {
|
||||||
rpc: RpcClient;
|
rpc: RpcClient;
|
||||||
@@ -42,23 +46,32 @@ export class PiProcessManager {
|
|||||||
private runtimes = new Map<string, SessionRuntime>();
|
private runtimes = new Map<string, SessionRuntime>();
|
||||||
private agentSnapshotCleanups = new WeakMap<ChildProcessWithoutNullStreams, () => void>();
|
private agentSnapshotCleanups = new WeakMap<ChildProcessWithoutNullStreams, () => void>();
|
||||||
private spawnFn: (
|
private spawnFn: (
|
||||||
sessionId: string, author: string, provider: string | undefined, principal?: PrincipalContext,
|
sessionId: string, author: string, provider: string | undefined, model: string | undefined,
|
||||||
runtimeConfigPath?: string,
|
principal?: PrincipalContext, runtimeConfigPath?: string,
|
||||||
) => ChildProcessWithoutNullStreams;
|
) => ChildProcessWithoutNullStreams;
|
||||||
private loadAuthProviders: (agentDir: string) => ReadonlySet<string>;
|
private loadAuthProviders: (agentDir: string) => ReadonlySet<string>;
|
||||||
|
private modelCatalog: RuntimeModelCatalog;
|
||||||
|
private modelCatalogConfigured: boolean;
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
private cfg: AppConfig,
|
private cfg: AppConfig,
|
||||||
opts?: { spawnFn?: SpawnFn; authProviders?: (agentDir: string) => ReadonlySet<string> },
|
opts?: {
|
||||||
|
spawnFn?: SpawnFn;
|
||||||
|
authProviders?: (agentDir: string) => ReadonlySet<string>;
|
||||||
|
modelCatalog?: RuntimeModelCatalog;
|
||||||
|
},
|
||||||
) {
|
) {
|
||||||
|
this.modelCatalog = opts?.modelCatalog ?? loadRuntimeModelCatalog(cfg.modelCatalogFile);
|
||||||
|
this.modelCatalogConfigured = cfg.modelCatalogFile !== undefined
|
||||||
|
|| this.modelCatalog.defaultSession !== null;
|
||||||
this.loadAuthProviders = opts?.authProviders
|
this.loadAuthProviders = opts?.authProviders
|
||||||
?? ((agentDir) => loadPiAuthProviders({ agentDir }));
|
?? ((agentDir) => loadPiAuthProviders({ agentDir }));
|
||||||
if (opts?.spawnFn) {
|
if (opts?.spawnFn) {
|
||||||
this.spawnFn = (sessionId, author, provider, principal, runtimeConfigPath) =>
|
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath) =>
|
||||||
this.spawnPi(opts.spawnFn!, sessionId, author, provider, principal, runtimeConfigPath);
|
this.spawnPi(opts.spawnFn!, sessionId, author, provider, model, principal, runtimeConfigPath);
|
||||||
} else {
|
} else {
|
||||||
this.spawnFn = (sessionId, author, provider, principal, runtimeConfigPath) =>
|
this.spawnFn = (sessionId, author, provider, model, principal, runtimeConfigPath) =>
|
||||||
this.spawnPi(nodeSpawn, sessionId, author, provider, principal, runtimeConfigPath);
|
this.spawnPi(nodeSpawn, sessionId, author, provider, model, principal, runtimeConfigPath);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -71,7 +84,7 @@ export class PiProcessManager {
|
|||||||
|
|
||||||
private spawnPi(
|
private spawnPi(
|
||||||
spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined,
|
spawnFn: SpawnFn, sessionId: string, author: string, provider: string | undefined,
|
||||||
principal?: PrincipalContext, runtimeConfigPath?: string,
|
model: string | undefined, principal?: PrincipalContext, runtimeConfigPath?: string,
|
||||||
): ChildProcessWithoutNullStreams {
|
): ChildProcessWithoutNullStreams {
|
||||||
// This is the final shared boundary for createFor(), spawnFor(), and resume(). Validate
|
// This is the final shared boundary for createFor(), spawnFor(), and resume(). Validate
|
||||||
// before auth-provider inspection, then make Pi consume the exact copied bytes rather than
|
// before auth-provider inspection, then make Pi consume the exact copied bytes rather than
|
||||||
@@ -79,12 +92,23 @@ export class PiProcessManager {
|
|||||||
const agent = createPiRuntimeAgentSnapshot();
|
const agent = createPiRuntimeAgentSnapshot();
|
||||||
let child: ChildProcessWithoutNullStreams | undefined;
|
let child: ChildProcessWithoutNullStreams | undefined;
|
||||||
try {
|
try {
|
||||||
|
const catalogModel = provider && model
|
||||||
|
? this.modelCatalog.sessionModels()
|
||||||
|
.find((entry) => entry.provider === provider && entry.model === model)
|
||||||
|
: undefined;
|
||||||
|
const credentialName = catalogModel?.authentication.mode === "secret_env"
|
||||||
|
? catalogModel.authentication.apiKeyEnv
|
||||||
|
: undefined;
|
||||||
|
const projectedApiKey = configuredPiProviderApiKey(agent.models, provider)
|
||||||
|
?? (credentialName ? `$${credentialName}` : undefined);
|
||||||
const env = buildPiChildEnv({
|
const env = buildPiChildEnv({
|
||||||
provider,
|
provider,
|
||||||
authProviders: this.loadAuthProviders(agent.agentDir),
|
authProviders: this.loadAuthProviders(agent.agentDir),
|
||||||
credentialValue: secretValue(this.cfg, "THT_MODEL_API_KEY"),
|
credentialValue: credentialName
|
||||||
|
? secretValue(this.cfg, credentialName)
|
||||||
|
: this.modelCatalogConfigured ? undefined : secretValue(this.cfg, "THT_MODEL_API_KEY"),
|
||||||
credentialFile: this.cfg.modelApiKeyFile,
|
credentialFile: this.cfg.modelApiKeyFile,
|
||||||
configuredApiKey: configuredPiProviderApiKey(agent.models, provider),
|
configuredApiKey: projectedApiKey,
|
||||||
additions: { THT_SESSION: sessionId, THT_AUTHOR: author },
|
additions: { THT_SESSION: sessionId, THT_AUTHOR: author },
|
||||||
});
|
});
|
||||||
env.PI_CODING_AGENT_DIR = agent.agentDir;
|
env.PI_CODING_AGENT_DIR = agent.agentDir;
|
||||||
@@ -162,9 +186,10 @@ export class PiProcessManager {
|
|||||||
}
|
}
|
||||||
const author = o.author ?? "dev@local";
|
const author = o.author ?? "dev@local";
|
||||||
const provider = canonicalPiProvider(o.provider ?? this.cfg.defaults.provider);
|
const provider = canonicalPiProvider(o.provider ?? this.cfg.defaults.provider);
|
||||||
|
const model = o.model ?? this.cfg.defaults.model;
|
||||||
let child: ChildProcessWithoutNullStreams;
|
let child: ChildProcessWithoutNullStreams;
|
||||||
try {
|
try {
|
||||||
child = this.spawnFn(sessionId, author, provider, o.principal, o.runtimeConfig?.path);
|
child = this.spawnFn(sessionId, author, provider, model, o.principal, o.runtimeConfig?.path);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
o.runtimeConfig?.release();
|
o.runtimeConfig?.release();
|
||||||
throw error;
|
throw error;
|
||||||
|
|||||||
@@ -17,6 +17,10 @@ import {
|
|||||||
readConfiguredPiAgentFile,
|
readConfiguredPiAgentFile,
|
||||||
validateDeclarativePiConfig,
|
validateDeclarativePiConfig,
|
||||||
} from "./managed-config.js";
|
} from "./managed-config.js";
|
||||||
|
import {
|
||||||
|
loadRuntimeModelCatalog,
|
||||||
|
type RuntimeModelCatalog,
|
||||||
|
} from "../models/runtime-model-catalog.js";
|
||||||
|
|
||||||
const SMOKE_PROMPT = "Provider health check. Reply with exactly OK.";
|
const SMOKE_PROMPT = "Provider health check. Reply with exactly OK.";
|
||||||
const SMOKE_ARGS = [
|
const SMOKE_ARGS = [
|
||||||
@@ -49,6 +53,7 @@ interface ProviderSmokeOptions {
|
|||||||
authProviders?: () => ReadonlySet<string>;
|
authProviders?: () => ReadonlySet<string>;
|
||||||
readAuthStore?: () => string;
|
readAuthStore?: () => string;
|
||||||
readModelsStore?: () => string | undefined;
|
readModelsStore?: () => string | undefined;
|
||||||
|
modelCatalog?: RuntimeModelCatalog;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function createPiProviderSmoke(
|
export function createPiProviderSmoke(
|
||||||
@@ -69,12 +74,24 @@ export function createPiProviderSmoke(
|
|||||||
const configuredModels = options.readModelsStore
|
const configuredModels = options.readModelsStore
|
||||||
? options.readModelsStore()
|
? options.readModelsStore()
|
||||||
: readConfiguredPiAgentFile("models.json", true);
|
: readConfiguredPiAgentFile("models.json", true);
|
||||||
|
const catalog = options.modelCatalog ?? loadRuntimeModelCatalog(config.modelCatalogFile);
|
||||||
|
const catalogConfigured = config.modelCatalogFile !== undefined
|
||||||
|
|| catalog.defaultSession !== null;
|
||||||
|
const catalogModel = catalog.sessionModels()
|
||||||
|
.find((entry) => entry.provider === canonicalProvider && entry.model === model);
|
||||||
|
const credentialName = catalogModel?.authentication.mode === "secret_env"
|
||||||
|
? catalogModel.authentication.apiKeyEnv
|
||||||
|
: undefined;
|
||||||
|
const projectedApiKey = configuredPiProviderApiKey(configuredModels, canonicalProvider)
|
||||||
|
?? (credentialName ? `$${credentialName}` : undefined);
|
||||||
const env = buildPiChildEnv({
|
const env = buildPiChildEnv({
|
||||||
provider: canonicalProvider,
|
provider: canonicalProvider,
|
||||||
authProviders: configuredAuthProviders,
|
authProviders: configuredAuthProviders,
|
||||||
credentialValue: secretValue(config, "THT_MODEL_API_KEY"),
|
credentialValue: credentialName
|
||||||
|
? secretValue(config, credentialName)
|
||||||
|
: catalogConfigured ? undefined : secretValue(config, "THT_MODEL_API_KEY"),
|
||||||
credentialFile: config.modelApiKeyFile,
|
credentialFile: config.modelApiKeyFile,
|
||||||
configuredApiKey: configuredPiProviderApiKey(configuredModels, canonicalProvider),
|
configuredApiKey: projectedApiKey,
|
||||||
});
|
});
|
||||||
clearPrincipalEnvironment(env);
|
clearPrincipalEnvironment(env);
|
||||||
delete env.THT_DATA_ROOT;
|
delete env.THT_DATA_ROOT;
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ import {
|
|||||||
} from "../catalog/types.js";
|
} from "../catalog/types.js";
|
||||||
|
|
||||||
const idSchema = z.uuid();
|
const idSchema = z.uuid();
|
||||||
const modelIdSchema = z.string().regex(/^[a-z][a-z0-9._-]{0,63}$/);
|
const modelIdSchema = z.string().regex(/^[a-z][a-z0-9._-]{0,63}\/[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$/);
|
||||||
const selectedTargetIdsSchema = z.array(idSchema).min(1);
|
const selectedTargetIdsSchema = z.array(idSchema).min(1);
|
||||||
const suggestionSchema = z.discriminatedUnion("scope", [
|
const suggestionSchema = z.discriminatedUnion("scope", [
|
||||||
z.object({ modelId: modelIdSchema, scope: z.literal("all") }).strict(),
|
z.object({ modelId: modelIdSchema, scope: z.literal("all") }).strict(),
|
||||||
|
|||||||
+10
-15
@@ -1,10 +1,8 @@
|
|||||||
import { readdirSync } from "node:fs";
|
import { readdirSync } from "node:fs";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import type { FastifyInstance } from "fastify";
|
import type { FastifyInstance } from "fastify";
|
||||||
import type { PiModel } from "../pi/list-models.js";
|
|
||||||
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||||
|
import type { RuntimeModelCatalog } from "../models/runtime-model-catalog.js";
|
||||||
export type ListModelsFn = () => Promise<PiModel[]>;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* List YAML workspace configs found in <harnessDir>/workspaces/*.yaml.
|
* List YAML workspace configs found in <harnessDir>/workspaces/*.yaml.
|
||||||
@@ -25,20 +23,17 @@ export function listWorkspaces(harnessDir: string): { name: string; file: string
|
|||||||
|
|
||||||
export function metaRoutes(
|
export function metaRoutes(
|
||||||
app: FastifyInstance,
|
app: FastifyInstance,
|
||||||
deps: { harnessDir: string; listModels?: ListModelsFn },
|
deps: { harnessDir: string; modelCatalog: RuntimeModelCatalog },
|
||||||
): void {
|
): void {
|
||||||
app.get("/models", async (request, reply) => {
|
app.get("/models", async (request, reply) => {
|
||||||
if (!isPrincipalContext(requirePermission(request, reply, "session.use"))) return reply;
|
if (!isPrincipalContext(requirePermission(request, reply, "session.use"))) return reply;
|
||||||
const fn = deps.listModels ?? (async () => []);
|
return {
|
||||||
try {
|
models: deps.modelCatalog.sessionModels().map((entry) => ({
|
||||||
return { models: await fn() };
|
provider: entry.provider,
|
||||||
} catch (error) {
|
id: entry.model,
|
||||||
app.log.warn({
|
name: entry.label,
|
||||||
component: "pi-model-list",
|
reasoning: entry.session?.reasoning ?? false,
|
||||||
errorType: error instanceof Error ? error.name : typeof error,
|
})),
|
||||||
}, "Pi model listing failed");
|
};
|
||||||
// Graceful fallback: Pi may not be running; don't crash the server.
|
|
||||||
return { models: [] as PiModel[] };
|
|
||||||
}
|
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,13 +11,6 @@ export function piManagementRoutes(
|
|||||||
deps: { service: PiManagementService },
|
deps: { service: PiManagementService },
|
||||||
): void {
|
): void {
|
||||||
app.get("/pi-management/status", async (request, reply) => run(request, reply, deps, () => deps.service.status()));
|
app.get("/pi-management/status", async (request, reply) => run(request, reply, deps, () => deps.service.status()));
|
||||||
app.get("/pi-management/options", async (request, reply) => run(request, reply, deps, () => deps.service.options()));
|
|
||||||
app.put("/pi-management/config", async (request, reply) => run(
|
|
||||||
request,
|
|
||||||
reply,
|
|
||||||
deps,
|
|
||||||
() => deps.service.configure((request.body ?? {}) as Record<string, unknown>),
|
|
||||||
));
|
|
||||||
app.post("/pi-management/test", async (request, reply) => run(request, reply, deps, () => deps.service.test()));
|
app.post("/pi-management/test", async (request, reply) => run(request, reply, deps, () => deps.service.test()));
|
||||||
app.get("/pi-management/logs", async (request, reply) => run(request, reply, deps, () => deps.service.logs()));
|
app.get("/pi-management/logs", async (request, reply) => run(request, reply, deps, () => deps.service.logs()));
|
||||||
}
|
}
|
||||||
@@ -38,8 +31,7 @@ async function run<T>(
|
|||||||
return await action();
|
return await action();
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
if (error instanceof PiManagementError) {
|
if (error instanceof PiManagementError) {
|
||||||
const statusCode = error.code === "pi_management_invalid_config" ? 400 : 503;
|
return reply.code(503).send({ code: error.code, error: error.message });
|
||||||
return reply.code(statusCode).send({ code: error.code, error: error.message });
|
|
||||||
}
|
}
|
||||||
return reply.code(503).send({ code: "pi_management_unavailable", error: "Pi management is unavailable" });
|
return reply.code(503).send({ code: "pi_management_unavailable", error: "Pi management is unavailable" });
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,12 +6,13 @@ import type { Settings } from "../settings/settings-store.js";
|
|||||||
import { getPrincipal } from "../auth/auth.js";
|
import { getPrincipal } from "../auth/auth.js";
|
||||||
import type { PrincipalContext } from "../auth/principal.js";
|
import type { PrincipalContext } from "../auth/principal.js";
|
||||||
import type { ReadinessManager } from "../runtime/readiness-manager.js";
|
import type { ReadinessManager } from "../runtime/readiness-manager.js";
|
||||||
import type { ListModelsFn } from "./meta.js";
|
import type { ListModelsFn } from "../pi/list-models.js";
|
||||||
import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
import type { WorkspaceRegistry } from "../workspaces/registry.js";
|
||||||
import { validateOperationalWorkspace, type WorkspaceDescriptor } from "../workspaces/schema.js";
|
import { validateOperationalWorkspace, type WorkspaceDescriptor } from "../workspaces/schema.js";
|
||||||
import type { MaintenanceBarrier } from "../runtime/maintenance-gate.js";
|
import type { MaintenanceBarrier } from "../runtime/maintenance-gate.js";
|
||||||
import { hasPermission, isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
import { hasPermission, isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||||
import type { EffectiveRelationshipSnapshotProvider } from "../catalog/effective-relationship-snapshot.js";
|
import type { EffectiveRelationshipSnapshotProvider } from "../catalog/effective-relationship-snapshot.js";
|
||||||
|
import { splitCanonicalModelId, type RuntimeModelCatalog } from "../models/runtime-model-catalog.js";
|
||||||
|
|
||||||
const BOOTSTRAP_FAILURE_MESSAGE =
|
const BOOTSTRAP_FAILURE_MESSAGE =
|
||||||
"Session startup failed. Check configuration and connectivity, then Resume the session.";
|
"Session startup failed. Check configuration and connectivity, then Resume the session.";
|
||||||
@@ -43,6 +44,7 @@ export function sessionRoutes(
|
|||||||
maintenanceBarrier: MaintenanceBarrier;
|
maintenanceBarrier: MaintenanceBarrier;
|
||||||
/** Optional only for narrow route-test stubs and installations without a Catalog database. */
|
/** Optional only for narrow route-test stubs and installations without a Catalog database. */
|
||||||
effectiveRelationships?: EffectiveRelationshipSnapshotProvider;
|
effectiveRelationships?: EffectiveRelationshipSnapshotProvider;
|
||||||
|
modelCatalog: RuntimeModelCatalog;
|
||||||
},
|
},
|
||||||
) {
|
) {
|
||||||
const lifecycleTails = new Map<string, Promise<void>>();
|
const lifecycleTails = new Map<string, Promise<void>>();
|
||||||
@@ -358,7 +360,6 @@ export function sessionRoutes(
|
|||||||
let workspaceId: string | undefined;
|
let workspaceId: string | undefined;
|
||||||
let workspaceRevision: string | undefined;
|
let workspaceRevision: string | undefined;
|
||||||
let workspaceDescriptor: WorkspaceDescriptor | undefined;
|
let workspaceDescriptor: WorkspaceDescriptor | undefined;
|
||||||
let allowedModels: readonly string[] | undefined;
|
|
||||||
if (requestedWorkspaceId) {
|
if (requestedWorkspaceId) {
|
||||||
try {
|
try {
|
||||||
const registry = d.workspaceRegistry as Partial<WorkspaceRegistry>;
|
const registry = d.workspaceRegistry as Partial<WorkspaceRegistry>;
|
||||||
@@ -378,7 +379,6 @@ export function sessionRoutes(
|
|||||||
workspaceId = resolved.revision.id;
|
workspaceId = resolved.revision.id;
|
||||||
workspaceRevision = resolved.revision.commit;
|
workspaceRevision = resolved.revision.commit;
|
||||||
workspaceDescriptor = resolved.workspace;
|
workspaceDescriptor = resolved.workspace;
|
||||||
allowedModels = resolved.workspace.llm_policy.allowed;
|
|
||||||
} catch {
|
} catch {
|
||||||
return reply.code(409).send({
|
return reply.code(409).send({
|
||||||
error: WORKSPACE_REVISION_UNAVAILABLE_MESSAGE,
|
error: WORKSPACE_REVISION_UNAVAILABLE_MESSAGE,
|
||||||
@@ -386,12 +386,17 @@ export function sessionRoutes(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const provider = b.provider ?? s.provider;
|
const requestedCanonical = b.provider && b.model ? `${b.provider}/${b.model}` : undefined;
|
||||||
const model = b.model ?? s.model;
|
let selectedCanonical = requestedCanonical ?? d.modelCatalog.defaultSession;
|
||||||
|
let modelWarning: string | undefined;
|
||||||
|
if (selectedCanonical && d.modelCatalog.defaultSession && !d.modelCatalog.hasSession(selectedCanonical)) {
|
||||||
|
selectedCanonical = d.modelCatalog.defaultSession;
|
||||||
|
modelWarning = `Configured model ${requestedCanonical ?? "selection"} is unavailable; using ${selectedCanonical}.`;
|
||||||
|
}
|
||||||
|
const selected = selectedCanonical ? splitCanonicalModelId(selectedCanonical) : undefined;
|
||||||
|
const provider = selected?.provider ?? b.provider;
|
||||||
|
const model = selected?.model ?? b.model;
|
||||||
const thinking = b.thinking ?? s.thinking;
|
const thinking = b.thinking ?? s.thinking;
|
||||||
if (allowedModels && provider && model && !allowedModels.includes(`${provider}/${model}`)) {
|
|
||||||
return reply.code(400).send({ error: "Selected model is not allowed by this workspace." });
|
|
||||||
}
|
|
||||||
// A persisted session is resumable without keeping Pi alive. New work replaces every
|
// A persisted session is resumable without keeping Pi alive. New work replaces every
|
||||||
// runtime owned by this principal, while runtimes belonging to other users remain intact.
|
// runtime owned by this principal, while runtimes belonging to other users remain intact.
|
||||||
// Optional chaining preserves the deliberately narrow manager stubs used by route tests.
|
// Optional chaining preserves the deliberately narrow manager stubs used by route tests.
|
||||||
@@ -490,7 +495,7 @@ export function sessionRoutes(
|
|||||||
),
|
),
|
||||||
() => d.mgr.start(id, rt, runtimeOptions),
|
() => d.mgr.start(id, rt, runtimeOptions),
|
||||||
);
|
);
|
||||||
return { id };
|
return { id, ...(modelWarning ? { warning: modelWarning } : {}) };
|
||||||
} finally {
|
} finally {
|
||||||
if (revisionLease && !manifestPersisted) {
|
if (revisionLease && !manifestPersisted) {
|
||||||
await revisionLease.abort().catch((error: unknown) => {
|
await revisionLease.abort().catch((error: unknown) => {
|
||||||
@@ -598,6 +603,10 @@ export function sessionRoutes(
|
|||||||
provider?: string; model?: string; thinking?: string;
|
provider?: string; model?: string; thinking?: string;
|
||||||
workspace_id?: string; workspace_revision?: string;
|
workspace_id?: string; workspace_revision?: string;
|
||||||
};
|
};
|
||||||
|
const savedCanonical = saved.provider && saved.model ? `${saved.provider}/${saved.model}` : "";
|
||||||
|
if (d.modelCatalog.defaultSession && (!savedCanonical || !d.modelCatalog.hasSession(savedCanonical))) {
|
||||||
|
return reply.code(503).send({ error: MODEL_UNAVAILABLE_MESSAGE, code: "model_unavailable" });
|
||||||
|
}
|
||||||
let workspaceConfigPath: string;
|
let workspaceConfigPath: string;
|
||||||
let workspaceDescriptor: WorkspaceDescriptor | undefined;
|
let workspaceDescriptor: WorkspaceDescriptor | undefined;
|
||||||
try {
|
try {
|
||||||
|
|||||||
@@ -1,17 +1,27 @@
|
|||||||
import type { FastifyInstance } from "fastify";
|
import type { FastifyInstance } from "fastify";
|
||||||
import type { AppConfig } from "../config.js";
|
import type { AppConfig } from "../config.js";
|
||||||
import type { Settings } from "../settings/settings-store.js";
|
import type { Settings } from "../settings/settings-store.js";
|
||||||
import { listWorkspaces, type ListModelsFn } from "./meta.js";
|
import { listWorkspaces } from "./meta.js";
|
||||||
import type { PrincipalContext } from "../auth/principal.js";
|
import type { PrincipalContext } from "../auth/principal.js";
|
||||||
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
import { isPrincipalContext, requirePermission } from "../auth/authorization.js";
|
||||||
|
import {
|
||||||
|
splitCanonicalModelId,
|
||||||
|
type RuntimeModelCatalog,
|
||||||
|
} from "../models/runtime-model-catalog.js";
|
||||||
|
|
||||||
/** Merge stored settings over env/first-workspace defaults. */
|
/** Merge only workspace and runtime-thinking preferences; model defaults belong to modelCatalog. */
|
||||||
export function effectiveSettings(cfg: AppConfig, stored: Settings): Settings {
|
export function effectiveSettings(
|
||||||
|
cfg: AppConfig,
|
||||||
|
stored: Settings,
|
||||||
|
modelCatalog?: RuntimeModelCatalog,
|
||||||
|
): Settings {
|
||||||
const workspaces = listWorkspaces(cfg.harnessDir);
|
const workspaces = listWorkspaces(cfg.harnessDir);
|
||||||
|
const selected = modelCatalog?.defaultSession
|
||||||
|
? splitCanonicalModelId(modelCatalog.defaultSession)
|
||||||
|
: undefined;
|
||||||
return {
|
return {
|
||||||
workspace: stored.workspace ?? workspaces[0]?.name,
|
workspace: stored.workspace ?? workspaces[0]?.name,
|
||||||
provider: cfg.defaults.provider ?? stored.provider,
|
...(selected ?? {}),
|
||||||
model: cfg.defaults.model ?? stored.model,
|
|
||||||
thinking: cfg.defaults.thinking ?? stored.thinking,
|
thinking: cfg.defaults.thinking ?? stored.thinking,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
@@ -19,7 +29,7 @@ export function effectiveSettings(cfg: AppConfig, stored: Settings): Settings {
|
|||||||
export function settingsRoutes(
|
export function settingsRoutes(
|
||||||
app: FastifyInstance,
|
app: FastifyInstance,
|
||||||
deps: {
|
deps: {
|
||||||
cfg: AppConfig; listModels: ListModelsFn;
|
cfg: AppConfig;
|
||||||
getSettings: (principal: PrincipalContext) => Promise<Settings>;
|
getSettings: (principal: PrincipalContext) => Promise<Settings>;
|
||||||
},
|
},
|
||||||
): void {
|
): void {
|
||||||
@@ -37,22 +47,6 @@ export function settingsRoutes(
|
|||||||
const principal = requirePermission(req, reply, "settings.manage");
|
const principal = requirePermission(req, reply, "settings.manage");
|
||||||
if (!isPrincipalContext(principal)) return principal;
|
if (!isPrincipalContext(principal)) return principal;
|
||||||
const b = (req.body ?? {}) as Settings;
|
const b = (req.body ?? {}) as Settings;
|
||||||
if (b.model) {
|
|
||||||
let available: { provider: string; id: string }[] = [];
|
|
||||||
try {
|
|
||||||
available = await deps.listModels();
|
|
||||||
} catch {
|
|
||||||
available = [];
|
|
||||||
}
|
|
||||||
// Only validate when Pi gave us a non-empty list; otherwise allow (degraded).
|
|
||||||
if (available.length > 0 && !available.some(
|
|
||||||
(candidate) => candidate.provider === b.provider && candidate.id === b.model,
|
|
||||||
)) {
|
|
||||||
return reply.code(400).send({
|
|
||||||
error: `Unknown model: ${b.provider ?? "unknown"}/${b.model}`,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
try {
|
try {
|
||||||
// Retain this endpoint as a validating compatibility surface for older clients, but do
|
// Retain this endpoint as a validating compatibility surface for older clients, but do
|
||||||
// not write anonymous users' choices to shared server storage.
|
// not write anonymous users' choices to shared server storage.
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import type { AppConfig } from "../config.js";
|
|||||||
|
|
||||||
export interface Settings {
|
export interface Settings {
|
||||||
workspace?: string;
|
workspace?: string;
|
||||||
|
/** Legacy input fields are ignored when loading/evaluating installation settings. */
|
||||||
provider?: string;
|
provider?: string;
|
||||||
model?: string;
|
model?: string;
|
||||||
thinking?: string;
|
thinking?: string;
|
||||||
@@ -37,7 +38,13 @@ export function loadSettings(cfg: AppConfig): Settings {
|
|||||||
try {
|
try {
|
||||||
const raw = readFileSync(cfg.settingsFile, "utf8");
|
const raw = readFileSync(cfg.settingsFile, "utf8");
|
||||||
const parsed = JSON.parse(raw);
|
const parsed = JSON.parse(raw);
|
||||||
if (parsed && typeof parsed === "object") return parsed as Settings;
|
if (parsed && typeof parsed === "object") {
|
||||||
|
const value = parsed as Record<string, unknown>;
|
||||||
|
return {
|
||||||
|
...(typeof value.workspace === "string" ? { workspace: value.workspace } : {}),
|
||||||
|
...(typeof value.thinking === "string" ? { thinking: value.thinking } : {}),
|
||||||
|
};
|
||||||
|
}
|
||||||
return {};
|
return {};
|
||||||
} catch {
|
} catch {
|
||||||
return {};
|
return {};
|
||||||
|
|||||||
@@ -598,7 +598,11 @@ export class ThtRunner {
|
|||||||
} catch {
|
} catch {
|
||||||
return { ok: false, code: "workspace_not_activatable" };
|
return { ok: false, code: "workspace_not_activatable" };
|
||||||
}
|
}
|
||||||
const collection = descriptor.semantic_index.vector_store;
|
const collection = {
|
||||||
|
collection: descriptor.workspace.id,
|
||||||
|
dimensions: this.cfg.semanticRuntime.internalEmbeddingDimensions,
|
||||||
|
distance: "cosine" as const,
|
||||||
|
};
|
||||||
const controller = new AbortController();
|
const controller = new AbortController();
|
||||||
const timer = setTimeout(() => controller.abort(), Math.max(1, timeoutSec) * 1000);
|
const timer = setTimeout(() => controller.abort(), Math.max(1, timeoutSec) * 1000);
|
||||||
try {
|
try {
|
||||||
|
|||||||
@@ -238,6 +238,7 @@ function createProductionService(): WorkspacePreprocessingService {
|
|||||||
});
|
});
|
||||||
return new WorkspacePreprocessingService({
|
return new WorkspacePreprocessingService({
|
||||||
dataRoot: config.dataRoot ?? "/data",
|
dataRoot: config.dataRoot ?? "/data",
|
||||||
|
embeddingDimensions: config.internalEmbeddingDimensions,
|
||||||
httpPrivateHostAllowlist: (process.env.THT_EVIDENCE_PRIVATE_HOST_ALLOWLIST ?? "")
|
httpPrivateHostAllowlist: (process.env.THT_EVIDENCE_PRIVATE_HOST_ALLOWLIST ?? "")
|
||||||
.split(",").map((value) => value.trim()).filter((value) => value.length > 0),
|
.split(",").map((value) => value.trim()).filter((value) => value.length > 0),
|
||||||
acquireActiveRuntime: async (workspaceId) => {
|
acquireActiveRuntime: async (workspaceId) => {
|
||||||
|
|||||||
@@ -59,12 +59,12 @@ function safeSecretFilePath(path: string, secretRoots: readonly string[]): strin
|
|||||||
|
|
||||||
function requireSupportedDescriptor(workspace: unknown): void {
|
function requireSupportedDescriptor(workspace: unknown): void {
|
||||||
if (typeof workspace !== "object" || workspace === null) {
|
if (typeof workspace !== "object" || workspace === null) {
|
||||||
throw new Error("Workspace bindings support only workspace schema version 3");
|
throw new Error("Workspace bindings support only workspace schema version 4");
|
||||||
}
|
}
|
||||||
const metadata = Reflect.get(workspace, "workspace");
|
const metadata = Reflect.get(workspace, "workspace");
|
||||||
if (typeof metadata !== "object" || metadata === null
|
if (typeof metadata !== "object" || metadata === null
|
||||||
|| Reflect.get(metadata, "schema_version") !== 3) {
|
|| Reflect.get(metadata, "schema_version") !== 4) {
|
||||||
throw new Error("Workspace bindings support only workspace schema version 3");
|
throw new Error("Workspace bindings support only workspace schema version 4");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -155,7 +155,7 @@ export function resolveEvidenceBinding(
|
|||||||
return { values, missing };
|
return { values, missing };
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Resolve the complete schema-v3 runtime binding set. */
|
/** Resolve the complete schema-v4 runtime binding set. */
|
||||||
export function resolveRuntimeBindings(
|
export function resolveRuntimeBindings(
|
||||||
workspace: WorkspaceDescriptor,
|
workspace: WorkspaceDescriptor,
|
||||||
env: NodeJS.ProcessEnv,
|
env: NodeJS.ProcessEnv,
|
||||||
|
|||||||
@@ -137,12 +137,12 @@ function evidenceVariables(
|
|||||||
|
|
||||||
function requireSupportedDescriptor(workspace: unknown): void {
|
function requireSupportedDescriptor(workspace: unknown): void {
|
||||||
if (typeof workspace !== "object" || workspace === null) {
|
if (typeof workspace !== "object" || workspace === null) {
|
||||||
throw new Error("Installation contract supports only workspace schema version 3");
|
throw new Error("Installation contract supports only workspace schema version 4");
|
||||||
}
|
}
|
||||||
const metadata = Reflect.get(workspace, "workspace");
|
const metadata = Reflect.get(workspace, "workspace");
|
||||||
if (typeof metadata !== "object" || metadata === null
|
if (typeof metadata !== "object" || metadata === null
|
||||||
|| Reflect.get(metadata, "schema_version") !== 3) {
|
|| Reflect.get(metadata, "schema_version") !== 4) {
|
||||||
throw new Error("Installation contract supports only workspace schema version 3");
|
throw new Error("Installation contract supports only workspace schema version 4");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -406,12 +406,12 @@ function numericBinding(binding: Record<string, string>, name: string): number |
|
|||||||
|
|
||||||
function requireSupportedDescriptor(workspace: unknown): void {
|
function requireSupportedDescriptor(workspace: unknown): void {
|
||||||
if (typeof workspace !== "object" || workspace === null) {
|
if (typeof workspace !== "object" || workspace === null) {
|
||||||
throw new Error("Workspace diagnoser supports only workspace schema version 3");
|
throw new Error("Workspace diagnoser supports only workspace schema version 4");
|
||||||
}
|
}
|
||||||
const metadata = Reflect.get(workspace, "workspace");
|
const metadata = Reflect.get(workspace, "workspace");
|
||||||
if (typeof metadata !== "object" || metadata === null
|
if (typeof metadata !== "object" || metadata === null
|
||||||
|| Reflect.get(metadata, "schema_version") !== 3) {
|
|| Reflect.get(metadata, "schema_version") !== 4) {
|
||||||
throw new Error("Workspace diagnoser supports only workspace schema version 3");
|
throw new Error("Workspace diagnoser supports only workspace schema version 4");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -506,11 +506,15 @@ async function diagnoseValidatedWorkspace(
|
|||||||
try {
|
try {
|
||||||
const vector = await withTimeout(timeoutMs, (signal) => adapters.inspectQdrant({
|
const vector = await withTimeout(timeoutMs, (signal) => adapters.inspectQdrant({
|
||||||
baseUrl: semanticRuntime.internalQdrantUrl,
|
baseUrl: semanticRuntime.internalQdrantUrl,
|
||||||
collection: descriptor.semantic_index.vector_store.collection,
|
collection: descriptor.workspace.id,
|
||||||
timeoutMs,
|
timeoutMs,
|
||||||
signal,
|
signal,
|
||||||
}));
|
}));
|
||||||
const expected = descriptor.semantic_index.vector_store;
|
const expected = {
|
||||||
|
collection: descriptor.workspace.id,
|
||||||
|
dimensions: semanticRuntime.internalEmbeddingDimensions,
|
||||||
|
distance: "cosine",
|
||||||
|
};
|
||||||
if (vector.collection !== expected.collection
|
if (vector.collection !== expected.collection
|
||||||
|| vector.dimensions !== expected.dimensions
|
|| vector.dimensions !== expected.dimensions
|
||||||
|| vector.distance !== expected.distance) {
|
|| vector.distance !== expected.distance) {
|
||||||
@@ -529,10 +533,8 @@ async function diagnoseValidatedWorkspace(
|
|||||||
timeoutMs,
|
timeoutMs,
|
||||||
signal,
|
signal,
|
||||||
}));
|
}));
|
||||||
if (semanticRuntime.internalEmbeddingModel !== descriptor.semantic_index.embedding.model
|
if (!embedding.available
|
||||||
|| semanticRuntime.internalEmbeddingDimensions !== descriptor.semantic_index.embedding.dimensions
|
|| embedding.dimensions !== semanticRuntime.internalEmbeddingDimensions) {
|
||||||
|| !embedding.available
|
|
||||||
|| embedding.dimensions !== descriptor.semantic_index.embedding.dimensions) {
|
|
||||||
diagnostics.push(diagnosticError("semantic_index_incompatible"));
|
diagnostics.push(diagnosticError("semantic_index_incompatible"));
|
||||||
activatable = false;
|
activatable = false;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ import { createHash } from "node:crypto";
|
|||||||
import { normalize } from "node:path";
|
import { normalize } from "node:path";
|
||||||
|
|
||||||
export interface CanonicalEffectiveConfig {
|
export interface CanonicalEffectiveConfig {
|
||||||
schemaVersion: 1;
|
schemaVersion: 2;
|
||||||
dwh: CanonicalDwhConfig;
|
dwh: CanonicalDwhConfig;
|
||||||
vector: CanonicalVectorConfig;
|
vector: CanonicalVectorConfig;
|
||||||
embedding: CanonicalEmbeddingConfig;
|
embedding: CanonicalEmbeddingConfig;
|
||||||
@@ -27,6 +27,7 @@ export interface CanonicalVectorConfig {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export interface CanonicalEmbeddingConfig {
|
export interface CanonicalEmbeddingConfig {
|
||||||
|
id: string;
|
||||||
model: string;
|
model: string;
|
||||||
dimensions: number;
|
dimensions: number;
|
||||||
}
|
}
|
||||||
@@ -137,8 +138,10 @@ function buildEmbeddingConfig(rendered: Record<string, unknown>): CanonicalEmbed
|
|||||||
if (!embeddings) {
|
if (!embeddings) {
|
||||||
throw new TypeError("effective config is missing embedding resources");
|
throw new TypeError("effective config is missing embedding resources");
|
||||||
}
|
}
|
||||||
|
const model = requireString(embeddings, "model");
|
||||||
return {
|
return {
|
||||||
model: requireString(embeddings, "model"),
|
id: `ollama/${model}`,
|
||||||
|
model,
|
||||||
dimensions: requireNumber(embeddings, "dimensions"),
|
dimensions: requireNumber(embeddings, "dimensions"),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
@@ -166,7 +169,7 @@ export function buildCanonicalEffectiveConfig(renderedConfig: unknown): Canonica
|
|||||||
throw new TypeError("effective config requires a rendered configuration object");
|
throw new TypeError("effective config requires a rendered configuration object");
|
||||||
}
|
}
|
||||||
return {
|
return {
|
||||||
schemaVersion: 1,
|
schemaVersion: 2,
|
||||||
dwh: buildDwhConfig(rendered),
|
dwh: buildDwhConfig(rendered),
|
||||||
vector: buildVectorConfig(rendered),
|
vector: buildVectorConfig(rendered),
|
||||||
embedding: buildEmbeddingConfig(rendered),
|
embedding: buildEmbeddingConfig(rendered),
|
||||||
|
|||||||
@@ -77,6 +77,7 @@ export interface WorkspacePreprocessingServiceDeps {
|
|||||||
{ ok: true } | { ok: false; code: "workspace_not_activatable" | "semantic_index_incompatible" }
|
{ ok: true } | { ok: false; code: "workspace_not_activatable" | "semantic_index_incompatible" }
|
||||||
>;
|
>;
|
||||||
httpPrivateHostAllowlist?: readonly string[];
|
httpPrivateHostAllowlist?: readonly string[];
|
||||||
|
embeddingDimensions?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface RunScope {
|
interface RunScope {
|
||||||
@@ -118,7 +119,7 @@ export class WorkspacePreprocessingService {
|
|||||||
|
|
||||||
async vectorInspect(options: { workspaceId: string }): Promise<WorkspaceOperationResult> {
|
async vectorInspect(options: { workspaceId: string }): Promise<WorkspaceOperationResult> {
|
||||||
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
|
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
|
||||||
const collection = runtime.workspace.semantic_index.vector_store.collection;
|
const collection = runtime.workspace.workspace.id;
|
||||||
const res = await fetch(`${runtime.configLease.semanticQdrantUrl}/collections/${encodeURIComponent(collection)}`, { method: "GET" });
|
const res = await fetch(`${runtime.configLease.semanticQdrantUrl}/collections/${encodeURIComponent(collection)}`, { method: "GET" });
|
||||||
if (!res.ok) return baseResult(runtime, "vector inspect", "failed", "semantic_index_incompatible", { warnings: ["collection unavailable"] });
|
if (!res.ok) return baseResult(runtime, "vector inspect", "failed", "semantic_index_incompatible", { warnings: ["collection unavailable"] });
|
||||||
const body = await res.json() as any;
|
const body = await res.json() as any;
|
||||||
@@ -132,7 +133,7 @@ export class WorkspacePreprocessingService {
|
|||||||
|
|
||||||
async vectorRebuild(options: { workspaceId: string; collection?: string; confirm?: string; destroy?: boolean }): Promise<WorkspaceOperationResult> {
|
async vectorRebuild(options: { workspaceId: string; collection?: string; confirm?: string; destroy?: boolean }): Promise<WorkspaceOperationResult> {
|
||||||
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
|
const runtime = await this.deps.acquireActiveRuntime(options.workspaceId);
|
||||||
const collection = runtime.workspace.semantic_index.vector_store.collection;
|
const collection = runtime.workspace.workspace.id;
|
||||||
if (options.collection !== collection || options.confirm !== collection || options.destroy !== true) {
|
if (options.collection !== collection || options.confirm !== collection || options.destroy !== true) {
|
||||||
return baseResult(runtime, "vector rebuild", "failed", "semantic_index_incompatible", { warnings: ["rebuild requires exact confirmation and --destroy"] });
|
return baseResult(runtime, "vector rebuild", "failed", "semantic_index_incompatible", { warnings: ["rebuild requires exact confirmation and --destroy"] });
|
||||||
}
|
}
|
||||||
@@ -143,8 +144,8 @@ export class WorkspacePreprocessingService {
|
|||||||
const recreated = await reconcileCollection({
|
const recreated = await reconcileCollection({
|
||||||
baseUrl: runtime.configLease.semanticQdrantUrl,
|
baseUrl: runtime.configLease.semanticQdrantUrl,
|
||||||
collection,
|
collection,
|
||||||
dimensions: runtime.workspace.semantic_index.vector_store.dimensions,
|
dimensions: this.deps.embeddingDimensions ?? 1024,
|
||||||
distance: runtime.workspace.semantic_index.vector_store.distance,
|
distance: "cosine",
|
||||||
mode: "self_heal",
|
mode: "self_heal",
|
||||||
});
|
});
|
||||||
if (!recreated.ok) return baseResult(runtime, "vector rebuild", "failed", "semantic_index_incompatible", { warnings: ["collection recreate failed"] });
|
if (!recreated.ok) return baseResult(runtime, "vector rebuild", "failed", "semantic_index_incompatible", { warnings: ["collection recreate failed"] });
|
||||||
@@ -400,6 +401,8 @@ export class WorkspacePreprocessingService {
|
|||||||
catalogBlob: runtime.catalogBlob,
|
catalogBlob: runtime.catalogBlob,
|
||||||
configDigest: runtime.configLease.configDigest,
|
configDigest: runtime.configLease.configDigest,
|
||||||
bindingDigest: runtime.configLease.bindingDigest,
|
bindingDigest: runtime.configLease.bindingDigest,
|
||||||
|
embeddingId: runtime.configLease.effectiveConfig.embedding.id,
|
||||||
|
embeddingDimensions: runtime.configLease.effectiveConfig.embedding.dimensions,
|
||||||
});
|
});
|
||||||
return { runtime, state, job };
|
return { runtime, state, job };
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -44,11 +44,13 @@ export interface BeginPreprocessingJobOptions {
|
|||||||
catalogBlob: string;
|
catalogBlob: string;
|
||||||
configDigest: string;
|
configDigest: string;
|
||||||
bindingDigest: string;
|
bindingDigest: string;
|
||||||
|
embeddingId: string;
|
||||||
|
embeddingDimensions: number;
|
||||||
runId?: string;
|
runId?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface PreprocessingJobState {
|
export interface PreprocessingJobState {
|
||||||
schemaVersion: 1;
|
schemaVersion: 2;
|
||||||
runId: string;
|
runId: string;
|
||||||
operation: string;
|
operation: string;
|
||||||
workspaceId: string;
|
workspaceId: string;
|
||||||
@@ -57,6 +59,8 @@ export interface PreprocessingJobState {
|
|||||||
catalogBlob: string;
|
catalogBlob: string;
|
||||||
configDigest: string;
|
configDigest: string;
|
||||||
bindingDigest: string;
|
bindingDigest: string;
|
||||||
|
embeddingId: string;
|
||||||
|
embeddingDimensions: number;
|
||||||
completedStages: string[];
|
completedStages: string[];
|
||||||
childRuns: Record<string, string>;
|
childRuns: Record<string, string>;
|
||||||
status: "active" | "succeeded" | "blocked" | "failed";
|
status: "active" | "succeeded" | "blocked" | "failed";
|
||||||
@@ -146,7 +150,7 @@ function decodeJob(value: unknown): PreprocessingJobState {
|
|||||||
}
|
}
|
||||||
const record = value as Record<string, unknown>;
|
const record = value as Record<string, unknown>;
|
||||||
if (
|
if (
|
||||||
record.schemaVersion !== 1
|
record.schemaVersion !== 2
|
||||||
|| typeof record.runId !== "string"
|
|| typeof record.runId !== "string"
|
||||||
|| typeof record.operation !== "string"
|
|| typeof record.operation !== "string"
|
||||||
|| typeof record.workspaceId !== "string"
|
|| typeof record.workspaceId !== "string"
|
||||||
@@ -155,6 +159,8 @@ function decodeJob(value: unknown): PreprocessingJobState {
|
|||||||
|| typeof record.catalogBlob !== "string"
|
|| typeof record.catalogBlob !== "string"
|
||||||
|| typeof record.configDigest !== "string"
|
|| typeof record.configDigest !== "string"
|
||||||
|| typeof record.bindingDigest !== "string"
|
|| typeof record.bindingDigest !== "string"
|
||||||
|
|| typeof record.embeddingId !== "string"
|
||||||
|
|| typeof record.embeddingDimensions !== "number"
|
||||||
|| !Array.isArray(record.completedStages)
|
|| !Array.isArray(record.completedStages)
|
||||||
|| typeof record.childRuns !== "object" || record.childRuns === null || Array.isArray(record.childRuns)
|
|| typeof record.childRuns !== "object" || record.childRuns === null || Array.isArray(record.childRuns)
|
||||||
|| !["active", "succeeded", "blocked", "failed"].includes(String(record.status))
|
|| !["active", "succeeded", "blocked", "failed"].includes(String(record.status))
|
||||||
@@ -275,6 +281,8 @@ export class PreprocessingStateStore {
|
|||||||
|| existing.catalogBlob !== options.catalogBlob
|
|| existing.catalogBlob !== options.catalogBlob
|
||||||
|| existing.configDigest !== options.configDigest
|
|| existing.configDigest !== options.configDigest
|
||||||
|| existing.bindingDigest !== options.bindingDigest
|
|| existing.bindingDigest !== options.bindingDigest
|
||||||
|
|| existing.embeddingId !== options.embeddingId
|
||||||
|
|| existing.embeddingDimensions !== options.embeddingDimensions
|
||||||
) {
|
) {
|
||||||
throw new PreprocessingStateError(
|
throw new PreprocessingStateError(
|
||||||
"preprocessing_resume_mismatch",
|
"preprocessing_resume_mismatch",
|
||||||
@@ -295,7 +303,7 @@ export class PreprocessingStateStore {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
const job: PreprocessingJobState = {
|
const job: PreprocessingJobState = {
|
||||||
schemaVersion: 1,
|
schemaVersion: 2,
|
||||||
runId,
|
runId,
|
||||||
operation: options.operation,
|
operation: options.operation,
|
||||||
workspaceId: this.options.workspaceId,
|
workspaceId: this.options.workspaceId,
|
||||||
@@ -304,6 +312,8 @@ export class PreprocessingStateStore {
|
|||||||
catalogBlob: options.catalogBlob,
|
catalogBlob: options.catalogBlob,
|
||||||
configDigest: options.configDigest,
|
configDigest: options.configDigest,
|
||||||
bindingDigest: options.bindingDigest,
|
bindingDigest: options.bindingDigest,
|
||||||
|
embeddingId: options.embeddingId,
|
||||||
|
embeddingDimensions: options.embeddingDimensions,
|
||||||
completedStages: [],
|
completedStages: [],
|
||||||
childRuns: {},
|
childRuns: {},
|
||||||
status: "active",
|
status: "active",
|
||||||
|
|||||||
@@ -560,7 +560,7 @@ export class WorkspaceRegistry {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const collection = workspace.semantic_index.vector_store.collection;
|
const collection = workspace.workspace.id;
|
||||||
const owner = collectionOwners.get(collection);
|
const owner = collectionOwners.get(collection);
|
||||||
if (owner !== undefined) {
|
if (owner !== undefined) {
|
||||||
throw new Error(`duplicate qdrant collection ownership: ${collection} (${owner}, ${id})`);
|
throw new Error(`duplicate qdrant collection ownership: ${collection} (${owner}, ${id})`);
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ export interface RuntimeInstallationOverlay {
|
|||||||
export interface SemanticRuntimeConfig {
|
export interface SemanticRuntimeConfig {
|
||||||
internalQdrantUrl: string;
|
internalQdrantUrl: string;
|
||||||
internalEmbeddingUrl: string;
|
internalEmbeddingUrl: string;
|
||||||
|
internalEmbeddingId?: string;
|
||||||
internalEmbeddingModel: string;
|
internalEmbeddingModel: string;
|
||||||
internalEmbeddingDimensions: number;
|
internalEmbeddingDimensions: number;
|
||||||
}
|
}
|
||||||
@@ -41,6 +42,7 @@ export interface SemanticRuntimeConfig {
|
|||||||
export const DEFAULT_SEMANTIC_RUNTIME: SemanticRuntimeConfig = {
|
export const DEFAULT_SEMANTIC_RUNTIME: SemanticRuntimeConfig = {
|
||||||
internalQdrantUrl: "http://qdrant:6333",
|
internalQdrantUrl: "http://qdrant:6333",
|
||||||
internalEmbeddingUrl: "http://embedding:11434",
|
internalEmbeddingUrl: "http://embedding:11434",
|
||||||
|
internalEmbeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
||||||
internalEmbeddingDimensions: 1024,
|
internalEmbeddingDimensions: 1024,
|
||||||
};
|
};
|
||||||
@@ -202,16 +204,16 @@ function placeholderConnection(identity: { database: string; schema: string }):
|
|||||||
|
|
||||||
function requireSupportedDescriptor(workspace: unknown): void {
|
function requireSupportedDescriptor(workspace: unknown): void {
|
||||||
if (typeof workspace !== "object" || workspace === null) {
|
if (typeof workspace !== "object" || workspace === null) {
|
||||||
throw new Error("Runtime renderer supports only workspace schema version 3");
|
throw new Error("Runtime renderer supports only workspace schema version 4");
|
||||||
}
|
}
|
||||||
const metadata = Reflect.get(workspace, "workspace");
|
const metadata = Reflect.get(workspace, "workspace");
|
||||||
if (typeof metadata !== "object" || metadata === null
|
if (typeof metadata !== "object" || metadata === null
|
||||||
|| Reflect.get(metadata, "schema_version") !== 3) {
|
|| Reflect.get(metadata, "schema_version") !== 4) {
|
||||||
throw new Error("Runtime renderer supports only workspace schema version 3");
|
throw new Error("Runtime renderer supports only workspace schema version 4");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Render the schema-v3 compatibility fields consumed by the current Python harness. */
|
/** Render the schema-v4 compatibility fields consumed by the current Python harness. */
|
||||||
export function renderRuntimeConfig(
|
export function renderRuntimeConfig(
|
||||||
workspace: WorkspaceDescriptor,
|
workspace: WorkspaceDescriptor,
|
||||||
bindings: RuntimeBindings,
|
bindings: RuntimeBindings,
|
||||||
@@ -263,12 +265,24 @@ export function renderRuntimeConfig(
|
|||||||
...(installation.profile === undefined ? {} : { profile: installation.profile }),
|
...(installation.profile === undefined ? {} : { profile: installation.profile }),
|
||||||
language: descriptor.workspace.language,
|
language: descriptor.workspace.language,
|
||||||
database,
|
database,
|
||||||
semantic_index: descriptor.semantic_index,
|
semantic_index: {
|
||||||
|
vector_store: {
|
||||||
|
engine: "qdrant",
|
||||||
|
collection: descriptor.workspace.id,
|
||||||
|
dimensions: semanticRuntime.internalEmbeddingDimensions,
|
||||||
|
distance: "cosine",
|
||||||
|
},
|
||||||
|
embedding: {
|
||||||
|
provider: "ollama_internal",
|
||||||
|
model: semanticRuntime.internalEmbeddingModel,
|
||||||
|
dimensions: semanticRuntime.internalEmbeddingDimensions,
|
||||||
|
},
|
||||||
|
},
|
||||||
resources: {
|
resources: {
|
||||||
vector: {
|
vector: {
|
||||||
engine: "qdrant",
|
engine: "qdrant",
|
||||||
base_url: semanticRuntime.internalQdrantUrl,
|
base_url: semanticRuntime.internalQdrantUrl,
|
||||||
collection: descriptor.semantic_index.vector_store.collection,
|
collection: descriptor.workspace.id,
|
||||||
},
|
},
|
||||||
embeddings: {
|
embeddings: {
|
||||||
provider: "ollama_internal",
|
provider: "ollama_internal",
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ export interface CanonicalDiagnostics {
|
|||||||
}
|
}
|
||||||
|
|
||||||
interface WorkspaceMetadata {
|
interface WorkspaceMetadata {
|
||||||
schema_version: 3;
|
schema_version: 4;
|
||||||
id: string;
|
id: string;
|
||||||
name: string;
|
name: string;
|
||||||
description?: string;
|
description?: string;
|
||||||
@@ -51,31 +51,12 @@ interface WorkspaceDwh {
|
|||||||
supported_transports: DwhTransport[];
|
supported_transports: DwhTransport[];
|
||||||
}
|
}
|
||||||
|
|
||||||
interface WorkspaceBase<TVectorStore> {
|
interface WorkspaceBase {
|
||||||
workspace: WorkspaceMetadata;
|
workspace: WorkspaceMetadata;
|
||||||
dwh: WorkspaceDwh;
|
dwh: WorkspaceDwh;
|
||||||
semantic_index: {
|
|
||||||
vector_store: TVectorStore;
|
|
||||||
embedding: {
|
|
||||||
provider: "ollama_internal";
|
|
||||||
model: "qwen3-embedding:0.6b";
|
|
||||||
dimensions: 1024;
|
|
||||||
};
|
|
||||||
};
|
|
||||||
llm_policy: {
|
|
||||||
default?: `${string}/${string}`;
|
|
||||||
allowed: `${string}/${string}`[];
|
|
||||||
};
|
|
||||||
diagnostics?: Pick<CanonicalDiagnostics, "dwh_rest">;
|
diagnostics?: Pick<CanonicalDiagnostics, "dwh_rest">;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface QdrantVectorStore {
|
|
||||||
engine: "qdrant";
|
|
||||||
collection: string;
|
|
||||||
dimensions: 1024;
|
|
||||||
distance: "cosine";
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface EvidencePolicy {
|
export interface EvidencePolicy {
|
||||||
max_chunk_chars: number;
|
max_chunk_chars: number;
|
||||||
retain_published_generations: number;
|
retain_published_generations: number;
|
||||||
@@ -120,12 +101,12 @@ export interface WorkspaceEvidence {
|
|||||||
policy: EvidencePolicy;
|
policy: EvidencePolicy;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface WorkspaceV3 extends WorkspaceBase<QdrantVectorStore> {
|
export interface WorkspaceV4 extends WorkspaceBase {
|
||||||
evidence?: WorkspaceEvidence;
|
evidence?: WorkspaceEvidence;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type CanonicalWorkspace = WorkspaceV3;
|
export type CanonicalWorkspace = WorkspaceV4;
|
||||||
export type WorkspaceDescriptor = WorkspaceV3;
|
export type WorkspaceDescriptor = WorkspaceV4;
|
||||||
|
|
||||||
const workspaceId = z.string().regex(/^[a-z][a-z0-9-]{2,62}$/, {
|
const workspaceId = z.string().regex(/^[a-z][a-z0-9-]{2,62}$/, {
|
||||||
message: "workspace id must match ^[a-z][a-z0-9-]{2,62}$",
|
message: "workspace id must match ^[a-z][a-z0-9-]{2,62}$",
|
||||||
@@ -135,9 +116,6 @@ const identifier = z.string().regex(/^[A-Za-z_][A-Za-z0-9_]*$/, {
|
|||||||
});
|
});
|
||||||
const port = z.number().int().min(1).max(65_535);
|
const port = z.number().int().min(1).max(65_535);
|
||||||
const timeoutMs = z.number().int().positive();
|
const timeoutMs = z.number().int().positive();
|
||||||
const modelReference = z.string().regex(/^[^/\s]+\/[^/\s]+$/, {
|
|
||||||
message: "model must use provider/model syntax",
|
|
||||||
});
|
|
||||||
|
|
||||||
function isOriginRelativeDiagnosticPath(value: string): boolean {
|
function isOriginRelativeDiagnosticPath(value: string): boolean {
|
||||||
return /^\/(?!\/)[^\\\u0000-\u001F\u007F?#]*$/.test(value) && !/%5c/i.test(value);
|
return /^\/(?!\/)[^\\\u0000-\u001F\u007F?#]*$/.test(value) && !/%5c/i.test(value);
|
||||||
@@ -166,21 +144,6 @@ const dwhSchema = z.object({
|
|||||||
timeout_ms: timeoutMs.optional(),
|
timeout_ms: timeoutMs.optional(),
|
||||||
supported_transports: z.array(z.enum(DWH_TRANSPORTS)).min(1),
|
supported_transports: z.array(z.enum(DWH_TRANSPORTS)).min(1),
|
||||||
}).strict();
|
}).strict();
|
||||||
const internalEmbeddingSchema = z.object({
|
|
||||||
provider: z.literal("ollama_internal"),
|
|
||||||
model: z.literal("qwen3-embedding:0.6b"),
|
|
||||||
dimensions: z.literal(1024),
|
|
||||||
}).strict();
|
|
||||||
const qdrantVectorStoreSchema = z.object({
|
|
||||||
engine: z.literal("qdrant"),
|
|
||||||
collection: workspaceId,
|
|
||||||
dimensions: z.literal(1024),
|
|
||||||
distance: z.literal("cosine"),
|
|
||||||
}).strict();
|
|
||||||
const llmPolicySchema = z.object({
|
|
||||||
default: modelReference.optional(),
|
|
||||||
allowed: z.array(modelReference).min(1),
|
|
||||||
}).strict();
|
|
||||||
|
|
||||||
const positiveSafeInteger = z.number().int().safe().positive();
|
const positiveSafeInteger = z.number().int().safe().positive();
|
||||||
const nonnegativeSafeInteger = z.number().int().safe().nonnegative();
|
const nonnegativeSafeInteger = z.number().int().safe().nonnegative();
|
||||||
@@ -366,7 +329,6 @@ function unique<T>(values: readonly T[], context: z.RefinementCtx, path: Propert
|
|||||||
|
|
||||||
function workspaceInvariants(workspace: any, context: z.RefinementCtx): void {
|
function workspaceInvariants(workspace: any, context: z.RefinementCtx): void {
|
||||||
unique(workspace.dwh.supported_transports, context, ["dwh", "supported_transports"]);
|
unique(workspace.dwh.supported_transports, context, ["dwh", "supported_transports"]);
|
||||||
unique(workspace.llm_policy.allowed, context, ["llm_policy", "allowed"]);
|
|
||||||
|
|
||||||
if (workspace.evidence?.source.type === "filesystem") {
|
if (workspace.evidence?.source.type === "filesystem") {
|
||||||
const expected = `${workspace.workspace.id}/evidence`;
|
const expected = `${workspace.workspace.id}/evidence`;
|
||||||
@@ -379,20 +341,6 @@ function workspaceInvariants(workspace: any, context: z.RefinementCtx): void {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (workspace.semantic_index.vector_store.dimensions !== workspace.semantic_index.embedding.dimensions) {
|
|
||||||
context.addIssue({
|
|
||||||
code: "custom",
|
|
||||||
path: ["semantic_index", "embedding", "dimensions"],
|
|
||||||
message: "embedding dimensions must match vector store dimensions",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
if (workspace.llm_policy.default && !workspace.llm_policy.allowed.includes(workspace.llm_policy.default)) {
|
|
||||||
context.addIssue({
|
|
||||||
code: "custom",
|
|
||||||
path: ["llm_policy", "default"],
|
|
||||||
message: "LLM default must be included in the allowlist",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
if (workspace.diagnostics?.dwh_rest && !workspace.dwh.supported_transports.includes("rest_api")) {
|
if (workspace.diagnostics?.dwh_rest && !workspace.dwh.supported_transports.includes("rest_api")) {
|
||||||
context.addIssue({
|
context.addIssue({
|
||||||
code: "custom",
|
code: "custom",
|
||||||
@@ -402,23 +350,18 @@ function workspaceInvariants(workspace: any, context: z.RefinementCtx): void {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
const WorkspaceV3Schema = z.object({
|
const WorkspaceV4Schema = z.object({
|
||||||
dwh: dwhSchema,
|
dwh: dwhSchema,
|
||||||
llm_policy: llmPolicySchema,
|
|
||||||
evidence: workspaceEvidenceSchema.optional(),
|
evidence: workspaceEvidenceSchema.optional(),
|
||||||
diagnostics: z.object({
|
diagnostics: z.object({
|
||||||
dwh_rest: dwhRestDiagnostic.optional(),
|
dwh_rest: dwhRestDiagnostic.optional(),
|
||||||
}).strict().optional(),
|
}).strict().optional(),
|
||||||
workspace: z.object({
|
workspace: z.object({
|
||||||
schema_version: z.literal(3), id: workspaceId, name: z.string().trim().min(1),
|
schema_version: z.literal(4), id: workspaceId, name: z.string().trim().min(1),
|
||||||
description: z.string().trim().min(1).optional(), language: z.enum(["en", "it"]),
|
description: z.string().trim().min(1).optional(), language: z.enum(["en", "it"]),
|
||||||
}).strict(),
|
}).strict(),
|
||||||
semantic_index: z.object({
|
|
||||||
vector_store: qdrantVectorStoreSchema,
|
|
||||||
embedding: internalEmbeddingSchema,
|
|
||||||
}).strict(),
|
|
||||||
}).strict().superRefine(workspaceInvariants);
|
}).strict().superRefine(workspaceInvariants);
|
||||||
const WorkspaceDescriptorSchema = WorkspaceV3Schema;
|
const WorkspaceDescriptorSchema = WorkspaceV4Schema;
|
||||||
|
|
||||||
export function parseWorkspaceYaml(source: string): WorkspaceDescriptor {
|
export function parseWorkspaceYaml(source: string): WorkspaceDescriptor {
|
||||||
const documents = parseAllDocuments(source, { uniqueKeys: true });
|
const documents = parseAllDocuments(source, { uniqueKeys: true });
|
||||||
@@ -431,6 +374,25 @@ export function parseWorkspaceYaml(source: string): WorkspaceDescriptor {
|
|||||||
return validateWorkspaceDescriptor(document.toJSON());
|
return validateWorkspaceDescriptor(document.toJSON());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Deterministically removes the two installation-owned v3 blocks without altering workspace data. */
|
||||||
|
export function migrateWorkspaceV3Yaml(source: string): string {
|
||||||
|
const documents = parseAllDocuments(source, { uniqueKeys: true });
|
||||||
|
if (documents.length !== 1) throw new Error("Workspace YAML must contain exactly one document");
|
||||||
|
const document = documents[0];
|
||||||
|
if (document.errors.length > 0 || document.warnings.length > 0) {
|
||||||
|
throw new Error("Invalid workspace YAML");
|
||||||
|
}
|
||||||
|
const value = document.toJSON() as Record<string, unknown>;
|
||||||
|
const metadata = value.workspace as Record<string, unknown> | undefined;
|
||||||
|
if (!metadata || metadata.schema_version !== 3) {
|
||||||
|
throw new Error("Workspace migration requires schema version 3");
|
||||||
|
}
|
||||||
|
metadata.schema_version = 4;
|
||||||
|
delete value.semantic_index;
|
||||||
|
delete value.llm_policy;
|
||||||
|
return serializeWorkspaceYaml(validateWorkspaceDescriptor(value));
|
||||||
|
}
|
||||||
|
|
||||||
export function validateWorkspaceDescriptor(workspace: unknown): WorkspaceDescriptor {
|
export function validateWorkspaceDescriptor(workspace: unknown): WorkspaceDescriptor {
|
||||||
return WorkspaceDescriptorSchema.parse(workspace) as WorkspaceDescriptor;
|
return WorkspaceDescriptorSchema.parse(workspace) as WorkspaceDescriptor;
|
||||||
}
|
}
|
||||||
@@ -439,11 +401,11 @@ export function isCanonicalWorkspace(workspace: unknown): workspace is Canonical
|
|||||||
return WorkspaceDescriptorSchema.safeParse(workspace).success;
|
return WorkspaceDescriptorSchema.safeParse(workspace).success;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function isOperationalWorkspace(workspace: unknown): workspace is WorkspaceV3 {
|
export function isOperationalWorkspace(workspace: unknown): workspace is WorkspaceV4 {
|
||||||
return WorkspaceDescriptorSchema.safeParse(workspace).success;
|
return WorkspaceDescriptorSchema.safeParse(workspace).success;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function validateOperationalWorkspace(workspace: unknown): WorkspaceV3 {
|
export function validateOperationalWorkspace(workspace: unknown): WorkspaceV4 {
|
||||||
return validateWorkspaceDescriptor(workspace);
|
return validateWorkspaceDescriptor(workspace);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -19,4 +19,4 @@ export type WorkspaceErrorCode =
|
|||||||
| "workspace_stale" | "git_unavailable" | "git_auth_failed" | "git_non_fast_forward"
|
| "workspace_stale" | "git_unavailable" | "git_auth_failed" | "git_non_fast_forward"
|
||||||
| "connector_unavailable" | "semantic_index_incompatible";
|
| "connector_unavailable" | "semantic_index_incompatible";
|
||||||
|
|
||||||
export type { WorkspaceV3 } from "./schema.js";
|
export type { WorkspaceV4 } from "./schema.js";
|
||||||
|
|||||||
@@ -5,14 +5,12 @@ import { createLocalAuthFixture } from "./auth-test-fixtures.js";
|
|||||||
function fakeService(): PiManagementService {
|
function fakeService(): PiManagementService {
|
||||||
return {
|
return {
|
||||||
status: vi.fn(async () => ({ ready: true })),
|
status: vi.fn(async () => ({ ready: true })),
|
||||||
options: vi.fn(async () => ({ providers: [], models: [], reasoning: [], checkedAt: "2026-08-17T00:00:00.000Z" })),
|
|
||||||
configure: vi.fn(async (value) => ({ ...value, updatedAt: "2026-08-17T00:00:00.000Z" })),
|
|
||||||
test: vi.fn(async () => ({ ready: true, checkedAt: "2026-08-17T00:00:00.000Z" })),
|
test: vi.fn(async () => ({ ready: true, checkedAt: "2026-08-17T00:00:00.000Z" })),
|
||||||
logs: vi.fn(async () => ({ lines: [] })),
|
logs: vi.fn(async () => ({ lines: [] })),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
test("a local HTTPS cookie session authorizes Pi writes through an untrusted internal HTTP hop", async () => {
|
test("a local HTTPS cookie session authorizes the Pi smoke check through an untrusted internal HTTP hop", async () => {
|
||||||
const service = fakeService();
|
const service = fakeService();
|
||||||
const fixture = await createLocalAuthFixture(
|
const fixture = await createLocalAuthFixture(
|
||||||
{ piManagement: service },
|
{ piManagement: service },
|
||||||
@@ -25,31 +23,21 @@ test("a local HTTPS cookie session authorizes Pi writes through an untrusted int
|
|||||||
expect(fixture.publicUrl).toBe("HTTPS://thothii.example.test");
|
expect(fixture.publicUrl).toBe("HTTPS://thothii.example.test");
|
||||||
|
|
||||||
const proxyHeaders = fixture.sessionHeaders({ host: "127.0.0.1:8080" });
|
const proxyHeaders = fixture.sessionHeaders({ host: "127.0.0.1:8080" });
|
||||||
const configured = await fixture.app.inject({
|
|
||||||
method: "PUT",
|
|
||||||
url: "/pi-management/config",
|
|
||||||
headers: proxyHeaders,
|
|
||||||
payload: { provider: "zai", model: "glm-5.2", reasoning: "high" },
|
|
||||||
});
|
|
||||||
const smoke = await fixture.app.inject({
|
const smoke = await fixture.app.inject({
|
||||||
method: "POST",
|
method: "POST",
|
||||||
url: "/pi-management/test",
|
url: "/pi-management/test",
|
||||||
headers: proxyHeaders,
|
headers: proxyHeaders,
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(configured.statusCode).toBe(200);
|
|
||||||
expect(smoke.statusCode).toBe(200);
|
expect(smoke.statusCode).toBe(200);
|
||||||
expect(service.configure).toHaveBeenCalledTimes(1);
|
|
||||||
expect(service.test).toHaveBeenCalledTimes(1);
|
expect(service.test).toHaveBeenCalledTimes(1);
|
||||||
|
|
||||||
fixture.resetDownstreamHits();
|
fixture.resetDownstreamHits();
|
||||||
vi.mocked(service.configure).mockClear();
|
|
||||||
vi.mocked(service.test).mockClear();
|
vi.mocked(service.test).mockClear();
|
||||||
const wrongOrigin = await fixture.app.inject({
|
const wrongOrigin = await fixture.app.inject({
|
||||||
method: "PUT",
|
method: "POST",
|
||||||
url: "/pi-management/config",
|
url: "/pi-management/test",
|
||||||
headers: fixture.sessionHeaders({ host: "127.0.0.1:8080", origin: "https://evil.example" }),
|
headers: fixture.sessionHeaders({ host: "127.0.0.1:8080", origin: "https://evil.example" }),
|
||||||
payload: { provider: "zai", model: "glm-5.2", reasoning: "high" },
|
|
||||||
});
|
});
|
||||||
const wrongCsrf = await fixture.app.inject({
|
const wrongCsrf = await fixture.app.inject({
|
||||||
method: "POST",
|
method: "POST",
|
||||||
@@ -62,7 +50,6 @@ test("a local HTTPS cookie session authorizes Pi writes through an untrusted int
|
|||||||
expect(response.json()).toEqual({ code: "csrf_failed", error: "Request origin validation failed" });
|
expect(response.json()).toEqual({ code: "csrf_failed", error: "Request origin validation failed" });
|
||||||
}
|
}
|
||||||
expect(fixture.downstreamHits()).toBe(0);
|
expect(fixture.downstreamHits()).toBe(0);
|
||||||
expect(service.configure).not.toHaveBeenCalled();
|
|
||||||
expect(service.test).not.toHaveBeenCalled();
|
expect(service.test).not.toHaveBeenCalled();
|
||||||
} finally {
|
} finally {
|
||||||
await fixture.close();
|
await fixture.close();
|
||||||
|
|||||||
@@ -19,16 +19,11 @@ afterEach(() => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
const workspace: WorkspaceDescriptor = {
|
const workspace: WorkspaceDescriptor = {
|
||||||
workspace: { schema_version: 3, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
workspace: { schema_version: 4, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres", database: "warehouse", schema: "datawarehouse", port: 5432,
|
engine: "postgres", database: "warehouse", schema: "datawarehouse", port: 5432,
|
||||||
supported_transports: ["postgres_direct", "rest_api"],
|
supported_transports: ["postgres_direct", "rest_api"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: "psd", dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
diagnostics: { dwh_rest: { method: "GET", path: "/health", auth: "bearer", response: { database: "database", schema: "schema" } } },
|
diagnostics: { dwh_rest: { method: "GET", path: "/health", auth: "bearer", response: { database: "database", schema: "schema" } } },
|
||||||
};
|
};
|
||||||
const revision: WorkspaceRevision = { id: "psd-clinical", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/tmp/psd.yaml" };
|
const revision: WorkspaceRevision = { id: "psd-clinical", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/tmp/psd.yaml" };
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ import type { WorkspaceDescriptor } from "../src/workspaces/schema.js";
|
|||||||
|
|
||||||
const workspace: WorkspaceDescriptor = {
|
const workspace: WorkspaceDescriptor = {
|
||||||
workspace: {
|
workspace: {
|
||||||
schema_version: 3,
|
schema_version: 4,
|
||||||
id: "psd-clinical",
|
id: "psd-clinical",
|
||||||
name: "Policlinico San Donato",
|
name: "Policlinico San Donato",
|
||||||
language: "it",
|
language: "it",
|
||||||
@@ -36,11 +36,6 @@ const workspace: WorkspaceDescriptor = {
|
|||||||
port: 5432,
|
port: 5432,
|
||||||
supported_transports: ["postgres_direct"],
|
supported_transports: ["postgres_direct"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: "psd", dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
};
|
};
|
||||||
const revision: WorkspaceRevision = {
|
const revision: WorkspaceRevision = {
|
||||||
id: "psd-clinical",
|
id: "psd-clinical",
|
||||||
@@ -49,7 +44,7 @@ const revision: WorkspaceRevision = {
|
|||||||
snapshotPath: "/tmp/psd.yaml",
|
snapshotPath: "/tmp/psd.yaml",
|
||||||
};
|
};
|
||||||
const configuredModel: ResolvedMetadataGenerationModel = {
|
const configuredModel: ResolvedMetadataGenerationModel = {
|
||||||
id: "openai-mini",
|
id: "openai/gpt-4.1-mini",
|
||||||
provider: "openai",
|
provider: "openai",
|
||||||
model: "gpt-4.1-mini",
|
model: "gpt-4.1-mini",
|
||||||
apiKeyEnv: "OPENAI_API_KEY",
|
apiKeyEnv: "OPENAI_API_KEY",
|
||||||
@@ -705,7 +700,7 @@ test("generates one selected Catalog Column from a single JSON code fence", asyn
|
|||||||
expect(completionRequest.messages[0]?.content).toContain('{"results":[');
|
expect(completionRequest.messages[0]?.content).toContain('{"results":[');
|
||||||
expect(completionRequest.messages[1]?.content).toContain(`"targetId":"${column.id}"`);
|
expect(completionRequest.messages[1]?.content).toContain(`"targetId":"${column.id}"`);
|
||||||
expect(completionRequest.messages[1]?.content).not.toMatch(/source rows|samples|example values/i);
|
expect(completionRequest.messages[1]?.content).not.toMatch(/source rows|samples|example values/i);
|
||||||
expect(start.body).not.toMatch(/test-provider-secret|gpt-4\.1|openai\/gpt|Catalog metadata/);
|
expect(start.body).not.toMatch(/test-provider-secret|Catalog metadata/);
|
||||||
|
|
||||||
resolveCompletion(`\`\`\`json\n${JSON.stringify({
|
resolveCompletion(`\`\`\`json\n${JSON.stringify({
|
||||||
results: [{
|
results: [{
|
||||||
@@ -2909,7 +2904,7 @@ test("validates selected targets and resolves every requested target before laun
|
|||||||
const unknownModel = await app.inject({
|
const unknownModel = await app.inject({
|
||||||
method: "POST",
|
method: "POST",
|
||||||
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
||||||
payload: { modelId: "unknown-model", scope: "selected_columns", targetIds: [column.id] },
|
payload: { modelId: "openai/unknown-model", scope: "selected_columns", targetIds: [column.id] },
|
||||||
});
|
});
|
||||||
expect(unknownModel.statusCode).toBe(409);
|
expect(unknownModel.statusCode).toBe(409);
|
||||||
expect(unknownModel.json().code).toBe("metadata_generation_model_unavailable");
|
expect(unknownModel.json().code).toBe("metadata_generation_model_unavailable");
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import { up as upDescriptionGeneration } from "../src/catalog/migrations/005_des
|
|||||||
import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js";
|
import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensitive_data_flag.js";
|
||||||
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
|
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
|
||||||
import { up as upAiTokenUsage } from "../src/catalog/migrations/009_ai_token_usage.js";
|
import { up as upAiTokenUsage } from "../src/catalog/migrations/009_ai_token_usage.js";
|
||||||
|
import { up as upCanonicalModelIds } from "../src/catalog/migrations/010_canonical_model_ids.js";
|
||||||
import { KyselyCatalogRepository, type CatalogDatabase } from "../src/catalog/repository.js";
|
import { KyselyCatalogRepository, type CatalogDatabase } from "../src/catalog/repository.js";
|
||||||
import { loadConfig } from "../src/config.js";
|
import { loadConfig } from "../src/config.js";
|
||||||
import type { WorkspaceRegistry } from "../src/workspaces/registry.js";
|
import type { WorkspaceRegistry } from "../src/workspaces/registry.js";
|
||||||
@@ -50,6 +51,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
|||||||
await upDescriptionGeneration(db);
|
await upDescriptionGeneration(db);
|
||||||
await upSensitiveSuggestionRuns(db);
|
await upSensitiveSuggestionRuns(db);
|
||||||
await upAiTokenUsage(db);
|
await upAiTokenUsage(db);
|
||||||
|
await upCanonicalModelIds(db);
|
||||||
const repository = new KyselyCatalogRepository(db);
|
const repository = new KyselyCatalogRepository(db);
|
||||||
const database = await repository.create({
|
const database = await repository.create({
|
||||||
workspaceId: "psd-clinical",
|
workspaceId: "psd-clinical",
|
||||||
@@ -158,9 +160,9 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
|||||||
}),
|
}),
|
||||||
};
|
};
|
||||||
const models: MetadataGenerationModels = {
|
const models: MetadataGenerationModels = {
|
||||||
catalog: () => ({ models: [{ id: "openai-mini", label: "OpenAI Mini" }], default: "openai-mini" }),
|
catalog: () => ({ models: [{ id: "openai/gpt-4.1-mini", label: "OpenAI Mini" }], default: "openai/gpt-4.1-mini" }),
|
||||||
resolve: () => ({
|
resolve: () => ({
|
||||||
id: "openai-mini",
|
id: "openai/gpt-4.1-mini",
|
||||||
provider: "openai",
|
provider: "openai",
|
||||||
model: "gpt-4.1-mini",
|
model: "gpt-4.1-mini",
|
||||||
apiKeyEnv: "OPENAI_API_KEY",
|
apiKeyEnv: "OPENAI_API_KEY",
|
||||||
@@ -205,12 +207,12 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
|||||||
method: "POST",
|
method: "POST",
|
||||||
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
||||||
payload: {
|
payload: {
|
||||||
modelId: "openai-mini",
|
modelId: "openai/gpt-4.1-mini",
|
||||||
scope: "selected_columns",
|
scope: "selected_columns",
|
||||||
targetIds: [status.id, birthDate.id],
|
targetIds: [status.id, birthDate.id],
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
expect(successfulStart.statusCode).toBe(202);
|
expect(successfulStart.statusCode, successfulStart.body).toBe(202);
|
||||||
expect(await terminalRun(app, successfulStart.json().id)).toMatchObject({
|
expect(await terminalRun(app, successfulStart.json().id)).toMatchObject({
|
||||||
status: "completed",
|
status: "completed",
|
||||||
total: 2,
|
total: 2,
|
||||||
@@ -233,7 +235,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
|||||||
const tableStart = await app.inject({
|
const tableStart = await app.inject({
|
||||||
method: "POST",
|
method: "POST",
|
||||||
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
||||||
payload: { modelId: "openai-mini", scope: "selected_tables", targetIds: [table.id] },
|
payload: { modelId: "openai/gpt-4.1-mini", scope: "selected_tables", targetIds: [table.id] },
|
||||||
});
|
});
|
||||||
expect(tableStart.statusCode).toBe(202);
|
expect(tableStart.statusCode).toBe(202);
|
||||||
expect(await terminalRun(app, tableStart.json().id)).toMatchObject({
|
expect(await terminalRun(app, tableStart.json().id)).toMatchObject({
|
||||||
@@ -253,7 +255,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
|||||||
const failedStart = await app.inject({
|
const failedStart = await app.inject({
|
||||||
method: "POST",
|
method: "POST",
|
||||||
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
||||||
payload: { modelId: "openai-mini", scope: "selected_columns", targetIds: [status.id] },
|
payload: { modelId: "openai/gpt-4.1-mini", scope: "selected_columns", targetIds: [status.id] },
|
||||||
});
|
});
|
||||||
expect(failedStart.statusCode).toBe(202);
|
expect(failedStart.statusCode).toBe(202);
|
||||||
const failedRun = await terminalRun(app, failedStart.json().id);
|
const failedRun = await terminalRun(app, failedStart.json().id);
|
||||||
@@ -283,7 +285,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
|||||||
const allStart = await app.inject({
|
const allStart = await app.inject({
|
||||||
method: "POST",
|
method: "POST",
|
||||||
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
||||||
payload: { modelId: "openai-mini", scope: "all" },
|
payload: { modelId: "openai/gpt-4.1-mini", scope: "all" },
|
||||||
});
|
});
|
||||||
expect(allStart.statusCode).toBe(202);
|
expect(allStart.statusCode).toBe(202);
|
||||||
const allRun = await terminalRun(app, allStart.json().id);
|
const allRun = await terminalRun(app, allStart.json().id);
|
||||||
@@ -327,7 +329,7 @@ test.skipIf(!dockerAvailable)("Fastify persists Description Generation success a
|
|||||||
const missingStart = await app.inject({
|
const missingStart = await app.inject({
|
||||||
method: "POST",
|
method: "POST",
|
||||||
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
url: `/catalog/databases/${database.id}/description-generation-runs`,
|
||||||
payload: { modelId: "openai-mini", scope: "missing" },
|
payload: { modelId: "openai/gpt-4.1-mini", scope: "missing" },
|
||||||
});
|
});
|
||||||
expect(missingStart.statusCode).toBe(202);
|
expect(missingStart.statusCode).toBe(202);
|
||||||
expect(await terminalRun(app, missingStart.json().id)).toMatchObject({
|
expect(await terminalRun(app, missingStart.json().id)).toMatchObject({
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import { up as upSensitiveDataFlag } from "../src/catalog/migrations/006_sensiti
|
|||||||
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
|
import { up as upSensitiveSuggestionRuns } from "../src/catalog/migrations/007_sensitive_data_suggestion_runs.js";
|
||||||
import { up as upLogicalRelationships } from "../src/catalog/migrations/008_catalog_logical_relationships.js";
|
import { up as upLogicalRelationships } from "../src/catalog/migrations/008_catalog_logical_relationships.js";
|
||||||
import { up as upAiTokenUsage } from "../src/catalog/migrations/009_ai_token_usage.js";
|
import { up as upAiTokenUsage } from "../src/catalog/migrations/009_ai_token_usage.js";
|
||||||
|
import { up as upCanonicalModelIds } from "../src/catalog/migrations/010_canonical_model_ids.js";
|
||||||
|
|
||||||
const dockerAvailable = spawnSync("docker", ["info"], { stdio: "ignore" }).status === 0;
|
const dockerAvailable = spawnSync("docker", ["info"], { stdio: "ignore" }).status === 0;
|
||||||
|
|
||||||
@@ -32,6 +33,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL migration enforces one database per wo
|
|||||||
await upDescriptionGeneration(db);
|
await upDescriptionGeneration(db);
|
||||||
await upSensitiveSuggestionRuns(db);
|
await upSensitiveSuggestionRuns(db);
|
||||||
await upAiTokenUsage(db);
|
await upAiTokenUsage(db);
|
||||||
|
await upCanonicalModelIds(db);
|
||||||
await sql`CREATE ROLE thothii_catalog_runtime`.execute(db);
|
await sql`CREATE ROLE thothii_catalog_runtime`.execute(db);
|
||||||
await upRuntimeSequencePrivileges(db);
|
await upRuntimeSequencePrivileges(db);
|
||||||
const sequencePrivilege = await sql<{ allowed: boolean }>`
|
const sequencePrivilege = await sql<{ allowed: boolean }>`
|
||||||
@@ -391,6 +393,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
await upDescriptionGeneration(db);
|
await upDescriptionGeneration(db);
|
||||||
await upSensitiveSuggestionRuns(db);
|
await upSensitiveSuggestionRuns(db);
|
||||||
await upAiTokenUsage(db);
|
await upAiTokenUsage(db);
|
||||||
|
await upCanonicalModelIds(db);
|
||||||
const repository = new KyselyCatalogRepository(db);
|
const repository = new KyselyCatalogRepository(db);
|
||||||
const firstDatabase = await repository.create({
|
const firstDatabase = await repository.create({
|
||||||
workspaceId: "generation-one",
|
workspaceId: "generation-one",
|
||||||
@@ -428,14 +431,14 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
const run = await repository.createDescriptionGenerationRun(
|
const run = await repository.createDescriptionGenerationRun(
|
||||||
firstDatabase.id,
|
firstDatabase.id,
|
||||||
"selected_columns",
|
"selected_columns",
|
||||||
"openai-mini",
|
"openai/gpt-4.1-mini",
|
||||||
"it",
|
"it",
|
||||||
1,
|
1,
|
||||||
);
|
);
|
||||||
expect(run).toMatchObject({
|
expect(run).toMatchObject({
|
||||||
databaseId: firstDatabase.id,
|
databaseId: firstDatabase.id,
|
||||||
scope: "selected_columns",
|
scope: "selected_columns",
|
||||||
modelId: "openai-mini",
|
modelId: "openai/gpt-4.1-mini",
|
||||||
language: "it",
|
language: "it",
|
||||||
status: "queued",
|
status: "queued",
|
||||||
total: 1,
|
total: 1,
|
||||||
@@ -450,7 +453,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
await expect(repository.createDescriptionGenerationRun(
|
await expect(repository.createDescriptionGenerationRun(
|
||||||
secondDatabase.id,
|
secondDatabase.id,
|
||||||
"selected_columns",
|
"selected_columns",
|
||||||
"openai-mini",
|
"openai/gpt-4.1-mini",
|
||||||
"en",
|
"en",
|
||||||
1,
|
1,
|
||||||
)).rejects.toThrow("A description generation run is already active");
|
)).rejects.toThrow("A description generation run is already active");
|
||||||
@@ -461,7 +464,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
await expect(repository.createDescriptionGenerationRun(
|
await expect(repository.createDescriptionGenerationRun(
|
||||||
secondDatabase.id,
|
secondDatabase.id,
|
||||||
"selected_columns",
|
"selected_columns",
|
||||||
"openai-mini",
|
"openai/gpt-4.1-mini",
|
||||||
"en",
|
"en",
|
||||||
1,
|
1,
|
||||||
)).rejects.toThrow("A description generation run is already active");
|
)).rejects.toThrow("A description generation run is already active");
|
||||||
@@ -505,7 +508,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
const next = await repository.createDescriptionGenerationRun(
|
const next = await repository.createDescriptionGenerationRun(
|
||||||
secondDatabase.id,
|
secondDatabase.id,
|
||||||
"missing",
|
"missing",
|
||||||
"openai-mini",
|
"openai/gpt-4.1-mini",
|
||||||
"en",
|
"en",
|
||||||
1,
|
1,
|
||||||
);
|
);
|
||||||
@@ -533,7 +536,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
const allRun = await repository.createDescriptionGenerationRun(
|
const allRun = await repository.createDescriptionGenerationRun(
|
||||||
firstDatabase.id,
|
firstDatabase.id,
|
||||||
"all",
|
"all",
|
||||||
"openai-mini",
|
"openai/gpt-4.1-mini",
|
||||||
"it",
|
"it",
|
||||||
2,
|
2,
|
||||||
);
|
);
|
||||||
@@ -562,7 +565,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
const suggestionRun = await repository.createSensitiveDataSuggestionRun(
|
const suggestionRun = await repository.createSensitiveDataSuggestionRun(
|
||||||
firstDatabase.id,
|
firstDatabase.id,
|
||||||
"selected_columns",
|
"selected_columns",
|
||||||
"openai-mini",
|
"openai/gpt-4.1-mini",
|
||||||
);
|
);
|
||||||
expect(suggestionRun).toMatchObject({
|
expect(suggestionRun).toMatchObject({
|
||||||
databaseId: firstDatabase.id,
|
databaseId: firstDatabase.id,
|
||||||
@@ -604,7 +607,7 @@ test.skipIf(!dockerAvailable)("PostgreSQL repository persists description and se
|
|||||||
const interruptedSuggestionRun = await repository.createSensitiveDataSuggestionRun(
|
const interruptedSuggestionRun = await repository.createSensitiveDataSuggestionRun(
|
||||||
secondDatabase.id,
|
secondDatabase.id,
|
||||||
"all",
|
"all",
|
||||||
"openai-mini",
|
"openai/gpt-4.1-mini",
|
||||||
);
|
);
|
||||||
expect(await repository.interruptActiveSensitiveDataSuggestionRuns(
|
expect(await repository.interruptActiveSensitiveDataSuggestionRuns(
|
||||||
"Sensitive-field suggestion generation was interrupted by backend restart.",
|
"Sensitive-field suggestion generation was interrupted by backend restart.",
|
||||||
|
|||||||
@@ -16,13 +16,8 @@ const roots: string[] = [];
|
|||||||
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
|
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
|
||||||
|
|
||||||
const workspace: WorkspaceDescriptor = {
|
const workspace: WorkspaceDescriptor = {
|
||||||
workspace: { schema_version: 3, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
workspace: { schema_version: 4, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
||||||
dwh: { engine: "postgres", database: "warehouse", schema: "datawarehouse", port: 5432, supported_transports: ["postgres_direct"] },
|
dwh: { engine: "postgres", database: "warehouse", schema: "datawarehouse", port: 5432, supported_transports: ["postgres_direct"] },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: "psd", dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
};
|
};
|
||||||
const revision: WorkspaceRevision = { id: "psd-clinical", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/tmp/psd.yaml" };
|
const revision: WorkspaceRevision = { id: "psd-clinical", commit: "a".repeat(40), blob: "b".repeat(40), snapshotPath: "/tmp/psd.yaml" };
|
||||||
|
|
||||||
|
|||||||
@@ -13,16 +13,11 @@ const roots: string[] = [];
|
|||||||
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
|
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
|
||||||
|
|
||||||
const workspace: WorkspaceDescriptor = {
|
const workspace: WorkspaceDescriptor = {
|
||||||
workspace: { schema_version: 3, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
workspace: { schema_version: 4, id: "psd-clinical", name: "Policlinico San Donato", language: "it" },
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres", database: "warehouse", schema: "datawarehouse", port: 5432,
|
engine: "postgres", database: "warehouse", schema: "datawarehouse", port: 5432,
|
||||||
supported_transports: ["postgres_direct", "rest_api"],
|
supported_transports: ["postgres_direct", "rest_api"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: "psd", dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
diagnostics: { dwh_rest: { method: "GET", path: "/health", auth: "bearer", response: { database: "database", schema: "schema" } } },
|
diagnostics: { dwh_rest: { method: "GET", path: "/health", auth: "bearer", response: { database: "database", schema: "schema" } } },
|
||||||
};
|
};
|
||||||
const revision: WorkspaceRevision = {
|
const revision: WorkspaceRevision = {
|
||||||
|
|||||||
@@ -71,6 +71,7 @@ test("loadConfig keeps local development defaults", () => {
|
|||||||
workspaceSecretRuntimeRoot: "/tmp/thothii-workspace-secrets",
|
workspaceSecretRuntimeRoot: "/tmp/thothii-workspace-secrets",
|
||||||
internalQdrantUrl: "http://qdrant:6333",
|
internalQdrantUrl: "http://qdrant:6333",
|
||||||
internalEmbeddingUrl: "http://embedding:11434",
|
internalEmbeddingUrl: "http://embedding:11434",
|
||||||
|
internalEmbeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
||||||
internalEmbeddingDimensions: 1024,
|
internalEmbeddingDimensions: 1024,
|
||||||
authMode: "none",
|
authMode: "none",
|
||||||
@@ -198,6 +199,22 @@ test("loadConfig accepts only the allowed internal semantic runtime hosts", () =
|
|||||||
.toThrow(/internal.*embedding|invalid/i);
|
.toThrow(/internal.*embedding|invalid/i);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("loadConfig derives the embedding runtime model from its canonical catalog identity", () => {
|
||||||
|
expect(loadConfig({
|
||||||
|
THT_INTERNAL_EMBEDDING_ID: "ollama/nomic-embed-text",
|
||||||
|
THT_INTERNAL_EMBEDDING_MODEL: "nomic-embed-text",
|
||||||
|
})).toMatchObject({
|
||||||
|
internalEmbeddingId: "ollama/nomic-embed-text",
|
||||||
|
internalEmbeddingModel: "nomic-embed-text",
|
||||||
|
});
|
||||||
|
expect(() => loadConfig({
|
||||||
|
THT_INTERNAL_EMBEDDING_ID: "ollama/nomic-embed-text",
|
||||||
|
THT_INTERNAL_EMBEDDING_MODEL: "different-model",
|
||||||
|
})).toThrow("does not match its canonical identity");
|
||||||
|
expect(() => loadConfig({ THT_INTERNAL_EMBEDDING_ID: "not-canonical" }))
|
||||||
|
.toThrow("embedding identity configuration is invalid");
|
||||||
|
});
|
||||||
|
|
||||||
test("loadConfig enables the legacy workspace request only through explicit local mode", () => {
|
test("loadConfig enables the legacy workspace request only through explicit local mode", () => {
|
||||||
expect(loadConfig({ THT_LEGACY_WORKSPACE_MODE: "local" }).legacyWorkspaceMode).toBe(true);
|
expect(loadConfig({ THT_LEGACY_WORKSPACE_MODE: "local" }).legacyWorkspaceMode).toBe(true);
|
||||||
|
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import {
|
|||||||
const semanticRuntime = {
|
const semanticRuntime = {
|
||||||
internalQdrantUrl: "http://qdrant:6333",
|
internalQdrantUrl: "http://qdrant:6333",
|
||||||
internalEmbeddingUrl: "http://embedding:11434",
|
internalEmbeddingUrl: "http://embedding:11434",
|
||||||
|
internalEmbeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
||||||
internalEmbeddingDimensions: 1024,
|
internalEmbeddingDimensions: 1024,
|
||||||
};
|
};
|
||||||
@@ -102,11 +103,11 @@ function restRendered(): Record<string, unknown> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const directCanonical =
|
const directCanonical =
|
||||||
`{"schemaVersion":1,"dwh":{` +
|
`{"schemaVersion":2,"dwh":{` +
|
||||||
`"engine":"postgres","database":"postgres","schema":"datawarehouse",` +
|
`"engine":"postgres","database":"postgres","schema":"datawarehouse",` +
|
||||||
`"transport":"postgres_direct","host":"dwh.internal","port":5432,"user":"thoth_reader"},` +
|
`"transport":"postgres_direct","host":"dwh.internal","port":5432,"user":"thoth_reader"},` +
|
||||||
`"vector":{"collection":"psd-clinical","dimensions":1024,"distance":"cosine"},` +
|
`"vector":{"collection":"psd-clinical","dimensions":1024,"distance":"cosine"},` +
|
||||||
`"embedding":{"model":"qwen3-embedding:0.6b","dimensions":1024},` +
|
`"embedding":{"id":"ollama/qwen3-embedding:0.6b","model":"qwen3-embedding:0.6b","dimensions":1024},` +
|
||||||
`"roots":{"artifacts":"/data/sessions/psd-clinical/artifacts",` +
|
`"roots":{"artifacts":"/data/sessions/psd-clinical/artifacts",` +
|
||||||
`"indexes":"/data/sessions/psd-clinical/indexes"}}`;
|
`"indexes":"/data/sessions/psd-clinical/indexes"}}`;
|
||||||
|
|
||||||
@@ -192,6 +193,9 @@ test("DWH-affecting changes alter the effective config identity", () => {
|
|||||||
const changedCollection = { ...base, resources: { ...base.resources, vector: { ...(base.resources as Record<string, any>).vector, collection: "other" } } };
|
const changedCollection = { ...base, resources: { ...base.resources, vector: { ...(base.resources as Record<string, any>).vector, collection: "other" } } };
|
||||||
expect(effectiveConfigIdentity("psd-clinical", changedCollection)).not.toBe(identityBefore);
|
expect(effectiveConfigIdentity("psd-clinical", changedCollection)).not.toBe(identityBefore);
|
||||||
|
|
||||||
|
const changedEmbeddingIdentity = { ...base, resources: { ...base.resources, embeddings: { ...(base.resources as Record<string, any>).embeddings, model: "other-embedding" } } };
|
||||||
|
expect(effectiveConfigIdentity("psd-clinical", changedEmbeddingIdentity)).not.toBe(identityBefore);
|
||||||
|
|
||||||
const changedTransport = restRendered();
|
const changedTransport = restRendered();
|
||||||
expect(effectiveConfigIdentity("psd-clinical", changedTransport)).not.toBe(identityBefore);
|
expect(effectiveConfigIdentity("psd-clinical", changedTransport)).not.toBe(identityBefore);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,15 +1,12 @@
|
|||||||
import { chmodSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
import { chmodSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { afterEach, expect, test, vi } from "vitest";
|
import { afterEach, expect, test } from "vitest";
|
||||||
import { buildApp } from "../src/app.js";
|
|
||||||
import { MemoryCatalogRepository } from "../src/catalog/memory-repository.js";
|
|
||||||
import {
|
import {
|
||||||
loadMetadataGenerationModels,
|
loadMetadataGenerationModels,
|
||||||
MetadataGenerationModelUnavailableError,
|
MetadataGenerationModelUnavailableError,
|
||||||
} from "../src/catalog/metadata-generation-models.js";
|
} from "../src/catalog/metadata-generation-models.js";
|
||||||
import { loadConfig } from "../src/config.js";
|
import { loadRuntimeModelCatalog, splitCanonicalModelId } from "../src/models/runtime-model-catalog.js";
|
||||||
import type { WorkspaceRegistry } from "../src/workspaces/registry.js";
|
|
||||||
|
|
||||||
const roots: string[] = [];
|
const roots: string[] = [];
|
||||||
|
|
||||||
@@ -17,260 +14,101 @@ afterEach(() => {
|
|||||||
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
|
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
|
||||||
});
|
});
|
||||||
|
|
||||||
function metadataConfiguration(
|
function runtimeCatalog(overrides: Record<string, unknown> = {}, secrets = "OPENAI_API_KEY=raw-provider-secret\n") {
|
||||||
metadataGeneration: string,
|
const root = mkdtempSync(join(tmpdir(), "thothii-runtime-models-"));
|
||||||
secrets = "OPENAI_API_KEY=raw-provider-secret\n",
|
|
||||||
) {
|
|
||||||
const root = mkdtempSync(join(tmpdir(), "thothii-metadata-models-"));
|
|
||||||
roots.push(root);
|
roots.push(root);
|
||||||
const installationFile = join(root, "thothii-installation.yaml");
|
const catalogFile = join(root, "catalog.json");
|
||||||
const secretsFile = join(root, "thothii.secrets");
|
const secretsFile = join(root, "thothii.secrets");
|
||||||
writeFileSync(installationFile, metadataGeneration, { mode: 0o600 });
|
const catalog = {
|
||||||
writeFileSync(secretsFile, secrets, { mode: 0o600 });
|
schemaVersion: 1,
|
||||||
chmodSync(installationFile, 0o600);
|
defaultSession: "zai/glm-5.3",
|
||||||
chmodSync(secretsFile, 0o600);
|
defaultMetadataGeneration: "zai/glm-5.3",
|
||||||
return { installationFile, secretsFile };
|
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||||
}
|
|
||||||
|
|
||||||
function appFor(installationFile: string, secretsFile: string) {
|
|
||||||
const config = loadConfig({
|
|
||||||
NODE_ENV: "test",
|
|
||||||
THT_HARNESS_DIR: "/missing",
|
|
||||||
THT_INSTALLATION_CONFIG_FILE: installationFile,
|
|
||||||
THT_SECRETS_FILE: secretsFile,
|
|
||||||
PI_PROVIDER: "unrelated-pi-provider",
|
|
||||||
PI_MODEL: "unrelated-pi-model",
|
|
||||||
});
|
|
||||||
return buildApp(config, {
|
|
||||||
thtRunner: {} as never,
|
|
||||||
workspaceRegistry: { list: vi.fn(async () => []) } as unknown as WorkspaceRegistry,
|
|
||||||
workspaceDiagnoser: vi.fn(),
|
|
||||||
catalogRepository: new MemoryCatalogRepository(),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
test("exposes only safe metadata-generation choices and their configured default", async () => {
|
|
||||||
const { installationFile, secretsFile } = metadataConfiguration(`metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- id: openai-mini
|
|
||||||
label: OpenAI Mini
|
|
||||||
litellm:
|
|
||||||
provider: openai
|
|
||||||
model: gpt-4.1-mini
|
|
||||||
endpoint:
|
|
||||||
baseUrl: https://api.openai.example/v1
|
|
||||||
apiVersion: "2026-08-01"
|
|
||||||
apiKeyEnv: OPENAI_API_KEY
|
|
||||||
`);
|
|
||||||
const app = appFor(installationFile, secretsFile);
|
|
||||||
|
|
||||||
const response = await app.inject({ method: "GET", url: "/catalog/metadata-generation/models" });
|
|
||||||
|
|
||||||
expect(response.statusCode).toBe(200);
|
|
||||||
expect(response.json()).toEqual({
|
|
||||||
models: [{ id: "openai-mini", label: "OpenAI Mini" }],
|
|
||||||
default: "openai-mini",
|
|
||||||
});
|
|
||||||
expect(response.body).not.toMatch(/openai\/gpt|gpt-4\.1|api\.openai|OPENAI_API_KEY|raw-provider-secret/);
|
|
||||||
await app.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects an unprotected installation descriptor", () => {
|
|
||||||
const { installationFile, secretsFile } = metadataConfiguration(`metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- id: openai-mini
|
|
||||||
label: OpenAI Mini
|
|
||||||
litellm: {provider: openai, model: gpt-4.1-mini}
|
|
||||||
apiKeyEnv: OPENAI_API_KEY
|
|
||||||
`);
|
|
||||||
chmodSync(installationFile, 0o644);
|
|
||||||
|
|
||||||
expect(() => loadMetadataGenerationModels({ installationFile, secretsFile }))
|
|
||||||
.toThrow("metadata-generation installation is unavailable");
|
|
||||||
});
|
|
||||||
|
|
||||||
test("returns an empty safe catalog when no metadata-generation model is configured", async () => {
|
|
||||||
const { installationFile, secretsFile } = metadataConfiguration("profile: local\n");
|
|
||||||
const app = appFor(installationFile, secretsFile);
|
|
||||||
|
|
||||||
const response = await app.inject({ method: "GET", url: "/catalog/metadata-generation/models" });
|
|
||||||
|
|
||||||
expect(response.statusCode).toBe(200);
|
|
||||||
expect(response.json()).toEqual({ models: [], default: null });
|
|
||||||
await app.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
test("resolves only a configured selection for the later generation boundary", () => {
|
|
||||||
const { installationFile, secretsFile } = metadataConfiguration(`metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- id: openai-mini
|
|
||||||
label: OpenAI Mini
|
|
||||||
litellm:
|
|
||||||
provider: openai
|
|
||||||
model: gpt-4.1-mini
|
|
||||||
endpoint: {baseUrl: https://api.openai.example/v1, apiVersion: "2026-08-01"}
|
|
||||||
apiKeyEnv: OPENAI_API_KEY
|
|
||||||
`);
|
|
||||||
const models = loadMetadataGenerationModels({ installationFile, secretsFile });
|
|
||||||
|
|
||||||
expect(models.resolve("openai-mini")).toEqual({
|
|
||||||
id: "openai-mini",
|
|
||||||
provider: "openai",
|
|
||||||
model: "gpt-4.1-mini",
|
|
||||||
endpoint: { baseUrl: "https://api.openai.example/v1", apiVersion: "2026-08-01" },
|
|
||||||
apiKeyEnv: "OPENAI_API_KEY",
|
|
||||||
apiKey: "raw-provider-secret",
|
|
||||||
});
|
|
||||||
expect(() => models.resolve("unknown-model")).toThrow(MetadataGenerationModelUnavailableError);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("loads DeepSeek models, GLM, and an explicit keyless Qwen endpoint from installation setup", () => {
|
|
||||||
const { installationFile, secretsFile } = metadataConfiguration(`metadataGeneration:
|
|
||||||
default: glm-53
|
|
||||||
models:
|
|
||||||
- id: deepseek-v4-pro
|
|
||||||
label: DeepSeek V4 Pro
|
|
||||||
litellm: {provider: deepseek, model: deepseek-v4-pro}
|
|
||||||
apiKeyEnv: DEEPSEEK_API_KEY
|
|
||||||
- id: deepseek-v4-flash
|
|
||||||
label: DeepSeek V4 Flash
|
|
||||||
litellm: {provider: deepseek, model: deepseek-v4-flash}
|
|
||||||
apiKeyEnv: DEEPSEEK_API_KEY
|
|
||||||
- id: glm-53
|
|
||||||
label: GLM 5.3
|
|
||||||
litellm:
|
|
||||||
provider: openai
|
|
||||||
model: glm-5.3
|
|
||||||
endpoint: {baseUrl: https://api.z.ai/api/coding/paas/v4}
|
|
||||||
apiKeyEnv: ZAI_API_KEY
|
|
||||||
- id: qwen-36
|
|
||||||
label: Qwen 3.6
|
|
||||||
litellm:
|
|
||||||
provider: openai
|
|
||||||
model: qwen3.6-35b-a3b
|
|
||||||
disableThinking: true
|
|
||||||
endpoint: {baseUrl: https://models.internal.example/v1}
|
|
||||||
`, "DEEPSEEK_API_KEY=deepseek-secret\nZAI_API_KEY=zai-secret\n");
|
|
||||||
|
|
||||||
const models = loadMetadataGenerationModels({ installationFile, secretsFile });
|
|
||||||
|
|
||||||
expect(models.catalog()).toEqual({
|
|
||||||
models: [
|
models: [
|
||||||
{ id: "deepseek-v4-pro", label: "DeepSeek V4 Pro" },
|
{
|
||||||
{ id: "deepseek-v4-flash", label: "DeepSeek V4 Flash" },
|
id: "zai/glm-5.3", provider: "zai", model: "glm-5.3", label: "GLM 5.3",
|
||||||
{ id: "glm-53", label: "GLM 5.3" },
|
upstreamModel: "glm-5.3", endpoint: { baseUrl: "https://api.z.ai/v1" },
|
||||||
{ id: "qwen-36", label: "Qwen 3.6" },
|
authentication: { mode: "secret_env", apiKeyEnv: "OPENAI_API_KEY" },
|
||||||
|
sessionAdapter: { mode: "openai_compatible" },
|
||||||
|
metadataAdapter: { litellmProvider: "openai" },
|
||||||
|
session: { reasoning: true, contextWindow: 200000, maxTokens: 131072 },
|
||||||
|
metadataGeneration: { disableThinking: false },
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "deepseek/deepseek-v4-pro", provider: "deepseek", model: "deepseek-v4-pro",
|
||||||
|
label: "DeepSeek V4 Pro", upstreamModel: "deepseek-v4-pro",
|
||||||
|
authentication: { mode: "pi_auth" }, sessionAdapter: { mode: "pi_builtin" },
|
||||||
|
session: { reasoning: false },
|
||||||
|
},
|
||||||
],
|
],
|
||||||
default: "glm-53",
|
...overrides,
|
||||||
|
};
|
||||||
|
writeFileSync(catalogFile, JSON.stringify(catalog), { mode: 0o600 });
|
||||||
|
writeFileSync(secretsFile, secrets, { mode: 0o600 });
|
||||||
|
chmodSync(catalogFile, 0o600);
|
||||||
|
chmodSync(secretsFile, 0o600);
|
||||||
|
return { catalogFile, secretsFile };
|
||||||
|
}
|
||||||
|
|
||||||
|
test("loads session default and safe metadata choices from the normalized runtime catalog", () => {
|
||||||
|
const { catalogFile, secretsFile } = runtimeCatalog();
|
||||||
|
const runtime = loadRuntimeModelCatalog(catalogFile);
|
||||||
|
const metadata = loadMetadataGenerationModels({ catalogFile, secretsFile });
|
||||||
|
|
||||||
|
expect(runtime.defaultSession).toBe("zai/glm-5.3");
|
||||||
|
expect(runtime.hasSession("deepseek/deepseek-v4-pro")).toBe(true);
|
||||||
|
expect(metadata.catalog()).toEqual({
|
||||||
|
models: [{ id: "zai/glm-5.3", label: "GLM 5.3" }],
|
||||||
|
default: "zai/glm-5.3",
|
||||||
});
|
});
|
||||||
expect(models.resolve("deepseek-v4-pro")).toMatchObject({
|
expect(metadata.resolve("zai/glm-5.3")).toEqual({
|
||||||
apiKeyEnv: "DEEPSEEK_API_KEY",
|
id: "zai/glm-5.3", provider: "openai", model: "glm-5.3",
|
||||||
apiKey: "deepseek-secret",
|
endpoint: { baseUrl: "https://api.z.ai/v1" },
|
||||||
});
|
apiKeyEnv: "OPENAI_API_KEY", apiKey: "raw-provider-secret",
|
||||||
expect(models.resolve("qwen-36")).toEqual({
|
|
||||||
id: "qwen-36",
|
|
||||||
provider: "openai",
|
|
||||||
model: "qwen3.6-35b-a3b",
|
|
||||||
disableThinking: true,
|
|
||||||
endpoint: { baseUrl: "https://models.internal.example/v1" },
|
|
||||||
});
|
});
|
||||||
|
expect(() => metadata.resolve("zai/missing")).toThrow(MetadataGenerationModelUnavailableError);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("loads an explicit keyless endpoint without a secret bundle", () => {
|
test("returns empty catalogs when no runtime projection is configured", () => {
|
||||||
const { installationFile } = metadataConfiguration(`metadataGeneration:
|
expect(loadRuntimeModelCatalog().defaultSession).toBeNull();
|
||||||
default: qwen-36
|
expect(loadMetadataGenerationModels({}).catalog()).toEqual({ models: [], default: null });
|
||||||
models:
|
});
|
||||||
- id: qwen-36
|
|
||||||
label: Qwen 3.6
|
|
||||||
litellm:
|
|
||||||
provider: openai
|
|
||||||
model: qwen3.6-35b-a3b
|
|
||||||
disableThinking: true
|
|
||||||
endpoint: {baseUrl: https://models.internal.example/v1}
|
|
||||||
`);
|
|
||||||
|
|
||||||
expect(loadMetadataGenerationModels({ installationFile }).resolve("qwen-36")).toEqual({
|
test("rejects a drifted default and an unprotected projection", () => {
|
||||||
id: "qwen-36",
|
const drifted = runtimeCatalog({ defaultSession: "zai/missing" });
|
||||||
provider: "openai",
|
expect(() => loadRuntimeModelCatalog(drifted.catalogFile)).toThrow("session default is invalid");
|
||||||
model: "qwen3.6-35b-a3b",
|
|
||||||
disableThinking: true,
|
const unprotected = runtimeCatalog();
|
||||||
endpoint: { baseUrl: "https://models.internal.example/v1" },
|
chmodSync(unprotected.catalogFile, 0o666);
|
||||||
|
expect(() => loadRuntimeModelCatalog(unprotected.catalogFile)).toThrow("runtime model catalog is unavailable");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("rejects authentication semantics that cannot come from the installation catalog", () => {
|
||||||
|
const invalid = runtimeCatalog({
|
||||||
|
defaultMetadataGeneration: undefined,
|
||||||
|
models: [{
|
||||||
|
id: "zai/glm-5.3",
|
||||||
|
provider: "zai",
|
||||||
|
model: "glm-5.3",
|
||||||
|
label: "GLM 5.3",
|
||||||
|
upstreamModel: "glm-5.3",
|
||||||
|
authentication: { mode: "secret_env" },
|
||||||
|
sessionAdapter: { mode: "pi_builtin" },
|
||||||
|
session: { reasoning: true },
|
||||||
|
}],
|
||||||
});
|
});
|
||||||
|
expect(() => loadRuntimeModelCatalog(invalid.catalogFile)).toThrow("runtime model catalog is invalid");
|
||||||
});
|
});
|
||||||
|
|
||||||
test.each([
|
test("fails closed for missing or unusable provider secrets", () => {
|
||||||
["invalid YAML", "metadataGeneration: [\n", "OPENAI_API_KEY=secret\n", /invalid YAML/],
|
const missing = runtimeCatalog({}, "THT_DWH_API_KEY=other\n");
|
||||||
["duplicate ids", `metadataGeneration:
|
expect(() => loadMetadataGenerationModels(missing)).toThrow('secret "OPENAI_API_KEY" is missing');
|
||||||
default: openai-mini
|
|
||||||
models:
|
const unusable = runtimeCatalog({}, "OPENAI_API_KEY=contains whitespace\n");
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}, apiKeyEnv: OPENAI_API_KEY}
|
expect(() => loadMetadataGenerationModels(unusable)).toThrow('secret "OPENAI_API_KEY" is unusable');
|
||||||
- {id: openai-mini, label: Two, litellm: {provider: openai, model: gpt-4.1}, apiKeyEnv: OPENAI_API_KEY}
|
|
||||||
`, "OPENAI_API_KEY=secret\n", /model id "openai-mini" is duplicated/],
|
|
||||||
["missing default", `metadataGeneration:
|
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}, apiKeyEnv: OPENAI_API_KEY}
|
|
||||||
`, "OPENAI_API_KEY=secret\n", /default is required/],
|
|
||||||
["unknown default", `metadataGeneration:
|
|
||||||
default: absent
|
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}, apiKeyEnv: OPENAI_API_KEY}
|
|
||||||
`, "OPENAI_API_KEY=secret\n", /default "absent" is not configured/],
|
|
||||||
["malformed settings", `metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: "open ai", model: gpt-4.1-mini}, apiKeyEnv: OPENAI_API_KEY}
|
|
||||||
`, "OPENAI_API_KEY=secret\n", /configuration is invalid/],
|
|
||||||
["malformed endpoint", `metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- id: openai-mini
|
|
||||||
label: One
|
|
||||||
litellm: {provider: openai, model: gpt-4.1-mini, endpoint: {baseUrl: not-a-url}}
|
|
||||||
apiKeyEnv: OPENAI_API_KEY
|
|
||||||
`, "OPENAI_API_KEY=secret\n", /configuration is invalid/],
|
|
||||||
["keyless hosted model without endpoint", `metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}}
|
|
||||||
`, "", /configuration is invalid/],
|
|
||||||
["disable thinking without endpoint", `metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- id: openai-mini
|
|
||||||
label: One
|
|
||||||
litellm: {provider: openai, model: gpt-4.1-mini, disableThinking: true}
|
|
||||||
apiKeyEnv: OPENAI_API_KEY
|
|
||||||
`, "OPENAI_API_KEY=secret\n", /configuration is invalid/],
|
|
||||||
["unallowed secret reference", `metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}, apiKeyEnv: THT_DWH_API_KEY}
|
|
||||||
`, "THT_DWH_API_KEY=secret\n", /configuration is invalid/],
|
|
||||||
["missing referenced secret", `metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}, apiKeyEnv: OPENAI_API_KEY}
|
|
||||||
`, "THT_DWH_API_KEY=secret\n", /secret "OPENAI_API_KEY" is missing/],
|
|
||||||
["unusable referenced secret", `metadataGeneration:
|
|
||||||
default: openai-mini
|
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}, apiKeyEnv: OPENAI_API_KEY}
|
|
||||||
`, "OPENAI_API_KEY=secret with whitespace\n", /secret "OPENAI_API_KEY" is unusable/],
|
|
||||||
] as const)("rejects %s metadata-generation configuration", (_name, yaml, secrets, expected) => {
|
|
||||||
const { installationFile, secretsFile } = metadataConfiguration(yaml, secrets);
|
|
||||||
expect(() => loadMetadataGenerationModels({ installationFile, secretsFile })).toThrow(expected);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("rejects a missing secret-bundle declaration for configured models", () => {
|
test("splits canonical session identities without provider aliases", () => {
|
||||||
const { installationFile } = metadataConfiguration(`metadataGeneration:
|
expect(splitCanonicalModelId("zai/glm-5.3")).toEqual({ provider: "zai", model: "glm-5.3" });
|
||||||
default: openai-mini
|
expect(() => splitCanonicalModelId("glm-5.3")).toThrow("model identity is invalid");
|
||||||
models:
|
|
||||||
- {id: openai-mini, label: One, litellm: {provider: openai, model: gpt-4.1-mini}, apiKeyEnv: OPENAI_API_KEY}
|
|
||||||
`);
|
|
||||||
|
|
||||||
expect(() => loadMetadataGenerationModels({ installationFile }))
|
|
||||||
.toThrow("metadata-generation keyed models require THT_SECRETS_FILE");
|
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
import { mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs";
|
import { mkdtempSync } from "node:fs";
|
||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { expect, test, vi } from "vitest";
|
import { expect, test, vi } from "vitest";
|
||||||
import { loadConfig } from "../src/config.js";
|
import { loadConfig } from "../src/config.js";
|
||||||
import {
|
import {
|
||||||
PiManagementError,
|
|
||||||
createPiManagement,
|
createPiManagement,
|
||||||
type PiExecFile,
|
type PiExecFile,
|
||||||
} from "../src/pi/management.js";
|
} from "../src/pi/management.js";
|
||||||
|
import type { RuntimeModelCatalog } from "../src/models/runtime-model-catalog.js";
|
||||||
|
|
||||||
function configFor(settingsFile = join(mkdtempSync(join(tmpdir(), "tht-pi-management-")), "settings.json")) {
|
function configFor(settingsFile = join(mkdtempSync(join(tmpdir(), "tht-pi-management-")), "settings.json")) {
|
||||||
return loadConfig({
|
return loadConfig({
|
||||||
@@ -18,10 +18,14 @@ function configFor(settingsFile = join(mkdtempSync(join(tmpdir(), "tht-pi-manage
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
const supportedModels = [
|
const modelCatalog: RuntimeModelCatalog = {
|
||||||
{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true },
|
defaultSession: "zai/glm-5.2",
|
||||||
{ provider: "deepseek", id: "deepseek-v4", name: "DeepSeek V4", reasoning: true },
|
defaultMetadataGeneration: null,
|
||||||
];
|
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||||
|
sessionModels: () => [],
|
||||||
|
metadataModels: () => [],
|
||||||
|
hasSession: (id) => id === "zai/glm-5.2",
|
||||||
|
};
|
||||||
|
|
||||||
function successfulExec(calls: Array<{ command: string; args: string[]; timeout: number }>): PiExecFile {
|
function successfulExec(calls: Array<{ command: string; args: string[]; timeout: number }>): PiExecFile {
|
||||||
return async (command, args, options) => {
|
return async (command, args, options) => {
|
||||||
@@ -36,7 +40,7 @@ test("status parses only a Pi version from a fixed execFile argument array", asy
|
|||||||
const calls: Array<{ command: string; args: string[]; timeout: number }> = [];
|
const calls: Array<{ command: string; args: string[]; timeout: number }> = [];
|
||||||
const service = createPiManagement(configFor(), {
|
const service = createPiManagement(configFor(), {
|
||||||
execute: successfulExec(calls),
|
execute: successfulExec(calls),
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
||||||
credentialStatus: () => "missing",
|
credentialStatus: () => "missing",
|
||||||
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
||||||
@@ -63,7 +67,7 @@ test.each(["present", "missing"] as const)(
|
|||||||
const checkedProviders: Array<string | undefined> = [];
|
const checkedProviders: Array<string | undefined> = [];
|
||||||
const service = createPiManagement(configFor(), {
|
const service = createPiManagement(configFor(), {
|
||||||
execute: successfulExec([]),
|
execute: successfulExec([]),
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
||||||
credentialStatus: (provider) => {
|
credentialStatus: (provider) => {
|
||||||
checkedProviders.push(provider);
|
checkedProviders.push(provider);
|
||||||
@@ -85,100 +89,6 @@ test.each(["present", "missing"] as const)(
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
// Catches an options response that leaks provider metadata or lets callers choose model IDs that
|
|
||||||
// Pi did not explicitly enable for this installation.
|
|
||||||
test("options expose only closed provider, model, and reasoning choices", async () => {
|
|
||||||
const service = createPiManagement(configFor(), {
|
|
||||||
execute: successfulExec([]),
|
|
||||||
listModels: async () => supportedModels,
|
|
||||||
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
|
||||||
});
|
|
||||||
|
|
||||||
await expect(service.options()).resolves.toEqual({
|
|
||||||
providers: ["zai", "deepseek"],
|
|
||||||
models: [
|
|
||||||
{ provider: "zai", id: "glm-5.2" },
|
|
||||||
{ provider: "deepseek", id: "deepseek-v4" },
|
|
||||||
],
|
|
||||||
reasoning: ["low", "medium", "high"],
|
|
||||||
checkedAt: "2026-08-05T10:00:00.000Z",
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// Catches raw managed models.json validation details being collapsed into an ambiguous model-list
|
|
||||||
// failure or escaping through the Pi Management options API.
|
|
||||||
test("options report invalid managed model configuration with a stable sanitized error", async () => {
|
|
||||||
const service = createPiManagement(configFor(), {
|
|
||||||
execute: successfulExec([]),
|
|
||||||
listModels: async () => {
|
|
||||||
throw Object.assign(
|
|
||||||
new Error("!sensitive-command /private/models.json raw-secret"),
|
|
||||||
{ code: "PI_MANAGED_CONFIG_INVALID" },
|
|
||||||
);
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
let caught: unknown;
|
|
||||||
try {
|
|
||||||
await service.options();
|
|
||||||
} catch (error) {
|
|
||||||
caught = error;
|
|
||||||
}
|
|
||||||
expect(caught).toMatchObject<PiManagementError>({
|
|
||||||
code: "pi_management_unavailable",
|
|
||||||
message: "Pi provider/model configuration is invalid",
|
|
||||||
});
|
|
||||||
expect(String(caught)).not.toMatch(/sensitive|private|models\.json|secret/i);
|
|
||||||
});
|
|
||||||
|
|
||||||
// Catches configuration writes that accept whitespace, unknown choices, or extra free-form fields
|
|
||||||
// before reaching the durable installation settings file.
|
|
||||||
test("config rejects invalid free-form values before writing settings", async () => {
|
|
||||||
const directory = mkdtempSync(join(tmpdir(), "tht-pi-management-invalid-"));
|
|
||||||
try {
|
|
||||||
let writes = 0;
|
|
||||||
const service = createPiManagement(configFor(join(directory, "settings.json")), {
|
|
||||||
execute: successfulExec([]),
|
|
||||||
listModels: async () => supportedModels,
|
|
||||||
readSettings: () => ({}),
|
|
||||||
saveSettings: () => { writes += 1; return {}; },
|
|
||||||
});
|
|
||||||
|
|
||||||
await expect(service.configure({
|
|
||||||
provider: "zai ", model: "glm-5.2", reasoning: "medium", unexpected: "value",
|
|
||||||
} as any)).rejects.toMatchObject<PiManagementError>({ code: "pi_management_invalid_config" });
|
|
||||||
expect(writes).toBe(0);
|
|
||||||
} finally {
|
|
||||||
rmSync(directory, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Catches a non-atomic implementation that can leave partial settings or temporary files after a
|
|
||||||
// normal installation-default update.
|
|
||||||
test("config validates closed choices and atomically persists non-secret defaults", async () => {
|
|
||||||
const directory = mkdtempSync(join(tmpdir(), "tht-pi-management-write-"));
|
|
||||||
const settingsFile = join(directory, "settings.json");
|
|
||||||
try {
|
|
||||||
const service = createPiManagement(configFor(settingsFile), {
|
|
||||||
execute: successfulExec([]),
|
|
||||||
listModels: async () => supportedModels,
|
|
||||||
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
|
||||||
});
|
|
||||||
|
|
||||||
await expect(service.configure({
|
|
||||||
provider: "zai", model: "glm-5.2", reasoning: "high",
|
|
||||||
})).resolves.toEqual({
|
|
||||||
provider: "zai", model: "glm-5.2", reasoning: "high", updatedAt: "2026-08-05T10:00:00.000Z",
|
|
||||||
});
|
|
||||||
expect(JSON.parse(readFileSync(settingsFile, "utf8"))).toEqual({
|
|
||||||
provider: "zai", model: "glm-5.2", thinking: "high",
|
|
||||||
});
|
|
||||||
expect(readdirSync(directory)).toEqual(["settings.json"]);
|
|
||||||
} finally {
|
|
||||||
rmSync(directory, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// Catches a hung Pi smoke check that leaves an operator waiting indefinitely or returns raw child
|
// Catches a hung Pi smoke check that leaves an operator waiting indefinitely or returns raw child
|
||||||
// diagnostics containing provider credentials.
|
// diagnostics containing provider credentials.
|
||||||
test("smoke uses the configured timeout and reports a sanitized timeout", async () => {
|
test("smoke uses the configured timeout and reports a sanitized timeout", async () => {
|
||||||
@@ -188,7 +98,7 @@ test("smoke uses the configured timeout and reports a sanitized timeout", async
|
|||||||
calls.push({ command, args, timeout: options.timeout });
|
calls.push({ command, args, timeout: options.timeout });
|
||||||
throw Object.assign(new Error("provider token=raw-provider-token"), { code: "ETIMEDOUT" });
|
throw Object.assign(new Error("provider token=raw-provider-token"), { code: "ETIMEDOUT" });
|
||||||
},
|
},
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
||||||
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
||||||
});
|
});
|
||||||
@@ -210,7 +120,7 @@ test("smoke exercises the configured provider and model", async () => {
|
|||||||
const providerChecks: unknown[] = [];
|
const providerChecks: unknown[] = [];
|
||||||
const service = createPiManagement(configFor(), {
|
const service = createPiManagement(configFor(), {
|
||||||
execute: successfulExec([]),
|
execute: successfulExec([]),
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
||||||
smokeProvider: async (request) => { providerChecks.push(request); },
|
smokeProvider: async (request) => { providerChecks.push(request); },
|
||||||
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
||||||
@@ -230,7 +140,7 @@ test("smoke exercises the configured provider and model", async () => {
|
|||||||
test("smoke fails closed and sanitizes configured-provider authentication errors", async () => {
|
test("smoke fails closed and sanitizes configured-provider authentication errors", async () => {
|
||||||
const service = createPiManagement(configFor(), {
|
const service = createPiManagement(configFor(), {
|
||||||
execute: successfulExec([]),
|
execute: successfulExec([]),
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
||||||
smokeProvider: async () => {
|
smokeProvider: async () => {
|
||||||
throw new Error('401 {"token":"raw-expired-token","output":"raw-provider-output"}');
|
throw new Error('401 {"token":"raw-expired-token","output":"raw-provider-output"}');
|
||||||
@@ -252,7 +162,7 @@ test("smoke fails closed and sanitizes configured-provider authentication errors
|
|||||||
test("smoke reports invalid managed provider configuration with a stable sanitized error", async () => {
|
test("smoke reports invalid managed provider configuration with a stable sanitized error", async () => {
|
||||||
const service = createPiManagement(configFor(), {
|
const service = createPiManagement(configFor(), {
|
||||||
execute: successfulExec([]),
|
execute: successfulExec([]),
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
||||||
smokeProvider: async () => {
|
smokeProvider: async () => {
|
||||||
throw Object.assign(
|
throw Object.assign(
|
||||||
@@ -284,7 +194,7 @@ test("smoke applies one deadline across version and a hung provider turn", async
|
|||||||
() => resolve({ stdout: "pi 0.80.3\n", stderr: "" }),
|
() => resolve({ stdout: "pi 0.80.3\n", stderr: "" }),
|
||||||
500,
|
500,
|
||||||
)),
|
)),
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
readSettings: () => ({ provider: "zai", model: "glm-5.2", thinking: "medium" }),
|
||||||
smokeProvider: async ({ timeoutMs }) => {
|
smokeProvider: async ({ timeoutMs }) => {
|
||||||
providerTimeouts.push(timeoutMs);
|
providerTimeouts.push(timeoutMs);
|
||||||
@@ -320,7 +230,7 @@ test("logs keep only the latest 200 redacted lines", async () => {
|
|||||||
source[201] = "THT_MODEL_API_KEY=raw-env-secret";
|
source[201] = "THT_MODEL_API_KEY=raw-env-secret";
|
||||||
const service = createPiManagement(configFor(), {
|
const service = createPiManagement(configFor(), {
|
||||||
execute: successfulExec([]),
|
execute: successfulExec([]),
|
||||||
listModels: async () => supportedModels,
|
modelCatalog,
|
||||||
readLogs: () => source.join("\n"),
|
readLogs: () => source.join("\n"),
|
||||||
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
now: () => new Date("2026-08-05T10:00:00.000Z"),
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import {
|
|||||||
} from "node:fs";
|
} from "node:fs";
|
||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { PiProcessManager } from "../src/pi/pi-process-manager.js";
|
import { PiProcessManager } from "../src/pi/pi-process-manager.js";
|
||||||
|
import type { RuntimeModelCatalog, RuntimeModel } from "../src/models/runtime-model-catalog.js";
|
||||||
import { loadConfig } from "../src/config.js";
|
import { loadConfig } from "../src/config.js";
|
||||||
import {
|
import {
|
||||||
PI_MANAGED_CONFIG_ERROR_MESSAGE,
|
PI_MANAGED_CONFIG_ERROR_MESSAGE,
|
||||||
@@ -641,6 +642,53 @@ test("session Pi spawn reads the single secret bundle and scrubs its path", asyn
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("session Pi spawn resolves the selected catalog credential from the secret bundle", () => {
|
||||||
|
const root = mkdtempSync(path.join(tmpdir(), "thothii-catalog-credential-"));
|
||||||
|
const agentDir = path.join(root, "agent");
|
||||||
|
mkdirSync(agentDir, { mode: 0o700 });
|
||||||
|
writeFileSync(path.join(agentDir, "auth.json"), "{}\n", { mode: 0o600 });
|
||||||
|
writeFileSync(path.join(agentDir, "models.json"), '{"providers":{}}\n', { mode: 0o600 });
|
||||||
|
const secret = path.join(root, "thothii.secrets");
|
||||||
|
writeFileSync(secret, "ZAI_API_KEY=catalog-secret\nTHT_MODEL_API_KEY=legacy-secret\n", { mode: 0o600 });
|
||||||
|
const model: RuntimeModel = {
|
||||||
|
id: "openai/test-model",
|
||||||
|
provider: "openai",
|
||||||
|
model: "test-model",
|
||||||
|
label: "Test model",
|
||||||
|
upstreamModel: "test-model",
|
||||||
|
authentication: { mode: "secret_env", apiKeyEnv: "ZAI_API_KEY" },
|
||||||
|
sessionAdapter: { mode: "pi_builtin" },
|
||||||
|
session: { reasoning: false },
|
||||||
|
};
|
||||||
|
const modelCatalog: RuntimeModelCatalog = {
|
||||||
|
defaultSession: model.id,
|
||||||
|
defaultMetadataGeneration: null,
|
||||||
|
embedding: null,
|
||||||
|
sessionModels: () => [model],
|
||||||
|
metadataModels: () => [],
|
||||||
|
hasSession: (id) => id === model.id,
|
||||||
|
};
|
||||||
|
const calls: any[][] = [];
|
||||||
|
const child = recordingChild();
|
||||||
|
child.stderr.resume = () => {};
|
||||||
|
vi.stubEnv("PI_CODING_AGENT_DIR", agentDir);
|
||||||
|
const mgr = new PiProcessManager(loadConfig({ THT_SECRETS_FILE: secret }), {
|
||||||
|
modelCatalog,
|
||||||
|
authProviders: () => new Set(),
|
||||||
|
spawnFn: (...args: any[]) => { calls.push(args); return child as any; },
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
mgr.createFor("catalog-credential", { provider: "openai", model: "test-model" });
|
||||||
|
expect(calls[0][2].env.ZAI_API_KEY).toBe("catalog-secret");
|
||||||
|
expect(calls[0][2].env).not.toHaveProperty("OPENAI_API_KEY");
|
||||||
|
expect(calls[0][2].env).not.toHaveProperty("THT_MODEL_API_KEY");
|
||||||
|
} finally {
|
||||||
|
mgr.teardown("catalog-credential");
|
||||||
|
vi.unstubAllEnvs();
|
||||||
|
rmSync(root, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
test.each([["OpenAI", "openai"], ["gemini", "google"]])(
|
test.each([["OpenAI", "openai"], ["gemini", "google"]])(
|
||||||
"set_model uses canonical packaged provider ID for %s", async (provider, canonical) => {
|
"set_model uses canonical packaged provider ID for %s", async (provider, canonical) => {
|
||||||
const secret = path.resolve(__dirname, `.canonical-key-${process.pid}-${provider}`);
|
const secret = path.resolve(__dirname, `.canonical-key-${process.pid}-${provider}`);
|
||||||
|
|||||||
@@ -1,9 +1,11 @@
|
|||||||
import { EventEmitter } from "node:events";
|
import { EventEmitter } from "node:events";
|
||||||
import { existsSync, readFileSync, readdirSync } from "node:fs";
|
import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs";
|
||||||
import { dirname } from "node:path";
|
import { tmpdir } from "node:os";
|
||||||
|
import { dirname, join } from "node:path";
|
||||||
import { afterEach, expect, test, vi } from "vitest";
|
import { afterEach, expect, test, vi } from "vitest";
|
||||||
import { loadConfig } from "../src/config.js";
|
import { loadConfig } from "../src/config.js";
|
||||||
import { createPiProviderSmoke } from "../src/pi/provider-smoke.js";
|
import { createPiProviderSmoke } from "../src/pi/provider-smoke.js";
|
||||||
|
import type { RuntimeModel, RuntimeModelCatalog } from "../src/models/runtime-model-catalog.js";
|
||||||
|
|
||||||
afterEach(() => vi.unstubAllEnvs());
|
afterEach(() => vi.unstubAllEnvs());
|
||||||
|
|
||||||
@@ -44,6 +46,50 @@ function successfulProviderChild() {
|
|||||||
|
|
||||||
const MANAGED_CONFIG_ERROR = "Pi provider/model configuration is invalid";
|
const MANAGED_CONFIG_ERROR = "Pi provider/model configuration is invalid";
|
||||||
|
|
||||||
|
test("provider smoke resolves the selected catalog credential from the secret bundle", async () => {
|
||||||
|
const root = mkdtempSync(join(tmpdir(), "thothii-smoke-catalog-credential-"));
|
||||||
|
const secret = join(root, "thothii.secrets");
|
||||||
|
writeFileSync(secret, "ZAI_API_KEY=catalog-secret\nTHT_MODEL_API_KEY=legacy-secret\n", { mode: 0o600 });
|
||||||
|
const model: RuntimeModel = {
|
||||||
|
id: "openai/test-model",
|
||||||
|
provider: "openai",
|
||||||
|
model: "test-model",
|
||||||
|
label: "Test model",
|
||||||
|
upstreamModel: "test-model",
|
||||||
|
authentication: { mode: "secret_env", apiKeyEnv: "ZAI_API_KEY" },
|
||||||
|
sessionAdapter: { mode: "pi_builtin" },
|
||||||
|
session: { reasoning: false },
|
||||||
|
};
|
||||||
|
const modelCatalog: RuntimeModelCatalog = {
|
||||||
|
defaultSession: model.id,
|
||||||
|
defaultMetadataGeneration: null,
|
||||||
|
embedding: null,
|
||||||
|
sessionModels: () => [model],
|
||||||
|
metadataModels: () => [],
|
||||||
|
hasSession: (id) => id === model.id,
|
||||||
|
};
|
||||||
|
let spawnEnv: NodeJS.ProcessEnv | undefined;
|
||||||
|
const smoke = createPiProviderSmoke(loadConfig({ THT_SECRETS_FILE: secret }), {
|
||||||
|
modelCatalog,
|
||||||
|
authProviders: () => new Set(),
|
||||||
|
readModelsStore: () => undefined,
|
||||||
|
spawnFn: (_command, _args, options) => {
|
||||||
|
spawnEnv = options.env;
|
||||||
|
return successfulProviderChild();
|
||||||
|
},
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
await expect(smoke({
|
||||||
|
provider: "openai", model: "test-model", reasoning: "medium", timeoutMs: 750,
|
||||||
|
})).resolves.toBeUndefined();
|
||||||
|
expect(spawnEnv?.ZAI_API_KEY).toBe("catalog-secret");
|
||||||
|
expect(spawnEnv).not.toHaveProperty("OPENAI_API_KEY");
|
||||||
|
expect(spawnEnv).not.toHaveProperty("THT_MODEL_API_KEY");
|
||||||
|
} finally {
|
||||||
|
rmSync(root, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
// Catches an isolated smoke agent that copies auth.json but drops the selected custom
|
// Catches an isolated smoke agent that copies auth.json but drops the selected custom
|
||||||
// provider/model from models.json, causing set_model to fail before the real request.
|
// provider/model from models.json, causing set_model to fail before the real request.
|
||||||
test("provider smoke reaches the selected custom provider from an isolated models.json", async () => {
|
test("provider smoke reaches the selected custom provider from an isolated models.json", async () => {
|
||||||
|
|||||||
@@ -2,18 +2,11 @@ import { expect, test } from "vitest";
|
|||||||
import { ReadinessManager } from "../src/runtime/readiness-manager.js";
|
import { ReadinessManager } from "../src/runtime/readiness-manager.js";
|
||||||
|
|
||||||
const workspace = {
|
const workspace = {
|
||||||
workspace: { schema_version: 3, id: "psd", name: "PSD", language: "it" },
|
workspace: { schema_version: 4, id: "psd", name: "PSD", language: "it" },
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres", database: "warehouse", schema: "public",
|
engine: "postgres", database: "warehouse", schema: "public",
|
||||||
supported_transports: ["postgres_direct"],
|
supported_transports: ["postgres_direct"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: "psd", dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: {
|
|
||||||
provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
} as const;
|
} as const;
|
||||||
|
|
||||||
function deferred<T>() {
|
function deferred<T>() {
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ afterEach(() => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
const validYaml = `workspace:
|
const validYaml = `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -24,18 +24,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`;
|
`;
|
||||||
|
|
||||||
async function git(cwd: string, args: string[]): Promise<string> {
|
async function git(cwd: string, args: string[]): Promise<string> {
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ async function git(cwd: string, args: string[]): Promise<string> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const descriptor = `workspace:
|
const descriptor = `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: research
|
id: research
|
||||||
name: Research
|
name: Research
|
||||||
language: en
|
language: en
|
||||||
@@ -29,18 +29,6 @@ dwh:
|
|||||||
database: analytics
|
database: analytics
|
||||||
schema: mart
|
schema: mart
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: research
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
evidence:
|
evidence:
|
||||||
source:
|
source:
|
||||||
type: filesystem
|
type: filesystem
|
||||||
|
|||||||
@@ -14,11 +14,6 @@ function fakeService(): PiManagementService {
|
|||||||
config: { provider: "zai", model: "glm-5.2", reasoning: "medium" },
|
config: { provider: "zai", model: "glm-5.2", reasoning: "medium" },
|
||||||
checkedAt: "2026-08-05T10:00:00.000Z",
|
checkedAt: "2026-08-05T10:00:00.000Z",
|
||||||
})),
|
})),
|
||||||
options: vi.fn(async () => ({
|
|
||||||
providers: ["zai"], models: [{ provider: "zai", id: "glm-5.2" }],
|
|
||||||
reasoning: ["low", "medium", "high"], checkedAt: "2026-08-05T10:00:00.000Z",
|
|
||||||
})),
|
|
||||||
configure: vi.fn(async (value) => ({ ...value, updatedAt: "2026-08-05T10:00:00.000Z" })),
|
|
||||||
test: vi.fn(async () => ({ ready: true, checkedAt: "2026-08-05T10:00:00.000Z" })),
|
test: vi.fn(async () => ({ ready: true, checkedAt: "2026-08-05T10:00:00.000Z" })),
|
||||||
logs: vi.fn(async () => ({ lines: ["Pi smoke check succeeded"], checkedAt: "2026-08-05T10:00:00.000Z" })),
|
logs: vi.fn(async () => ({ lines: ["Pi smoke check succeeded"], checkedAt: "2026-08-05T10:00:00.000Z" })),
|
||||||
};
|
};
|
||||||
@@ -106,24 +101,17 @@ test("loopback-only AUTH_MODE=none may read the sanitized Pi status", async () =
|
|||||||
});
|
});
|
||||||
|
|
||||||
// A local implicit administrator has pi.manage, but a browser origin still cannot borrow that
|
// A local implicit administrator has pi.manage, but a browser origin still cannot borrow that
|
||||||
// authority to mutate local configuration or trigger provider work.
|
// authority to trigger provider work.
|
||||||
test("loopback-only management rejects cross-origin writes for its local administrator", async () => {
|
test("loopback-only management rejects cross-origin writes for its local administrator", async () => {
|
||||||
const service = fakeService();
|
const service = fakeService();
|
||||||
const app = appWith(service);
|
const app = appWith(service);
|
||||||
try {
|
try {
|
||||||
const configured = await app.inject({
|
|
||||||
method: "PUT", url: "/pi-management/config",
|
|
||||||
headers: { host: "127.0.0.1:8080", origin: "https://evil.example" },
|
|
||||||
payload: { provider: "zai", model: "glm-5.2", reasoning: "high" },
|
|
||||||
});
|
|
||||||
const smoke = await app.inject({
|
const smoke = await app.inject({
|
||||||
method: "POST", url: "/pi-management/test",
|
method: "POST", url: "/pi-management/test",
|
||||||
headers: { host: "127.0.0.1:8080", origin: "https://evil.example" },
|
headers: { host: "127.0.0.1:8080", origin: "https://evil.example" },
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(configured.statusCode).toBe(403);
|
|
||||||
expect(smoke.statusCode).toBe(403);
|
expect(smoke.statusCode).toBe(403);
|
||||||
expect(service.configure).not.toHaveBeenCalled();
|
|
||||||
expect(service.test).not.toHaveBeenCalled();
|
expect(service.test).not.toHaveBeenCalled();
|
||||||
} finally {
|
} finally {
|
||||||
await app.close();
|
await app.close();
|
||||||
@@ -132,14 +120,13 @@ test("loopback-only management rejects cross-origin writes for its local adminis
|
|||||||
|
|
||||||
// Catches an origin guard that also blocks the same-origin Docker frontend or non-browser local
|
// Catches an origin guard that also blocks the same-origin Docker frontend or non-browser local
|
||||||
// lifecycle clients that do not send Origin.
|
// lifecycle clients that do not send Origin.
|
||||||
test("loopback-only management preserves same-origin frontend and origin-less local writes", async () => {
|
test("loopback-only management preserves same-origin and origin-less smoke checks", async () => {
|
||||||
const service = fakeService();
|
const service = fakeService();
|
||||||
const app = appWith(service);
|
const app = appWith(service);
|
||||||
try {
|
try {
|
||||||
const sameOrigin = await app.inject({
|
const sameOrigin = await app.inject({
|
||||||
method: "PUT", url: "/pi-management/config",
|
method: "POST", url: "/pi-management/test",
|
||||||
headers: { host: "127.0.0.1:8080", origin: "http://127.0.0.1:8080" },
|
headers: { host: "127.0.0.1:8080", origin: "http://127.0.0.1:8080" },
|
||||||
payload: { provider: "zai", model: "glm-5.2", reasoning: "high" },
|
|
||||||
});
|
});
|
||||||
const lifecycleClient = await app.inject({ method: "POST", url: "/pi-management/test" });
|
const lifecycleClient = await app.inject({ method: "POST", url: "/pi-management/test" });
|
||||||
|
|
||||||
@@ -150,25 +137,20 @@ test("loopback-only management preserves same-origin frontend and origin-less lo
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Catches route wiring that bypasses closed service validation or gives the browser a Docker/image
|
// Catches a regression that reintroduces a browser-writable provider/model source.
|
||||||
// lifecycle endpoint rather than only installation-default configuration and diagnostics.
|
test("trusted admins receive only status, smoke, and log endpoints", async () => {
|
||||||
test("trusted admins receive only configuration, smoke, options, and log endpoints", async () => {
|
|
||||||
const service = fakeService();
|
const service = fakeService();
|
||||||
const app = appWith(service, exposedServerEnv);
|
const app = appWith(service, exposedServerEnv);
|
||||||
try {
|
try {
|
||||||
const options = await app.inject({ method: "GET", url: "/pi-management/options", headers: adminHeaders });
|
const status = await app.inject({ method: "GET", url: "/pi-management/status", headers: adminHeaders });
|
||||||
const configured = await app.inject({
|
|
||||||
method: "PUT", url: "/pi-management/config", headers: adminHeaders,
|
|
||||||
payload: { provider: "zai", model: "glm-5.2", reasoning: "high" },
|
|
||||||
});
|
|
||||||
const smoke = await app.inject({ method: "POST", url: "/pi-management/test", headers: adminHeaders });
|
const smoke = await app.inject({ method: "POST", url: "/pi-management/test", headers: adminHeaders });
|
||||||
const logs = await app.inject({ method: "GET", url: "/pi-management/logs", headers: adminHeaders });
|
const logs = await app.inject({ method: "GET", url: "/pi-management/logs", headers: adminHeaders });
|
||||||
|
|
||||||
expect(options.statusCode).toBe(200);
|
expect(status.statusCode).toBe(200);
|
||||||
expect(configured.statusCode).toBe(200);
|
|
||||||
expect(configured.json()).toMatchObject({ provider: "zai", model: "glm-5.2", reasoning: "high" });
|
|
||||||
expect(smoke.statusCode).toBe(200);
|
expect(smoke.statusCode).toBe(200);
|
||||||
expect(logs.statusCode).toBe(200);
|
expect(logs.statusCode).toBe(200);
|
||||||
|
expect((await app.inject({ method: "GET", url: "/pi-management/options", headers: adminHeaders })).statusCode).toBe(404);
|
||||||
|
expect((await app.inject({ method: "PUT", url: "/pi-management/config", headers: adminHeaders })).statusCode).toBe(404);
|
||||||
expect(app.printRoutes()).not.toContain("update");
|
expect(app.printRoutes()).not.toContain("update");
|
||||||
expect(app.printRoutes()).not.toContain("rollback");
|
expect(app.printRoutes()).not.toContain("rollback");
|
||||||
} finally {
|
} finally {
|
||||||
|
|||||||
@@ -18,25 +18,25 @@ const SCRIPT = path.resolve("../harness/tests/fake_pi/scripts/f1_disambiguation.
|
|||||||
|
|
||||||
function operationalWorkspace(id = "default") {
|
function operationalWorkspace(id = "default") {
|
||||||
return {
|
return {
|
||||||
workspace: { schema_version: 3, id, name: id, language: "en" },
|
workspace: { schema_version: 4, id, name: id, language: "en" },
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres", database: "warehouse", schema: "public",
|
engine: "postgres", database: "warehouse", schema: "public",
|
||||||
supported_transports: ["postgres_direct"],
|
supported_transports: ["postgres_direct"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: {
|
|
||||||
engine: "qdrant", collection: id, dimensions: 1024, distance: "cosine",
|
|
||||||
},
|
|
||||||
embedding: {
|
|
||||||
provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
llm_policy: {
|
|
||||||
allowed: ["zai/glm-5.2", "deepseek/deepseek-v4-pro", "local-qwen/qwen3.6-35b-a3b"],
|
|
||||||
},
|
|
||||||
} as const;
|
} as const;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function sessionCatalog(defaultSession = "zai/glm-5.2", available = [defaultSession]) {
|
||||||
|
return {
|
||||||
|
defaultSession,
|
||||||
|
defaultMetadataGeneration: null,
|
||||||
|
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||||
|
sessionModels: () => [],
|
||||||
|
metadataModels: () => [],
|
||||||
|
hasSession: (id: string) => available.includes(id),
|
||||||
|
} as any;
|
||||||
|
}
|
||||||
|
|
||||||
const defaultWorkspaceRegistry = {
|
const defaultWorkspaceRegistry = {
|
||||||
list: vi.fn(async () => [{
|
list: vi.fn(async () => [{
|
||||||
id: "default", commit: "e".repeat(40), blob: "f".repeat(40),
|
id: "default", commit: "e".repeat(40), blob: "f".repeat(40),
|
||||||
@@ -495,8 +495,8 @@ test("creates a session from the active immutable workspace revision", async ()
|
|||||||
workspaceRegistry: {
|
workspaceRegistry: {
|
||||||
read: vi.fn(async () => ({
|
read: vi.fn(async () => ({
|
||||||
workspace: {
|
workspace: {
|
||||||
workspace: { schema_version: 2, id: "psd-clinical", name: "PSD", language: "it" },
|
workspace: { schema_version: 4, id: "psd-clinical", name: "PSD", language: "it" },
|
||||||
dwh: {}, semantic_index: {}, llm_policy: { allowed: ["zai/glm-5.2"] },
|
dwh: {},
|
||||||
},
|
},
|
||||||
revision: {
|
revision: {
|
||||||
id: "psd-clinical", commit: "a".repeat(40), blob: "b".repeat(40),
|
id: "psd-clinical", commit: "a".repeat(40), blob: "b".repeat(40),
|
||||||
@@ -589,22 +589,11 @@ test("rejects an SSH-only workspace before persisting or starting a session", as
|
|||||||
workspaceRegistry: {
|
workspaceRegistry: {
|
||||||
acquireSessionRevision: vi.fn(async () => ({
|
acquireSessionRevision: vi.fn(async () => ({
|
||||||
workspace: {
|
workspace: {
|
||||||
workspace: { schema_version: 2, id: "ssh-workspace", name: "SSH", language: "en" },
|
workspace: { schema_version: 4, id: "ssh-workspace", name: "SSH", language: "en" },
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres", database: "postgres", schema: "public",
|
engine: "postgres", database: "postgres", schema: "public",
|
||||||
supported_transports: ["ssh_tunnel"],
|
supported_transports: ["ssh_tunnel"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: {
|
|
||||||
engine: "pgvector", database: "postgres", schema: "vectors",
|
|
||||||
collection: "documents", dimensions: 768, distance: "cosine",
|
|
||||||
supported_transports: ["ssh_tunnel"],
|
|
||||||
},
|
|
||||||
embedding: {
|
|
||||||
provider: "ollama_compatible", model: "nomic-embed-text", dimensions: 768,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
},
|
},
|
||||||
revision: {
|
revision: {
|
||||||
id: "ssh-workspace", commit: "a".repeat(40), blob: "b".repeat(40),
|
id: "ssh-workspace", commit: "a".repeat(40), blob: "b".repeat(40),
|
||||||
@@ -632,7 +621,7 @@ test("hands a revision lease to retention only after the session manifest is dur
|
|||||||
const markPersisted = vi.fn(async () => {});
|
const markPersisted = vi.fn(async () => {});
|
||||||
const abort = vi.fn(async () => {});
|
const abort = vi.fn(async () => {});
|
||||||
const acquireSessionRevision = vi.fn(async () => ({
|
const acquireSessionRevision = vi.fn(async () => ({
|
||||||
workspace: { llm_policy: { allowed: ["zai/glm-5.2"] } },
|
workspace: operationalWorkspace("leased"),
|
||||||
revision: {
|
revision: {
|
||||||
id: "leased", commit: "a".repeat(40), blob: "b".repeat(40),
|
id: "leased", commit: "a".repeat(40), blob: "b".repeat(40),
|
||||||
snapshotPath: `/data/workspace-registry/snapshots/${"a".repeat(40)}/leased.yaml`,
|
snapshotPath: `/data/workspace-registry/snapshots/${"a".repeat(40)}/leased.yaml`,
|
||||||
@@ -670,7 +659,7 @@ test("creates a session from the configured default workspace revision when work
|
|||||||
const sessionNew = vi.fn(async () => ({ id: "default-pinned" }));
|
const sessionNew = vi.fn(async () => ({ id: "default-pinned" }));
|
||||||
const registry = {
|
const registry = {
|
||||||
read: vi.fn(async (id: string) => ({
|
read: vi.fn(async (id: string) => ({
|
||||||
workspace: { llm_policy: { allowed: ["zai/glm-5.2"] } },
|
workspace: operationalWorkspace(id),
|
||||||
revision: {
|
revision: {
|
||||||
id, commit: "c".repeat(40), blob: "d".repeat(40),
|
id, commit: "c".repeat(40), blob: "d".repeat(40),
|
||||||
snapshotPath: `/data/workspace-registry/snapshots/${"c".repeat(40)}/${id}.yaml`,
|
snapshotPath: `/data/workspace-registry/snapshots/${"c".repeat(40)}/${id}.yaml`,
|
||||||
@@ -758,7 +747,7 @@ test("session lifecycle locates a B session when installation default is A", asy
|
|||||||
listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }],
|
listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }],
|
||||||
workspaceRegistry: {
|
workspaceRegistry: {
|
||||||
read: async (id: string) => ({
|
read: async (id: string) => ({
|
||||||
workspace: { llm_policy: { allowed: ["zai/glm-5.2"] } },
|
workspace: operationalWorkspace(id),
|
||||||
revision: { id, commit: "b".repeat(40), blob: "d".repeat(40), snapshotPath: bPath },
|
revision: { id, commit: "b".repeat(40), blob: "d".repeat(40), snapshotPath: bPath },
|
||||||
}),
|
}),
|
||||||
list: async () => [
|
list: async () => [
|
||||||
@@ -793,7 +782,7 @@ test("session lifecycle locates a B session when installation default is A", asy
|
|||||||
expect(runtimeOptions).toEqual(["/runtime/1.yaml", "/runtime/2.yaml"]);
|
expect(runtimeOptions).toEqual(["/runtime/1.yaml", "/runtime/2.yaml"]);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("POST /sessions usa i settings (workspace/provider/model/thinking) e crea+avvia", async () => {
|
test("POST /sessions uses the catalog default with workspace/thinking settings and starts", async () => {
|
||||||
const modelKey = path.join(os.tmpdir(), `thoth-model-key-${process.pid}`);
|
const modelKey = path.join(os.tmpdir(), `thoth-model-key-${process.pid}`);
|
||||||
writeFileSync(modelKey, "test-model-key", { mode: 0o600 });
|
writeFileSync(modelKey, "test-model-key", { mode: 0o600 });
|
||||||
chmodSync(modelKey, 0o600);
|
chmodSync(modelKey, 0o600);
|
||||||
@@ -808,7 +797,8 @@ test("POST /sessions usa i settings (workspace/provider/model/thinking) e crea+a
|
|||||||
sessionNew: async (o: any) => { sessionNewArg = o; return { id: "s1" }; },
|
sessionNew: async (o: any) => { sessionNewArg = o; return { id: "s1" }; },
|
||||||
sessionList: async () => [{ id: "s1" }],
|
sessionList: async () => [{ id: "s1" }],
|
||||||
} as any,
|
} as any,
|
||||||
getSettings: () => ({ workspace: "w", provider: "zai", model: "glm-5.2", thinking: "high" }),
|
getSettings: () => ({ workspace: "w", thinking: "high" }),
|
||||||
|
runtimeModelCatalog: sessionCatalog(),
|
||||||
listModels: async () => [
|
listModels: async () => [
|
||||||
{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true },
|
{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true },
|
||||||
],
|
],
|
||||||
@@ -2565,32 +2555,43 @@ test("POST /sessions proceeds when ollamaEnsure succeeds", async () => {
|
|||||||
expect(ensureWs).toContain(`/snapshots/${"e".repeat(40)}/psd.yaml`);
|
expect(ensureWs).toContain(`/snapshots/${"e".repeat(40)}/psd.yaml`);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("POST /sessions rejects an unavailable saved model before persisting a session", async () => {
|
test("POST /sessions falls back from a stale requested model to the catalog default", async () => {
|
||||||
let created = 0;
|
let created = 0;
|
||||||
|
let persisted: any;
|
||||||
|
const runtime = { bridge: { onClientEvent: () => {} } };
|
||||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||||
thtRunner: {
|
thtRunner: {
|
||||||
sessionNew: async () => { created += 1; return { id: "must-not-exist" }; },
|
sessionNew: async (options: any) => { created += 1; persisted = options; return { id: "fallback" }; },
|
||||||
|
searchPack: async () => {},
|
||||||
} as any,
|
} as any,
|
||||||
readiness: { ensure: async () => ({ ok: true }) } as any,
|
readiness: { ensure: async () => ({ ok: true }) } as any,
|
||||||
getSettings: () => ({
|
getSettings: () => ({ workspace: "psd", thinking: "medium" }) as any,
|
||||||
workspace: "psd",
|
runtimeModelCatalog: sessionCatalog(),
|
||||||
provider: "deepseek",
|
|
||||||
model: "deepseek-v4-pro",
|
|
||||||
thinking: "medium",
|
|
||||||
}) as any,
|
|
||||||
listModels: async () => [
|
listModels: async () => [
|
||||||
{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true },
|
{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true },
|
||||||
],
|
],
|
||||||
|
mgr: {
|
||||||
|
teardownForPrincipal: () => [],
|
||||||
|
createFor: () => runtime,
|
||||||
|
get: () => runtime,
|
||||||
|
configure: async () => {},
|
||||||
|
start: () => {},
|
||||||
|
} as any,
|
||||||
});
|
});
|
||||||
|
|
||||||
const res = await app.inject({ method: "POST", url: "/sessions", payload: { question: "q" } });
|
const res = await app.inject({
|
||||||
|
method: "POST",
|
||||||
|
url: "/sessions",
|
||||||
|
payload: { question: "q", provider: "deepseek", model: "deepseek-v4-pro" },
|
||||||
|
});
|
||||||
|
|
||||||
expect(res.statusCode).toBe(503);
|
expect(res.statusCode).toBe(200);
|
||||||
expect(res.json()).toEqual({
|
expect(res.json()).toEqual({
|
||||||
error: "Selected model is unavailable. Check Pi authentication and model settings, then try again.",
|
id: "fallback",
|
||||||
code: "model_unavailable",
|
warning: "Configured model deepseek/deepseek-v4-pro is unavailable; using zai/glm-5.2.",
|
||||||
});
|
});
|
||||||
expect(created).toBe(0);
|
expect(persisted).toMatchObject({ provider: "zai", model: "glm-5.2" });
|
||||||
|
expect(created).toBe(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("POST /sessions marks a persisted session failed when runtime construction throws", async () => {
|
test("POST /sessions marks a persisted session failed when runtime construction throws", async () => {
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ function appWithTmpSettings(extraEnv: Record<string, string> = {}, deps = {}) {
|
|||||||
return { app, dir };
|
return { app, dir };
|
||||||
}
|
}
|
||||||
|
|
||||||
test("GET /settings returns effective defaults (env provider/model/thinking, first workspace)", async () => {
|
test("GET /settings returns thinking and the first workspace without legacy model defaults", async () => {
|
||||||
const { app, dir } = appWithTmpSettings({ PI_PROVIDER: "zai", PI_MODEL: "glm-5.2", PI_THINKING: "medium" }, {
|
const { app, dir } = appWithTmpSettings({ PI_PROVIDER: "zai", PI_MODEL: "glm-5.2", PI_THINKING: "medium" }, {
|
||||||
listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }],
|
listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }],
|
||||||
});
|
});
|
||||||
@@ -29,8 +29,8 @@ test("GET /settings returns effective defaults (env provider/model/thinking, fir
|
|||||||
const res = await app.inject({ method: "GET", url: "/settings" });
|
const res = await app.inject({ method: "GET", url: "/settings" });
|
||||||
expect(res.statusCode).toBe(200);
|
expect(res.statusCode).toBe(200);
|
||||||
const body = res.json();
|
const body = res.json();
|
||||||
expect(body.provider).toBe("zai");
|
expect(body).not.toHaveProperty("provider");
|
||||||
expect(body.model).toBe("glm-5.2");
|
expect(body).not.toHaveProperty("model");
|
||||||
expect(body.thinking).toBe("medium");
|
expect(body.thinking).toBe("medium");
|
||||||
expect(typeof body.workspace).toBe("string"); // first workspace from ../harness/workspaces
|
expect(typeof body.workspace).toBe("string"); // first workspace from ../harness/workspaces
|
||||||
} finally {
|
} finally {
|
||||||
@@ -62,7 +62,9 @@ test("PUT /settings does not persist personal workspace or LLM choices", async (
|
|||||||
});
|
});
|
||||||
expect(put.statusCode).toBe(200);
|
expect(put.statusCode).toBe(200);
|
||||||
const got = await app.inject({ method: "GET", url: "/settings" });
|
const got = await app.inject({ method: "GET", url: "/settings" });
|
||||||
expect(got.json()).toMatchObject({ provider: "zai", model: "glm-5.2", thinking: "medium" });
|
expect(got.json()).toMatchObject({ thinking: "medium" });
|
||||||
|
expect(got.json()).not.toHaveProperty("provider");
|
||||||
|
expect(got.json()).not.toHaveProperty("model");
|
||||||
expect(got.json()).not.toMatchObject({ workspace: "psd", thinking: "high" });
|
expect(got.json()).not.toMatchObject({ workspace: "psd", thinking: "high" });
|
||||||
} finally {
|
} finally {
|
||||||
rmSync(dir, { recursive: true, force: true });
|
rmSync(dir, { recursive: true, force: true });
|
||||||
@@ -114,7 +116,7 @@ test("settings no longer read or write principal-specific preferences", async ()
|
|||||||
expect(preferences.size).toBe(0);
|
expect(preferences.size).toBe(0);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("GET /settings retains complete legacy installation defaults without seeding a private profile", async () => {
|
test("GET /settings drops legacy installation model fields without seeding a private profile", async () => {
|
||||||
let preferences: Record<string, unknown> = {};
|
let preferences: Record<string, unknown> = {};
|
||||||
const writes: Record<string, unknown>[] = [];
|
const writes: Record<string, unknown>[] = [];
|
||||||
const runner = {
|
const runner = {
|
||||||
@@ -135,9 +137,7 @@ test("GET /settings retains complete legacy installation defaults without seedin
|
|||||||
const first = await app.inject({ method: "GET", url: "/settings" });
|
const first = await app.inject({ method: "GET", url: "/settings" });
|
||||||
const second = await app.inject({ method: "GET", url: "/settings" });
|
const second = await app.inject({ method: "GET", url: "/settings" });
|
||||||
|
|
||||||
const expected = {
|
const expected = { workspace: "local", thinking: "low" };
|
||||||
workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low",
|
|
||||||
};
|
|
||||||
expect(first.statusCode).toBe(200);
|
expect(first.statusCode).toBe(200);
|
||||||
expect(first.json()).toEqual(expected);
|
expect(first.json()).toEqual(expected);
|
||||||
expect(second.json()).toEqual(expected);
|
expect(second.json()).toEqual(expected);
|
||||||
@@ -167,15 +167,13 @@ test("GET /settings ignores stale private preferences in favor of installation d
|
|||||||
const response = await app.inject({ method: "GET", url: "/settings" });
|
const response = await app.inject({ method: "GET", url: "/settings" });
|
||||||
|
|
||||||
expect(response.statusCode).toBe(200);
|
expect(response.statusCode).toBe(200);
|
||||||
expect(response.json()).toEqual({
|
expect(response.json()).toEqual({ workspace: "local", thinking: "low" });
|
||||||
workspace: "local", provider: "local-qwen", model: "qwen3.6-35b-a3b", thinking: "low",
|
|
||||||
});
|
|
||||||
} finally {
|
} finally {
|
||||||
rmSync(dir, { recursive: true, force: true });
|
rmSync(dir, { recursive: true, force: true });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
test("PUT /settings rejects an unknown model when a model list is available", async () => {
|
test("PUT /settings ignores a legacy unknown model because the catalog owns model validity", async () => {
|
||||||
const { app, dir } = appWithTmpSettings({}, {
|
const { app, dir } = appWithTmpSettings({}, {
|
||||||
listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }],
|
listModels: async () => [{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true }],
|
||||||
});
|
});
|
||||||
@@ -184,14 +182,14 @@ test("PUT /settings rejects an unknown model when a model list is available", as
|
|||||||
method: "PUT", url: "/settings",
|
method: "PUT", url: "/settings",
|
||||||
payload: { workspace: "psd", provider: "zai", model: "does-not-exist", thinking: "low" },
|
payload: { workspace: "psd", provider: "zai", model: "does-not-exist", thinking: "low" },
|
||||||
});
|
});
|
||||||
expect(put.statusCode).toBe(400);
|
expect(put.statusCode).toBe(200);
|
||||||
expect(put.json()).toMatchObject({ error: expect.stringMatching(/model/i) });
|
expect(put.json()).not.toHaveProperty("model");
|
||||||
} finally {
|
} finally {
|
||||||
rmSync(dir, { recursive: true, force: true });
|
rmSync(dir, { recursive: true, force: true });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
test("PUT /settings validates provider and model as one composite identifier", async () => {
|
test("PUT /settings ignores legacy provider/model pairs", async () => {
|
||||||
const { app, dir } = appWithTmpSettings({}, {
|
const { app, dir } = appWithTmpSettings({}, {
|
||||||
listModels: async () => [
|
listModels: async () => [
|
||||||
{ provider: "provider-a", id: "shared-id", name: "A", reasoning: false },
|
{ provider: "provider-a", id: "shared-id", name: "A", reasoning: false },
|
||||||
@@ -204,7 +202,7 @@ test("PUT /settings validates provider and model as one composite identifier", a
|
|||||||
workspace: "psd", provider: "provider-b", model: "shared-id", thinking: "low",
|
workspace: "psd", provider: "provider-b", model: "shared-id", thinking: "low",
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
expect(wrongProvider.statusCode).toBe(400);
|
expect(wrongProvider.statusCode).toBe(200);
|
||||||
|
|
||||||
const exactPair = await app.inject({
|
const exactPair = await app.inject({
|
||||||
method: "PUT", url: "/settings",
|
method: "PUT", url: "/settings",
|
||||||
@@ -213,6 +211,8 @@ test("PUT /settings validates provider and model as one composite identifier", a
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
expect(exactPair.statusCode).toBe(200);
|
expect(exactPair.statusCode).toBe(200);
|
||||||
|
expect(exactPair.json()).not.toHaveProperty("provider");
|
||||||
|
expect(exactPair.json()).not.toHaveProperty("model");
|
||||||
} finally {
|
} finally {
|
||||||
rmSync(dir, { recursive: true, force: true });
|
rmSync(dir, { recursive: true, force: true });
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { test, expect, vi } from "vitest";
|
import { test, expect } from "vitest";
|
||||||
import Fastify from "fastify";
|
import Fastify from "fastify";
|
||||||
import { buildApp } from "../src/app.js";
|
import { buildApp } from "../src/app.js";
|
||||||
import { loadConfig } from "../src/config.js";
|
import { loadConfig } from "../src/config.js";
|
||||||
@@ -156,12 +156,23 @@ test("registry-backed SQL preview resolves and uses the session's pinned runtime
|
|||||||
// Workspace registry route coverage lives in routes-workspaces.test.ts. `/workspaces` no longer
|
// Workspace registry route coverage lives in routes-workspaces.test.ts. `/workspaces` no longer
|
||||||
// reads legacy harness files: the Git registry is the single shared source of truth.
|
// reads legacy harness files: the Git registry is the single shared source of truth.
|
||||||
|
|
||||||
test("GET /models returns {models:[...]} from injected listModels stub", async () => {
|
test("GET /models returns session choices from the installation model catalog", async () => {
|
||||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||||
thtRunner: {} as any,
|
thtRunner: {} as any,
|
||||||
listModels: async () => [
|
runtimeModelCatalog: {
|
||||||
{ provider: "zai", id: "glm-5.2", name: "GLM 5.2", reasoning: true },
|
defaultSession: "zai/glm-5.2",
|
||||||
],
|
defaultMetadataGeneration: null,
|
||||||
|
embedding: { id: "ollama/qwen3-embedding:0.6b", dimensions: 1024 },
|
||||||
|
sessionModels: () => [{
|
||||||
|
id: "zai/glm-5.2", provider: "zai", model: "glm-5.2", label: "GLM 5.2",
|
||||||
|
upstreamModel: "glm-5.2", authentication: { mode: "pi_auth" },
|
||||||
|
sessionAdapter: { mode: "pi_builtin" }, session: { reasoning: true },
|
||||||
|
}],
|
||||||
|
metadataModels: () => [],
|
||||||
|
hasSession: (id: string) => id === "zai/glm-5.2",
|
||||||
|
},
|
||||||
|
// Runtime introspection is a health gate for starting a session, not a second catalog.
|
||||||
|
listModels: async () => { throw new Error("Pi is unavailable"); },
|
||||||
});
|
});
|
||||||
|
|
||||||
const res = await app.inject({ method: "GET", url: "/models" });
|
const res = await app.inject({ method: "GET", url: "/models" });
|
||||||
@@ -172,36 +183,17 @@ test("GET /models returns {models:[...]} from injected listModels stub", async (
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("GET /models returns {models:[]} when listModels throws (graceful fallback)", async () => {
|
test("GET /models returns an empty list when the catalog has no session models", async () => {
|
||||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
||||||
thtRunner: {} as any,
|
thtRunner: {} as any,
|
||||||
listModels: async () => { throw new Error("Pi not running"); },
|
runtimeModelCatalog: {
|
||||||
});
|
defaultSession: null,
|
||||||
|
defaultMetadataGeneration: null,
|
||||||
const res = await app.inject({ method: "GET", url: "/models" });
|
embedding: null,
|
||||||
|
sessionModels: () => [],
|
||||||
expect(res.statusCode).toBe(200);
|
metadataModels: () => [],
|
||||||
expect(res.json()).toEqual({ models: [] });
|
hasSession: () => false,
|
||||||
});
|
},
|
||||||
|
|
||||||
test("GET /models logs a sanitized warning when listing fails", async () => {
|
|
||||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
|
||||||
thtRunner: {} as any,
|
|
||||||
listModels: async () => { throw new Error("credential-value-must-not-appear"); },
|
|
||||||
});
|
|
||||||
const warn = vi.spyOn(app.log, "warn");
|
|
||||||
|
|
||||||
const res = await app.inject({ method: "GET", url: "/models" });
|
|
||||||
|
|
||||||
expect(res.json()).toEqual({ models: [] });
|
|
||||||
expect(JSON.stringify(warn.mock.calls)).not.toContain("credential-value-must-not-appear");
|
|
||||||
expect(warn).toHaveBeenCalled();
|
|
||||||
});
|
|
||||||
|
|
||||||
test("GET /models with empty listModels stub returns empty array", async () => {
|
|
||||||
const app = buildApp(loadConfig({ THT_HARNESS_DIR: "../harness" }), {
|
|
||||||
thtRunner: {} as any,
|
|
||||||
listModels: async () => [],
|
|
||||||
});
|
});
|
||||||
|
|
||||||
const res = await app.inject({ method: "GET", url: "/models" });
|
const res = await app.inject({ method: "GET", url: "/models" });
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import type { AuthDiagnoser, AuthDiagnostics } from "../src/auth/diagnostics.js"
|
|||||||
|
|
||||||
const workspace: CanonicalWorkspace = {
|
const workspace: CanonicalWorkspace = {
|
||||||
workspace: {
|
workspace: {
|
||||||
schema_version: 3,
|
schema_version: 4,
|
||||||
id: "psd-clinical",
|
id: "psd-clinical",
|
||||||
name: "Policlinico San Donato",
|
name: "Policlinico San Donato",
|
||||||
description: "Clinical analytics workspace",
|
description: "Clinical analytics workspace",
|
||||||
@@ -26,20 +26,6 @@ const workspace: CanonicalWorkspace = {
|
|||||||
schema: "datawarehouse",
|
schema: "datawarehouse",
|
||||||
supported_transports: ["postgres_direct"],
|
supported_transports: ["postgres_direct"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: {
|
|
||||||
engine: "qdrant",
|
|
||||||
collection: "psd-clinical",
|
|
||||||
dimensions: 1024,
|
|
||||||
distance: "cosine",
|
|
||||||
},
|
|
||||||
embedding: {
|
|
||||||
provider: "ollama_internal",
|
|
||||||
model: "qwen3-embedding:0.6b",
|
|
||||||
dimensions: 1024,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
};
|
};
|
||||||
|
|
||||||
const revision: WorkspaceRevision = {
|
const revision: WorkspaceRevision = {
|
||||||
@@ -194,7 +180,7 @@ test("lists workspace summaries and reads a validated immutable workspace", asyn
|
|||||||
expect(read.json()).toEqual({ workspace, revision });
|
expect(read.json()).toEqual({ workspace, revision });
|
||||||
});
|
});
|
||||||
|
|
||||||
test("validates a schema v3 workspace without mutating the repository", async () => {
|
test("validates a schema v4 workspace without mutating the repository", async () => {
|
||||||
const app = appFor(registryFake());
|
const app = appFor(registryFake());
|
||||||
|
|
||||||
const response = await app.inject({
|
const response = await app.inject({
|
||||||
@@ -284,7 +270,7 @@ test.each([1, 2])("rejects schema v%s at the validation boundary with a sanitize
|
|||||||
expect(response.body).not.toMatch(/migration_required|schema version/i);
|
expect(response.body).not.toMatch(/migration_required|schema version/i);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("runs diagnostics for a schema v3 workspace", async () => {
|
test("runs diagnostics for a schema v4 workspace", async () => {
|
||||||
const diagnose = vi.fn(async () => ({ activatable: true, diagnostics: [] }));
|
const diagnose = vi.fn(async () => ({ activatable: true, diagnostics: [] }));
|
||||||
const app = appFor(registryFake(), diagnose);
|
const app = appFor(registryFake(), diagnose);
|
||||||
|
|
||||||
|
|||||||
@@ -27,9 +27,9 @@ test("saveSettings writes the file and loadSettings reads it back", () => {
|
|||||||
const dir = mkdtempSync(join(tmpdir(), "tht-set-"));
|
const dir = mkdtempSync(join(tmpdir(), "tht-set-"));
|
||||||
try {
|
try {
|
||||||
const cfg = cfgWith(join(dir, "nested", "settings.json"));
|
const cfg = cfgWith(join(dir, "nested", "settings.json"));
|
||||||
const saved = saveSettings(cfg, { workspace: "psd", provider: "zai", model: "glm-5.2", thinking: "medium" });
|
const saved = saveSettings(cfg, { workspace: "psd", thinking: "medium" });
|
||||||
expect(saved.model).toBe("glm-5.2");
|
expect(saved.thinking).toBe("medium");
|
||||||
expect(loadSettings(cfg)).toEqual({ workspace: "psd", provider: "zai", model: "glm-5.2", thinking: "medium" });
|
expect(loadSettings(cfg)).toEqual({ workspace: "psd", thinking: "medium" });
|
||||||
} finally {
|
} finally {
|
||||||
rmSync(dir, { recursive: true, force: true });
|
rmSync(dir, { recursive: true, force: true });
|
||||||
}
|
}
|
||||||
@@ -55,11 +55,11 @@ test("saveSettings restores the previous file when post-rename directory durabil
|
|||||||
const dir = mkdtempSync(join(tmpdir(), "tht-set-transaction-"));
|
const dir = mkdtempSync(join(tmpdir(), "tht-set-transaction-"));
|
||||||
try {
|
try {
|
||||||
const cfg = cfgWith(join(dir, "settings.json"));
|
const cfg = cfgWith(join(dir, "settings.json"));
|
||||||
saveSettings(cfg, { provider: "old", model: "old-model", thinking: "low" });
|
saveSettings(cfg, { thinking: "low" });
|
||||||
let syncs = 0;
|
let syncs = 0;
|
||||||
expect(() => saveSettings(
|
expect(() => saveSettings(
|
||||||
cfg,
|
cfg,
|
||||||
{ provider: "new", model: "new-model", thinking: "high" },
|
{ thinking: "high" },
|
||||||
{
|
{
|
||||||
syncDirectory(directory: string) {
|
syncDirectory(directory: string) {
|
||||||
syncs += 1;
|
syncs += 1;
|
||||||
@@ -69,7 +69,7 @@ test("saveSettings restores the previous file when post-rename directory durabil
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
)).toThrow(/directory fsync failure/);
|
)).toThrow(/directory fsync failure/);
|
||||||
expect(loadSettings(cfg)).toEqual({ provider: "old", model: "old-model", thinking: "low" });
|
expect(loadSettings(cfg)).toEqual({ thinking: "low" });
|
||||||
expect(syncs).toBeGreaterThanOrEqual(2);
|
expect(syncs).toBeGreaterThanOrEqual(2);
|
||||||
} finally {
|
} finally {
|
||||||
rmSync(dir, { recursive: true, force: true });
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
|||||||
@@ -8,18 +8,11 @@ const keywordIndexes = [
|
|||||||
];
|
];
|
||||||
|
|
||||||
const workspace: CanonicalWorkspace = {
|
const workspace: CanonicalWorkspace = {
|
||||||
workspace: { schema_version: 3, id: "psd", name: "PSD", language: "it" },
|
workspace: { schema_version: 4, id: "psd", name: "PSD", language: "it" },
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres", database: "warehouse", schema: "public",
|
engine: "postgres", database: "warehouse", schema: "public",
|
||||||
supported_transports: ["postgres_direct"],
|
supported_transports: ["postgres_direct"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", collection: "psd", dimensions: 1024, distance: "cosine" },
|
|
||||||
embedding: {
|
|
||||||
provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
};
|
};
|
||||||
|
|
||||||
function runner(request: (...args: any[]) => Promise<any>) {
|
function runner(request: (...args: any[]) => Promise<any>) {
|
||||||
|
|||||||
@@ -19,12 +19,13 @@ afterEach(() => {
|
|||||||
const semanticRuntime = {
|
const semanticRuntime = {
|
||||||
internalQdrantUrl: "http://qdrant:6333",
|
internalQdrantUrl: "http://qdrant:6333",
|
||||||
internalEmbeddingUrl: "http://embedding:11434",
|
internalEmbeddingUrl: "http://embedding:11434",
|
||||||
|
internalEmbeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
||||||
internalEmbeddingDimensions: 1024,
|
internalEmbeddingDimensions: 1024,
|
||||||
};
|
};
|
||||||
|
|
||||||
const baseWorkspace = parseWorkspaceYaml(`workspace:
|
const baseWorkspace = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Runtime Lease
|
name: Runtime Lease
|
||||||
language: en
|
language: en
|
||||||
@@ -33,21 +34,9 @@ dwh:
|
|||||||
database: analytics
|
database: analytics
|
||||||
schema: mart
|
schema: mart
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`);
|
`);
|
||||||
const filesystemWorkspace = parseWorkspaceYaml(`${baseWorkspace ? '' : ''}workspace:
|
const filesystemWorkspace = parseWorkspaceYaml(`${baseWorkspace ? '' : ''}workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: fs-workspace
|
id: fs-workspace
|
||||||
name: Filesystem
|
name: Filesystem
|
||||||
language: en
|
language: en
|
||||||
@@ -56,25 +45,13 @@ dwh:
|
|||||||
database: analytics
|
database: analytics
|
||||||
schema: mart
|
schema: mart
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: fs-workspace
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
evidence:
|
evidence:
|
||||||
source:
|
source:
|
||||||
type: filesystem
|
type: filesystem
|
||||||
uri: fs-workspace/evidence
|
uri: fs-workspace/evidence
|
||||||
`);
|
`);
|
||||||
const privateHttpWorkspace = parseWorkspaceYaml(`workspace:
|
const privateHttpWorkspace = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: http-workspace
|
id: http-workspace
|
||||||
name: Http
|
name: Http
|
||||||
language: en
|
language: en
|
||||||
@@ -83,18 +60,6 @@ dwh:
|
|||||||
database: analytics
|
database: analytics
|
||||||
schema: mart
|
schema: mart
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: http-workspace
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
evidence:
|
evidence:
|
||||||
source:
|
source:
|
||||||
type: http
|
type: http
|
||||||
@@ -125,7 +90,7 @@ function runtime(workspace = baseWorkspace, workspaceId = workspace.workspace.id
|
|||||||
bindingDigest: "sha256:bindings",
|
bindingDigest: "sha256:bindings",
|
||||||
semanticQdrantUrl: "http://qdrant:6333",
|
semanticQdrantUrl: "http://qdrant:6333",
|
||||||
effectiveConfig: {
|
effectiveConfig: {
|
||||||
schemaVersion: 1,
|
schemaVersion: 2,
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres",
|
engine: "postgres",
|
||||||
database: "analytics",
|
database: "analytics",
|
||||||
@@ -136,7 +101,7 @@ function runtime(workspace = baseWorkspace, workspaceId = workspace.workspace.id
|
|||||||
user: "reader",
|
user: "reader",
|
||||||
},
|
},
|
||||||
vector: { collection: workspaceId, dimensions: 1024, distance: "cosine" },
|
vector: { collection: workspaceId, dimensions: 1024, distance: "cosine" },
|
||||||
embedding: { model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
embedding: { id: "ollama/qwen3-embedding:0.6b", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
||||||
roots: { artifacts: "/data/artifacts", indexes: "/data/indexes" },
|
roots: { artifacts: "/data/artifacts", indexes: "/data/indexes" },
|
||||||
},
|
},
|
||||||
effectiveConfigIdentity: "workspace://psd-clinical@v1:" + "d".repeat(64),
|
effectiveConfigIdentity: "workspace://psd-clinical@v1:" + "d".repeat(64),
|
||||||
|
|||||||
@@ -28,13 +28,15 @@ test("job state creates durable 0600 JSON and enforces same-revision resume", as
|
|||||||
catalogBlob: "c".repeat(40),
|
catalogBlob: "c".repeat(40),
|
||||||
configDigest: "sha256:config",
|
configDigest: "sha256:config",
|
||||||
bindingDigest: "sha256:bindings",
|
bindingDigest: "sha256:bindings",
|
||||||
|
embeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
|
embeddingDimensions: 1024,
|
||||||
});
|
});
|
||||||
|
|
||||||
const path = store.jobPath(job.runId);
|
const path = store.jobPath(job.runId);
|
||||||
expect(existsSync(path)).toBe(true);
|
expect(existsSync(path)).toBe(true);
|
||||||
expect(statSync(path).mode & 0o777).toBe(0o600);
|
expect(statSync(path).mode & 0o777).toBe(0o600);
|
||||||
expect(JSON.parse(readFileSync(path, "utf8"))).toMatchObject({
|
expect(JSON.parse(readFileSync(path, "utf8"))).toMatchObject({
|
||||||
schemaVersion: 1,
|
schemaVersion: 2,
|
||||||
operation: "preprocess dwh",
|
operation: "preprocess dwh",
|
||||||
workspaceId: "psd-clinical",
|
workspaceId: "psd-clinical",
|
||||||
workspaceRevision: "a".repeat(40),
|
workspaceRevision: "a".repeat(40),
|
||||||
@@ -42,6 +44,8 @@ test("job state creates durable 0600 JSON and enforces same-revision resume", as
|
|||||||
catalogBlob: "c".repeat(40),
|
catalogBlob: "c".repeat(40),
|
||||||
configDigest: "sha256:config",
|
configDigest: "sha256:config",
|
||||||
bindingDigest: "sha256:bindings",
|
bindingDigest: "sha256:bindings",
|
||||||
|
embeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
|
embeddingDimensions: 1024,
|
||||||
});
|
});
|
||||||
|
|
||||||
await expect(store.beginJob({
|
await expect(store.beginJob({
|
||||||
@@ -52,6 +56,20 @@ test("job state creates durable 0600 JSON and enforces same-revision resume", as
|
|||||||
catalogBlob: "c".repeat(40),
|
catalogBlob: "c".repeat(40),
|
||||||
configDigest: "sha256:config",
|
configDigest: "sha256:config",
|
||||||
bindingDigest: "sha256:bindings",
|
bindingDigest: "sha256:bindings",
|
||||||
|
embeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
|
embeddingDimensions: 1024,
|
||||||
|
})).rejects.toMatchObject({ code: "preprocessing_resume_mismatch" });
|
||||||
|
|
||||||
|
await expect(store.beginJob({
|
||||||
|
operation: "preprocess dwh",
|
||||||
|
runId: job.runId,
|
||||||
|
workspaceRevision: "a".repeat(40),
|
||||||
|
descriptorBlob: "b".repeat(40),
|
||||||
|
catalogBlob: "c".repeat(40),
|
||||||
|
configDigest: "sha256:config",
|
||||||
|
bindingDigest: "sha256:bindings",
|
||||||
|
embeddingId: "ollama/replacement-embedding",
|
||||||
|
embeddingDimensions: 1024,
|
||||||
})).rejects.toMatchObject({ code: "preprocessing_resume_mismatch" });
|
})).rejects.toMatchObject({ code: "preprocessing_resume_mismatch" });
|
||||||
|
|
||||||
const resumed = await store.beginJob({
|
const resumed = await store.beginJob({
|
||||||
@@ -62,6 +80,8 @@ test("job state creates durable 0600 JSON and enforces same-revision resume", as
|
|||||||
catalogBlob: "c".repeat(40),
|
catalogBlob: "c".repeat(40),
|
||||||
configDigest: "sha256:config",
|
configDigest: "sha256:config",
|
||||||
bindingDigest: "sha256:bindings",
|
bindingDigest: "sha256:bindings",
|
||||||
|
embeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
|
embeddingDimensions: 1024,
|
||||||
});
|
});
|
||||||
expect(resumed.runId).toBe(job.runId);
|
expect(resumed.runId).toBe(job.runId);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -75,10 +75,6 @@ function workspaceVariant(
|
|||||||
return {
|
return {
|
||||||
...workspace,
|
...workspace,
|
||||||
workspace: { ...workspace.workspace, ...changes, id },
|
workspace: { ...workspace.workspace, ...changes, id },
|
||||||
semantic_index: {
|
|
||||||
...workspace.semantic_index,
|
|
||||||
vector_store: { ...workspace.semantic_index.vector_store, collection: id },
|
|
||||||
},
|
|
||||||
...(workspace.evidence?.source.type === "filesystem"
|
...(workspace.evidence?.source.type === "filesystem"
|
||||||
? {
|
? {
|
||||||
evidence: {
|
evidence: {
|
||||||
@@ -182,7 +178,7 @@ test("shared deployment fixtures remain valid standalone descriptors with canoni
|
|||||||
|
|
||||||
expect(smoke).toMatchObject({
|
expect(smoke).toMatchObject({
|
||||||
workspace: {
|
workspace: {
|
||||||
schema_version: 3,
|
schema_version: 4,
|
||||||
id: "local",
|
id: "local",
|
||||||
name: "Local",
|
name: "Local",
|
||||||
description: "Isolated workspace registry smoke fixture.",
|
description: "Isolated workspace registry smoke fixture.",
|
||||||
@@ -193,14 +189,14 @@ test("shared deployment fixtures remain valid standalone descriptors with canoni
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
expect(task13).toMatchObject({
|
expect(task13).toMatchObject({
|
||||||
workspace: { schema_version: 3, id: "task13-smoke", name: "Task 13 Smoke" },
|
workspace: { schema_version: 4, id: "task13-smoke", name: "Task 13 Smoke" },
|
||||||
evidence: {
|
evidence: {
|
||||||
source: { type: "filesystem", uri: "task13-smoke/evidence", patterns: ["**/*.md"] },
|
source: { type: "filesystem", uri: "task13-smoke/evidence", patterns: ["**/*.md"] },
|
||||||
policy: { max_chunk_chars: 4000, retain_published_generations: 3 },
|
policy: { max_chunk_chars: 4000, retain_published_generations: 3 },
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
expect(windows).toMatchObject({
|
expect(windows).toMatchObject({
|
||||||
workspace: { schema_version: 3, id: "task13-windows", name: "Task 13 Windows" },
|
workspace: { schema_version: 4, id: "task13-windows", name: "Task 13 Windows" },
|
||||||
evidence: {
|
evidence: {
|
||||||
source: { type: "filesystem", uri: "task13-windows/evidence", patterns: ["**/*.md"] },
|
source: { type: "filesystem", uri: "task13-windows/evidence", patterns: ["**/*.md"] },
|
||||||
policy: { max_chunk_chars: 4000, retain_published_generations: 3 },
|
policy: { max_chunk_chars: 4000, retain_published_generations: 3 },
|
||||||
@@ -282,7 +278,7 @@ test("registry rejects orphan descriptors, metadata mismatches, and the retired
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("Windows clone contract copies the shared complete schema v3 descriptor into the nested registry layout", () => {
|
test("Windows clone contract copies the shared complete schema v4 descriptor into the nested registry layout", () => {
|
||||||
const descriptor = parseWorkspaceYaml(readFixture("workspace-registry-windows.yaml"));
|
const descriptor = parseWorkspaceYaml(readFixture("workspace-registry-windows.yaml"));
|
||||||
const windows = readFileSync(
|
const windows = readFileSync(
|
||||||
new URL("../../scripts/test-windows-clone-contract.ps1", import.meta.url),
|
new URL("../../scripts/test-windows-clone-contract.ps1", import.meta.url),
|
||||||
@@ -299,7 +295,7 @@ test("Windows clone contract copies the shared complete schema v3 descriptor int
|
|||||||
expect(windows).not.toContain('schema_version: 3');
|
expect(windows).not.toContain('schema_version: 3');
|
||||||
expect(descriptor).toMatchObject({
|
expect(descriptor).toMatchObject({
|
||||||
workspace: {
|
workspace: {
|
||||||
schema_version: 3,
|
schema_version: 4,
|
||||||
id: "task13-windows",
|
id: "task13-windows",
|
||||||
name: "Task 13 Windows",
|
name: "Task 13 Windows",
|
||||||
language: "en",
|
language: "en",
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ import {
|
|||||||
import type { WorkspaceRegistryConfig } from "../src/workspaces/types.js";
|
import type { WorkspaceRegistryConfig } from "../src/workspaces/types.js";
|
||||||
|
|
||||||
const validYaml = `workspace:
|
const validYaml = `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -24,18 +24,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`;
|
`;
|
||||||
|
|
||||||
function withFilesystemEvidence(source: string, id = "psd-clinical"): string {
|
function withFilesystemEvidence(source: string, id = "psd-clinical"): string {
|
||||||
@@ -146,7 +134,7 @@ function legacyV1Yaml(source = validYaml): string {
|
|||||||
.replace(" model: qwen3-embedding:0.6b", " model: nomic-embed-text-v2-moe")
|
.replace(" model: qwen3-embedding:0.6b", " model: nomic-embed-text-v2-moe")
|
||||||
.replace(" dimensions: 1024", " dimensions: 768")
|
.replace(" dimensions: 1024", " dimensions: 768")
|
||||||
.replace("distance: cosine\n", "distance: cosine\n supported_transports: [pgvector_direct]\n")
|
.replace("distance: cosine\n", "distance: cosine\n supported_transports: [pgvector_direct]\n")
|
||||||
.replace("schema_version: 3", "schema_version: 1");
|
.replace("schema_version: 4", "schema_version: 1");
|
||||||
}
|
}
|
||||||
|
|
||||||
function legacyV2Yaml(source = validYaml): string {
|
function legacyV2Yaml(source = validYaml): string {
|
||||||
@@ -158,7 +146,7 @@ function legacyV2Yaml(source = validYaml): string {
|
|||||||
.replace(" model: qwen3-embedding:0.6b", " model: nomic-embed-text-v2-moe")
|
.replace(" model: qwen3-embedding:0.6b", " model: nomic-embed-text-v2-moe")
|
||||||
.replace(" dimensions: 1024", " dimensions: 768")
|
.replace(" dimensions: 1024", " dimensions: 768")
|
||||||
.replace("distance: cosine\n", "distance: cosine\n supported_transports: [pgvector_direct]\n")
|
.replace("distance: cosine\n", "distance: cosine\n supported_transports: [pgvector_direct]\n")
|
||||||
.replace("schema_version: 3", "schema_version: 2");
|
.replace("schema_version: 4", "schema_version: 2");
|
||||||
}
|
}
|
||||||
|
|
||||||
const runFile = promisify(execFile);
|
const runFile = promisify(execFile);
|
||||||
@@ -197,7 +185,7 @@ async function fixture(workspaceSource = validYaml): Promise<{
|
|||||||
await git(source, ["init", "--initial-branch=main"]);
|
await git(source, ["init", "--initial-branch=main"]);
|
||||||
await git(source, ["config", "user.name", "Workspace Registry Test"]);
|
await git(source, ["config", "user.name", "Workspace Registry Test"]);
|
||||||
await git(source, ["config", "user.email", "workspace-registry@example.invalid"]);
|
await git(source, ["config", "user.email", "workspace-registry@example.invalid"]);
|
||||||
const workspace = workspaceSource.includes("schema_version: 3")
|
const workspace = workspaceSource.includes("schema_version: 4")
|
||||||
? parseWorkspaceYaml(workspaceSource) : undefined;
|
? parseWorkspaceYaml(workspaceSource) : undefined;
|
||||||
mkdirSync(join(source, "psd-clinical"), { recursive: true });
|
mkdirSync(join(source, "psd-clinical"), { recursive: true });
|
||||||
writeFileSync(join(source, "thoth-workspaces.yaml"), catalogYaml([
|
writeFileSync(join(source, "thoth-workspaces.yaml"), catalogYaml([
|
||||||
@@ -256,7 +244,7 @@ async function multiWorkspaceFixture(workspaces: Record<string, string>): Promis
|
|||||||
await git(source, ["config", "user.email", "workspace-registry@example.invalid"]);
|
await git(source, ["config", "user.email", "workspace-registry@example.invalid"]);
|
||||||
const entries = [];
|
const entries = [];
|
||||||
for (const [id, workspaceSource] of Object.entries(workspaces)) {
|
for (const [id, workspaceSource] of Object.entries(workspaces)) {
|
||||||
const workspace = workspaceSource.includes("schema_version: 3") ? parseWorkspaceYaml(workspaceSource) : undefined;
|
const workspace = workspaceSource.includes("schema_version: 4") ? parseWorkspaceYaml(workspaceSource) : undefined;
|
||||||
entries.push({ id, name: workspace.workspace.name, ...(workspace.workspace.description ? { description: workspace.workspace.description } : {}) });
|
entries.push({ id, name: workspace.workspace.name, ...(workspace.workspace.description ? { description: workspace.workspace.description } : {}) });
|
||||||
mkdirSync(join(source, id), { recursive: true });
|
mkdirSync(join(source, id), { recursive: true });
|
||||||
writeFileSync(join(source, id, "workspace.yaml"), workspaceSource);
|
writeFileSync(join(source, id, "workspace.yaml"), workspaceSource);
|
||||||
@@ -788,7 +776,7 @@ test("normalizes historical operational state during offline fallback after rest
|
|||||||
expect(read.revision).not.toHaveProperty("state");
|
expect(read.revision).not.toHaveProperty("state");
|
||||||
});
|
});
|
||||||
|
|
||||||
test("fails closed when a retained snapshot descriptor is not schema v3", async () => {
|
test("fails closed when a retained snapshot descriptor is not schema v4", async () => {
|
||||||
const remote = await fixture();
|
const remote = await fixture();
|
||||||
const root = join(remote.root, "registry");
|
const root = join(remote.root, "registry");
|
||||||
await new WorkspaceRegistry(config(root, remote.remote)).bootstrap();
|
await new WorkspaceRegistry(config(root, remote.remote)).bootstrap();
|
||||||
@@ -820,45 +808,6 @@ test("keeps the last valid snapshot when a pulled commit has invalid YAML", asyn
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("rejects duplicate schema v3 collection ownership and keeps the previous active snapshot", async () => {
|
|
||||||
const v3Yaml = validYaml;
|
|
||||||
const remote = await multiWorkspaceFixture({
|
|
||||||
"psd-clinical": v3Yaml,
|
|
||||||
"research-clinical": v3Yaml
|
|
||||||
.replace("id: psd-clinical", "id: research-clinical")
|
|
||||||
.replace("name: Policlinico San Donato", "name: Research Clinical")
|
|
||||||
.replace("collection: psd-clinical", "collection: research-clinical"),
|
|
||||||
});
|
|
||||||
const registry = new WorkspaceRegistry(config(join(remote.root, "registry"), remote.remote));
|
|
||||||
await registry.bootstrap();
|
|
||||||
|
|
||||||
writeFileSync(
|
|
||||||
join(remote.source, "research-clinical", "workspace.yaml"),
|
|
||||||
v3Yaml
|
|
||||||
.replace("id: psd-clinical", "id: research-clinical")
|
|
||||||
.replace("name: Policlinico San Donato", "name: Research Clinical")
|
|
||||||
.replace("collection: psd-clinical", "collection: shared"),
|
|
||||||
);
|
|
||||||
writeFileSync(
|
|
||||||
join(remote.source, "psd-clinical", "workspace.yaml"),
|
|
||||||
v3Yaml.replace("collection: psd-clinical", "collection: shared"),
|
|
||||||
);
|
|
||||||
await git(remote.source, ["add", "-A"]);
|
|
||||||
await git(remote.source, ["commit", "-m", "Duplicate collection ownership"]);
|
|
||||||
await git(remote.source, ["push", "origin", "main"]);
|
|
||||||
|
|
||||||
await expect(registry.pull()).rejects.toMatchObject({
|
|
||||||
code: "workspace_invalid",
|
|
||||||
message: "Workspace repository content is invalid",
|
|
||||||
});
|
|
||||||
await expect(registry.read("psd-clinical")).resolves.toMatchObject({
|
|
||||||
revision: { commit: remote.initialCommit },
|
|
||||||
});
|
|
||||||
await expect(registry.read("research-clinical")).resolves.toMatchObject({
|
|
||||||
revision: { commit: remote.initialCommit },
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
test("retains a historical snapshot while a resumable manifest still references its revision", async () => {
|
test("retains a historical snapshot while a resumable manifest still references its revision", async () => {
|
||||||
const remote = await fixture();
|
const remote = await fixture();
|
||||||
const root = join(remote.root, "registry");
|
const root = join(remote.root, "registry");
|
||||||
|
|||||||
@@ -66,7 +66,7 @@ workspaces: [{id: psd-clinical, name: Runtime Lease}]
|
|||||||
`);
|
`);
|
||||||
mkdirSync(join(source, "psd-clinical", "evidence"), { recursive: true });
|
mkdirSync(join(source, "psd-clinical", "evidence"), { recursive: true });
|
||||||
writeFileSync(join(source, "psd-clinical", "workspace.yaml"), `workspace:
|
writeFileSync(join(source, "psd-clinical", "workspace.yaml"), `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Runtime Lease
|
name: Runtime Lease
|
||||||
language: en
|
language: en
|
||||||
@@ -75,18 +75,6 @@ dwh:
|
|||||||
database: analytics
|
database: analytics
|
||||||
schema: mart
|
schema: mart
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
evidence:
|
evidence:
|
||||||
source:
|
source:
|
||||||
type: filesystem
|
type: filesystem
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ const thtBin = join(harnessDir, ".venv", "bin", "tht");
|
|||||||
const roots: string[] = [];
|
const roots: string[] = [];
|
||||||
|
|
||||||
const canonicalWorkspace = `workspace:
|
const canonicalWorkspace = `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Runtime handoff
|
name: Runtime handoff
|
||||||
language: en
|
language: en
|
||||||
@@ -30,18 +30,6 @@ dwh:
|
|||||||
database: analytics
|
database: analytics
|
||||||
schema: mart
|
schema: mart
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`;
|
`;
|
||||||
|
|
||||||
const filesystemWorkspace = `${canonicalWorkspace}evidence:
|
const filesystemWorkspace = `${canonicalWorkspace}evidence:
|
||||||
@@ -145,7 +133,7 @@ function runnerFor(f: Awaited<ReturnType<typeof fixture>>): ThtRunner {
|
|||||||
} as any);
|
} as any);
|
||||||
}
|
}
|
||||||
|
|
||||||
test("real schema-v3 registry revision loads through ThtRunner and the harness contract", async () => {
|
test("real schema-v4 registry revision loads through ThtRunner and the harness contract", async () => {
|
||||||
const f = await fixture();
|
const f = await fixture();
|
||||||
const runner = runnerFor(f);
|
const runner = runnerFor(f);
|
||||||
|
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ import {
|
|||||||
import { supportsSessionRuntime } from "../src/workspaces/bindings.js";
|
import { supportsSessionRuntime } from "../src/workspaces/bindings.js";
|
||||||
import { parseWorkspaceYaml } from "../src/workspaces/schema.js";
|
import { parseWorkspaceYaml } from "../src/workspaces/schema.js";
|
||||||
|
|
||||||
const workspaceV3 = parseWorkspaceYaml(`workspace:
|
const workspaceV4 = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -22,19 +22,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
default: zai/glm-5.2
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`);
|
`);
|
||||||
const paths: RuntimePaths = {
|
const paths: RuntimePaths = {
|
||||||
sessions: "/data/workspaces/psd-clinical/sessions",
|
sessions: "/data/workspaces/psd-clinical/sessions",
|
||||||
@@ -45,6 +32,7 @@ const paths: RuntimePaths = {
|
|||||||
const semanticRuntime: SemanticRuntimeConfig = {
|
const semanticRuntime: SemanticRuntimeConfig = {
|
||||||
internalQdrantUrl: "http://qdrant:6333",
|
internalQdrantUrl: "http://qdrant:6333",
|
||||||
internalEmbeddingUrl: "http://embedding:11434",
|
internalEmbeddingUrl: "http://embedding:11434",
|
||||||
|
internalEmbeddingId: "ollama/qwen3-embedding:0.6b",
|
||||||
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
internalEmbeddingModel: "qwen3-embedding:0.6b",
|
||||||
internalEmbeddingDimensions: 1024,
|
internalEmbeddingDimensions: 1024,
|
||||||
};
|
};
|
||||||
@@ -63,8 +51,8 @@ const directBindings: RuntimeBindings = {
|
|||||||
evidence: { missing: [], values: {} },
|
evidence: { missing: [], values: {} },
|
||||||
};
|
};
|
||||||
|
|
||||||
test("renders only the schema-v3 internal Qdrant and Ollama runtime shape", () => {
|
test("derives the internal Qdrant and Ollama runtime shape from workspace v4 plus installation config", () => {
|
||||||
const rendered = parse(renderRuntimeConfig(workspaceV3, directBindings, paths, {
|
const rendered = parse(renderRuntimeConfig(workspaceV4, directBindings, paths, {
|
||||||
workspaceId: "psd-clinical", workspaceRevision: "a".repeat(40),
|
workspaceId: "psd-clinical", workspaceRevision: "a".repeat(40),
|
||||||
}, {}, semanticRuntime));
|
}, {}, semanticRuntime));
|
||||||
|
|
||||||
@@ -95,8 +83,8 @@ test("renders only the schema-v3 internal Qdrant and Ollama runtime shape", () =
|
|||||||
expect(rendered).not.toHaveProperty("vector_rest");
|
expect(rendered).not.toHaveProperty("vector_rest");
|
||||||
});
|
});
|
||||||
|
|
||||||
test("renders schema-v3 DWH REST without exposing secret contents", () => {
|
test("renders workspace-v4 DWH REST without exposing secret contents", () => {
|
||||||
const rendered = parse(renderRuntimeConfig(workspaceV3, {
|
const rendered = parse(renderRuntimeConfig(workspaceV4, {
|
||||||
dwh: {
|
dwh: {
|
||||||
transport: "rest_api", missing: [], values: {
|
transport: "rest_api", missing: [], values: {
|
||||||
THT_WS_PSD_CLINICAL_DWH_BASE_URL: "https://dwh.example.test",
|
THT_WS_PSD_CLINICAL_DWH_BASE_URL: "https://dwh.example.test",
|
||||||
@@ -152,7 +140,7 @@ function evidenceWorkspace(
|
|||||||
}
|
}
|
||||||
|
|
||||||
const canonicalEvidenceWorkspace = `workspace:
|
const canonicalEvidenceWorkspace = `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Runtime Evidence
|
name: Runtime Evidence
|
||||||
language: en
|
language: en
|
||||||
@@ -161,18 +149,6 @@ dwh:
|
|||||||
database: analytics
|
database: analytics
|
||||||
schema: mart
|
schema: mart
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`;
|
`;
|
||||||
|
|
||||||
const evidenceRevision = "1".repeat(40);
|
const evidenceRevision = "1".repeat(40);
|
||||||
@@ -374,7 +350,7 @@ test("renders static S3 Evidence with endpoint policy, limits, and file paths bu
|
|||||||
|
|
||||||
test("omits Evidence configuration and policy when the descriptor has no Evidence", () => {
|
test("omits Evidence configuration and policy when the descriptor has no Evidence", () => {
|
||||||
const rendered = parse(renderRuntimeConfig(
|
const rendered = parse(renderRuntimeConfig(
|
||||||
workspaceV3,
|
workspaceV4,
|
||||||
directBindings,
|
directBindings,
|
||||||
paths,
|
paths,
|
||||||
evidenceContext,
|
evidenceContext,
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ const roots: string[] = [];
|
|||||||
|
|
||||||
function workspace(extra = "") {
|
function workspace(extra = "") {
|
||||||
return parseWorkspaceYaml(`workspace:
|
return parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: en
|
language: en
|
||||||
@@ -23,10 +23,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
${extra}`);
|
${extra}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
import { expect, test } from "vitest";
|
||||||
|
import { migrateWorkspaceV3Yaml, parseWorkspaceYaml } from "../src/workspaces/schema.js";
|
||||||
|
|
||||||
|
const legacy = `workspace:
|
||||||
|
schema_version: 3
|
||||||
|
id: abc
|
||||||
|
name: Example
|
||||||
|
language: en
|
||||||
|
dwh:
|
||||||
|
engine: postgres
|
||||||
|
database: warehouse
|
||||||
|
schema: public
|
||||||
|
supported_transports: [postgres_direct]
|
||||||
|
semantic_index:
|
||||||
|
vector_store: {engine: qdrant, collection: abc, dimensions: 1024, distance: cosine}
|
||||||
|
embedding: {provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024}
|
||||||
|
llm_policy:
|
||||||
|
default: zai/glm-5.3
|
||||||
|
allowed: [zai/glm-5.3]
|
||||||
|
`;
|
||||||
|
|
||||||
|
test("strict workspace v4 rejects model-bearing v3 descriptors", () => {
|
||||||
|
expect(() => parseWorkspaceYaml(legacy)).toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
test("v3 to v4 migration removes only model/vector policy and bumps the version", () => {
|
||||||
|
const migrated = migrateWorkspaceV3Yaml(legacy);
|
||||||
|
const workspace = parseWorkspaceYaml(migrated);
|
||||||
|
|
||||||
|
expect(workspace.workspace).toMatchObject({ schema_version: 4, id: "abc" });
|
||||||
|
expect(migrated).not.toMatch(/semantic_index|llm_policy/);
|
||||||
|
expect(workspace.dwh).toEqual({
|
||||||
|
engine: "postgres", database: "warehouse", schema: "public",
|
||||||
|
supported_transports: ["postgres_direct"],
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -9,8 +9,8 @@ import {
|
|||||||
} from "../src/workspaces/bindings.js";
|
} from "../src/workspaces/bindings.js";
|
||||||
import { parseWorkspaceYaml } from "../src/workspaces/schema.js";
|
import { parseWorkspaceYaml } from "../src/workspaces/schema.js";
|
||||||
|
|
||||||
const workspaceV3 = parseWorkspaceYaml(`workspace:
|
const workspaceV4 = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -19,10 +19,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
`);
|
`);
|
||||||
const temporaryRoots: string[] = [];
|
const temporaryRoots: string[] = [];
|
||||||
|
|
||||||
@@ -40,9 +36,9 @@ function secretPath(name: string): { root: string; path: string } {
|
|||||||
return { root: secrets, path };
|
return { root: secrets, path };
|
||||||
}
|
}
|
||||||
|
|
||||||
test("resolves schema-v3 direct DWH bindings from the stable namespace", () => {
|
test("resolves workspace-v4 direct DWH bindings from the stable namespace", () => {
|
||||||
const password = secretPath("dwh-password");
|
const password = secretPath("dwh-password");
|
||||||
const result = resolveBinding(workspaceV3, "DWH", {
|
const result = resolveBinding(workspaceV4, "DWH", {
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "postgres_direct",
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "postgres_direct",
|
||||||
THT_WS_PSD_CLINICAL_DWH_HOST: "dwh.internal",
|
THT_WS_PSD_CLINICAL_DWH_HOST: "dwh.internal",
|
||||||
THT_WS_PSD_CLINICAL_DWH_PORT: "5432",
|
THT_WS_PSD_CLINICAL_DWH_PORT: "5432",
|
||||||
@@ -61,14 +57,14 @@ test("resolves schema-v3 direct DWH bindings from the stable namespace", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("requires schema-v3 REST credentials unless the DWH diagnostic declares auth none", () => {
|
test("requires workspace-v4 REST credentials unless the DWH diagnostic declares auth none", () => {
|
||||||
expect(resolveBinding(workspaceV3, "DWH", {
|
expect(resolveBinding(workspaceV4, "DWH", {
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "rest_api",
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "rest_api",
|
||||||
THT_WS_PSD_CLINICAL_DWH_BASE_URL: "https://dwh.example.test",
|
THT_WS_PSD_CLINICAL_DWH_BASE_URL: "https://dwh.example.test",
|
||||||
}, []).missing).toContain("THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE");
|
}, []).missing).toContain("THT_WS_PSD_CLINICAL_DWH_API_KEY_FILE");
|
||||||
|
|
||||||
const noAuth = parseWorkspaceYaml(`workspace:
|
const noAuth = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: No auth
|
name: No auth
|
||||||
language: en
|
language: en
|
||||||
@@ -77,16 +73,12 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: public
|
schema: public
|
||||||
supported_transports: [rest_api]
|
supported_transports: [rest_api]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
diagnostics:
|
diagnostics:
|
||||||
dwh_rest:
|
dwh_rest:
|
||||||
method: GET
|
method: GET
|
||||||
path: /health
|
path: /health
|
||||||
auth: none
|
auth: none
|
||||||
response: { database: database, schema: schema }
|
response: { database: database, schema: schema }
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
`);
|
`);
|
||||||
expect(resolveBinding(noAuth, "DWH", {
|
expect(resolveBinding(noAuth, "DWH", {
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "rest_api",
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "rest_api",
|
||||||
@@ -98,7 +90,7 @@ test("rejects unsupported transports and secret paths outside configured roots",
|
|||||||
const outside = secretPath("outside-password");
|
const outside = secretPath("outside-password");
|
||||||
const allowed = secretPath("allowed-password");
|
const allowed = secretPath("allowed-password");
|
||||||
const directOnly = parseWorkspaceYaml(`workspace:
|
const directOnly = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Direct only
|
name: Direct only
|
||||||
language: en
|
language: en
|
||||||
@@ -107,15 +99,11 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: public
|
schema: public
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
`);
|
`);
|
||||||
expect(resolveBinding(directOnly, "DWH", {
|
expect(resolveBinding(directOnly, "DWH", {
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "rest_api",
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "rest_api",
|
||||||
}, []).missing).toContain("THT_WS_PSD_CLINICAL_DWH_TRANSPORT");
|
}, []).missing).toContain("THT_WS_PSD_CLINICAL_DWH_TRANSPORT");
|
||||||
const result = resolveBinding(workspaceV3, "DWH", {
|
const result = resolveBinding(workspaceV4, "DWH", {
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "postgres_direct",
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "postgres_direct",
|
||||||
THT_WS_PSD_CLINICAL_DWH_HOST: "dwh.internal",
|
THT_WS_PSD_CLINICAL_DWH_HOST: "dwh.internal",
|
||||||
THT_WS_PSD_CLINICAL_DWH_PORT: "5432",
|
THT_WS_PSD_CLINICAL_DWH_PORT: "5432",
|
||||||
@@ -128,7 +116,7 @@ llm_policy: { allowed: [zai/glm-5.2] }
|
|||||||
|
|
||||||
test("runtime bindings contain only DWH and Evidence roles", () => {
|
test("runtime bindings contain only DWH and Evidence roles", () => {
|
||||||
const password = secretPath("dwh-password");
|
const password = secretPath("dwh-password");
|
||||||
const bindings = resolveRuntimeBindings(workspaceV3, {
|
const bindings = resolveRuntimeBindings(workspaceV4, {
|
||||||
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "postgres_direct",
|
THT_WS_PSD_CLINICAL_DWH_TRANSPORT: "postgres_direct",
|
||||||
THT_WS_PSD_CLINICAL_DWH_HOST: "dwh.internal",
|
THT_WS_PSD_CLINICAL_DWH_HOST: "dwh.internal",
|
||||||
THT_WS_PSD_CLINICAL_DWH_PORT: "5432",
|
THT_WS_PSD_CLINICAL_DWH_PORT: "5432",
|
||||||
@@ -143,7 +131,7 @@ test("runtime bindings contain only DWH and Evidence roles", () => {
|
|||||||
|
|
||||||
function withEvidence(source: Record<string, unknown>) {
|
function withEvidence(source: Record<string, unknown>) {
|
||||||
return parseWorkspaceYaml(`workspace:
|
return parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -152,10 +140,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
evidence:
|
evidence:
|
||||||
source: ${JSON.stringify(source)}
|
source: ${JSON.stringify(source)}
|
||||||
`);
|
`);
|
||||||
@@ -260,7 +244,7 @@ test("rejects relative, missing, directory, unreadable, and escaping symlink Evi
|
|||||||
});
|
});
|
||||||
|
|
||||||
test("includes Evidence binding completeness in session runtime support without changing v3 compatibility", () => {
|
test("includes Evidence binding completeness in session runtime support without changing v3 compatibility", () => {
|
||||||
const unsigned = resolveRuntimeBindings(workspaceV3, {}, ["/run/secrets"]);
|
const unsigned = resolveRuntimeBindings(workspaceV4, {}, ["/run/secrets"]);
|
||||||
expect(unsigned.evidence).toEqual({ values: {}, missing: [] });
|
expect(unsigned.evidence).toEqual({ values: {}, missing: [] });
|
||||||
expect(supportsSessionRuntime(unsigned)).toBe(true);
|
expect(supportsSessionRuntime(unsigned)).toBe(true);
|
||||||
|
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import {
|
|||||||
} from "../src/workspaces/catalog.js";
|
} from "../src/workspaces/catalog.js";
|
||||||
|
|
||||||
const descriptor = parseWorkspaceYaml(`workspace:
|
const descriptor = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd
|
id: psd
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
description: Clinical warehouse
|
description: Clinical warehouse
|
||||||
@@ -17,10 +17,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [rest_api]
|
supported_transports: [rest_api]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
`);
|
`);
|
||||||
|
|
||||||
test("parses the strict ordered root catalog", () => {
|
test("parses the strict ordered root catalog", () => {
|
||||||
|
|||||||
@@ -5,8 +5,8 @@ import { join } from "node:path";
|
|||||||
import { buildInstallationContract, renderWorkspaceDocs } from "../src/workspaces/contracts.js";
|
import { buildInstallationContract, renderWorkspaceDocs } from "../src/workspaces/contracts.js";
|
||||||
import { type CanonicalWorkspace, parseWorkspaceYaml } from "../src/workspaces/schema.js";
|
import { type CanonicalWorkspace, parseWorkspaceYaml } from "../src/workspaces/schema.js";
|
||||||
|
|
||||||
const workspaceV3 = parseWorkspaceYaml(`workspace:
|
const workspaceV4 = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -15,17 +15,12 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`);
|
`);
|
||||||
|
|
||||||
test("schema-v3 installation contracts expose only DWH bindings and no semantic variables", () => {
|
test("workspace-v4 installation contracts expose only DWH bindings and no semantic variables", () => {
|
||||||
const contract = buildInstallationContract(workspaceV3);
|
const contract = buildInstallationContract(workspaceV4);
|
||||||
const names = contract.variables.map((variable) => variable.name);
|
const names = contract.variables.map((variable) => variable.name);
|
||||||
const docs = renderWorkspaceDocs(workspaceV3);
|
const docs = renderWorkspaceDocs(workspaceV4);
|
||||||
|
|
||||||
expect(contract.workspaceId).toBe("psd-clinical");
|
expect(contract.workspaceId).toBe("psd-clinical");
|
||||||
expect(contract.namespace).toBe("PSD_CLINICAL");
|
expect(contract.namespace).toBe("PSD_CLINICAL");
|
||||||
@@ -54,22 +49,22 @@ test.each([
|
|||||||
|
|
||||||
test("validates public contract and documentation inputs at runtime", () => {
|
test("validates public contract and documentation inputs at runtime", () => {
|
||||||
const unsafeWorkspace = {
|
const unsafeWorkspace = {
|
||||||
...workspaceV3,
|
...workspaceV4,
|
||||||
workspace: { ...workspaceV3.workspace, id: "psd\nclinical" },
|
workspace: { ...workspaceV4.workspace, id: "psd\nclinical" },
|
||||||
} as CanonicalWorkspace;
|
} as CanonicalWorkspace;
|
||||||
|
|
||||||
expect(() => buildInstallationContract(unsafeWorkspace)).toThrow(/id/i);
|
expect(() => buildInstallationContract(unsafeWorkspace)).toThrow(/id/i);
|
||||||
expect(() => renderWorkspaceDocs(unsafeWorkspace)).toThrow(/id/i);
|
expect(() => renderWorkspaceDocs(unsafeWorkspace)).toThrow(/id/i);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("v3 installation contract omits external vector and embedding bindings", () => {
|
test("v4 installation contract omits external vector and embedding bindings", () => {
|
||||||
const contract = buildInstallationContract(workspaceV3);
|
const contract = buildInstallationContract(workspaceV4);
|
||||||
const names = contract.variables.map((variable) => variable.name);
|
const names = contract.variables.map((variable) => variable.name);
|
||||||
|
|
||||||
expect(names).toContain("THT_WS_PSD_CLINICAL_DWH_TRANSPORT");
|
expect(names).toContain("THT_WS_PSD_CLINICAL_DWH_TRANSPORT");
|
||||||
expect(names.some((name) => name.includes("_VECTOR_"))).toBe(false);
|
expect(names.some((name) => name.includes("_VECTOR_"))).toBe(false);
|
||||||
expect(names.some((name) => name.includes("_EMBEDDING_"))).toBe(false);
|
expect(names.some((name) => name.includes("_EMBEDDING_"))).toBe(false);
|
||||||
expect(renderWorkspaceDocs(workspaceV3).markdown).not.toContain("Embedding service");
|
expect(renderWorkspaceDocs(workspaceV4).markdown).not.toContain("Embedding service");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
||||||
@@ -114,7 +109,7 @@ test.each([
|
|||||||
|
|
||||||
function renderWorkspaceWithoutEvidence(): string {
|
function renderWorkspaceWithoutEvidence(): string {
|
||||||
return `workspace:
|
return `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -123,10 +118,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
`;
|
`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import type { RuntimeBindings } from "../src/workspaces/runtime-renderer.js";
|
|||||||
import { parseWorkspaceYaml, resolveDiagnosticUrl } from "../src/workspaces/schema.js";
|
import { parseWorkspaceYaml, resolveDiagnosticUrl } from "../src/workspaces/schema.js";
|
||||||
|
|
||||||
const workspace = parseWorkspaceYaml(`workspace:
|
const workspace = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -22,18 +22,6 @@ dwh:
|
|||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
timeout_ms: 8000
|
timeout_ms: 8000
|
||||||
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
supported_transports: [postgres_direct, rest_api, ssh_tunnel]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`);
|
`);
|
||||||
|
|
||||||
const bindings: RuntimeBindings = {
|
const bindings: RuntimeBindings = {
|
||||||
@@ -78,7 +66,7 @@ afterEach(() => {
|
|||||||
vi.restoreAllMocks();
|
vi.restoreAllMocks();
|
||||||
});
|
});
|
||||||
|
|
||||||
test("diagnoses schema-v3 DWH, internal Qdrant, and internal Ollama without semantic bindings", async () => {
|
test("diagnoses workspace-v4 DWH plus installation-derived Qdrant and Ollama", async () => {
|
||||||
const adapters = successfulAdapters();
|
const adapters = successfulAdapters();
|
||||||
const result = await diagnose(adapters)(workspace, bindings, { writeProbe: false });
|
const result = await diagnose(adapters)(workspace, bindings, { writeProbe: false });
|
||||||
|
|
||||||
@@ -139,7 +127,7 @@ test("reports only sanitized DWH and Evidence binding names before network diagn
|
|||||||
expect(adapters.inspectQdrant).not.toHaveBeenCalled();
|
expect(adapters.inspectQdrant).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
test("keeps schema-v3 DWH SSH diagnostic-only and runtime-inactive", async () => {
|
test("keeps workspace-v4 DWH SSH diagnostic-only and runtime-inactive", async () => {
|
||||||
const adapters = successfulAdapters();
|
const adapters = successfulAdapters();
|
||||||
const result = await diagnose(adapters)(workspace, {
|
const result = await diagnose(adapters)(workspace, {
|
||||||
...bindings,
|
...bindings,
|
||||||
@@ -152,9 +140,9 @@ test("keeps schema-v3 DWH SSH diagnostic-only and runtime-inactive", async () =>
|
|||||||
expect(adapters.probeConnector).not.toHaveBeenCalled();
|
expect(adapters.probeConnector).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
test("uses the schema-v3 declared DWH REST diagnostic and auth policy", async () => {
|
test("uses the workspace-v4 declared DWH REST diagnostic and auth policy", async () => {
|
||||||
const restWorkspace = parseWorkspaceYaml(`workspace:
|
const restWorkspace = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: REST workspace
|
name: REST workspace
|
||||||
language: en
|
language: en
|
||||||
@@ -163,16 +151,12 @@ dwh:
|
|||||||
database: warehouse
|
database: warehouse
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [rest_api]
|
supported_transports: [rest_api]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
diagnostics:
|
diagnostics:
|
||||||
dwh_rest:
|
dwh_rest:
|
||||||
method: POST
|
method: POST
|
||||||
path: /rpc/ping
|
path: /rpc/ping
|
||||||
auth: bearer
|
auth: bearer
|
||||||
response: { database: database, schema: schema }
|
response: { database: database, schema: schema }
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
`);
|
`);
|
||||||
const adapters = successfulAdapters();
|
const adapters = successfulAdapters();
|
||||||
const result = await diagnose(adapters)(restWorkspace, {
|
const result = await diagnose(adapters)(restWorkspace, {
|
||||||
@@ -423,7 +407,7 @@ test("uses a REST secret only as a header and redacts it from failed diagnostics
|
|||||||
const canary = "CANARY-REST-AUTH-SECRET";
|
const canary = "CANARY-REST-AUTH-SECRET";
|
||||||
await writeFile(credentialFile, canary);
|
await writeFile(credentialFile, canary);
|
||||||
const restDescriptor = parseWorkspaceYaml(`workspace:
|
const restDescriptor = parseWorkspaceYaml(`workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: REST auth
|
name: REST auth
|
||||||
language: en
|
language: en
|
||||||
@@ -432,16 +416,12 @@ dwh:
|
|||||||
database: warehouse
|
database: warehouse
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [rest_api]
|
supported_transports: [rest_api]
|
||||||
semantic_index:
|
|
||||||
vector_store: { engine: qdrant, collection: psd-clinical, dimensions: 1024, distance: cosine }
|
|
||||||
embedding: { provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024 }
|
|
||||||
diagnostics:
|
diagnostics:
|
||||||
dwh_rest:
|
dwh_rest:
|
||||||
method: GET
|
method: GET
|
||||||
path: /health
|
path: /health
|
||||||
auth: bearer
|
auth: bearer
|
||||||
response: { database: database, schema: schema }
|
response: { database: database, schema: schema }
|
||||||
llm_policy: { allowed: [zai/glm-5.2] }
|
|
||||||
`);
|
`);
|
||||||
const fetchMock = vi.fn()
|
const fetchMock = vi.fn()
|
||||||
.mockResolvedValueOnce(new Response("upstream CANARY-REST-AUTH-SECRET", { status: 503 }))
|
.mockResolvedValueOnce(new Response("upstream CANARY-REST-AUTH-SECRET", { status: 503 }))
|
||||||
|
|||||||
@@ -53,7 +53,7 @@ async function makeRepo(id: string, annotations: string | Buffer | "dir" | "syml
|
|||||||
writeFileSync(join(source, "thoth-workspaces.yaml"),
|
writeFileSync(join(source, "thoth-workspaces.yaml"),
|
||||||
`schema_version: 1\nworkspaces: [{id: ${id}, name: Workspace}]\n`);
|
`schema_version: 1\nworkspaces: [{id: ${id}, name: Workspace}]\n`);
|
||||||
mkdirSync(join(source, id, "schema"), { recursive: true });
|
mkdirSync(join(source, id, "schema"), { recursive: true });
|
||||||
writeFileSync(join(source, id, "workspace.yaml"), `workspace:\n schema_version: 3\n id: ${id}\n`);
|
writeFileSync(join(source, id, "workspace.yaml"), `workspace:\n schema_version: 4\n id: ${id}\n`);
|
||||||
const annotationsPath = join(source, id, "schema", "annotations.yaml");
|
const annotationsPath = join(source, id, "schema", "annotations.yaml");
|
||||||
if (annotations === "dir") {
|
if (annotations === "dir") {
|
||||||
mkdirSync(annotationsPath, { recursive: true });
|
mkdirSync(annotationsPath, { recursive: true });
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ async function fixture(layout: EvidenceLayout): Promise<{ root: string; remote:
|
|||||||
await git(source, ["config", "user.email", "evidence@example.invalid"]);
|
await git(source, ["config", "user.email", "evidence@example.invalid"]);
|
||||||
writeFileSync(join(source, "thoth-workspaces.yaml"), "schema_version: 1\nworkspaces: [{id: research, name: Research}]\n");
|
writeFileSync(join(source, "thoth-workspaces.yaml"), "schema_version: 1\nworkspaces: [{id: research, name: Research}]\n");
|
||||||
mkdirSync(join(source, "research"), { recursive: true });
|
mkdirSync(join(source, "research"), { recursive: true });
|
||||||
writeFileSync(join(source, "research", "workspace.yaml"), "workspace:\n schema_version: 3\n id: research\n");
|
writeFileSync(join(source, "research", "workspace.yaml"), "workspace:\n schema_version: 4\n id: research\n");
|
||||||
const evidence = join(source, "research", "evidence");
|
const evidence = join(source, "research", "evidence");
|
||||||
if (layout === "tree") {
|
if (layout === "tree") {
|
||||||
mkdirSync(join(evidence, "nested"), { recursive: true });
|
mkdirSync(join(evidence, "nested"), { recursive: true });
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import {
|
|||||||
import type { WorkspaceRegistryConfig } from "../src/workspaces/types.js";
|
import type { WorkspaceRegistryConfig } from "../src/workspaces/types.js";
|
||||||
|
|
||||||
const validYaml = `workspace:
|
const validYaml = `workspace:
|
||||||
schema_version: 2
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
language: it
|
language: it
|
||||||
@@ -21,21 +21,6 @@ dwh:
|
|||||||
database: postgres
|
database: postgres
|
||||||
schema: datawarehouse
|
schema: datawarehouse
|
||||||
supported_transports: [postgres_direct]
|
supported_transports: [postgres_direct]
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: pgvector
|
|
||||||
database: postgres
|
|
||||||
schema: vectors
|
|
||||||
collection: clinical_documents
|
|
||||||
dimensions: 768
|
|
||||||
distance: cosine
|
|
||||||
supported_transports: [pgvector_direct]
|
|
||||||
embedding:
|
|
||||||
provider: ollama_compatible
|
|
||||||
model: nomic-embed-text-v2-moe
|
|
||||||
dimensions: 768
|
|
||||||
llm_policy:
|
|
||||||
allowed: [zai/glm-5.2]
|
|
||||||
`;
|
`;
|
||||||
|
|
||||||
const runFile = promisify(execFile);
|
const runFile = promisify(execFile);
|
||||||
|
|||||||
+12
-21
@@ -13,20 +13,11 @@ import {
|
|||||||
import { renderRuntimeConfig, type RuntimeBindings } from "../src/workspaces/runtime-renderer.js";
|
import { renderRuntimeConfig, type RuntimeBindings } from "../src/workspaces/runtime-renderer.js";
|
||||||
|
|
||||||
const unsupportedWorkspace = {
|
const unsupportedWorkspace = {
|
||||||
workspace: { schema_version: 2, id: "legacy-workspace", name: "Legacy", language: "en" },
|
workspace: { schema_version: 3, id: "legacy-workspace", name: "Legacy", language: "en" },
|
||||||
dwh: {
|
dwh: {
|
||||||
engine: "postgres", database: "warehouse", schema: "public",
|
engine: "postgres", database: "warehouse", schema: "public",
|
||||||
supported_transports: ["postgres_direct"],
|
supported_transports: ["postgres_direct"],
|
||||||
},
|
},
|
||||||
semantic_index: {
|
|
||||||
vector_store: {
|
|
||||||
engine: "pgvector", database: "warehouse", schema: "vectors",
|
|
||||||
collection: "documents", dimensions: 768, distance: "cosine",
|
|
||||||
supported_transports: ["pgvector_direct"],
|
|
||||||
},
|
|
||||||
embedding: { provider: "ollama_compatible", model: "legacy", dimensions: 768 },
|
|
||||||
},
|
|
||||||
llm_policy: { allowed: ["zai/glm-5.2"] },
|
|
||||||
};
|
};
|
||||||
|
|
||||||
const bindings: RuntimeBindings = {
|
const bindings: RuntimeBindings = {
|
||||||
@@ -40,31 +31,31 @@ const adapters: DiagnosticAdapters = {
|
|||||||
probeEmbedding: vi.fn(),
|
probeEmbedding: vi.fn(),
|
||||||
};
|
};
|
||||||
|
|
||||||
test("renderer rejects callers that bypass the schema-v3 type contract", () => {
|
test("renderer rejects callers that bypass the schema-v4 type contract", () => {
|
||||||
expect(() => renderRuntimeConfig(unsupportedWorkspace as never, bindings, {
|
expect(() => renderRuntimeConfig(unsupportedWorkspace as never, bindings, {
|
||||||
sessions: "/data/sessions", artifacts: "/data/artifacts", indexes: "/data/indexes",
|
sessions: "/data/sessions", artifacts: "/data/artifacts", indexes: "/data/indexes",
|
||||||
})).toThrow("Runtime renderer supports only workspace schema version 3");
|
})).toThrow("Runtime renderer supports only workspace schema version 4");
|
||||||
});
|
});
|
||||||
|
|
||||||
test("installation contract rejects callers that bypass the schema-v3 type contract", () => {
|
test("installation contract rejects callers that bypass the schema-v4 type contract", () => {
|
||||||
expect(() => buildInstallationContract(unsupportedWorkspace as never))
|
expect(() => buildInstallationContract(unsupportedWorkspace as never))
|
||||||
.toThrow("Installation contract supports only workspace schema version 3");
|
.toThrow("Installation contract supports only workspace schema version 4");
|
||||||
});
|
});
|
||||||
|
|
||||||
test("binding entry points reject callers that bypass the schema-v3 type contract", () => {
|
test("binding entry points reject callers that bypass the schema-v4 type contract", () => {
|
||||||
expect(() => resolveBinding(unsupportedWorkspace as never, "DWH", {}, []))
|
expect(() => resolveBinding(unsupportedWorkspace as never, "DWH", {}, []))
|
||||||
.toThrow("Workspace bindings support only workspace schema version 3");
|
.toThrow("Workspace bindings support only workspace schema version 4");
|
||||||
expect(() => resolveEvidenceBinding(unsupportedWorkspace as never, {}, []))
|
expect(() => resolveEvidenceBinding(unsupportedWorkspace as never, {}, []))
|
||||||
.toThrow("Workspace bindings support only workspace schema version 3");
|
.toThrow("Workspace bindings support only workspace schema version 4");
|
||||||
expect(() => resolveRuntimeBindings(unsupportedWorkspace as never, {}, []))
|
expect(() => resolveRuntimeBindings(unsupportedWorkspace as never, {}, []))
|
||||||
.toThrow("Workspace bindings support only workspace schema version 3");
|
.toThrow("Workspace bindings support only workspace schema version 4");
|
||||||
});
|
});
|
||||||
|
|
||||||
test("diagnoser factories reject callers that bypass the schema-v3 type contract", async () => {
|
test("diagnoser factories reject callers that bypass the schema-v4 type contract", async () => {
|
||||||
await expect(createWorkspaceDiagnoser(adapters)(unsupportedWorkspace as never, bindings, {
|
await expect(createWorkspaceDiagnoser(adapters)(unsupportedWorkspace as never, bindings, {
|
||||||
writeProbe: false,
|
writeProbe: false,
|
||||||
})).rejects.toThrow("Workspace diagnoser supports only workspace schema version 3");
|
})).rejects.toThrow("Workspace diagnoser supports only workspace schema version 4");
|
||||||
await expect(createProductionWorkspaceDiagnoser(5_000, adapters)(
|
await expect(createProductionWorkspaceDiagnoser(5_000, adapters)(
|
||||||
unsupportedWorkspace as never, bindings, { writeProbe: false },
|
unsupportedWorkspace as never, bindings, { writeProbe: false },
|
||||||
)).rejects.toThrow("Workspace diagnoser supports only workspace schema version 3");
|
)).rejects.toThrow("Workspace diagnoser supports only workspace schema version 4");
|
||||||
});
|
});
|
||||||
@@ -10,7 +10,7 @@ import {
|
|||||||
} from "../src/workspaces/schema.js";
|
} from "../src/workspaces/schema.js";
|
||||||
|
|
||||||
export const validYaml = `workspace:
|
export const validYaml = `workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: psd-clinical
|
id: psd-clinical
|
||||||
name: Policlinico San Donato
|
name: Policlinico San Donato
|
||||||
description: Clinical data warehouse workspace
|
description: Clinical data warehouse workspace
|
||||||
@@ -25,33 +25,8 @@ dwh:
|
|||||||
- postgres_direct
|
- postgres_direct
|
||||||
- rest_api
|
- rest_api
|
||||||
- ssh_tunnel
|
- ssh_tunnel
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: psd-clinical
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
llm_policy:
|
|
||||||
default: zai/glm-5.2
|
|
||||||
allowed:
|
|
||||||
- zai/glm-5.2
|
|
||||||
- openai/gpt-5
|
|
||||||
`;
|
`;
|
||||||
|
|
||||||
test("rejects a workspace whose embedding dimensions differ from its collection", () => {
|
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace("dimensions: 1024", "dimensions: 1536")))
|
|
||||||
.toThrow(/dimensions/i);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects an LLM default outside its allowlist", () => {
|
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace("- zai/glm-5.2", "- openai/gpt-5")))
|
|
||||||
.toThrow(/allowlist/i);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects unknown keys and invalid immutable IDs", () => {
|
test("rejects unknown keys and invalid immutable IDs", () => {
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace(" language: it", " language: it\n label: PSD")))
|
expect(() => parseWorkspaceYaml(validYaml.replace(" language: it", " language: it\n label: PSD")))
|
||||||
.toThrow(/unrecognized key/i);
|
.toThrow(/unrecognized key/i);
|
||||||
@@ -74,94 +49,34 @@ test("accepts optional connection ports and timeouts but rejects unsafe values",
|
|||||||
.toThrow(/port/i);
|
.toThrow(/port/i);
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace("timeout_ms: 5000", "timeout_ms: 0")))
|
expect(() => parseWorkspaceYaml(validYaml.replace("timeout_ms: 5000", "timeout_ms: 0")))
|
||||||
.toThrow(/timeout/i);
|
.toThrow(/timeout/i);
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace("dimensions: 1024", "dimensions: 2048")))
|
|
||||||
.toThrow(/1024|dimensions/i);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test("accepts only the schema v3 internal qdrant semantic shape", () => {
|
test("accepts a model-free schema v4 workspace", () => {
|
||||||
expect(parseWorkspaceYaml(validYaml)).toMatchObject({
|
expect(parseWorkspaceYaml(validYaml)).toMatchObject({
|
||||||
workspace: { schema_version: 3, id: "psd-clinical" },
|
workspace: { schema_version: 4, id: "psd-clinical" },
|
||||||
semantic_index: {
|
dwh: { database: "postgres", schema: "datawarehouse" },
|
||||||
vector_store: {
|
|
||||||
engine: "qdrant",
|
|
||||||
collection: "psd-clinical",
|
|
||||||
dimensions: 1024,
|
|
||||||
distance: "cosine",
|
|
||||||
},
|
|
||||||
embedding: {
|
|
||||||
provider: "ollama_internal",
|
|
||||||
model: "qwen3-embedding:0.6b",
|
|
||||||
dimensions: 1024,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("committed example descriptors parse as exact schema v3 workspaces", () => {
|
test("committed example descriptors parse as exact schema v4 workspaces", () => {
|
||||||
const example = readFileSync(resolve(process.cwd(), "../deploy/workspaces/example.yaml"), "utf8");
|
const example = readFileSync(resolve(process.cwd(), "../deploy/workspaces/example.yaml"), "utf8");
|
||||||
const psdExample = readFileSync(resolve(process.cwd(), "../deploy/workspaces/psd.yaml.example"), "utf8");
|
const psdExample = readFileSync(resolve(process.cwd(), "../deploy/workspaces/psd.yaml.example"), "utf8");
|
||||||
|
|
||||||
expect(() => parseWorkspaceYaml(example)).not.toThrow();
|
expect(() => parseWorkspaceYaml(example)).not.toThrow();
|
||||||
expect(() => parseWorkspaceYaml(psdExample)).not.toThrow();
|
expect(() => parseWorkspaceYaml(psdExample)).not.toThrow();
|
||||||
expect(parseWorkspaceYaml(example)).toMatchObject({
|
expect(parseWorkspaceYaml(example)).toMatchObject({
|
||||||
workspace: { schema_version: 3 },
|
workspace: { schema_version: 4 },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", distance: "cosine", dimensions: 1024 },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
expect(parseWorkspaceYaml(psdExample)).toMatchObject({
|
expect(parseWorkspaceYaml(psdExample)).toMatchObject({
|
||||||
workspace: { schema_version: 3 },
|
workspace: { schema_version: 4 },
|
||||||
semantic_index: {
|
|
||||||
vector_store: { engine: "qdrant", distance: "cosine", dimensions: 1024 },
|
|
||||||
embedding: { provider: "ollama_internal", model: "qwen3-embedding:0.6b", dimensions: 1024 },
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("rejects pgvector semantic stores in schema v3", () => {
|
test("rejects installation-owned model and vector fields", () => {
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace("engine: qdrant", "engine: pgvector")))
|
expect(() => parseWorkspaceYaml(`${validYaml}llm_policy:\n allowed: [zai/glm-5.3]\n`))
|
||||||
.toThrow(/qdrant|pgvector/i);
|
.toThrow(/unrecognized key|llm_policy/i);
|
||||||
});
|
expect(() => parseWorkspaceYaml(`${validYaml}semantic_index: {}\n`))
|
||||||
|
.toThrow(/unrecognized key|semantic_index/i);
|
||||||
test("rejects supported_transports inside schema v3 semantic identity", () => {
|
|
||||||
const withTransport = validYaml.replace(
|
|
||||||
" distance: cosine\n",
|
|
||||||
" distance: cosine\n supported_transports:\n - rest_api\n",
|
|
||||||
);
|
|
||||||
|
|
||||||
expect(() => parseWorkspaceYaml(withTransport)).toThrow(/unrecognized key|supported_transports/i);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects external embedding providers in schema v3", () => {
|
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace("provider: ollama_internal", "provider: openai_compatible")))
|
|
||||||
.toThrow(/ollama_internal|provider/i);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects non-cosine distance in schema v3", () => {
|
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace("distance: cosine", "distance: l2")))
|
|
||||||
.toThrow(/cosine|distance/i);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects unknown fields in schema v3 semantic identity", () => {
|
|
||||||
const withUnknownField = validYaml.replace(
|
|
||||||
" collection: psd-clinical\n",
|
|
||||||
" collection: psd-clinical\n namespace: psd\n",
|
|
||||||
);
|
|
||||||
|
|
||||||
expect(() => parseWorkspaceYaml(withUnknownField)).toThrow(/unrecognized key/i);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("rejects legacy semantic connector fields and diagnostics in schema v3", () => {
|
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace(
|
|
||||||
" collection: psd-clinical\n",
|
|
||||||
" collection: psd-clinical\n database: postgres\n",
|
|
||||||
))).toThrow(/unrecognized key|database/i);
|
|
||||||
|
|
||||||
expect(() => parseWorkspaceYaml(validYaml.replace(
|
|
||||||
"llm_policy:\n",
|
|
||||||
"diagnostics:\n vector_rest:\n metadata:\n method: GET\n path: /metadata\n auth: bearer\n response:\n collection: collection\n dimensions: dimensions\n distance: distance\nllm_policy:\n",
|
|
||||||
))).toThrow(/unrecognized key|vector_rest/i);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
test.each([
|
test.each([
|
||||||
@@ -222,8 +137,8 @@ llm_policy:
|
|||||||
- zai/glm-5.2
|
- zai/glm-5.2
|
||||||
`],
|
`],
|
||||||
])("rejects schema %s descriptors at parser and object-validator boundaries", (_version, yaml) => {
|
])("rejects schema %s descriptors at parser and object-validator boundaries", (_version, yaml) => {
|
||||||
expect(() => parseWorkspaceYaml(yaml)).toThrow(/schema_version|invalid literal|3/i);
|
expect(() => parseWorkspaceYaml(yaml)).toThrow(/schema_version|invalid literal|4/i);
|
||||||
expect(() => validateWorkspaceDescriptor(parse(yaml))).toThrow(/schema_version|invalid literal|3/i);
|
expect(() => validateWorkspaceDescriptor(parse(yaml))).toThrow(/schema_version|invalid literal|4/i);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("does not expose the redundant canonical validator or v1 migration", () => {
|
test("does not expose the redundant canonical validator or v1 migration", () => {
|
||||||
@@ -253,9 +168,8 @@ test("rejects REST diagnostic declarations without their matching connector tran
|
|||||||
response:
|
response:
|
||||||
database: database
|
database: database
|
||||||
schema: schema
|
schema: schema
|
||||||
llm_policy:
|
|
||||||
`;
|
`;
|
||||||
const declared = validYaml.replace("llm_policy:\n", diagnostics);
|
const declared = `${validYaml}${diagnostics}`;
|
||||||
|
|
||||||
expect(() => parseWorkspaceYaml(declared.replace(" - rest_api\n", ""))).toThrow(/dwh_rest/i);
|
expect(() => parseWorkspaceYaml(declared.replace(" - rest_api\n", ""))).toThrow(/dwh_rest/i);
|
||||||
});
|
});
|
||||||
@@ -462,7 +376,7 @@ test.each(["curated/**/*.yaml", "curated/**"])(
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
test("keeps evidence optional on schema v3", () => {
|
test("keeps evidence optional on schema v4", () => {
|
||||||
expect(validateWorkspaceDescriptor(validWorkspaceObject())).not.toHaveProperty("evidence");
|
expect(validateWorkspaceDescriptor(validWorkspaceObject())).not.toHaveProperty("evidence");
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -471,7 +385,7 @@ test("serializes defaulted evidence canonically and parses it without loss", ()
|
|||||||
type: "filesystem",
|
type: "filesystem",
|
||||||
uri: "psd-clinical/evidence",
|
uri: "psd-clinical/evidence",
|
||||||
}));
|
}));
|
||||||
if (canonical.workspace.schema_version !== 3) throw new Error("expected schema v3");
|
if (canonical.workspace.schema_version !== 4) throw new Error("expected schema v4");
|
||||||
|
|
||||||
expect(parseWorkspaceYaml(serializeWorkspaceYaml(canonical))).toEqual(canonical);
|
expect(parseWorkspaceYaml(serializeWorkspaceYaml(canonical))).toEqual(canonical);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ async function fixture(): Promise<{ root: string; remote: string; commit: string
|
|||||||
await git(source, ["config", "user.email", "evidence-materializer@example.invalid"]);
|
await git(source, ["config", "user.email", "evidence-materializer@example.invalid"]);
|
||||||
writeFileSync(join(source, "thoth-workspaces.yaml"), "schema_version: 1\nworkspaces: [{id: research, name: Research}]\n");
|
writeFileSync(join(source, "thoth-workspaces.yaml"), "schema_version: 1\nworkspaces: [{id: research, name: Research}]\n");
|
||||||
mkdirSync(join(source, "research", "evidence", "nested"), { recursive: true });
|
mkdirSync(join(source, "research", "evidence", "nested"), { recursive: true });
|
||||||
writeFileSync(join(source, "research", "workspace.yaml"), "workspace:\n schema_version: 3\n id: research\n");
|
writeFileSync(join(source, "research", "workspace.yaml"), "workspace:\n schema_version: 4\n id: research\n");
|
||||||
writeFileSync(join(source, "research", "evidence", "guide.md"), "# guide\n");
|
writeFileSync(join(source, "research", "evidence", "guide.md"), "# guide\n");
|
||||||
writeFileSync(join(source, "research", "evidence", "nested", "deep.md"), "# deep\n");
|
writeFileSync(join(source, "research", "evidence", "nested", "deep.md"), "# deep\n");
|
||||||
await git(source, ["add", "-A"]);
|
await git(source, ["add", "-A"]);
|
||||||
@@ -102,7 +102,7 @@ test("refuses symlink-containing trees and bound violations without publishing",
|
|||||||
await git(source, ["config", "user.email", "e@e.invalid"]);
|
await git(source, ["config", "user.email", "e@e.invalid"]);
|
||||||
writeFileSync(join(source, "thoth-workspaces.yaml"), "schema_version: 1\nworkspaces: [{id: research, name: Research}]\n");
|
writeFileSync(join(source, "thoth-workspaces.yaml"), "schema_version: 1\nworkspaces: [{id: research, name: Research}]\n");
|
||||||
mkdirSync(join(source, "research", "evidence"), { recursive: true });
|
mkdirSync(join(source, "research", "evidence"), { recursive: true });
|
||||||
writeFileSync(join(source, "research", "workspace.yaml"), "workspace:\n schema_version: 3\n id: research\n");
|
writeFileSync(join(source, "research", "workspace.yaml"), "workspace:\n schema_version: 4\n id: research\n");
|
||||||
writeFileSync(join(source, "research", "outside.md"), "# outside\n");
|
writeFileSync(join(source, "research", "outside.md"), "# outside\n");
|
||||||
symlinkSync("../outside.md", join(source, "research", "evidence", "link.md"));
|
symlinkSync("../outside.md", join(source, "research", "evidence", "link.md"));
|
||||||
await git(source, ["add", "-A"]);
|
await git(source, ["add", "-A"]);
|
||||||
|
|||||||
@@ -36,15 +36,11 @@ services:
|
|||||||
THT_LLM_URL: ${THT_LLM_URL:-}
|
THT_LLM_URL: ${THT_LLM_URL:-}
|
||||||
THT_INTERNAL_QDRANT_URL: http://qdrant:6333
|
THT_INTERNAL_QDRANT_URL: http://qdrant:6333
|
||||||
THT_INTERNAL_EMBEDDING_URL: http://embedding:11434
|
THT_INTERNAL_EMBEDDING_URL: http://embedding:11434
|
||||||
THT_INTERNAL_EMBEDDING_MODEL: qwen3-embedding:0.6b
|
|
||||||
THT_INTERNAL_EMBEDDING_DIMENSIONS: "1024"
|
|
||||||
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
||||||
volumes:
|
volumes:
|
||||||
- settings:/data/settings
|
- settings:/data/settings
|
||||||
- pi-state:/home/thoth/.pi
|
- pi-state:/home/thoth/.pi
|
||||||
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
||||||
- ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro
|
|
||||||
- ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro
|
|
||||||
- workspace-registry:/data/workspace-registry
|
- workspace-registry:/data/workspace-registry
|
||||||
- workspace-secrets:/data/workspace-secrets
|
- workspace-secrets:/data/workspace-secrets
|
||||||
- sessions:/data/sessions
|
- sessions:/data/sessions
|
||||||
@@ -140,8 +136,6 @@ services:
|
|||||||
THT_LLM_URL: ${THT_LLM_URL:-}
|
THT_LLM_URL: ${THT_LLM_URL:-}
|
||||||
THT_INTERNAL_QDRANT_URL: http://qdrant:6333
|
THT_INTERNAL_QDRANT_URL: http://qdrant:6333
|
||||||
THT_INTERNAL_EMBEDDING_URL: http://embedding:11434
|
THT_INTERNAL_EMBEDDING_URL: http://embedding:11434
|
||||||
THT_INTERNAL_EMBEDDING_MODEL: qwen3-embedding:0.6b
|
|
||||||
THT_INTERNAL_EMBEDDING_DIMENSIONS: "1024"
|
|
||||||
HOME: /tmp/thoth
|
HOME: /tmp/thoth
|
||||||
AWS_ACCESS_KEY_ID: ""
|
AWS_ACCESS_KEY_ID: ""
|
||||||
AWS_SECRET_ACCESS_KEY: ""
|
AWS_SECRET_ACCESS_KEY: ""
|
||||||
@@ -243,7 +237,6 @@ services:
|
|||||||
entrypoint: ["/usr/bin/bash", "/opt/thoth/embedding-model-init.sh"]
|
entrypoint: ["/usr/bin/bash", "/opt/thoth/embedding-model-init.sh"]
|
||||||
environment:
|
environment:
|
||||||
OLLAMA_BASE_URL: http://embedding:11434
|
OLLAMA_BASE_URL: http://embedding:11434
|
||||||
OLLAMA_MODEL: qwen3-embedding:0.6b
|
|
||||||
OLLAMA_WAIT_TIMEOUT_SEC: "180"
|
OLLAMA_WAIT_TIMEOUT_SEC: "180"
|
||||||
volumes:
|
volumes:
|
||||||
- embedding-models:/root/.ollama
|
- embedding-models:/root/.ollama
|
||||||
|
|||||||
@@ -21,14 +21,6 @@ services:
|
|||||||
source: ${PI_AUTH_FILE:?set PI_AUTH_FILE}
|
source: ${PI_AUTH_FILE:?set PI_AUTH_FILE}
|
||||||
target: /home/thoth/.pi/agent/auth.json
|
target: /home/thoth/.pi/agent/auth.json
|
||||||
read_only: true
|
read_only: true
|
||||||
- type: bind
|
|
||||||
source: ./deploy/pi/models.json
|
|
||||||
target: /home/thoth/.pi/agent/models.json
|
|
||||||
read_only: true
|
|
||||||
- type: bind
|
|
||||||
source: ./deploy/pi/settings.json
|
|
||||||
target: /home/thoth/.pi/agent/settings.json
|
|
||||||
read_only: true
|
|
||||||
- type: bind
|
- type: bind
|
||||||
source: ${THT_WORKSPACE_REGISTRY_ROOT:?set THT_WORKSPACE_REGISTRY_ROOT}
|
source: ${THT_WORKSPACE_REGISTRY_ROOT:?set THT_WORKSPACE_REGISTRY_ROOT}
|
||||||
target: /data/workspace-registry
|
target: /data/workspace-registry
|
||||||
|
|||||||
@@ -1,46 +0,0 @@
|
|||||||
{
|
|
||||||
"providers": {
|
|
||||||
"zai": {
|
|
||||||
"baseUrl": "https://api.z.ai/api/coding/paas/v4",
|
|
||||||
"api": "openai-completions",
|
|
||||||
"apiKey": "$ZAI_API_KEY",
|
|
||||||
"models": [
|
|
||||||
{
|
|
||||||
"id": "glm-5.3",
|
|
||||||
"name": "GLM-5.3",
|
|
||||||
"reasoning": true,
|
|
||||||
"contextWindow": 200000,
|
|
||||||
"maxTokens": 131072
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"local-qwen": {
|
|
||||||
"name": "Local Qwen",
|
|
||||||
"baseUrl": "https://ml-aritmolab.policlinicosandonato.it/v1",
|
|
||||||
"api": "openai-completions",
|
|
||||||
"apiKey": "local",
|
|
||||||
"models": [
|
|
||||||
{
|
|
||||||
"id": "qwen3.6-35b-a3b",
|
|
||||||
"name": "Qwen3.6 35B A3B",
|
|
||||||
"reasoning": false,
|
|
||||||
"input": ["text"],
|
|
||||||
"cost": {
|
|
||||||
"input": 0,
|
|
||||||
"output": 0,
|
|
||||||
"cacheRead": 0,
|
|
||||||
"cacheWrite": 0
|
|
||||||
},
|
|
||||||
"contextWindow": 131072,
|
|
||||||
"maxTokens": 16384,
|
|
||||||
"compat": {
|
|
||||||
"supportsDeveloperRole": false,
|
|
||||||
"supportsReasoningEffort": false,
|
|
||||||
"supportsStore": false,
|
|
||||||
"maxTokensField": "max_tokens"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
{
|
|
||||||
"defaultProjectTrust": "always",
|
|
||||||
"enabledModels": [
|
|
||||||
"zai/glm-5.3",
|
|
||||||
"deepseek/deepseek-v4-flash",
|
|
||||||
"deepseek/deepseek-v4-pro",
|
|
||||||
"local-qwen/qwen3.6-35b-a3b"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
@@ -14,9 +14,7 @@ THT_AUTH_CONFIG_ROOT=<abs>/deploy/psd/auth
|
|||||||
# DWH and Evidence credentials are entered later in Workspace management and stored encrypted
|
# DWH and Evidence credentials are entered later in Workspace management and stored encrypted
|
||||||
# by the backend. They do not depend on host filesystem paths.
|
# by the backend. They do not depend on host filesystem paths.
|
||||||
|
|
||||||
# Pi (LLM)
|
# Pi runtime preference; provider/model defaults live only in installation modelCatalog.
|
||||||
PI_PROVIDER=zai
|
|
||||||
PI_MODEL=glm-5.3
|
|
||||||
PI_THINKING=medium
|
PI_THINKING=medium
|
||||||
|
|
||||||
# App defaults
|
# App defaults
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Esempio soltanto: `tht setup` genera deploy/<installation-id>/thothii-installation.yaml.
|
# Example only: `tht setup` generates deploy/<installation-id>/thothii-installation.yaml.
|
||||||
# Sostituisci i path assoluti se usi questo riferimento per una configurazione avanzata.
|
# Replace every absolute path before using this as an advanced reference.
|
||||||
# Seleziona UN solo override Git (https o ssh).
|
schemaVersion: 2
|
||||||
profile: local
|
profile: local
|
||||||
projectDirectory: "<abs>/projects/ThothII"
|
projectDirectory: "<abs>/projects/ThothII"
|
||||||
envFile: "<abs>/projects/ThothII/deploy/psd/operator.env"
|
envFile: "<abs>/projects/ThothII/deploy/psd/operator.env"
|
||||||
@@ -8,38 +8,67 @@ workspaceRepository:
|
|||||||
remote: git@github.com:mptyl/tht-workspace-psd.git
|
remote: git@github.com:mptyl/tht-workspace-psd.git
|
||||||
branch: main
|
branch: main
|
||||||
access: ssh
|
access: ssh
|
||||||
metadataGeneration:
|
modelCatalog:
|
||||||
default: glm-53
|
defaults:
|
||||||
models:
|
session: zai/glm-5.3
|
||||||
- id: deepseek-v4-pro
|
metadataGeneration: zai/glm-5.3
|
||||||
label: DeepSeek V4 Pro
|
embedding:
|
||||||
litellm:
|
id: ollama/qwen3-embedding:0.6b
|
||||||
provider: deepseek
|
dimensions: 1024
|
||||||
model: deepseek-v4-pro
|
providers:
|
||||||
apiKeyEnv: DEEPSEEK_API_KEY
|
deepseek:
|
||||||
- id: deepseek-v4-flash
|
authentication:
|
||||||
label: DeepSeek V4 Flash
|
mode: pi_auth
|
||||||
litellm:
|
session:
|
||||||
provider: deepseek
|
mode: pi_builtin
|
||||||
model: deepseek-v4-flash
|
models:
|
||||||
apiKeyEnv: DEEPSEEK_API_KEY
|
deepseek-v4-pro:
|
||||||
- id: glm-53
|
session: {}
|
||||||
label: GLM 5.3
|
deepseek-v4-flash:
|
||||||
litellm:
|
session: {}
|
||||||
provider: openai
|
zai:
|
||||||
model: glm-5.3
|
endpoint:
|
||||||
endpoint:
|
baseUrl: https://api.z.ai/api/coding/paas/v4
|
||||||
baseUrl: https://api.z.ai/api/coding/paas/v4
|
authentication:
|
||||||
apiKeyEnv: ZAI_API_KEY
|
mode: secret_env
|
||||||
- id: qwen-36
|
apiKeyEnv: ZAI_API_KEY
|
||||||
label: AritmoLab Qwen 3.6 35B A3B
|
session:
|
||||||
litellm:
|
mode: openai_compatible
|
||||||
provider: openai
|
metadataGeneration:
|
||||||
model: qwen3.6-35b-a3b
|
litellmProvider: openai
|
||||||
disableThinking: true
|
models:
|
||||||
endpoint:
|
glm-5.3:
|
||||||
baseUrl: https://ml-aritmolab.policlinicosandonato.it/v1
|
label: GLM 5.3
|
||||||
# Qwen omette apiKeyEnv: l'endpoint VPN non autentica le richieste.
|
session:
|
||||||
|
reasoning: true
|
||||||
|
contextWindow: 200000
|
||||||
|
maxTokens: 131072
|
||||||
|
metadataGeneration: {}
|
||||||
|
local-qwen:
|
||||||
|
endpoint:
|
||||||
|
baseUrl: https://ml-aritmolab.policlinicosandonato.it/v1
|
||||||
|
authentication:
|
||||||
|
mode: none
|
||||||
|
session:
|
||||||
|
mode: openai_compatible
|
||||||
|
metadataGeneration:
|
||||||
|
litellmProvider: openai
|
||||||
|
models:
|
||||||
|
qwen3.6-35b-a3b:
|
||||||
|
label: AritmoLab Qwen 3.6 35B A3B
|
||||||
|
session:
|
||||||
|
reasoning: false
|
||||||
|
input: [text]
|
||||||
|
cost: {input: 0, output: 0, cacheRead: 0, cacheWrite: 0}
|
||||||
|
contextWindow: 131072
|
||||||
|
maxTokens: 16384
|
||||||
|
compatibility:
|
||||||
|
supportsDeveloperRole: false
|
||||||
|
supportsReasoningEffort: false
|
||||||
|
supportsStore: false
|
||||||
|
maxTokensField: max_tokens
|
||||||
|
metadataGeneration:
|
||||||
|
disableThinking: true
|
||||||
authentication:
|
authentication:
|
||||||
configDirectory: "<abs>/projects/ThothII/deploy/psd/auth"
|
configDirectory: "<abs>/projects/ThothII/deploy/psd/auth"
|
||||||
overrides:
|
overrides:
|
||||||
|
|||||||
@@ -10,8 +10,9 @@ chmod 600 deploy/secrets/thothii.secrets
|
|||||||
|
|
||||||
The file uses strict `KEY=VALUE` lines (comments and blank lines are allowed). The supported
|
The file uses strict `KEY=VALUE` lines (comments and blank lines are allowed). The supported
|
||||||
installation keys are `THT_MODEL_API_KEY`, `THT_DWH_API_KEY`, `THT_CA`, `THT_SSL_CA`,
|
installation keys are `THT_MODEL_API_KEY`, `THT_DWH_API_KEY`, `THT_CA`, `THT_SSL_CA`,
|
||||||
`THT_OIDC_CLIENT_SECRET`, and `THT_AUTHENTIK_API_TOKEN`. Metadata-generation models may reference
|
`THT_OIDC_CLIENT_SECRET`, and `THT_AUTHENTIK_API_TOKEN`. Installation Model Catalog providers may
|
||||||
exactly one of `THT_METADATA_API_KEY`, `ANTHROPIC_API_KEY`, `AZURE_API_KEY`, `GEMINI_API_KEY`,
|
reference exactly one of `THT_MODEL_API_KEY`, `THT_METADATA_API_KEY`, `ANTHROPIC_API_KEY`,
|
||||||
|
`AZURE_API_KEY`, `GEMINI_API_KEY`,
|
||||||
`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`, or `ZAI_API_KEY` through their
|
`DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`, or `ZAI_API_KEY` through their
|
||||||
descriptor `apiKeyEnv`. An entry for an explicitly configured endpoint that accepts unauthenticated
|
descriptor `apiKeyEnv`. An entry for an explicitly configured endpoint that accepts unauthenticated
|
||||||
requests may omit `apiKeyEnv`; hosted/default endpoints must always reference a key.
|
requests may omit `apiKeyEnv`; hosted/default endpoints must always reference a key.
|
||||||
@@ -23,10 +24,10 @@ installation. Other configured values must be non-empty and contain no
|
|||||||
whitespace. Do not put secrets in the root `.env`, installation YAML, workspace YAML, URLs, logs,
|
whitespace. Do not put secrets in the root `.env`, installation YAML, workspace YAML, URLs, logs,
|
||||||
or rendered Compose output.
|
or rendered Compose output.
|
||||||
|
|
||||||
`THT_MODEL_API_KEY` remains the generic Pi child credential. Metadata generation is a separate
|
Session and metadata-generation runtimes read only the provider key named by
|
||||||
backend-owned runtime and reads only the key named by its own `metadataGeneration.models[].apiKeyEnv`
|
`modelCatalog.providers.<provider>.authentication.apiKeyEnv`. They share the declaration and
|
||||||
(when present);
|
credential reference, not their execution lifecycle. Pi-owned authentication remains available only
|
||||||
it does not read Pi settings, `PI_AUTH_FILE`, or workspace `llm_policy`.
|
to session-only built-in providers through `authentication.mode: pi_auth`.
|
||||||
|
|
||||||
Do not add vector or embedding endpoint credentials to the bundle. Active operator manuals use
|
Do not add vector or embedding endpoint credentials to the bundle. Active operator manuals use
|
||||||
internal Qdrant and Ollama services, so vector/embedding runtime endpoint secrets are not part of
|
internal Qdrant and Ollama services, so vector/embedding runtime endpoint secrets are not part of
|
||||||
@@ -56,7 +57,7 @@ and only then deleting the old files. The old variables remain a compatibility p
|
|||||||
upgrades, but the documented and tested default is an absolute `THT_SECRETS_FILE` path to the
|
upgrades, but the documented and tested default is an absolute `THT_SECRETS_FILE` path to the
|
||||||
protected bundle.
|
protected bundle.
|
||||||
|
|
||||||
Hosted Pi providers must use a single model key through `THT_MODEL_API_KEY`. Compound providers
|
Hosted providers must use one explicitly named catalog key. Compound providers
|
||||||
(Bedrock, Azure OpenAI Responses, Cloudflare Workers AI/Gateway) fail closed until a
|
(Bedrock, Azure OpenAI Responses, Cloudflare Workers AI/Gateway) fail closed until a
|
||||||
provider-specific credential adapter is implemented.
|
provider-specific credential adapter is implemented.
|
||||||
|
|
||||||
|
|||||||
@@ -2,14 +2,13 @@
|
|||||||
# Values are read as literal strings (no shell expansion or command substitution).
|
# Values are read as literal strings (no shell expansion or command substitution).
|
||||||
# Leave unused keys out of the file.
|
# Leave unused keys out of the file.
|
||||||
|
|
||||||
# Hosted model provider (single-key providers only).
|
# Hosted catalog provider (single-key providers only). The selected name must match
|
||||||
# THT_MODEL_API_KEY=replace-me
|
# modelCatalog.providers.<provider>.authentication.apiKeyEnv.
|
||||||
|
# Allowed names include THT_MODEL_API_KEY, THT_METADATA_API_KEY, ANTHROPIC_API_KEY,
|
||||||
# Description Generation only. The selected name must match apiKeyEnv in metadataGeneration.
|
# AZURE_API_KEY, GEMINI_API_KEY,
|
||||||
# Allowed names: THT_METADATA_API_KEY, ANTHROPIC_API_KEY, AZURE_API_KEY, GEMINI_API_KEY,
|
|
||||||
# DEEPSEEK_API_KEY, OPENAI_API_KEY, OPENROUTER_API_KEY, or ZAI_API_KEY.
|
# DEEPSEEK_API_KEY, OPENAI_API_KEY, OPENROUTER_API_KEY, or ZAI_API_KEY.
|
||||||
# OPENAI_API_KEY=replace-me
|
# OPENAI_API_KEY=replace-me
|
||||||
# A model with an explicit unauthenticated endpoint omits apiKeyEnv and needs no bundle entry.
|
# A provider with an explicit keyless endpoint uses authentication.mode: none.
|
||||||
|
|
||||||
# External DWH adapter.
|
# External DWH adapter.
|
||||||
# THT_DWH_API_KEY=replace-me
|
# THT_DWH_API_KEY=replace-me
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
workspace:
|
workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: example
|
id: example
|
||||||
name: Example workspace
|
name: Example workspace
|
||||||
description: Generic example WorkspaceV3 descriptor.
|
description: Generic example WorkspaceV4 descriptor.
|
||||||
language: en
|
language: en
|
||||||
|
|
||||||
dwh:
|
dwh:
|
||||||
@@ -13,17 +13,6 @@ dwh:
|
|||||||
- postgres_direct
|
- postgres_direct
|
||||||
- rest_api
|
- rest_api
|
||||||
|
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: example
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
|
|
||||||
evidence:
|
evidence:
|
||||||
source:
|
source:
|
||||||
type: filesystem
|
type: filesystem
|
||||||
@@ -35,11 +24,6 @@ evidence:
|
|||||||
max_chunk_chars: 4000
|
max_chunk_chars: 4000
|
||||||
retain_published_generations: 3
|
retain_published_generations: 3
|
||||||
|
|
||||||
llm_policy:
|
|
||||||
default: zai/glm-5.2
|
|
||||||
allowed:
|
|
||||||
- zai/glm-5.2
|
|
||||||
|
|
||||||
diagnostics:
|
diagnostics:
|
||||||
dwh_rest:
|
dwh_rest:
|
||||||
method: GET
|
method: GET
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
workspace:
|
workspace:
|
||||||
schema_version: 3
|
schema_version: 4
|
||||||
id: example-workspace
|
id: example-workspace
|
||||||
name: Example Workspace
|
name: Example Workspace
|
||||||
description: Example WorkspaceV3 descriptor.
|
description: Example WorkspaceV4 descriptor.
|
||||||
language: en
|
language: en
|
||||||
|
|
||||||
dwh:
|
dwh:
|
||||||
@@ -13,17 +13,6 @@ dwh:
|
|||||||
- postgres_direct
|
- postgres_direct
|
||||||
- rest_api
|
- rest_api
|
||||||
|
|
||||||
semantic_index:
|
|
||||||
vector_store:
|
|
||||||
engine: qdrant
|
|
||||||
collection: example-workspace
|
|
||||||
dimensions: 1024
|
|
||||||
distance: cosine
|
|
||||||
embedding:
|
|
||||||
provider: ollama_internal
|
|
||||||
model: qwen3-embedding:0.6b
|
|
||||||
dimensions: 1024
|
|
||||||
|
|
||||||
evidence:
|
evidence:
|
||||||
source:
|
source:
|
||||||
type: filesystem
|
type: filesystem
|
||||||
@@ -35,11 +24,6 @@ evidence:
|
|||||||
max_chunk_chars: 4000
|
max_chunk_chars: 4000
|
||||||
retain_published_generations: 3
|
retain_published_generations: 3
|
||||||
|
|
||||||
llm_policy:
|
|
||||||
default: zai/glm-5.2
|
|
||||||
allowed:
|
|
||||||
- zai/glm-5.2
|
|
||||||
|
|
||||||
diagnostics:
|
diagnostics:
|
||||||
dwh_rest:
|
dwh_rest:
|
||||||
method: GET
|
method: GET
|
||||||
|
|||||||
@@ -1,6 +1,5 @@
|
|||||||
# ThothII standalone development/smoke stack.
|
# ThothII standalone development/smoke base. Model settings are intentionally absent;
|
||||||
# Run with the canonical local env file:
|
# use `tht start --build`, which adds the generated modelCatalog projection.
|
||||||
# docker compose --env-file deploy/env/local.env -f docker-compose.dev.yml up -d --build
|
|
||||||
# frontend: http://localhost:8090 backend: http://localhost:8787
|
# frontend: http://localhost:8090 backend: http://localhost:8787
|
||||||
name: thothii-dev
|
name: thothii-dev
|
||||||
|
|
||||||
@@ -36,8 +35,6 @@ services:
|
|||||||
THT_LLM_URL: ${THT_LLM_URL:-}
|
THT_LLM_URL: ${THT_LLM_URL:-}
|
||||||
THT_INTERNAL_QDRANT_URL: http://qdrant:6333
|
THT_INTERNAL_QDRANT_URL: http://qdrant:6333
|
||||||
THT_INTERNAL_EMBEDDING_URL: http://embedding:11434
|
THT_INTERNAL_EMBEDDING_URL: http://embedding:11434
|
||||||
THT_INTERNAL_EMBEDDING_MODEL: qwen3-embedding:0.6b
|
|
||||||
THT_INTERNAL_EMBEDDING_DIMENSIONS: "1024"
|
|
||||||
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
MAX_PI_PROCESSES: ${MAX_PI_PROCESSES:-4}
|
||||||
extra_hosts:
|
extra_hosts:
|
||||||
- "host.docker.internal:host-gateway"
|
- "host.docker.internal:host-gateway"
|
||||||
@@ -45,8 +42,6 @@ services:
|
|||||||
- dev-data:/data
|
- dev-data:/data
|
||||||
- dev-pi-state:/home/thoth/.pi
|
- dev-pi-state:/home/thoth/.pi
|
||||||
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
- ${PI_AUTH_FILE:?set PI_AUTH_FILE}:/home/thoth/.pi/agent/auth.json:ro
|
||||||
- ./deploy/pi/models.json:/home/thoth/.pi/agent/models.json:ro
|
|
||||||
- ./deploy/pi/settings.json:/home/thoth/.pi/agent/settings.json:ro
|
|
||||||
- workspace-registry:/data/workspace-registry
|
- workspace-registry:/data/workspace-registry
|
||||||
- workspace-secrets:/data/workspace-secrets
|
- workspace-secrets:/data/workspace-secrets
|
||||||
- ${THT_DEV_EVIDENCE_HOST_PATH:-./evidence}:/data/evidence:ro
|
- ${THT_DEV_EVIDENCE_HOST_PATH:-./evidence}:/data/evidence:ro
|
||||||
@@ -129,7 +124,6 @@ services:
|
|||||||
entrypoint: ["/usr/bin/bash", "/opt/thoth/embedding-model-init.sh"]
|
entrypoint: ["/usr/bin/bash", "/opt/thoth/embedding-model-init.sh"]
|
||||||
environment:
|
environment:
|
||||||
OLLAMA_BASE_URL: http://embedding:11434
|
OLLAMA_BASE_URL: http://embedding:11434
|
||||||
OLLAMA_MODEL: qwen3-embedding:0.6b
|
|
||||||
OLLAMA_WAIT_TIMEOUT_SEC: "180"
|
OLLAMA_WAIT_TIMEOUT_SEC: "180"
|
||||||
volumes:
|
volumes:
|
||||||
- embedding-models:/root/.ollama
|
- embedding-models:/root/.ollama
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user