378 lines
21 KiB
HTML
378 lines
21 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
|
|
|
|
<meta charset="utf-8">
|
|
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
|
|
|
|
<link rel="canonical" href="https://git.tylconsulting.it/thothii-docs/general/pi-configuration/">
|
|
<link rel="shortcut icon" href="../../img/favicon.ico">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=0" />
|
|
<title>Model configuration - ThothII Docs</title>
|
|
<link href="../../css/bootstrap-3.3.7.min.css" rel="stylesheet">
|
|
<link href="../../css/font-awesome-4.7.0.css" rel="stylesheet">
|
|
<link href="../../css/base.css" rel="stylesheet">
|
|
<link rel="stylesheet" href="../../css/highlight.css">
|
|
<link href="../../stylesheets/extra.css" rel="stylesheet">
|
|
<!-- HTML5 shim and Respond.js IE8 support of HTML5 elements and media queries -->
|
|
<!--[if lt IE 9]>
|
|
<script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
|
|
<script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
|
|
<![endif]-->
|
|
|
|
<script src="../../js/jquery-3.2.1.min.js"></script>
|
|
<script src="../../js/bootstrap-3.3.7.min.js"></script>
|
|
<script src="../../js/highlight.pack.js"></script>
|
|
|
|
<base target="_top">
|
|
<script>
|
|
var base_url = '../..';
|
|
var is_top_frame = false;
|
|
|
|
var pageToc = [
|
|
{title: "Installation Model Catalog", url: "#_top", children: [
|
|
{title: "Host instructions in Administration", url: "#host-instructions-in-administration" },
|
|
{title: "Minimal catalog", url: "#minimal-catalog" },
|
|
{title: "Session adapters", url: "#session-adapters" },
|
|
{title: "Authentication", url: "#authentication" },
|
|
{title: "Generated runtime projections", url: "#generated-runtime-projections" },
|
|
{title: "Migrating a legacy installation", url: "#migrating-a-legacy-installation" },
|
|
{title: "Troubleshooting", url: "#troubleshooting" },
|
|
]},
|
|
];
|
|
|
|
</script>
|
|
<script src="../../js/base.js"></script>
|
|
<script src="../../javascripts/layout-init.js"></script>
|
|
</head>
|
|
|
|
<body>
|
|
<script>
|
|
if (is_top_frame) { $('body').addClass('wm-top-page'); }
|
|
</script>
|
|
|
|
|
|
|
|
<div class="container-fluid wm-page-content">
|
|
<a name="_top"></a>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div class="row wm-article-nav-buttons" role="navigation" aria-label="navigation">
|
|
|
|
<div class="wm-article-nav pull-right">
|
|
<a href="../../operations/workspaces/" class="btn btn-xs btn-default pull-right">
|
|
Next
|
|
<i class="fa fa-chevron-right" aria-hidden="true"></i>
|
|
</a>
|
|
<a href="../../operations/workspaces/" class="btn btn-xs btn-link">
|
|
Workspaces
|
|
</a>
|
|
</div>
|
|
|
|
<div class="wm-article-nav">
|
|
<a href="../../install/authentik/" class="btn btn-xs btn-default pull-left">
|
|
<i class="fa fa-chevron-left" aria-hidden="true"></i>
|
|
Previous</a><a href="../../install/authentik/" class="btn btn-xs btn-link">
|
|
Authentik
|
|
</a>
|
|
</div>
|
|
|
|
</div>
|
|
|
|
|
|
|
|
<h1 id="installation-model-catalog">Installation Model Catalog<a class="headerlink" href="#installation-model-catalog" title="Permanent link">¶</a></h1>
|
|
<p>ThothII has one operator-authored model source: <code>modelCatalog</code> in
|
|
<code>deploy/<installation-id>/thothii-installation.yaml</code>. It declares models used by interactive Pi
|
|
sessions, metadata generation, and the internal embedding service. Workspace descriptors never
|
|
declare providers, model allowlists, defaults, embeddings, dimensions, or vector-store settings.</p>
|
|
<p>Do not edit <code>deploy/pi/models.json</code>, <code>deploy/pi/settings.json</code>, files under <code>generated/</code>, or
|
|
provider/model environment defaults. Those former sources are retired.</p>
|
|
<h2 id="host-instructions-in-administration">Host instructions in Administration<a class="headerlink" href="#host-instructions-in-administration" title="Permanent link">¶</a></h2>
|
|
<p>The <strong>Pi configuration</strong> navigation button opens <strong>Pi management</strong>. Its Host maintenance
|
|
section selects the installation host's Linux, macOS, or Windows tab automatically; the
|
|
browser's operating system does not affect it. Selecting another tab is still possible.</p>
|
|
<p>The host CLI includes <code>THT_HOST_PLATFORM</code> in the generated Compose projection using the OS
|
|
on which it runs. Generate projections on the destination host, not on another computer,
|
|
and do not edit generated files. Without this projection (older deployments or native
|
|
development), the backend reports its own OS; a Linux Docker container cannot discover
|
|
whether the physical host is macOS or Windows. Run the updated host CLI's normal
|
|
configuration reload on the destination installation to regenerate this information.</p>
|
|
<h2 id="minimal-catalog">Minimal catalog<a class="headerlink" href="#minimal-catalog" title="Permanent link">¶</a></h2>
|
|
<pre class="highlight"><code class="language-yaml">schemaVersion: 2
|
|
modelCatalog:
|
|
defaults:
|
|
interaction: zai/glm-5.3
|
|
|
|
embedding:
|
|
id: ollama/qwen3-embedding:0.6b
|
|
dimensions: 1024
|
|
|
|
providers:
|
|
zai:
|
|
endpoint:
|
|
baseUrl: https://api.z.ai/api/coding/paas/v4
|
|
authentication:
|
|
mode: secret_env
|
|
apiKeyEnv: ZAI_API_KEY
|
|
session:
|
|
mode: openai_compatible
|
|
metadataGeneration:
|
|
litellmProvider: openai
|
|
models:
|
|
glm-5.3:
|
|
label: GLM 5.3
|
|
session:
|
|
reasoning: true
|
|
contextWindow: 200000
|
|
maxTokens: 131072
|
|
metadataGeneration: {}</code></pre>
|
|
<p>Provider and model entries are maps. The keys form the canonical identity
|
|
<code><provider-key>/<model-key></code>; <code>label</code> is only display text. A model is eligible for a use only when
|
|
it contains that use block:</p>
|
|
<ul>
|
|
<li><code>session</code> makes it selectable for interactive sessions;</li>
|
|
<li><code>metadataGeneration</code> makes it selectable for description generation;</li>
|
|
<li><code>embedding</code> is a single installation-level model rather than a selectable list.</li>
|
|
</ul>
|
|
<p><code>defaults.interaction</code> is the only LLM default, required once per installation, never per workspace.
|
|
Core and Administration share the user's operational model choice. An explicit choice takes priority
|
|
over the default and is remembered in this browser for the authenticated user and application mount.
|
|
Switching workspace does not change the model. Browser-storage restrictions may limit remembering
|
|
to the current visit; preferences do not synchronize across devices or change installation YAML.
|
|
An unavailable remembered model is not silently replaced: select another configured model.</p>
|
|
<p>When any metadata-generation models are configured, the default and the operational list must
|
|
support both <code>session</code> and <code>metadataGeneration</code>. Entries for just one adapter may remain in the
|
|
installation inventory, but are not selectable for global interaction. Core-only installations remain
|
|
supported when no metadata-generation model is configured; Admin AI is then unavailable.
|
|
The embedding model remains separate and is unaffected by the interaction selector.</p>
|
|
<p>New sessions record the selected model in their manifest. Resume retains the session's workspace and
|
|
revision, but uses the current global model (the installation default for clients that omit a model).
|
|
Historical manifest model fields are not rewritten by resume. Archived/finalized sessions remain read-only.</p>
|
|
<h3 id="required-operator-verification-for-every-model">Required operator verification for every model<a class="headerlink" href="#required-operator-verification-for-every-model" title="Permanent link">¶</a></h3>
|
|
<p>Catalog validation checks configuration, not model behavior. Before offering a model to users, and
|
|
after changing its endpoint, adapters, or the Pi/LiteLLM versions, the operator must verify <strong>both</strong>:</p>
|
|
<ol>
|
|
<li><strong>Core / Pi:</strong> select the model, start a test session in a prepared test workspace, exercise an
|
|
actual tool call and its returned result, a human review gate, and stop/resume. Check streaming,
|
|
tool arguments, authentication, and reasoning/token-limit compatibility. A plain chat reply or
|
|
<code>tht pi test</code> alone is not sufficient.</li>
|
|
<li><strong>Administration / LiteLLM:</strong> select the same model and generate descriptions for a small,
|
|
non-sensitive test table. Check the structured result is accepted and the generation completes.
|
|
Review the output quality before using it on real metadata. This action writes test metadata
|
|
and may incur provider charges: use an authorized test database and approved data.</li>
|
|
</ol>
|
|
<p>There is no automatic certification flag or startup model probe. The operator owns this verification;
|
|
do not infer compatibility from the model label or from success in just one path. Both adapters point
|
|
to one catalog identity; Pi does not need to route through a new LiteLLM proxy. Models using only
|
|
<code>pi_auth</code> cannot serve the current LiteLLM path and are excluded from shared selection.</p>
|
|
<h2 id="session-adapters">Session adapters<a class="headerlink" href="#session-adapters" title="Permanent link">¶</a></h2>
|
|
<p>Use <code>pi_builtin</code> for a model whose technical definition ships with Pi. This does not require
|
|
<code>pi_auth</code>: a shared bundle credential lets native Pi and LiteLLM use the same provider identity:</p>
|
|
<pre class="highlight"><code class="language-yaml">deepseek:
|
|
authentication:
|
|
mode: secret_env
|
|
apiKeyEnv: DEEPSEEK_API_KEY
|
|
session:
|
|
mode: pi_builtin
|
|
metadataGeneration:
|
|
litellmProvider: deepseek
|
|
models:
|
|
deepseek-v4-pro:
|
|
session: {}
|
|
metadataGeneration: {}
|
|
deepseek-v4-flash:
|
|
session: {}
|
|
metadataGeneration: {}</code></pre>
|
|
<p>Use <code>openai_compatible</code> for an explicit compatible endpoint. Each eligible session model must then
|
|
declare the technical limits Pi needs. <code>upstreamModel</code> is optional and is used only when the
|
|
endpoint expects a model name different from the catalog key.</p>
|
|
<p>Provider integrations remain declarative. Do not register providers from
|
|
<code>harness/.pi/extensions/</code>; those extensions implement the workflow and human gates only.</p>
|
|
<h3 id="qwen-36-sessions-and-thinking-controls">Qwen 3.6 sessions and thinking controls<a class="headerlink" href="#qwen-36-sessions-and-thinking-controls" title="Permanent link">¶</a></h3>
|
|
<p>For Qwen served through a vLLM-compatible chat template, declare the following inside the
|
|
model's <code>session</code> block, alongside its context and output limits:</p>
|
|
<pre class="highlight"><code class="language-yaml">reasoning: true
|
|
compatibility:
|
|
supportsDeveloperRole: false
|
|
supportsReasoningEffort: false
|
|
supportsStore: false
|
|
maxTokensField: max_tokens
|
|
thinkingFormat: qwen-chat-template</code></pre>
|
|
<p>This makes Pi send <code>chat_template_kwargs.enable_thinking</code> from the selected thinking level,
|
|
with <code>preserve_thinking: true</code>. Choose <strong>off</strong> to explicitly disable thinking. The alternative
|
|
<code>thinkingFormat: qwen</code> is for endpoints expecting top-level <code>enable_thinking</code>. Both formats
|
|
require <code>reasoning: true</code>; declaring <code>reasoning: false</code> does not tell the server to disable
|
|
thinking. Omit <code>thinkingFormat</code> to preserve Pi's default behavior for other providers.</p>
|
|
<p>Regenerate projections with the updated host CLI and recreate the local core container after
|
|
rebuilding it. Do not add these fields directly to generated Pi files. These controls do not
|
|
force tool calls or certify the workflow; perform the operator verification above.</p>
|
|
<pre class="highlight"><code class="language-sh">tht --installation /absolute/path/thothii-installation.yaml installation generate</code></pre>
|
|
<p>Use the model identifier exposed by your endpoint, such as <code>qwen3.6-35b-a3b</code>, and limits
|
|
supported by that deployment. The thinking format configures the Pi session adapter;
|
|
metadata generation continues to use its separate LiteLLM settings.</p>
|
|
<p>If a session displays text such as <code>{"type":"bash","command":"tht session show … --json"}</code>
|
|
and never opens a review widget, that text is not an executed tool call. A verified cause
|
|
was the Evidence JSON extension being loaded into interactive sessions and forcing
|
|
<code>response_format: {type: "json_object"}</code>. Upgrade to the core image containing the fix:
|
|
the extension belongs in <code>.pi/evidence-extensions/</code> and is loaded explicitly only by
|
|
Evidence authoring. It must not also remain in the automatically loaded <code>.pi/extensions/</code>
|
|
directory. Regenerating model configuration alone does not remove an extension from an old image.</p>
|
|
<p>After upgrading, reload the browser and resume the session. Verify that Pi executes
|
|
<code>tht session show</code> and opens a review widget. This fix does not require changing the Qwen
|
|
server, forcing every turn to call a tool, or teaching the model to print tool-call JSON.</p>
|
|
<h2 id="authentication">Authentication<a class="headerlink" href="#authentication" title="Permanent link">¶</a></h2>
|
|
<p>Every provider chooses one explicit mode:</p>
|
|
<ul>
|
|
<li><code>secret_env</code> names an approved key in the protected ThothII secret bundle through <code>apiKeyEnv</code>;</li>
|
|
<li><code>pi_auth</code> uses Pi's protected authentication projection and is valid only for session-only
|
|
<code>pi_builtin</code> providers;</li>
|
|
<li><code>none</code> is valid only with an explicit keyless endpoint.</li>
|
|
</ul>
|
|
<p>Secret values never belong in installation YAML, generated files, logs, CLI arguments, or browser
|
|
requests. The YAML contains only an environment-variable name or an authentication mode. Pi's
|
|
protected credential file remains selected by the installation authentication configuration.</p>
|
|
<p>For catalog providers using <code>secret_env</code>, the bundle is authoritative in both Core and Admin.
|
|
ThothII removes only the selected provider's old auth entry from the temporary Pi session snapshot;
|
|
the operator's original Pi auth store and other providers are unchanged. Provider smoke checks use
|
|
the same precedence, and model enumeration receives the catalog-declared bundle keys. A missing
|
|
declared key is an error, not permission to fall back to Pi auth or the legacy generic key file.
|
|
After rotating a bundle key, apply the normal installation lifecycle so processes reload it.</p>
|
|
<p>The PSD descriptor now declares only <code>deepseek/deepseek-v4-pro</code> and <code>deepseek/deepseek-v4-flash</code>
|
|
for both uses; it no longer duplicates them under <code>deepseek-metadata</code>. Historical records are not
|
|
rewritten. A saved obsolete identity must be explicitly reselected from the current catalog;
|
|
it is not silently remapped to another model or account.</p>
|
|
<h2 id="generated-runtime-projections">Generated runtime projections<a class="headerlink" href="#generated-runtime-projections" title="Permanent link">¶</a></h2>
|
|
<p>Before Compose starts, <code>tht</code> validates the installation and atomically writes deterministic files
|
|
under <code>deploy/<installation-id>/generated/</code>:</p>
|
|
<pre class="highlight"><code class="language-text">generated/
|
|
├── catalog.json
|
|
├── pi/
|
|
│ ├── models.json
|
|
│ └── settings.json
|
|
└── compose.models.yaml</code></pre>
|
|
<p>The normalized catalog is consumed by the backend. The Pi files and Compose override are boundary
|
|
adapters. They are not configuration sources and are excluded from backup. Restore regenerates
|
|
them from the installation descriptor.</p>
|
|
<p>Run all lifecycle commands from the project root and select the descriptor explicitly when more
|
|
than one installation exists:</p>
|
|
<pre class="highlight"><code class="language-bash">INSTALLATION=/absolute/path/deploy/example/thothii-installation.yaml
|
|
|
|
tht --installation "$INSTALLATION" start
|
|
tht --installation "$INSTALLATION" doctor</code></pre>
|
|
<p>After editing <code>modelCatalog</code> or provider credentials, apply the complete runtime projection with
|
|
the normal installation lifecycle, then run the Pi checks:</p>
|
|
<pre class="highlight"><code class="language-bash">tht --installation "$INSTALLATION" start
|
|
tht --installation "$INSTALLATION" pi doctor
|
|
tht --installation "$INSTALLATION" pi test</code></pre>
|
|
<p><code>tht pi restart</code>, <code>tht pi update</code>, and <code>tht pi rollback</code> refuse to run while generated model
|
|
projections differ from <code>modelCatalog</code>: those commands recreate only <code>core</code>, so they must never
|
|
partially apply an embedding change. <code>tht pi update</code> changes the Pi version; it is not the
|
|
configuration command. There is no <code>tht pi configure</code> and no separate apply command.</p>
|
|
<h2 id="migrating-a-legacy-installation">Migrating a legacy installation<a class="headerlink" href="#migrating-a-legacy-installation" title="Permanent link">¶</a></h2>
|
|
<p>For schema-v2 descriptors with the former <code>defaults.session</code> and <code>defaults.metadataGeneration</code>,
|
|
replace both with <code>defaults.interaction</code>. Equal legacy values are accepted and normalized in memory;
|
|
the loader never rewrites the descriptor. Different values fail with <code>migration_required</code>: explicitly
|
|
choose a model supporting both uses, remove both old fields, and set the single new field. Do not mix
|
|
new and legacy fields. A Core-only legacy session default can be normalized when Admin AI is absent.</p>
|
|
<p>The generated runtime catalog now uses schema version <strong>2</strong> and only <code>defaultInteraction</code>. Regenerate
|
|
and apply all runtime projections with the matching host/backend release using the normal installation
|
|
lifecycle; do not deploy only the backend against an old generated catalog or hand-edit generated JSON.</p>
|
|
<p>The migrator reads the former installation <code>metadataGeneration</code> block and the two former Pi JSON
|
|
files, but never modifies them. Supply the facts that cannot be inferred safely and write a separate
|
|
candidate:</p>
|
|
<pre class="highlight"><code class="language-bash">tht --installation /absolute/path/legacy/thothii-installation.yaml installation migrate \
|
|
--output /absolute/path/thothii-installation.v2.yaml \
|
|
--session-default zai/glm-5.3 \
|
|
--embedding-id ollama/qwen3-embedding:0.6b \
|
|
--embedding-dimensions 1024</code></pre>
|
|
<p>Review the candidate, move the legacy source files out of the installation only after approval,
|
|
then select the v2 descriptor. Ambiguous aliases, endpoint conflicts, or missing authentication
|
|
facts produce field-level errors; the migrator does not guess.
|
|
The legacy CLI flag <code>--session-default</code> now supplies the unified interaction default in the candidate;
|
|
if it conflicts with the legacy metadata default, align that choice explicitly before retrying.</p>
|
|
<h2 id="troubleshooting">Troubleshooting<a class="headerlink" href="#troubleshooting" title="Permanent link">¶</a></h2>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>Symptom</th>
|
|
<th>Meaning</th>
|
|
<th>Action</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
<tr>
|
|
<td><code>migration_required</code></td>
|
|
<td>A retired model source or installation schema is still present</td>
|
|
<td>Run the installation migrator and review its candidate</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Invalid interaction default</td>
|
|
<td>The canonical ID does not support all configured uses</td>
|
|
<td>Correct <code>defaults.interaction</code> or the intended adapter blocks</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Generated projection drift</td>
|
|
<td>Runtime files differ from the descriptor-derived bytes</td>
|
|
<td>Run <code>tht start</code> or <code>tht pi restart --yes --drain</code></td>
|
|
</tr>
|
|
<tr>
|
|
<td><code>model_unavailable</code> on create/resume</td>
|
|
<td>The selected global model is no longer eligible</td>
|
|
<td>Explicitly choose an eligible model; no fallback is applied</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Provider smoke failure</td>
|
|
<td>Credentials, endpoint, or provider availability is invalid</td>
|
|
<td>Correct the protected credential or catalog endpoint, restart, then run <code>tht pi test</code></td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<br>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div class="row wm-article-nav-buttons" role="navigation" aria-label="navigation">
|
|
|
|
<div class="wm-article-nav pull-right">
|
|
<a href="../../operations/workspaces/" class="btn btn-xs btn-default pull-right">
|
|
Next
|
|
<i class="fa fa-chevron-right" aria-hidden="true"></i>
|
|
</a>
|
|
<a href="../../operations/workspaces/" class="btn btn-xs btn-link">
|
|
Workspaces
|
|
</a>
|
|
</div>
|
|
|
|
<div class="wm-article-nav">
|
|
<a href="../../install/authentik/" class="btn btn-xs btn-default pull-left">
|
|
<i class="fa fa-chevron-left" aria-hidden="true"></i>
|
|
Previous</a><a href="../../install/authentik/" class="btn btn-xs btn-link">
|
|
Authentik
|
|
</a>
|
|
</div>
|
|
|
|
</div>
|
|
|
|
<br>
|
|
</div>
|
|
|
|
<footer class="container-fluid wm-page-content">
|
|
<p>Documentation built with <a href="https://www.mkdocs.org/">MkDocs</a> using <a href="https://github.com/gristlabs/mkdocs-windmill">Windmill</a> theme by Grist Labs.</p>
|
|
</footer>
|
|
|
|
</body>
|
|
</html> |