Files

T

User bc1b5e79ca docs: update checkout path to Thoth

2026-08-22 22:50:36 +02:00

17 KiB

Raw Blame History

Workspace PostgreSQL Diagnostic Alignment Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task.

Goal: Make the psd-clinical workspace connection diagnostic match the PostgreSQL transport actually used by sessions, then prove that the isolated /datamart-builder-test/ installation can validate the workspace and create a session without cutting over the production route.

Architecture: Keep the workspace descriptor, credential store, DWH endpoint, portal authentication, and deployment topology unchanged. Correct the concrete Node PostgreSQL probe so TLS is enabled only by explicit TLS bindings and the declared schema is checked with parameterized privilege SQL. Make the already-selected trusted upstream authentication mode diagnostically ready only when its protected session root is valid, so the aggregate workspace contract reflects the authentication path that already admitted the request.

Tech Stack: TypeScript, Node 24, pg, Fastify, Vitest, Docker, Docker Compose, Django/Nginx test route.

Global Constraints

Work only in /home/chirone/Thoth for product code; preserve all unrelated dirty-worktree changes.
Do not change the psd-clinical workspace descriptor, DWH password, vault, Qdrant, embedding model, Pi provider/model, Django, Nginx, DNS, or external balancer.
Do not print credentials, decrypted secret files, database exception text, cookies, or session content.
Rebuild and recreate only the isolated thothii-test core service.
Do not touch the current /datamart-builder/ route and do not perform the cutover.
Use test-first development: capture the intended focused RED before modifying production code.
Make a commit only from the exact Task 1–3 source/test paths; never stage pre-existing changes or the temporary deployment overlay.

Task 1: Specify the PostgreSQL diagnostic transport contract

Files:

Modify: backend/test/workspaces-diagnostics.test.ts
Test: backend/test/workspaces-diagnostics.test.ts
Add a small injected PostgreSQL wire-client constructor to the test setup through the planned ConcreteDiagnosticAdapterDependencies.createPostgresClient seam. The fake must expose connect, query, and end; it must retain no password after the assertion.

Add concrete DWH direct diagnostics disable TLS when no TLS binding is declared:

const createPostgresClient = vi.fn(() => ({ connect, query, end }));
const adapter = createConcreteDiagnosticAdapters({ createPostgresClient });
await adapter.probeConnector({
  role: "dwh",
  transport: "postgres_direct",
  host: "127.0.0.1",
  port: 5432,
  user: "reader",
  credentialFile: passwordFile,
  resource: { database: "warehouse", schema: "datawarehouse" },
  timeoutMs: 1_000,
  signal: new AbortController().signal,
});
expect(createPostgresClient).toHaveBeenCalledWith(expect.objectContaining({ ssl: false }));

The fake query result is { database: "warehouse", schema: "datawarehouse" }.

Add concrete DWH direct diagnostics use strict TLS only for explicit TLS bindings. Create a temporary CA file, pass both tlsCaFile and tlsServername, and assert:

expect(createPostgresClient).toHaveBeenCalledWith(expect.objectContaining({
  ssl: {
    ca: "test-ca",
    servername: "dwh.example.test",
    rejectUnauthorized: true,
  },
}));

Strengthen concrete DWH direct diagnostics authenticate, verify resource identity, and close by retaining the fake query function and asserting both the SQL and values:
```
expect(query).toHaveBeenCalledWith(
  expect.stringContaining("pg_catalog.has_schema_privilege"),
  ["datawarehouse"],
);
```
Also assert the SQL contains $1 and does not contain a quoted/interpolated datawarehouse identifier.
Add concrete DWH direct diagnostics fail closed when the declared schema is inaccessible. Return { database: "warehouse", schema: null }, expect the fixed internal error direct probe failed, and assert end was called once. Do not assert or expose a raw PostgreSQL error.
Run the focused RED:
```
cd backend
npx vitest run test/workspaces-diagnostics.test.ts -t 'disable TLS|strict TLS|authenticate, verify resource identity|declared schema is inaccessible'
```
Expected: compilation/test failure because createPostgresClient is not yet accepted, followed by behavioral failures if the seam is introduced without the production correction.

Task 2: Align the concrete PostgreSQL diagnostic with runtime semantics

Files:

Modify: backend/src/workspaces/diagnostics.ts
Test: backend/test/workspaces-diagnostics.test.ts

Import ClientConfig from pg as a type and define the narrow injected client contract:

export interface PostgreSqlDiagnosticWireClient {
  connect(): Promise<void>;
  query(sql: string, values: readonly unknown[]): Promise<{ rows: Array<Record<string, unknown>> }>;
  end(): Promise<void>;
}

Add this optional dependency:

createPostgresClient?: (config: ClientConfig) => PostgreSqlDiagnosticWireClient;

In createConcreteDiagnosticAdapters, select the injected constructor or the real pg.Client:

const createPostgresClient = dependencies.createPostgresClient
  ?? ((config: ClientConfig): PostgreSqlDiagnosticWireClient => new Client(config));

Build SSL options from explicit bindings only:

const tlsConfigured = request.tlsCaFile !== undefined || request.tlsServername !== undefined;
const ssl: ClientConfig["ssl"] = tlsConfigured
  ? {
      ...(request.tlsCaFile ? { ca: await readFile(request.tlsCaFile, "utf8") } : {}),
      ...(request.tlsServername ? { servername: request.tlsServername } : {}),
      rejectUnauthorized: true,
    }
  : false;

Pass ssl to createPostgresClient. Do not infer TLS from hostnames and do not add sslmode fallbacks.

Replace the current_schema() comparison with a parameterized database/schema-access query:

SELECT
  current_database() AS database,
  CASE
    WHEN pg_catalog.has_schema_privilege(
      current_user,
      (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = $1),
      'USAGE'
    )
    THEN $1
    ELSE NULL
  END AS schema

Call it with [schema]. Continue requiring both returned database === database and schema === schema. This validates the declared namespace without changing search_path and without interpolating an identifier.

Keep tlsVerified: true on a successful direct probe. In this result type the boolean means that the declared transport-security policy was satisfied: strict verification when TLS was configured, or an explicitly unconfigured/plain connection when it was not.

Run the focused GREEN:

cd backend
npx vitest run test/workspaces-diagnostics.test.ts -t 'disable TLS|strict TLS|authenticate, verify resource identity|declared schema is inaccessible'

Expected: all selected tests pass.

Task 3: Make trusted upstream authentication diagnostic-ready

Files:

Modify: backend/test/auth-diagnostics.test.ts
Modify: backend/src/auth/diagnostics.ts
Test: backend/test/auth-diagnostics.test.ts
Test: backend/test/routes-workspaces.test.ts

Add reports upstream authentication ready when its protected session root is valid:

const report = await createAuthDiagnoser({
  authMode: "upstream",
  authStateRoot: "/safe/auth-state",
  sessionRootValidator: async () => undefined,
}).inspect({ live: true });
expect(report).toEqual({
  ready: true,
  mode: "upstream",
  checks: [{ level: "info", code: "auth_ready", message: "Authentication is ready." }],
});

Add the complementary test upstream authentication still fails when its protected session root is invalid. Make the validator throw a synthetic sentinel and assert ready: false, exactly one auth_session_store_invalid check, and no sentinel in serialized output.
Run the focused RED:
```
cd backend
npx vitest run test/auth-diagnostics.test.ts -t 'upstream authentication'
```
Expected: the valid-root test fails because upstream mode is currently always labelled deprecated and uncertifiable.

Replace the unconditional upstream failure in createAuthDiagnoser with the same ordered-check outcome used by none and mock:

if (deps.authMode === "upstream") {
  const result = ordered(checks);
  return result.length === 0
    ? {
        ready: true,
        mode: "upstream",
        checks: [{ level: "info", code: "auth_ready", message: "Authentication is ready." }],
      }
    : { ready: false, mode: "upstream", checks: result };
}

This does not accept browser-supplied identity headers or alter request authentication. It only makes the diagnostic match the explicitly selected AUTH_MODE=upstream; Nginx remains responsible for clearing client identity headers and adding the normalized portal principal.

Run the focused GREEN and route regression:
```
cd backend
npx vitest run test/auth-diagnostics.test.ts -t 'upstream authentication'
npx vitest run test/routes-workspaces.test.ts -t 'runs diagnostics for a schema v3 workspace'
```
Expected: all selected tests pass and the aggregate workspace contract can be activatable: true only when connector and authentication diagnostics are both ready.

Task 4: Run backend verification and commit only the correction

Files:

Verify: backend/src/workspaces/diagnostics.ts
Verify: backend/src/auth/diagnostics.ts
Verify: backend/test/workspaces-diagnostics.test.ts
Verify: backend/test/auth-diagnostics.test.ts

Run the focused workspace/auth/session regression set:

cd backend
npx vitest run \
  test/workspaces-diagnostics.test.ts \
  test/routes-workspaces.test.ts \
  test/auth-diagnostics.test.ts \
  test/app-auth-mode.test.ts \
  test/routes-sessions.test.ts

Run the mandatory TypeScript gate:
```
cd backend
npx tsc --noEmit -p .
```
Run the full backend suite because both diagnostics are production admission dependencies:
```
cd backend
npx vitest run
```

Inspect only the scoped diff and whitespace:

git diff --check
git diff -- \
  backend/src/workspaces/diagnostics.ts \
  backend/src/auth/diagnostics.ts \
  backend/test/workspaces-diagnostics.test.ts \
  backend/test/auth-diagnostics.test.ts
git status --short

Commit exactly those four files, leaving every unrelated modification unstaged:

git add \
  backend/src/workspaces/diagnostics.ts \
  backend/src/auth/diagnostics.ts \
  backend/test/workspaces-diagnostics.test.ts \
  backend/test/auth-diagnostics.test.ts
git commit -m "fix(workspaces): align postgres connection diagnostics"

Task 5: Rebuild and recreate only the isolated test core

Files:

Read only: docker/core.Dockerfile
Read only: deploy/compose.datamart-builder-test.yaml
Read only: /tmp/thothii-test-server.env

Confirm the active image/service names and that the current application remains separate:

docker compose -p thothii-test \
  --env-file /tmp/thothii-test-server.env \
  -f /srv/thothii/source/ThothII/compose.yaml \
  -f /srv/thothii/source/ThothII/deploy/compose.server.yaml \
  -f /srv/thothii/source/ThothII/deploy/compose.git-ssh.yaml \
  -f /srv/thothii/operator/project-a-private.yaml \
  -f /home/chirone/Thoth/deploy/compose.datamart-builder-test.yaml \
  ps

Build only the test core image from the corrected source:

docker build -f docker/core.Dockerfile -t thothii-project-a-core:test .

Recreate only thothii-test-core-1; do not recreate frontend, Qdrant, embedding, portal, or current ThothII:

docker compose -p thothii-test \
  --env-file /tmp/thothii-test-server.env \
  -f /srv/thothii/source/ThothII/compose.yaml \
  -f /srv/thothii/source/ThothII/deploy/compose.server.yaml \
  -f /srv/thothii/source/ThothII/deploy/compose.git-ssh.yaml \
  -f /srv/thothii/operator/project-a-private.yaml \
  -f /home/chirone/Thoth/deploy/compose.datamart-builder-test.yaml \
  up -d --no-deps --force-recreate core

Wait for health without dumping environment or logs containing payloads:

docker inspect --format '{{.State.Health.Status}}' thothii-test-core-1
docker compose -p thothii-test \
  --env-file /tmp/thothii-test-server.env \
  -f /srv/thothii/source/ThothII/compose.yaml \
  -f /srv/thothii/source/ThothII/deploy/compose.server.yaml \
  -f /srv/thothii/source/ThothII/deploy/compose.git-ssh.yaml \
  -f /srv/thothii/operator/project-a-private.yaml \
  -f /home/chirone/Thoth/deploy/compose.datamart-builder-test.yaml \
  ps

Expected: core becomes healthy; all other test services remain running with their prior container identities.

Task 6: Verify the live workspace and one disposable session

Files:

No source modifications.
Runtime scope: thothii-test-core-1 and https://aritmolab.policlinicosandonato.it/datamart-builder-test/ only.

Probe the backend through its loopback interface with a synthetic normalized administrator principal. Keep the response in shell memory or a mode-0600 temporary file; never print credentials:

docker exec thothii-test-core-1 node -e '
  const headers = {
    "x-thoth-principal-issuer": "portal",
    "x-thoth-principal-subject": "diagnostic-workspace-postgres",
    "x-thoth-principal-display-name": "Diagnostic",
    "x-thoth-is-admin": "true",
    "content-type": "application/json",
  };
  const response = await fetch("http://127.0.0.1:8787/workspaces/psd-clinical/test", { method: "POST", headers, body: "{}" });
  const body = await response.json();
  if (response.status !== 200 || body.activatable !== true || body.authentication?.ready !== true
    || body.diagnostics?.some((item) => item.level === "error")) process.exit(1);
'

Confirm the saved model remains available without printing API keys:

docker exec thothii-test-core-1 node -e '
  const headers = {
    "x-thoth-principal-issuer": "portal",
    "x-thoth-principal-subject": "diagnostic-workspace-postgres",
    "x-thoth-principal-display-name": "Diagnostic",
    "x-thoth-is-admin": "true",
  };
  const response = await fetch("http://127.0.0.1:8787/models", { headers });
  const models = await response.json();
  if (response.status !== 200 || !models.some((item) => item.provider === "deepseek" && item.id === "deepseek-v4-flash")) process.exit(1);
'

Create exactly one disposable session using the saved global workspace/model settings, retain only its returned ID, then delete it immediately:

docker exec thothii-test-core-1 node -e '
  const headers = {
    "x-thoth-principal-issuer": "portal",
    "x-thoth-principal-subject": "diagnostic-workspace-postgres",
    "x-thoth-principal-display-name": "Diagnostic",
    "x-thoth-is-admin": "true",
    "content-type": "application/json",
  };
  const created = await fetch("http://127.0.0.1:8787/sessions", {
    method: "POST", headers, body: JSON.stringify({ question: "Diagnostic session creation check" }),
  });
  const payload = await created.json();
  if (created.status !== 200 || typeof payload.id !== "string") process.exit(1);
  const { ["content-type"]: _contentType, ...deleteHeaders } = headers;
  const removed = await fetch(`http://127.0.0.1:8787/sessions/${encodeURIComponent(payload.id)}`, {
    method: "DELETE", headers: deleteHeaders,
  });
  if (removed.status !== 204) process.exit(1);
'

Verify the public test page still responds through the existing portal route without changing Nginx or Django:
```
curl -k -sS -o /dev/null -w '%{http_code}\n' \
  https://aritmolab.policlinicosandonato.it/datamart-builder-test/
```
Expected: an authenticated portal request reaches the page; an unauthenticated shell probe may return the existing login redirect and must not be treated as a route regression.

Task 7: Manual browser checkpoint

Report that only the isolated test core was rebuilt and provide this URL:

https://aritmolab.policlinicosandonato.it/datamart-builder-test/
Ask the user to:
1. refresh the page;
2. open workspace management and run Test Workspace Connection for psd-clinical;
3. create a new session and submit a short question.
Stop and wait for the user's manual result. Do not repoint Datamart Builder, remove the -test route, or modify the current ThothII instance.

17 KiB Raw Blame History Unescape Escape

Workspace PostgreSQL Diagnostic Alignment Implementation Plan

Global Constraints

Task 1: Specify the PostgreSQL diagnostic transport contract

Task 2: Align the concrete PostgreSQL diagnostic with runtime semantics

Task 3: Make trusted upstream authentication diagnostic-ready

Task 4: Run backend verification and commit only the correction

Task 5: Rebuild and recreate only the isolated test core

Task 6: Verify the live workspace and one disposable session

Task 7: Manual browser checkpoint

17 KiB

Raw Blame History