ThothII/docs/superpowers/specs/2026-08-11-p1-1-workspace-directory-registry-design.md

# P1.1 Workspace-Directory Git Registry Design

**Status:** Proposed for owner approval
**Date:** 2026-08-11
**Supersedes:** The repository-layout and descriptor-publication portions of P1; P1's Evidence configuration, immutable revision, local-secret, rendering, and verification contracts remain in force.
**Deferred:** Any edits to P2–P6. Their impact will be planned only after P1.1 manual acceptance.

## 1. Goal

Make one shared Git repository read naturally as a catalog of self-contained workspaces. Each
workspace owns one directory containing its technical descriptor and, when Evidence is embedded,
its curated Evidence tree. A root catalog establishes the canonical workspace IDs, names, and
descriptions. ThothII may create a missing descriptor once as a bootstrap convenience, but after
that the descriptor is curator-owned and may be changed or removed only through ordinary Git
review and push.

P1.1 is a correction to P1, not the preprocessing project. It performs no DWH introspection,
Evidence acquisition, materialization, embeddings, Qdrant writes, ACTIVE publication, FK curation,
or retention execution.

## 2. Chosen repository contract

```text
thoth-workspaces.git/
├── thoth-workspaces.yaml
├── psd/
│   ├── workspace.yaml
│   └── evidence/
│       └── ... curated Evidence files ...
├── external-research/
│   └── workspace.yaml            # Evidence may instead be HTTP or S3
└── workspace-docs/
    ├── psd/
    │   ├── contract.env.example
    │   └── README.md
    └── external-research/
        ├── contract.env.example
        └── README.md
```

The fixed paths are:

```text
catalog                         thoth-workspaces.yaml
workspace descriptor            <id>/workspace.yaml
embedded filesystem Evidence    <id>/evidence
future curated FK annotations   <id>/schema/annotations.yaml  # P5, not P1.1
public generated docs           workspace-docs/<id>/{contract.env.example,README.md}
```

Internal installation snapshots deliberately remain flat:

```text
<data>/workspace-registry/snapshots/<commit>/<id>.yaml
```

This avoids changing ThtRunner's trusted-snapshot contract, historical session pins, runtime lease
files, or resume behavior. Repository layout and internal snapshot layout are separate contracts.

## 3. Root catalog

The curator owns `thoth-workspaces.yaml`. The API never creates, edits, deletes, stages, or cleans
it. Its strict initial shape is:

```yaml
schema_version: 1
workspaces:
  - id: psd
    name: Policlinico San Donato
    description: Data warehouse clinico del Policlinico San Donato
```

Rules:

- IDs use the existing `^[a-z][a-z0-9-]{2,62}$` contract, are unique, and cannot equal the reserved API directory `workspace-docs`.
- `name` is required; `description` is optional. Existing trim/nonblank, Unicode, and safe-error behavior used by descriptor metadata are reused; P1.1 introduces no new string-length limit.
- Unknown keys, duplicate YAML keys, aliases/tags, multiple documents, malformed encodings, and
  duplicate IDs are rejected.
- Catalog order is the workspace display order.
- A catalog entry may temporarily have no descriptor. This is the only bootstrap state and is
  represented publicly as `configuration_required`; it is not session-activatable.
- `workspace-docs` is a reserved top-level API directory and cannot be a workspace ID.
- The dedicated registry accepts only catalog-listed workspace directories plus the reserved
  generated-docs directory and explicitly allowed root control files. An unlisted workspace
  directory/descriptor, or a descriptor whose `workspace.id`, `workspace.name`, or optional
  `workspace.description` differs from the catalog, is invalid. Activation fails atomically and
  retains the prior valid snapshot.
- A present but empty or malformed descriptor is not "empty" for bootstrap. It is curator content
  and is rejected; the API never replaces it.

The descriptor retains `id`, `name`, and `description` so exports and immutable runtime snapshots
remain self-contained. The catalog is authoritative, and exact equality prevents two names for one
workspace.

## 4. Ownership and write policy

There are three writers with disjoint authority:

| Path | Owner | ThothII API behavior |
| --- | --- | --- |
| `thoth-workspaces.yaml` | curator | read and validate only |
| `<id>/workspace.yaml` | curator after bootstrap | create only if absent at the exact base commit; never overwrite or delete |
| `<id>/evidence/**` | curator | read Git objects only; never write, stage, clean, or materialize in P1.1 |
| `workspaces/<id>/schema/**` | curator/future P5 | untouched by P1.1 |
| `workspace-docs/<id>/*` | API | deterministic generated files only |

"Absent" means no Git object exists at `<id>/workspace.yaml` in the exact pulled base
commit. A zero-byte file, comments-only YAML, symlink, submodule, tree, or malformed document counts
as present and is never overwritten.

A browser/API bootstrap succeeds only when:

1. the catalog entry already exists at the request's exact `baseCommit`;
2. the descriptor path is absent at that commit and remains absent after the pull;
3. request metadata exactly matches the catalog;
4. filesystem Evidence, when selected, already exists as a Git tree at
   `<id>/evidence` in that same base commit;
5. the complete descriptor passes schema-v3 and operational publication checks.

The API then commits only the new descriptor and generated docs. Update and delete requests against
an existing descriptor return a stable `workspace_curator_owned` conflict response and do not
change any Git object. Curator deletion means removing the descriptor or catalog/directory through
Git. A retained session snapshot remains available under the existing retention rules.

The managed checkout must no longer run a directory-wide clean under `workspaces/`. Failure cleanup
is confined to the exact descriptor/docs files written by the failed API operation and proves their
pre-operation identity before removal.

## 5. Synchronization and generated docs

Startup/bootstrap may pull, validate, and activate curator bytes but never pushes as a side effect
of a status/read request. This deliberately means committed `workspace-docs` can remain stale until
an explicit synchronization action; immutable local snapshots and exports always derive fresh docs
from the validated active descriptor and never consume stale Git docs. The explicit
`/workspace-registry/pull` operator action remains the synchronization boundary:

1. pull the curator commit;
2. validate catalog, descriptors, namespace ownership, Evidence roots, and semantic-index ownership;
3. compute deterministic `workspace-docs/<id>` bytes;
4. if docs differ, create one docs-only follow-up commit without touching catalog, descriptors, or
   workspace content;
5. validate and activate the resulting exact commit.

Every API write transaction records a bounded per-path journal before mutation: prior Git object type,
mode, blob identity and bytes for tracked generated docs, or explicit absence, plus the intended
post-write identity. If bootstrap/docs push races, is rejected, or fails, the prior valid active
snapshot remains active; overwritten/deleted generated docs are restored byte-for-byte to their
prior objects, newly created absent-before files are removed, and curator paths are never cleaned.
A later explicit pull retries from a fresh remote head. The API removes stale generated docs only for workspaces that the curator has
removed from the catalog or returned to `configuration_required`.

A docs-only follow-up commit is an authoritative workspace revision, as every active workspace is
pinned to the complete Git commit rather than only to its descriptor blob. Automated acceptance
must show that descriptor and Evidence blob identities are unchanged across that docs-only commit.

## 6. API and browser behavior

`GET /workspaces` is catalog-driven and returns every catalog entry in catalog order with:

- canonical ID, display name, and description from the catalog;
- `configurationState: ready | configuration_required`;
- `file: <id>/workspace.yaml`;
- an immutable revision only for `ready` entries.

`language` is deliberately not a summary field because an unconfigured catalog slot has no
descriptor language. It remains available from the descriptor detail for `ready` workspaces and is
selected in the bootstrap draft before creation.

Descriptor-only routes (`GET /workspaces/:id`, diagnostics, export, session admission) reject a
`configuration_required` entry as `workspace_not_activatable`.

`POST /workspaces/validate` remains a context-free schema check. It does not claim catalog
agreement or publication eligibility. `POST /workspaces/publish` becomes bootstrap-create only.
Legacy update/delete payloads are recognized and rejected as `workspace_curator_owned` rather than
silently reinterpreted.

The browser:

- lists catalog slots, including those requiring configuration;
- offers an editable, browser-local bootstrap draft only for `configuration_required` entries;
- locks catalog-owned ID/name/description in that form;
- requires explicit validation and confirmation before the one create;
- turns the workspace read-only immediately after creation;
- keeps Pull/Sync, Validate, installation Test, Export, and safe Evidence summary for existing
  workspaces;
- removes update, delete, duplicate, field-conflict merge, and publish-existing controls;
- versions or purges old update/deletion drafts so stale localStorage cannot restore write access;
- treats imported bundles as bootstrap drafts only when they match an existing unconfigured
  catalog slot.

Existing descriptors are edited in the curator clone and become active after commit, push, and
installation pull.

## 7. Evidence and external sources

For filesystem Evidence, schema v3 now requires exactly:

```yaml
evidence:
  source:
    type: filesystem
    uri: <workspace-id>/evidence
```

The lexical path invariant and same-commit Git-tree check remain P1.1 responsibilities. Recursive
materialization, nested symlink rejection, byte acquisition, preprocessing, and indexing remain P6
or later.

HTTP and S3 descriptor shapes, local `*_FILE` bindings, secret handling, timeout/limit policy,
runtime rendering, and `tht config check` remain as delivered by P1. Those workspaces need no local
`evidence/` directory. Evidence may also remain absent for compatibility.

## 8. Rejected alternatives

1. **Keep P1's three top-level source trees.** Rejected because it does not make a workspace a
   self-contained Git unit and does not match the desired curator model.
2. **Place descriptors and Evidence together but let both API and curator update descriptors.**
   Rejected because it creates two authorities, restores field-level conflict merging, and risks
   overwriting reviewed Git content.
3. **Chosen: API bootstrap once, then curator ownership.** This preserves a convenient initial
   form while making ordinary Git review the single authority for all subsequent descriptor and
   content changes.

## 9. Compatibility and migration

P1.1 is a repository-contract cutover, not a dual-format reader. New code rejects the old flat
layout and a repository without `thoth-workspaces.yaml`. Existing repositories are migrated in one
curator-reviewed commit:

```text
workspaces/<id>.yaml                    -> <id>/workspace.yaml
workspace-content/<id>/evidence/**      -> <id>/evidence/**
(create thoth-workspaces.yaml from reviewed descriptor metadata)
```

No automatic in-product migrator rewrites a remote. The installation upgrades only after the
migration commit is available. Historical immutable installation snapshots and retained session
pins keep their current internal shape.

P2–P6 currently assume P1's old source paths in several places. P1.1 records that impact but does
not edit those plans. After P1.1 automated and manual acceptance, a separate owner-approved plan
will revise P2–P6.

## 10. Verification boundary

P1.1 must have independent automated and manual evidence. Accepted retained P1 artifacts remain immutable historical evidence; the old P1 process commands are not release gates for the superseding repository contract. The automated run starts from a clean
local Git remote and proves catalog authority, missing-descriptor bootstrap, curator modification,
API non-overwrite, nested Evidence identity, docs-only reconciliation, immutable snapshots,
runtime render determinism, `tht config check`, negative cases, secret absence, and exact cleanup.
It does not invoke preprocessing or Qdrant/Ollama/DWH services.

The manual environment is new and independent. The reviewer personally performs the bootstrap,
refusal, curator-edit, pull/sync, UI read-only, Git-object, export, render, config-check, secret-scan,
and cleanup checks. Project state remains `P1.1 manual acceptance: PENDING` until the reviewer
records approval.