ThothII/docs/superpowers/specs/2026-08-11-p1-1-workspace-directory-registry-design.md

P1.1 Workspace-Directory Git Registry Design

Status: Proposed for owner approval Date: 2026-08-11 Supersedes: The repository-layout and descriptor-publication portions of P1; P1's Evidence configuration, immutable revision, local-secret, rendering, and verification contracts remain in force. Deferred: Any edits to P2–P6. Their impact will be planned only after P1.1 manual acceptance.

1. Goal

Make one shared Git repository read naturally as a catalog of self-contained workspaces. Each workspace owns one directory containing its technical descriptor and, when Evidence is embedded, its curated Evidence tree. A root catalog establishes the canonical workspace IDs, names, and descriptions. ThothII may create a missing descriptor once as a bootstrap convenience, but after that the descriptor is curator-owned and may be changed or removed only through ordinary Git review and push.

P1.1 is a correction to P1, not the preprocessing project. It performs no DWH introspection, Evidence acquisition, materialization, embeddings, Qdrant writes, ACTIVE publication, FK curation, or retention execution.

2. Chosen repository contract

thoth-workspaces.git/
├── thoth-workspaces.yaml
├── psd/
│   ├── workspace.yaml
│   └── evidence/
│       └── ... curated Evidence files ...
├── external-research/
│   └── workspace.yaml            # Evidence may instead be HTTP or S3
└── workspace-docs/
    ├── psd/
    │   ├── contract.env.example
    │   └── README.md
    └── external-research/
        ├── contract.env.example
        └── README.md

The fixed paths are:

catalog                         thoth-workspaces.yaml
workspace descriptor            <id>/workspace.yaml
embedded filesystem Evidence    <id>/evidence
future curated FK annotations   <id>/schema/annotations.yaml  # P5, not P1.1
public generated docs           workspace-docs/<id>/{contract.env.example,README.md}

Internal installation snapshots deliberately remain flat:

<data>/workspace-registry/snapshots/<commit>/<id>.yaml

This avoids changing ThtRunner's trusted-snapshot contract, historical session pins, runtime lease files, or resume behavior. Repository layout and internal snapshot layout are separate contracts.

3. Root catalog

The curator owns thoth-workspaces.yaml. The API never creates, edits, deletes, stages, or cleans it. Its strict initial shape is:

schema_version: 1
workspaces:
  - id: psd
    name: Policlinico San Donato
    description: Data warehouse clinico del Policlinico San Donato

Rules:

• IDs use the existing ^[a-z][a-z0-9-]{2,62}$ contract, are unique, and cannot equal the reserved API directory workspace-docs.
• name is required; description is optional. Existing trim/nonblank, Unicode, and safe-error behavior used by descriptor metadata are reused; P1.1 introduces no new string-length limit.
• Unknown keys, duplicate YAML keys, aliases/tags, multiple documents, malformed encodings, and duplicate IDs are rejected.
• Catalog order is the workspace display order.
• A catalog entry may temporarily have no descriptor. This is the only bootstrap state and is represented publicly as configuration_required; it is not session-activatable.
• workspace-docs is a reserved top-level API directory and cannot be a workspace ID.
• The dedicated registry accepts only catalog-listed workspace directories plus the reserved generated-docs directory and explicitly allowed root control files. An unlisted workspace directory/descriptor, or a descriptor whose workspace.id, workspace.name, or optional workspace.description differs from the catalog, is invalid. Activation fails atomically and retains the prior valid snapshot.
• A present but empty or malformed descriptor is not "empty" for bootstrap. It is curator content and is rejected; the API never replaces it.

The descriptor retains id, name, and description so exports and immutable runtime snapshots remain self-contained. The catalog is authoritative, and exact equality prevents two names for one workspace.

4. Ownership and write policy

There are three writers with disjoint authority:

Path	Owner	ThothII API behavior
thoth-workspaces.yaml	curator	read and validate only
<id>/workspace.yaml	curator after bootstrap	create only if absent at the exact base commit; never overwrite or delete
<id>/evidence/**	curator	read Git objects only; never write, stage, clean, or materialize in P1.1
workspaces/<id>/schema/**	curator/future P5	untouched by P1.1
workspace-docs/<id>/*	API	deterministic generated files only

"Absent" means no Git object exists at <id>/workspace.yaml in the exact pulled base commit. A zero-byte file, comments-only YAML, symlink, submodule, tree, or malformed document counts as present and is never overwritten.

A browser/API bootstrap succeeds only when:

1. the catalog entry already exists at the request's exact baseCommit;
2. the descriptor path is absent at that commit and remains absent after the pull;
3. request metadata exactly matches the catalog;
4. filesystem Evidence, when selected, already exists as a Git tree at <id>/evidence in that same base commit;
5. the complete descriptor passes schema-v3 and operational publication checks.

The API then commits only the new descriptor and generated docs. Update and delete requests against an existing descriptor return a stable workspace_curator_owned conflict response and do not change any Git object. Curator deletion means removing the descriptor or catalog/directory through Git. A retained session snapshot remains available under the existing retention rules.

The managed checkout must no longer run a directory-wide clean under workspaces/. Failure cleanup is confined to the exact descriptor/docs files written by the failed API operation and proves their pre-operation identity before removal.

5. Synchronization and generated docs

Startup/bootstrap may pull, validate, and activate curator bytes but never pushes as a side effect of a status/read request. This deliberately means committed workspace-docs can remain stale until an explicit synchronization action; immutable local snapshots and exports always derive fresh docs from the validated active descriptor and never consume stale Git docs. The explicit /workspace-registry/pull operator action remains the synchronization boundary:

1. pull the curator commit;
2. validate catalog, descriptors, namespace ownership, Evidence roots, and semantic-index ownership;
3. compute deterministic workspace-docs/<id> bytes;
4. if docs differ, create one docs-only follow-up commit without touching catalog, descriptors, or workspace content;
5. validate and activate the resulting exact commit.

Every API write transaction records a bounded per-path journal before mutation: prior Git object type, mode, blob identity and bytes for tracked generated docs, or explicit absence, plus the intended post-write identity. If bootstrap/docs push races, is rejected, or fails, the prior valid active snapshot remains active; overwritten/deleted generated docs are restored byte-for-byte to their prior objects, newly created absent-before files are removed, and curator paths are never cleaned. A later explicit pull retries from a fresh remote head. The API removes stale generated docs only for workspaces that the curator has removed from the catalog or returned to configuration_required.

A docs-only follow-up commit is an authoritative workspace revision, as every active workspace is pinned to the complete Git commit rather than only to its descriptor blob. Automated acceptance must show that descriptor and Evidence blob identities are unchanged across that docs-only commit.

6. API and browser behavior

GET /workspaces is catalog-driven and returns every catalog entry in catalog order with:

• canonical ID, display name, and description from the catalog;
• configurationState: ready | configuration_required;
• file: <id>/workspace.yaml;
• an immutable revision only for ready entries.

language is deliberately not a summary field because an unconfigured catalog slot has no descriptor language. It remains available from the descriptor detail for ready workspaces and is selected in the bootstrap draft before creation.

Descriptor-only routes (GET /workspaces/:id, diagnostics, export, session admission) reject a configuration_required entry as workspace_not_activatable.

POST /workspaces/validate remains a context-free schema check. It does not claim catalog agreement or publication eligibility. POST /workspaces/publish becomes bootstrap-create only. Legacy update/delete payloads are recognized and rejected as workspace_curator_owned rather than silently reinterpreted.

The browser:

• lists catalog slots, including those requiring configuration;
• offers an editable, browser-local bootstrap draft only for configuration_required entries;
• locks catalog-owned ID/name/description in that form;
• requires explicit validation and confirmation before the one create;
• turns the workspace read-only immediately after creation;
• keeps Pull/Sync, Validate, installation Test, Export, and safe Evidence summary for existing workspaces;
• removes update, delete, duplicate, field-conflict merge, and publish-existing controls;
• versions or purges old update/deletion drafts so stale localStorage cannot restore write access;
• treats imported bundles as bootstrap drafts only when they match an existing unconfigured catalog slot.

Existing descriptors are edited in the curator clone and become active after commit, push, and installation pull.

7. Evidence and external sources

For filesystem Evidence, schema v3 now requires exactly:

evidence:
  source:
    type: filesystem
    uri: <workspace-id>/evidence

The lexical path invariant and same-commit Git-tree check remain P1.1 responsibilities. Recursive materialization, nested symlink rejection, byte acquisition, preprocessing, and indexing remain P6 or later.

HTTP and S3 descriptor shapes, local *_FILE bindings, secret handling, timeout/limit policy, runtime rendering, and tht config check remain as delivered by P1. Those workspaces need no local evidence/ directory. Evidence may also remain absent for compatibility.

8. Rejected alternatives

1. Keep P1's three top-level source trees. Rejected because it does not make a workspace a self-contained Git unit and does not match the desired curator model.
2. Place descriptors and Evidence together but let both API and curator update descriptors. Rejected because it creates two authorities, restores field-level conflict merging, and risks overwriting reviewed Git content.
3. Chosen: API bootstrap once, then curator ownership. This preserves a convenient initial form while making ordinary Git review the single authority for all subsequent descriptor and content changes.

9. Compatibility and migration

P1.1 is a repository-contract cutover, not a dual-format reader. New code rejects the old flat layout and a repository without thoth-workspaces.yaml. Existing repositories are migrated in one curator-reviewed commit:

workspaces/<id>.yaml                    -> <id>/workspace.yaml
workspace-content/<id>/evidence/**      -> <id>/evidence/**
(create thoth-workspaces.yaml from reviewed descriptor metadata)

No automatic in-product migrator rewrites a remote. The installation upgrades only after the migration commit is available. Historical immutable installation snapshots and retained session pins keep their current internal shape.

P2–P6 currently assume P1's old source paths in several places. P1.1 records that impact but does not edit those plans. After P1.1 automated and manual acceptance, a separate owner-approved plan will revise P2–P6.

10. Verification boundary

P1.1 must have independent automated and manual evidence. Accepted retained P1 artifacts remain immutable historical evidence; the old P1 process commands are not release gates for the superseding repository contract. The automated run starts from a clean local Git remote and proves catalog authority, missing-descriptor bootstrap, curator modification, API non-overwrite, nested Evidence identity, docs-only reconciliation, immutable snapshots, runtime render determinism, tht config check, negative cases, secret absence, and exact cleanup. It does not invoke preprocessing or Qdrant/Ollama/DWH services.

The manual environment is new and independent. The reviewer personally performs the bootstrap, refusal, curator-edit, pull/sync, UI read-only, Git-object, export, render, config-check, secret-scan, and cleanup checks. Project state remains P1.1 manual acceptance: PENDING until the reviewer records approval.