docs: add P1.1 workspace directory plan
This commit is contained in:
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,255 @@
|
||||
# P1.1 Workspace-Directory Git Registry Design
|
||||
|
||||
**Status:** Proposed for owner approval
|
||||
**Date:** 2026-08-11
|
||||
**Supersedes:** The repository-layout and descriptor-publication portions of P1; P1's Evidence configuration, immutable revision, local-secret, rendering, and verification contracts remain in force.
|
||||
**Deferred:** Any edits to P2–P6. Their impact will be planned only after P1.1 manual acceptance.
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Make one shared Git repository read naturally as a catalog of self-contained workspaces. Each
|
||||
workspace owns one directory containing its technical descriptor and, when Evidence is embedded,
|
||||
its curated Evidence tree. A root catalog establishes the canonical workspace IDs, names, and
|
||||
descriptions. ThothII may create a missing descriptor once as a bootstrap convenience, but after
|
||||
that the descriptor is curator-owned and may be changed or removed only through ordinary Git
|
||||
review and push.
|
||||
|
||||
P1.1 is a correction to P1, not the preprocessing project. It performs no DWH introspection,
|
||||
Evidence acquisition, materialization, embeddings, Qdrant writes, ACTIVE publication, FK curation,
|
||||
or retention execution.
|
||||
|
||||
## 2. Chosen repository contract
|
||||
|
||||
```text
|
||||
thoth-workspaces.git/
|
||||
├── thoth-workspaces.yaml
|
||||
├── psd/
|
||||
│ ├── workspace.yaml
|
||||
│ └── evidence/
|
||||
│ └── ... curated Evidence files ...
|
||||
├── external-research/
|
||||
│ └── workspace.yaml # Evidence may instead be HTTP or S3
|
||||
└── workspace-docs/
|
||||
├── psd/
|
||||
│ ├── contract.env.example
|
||||
│ └── README.md
|
||||
└── external-research/
|
||||
├── contract.env.example
|
||||
└── README.md
|
||||
```
|
||||
|
||||
The fixed paths are:
|
||||
|
||||
```text
|
||||
catalog thoth-workspaces.yaml
|
||||
workspace descriptor <id>/workspace.yaml
|
||||
embedded filesystem Evidence <id>/evidence
|
||||
future curated FK annotations <id>/schema/annotations.yaml # P5, not P1.1
|
||||
public generated docs workspace-docs/<id>/{contract.env.example,README.md}
|
||||
```
|
||||
|
||||
Internal installation snapshots deliberately remain flat:
|
||||
|
||||
```text
|
||||
<data>/workspace-registry/snapshots/<commit>/<id>.yaml
|
||||
```
|
||||
|
||||
This avoids changing ThtRunner's trusted-snapshot contract, historical session pins, runtime lease
|
||||
files, or resume behavior. Repository layout and internal snapshot layout are separate contracts.
|
||||
|
||||
## 3. Root catalog
|
||||
|
||||
The curator owns `thoth-workspaces.yaml`. The API never creates, edits, deletes, stages, or cleans
|
||||
it. Its strict initial shape is:
|
||||
|
||||
```yaml
|
||||
schema_version: 1
|
||||
workspaces:
|
||||
- id: psd
|
||||
name: Policlinico San Donato
|
||||
description: Data warehouse clinico del Policlinico San Donato
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- IDs use the existing `^[a-z][a-z0-9-]{2,62}$` contract, are unique, and cannot equal the reserved API directory `workspace-docs`.
|
||||
- `name` is required; `description` is optional. Existing trim/nonblank, Unicode, and safe-error behavior used by descriptor metadata are reused; P1.1 introduces no new string-length limit.
|
||||
- Unknown keys, duplicate YAML keys, aliases/tags, multiple documents, malformed encodings, and
|
||||
duplicate IDs are rejected.
|
||||
- Catalog order is the workspace display order.
|
||||
- A catalog entry may temporarily have no descriptor. This is the only bootstrap state and is
|
||||
represented publicly as `configuration_required`; it is not session-activatable.
|
||||
- `workspace-docs` is a reserved top-level API directory and cannot be a workspace ID.
|
||||
- The dedicated registry accepts only catalog-listed workspace directories plus the reserved
|
||||
generated-docs directory and explicitly allowed root control files. An unlisted workspace
|
||||
directory/descriptor, or a descriptor whose `workspace.id`, `workspace.name`, or optional
|
||||
`workspace.description` differs from the catalog, is invalid. Activation fails atomically and
|
||||
retains the prior valid snapshot.
|
||||
- A present but empty or malformed descriptor is not "empty" for bootstrap. It is curator content
|
||||
and is rejected; the API never replaces it.
|
||||
|
||||
The descriptor retains `id`, `name`, and `description` so exports and immutable runtime snapshots
|
||||
remain self-contained. The catalog is authoritative, and exact equality prevents two names for one
|
||||
workspace.
|
||||
|
||||
## 4. Ownership and write policy
|
||||
|
||||
There are three writers with disjoint authority:
|
||||
|
||||
| Path | Owner | ThothII API behavior |
|
||||
| --- | --- | --- |
|
||||
| `thoth-workspaces.yaml` | curator | read and validate only |
|
||||
| `<id>/workspace.yaml` | curator after bootstrap | create only if absent at the exact base commit; never overwrite or delete |
|
||||
| `<id>/evidence/**` | curator | read Git objects only; never write, stage, clean, or materialize in P1.1 |
|
||||
| `workspaces/<id>/schema/**` | curator/future P5 | untouched by P1.1 |
|
||||
| `workspace-docs/<id>/*` | API | deterministic generated files only |
|
||||
|
||||
"Absent" means no Git object exists at `<id>/workspace.yaml` in the exact pulled base
|
||||
commit. A zero-byte file, comments-only YAML, symlink, submodule, tree, or malformed document counts
|
||||
as present and is never overwritten.
|
||||
|
||||
A browser/API bootstrap succeeds only when:
|
||||
|
||||
1. the catalog entry already exists at the request's exact `baseCommit`;
|
||||
2. the descriptor path is absent at that commit and remains absent after the pull;
|
||||
3. request metadata exactly matches the catalog;
|
||||
4. filesystem Evidence, when selected, already exists as a Git tree at
|
||||
`<id>/evidence` in that same base commit;
|
||||
5. the complete descriptor passes schema-v3 and operational publication checks.
|
||||
|
||||
The API then commits only the new descriptor and generated docs. Update and delete requests against
|
||||
an existing descriptor return a stable `workspace_curator_owned` conflict response and do not
|
||||
change any Git object. Curator deletion means removing the descriptor or catalog/directory through
|
||||
Git. A retained session snapshot remains available under the existing retention rules.
|
||||
|
||||
The managed checkout must no longer run a directory-wide clean under `workspaces/`. Failure cleanup
|
||||
is confined to the exact descriptor/docs files written by the failed API operation and proves their
|
||||
pre-operation identity before removal.
|
||||
|
||||
## 5. Synchronization and generated docs
|
||||
|
||||
Startup/bootstrap may pull, validate, and activate curator bytes but never pushes as a side effect
|
||||
of a status/read request. This deliberately means committed `workspace-docs` can remain stale until
|
||||
an explicit synchronization action; immutable local snapshots and exports always derive fresh docs
|
||||
from the validated active descriptor and never consume stale Git docs. The explicit
|
||||
`/workspace-registry/pull` operator action remains the synchronization boundary:
|
||||
|
||||
1. pull the curator commit;
|
||||
2. validate catalog, descriptors, namespace ownership, Evidence roots, and semantic-index ownership;
|
||||
3. compute deterministic `workspace-docs/<id>` bytes;
|
||||
4. if docs differ, create one docs-only follow-up commit without touching catalog, descriptors, or
|
||||
workspace content;
|
||||
5. validate and activate the resulting exact commit.
|
||||
|
||||
Every API write transaction records a bounded per-path journal before mutation: prior Git object type,
|
||||
mode, blob identity and bytes for tracked generated docs, or explicit absence, plus the intended
|
||||
post-write identity. If bootstrap/docs push races, is rejected, or fails, the prior valid active
|
||||
snapshot remains active; overwritten/deleted generated docs are restored byte-for-byte to their
|
||||
prior objects, newly created absent-before files are removed, and curator paths are never cleaned.
|
||||
A later explicit pull retries from a fresh remote head. The API removes stale generated docs only for workspaces that the curator has
|
||||
removed from the catalog or returned to `configuration_required`.
|
||||
|
||||
A docs-only follow-up commit is an authoritative workspace revision, as every active workspace is
|
||||
pinned to the complete Git commit rather than only to its descriptor blob. Automated acceptance
|
||||
must show that descriptor and Evidence blob identities are unchanged across that docs-only commit.
|
||||
|
||||
## 6. API and browser behavior
|
||||
|
||||
`GET /workspaces` is catalog-driven and returns every catalog entry in catalog order with:
|
||||
|
||||
- canonical ID, display name, and description from the catalog;
|
||||
- `configurationState: ready | configuration_required`;
|
||||
- `file: <id>/workspace.yaml`;
|
||||
- an immutable revision only for `ready` entries.
|
||||
|
||||
`language` is deliberately not a summary field because an unconfigured catalog slot has no
|
||||
descriptor language. It remains available from the descriptor detail for `ready` workspaces and is
|
||||
selected in the bootstrap draft before creation.
|
||||
|
||||
Descriptor-only routes (`GET /workspaces/:id`, diagnostics, export, session admission) reject a
|
||||
`configuration_required` entry as `workspace_not_activatable`.
|
||||
|
||||
`POST /workspaces/validate` remains a context-free schema check. It does not claim catalog
|
||||
agreement or publication eligibility. `POST /workspaces/publish` becomes bootstrap-create only.
|
||||
Legacy update/delete payloads are recognized and rejected as `workspace_curator_owned` rather than
|
||||
silently reinterpreted.
|
||||
|
||||
The browser:
|
||||
|
||||
- lists catalog slots, including those requiring configuration;
|
||||
- offers an editable, browser-local bootstrap draft only for `configuration_required` entries;
|
||||
- locks catalog-owned ID/name/description in that form;
|
||||
- requires explicit validation and confirmation before the one create;
|
||||
- turns the workspace read-only immediately after creation;
|
||||
- keeps Pull/Sync, Validate, installation Test, Export, and safe Evidence summary for existing
|
||||
workspaces;
|
||||
- removes update, delete, duplicate, field-conflict merge, and publish-existing controls;
|
||||
- versions or purges old update/deletion drafts so stale localStorage cannot restore write access;
|
||||
- treats imported bundles as bootstrap drafts only when they match an existing unconfigured
|
||||
catalog slot.
|
||||
|
||||
Existing descriptors are edited in the curator clone and become active after commit, push, and
|
||||
installation pull.
|
||||
|
||||
## 7. Evidence and external sources
|
||||
|
||||
For filesystem Evidence, schema v3 now requires exactly:
|
||||
|
||||
```yaml
|
||||
evidence:
|
||||
source:
|
||||
type: filesystem
|
||||
uri: <workspace-id>/evidence
|
||||
```
|
||||
|
||||
The lexical path invariant and same-commit Git-tree check remain P1.1 responsibilities. Recursive
|
||||
materialization, nested symlink rejection, byte acquisition, preprocessing, and indexing remain P6
|
||||
or later.
|
||||
|
||||
HTTP and S3 descriptor shapes, local `*_FILE` bindings, secret handling, timeout/limit policy,
|
||||
runtime rendering, and `tht config check` remain as delivered by P1. Those workspaces need no local
|
||||
`evidence/` directory. Evidence may also remain absent for compatibility.
|
||||
|
||||
## 8. Rejected alternatives
|
||||
|
||||
1. **Keep P1's three top-level source trees.** Rejected because it does not make a workspace a
|
||||
self-contained Git unit and does not match the desired curator model.
|
||||
2. **Place descriptors and Evidence together but let both API and curator update descriptors.**
|
||||
Rejected because it creates two authorities, restores field-level conflict merging, and risks
|
||||
overwriting reviewed Git content.
|
||||
3. **Chosen: API bootstrap once, then curator ownership.** This preserves a convenient initial
|
||||
form while making ordinary Git review the single authority for all subsequent descriptor and
|
||||
content changes.
|
||||
|
||||
## 9. Compatibility and migration
|
||||
|
||||
P1.1 is a repository-contract cutover, not a dual-format reader. New code rejects the old flat
|
||||
layout and a repository without `thoth-workspaces.yaml`. Existing repositories are migrated in one
|
||||
curator-reviewed commit:
|
||||
|
||||
```text
|
||||
workspaces/<id>.yaml -> <id>/workspace.yaml
|
||||
workspace-content/<id>/evidence/** -> <id>/evidence/**
|
||||
(create thoth-workspaces.yaml from reviewed descriptor metadata)
|
||||
```
|
||||
|
||||
No automatic in-product migrator rewrites a remote. The installation upgrades only after the
|
||||
migration commit is available. Historical immutable installation snapshots and retained session
|
||||
pins keep their current internal shape.
|
||||
|
||||
P2–P6 currently assume P1's old source paths in several places. P1.1 records that impact but does
|
||||
not edit those plans. After P1.1 automated and manual acceptance, a separate owner-approved plan
|
||||
will revise P2–P6.
|
||||
|
||||
## 10. Verification boundary
|
||||
|
||||
P1.1 must have independent automated and manual evidence. Accepted retained P1 artifacts remain immutable historical evidence; the old P1 process commands are not release gates for the superseding repository contract. The automated run starts from a clean
|
||||
local Git remote and proves catalog authority, missing-descriptor bootstrap, curator modification,
|
||||
API non-overwrite, nested Evidence identity, docs-only reconciliation, immutable snapshots,
|
||||
runtime render determinism, `tht config check`, negative cases, secret absence, and exact cleanup.
|
||||
It does not invoke preprocessing or Qdrant/Ollama/DWH services.
|
||||
|
||||
The manual environment is new and independent. The reviewer personally performs the bootstrap,
|
||||
refusal, curator-edit, pull/sync, UI read-only, Git-object, export, render, config-check, secret-scan,
|
||||
and cleanup checks. Project state remains `P1.1 manual acceptance: PENDING` until the reviewer
|
||||
records approval.
|
||||
Reference in New Issue
Block a user