docs: define workspace evidence registry contract
This commit is contained in:
@@ -0,0 +1,172 @@
|
||||
# Workspace Evidence v3 contract
|
||||
|
||||
This is the canonical public contract for the optional `evidence` object in a schema-v3
|
||||
workspace descriptor. Evidence is optional: a valid v3 descriptor without it remains operational.
|
||||
When present, `evidence` is strict: it contains `source` and a defaulted strict `policy`; every
|
||||
source variant and the policy reject unknown keys.
|
||||
|
||||
## Filesystem source
|
||||
|
||||
A filesystem source uses the exact URI `workspace-content/<workspace.id>/evidence`. `patterns` is
|
||||
a nonempty list of unique, normalized relative POSIX globs. Its defaults are
|
||||
`patterns: ["**/*.md"]` and `max_bytes: 10485760`.
|
||||
|
||||
### Example: filesystem
|
||||
|
||||
```yaml
|
||||
evidence:
|
||||
source:
|
||||
type: filesystem
|
||||
uri: workspace-content/example/evidence
|
||||
patterns:
|
||||
- "**/*.md"
|
||||
max_bytes: 10485760
|
||||
policy:
|
||||
max_chunk_chars: 4000
|
||||
retain_published_generations: 3
|
||||
```
|
||||
|
||||
Safe: `workspace-content/example/evidence`. Unsafe filesystem identities include `/srv/evidence`,
|
||||
`workspace-content/another/evidence`, and `workspace-content/example/../another/evidence` because
|
||||
absolute, cross-namespace, and traversal paths are not canonical.
|
||||
|
||||
## HTTP source
|
||||
|
||||
`uris` is a nonempty, unique list of canonical HTTP or HTTPS provenance identities. Each URI must
|
||||
have no whitespace, control character, backslash, userinfo, query, or fragment. Public mode uses
|
||||
`authentication: none`; signed mode uses `authentication: signed_urls_file`. Defaults are
|
||||
`authentication: none`, `connect_timeout_ms: 5000`, `read_timeout_ms: 30000`,
|
||||
`max_bytes: 10485760`, `max_redirects: 5`, `allow_private_hosts: false`, and
|
||||
`max_cache_bytes: 67108864`.
|
||||
|
||||
### Example: http
|
||||
|
||||
```yaml
|
||||
evidence:
|
||||
source:
|
||||
type: http
|
||||
uris:
|
||||
- https://evidence.example.invalid/report.md
|
||||
authentication: signed_urls_file
|
||||
connect_timeout_ms: 5000
|
||||
read_timeout_ms: 30000
|
||||
max_bytes: 10485760
|
||||
max_redirects: 5
|
||||
allow_private_hosts: false
|
||||
max_cache_bytes: 67108864
|
||||
policy:
|
||||
max_chunk_chars: 4000
|
||||
retain_published_generations: 3
|
||||
```
|
||||
|
||||
The shown provenance URI is safe and query-free. An HTTP fragment identity such as
|
||||
`https://evidence.example.invalid/report.md#section` is unsafe. Query-bearing and userinfo-bearing
|
||||
HTTP identities are rejected; public documentation must not spell or publish a signed transport
|
||||
URL.
|
||||
|
||||
## S3 source
|
||||
|
||||
`uri` is a canonical `s3://` identity with a valid bucket and no port, userinfo, query, or fragment.
|
||||
`endpoint_url`, when present, is an origin-only HTTP(S) URL; `region`, when present, is nonblank.
|
||||
Defaults are `credentials: ambient`, `trusted_endpoint: false`, `allow_private_endpoint: false`,
|
||||
`allow_insecure_endpoint: false`, `max_bytes: 10485760`, `max_objects: 10000`, `max_pages: 100`,
|
||||
and `page_size: 1000` (and page size cannot exceed 1000). A custom endpoint requires
|
||||
`trusted_endpoint: true`; an HTTP endpoint additionally requires `allow_insecure_endpoint: true`.
|
||||
Static mode uses `credentials: static_files`, requires access-key and secret-key files together,
|
||||
and permits an optional session-token file.
|
||||
|
||||
### Example: s3
|
||||
|
||||
```yaml
|
||||
evidence:
|
||||
source:
|
||||
type: s3
|
||||
uri: s3://example-evidence/curated/
|
||||
credentials: static_files
|
||||
trusted_endpoint: false
|
||||
allow_private_endpoint: false
|
||||
allow_insecure_endpoint: false
|
||||
max_bytes: 10485760
|
||||
max_objects: 10000
|
||||
max_pages: 100
|
||||
page_size: 1000
|
||||
policy:
|
||||
max_chunk_chars: 4000
|
||||
retain_published_generations: 3
|
||||
```
|
||||
|
||||
Safe: `s3://example-evidence/curated/`. Unsafe identities include
|
||||
`s3://Invalid_Bucket/evidence` and `s3://example-evidence/evidence#section`. S3 userinfo and query
|
||||
identities are rejected in prose and implementation; no credential-bearing example is published.
|
||||
|
||||
## Policy
|
||||
|
||||
The strict policy defaults to `max_chunk_chars: 4000` and
|
||||
`retain_published_generations: 3`.
|
||||
|
||||
## Installation files
|
||||
|
||||
The namespace is the workspace ID uppercased with every `-` changed to `_`. All Evidence variables
|
||||
hold file paths, never credential or signed-URL values.
|
||||
|
||||
| Mode | Variable | File contract |
|
||||
| --- | --- | --- |
|
||||
| Signed HTTP | `THT_WS_<NAMESPACE>_EVIDENCE_SIGNED_URLS_FILE` | Required for `signed_urls_file`; nonempty JSON string array in declared-URI order; query-stripped identities must match `uris`. |
|
||||
| Static S3 pair | `THT_WS_<NAMESPACE>_EVIDENCE_ACCESS_KEY_FILE` and `THT_WS_<NAMESPACE>_EVIDENCE_SECRET_KEY_FILE` | Required together for `static_files`. |
|
||||
| Static S3 session | `THT_WS_<NAMESPACE>_EVIDENCE_SESSION_TOKEN_FILE` | Optional, and valid only with the required access/secret pair. |
|
||||
|
||||
Every variable is an absolute path to a readable regular file whose resolved target is strictly below one of the roots configured by `THT_WORKSPACE_SECRET_ROOTS`. Scalar S3 files are nonempty
|
||||
UTF-8 tokens without whitespace. Public docs, exports, and rendered YAML never expose file contents. `changeme`, `replace-me`, `YOUR_SECRET`, `<secret>`, access-key-looking strings, and any
|
||||
credential-bearing or query-bearing URI are forbidden as public placeholder values.
|
||||
|
||||
## One shared registry repository
|
||||
|
||||
All workspace namespaces live in one Git repository:
|
||||
|
||||
```text
|
||||
registry.git/
|
||||
├── workspaces/
|
||||
│ ├── example.yaml
|
||||
│ └── another.yaml
|
||||
├── workspace-content/
|
||||
│ ├── example/evidence/...
|
||||
│ └── another/evidence/...
|
||||
└── workspace-docs/
|
||||
├── example/{contract.env.example,README.md}
|
||||
└── another/{contract.env.example,README.md}
|
||||
```
|
||||
|
||||
Curators change only `workspace-content/<id>/evidence/**` through a normal clone. The API publishes
|
||||
only `workspaces/<id>.yaml` and
|
||||
`workspace-docs/<id>/{contract.env.example,README.md}`. It never writes Evidence source bytes.
|
||||
|
||||
## Registry revision and phase ownership
|
||||
|
||||
| Relationship | Contract |
|
||||
| --- | --- |
|
||||
| Revision identity | The descriptor blob and filesystem Evidence root tree are checked at the same 40-hex Git commit. |
|
||||
| Content-only revision | An Evidence-only commit changes authoritative `revision.commit` even when the descriptor blob is unchanged. |
|
||||
| Browser | Create and edit flows preserve and show a read-only Evidence summary. |
|
||||
| Export | Export remains exactly manifest, descriptor, contract, and README; it excludes Evidence bytes. |
|
||||
| P1 | Validates the lexical URI and proves the declared filesystem root object is a Git tree at that same commit; it does not recursively inspect nested symlinks. |
|
||||
| P6 | Owns commit-addressed materialization, realpath and recursive containment, nested-symlink checks, and race checks. |
|
||||
|
||||
P1 performs no acquisition, extraction, preprocessing/indexing, embeddings, Qdrant writes, `ACTIVE` publication, retention, or GC.
|
||||
|
||||
## Operator validation
|
||||
|
||||
After the runtime configuration is rendered or acquired, validate it with the exact per-command
|
||||
option ordering:
|
||||
|
||||
```sh
|
||||
tht config check -c <path>
|
||||
```
|
||||
|
||||
Stop after validation. P2/P6 later owns preprocessing and materialization.
|
||||
|
||||
## Acceptance states
|
||||
|
||||
These gates are independent and are not implied by this documentation contract.
|
||||
|
||||
automated integration: PENDING
|
||||
manual acceptance: PENDING
|
||||
@@ -5,3 +5,10 @@ THT_WS_NORTH_STAR_RESEARCH_DWH_HOST=dwh.internal.example
|
||||
THT_WS_NORTH_STAR_RESEARCH_DWH_PORT=5432
|
||||
THT_WS_NORTH_STAR_RESEARCH_DWH_USER=thoth_reader
|
||||
THT_WS_NORTH_STAR_RESEARCH_DWH_PASSWORD_FILE=/run/secrets/north-star-research-dwh-password
|
||||
|
||||
# Evidence examples use separate illustrative namespaces because one descriptor selects one mode.
|
||||
# Values are container file paths only; signed URLs and credential contents stay in those files.
|
||||
THT_WS_SIGNED_HTTP_EVIDENCE_SIGNED_URLS_FILE=/run/secrets/signed-http-evidence-urls.json
|
||||
THT_WS_STATIC_S3_EVIDENCE_ACCESS_KEY_FILE=/run/secrets/static-s3-evidence-access-key
|
||||
THT_WS_STATIC_S3_EVIDENCE_SECRET_KEY_FILE=/run/secrets/static-s3-evidence-secret-key
|
||||
THT_WS_STATIC_S3_EVIDENCE_SESSION_TOKEN_FILE=/run/secrets/static-s3-evidence-session-token
|
||||
|
||||
@@ -39,10 +39,13 @@ Create one private repository such as `thoth-workspaces.git`. It contains canoni
|
||||
definitions and generated artifacts only:
|
||||
|
||||
```text
|
||||
thoth-workspaces.yaml
|
||||
workspaces/<workspace-id>.yaml
|
||||
workspaces/<workspace-id>.env.example
|
||||
workspaces/<workspace-id>.md
|
||||
registry.git/
|
||||
├── workspaces/
|
||||
│ └── <workspace-id>.yaml
|
||||
├── workspace-content/
|
||||
│ └── <workspace-id>/evidence/...
|
||||
└── workspace-docs/
|
||||
└── <workspace-id>/{contract.env.example,README.md}
|
||||
```
|
||||
|
||||
For SSH, use a scoped deploy key, a verified `known_hosts` file, and strict host-key checking. For
|
||||
@@ -66,6 +69,32 @@ THT_WORKSPACE_GIT_CA_FILE=/absolute/path/installation-secrets/git-ca.pem
|
||||
For HTTPS set `THT_WORKSPACE_GIT_CREDENTIALS_FILE` instead of the SSH key/known-hosts pair. Remote
|
||||
and branch are non-secret; every `*_FILE` is a local path whose content never enters Git or logs.
|
||||
|
||||
## Curator flow for shared-registry Evidence
|
||||
|
||||
Follow this order; the [canonical Evidence contract](../contracts/workspace-evidence-v3.md) defines
|
||||
the source shapes and safety boundary.
|
||||
|
||||
1. Clone the one shared registry, or update the review clone with `git pull --ff-only`.
|
||||
2. Add source bytes below `workspace-content/<id>/evidence`, then commit and push.
|
||||
3. Validate and publish the descriptor against that base commit.
|
||||
4. Inspect `workspace-docs/<id>/contract.env.example` and `workspace-docs/<id>/README.md`.
|
||||
5. Provision only the selected Evidence `*_FILE` files outside Git and strictly below a root in `THT_WORKSPACE_SECRET_ROOTS`; add matching host-only `*_SOURCE` paths for the generated connector override.
|
||||
6. Render or acquire the runtime config, then run `tht config check -c <path>`.
|
||||
7. Stop: P2/P6 later performs preprocessing and materialization.
|
||||
|
||||
For example, a signed-HTTP workspace and a different static-S3 workspace can use these host-only
|
||||
connector sources; the values are paths, not file contents:
|
||||
|
||||
```dotenv
|
||||
THT_WS_SIGNED_HTTP_EVIDENCE_SIGNED_URLS_SOURCE=/absolute/path/installation-secrets/signed-http-evidence-urls.json
|
||||
THT_WS_STATIC_S3_EVIDENCE_ACCESS_KEY_SOURCE=/absolute/path/installation-secrets/static-s3-evidence-access-key
|
||||
THT_WS_STATIC_S3_EVIDENCE_SECRET_KEY_SOURCE=/absolute/path/installation-secrets/static-s3-evidence-secret-key
|
||||
THT_WS_STATIC_S3_EVIDENCE_SESSION_TOKEN_SOURCE=/absolute/path/installation-secrets/static-s3-evidence-session-token
|
||||
```
|
||||
|
||||
The descriptor and declared filesystem root are validated at the same registry commit. The
|
||||
browser shows a read-only Evidence summary, while exports omit Evidence bytes.
|
||||
|
||||
## Shared Git values, local bindings, and secret files
|
||||
|
||||
| Location | Contains | Never contains |
|
||||
|
||||
@@ -50,8 +50,10 @@ account Gitea administration, database-superuser rights, or a shell in the Git h
|
||||
|
||||
Create a private Gitea (or compatible Git) repository such as `platform/thoth-workspaces`. Protect
|
||||
`main` according to the release policy and grant the ThothII publisher only the intended repository
|
||||
scope. Commit canonical schema-v3 descriptors and generated `.md`/`.env.example` artifacts only;
|
||||
do not commit installation bindings or secret material.
|
||||
scope. Commit canonical schema-v3 descriptors under `workspaces/<id>.yaml`, curated Evidence
|
||||
under `workspace-content/<id>/evidence/**`, and generated public artifacts only at
|
||||
`workspace-docs/<id>/README.md` and `workspace-docs/<id>/contract.env.example`; do not commit
|
||||
installation bindings or secret material.
|
||||
|
||||
For SSH, create a least-privilege deploy key, record Gitea's host key in managed known-hosts, and
|
||||
use `ssh://git@git.example.invalid/platform/thoth-workspaces.git`. For HTTPS, create a scoped
|
||||
@@ -62,6 +64,32 @@ Bootstrap an empty remote from a temporary review clone: migrate legacy descript
|
||||
schema-v3 identity and generated artifacts, commit, and push `main`. The running server is not an
|
||||
authoring environment for migration.
|
||||
|
||||
## Curator flow for shared-registry Evidence
|
||||
|
||||
Follow this order; the [canonical Evidence contract](../contracts/workspace-evidence-v3.md) defines
|
||||
the source shapes and safety boundary.
|
||||
|
||||
1. Clone the one shared registry, or update the review clone with `git pull --ff-only`.
|
||||
2. Add source bytes below `workspace-content/<id>/evidence`, then commit and push.
|
||||
3. Validate and publish the descriptor against that base commit.
|
||||
4. Inspect `workspace-docs/<id>/contract.env.example` and `workspace-docs/<id>/README.md`.
|
||||
5. Provision only the selected Evidence `*_FILE` files outside Git and strictly below a root in `THT_WORKSPACE_SECRET_ROOTS`; add matching host-only `*_SOURCE` paths for the generated connector override.
|
||||
6. Render or acquire the runtime config, then run `tht config check -c <path>`.
|
||||
7. Stop: P2/P6 later performs preprocessing and materialization.
|
||||
|
||||
For example, separate signed-HTTP and static-S3 workspaces can use these host-only connector source
|
||||
paths:
|
||||
|
||||
```dotenv
|
||||
THT_WS_SIGNED_HTTP_EVIDENCE_SIGNED_URLS_SOURCE=/srv/thothii/secrets/signed-http-evidence-urls.json
|
||||
THT_WS_STATIC_S3_EVIDENCE_ACCESS_KEY_SOURCE=/srv/thothii/secrets/static-s3-evidence-access-key
|
||||
THT_WS_STATIC_S3_EVIDENCE_SECRET_KEY_SOURCE=/srv/thothii/secrets/static-s3-evidence-secret-key
|
||||
THT_WS_STATIC_S3_EVIDENCE_SESSION_TOKEN_SOURCE=/srv/thothii/secrets/static-s3-evidence-session-token
|
||||
```
|
||||
|
||||
The descriptor and declared filesystem root are validated at the same registry commit. The
|
||||
browser shows a read-only Evidence summary, while exports omit Evidence bytes.
|
||||
|
||||
## Git credentials, CA, SSH key, and known-hosts mounts
|
||||
|
||||
Use the secret manager or a protected host-only procedure to create independent regular files under
|
||||
|
||||
Reference in New Issue
Block a user