From c7338969d7c7c1c396d9099b7ab2d309b70ab6cf Mon Sep 17 00:00:00 2001 From: mptyl Date: Mon, 10 Aug 2026 17:36:24 +0200 Subject: [PATCH] fix: bind complete P1 manual dist graph and snapshot identity prepare records an immutable manifest of every regular backend/dist file (path/size/sha256/dev/ino) in the owned root and binds its record identity in ownership; serve revalidates record and every file before spawn, passes the manifest to the child on fd 4, and the immutable preload hash-verifies all files at startup and serves only cached verified bytes for any import below backend/dist, so imported dependency replacement is refused before RUNNING or never executes. The render command validates the commit snapshot.json manifest, binds snapshot bytes to the manifest digest and the installed Git blob, and passes the expected digest to the renderer, which revalidates head/files digest with bounded no-follow reads and renders only verified bytes with lease release on refusal. --- .../reviews/task9-quality-audit-final5.md | 66 +++++++++++ backend/scripts/p1-manual-acceptance.mjs | 103 ++++++++++++------ backend/scripts/p1-manual-acceptance.test.mjs | 94 +++++++++++++++- backend/scripts/p1-render-snapshot.mjs | 66 ++++++++--- backend/scripts/p1-render-snapshot.test.mjs | 29 +++-- docs/testing/p1-manual-acceptance.md | 38 ++++--- 6 files changed, 323 insertions(+), 73 deletions(-) create mode 100644 .artifacts/reviews/task9-quality-audit-final5.md diff --git a/.artifacts/reviews/task9-quality-audit-final5.md b/.artifacts/reviews/task9-quality-audit-final5.md new file mode 100644 index 00000000..665653b3 --- /dev/null +++ b/.artifacts/reviews/task9-quality-audit-final5.md @@ -0,0 +1,66 @@ +# Task 9 quality audit — final 5 + +**Scope:** the two blocking findings from `task9-quality-audit-final4.md` — unbound production +module graph at manual serve, and commit-addressed snapshots accepted without content identity at +render. Manual acceptance remains **PENDING**; no `VERDICT.md` was created. + +## Verdict: APPROVED for the two final integrity blockers + +### 1. Manual serve binds the complete `backend/dist` module graph, not only `server.js` + +`prepare` now builds a post-build manifest of every regular `backend/dist` file +(relative path, size, SHA-256, device, inode) and writes it as an exclusive `0600` record +(`installation/runtime/backend-dist.manifest.json`) inside the owned root; `ownership.json` +records that record's path/device/inode/size/SHA-256. `serve` revalidates the manifest record +identity and bytes, revalidates every distribution file against it (no-follow, single inode, +size and digest), and refuses before spawning. The manifest descriptor is passed to the child on +fd 4 together with the entrypoint on fd 3. The immutable preload parses the manifest, verifies +the entrypoint cross-digest, reads and hash-verifies **every** file at startup, caches the +verified bytes, and its load hook serves **only** those cached bytes for any import below +`backend/dist` (entry URL still served from the bound fd-3 bytes). A same-path regular +replacement of any imported dependency is therefore refused before `RUNNING` (serve-time +validation), refused at child startup (startup verification), or rendered harmless (cached +bytes), and the parent revalidates the full manifest at `RUNNING` publication and at `stop`. + +### 2. Renderer binds snapshot content to its commit identity + +The generated render command validates the bounded saved read/publish revisions, the +commit-addressed owned snapshot path, the installed Git HEAD, and the bounded +`snapshot.json` manifest of that commit: `head` equals the commit, `files[.yaml]` is the +SHA-256 of the snapshot bytes, the manifest revision binds commit/blob/snapshot path, the saved +revision blob equals the manifest blob, and `git rev-parse :workspaces/.yaml` plus +`git hash-object` of the snapshot bytes both equal that blob. It passes the expected digest as +`--snapshot-sha256`. The renderer re-reads the bounded `snapshot.json` (`head`, +`files[.yaml]` must equal the carried digest), opens the snapshot once with no-follow +semantics and bounded reads, renders only the digest-verified bytes, re-verifies around lease +publication, releases the lease in `finally`, and publishes no output on any refusal. + +## Deterministic regressions added + +- static regular replacement of an imported production dependency after `prepare` is refused, + no marker, no accepted PID record, no orphan; +- deterministic dependency check/load swap (`beforeSpawn` rename) is refused by the child's + startup verification, no marker, no PID record, no orphan; +- after `RUNNING`, a same-path regular dependency replacement is never executed: the loader + serves the verified cached bytes (health-visible source stays the original) and the marker is + absent; +- renderer refuses a same-path regular snapshot byte replacement against the carried digest and + manifest, with lease release and no output; +- renderer refuses manifest `head`, `files` digest, expected-digest, missing, and malformed + cases, with lease release and no output; +- wrapper refuses missing manifest, manifest head/digest/revision tampering, saved-revision blob + mismatch, Git blob mismatch, and snapshot-vs-Git-bytes mismatch, and passes the exact + `--snapshot-sha256` on the valid path (stub renderer records arguments). + +## Verification + +- `bash scripts/test-p1-manual-acceptance.sh` (backend build + both suites): **59 tests, 59 + pass, 0 fail**; no `8791/8792` listener and no `--p1-manual-nonce` process remain. +- `npx tsc --noEmit -p .` (backend): PASS. +- Real-repository `prepare` + `cleanup` cycle: 39 distribution files bound, entrypoint + cross-digest verified, owned root fully removed afterwards. +- Diff check: only the seven Task 9 paths are touched; no Task 8 file was modified. +- This report and the implementation contain no fixture secret or canary values. + +Manual acceptance remains **PENDING** by design; the walkthrough and human verdict are +unchanged. diff --git a/backend/scripts/p1-manual-acceptance.mjs b/backend/scripts/p1-manual-acceptance.mjs index 78b7898f..f0664bab 100755 --- a/backend/scripts/p1-manual-acceptance.mjs +++ b/backend/scripts/p1-manual-acceptance.mjs @@ -45,17 +45,41 @@ function legacySupervisorPath(root){return join(root,"installation/runtime/p1-ba const CONTROL_PORT=8792; const PRELOAD_SOURCE=`import net from "node:net"; import { createHash } from "node:crypto"; -import { fstatSync, readFileSync } from "node:fs"; +import { closeSync, constants, fstatSync, openSync, readFileSync, readSync, realpathSync } from "node:fs"; import { registerHooks } from "node:module"; -import { pathToFileURL } from "node:url"; +import { dirname, join, sep } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; const HOST="127.0.0.1",PORT=8792,HTTP_PORT=8791,HEX=/^[0-9a-f]{64}$/; const argv=process.argv.slice(2),noncePrefix="--p1-manual-nonce=",rootPrefix="--p1-root=",controlPrefix="--p1-control-nonce=",shaPrefix="--p1-entry-sha256=",devPrefix="--p1-entry-dev=",inoPrefix="--p1-entry-ino="; const prefixes=[noncePrefix,rootPrefix,controlPrefix,shaPrefix,devPrefix,inoPrefix]; if(argv.length!==6||argv.some((value,index)=>!value.startsWith(prefixes[index])))throw new Error("manual control identity arguments refused"); const nonce=argv[0].slice(noncePrefix.length),root=argv[1].slice(rootPrefix.length),controlNonce=argv[2].slice(controlPrefix.length),entrySha=argv[3].slice(shaPrefix.length),entryDev=argv[4].slice(devPrefix.length),entryIno=argv[5].slice(inoPrefix.length); if(!HEX.test(nonce)||!root.startsWith("/")||!HEX.test(controlNonce)||!HEX.test(entrySha)||!/^[0-9]+$/.test(entryDev)||!/^[0-9]+$/.test(entryIno))throw new Error("manual control identity refused"); +if(!Number.isInteger(constants.O_NOFOLLOW))throw new Error("manual distribution no-follow protection is unavailable"); const entryStat=fstatSync(3),entrySource=readFileSync(3);if(!entryStat.isFile()||String(entryStat.dev)!==entryDev||String(entryStat.ino)!==entryIno||createHash("sha256").update(entrySource).digest("hex")!==entrySha)throw new Error("manual entrypoint FD identity refused"); -const entryUrl=pathToFileURL(process.argv[1]).href;registerHooks({load(url,context,nextLoad){if(url===entryUrl)return{format:"module",shortCircuit:true,source:entrySource};return nextLoad(url,context);}}); +const manifestStat=fstatSync(4);if(!manifestStat.isFile()||manifestStat.size<1||manifestStat.size>8388608)throw new Error("manual distribution manifest FD identity refused"); +let manifest;try{manifest=JSON.parse(readFileSync(4));}catch{throw new Error("manual distribution manifest is malformed");} +const entryPath=realpathSync(process.argv[1]),distRoot=dirname(entryPath); +if(manifest?.schemaVersion!==1||manifest.kind!=="p1-manual-dist-manifest"||manifest.root!==distRoot||!manifest.files||typeof manifest.files!=="object"||Array.isArray(manifest.files))throw new Error("manual distribution manifest identity refused"); +const distEntries=Object.entries(manifest.files);if(distEntries.length<1||distEntries.length>20000)throw new Error("manual distribution manifest identity refused"); +const distBytes=new Map(); +for(const[rel,file]of distEntries){ + if(typeof rel!=="string"||!rel||rel.startsWith("/")||rel.startsWith("..")||rel.includes("\\\\")||rel.includes("/./")||rel.endsWith("/")||!Number.isSafeInteger(file?.size)||file.size<1||file.size>33554432||!HEX.test(file?.sha256??"")||!/^[0-9]+$/.test(String(file?.dev))||!/^[0-9]+$/.test(String(file?.ino)))throw new Error("manual distribution manifest is malformed"); + const path=join(distRoot,rel),fd=openSync(path,constants.O_RDONLY|constants.O_NOFOLLOW); + try{ + const before=fstatSync(fd); + if(!before.isFile()||before.nlink!==1||String(before.dev)!==String(file.dev)||String(before.ino)!==String(file.ino)||before.size!==file.size)throw new Error("manual distribution module identity changed"); + const bytes=Buffer.alloc(before.size);let offset=0; + while(offset{const address=ownedListener?.listening?ownedListener.address():undefined;return{listening:Boolean(ownedListener?.listening&&address&&address.address===HOST&&address.port===HTTP_PORT),host:address?.address,port:address?.port,generation:listenGeneration};}; const originalListen=net.Server.prototype.listen;net.Server.prototype.listen=function(...args){const candidate=this;candidate.once("listening",()=>{const address=candidate.address();if(address&&address.address===HOST&&address.port===HTTP_PORT){ownedListener=candidate;listenGeneration++;}});candidate.on("close",()=>{if(ownedListener===candidate){ownedListener=undefined;if(state==="READY")state="LISTENER_CLOSED";}});return originalListen.apply(candidate,args);}; @@ -67,11 +91,6 @@ const watchdog=setTimeout(()=>{if(state!=="STARTING")return;console.error("manua const PRELOAD=`data:text/javascript;base64,${Buffer.from(PRELOAD_SOURCE,"utf8").toString("base64")}`; async function controlRequest(control,payload){if(control?.host!==HOST||!Number.isSafeInteger(control?.port)||control.port<1||control.port>65535)throw new Error("backend control identity mismatch");return await new Promise((resolvePromise,reject)=>{const socket=net.createConnection({host:control.host,port:control.port}),timer=setTimeout(()=>socket.destroy(new Error("backend control timeout")),2000);let bytes="";socket.setEncoding("utf8");socket.on("connect",()=>socket.end(JSON.stringify(payload)));socket.on("data",chunk=>{bytes+=chunk;if(bytes.length>2048)socket.destroy(new Error("backend control response too large"));});socket.on("error",reject);socket.on("close",()=>{clearTimeout(timer);let value;try{value=JSON.parse(bytes);}catch{return reject(new Error("backend control response is malformed"));}resolvePromise(value);});});} -async function distManifest(repo){ - const base=join(repo,"backend","dist"), files=[]; - async function walk(dir){ for(const entry of await readdir(dir,{withFileTypes:true})){ const path=join(dir,entry.name); if(entry.isSymbolicLink())throw new Error("production distribution contains a symlink"); if(entry.isDirectory())await walk(path); else if(entry.isFile()){ const bytes=await readFile(path); if(bytes.length>33554432)throw new Error("production distribution file is unbounded"); files.push({path:relative(base,path).split(sep).join("/"),size:bytes.length,sha256:createHash("sha256").update(bytes).digest("hex")}); } else throw new Error("production distribution entry is unsupported"); if(files.length>20000)throw new Error("production distribution is unbounded"); }} - await walk(base); files.sort((a,b)=>a.path.localeCompare(b.path)); const bytes=Buffer.from(JSON.stringify({root:base,files})); return {root:base,files,sha256:createHash("sha256").update(bytes).digest("hex")}; -} function ownedValue(repo,root,nonce,{backendLog=null,entrypoint,distManifest=null,stage="PREPARING",createdAt=new Date().toISOString()}={}){return{schemaVersion:1,kind:"p1-manual-acceptance",nonce,repositoryRoot:repo,root,status:"PENDING",stage,createdAt,listener:{host:HOST,port:PORT,state:"stopped"},backendLog,entrypoint,distManifest,resources:[root,{kind:"fastify",host:HOST,port:PORT}]};} function validEntrypoint(value,repo){return value?.path===join(repo,"backend/dist/server.js")&&Number.isSafeInteger(value.dev)&&Number.isSafeInteger(value.ino)&&Number.isSafeInteger(value.size)&&value.size>0&&HEX64.test(value.sha256??"");} async function readBoundEntrypoint(repo){ @@ -79,7 +98,16 @@ async function readBoundEntrypoint(repo){ try{handle=await open(path,constants.O_RDONLY|constants.O_NOFOLLOW);const before=await handle.stat(),pathEntry=await lstat(path);if(!before.isFile()||before.nlink!==1||pathEntry.isSymbolicLink()||!pathEntry.isFile()||!sameEntry(before,pathEntry))throw new Error("production server identity is unsafe");if(before.size<1||before.size>33554432)throw new Error("production entrypoint is unbounded");const bytes=Buffer.alloc(before.size);let offset=0;while(offset{});throw error;} } async function requireEntrypointPathIdentity(entrypoint){const entry=await lstat(entrypoint.path);if(!entry.isFile()||entry.isSymbolicLink()||entry.nlink!==1||entry.dev!==entrypoint.dev||entry.ino!==entrypoint.ino||entry.size!==entrypoint.size)throw new Error("production entrypoint identity changed");const bytes=await readFile(entrypoint.path);if(bytes.length!==entrypoint.size||createHash("sha256").update(bytes).digest("hex")!==entrypoint.sha256)throw new Error("production entrypoint bytes changed");return entry;} -export async function readManualOwnership({repositoryRoot=defaultRepositoryRoot}={}){const repo=realpathSync(repositoryRoot),root=fixedManualRoot(repo);noSymlinkExisting(repo,root);let rootEntry,ownershipEntry;try{rootEntry=await lstat(root);ownershipEntry=await lstat(join(root,"ownership.json"));}catch{throw new Error("manual ownership is missing");}if(!rootEntry.isDirectory()||rootEntry.isSymbolicLink()||await realpath(root)!==root||!ownershipEntry.isFile()||ownershipEntry.isSymbolicLink())throw new Error("manual ownership is unsafe");let value;try{value=JSON.parse(await readFile(join(root,"ownership.json"),"utf8"));}catch{throw new Error("manual ownership is malformed");}const baseValid=value.schemaVersion===1&&value.kind==="p1-manual-acceptance"&&HEX64.test(value.nonce??"")&&value.repositoryRoot===repo&&value.root===root&&value.status==="PENDING"&&["PREPARING","READY"].includes(value.stage)&&value.listener?.host===HOST&&value.listener?.port===PORT&&value.listener?.state==="stopped"&&typeof value.createdAt==="string"&&validEntrypoint(value.entrypoint,repo)&&JSON.stringify(value.resources)===JSON.stringify([root,{kind:"fastify",host:HOST,port:PORT}]);const readyLog=value.backendLog?.path===join(root,"logs/backend.log")&&Number.isSafeInteger(value.backendLog?.dev)&&Number.isSafeInteger(value.backendLog?.ino);if(!baseValid||(value.stage==="READY"?!readyLog:value.backendLog!==null))throw new Error("manual ownership identity mismatch");return value;} +function validDistManifest(value,root){return value?.path===join(root,"installation","runtime","backend-dist.manifest.json")&&Number.isSafeInteger(value.dev)&&Number.isSafeInteger(value.ino)&&Number.isSafeInteger(value.size)&&value.size>0&&HEX64.test(value.sha256??"");} +function parseDistManifest(bytes,distRoot){let value;try{value=JSON.parse(bytes.toString("utf8"));}catch{throw new Error("production distribution manifest is malformed");}const files=value?.files;if(value?.schemaVersion!==1||value.kind!=="p1-manual-dist-manifest"||value.root!==distRoot||!files||typeof files!=="object"||Array.isArray(files))throw new Error("production distribution manifest is malformed");const entries=Object.entries(files);if(entries.length<1||entries.length>20000)throw new Error("production distribution manifest is malformed");for(const[rel,file]of entries){if(!/^[^./\\][^/\\]*(?:\/[^./\\][^/\\]*)*$/.test(rel)||!Number.isSafeInteger(file?.size)||file.size<1||file.size>33554432||!HEX64.test(file?.sha256??"")||!Number.isSafeInteger(file?.dev)||!Number.isSafeInteger(file?.ino))throw new Error("production distribution manifest is malformed");}return{value,files};} +async function buildDistManifest(repo){const dist=join(repo,"backend","dist"),files={};let count=0,total=0;async function walk(dir){for(const entry of await readdir(dir,{withFileTypes:true})){const path=join(dir,entry.name);if(entry.isSymbolicLink())throw new Error("production distribution contains a symlink");if(entry.isDirectory()){await walk(path);continue;}if(!entry.isFile())throw new Error("production distribution contains a nonregular entry");if(++count>20000)throw new Error("production distribution is unbounded");const rel=relative(dist,path).split(sep).join("/");let handle;try{handle=await open(path,constants.O_RDONLY|constants.O_NOFOLLOW);const before=await handle.stat();if(!before.isFile()||before.nlink!==1||before.size<1||before.size>33554432)throw new Error("production distribution module is unsafe");total+=before.size;if(total>536870912)throw new Error("production distribution is unbounded");const bytes=Buffer.alloc(before.size);let offset=0;while(offset{});}}}await walk(dist);return{schemaVersion:1,kind:"p1-manual-dist-manifest",root:dist,files};} +async function readBoundDistManifest(repo,owned){ + if(!Number.isInteger(constants.O_NOFOLLOW))throw new Error("production distribution manifest no-follow protection is unavailable");const distManifest=owned.distManifest;let handle; + try{handle=await open(distManifest.path,constants.O_RDONLY|constants.O_NOFOLLOW);const before=await handle.stat(),pathEntry=await lstat(distManifest.path);if(!before.isFile()||before.nlink!==1||pathEntry.isSymbolicLink()||!pathEntry.isFile()||!sameEntry(before,pathEntry)||before.dev!==distManifest.dev||before.ino!==distManifest.ino||before.size!==distManifest.size)throw new Error("production distribution manifest identity changed");if(before.size<1||before.size>8388608)throw new Error("production distribution manifest is unbounded");const bytes=Buffer.alloc(before.size);let offset=0;while(offset{});throw error;} +} +async function validateDistFiles(repo,files){const dist=join(repo,"backend","dist");for(const[rel,file]of Object.entries(files)){const path=join(dist,...rel.split("/"));let handle;try{handle=await open(path,constants.O_RDONLY|constants.O_NOFOLLOW);const before=await handle.stat(),pathEntry=await lstat(path);if(!before.isFile()||before.nlink!==1||pathEntry.isSymbolicLink()||!pathEntry.isFile()||!sameEntry(before,pathEntry)||before.dev!==file.dev||before.ino!==file.ino||before.size!==file.size)throw new Error("production distribution module identity changed");const bytes=Buffer.alloc(before.size);let offset=0;while(offset{});}}} + +export async function readManualOwnership({repositoryRoot=defaultRepositoryRoot}={}){const repo=realpathSync(repositoryRoot),root=fixedManualRoot(repo);noSymlinkExisting(repo,root);let rootEntry,ownershipEntry;try{rootEntry=await lstat(root);ownershipEntry=await lstat(join(root,"ownership.json"));}catch{throw new Error("manual ownership is missing");}if(!rootEntry.isDirectory()||rootEntry.isSymbolicLink()||await realpath(root)!==root||!ownershipEntry.isFile()||ownershipEntry.isSymbolicLink())throw new Error("manual ownership is unsafe");let value;try{value=JSON.parse(await readFile(join(root,"ownership.json"),"utf8"));}catch{throw new Error("manual ownership is malformed");}const baseValid=value.schemaVersion===1&&value.kind==="p1-manual-acceptance"&&HEX64.test(value.nonce??"")&&value.repositoryRoot===repo&&value.root===root&&value.status==="PENDING"&&["PREPARING","READY"].includes(value.stage)&&value.listener?.host===HOST&&value.listener?.port===PORT&&value.listener?.state==="stopped"&&typeof value.createdAt==="string"&&validEntrypoint(value.entrypoint,repo)&&validDistManifest(value.distManifest,root)&&JSON.stringify(value.resources)===JSON.stringify([root,{kind:"fastify",host:HOST,port:PORT}]);const readyLog=value.backendLog?.path===join(root,"logs/backend.log")&&Number.isSafeInteger(value.backendLog?.dev)&&Number.isSafeInteger(value.backendLog?.ino);if(!baseValid||(value.stage==="READY"?!readyLog:value.backendLog!==null))throw new Error("manual ownership identity mismatch");return value;} async function run(executable,argv,options={}){return await exec(executable,argv,{...options,maxBuffer:2*1024*1024,encoding:"utf8"});} function descriptor(id,source){return{workspace:{schema_version:3,id,name:`P1 ${id}`,language:"en"},dwh:{engine:"postgres",database:"postgres",schema:"public",supported_transports:["postgres_direct"]},semantic_index:{vector_store:{engine:"qdrant",collection:id,dimensions:1024,distance:"cosine"},embedding:{provider:"ollama_internal",model:"qwen3-embedding:0.6b",dimensions:1024}},llm_policy:{allowed:["zai/glm-5.2"]},evidence:{source,policy:{max_chunk_chars:4000,retain_published_generations:3}}};} function descriptors(){return[descriptor("p1-filesystem",{type:"filesystem",uri:"workspace-content/p1-filesystem/evidence",patterns:["**/*.md"],max_bytes:10485760}),descriptor("p1-http",{type:"http",uris:["https://evidence.example.test/guide.md"],authentication:"signed_urls_file",connect_timeout_ms:1250,read_timeout_ms:30001,max_bytes:12345,max_redirects:2,allow_private_hosts:false,max_cache_bytes:67890}),descriptor("p1-s3",{type:"s3",uri:"s3://p1-evidence/published/",endpoint_url:"https://s3.example.test/",region:"eu-west-1",credentials:"static_files",trusted_endpoint:true,allow_private_endpoint:false,allow_insecure_endpoint:false,max_bytes:12345,max_objects:33,max_pages:4,page_size:5})];} @@ -108,29 +136,44 @@ set -a . ${quote(join(root,"installation","bindings.env"))} set +a node --input-type=module - "$root" ${quote(response)} ${quote(published)} ${quote(snapshots)} ${quote(checkout)} ${quote(output)} "$repo/backend/scripts/p1-render-snapshot.mjs" <<'NODE' +import { createHash } from "node:crypto"; import { readFile, realpath, stat } from "node:fs/promises"; -import { dirname, isAbsolute, relative, resolve, sep } from "node:path"; +import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import { spawnSync } from "node:child_process"; const [root,readPath,publishPath,snapshots,checkout,output,renderer]=process.argv.slice(2); -const bounded=async(path)=>{let s;try{s=await stat(path);}catch{throw new Error("saved response is missing or unbounded");}if(!s.isFile()||s.size<2||s.size>1048576)throw new Error("saved response is missing or unbounded");let v;try{v=JSON.parse(await readFile(path,"utf8"));}catch{throw new Error("saved response is malformed JSON");}return v;}; +const HEX40=/^[0-9a-f]{40}$/,HEX64=/^[0-9a-f]{64}$/; +const bounded=async(path,label="saved response")=>{let s;try{s=await stat(path);}catch{throw new Error(label+" is missing or unbounded");}if(!s.isFile()||s.size<2||s.size>1048576)throw new Error(label+" is missing or unbounded");let v;try{v=JSON.parse(await readFile(path,"utf8"));}catch{throw new Error(label+" is malformed JSON");}return v;}; +const boundedBytes=async(path)=>{let s;try{s=await stat(path);}catch{throw new Error("saved snapshot is missing or unbounded");}if(!s.isFile()||s.size<2||s.size>1048576)throw new Error("saved snapshot is missing or unbounded");return await readFile(path);}; const read=await bounded(readPath),published=await bounded(publishPath);const revision=read?.revision,commit=revision?.commit,snapshot=revision?.snapshotPath,publishedCommit=published?.head??published?.revision?.commit; -if(!/^[0-9a-f]{40}$/.test(commit??"")||commit!==publishedCommit)throw new Error("saved read/publish revisions differ"); +if(!HEX40.test(commit??"")||commit!==publishedCommit)throw new Error("saved read/publish revisions differ"); if(typeof snapshot!=="string"||!isAbsolute(snapshot))throw new Error("snapshot path is not absolute");const canonical=await realpath(snapshot);const rel=relative(snapshots,canonical);if(rel.startsWith("..")||isAbsolute(rel)||dirname(canonical)!==resolve(snapshots,commit))throw new Error("snapshot escapes owned commit root"); +const id=basename(canonical).slice(0,-".yaml".length);if(!/^[a-z][a-z0-9-]{2,62}$/.test(id))throw new Error("snapshot workspace identity is invalid"); const git=spawnSync("git",["-C",checkout,"rev-parse","HEAD"],{encoding:"utf8"});if(git.status!==0||git.stdout.trim()!==commit)throw new Error("saved revision differs from installed Git commit"); -const child=spawnSync(process.execPath,[renderer,"--ownership",resolve(root,"ownership.json"),"--snapshot",canonical,"--output",output],{stdio:"inherit",env:process.env});if(child.status!==0)process.exit(child.status??1); +const manifest=await bounded(join(snapshots,commit,"snapshot.json"),"snapshot manifest");const files=manifest?.files,revisions=manifest?.revisions; +if(manifest?.head!==commit||!files||typeof files!=="object"||Array.isArray(files))throw new Error("snapshot manifest identity is invalid"); +const expected=files[id+".yaml"];if(!HEX64.test(expected??""))throw new Error("snapshot manifest digest is invalid"); +const snapshotBytes=await boundedBytes(canonical);if(createHash("sha256").update(snapshotBytes).digest("hex")!==expected)throw new Error("snapshot bytes differ from manifest digest"); +const entry=Array.isArray(revisions)?revisions.find(candidate=>candidate?.id===id):undefined; +if(!entry||!HEX40.test(entry?.blob??"")||entry.commit!==commit||typeof entry.snapshotPath!=="string"||resolve(entry.snapshotPath)!==canonical||(entry.state!=="operational"&&entry.state!=="migration_required"))throw new Error("snapshot manifest revision is invalid"); +if(!HEX40.test(revision?.blob??"")||revision.blob!==entry.blob)throw new Error("saved revision blob differs from snapshot manifest"); +const blobCheck=spawnSync("git",["-C",checkout,"rev-parse",commit+":workspaces/"+id+".yaml"],{encoding:"utf8"}); +if(blobCheck.status!==0||blobCheck.stdout.trim()!==entry.blob)throw new Error("snapshot blob differs from installed Git commit"); +const hashObject=spawnSync("git",["hash-object","--stdin"],{input:snapshotBytes,encoding:"utf8"}); +if(hashObject.status!==0||hashObject.stdout.trim()!==entry.blob)throw new Error("snapshot bytes differ from Git blob"); +const child=spawnSync(process.execPath,[renderer,"--ownership",resolve(root,"ownership.json"),"--snapshot",canonical,"--output",output,"--snapshot-sha256",expected],{stdio:"inherit",env:process.env});if(child.status!==0)process.exit(child.status??1); NODE `;} function guide(repo,root){const base=`http://${HOST}:${PORT}`;return `# P1 manual configuration walkthrough Status: **PENDING**. The reviewer, not this helper, performs and judges every step. Never inspect raw secret-file contents. Every lifecycle action uses the stable repository-root \`.p1-manual-acceptance.lifecycle.lock\`; successful prepare has advanced its ownership-first recovery record from \`PREPARING\` to \`READY\`. -1. Inspect \`${root}/ownership.json\`, including the bound production entrypoint identity, the pre-publication Evidence tree under \`author/workspace-content/p1-filesystem/evidence\`, descriptor fixtures, and binding **paths and modes** in \`installation/bindings.env\`. -2. Run \`${repo}/scripts/p1-manual-acceptance.sh serve\`; verify one production Node PID owns both \`${HOST}:${PORT}\` and its authenticated \`${HOST}:${CONTROL_PORT}\` control listener (for example, use \`lsof -nP -iTCP:${PORT} -sTCP:LISTEN\` and repeat for port ${CONTROL_PORT}). Serve executes the ownership-bound production bytes from an opened no-follow descriptor and publishes \`RUNNING\` only after the same authenticated child acknowledges its owned HTTP listener and passes bounded health checks. +1. Inspect \`${root}/ownership.json\`, including the bound production entrypoint identity and the complete \`backend/dist\` module manifest identity, the pre-publication Evidence tree under \`author/workspace-content/p1-filesystem/evidence\`, descriptor fixtures, and binding **paths and modes** in \`installation/bindings.env\`. +2. Run \`${repo}/scripts/p1-manual-acceptance.sh serve\`; verify one production Node PID owns both \`${HOST}:${PORT}\` and its authenticated \`${HOST}:${CONTROL_PORT}\` control listener (for example, use \`lsof -nP -iTCP:${PORT} -sTCP:LISTEN\` and repeat for port ${CONTROL_PORT}). Serve executes the ownership-bound production entrypoint and complete verified \`backend/dist\` module graph from opened no-follow descriptors and publishes \`RUNNING\` only after the same authenticated child acknowledges its owned HTTP listener and passes bounded health checks. 3. Personally run each concrete \`commands/http-01-*.sh\` through \`commands/http-14-*.sh\` script, one at a time in numeric order: real curl status → three validates → three sequential publishes → pull → three reads → three exports against \`${base}\`. Each script saves the exact JSON response under \`responses/\` or ZIP bytes under \`exports/raw/\`; each publish derives its current base from the preceding bounded saved response. Do not advance on a non-2xx response. 4. Only after publish, run \`commands/git-inspect.sh \`: inspect \`git log\`, \`git ls-tree\`, \`git show :workspaces/.yaml\`, and \`git show :workspace-content//evidence/...\` at that same commit. 5. Inspect generated \`workspace-docs\`, the immutable commit-addressed descriptor snapshot, and its \`snapshot.json\` manifest. 6. Run \`commands/extract-export.sh exports/raw/p1-filesystem.zip exports/extracted/p1-filesystem p1-filesystem\`, then the equivalent exact commands for \`p1-http\` and \`p1-s3\`; verify each manifest and descriptor identity, hashes, and absence of Evidence bytes and secret/canary material. -7. After saving \`responses/read-p1-filesystem.json\` and the final API/Git head in \`responses/pull.json\`, run \`commands/render-1.sh\`, \`commands/render-2.sh\`, then \`commands/diff-rendered.sh\`. The renderer publishes through one opened no-follow \`rendered\` directory identity and refuses an ancestor swap. +7. After saving \`responses/read-p1-filesystem.json\` and the final API/Git head in \`responses/pull.json\`, run \`commands/render-1.sh\`, \`commands/render-2.sh\`, then \`commands/diff-rendered.sh\`. The render commands bind the snapshot bytes to the commit\'s \`snapshot.json\` digest and Git blob identity; the renderer revalidates that digest and renders only the verified bytes through one opened no-follow \`rendered\` directory identity, refusing an ancestor swap. 8. Inspect runtime identity, absolute reserved filesystem root, Evidence limits, and policy in the rendered YAML; do not inspect secret contents. 9. Personally execute \`${repo}/harness/.venv/bin/tht config check -c ${root}/rendered/runtime-1.yaml\` and the same command for \`runtime-2.yaml\` (or run \`commands/config-check.sh\`). 10. Personally run \`commands/http-15-*.sh\` through \`commands/http-19-*.sh\` to submit the invalid absolute, Evidence-URI traversal, cross-workspace, protocol, and credential validation requests; verify safe rejection, no Git/snapshot mutation, and no rejected canary outside the request fixture. @@ -418,17 +461,18 @@ export async function prepareManual(options={}){ const{repositoryRoot=defaultRepositoryRoot,skipBuild=false}=options,repo=realpathSync(repositoryRoot),root=fixedManualRoot(repo),lifecycle=await acquireLifecycle(repo,"prepare");let entryBinding,ownershipCreated=false; try{ await requireLifecycleContext(lifecycle);await checkPrerequisites(repo);if(!skipBuild)await run("npm",["--prefix",join(repo,"backend"),"run","build"]);await requireLifecycleContext(lifecycle); - entryBinding=await readBoundEntrypoint(repo);const entrypoint=entryBinding.identity;await entryBinding.handle.close();entryBinding=undefined;const distribution=await distManifest(repo); + entryBinding=await readBoundEntrypoint(repo);const entrypoint=entryBinding.identity;await entryBinding.handle.close();entryBinding=undefined; noSymlinkExisting(repo,root);try{await mkdir(root,{recursive:false,mode:0o700});}catch(error){if(error.code==="EEXIST")throw new Error("manual acceptance root already exists; stop/cleanup it explicitly");throw error;}await bindLifecycleRoot(lifecycle,root); - const nonce=randomBytes(32).toString("hex"),createdAt=new Date().toISOString();await exclusiveRecord(join(root,"ownership.json"),ownedValue(repo,root,nonce,{entrypoint,distManifest:distribution,createdAt}),"manual ownership");ownershipCreated=true;await requireLifecycleContext(lifecycle,{root:true}); for(const path of ["installation/registry","installation/data","installation/runtime","fixture-secrets","fixtures/descriptors","requests","responses","exports/raw","exports/extracted","rendered","logs","commands"]){await mkdir(join(root,path),{recursive:true,mode:path==="fixture-secrets"?0o700:0o755});await requireLifecycleContext(lifecycle,{root:true});} + const distManifestValue=await buildDistManifest(repo),distManifestRecord=await exclusiveRecord(join(root,"installation/runtime/backend-dist.manifest.json"),distManifestValue,"production distribution manifest"),distManifest={path:distManifestRecord.path,dev:distManifestRecord.dev,ino:distManifestRecord.ino,size:distManifestRecord.bytes.length,sha256:createHash("sha256").update(distManifestRecord.bytes).digest("hex")};await requireLifecycleContext(lifecycle,{root:true}); + const nonce=randomBytes(32).toString("hex"),createdAt=new Date().toISOString();await exclusiveRecord(join(root,"ownership.json"),ownedValue(repo,root,nonce,{entrypoint,distManifest,createdAt}),"manual ownership");ownershipCreated=true;await requireLifecycleContext(lifecycle,{root:true}); const backendLogPath=join(root,"logs/backend.log"),backendLogHandle=await open(backendLogPath,"wx",0o600);let backendLogEntry;try{await backendLogHandle.chmod(0o600);await backendLogHandle.sync();backendLogEntry=await backendLogHandle.stat();}finally{await backendLogHandle.close();}directorySync(dirname(backendLogPath));const backendLog={path:backendLogPath,dev:backendLogEntry.dev,ino:backendLogEntry.ino}; await requireLifecycleContext(lifecycle,{root:true}); try{await initializeGit(root);}catch(error){try{await requireLifecycleContext(lifecycle,{root:true});}catch{await cleanupFailedPrepare(repo,lifecycle);throw new Error("manual acceptance parent or root identity changed during prepare");}throw error;}await requireLifecycleContext(lifecycle,{root:true}); const items=descriptors();for(const workspace of items)await atomicWrite(join(root,"fixtures/descriptors",`${workspace.workspace.id}.json`),`${JSON.stringify(workspace,null,2)}\n`); const secrets={"dwh-password":`DWH-${randomBytes(16).toString("hex")}`,"evidence-signed-urls.json":JSON.stringify([`https://evidence.example.test/guide.md?token=SIGNED-${randomBytes(16).toString("hex")}`]),"evidence-access":`ACCESS-${randomBytes(16).toString("hex")}`,"evidence-secret":`SECRET-${randomBytes(16).toString("hex")}`,"evidence-session":`SESSION-${randomBytes(16).toString("hex")}`};for(const[name,value]of Object.entries(secrets))await atomicWrite(join(root,"fixture-secrets",name),value,0o600); const env={};for(const workspace of items){const ns=workspace.workspace.id.toUpperCase().replaceAll("-","_"),prefix=`THT_WS_${ns}`;Object.assign(env,{[`${prefix}_DWH_TRANSPORT`]:"postgres_direct",[`${prefix}_DWH_HOST`]:"dwh.invalid",[`${prefix}_DWH_PORT`]:"5432",[`${prefix}_DWH_USER`]:"reader",[`${prefix}_DWH_PASSWORD_FILE`]:join(root,"fixture-secrets/dwh-password")});}Object.assign(env,{THT_WORKSPACE_SECRET_ROOTS:join(root,"fixture-secrets"),THT_WS_P1_HTTP_EVIDENCE_SIGNED_URLS_FILE:join(root,"fixture-secrets/evidence-signed-urls.json"),THT_WS_P1_S3_EVIDENCE_ACCESS_KEY_FILE:join(root,"fixture-secrets/evidence-access"),THT_WS_P1_S3_EVIDENCE_SECRET_KEY_FILE:join(root,"fixture-secrets/evidence-secret"),THT_WS_P1_S3_EVIDENCE_SESSION_TOKEN_FILE:join(root,"fixture-secrets/evidence-session")}); - await atomicWrite(join(root,"installation/bindings.env"),Object.entries(env).map(([k,v])=>`${k}=${quote(v)}`).join("\n")+"\n");await atomicWrite(join(root,"installation/base.yaml"),"{}\n");for(const[name,value]of Object.entries(requestFixtures(items)))await atomicWrite(join(root,"requests",name),`${JSON.stringify(value,null,2)}\n`);await writeCommands(repo,root);await atomicWrite(join(root,"GUIDE.md"),guide(repo,root),0o600);await requireLifecycleContext(lifecycle,{root:true});await atomicWrite(join(root,"ownership.json"),`${JSON.stringify(ownedValue(repo,root,nonce,{backendLog,entrypoint,distManifest:distribution,stage:"READY",createdAt}),null,2)}\n`);await requireLifecycleContext(lifecycle,{root:true});return{repositoryRoot:repo,root,nonce}; + await atomicWrite(join(root,"installation/bindings.env"),Object.entries(env).map(([k,v])=>`${k}=${quote(v)}`).join("\n")+"\n");await atomicWrite(join(root,"installation/base.yaml"),"{}\n");for(const[name,value]of Object.entries(requestFixtures(items)))await atomicWrite(join(root,"requests",name),`${JSON.stringify(value,null,2)}\n`);await writeCommands(repo,root);await atomicWrite(join(root,"GUIDE.md"),guide(repo,root),0o600);await requireLifecycleContext(lifecycle,{root:true});await atomicWrite(join(root,"ownership.json"),`${JSON.stringify(ownedValue(repo,root,nonce,{backendLog,entrypoint,distManifest,stage:"READY",createdAt}),null,2)}\n`);await requireLifecycleContext(lifecycle,{root:true});return{repositoryRoot:repo,root,nonce}; }catch(error){ if(entryBinding)await entryBinding.handle.close().catch(()=>{}); if(ownershipCreated){try{await requireLifecycleContext(lifecycle,{root:true});}catch{await cleanupFailedPrepare(repo,lifecycle).catch(()=>{});throw new Error("manual acceptance parent or root identity changed during prepare");}} @@ -438,11 +482,6 @@ export async function prepareManual(options={}){ function portAvailable(port,label=`${HOST}:${port}`){return new Promise((resolvePromise,reject)=>{const server=net.createServer();server.once("error",error=>error.code==="EADDRINUSE"?reject(new Error(`${label} is occupied`)):reject(error));server.listen({host:HOST,port,exclusive:true},()=>server.close(()=>resolvePromise()));});} async function requireCanonicalDirectory(path,label){const entry=await lstat(path);if(!entry.isDirectory()||entry.isSymbolicLink()||await realpath(path)!==path)throw new Error(`${label} directory identity is unsafe`);return entry;} async function requireAbsent(path,label){try{await lstat(path);throw new Error(`${label} is legacy or unsafe`);}catch(error){if(error.code!=="ENOENT")throw error;}} -async function validateDistIntegrity(repo,owned){ - if(!owned.distManifest?.sha256||owned.distManifest.root!==join(repo,"backend","dist"))throw new Error("production distribution manifest is missing"); - const current=await distManifest(repo); if(current.sha256!==owned.distManifest.sha256||JSON.stringify(current.files)!==JSON.stringify(owned.distManifest.files))throw new Error("production distribution identity changed"); - return current; -} async function validateServeFilesystem(repo,root,owned){ if(root!==fixedManualRoot(repo))throw new Error("owned root identity is unsafe"); for(const [path,label] of [ @@ -453,7 +492,7 @@ async function validateServeFilesystem(repo,root,owned){ ])await requireCanonicalDirectory(path,label); await requireAbsent(legacySupervisorPath(root),"legacy supervisor"); if(owned.stage!=="READY")throw new Error("manual acceptance preparation is incomplete"); - const script=join(repo,"backend/dist/server.js");await requireEntrypointPathIdentity(owned.entrypoint);await validateDistIntegrity(repo,owned); + const script=join(repo,"backend/dist/server.js");await requireEntrypointPathIdentity(owned.entrypoint);const manifestRecord=await readBoundDistManifest(repo,owned);try{await validateDistFiles(repo,manifestRecord.files);}finally{await manifestRecord.handle.close();} const logPath=join(root,"logs/backend.log"); if(owned.backendLog?.path!==logPath)throw new Error("backend log ownership identity is unsafe"); return{script,logPath}; @@ -478,7 +517,7 @@ async function readPid(root){const path=join(root,"backend.pid"),entry=await lst async function validateProcess(repo,root,owned,pidRecord){ const script=join(repo,"backend/dist/server.js"),entrypoint=owned.entrypoint; if(pidRecord.schemaVersion!==1||pidRecord.kind!=="p1-manual-backend"||pidRecord.status!=="RUNNING"||!Number.isSafeInteger(pidRecord.pid)||pidRecord.pid<2||!HEX64.test(pidRecord.reservationNonce??"")||pidRecord.nonce!==owned.nonce||pidRecord.root!==root||pidRecord.repositoryRoot!==repo||pidRecord.executable!==process.execPath||pidRecord.preload!==PRELOAD||pidRecord.script!==script||JSON.stringify(pidRecord.entrypoint)!==JSON.stringify(entrypoint)||!pidRecord.startIdentity||pidRecord.control?.host!==HOST||pidRecord.control?.port!==CONTROL_PORT)throw new Error("backend process identity mismatch; refusing cooperative control"); - await requireEntrypointPathIdentity(entrypoint);await validateDistIntegrity(repo,owned);if(!alive(pidRecord.pid))throw new Error("backend PID is stale; operator inspection required"); + await requireEntrypointPathIdentity(entrypoint);const manifestRecord=await readBoundDistManifest(repo,owned);try{await validateDistFiles(repo,manifestRecord.files);}finally{await manifestRecord.handle.close();}if(!alive(pidRecord.pid))throw new Error("backend PID is stale; operator inspection required"); const[start,args,cwd,executable]=await Promise.all([processStart(pidRecord.pid),processArgs(pidRecord.pid),processCwd(pidRecord.pid),processExecutable(pidRecord.pid)]); const expectedArgs=[pidRecord.executable,"--import",pidRecord.preload,pidRecord.script,`--p1-manual-nonce=${owned.nonce}`,`--p1-root=${root}`,`--p1-control-nonce=${pidRecord.reservationNonce}`,`--p1-entry-sha256=${entrypoint.sha256}`,`--p1-entry-dev=${entrypoint.dev}`,`--p1-entry-ino=${entrypoint.ino}`].join(" "); if(start!==pidRecord.startIdentity||cwd!==repo||executable!==realpathSync(pidRecord.executable)||args!==expectedArgs)throw new Error("backend process identity mismatch; refusing cooperative control");return true; @@ -488,12 +527,12 @@ async function healthStatus(){return await new Promise((resolvePromise,reject)=> function exactControlIdentity(answer,child,owned,root,reservationNonce){return answer?.pid===child.pid&&answer?.nonce===owned.nonce&&answer?.controlNonce===reservationNonce&&answer?.root===root&&answer?.control?.host===HOST&&answer?.control?.port===CONTROL_PORT;} function exactOwnedListener(answer,generation){return answer?.listener?.listening===true&&answer.listener.host===HOST&&answer.listener.port===PORT&&Number.isSafeInteger(answer.listener.generation)&&answer.listener.generation>0&&(generation===undefined||answer.listener.generation===generation);} export async function serveManual({repositoryRoot=defaultRepositoryRoot,beforeSpawn}={}){ - const repo=realpathSync(repositoryRoot),lifecycle=await acquireLifecycle(repo,"serve");let pidRecord,child,controlObserved=false,logFd,entryBinding; + const repo=realpathSync(repositoryRoot),lifecycle=await acquireLifecycle(repo,"serve");let pidRecord,child,controlObserved=false,logFd,entryBinding,manifestBinding; try{ const owned=await readManualOwnership({repositoryRoot:repo}),root=owned.root;await bindLifecycleRoot(lifecycle,root); if(owned.listener.host!==HOST||owned.listener.port!==PORT)throw new Error("non-loopback or unexpected bind refused"); const{script,logPath}=await validateServeFilesystem(repo,root,owned);await requireLifecycleContext(lifecycle,{root:true}); - logFd=openOwnedBackendLog(owned,logPath);entryBinding=await readBoundEntrypoint(repo);if(JSON.stringify(entryBinding.identity)!==JSON.stringify(owned.entrypoint))throw new Error("production entrypoint identity changed"); + logFd=openOwnedBackendLog(owned,logPath);entryBinding=await readBoundEntrypoint(repo);if(JSON.stringify(entryBinding.identity)!==JSON.stringify(owned.entrypoint))throw new Error("production entrypoint identity changed");manifestBinding=await readBoundDistManifest(repo,owned); const reservationNonce=randomBytes(32).toString("hex");pidRecord=await exclusiveRecord(join(root,"backend.pid"),{schemaVersion:1,kind:"p1-manual-backend-reservation",status:"RESERVED",reservationNonce,nonce:owned.nonce,root,repositoryRoot:repo},"backend PID record"); await Promise.all([portAvailable(PORT),portAvailable(CONTROL_PORT,`${HOST}:${CONTROL_PORT} control port`)]);await requireLifecycleContext(lifecycle,{root:true}); await ensureRuntimeDirectory(join(root,"installation/runtime/home"));await ensureRuntimeDirectory(join(root,"installation/runtime/tmp"));await ensureRuntimeDirectory(join(root,"installation/runtime/tht-home"));await requireLifecycleContext(lifecycle,{root:true}); @@ -501,8 +540,8 @@ export async function serveManual({repositoryRoot=defaultRepositoryRoot,beforeSp const env={...inherited,HOME:join(root,"installation/runtime/home"),TMPDIR:join(root,"installation/runtime/tmp"),HOST,PORT:String(PORT),AUTH_MODE:"none",THT_BIN:join(repo,"harness/.venv/bin/tht"),THT_HARNESS_DIR:join(repo,"harness"),THT_DATA_ROOT:join(root,"installation/data"),SETTINGS_FILE:join(root,"installation/data/settings.json"),MAINTENANCE_STATE_FILE:join(root,"installation/data/maintenance.json"),THT_WORKSPACE_REGISTRY_ROOT:join(root,"installation/registry"),THT_WORKSPACE_GIT_REMOTE:join(root,"remote.git"),THT_WORKSPACE_GIT_BRANCH:"main",THT_WORKSPACE_GIT_AUTHOR_NAME:"P1 Manual API Publisher",THT_WORKSPACE_GIT_AUTHOR_EMAIL:"p1-manual-api@example.invalid",THT_WORKSPACE_INSTALLATION_ID:"p1-manual-acceptance",THT_WORKSPACE_SECRET_ROOTS:join(root,"fixture-secrets"),THT_HOME:join(root,"installation/runtime/tht-home")}; if(beforeSpawn)await beforeSpawn({script,entrypoint:{...owned.entrypoint}});await requireLifecycleContext(lifecycle,{root:true}); const entryArgs=[`--p1-entry-sha256=${owned.entrypoint.sha256}`,`--p1-entry-dev=${owned.entrypoint.dev}`,`--p1-entry-ino=${owned.entrypoint.ino}`]; - child=spawn(process.execPath,["--import",PRELOAD,script,`--p1-manual-nonce=${owned.nonce}`,`--p1-root=${root}`,`--p1-control-nonce=${reservationNonce}`,...entryArgs],{cwd:repo,env,detached:true,stdio:["ignore",logFd,logFd,entryBinding.handle.fd]}); - await entryBinding.handle.close();entryBinding=undefined;closeSync(logFd);logFd=undefined; + child=spawn(process.execPath,["--import",PRELOAD,script,`--p1-manual-nonce=${owned.nonce}`,`--p1-root=${root}`,`--p1-control-nonce=${reservationNonce}`,...entryArgs],{cwd:repo,env,detached:true,stdio:["ignore",logFd,logFd,entryBinding.handle.fd,manifestBinding.handle.fd]}); + await entryBinding.handle.close();entryBinding=undefined;await manifestBinding.handle.close();manifestBinding=undefined;closeSync(logFd);logFd=undefined; let start="";for(let n=0;n<80;n++){if(child.exitCode!==null)break;try{start=await processStart(child.pid);if(start)break;}catch{}await new Promise(r=>setTimeout(r,25));} if(!start)throw new Error("backend failed before process identity could be recorded");await requireLifecycleContext(lifecycle,{root:true}); pidRecord=await replaceExactRecord(pidRecord,{schemaVersion:1,kind:"p1-manual-backend-reservation",status:"STARTING",pid:child.pid,reservationNonce,nonce:owned.nonce,root,repositoryRoot:repo,executable:process.execPath,preload:PRELOAD,script,entrypoint:owned.entrypoint,startIdentity:start,control:{host:HOST,port:CONTROL_PORT}}); @@ -523,10 +562,10 @@ export async function serveManual({repositoryRoot=defaultRepositoryRoot,beforeSp const publishedStatus=await controlRequest(runningValue.control,{action:"status",nonce:reservationNonce});if(!exactControlIdentity(publishedStatus,child,owned,root,reservationNonce)||publishedStatus.status!=="READY"||!exactOwnedListener(publishedStatus,listenerGeneration)){await removeExactRecord(pidRecord);throw new Error("backend listener changed during RUNNING publication");} child.unref();return child.pid; }catch(error){ - if(entryBinding)await entryBinding.handle.close().catch(()=>{});if(logFd!==undefined){closeSync(logFd);logFd=undefined;} + if(entryBinding)await entryBinding.handle.close().catch(()=>{});if(manifestBinding)await manifestBinding.handle.close().catch(()=>{});if(logFd!==undefined){closeSync(logFd);logFd=undefined;} if(child&&controlObserved){try{const value=pidRecord?JSON.parse(pidRecord.bytes):undefined;await controlRequest({host:HOST,port:CONTROL_PORT},{action:"stop",nonce:value?.reservationNonce});}catch{}await waitForChildExit(child,3000);}else if(child)await waitForChildExit(child,8500); if(pidRecord&&(!child||child.exitCode!==null||child.signalCode!==null||!alive(child.pid)))await removeExactRecord(pidRecord).catch(()=>{});throw error; - }finally{if(entryBinding)await entryBinding.handle.close().catch(()=>{});if(logFd!==undefined)closeSync(logFd);await removeExactRecord(lifecycle);} + }finally{if(entryBinding)await entryBinding.handle.close().catch(()=>{});if(manifestBinding)await manifestBinding.handle.close().catch(()=>{});if(logFd!==undefined)closeSync(logFd);await removeExactRecord(lifecycle);} } export async function stopManual({repositoryRoot=defaultRepositoryRoot}={}){const repo=realpathSync(repositoryRoot),lifecycle=await acquireLifecycle(repo,"stop");try{const owned=await readManualOwnership({repositoryRoot:repo}),root=owned.root;if(owned.stage!=="READY")throw new Error("owned backend was never prepared");await bindLifecycleRoot(lifecycle,root);let record;try{record=await readPid(root);}catch(error){if(error.code==="ENOENT")throw new Error("owned backend is not served");throw error;}await requireLifecycleContext(lifecycle,{root:true});await validateProcess(repo,root,owned,record.value);const answer=await controlRequest(record.value.control,{action:"stop",nonce:record.value.reservationNonce});if(answer.status!=="STOPPING"||answer.pid!==record.value.pid||answer.nonce!==owned.nonce||answer.controlNonce!==record.value.reservationNonce||answer.root!==root||answer.control?.host!==HOST||answer.control?.port!==CONTROL_PORT)throw new Error("backend cooperative stop acknowledgement mismatch; PID record retained");for(let n=0;n<100;n++){if(!alive(record.value.pid)){await requireLifecycleContext(lifecycle,{root:true});await removeExactRecord(record);await requireLifecycleContext(lifecycle,{root:true});return;}await new Promise(r=>setTimeout(r,100));}throw new Error("owned backend did not stop cooperatively; operator must intervene; PID record retained");}finally{await removeExactRecord(lifecycle);}} const ANCHORED_REMOVE_SOURCE=String.raw`import os,stat,sys diff --git a/backend/scripts/p1-manual-acceptance.test.mjs b/backend/scripts/p1-manual-acceptance.test.mjs index c7d438af..5079fe27 100644 --- a/backend/scripts/p1-manual-acceptance.test.mjs +++ b/backend/scripts/p1-manual-acceptance.test.mjs @@ -402,6 +402,64 @@ test("generated render command validates saved responses and owned snapshot befo await assert.rejects(lstat(output)); }); +const renderSnapshotYaml=`workspace: + schema_version: 3 + id: p1-filesystem + name: P1 filesystem + language: en +dwh: + engine: postgres + database: postgres + schema: public + supported_transports: [postgres_direct] +semantic_index: + vector_store: {engine: qdrant, collection: p1-filesystem, dimensions: 1024, distance: cosine} + embedding: {provider: ollama_internal, model: qwen3-embedding:0.6b, dimensions: 1024} +llm_policy: + allowed: [zai/glm-5.2] +evidence: + source: {type: filesystem, uri: workspace-content/p1-filesystem/evidence, patterns: ["**/*.md"], max_bytes: 10485760} + policy: {max_chunk_chars: 4000, retain_published_generations: 3} +`; + +test("generated render command binds snapshot bytes to the commit manifest and Git blob end to end", async () => { + const repo=await fakeRepo(); const run=await prepareManual({repositoryRoot:repo,skipBuild:true}); + const author=join(run.root,"author"); await mkdir(join(author,"workspaces"),{recursive:true}); + await writeFile(join(author,"workspaces","p1-filesystem.yaml"),renderSnapshotYaml); + await execFileAsync("git",["add","workspaces"],{cwd:author}); await execFileAsync("git",["commit","-m","publish p1"],{cwd:author}); await execFileAsync("git",["push","origin","main"],{cwd:author}); + const commit=(await execFileAsync("git",["rev-parse","HEAD"],{cwd:author})).stdout.trim(); + const blob=(await execFileAsync("git",["rev-parse","HEAD:workspaces/p1-filesystem.yaml"],{cwd:author})).stdout.trim(); + await execFileAsync("git",["clone",join(run.root,"remote.git"),join(run.root,"installation/registry/repo")]); + const commitDir=join(run.root,"installation/registry/snapshots",commit); await mkdir(commitDir,{recursive:true}); + const snapshot=join(commitDir,"p1-filesystem.yaml"),snapshotPath=snapshot,snapshotSha=sha256(renderSnapshotYaml); + await writeFile(snapshot,renderSnapshotYaml); + const readPath=join(run.root,"responses/read-p1-filesystem.json"),pullPath=join(run.root,"responses/pull.json"),script=join(run.root,"commands/render-1.sh"),script2=join(run.root,"commands/render-2.sh"),output=join(run.root,"rendered/runtime-1.yaml"),output2=join(run.root,"rendered/runtime-2.yaml"); + const rendererStub=join(repo,"backend/scripts/p1-render-snapshot.mjs"),stubArgs=join(run.root,"rendered/stub-args.json"); + await writeFile(rendererStub,`import { writeFileSync } from "node:fs";\nwriteFileSync(${JSON.stringify(stubArgs)}, JSON.stringify(process.argv.slice(2)));\n`); + const manifest=()=>({head:commit,revisions:[{id:"p1-filesystem",commit,blob,snapshotPath,state:"operational"}],files:{"p1-filesystem.yaml":snapshotSha}}); + await writeFile(readPath,JSON.stringify({revision:{id:"p1-filesystem",commit,blob,snapshotPath,state:"operational"}})); await writeFile(pullPath,JSON.stringify({head:commit})); + await assert.rejects(execFileAsync("bash",[script],{cwd:repo}),/snapshot manifest.*(missing|unbounded)/i); + await assert.rejects(lstat(output)); + await writeFile(join(commitDir,"snapshot.json"),JSON.stringify(manifest())); + await execFileAsync("bash",[script],{cwd:repo}); + assert.deepEqual(JSON.parse(await readFile(stubArgs,"utf8")),["--ownership",join(run.root,"ownership.json"),"--snapshot",snapshot,"--output",output,"--snapshot-sha256",snapshotSha]); + await writeFile(snapshot,renderSnapshotYaml.replace("max_chunk_chars: 4000","max_chunk_chars: 3999")); + await assert.rejects(execFileAsync("bash",[script2],{cwd:repo}),/snapshot bytes differ from manifest digest/); + await assert.rejects(lstat(output2)); + await writeFile(snapshot,renderSnapshotYaml); + await writeFile(join(commitDir,"snapshot.json"),JSON.stringify({...manifest(),head:"c".repeat(40)})); + await assert.rejects(execFileAsync("bash",[script2],{cwd:repo}),/snapshot manifest identity is invalid/); + await assert.rejects(lstat(output2)); + await writeFile(join(commitDir,"snapshot.json"),JSON.stringify(manifest())); + await writeFile(join(commitDir,"snapshot.json"),JSON.stringify({...manifest(),revisions:[{id:"p1-filesystem",commit,blob:"f".repeat(40),snapshotPath,state:"operational"}]})); + await assert.rejects(execFileAsync("bash",[script2],{cwd:repo}),/saved revision blob differs from snapshot manifest/); + await assert.rejects(lstat(output2)); + await writeFile(join(commitDir,"snapshot.json"),JSON.stringify(manifest())); + await writeFile(readPath,JSON.stringify({revision:{id:"p1-filesystem",commit,blob:"f".repeat(40),snapshotPath,state:"operational"}})); + await assert.rejects(execFileAsync("bash",[script2],{cwd:repo}),/saved revision blob differs/); + await assert.rejects(lstat(output2)); +}); + const sha256=bytes=>createHash("sha256").update(bytes).digest("hex"); async function makeExportZip(directory,name,{payloads={},manifest,workspaceId="p1-filesystem",extra=false,symlinkReadme=false}={}) { @@ -617,12 +675,44 @@ test("serve refuses a replaced backend dist dependency after prepare", { concurr await assert.rejects(lstat(marker)); await assert.rejects(lstat(join(run.root,"backend.pid"))); }); +test("serve refuses a deterministic dependency check/load swap before execution", { concurrency: false }, async () => { + const repo=await fakeRepo(),marker=join(repo,"dep-swap-executed"); + await writeFile(join(repo,"backend/dist/dep.js"),`export function start(){}\n`); + await writeFile(join(repo,"backend/dist/server.js"),`import http from "node:http";\nimport { start } from "./dep.js";\nstart();\nconst server=http.createServer((req,res)=>{res.statusCode=req.url==="/health"?200:200;res.setHeader("content-type","application/json");res.end(JSON.stringify({status:"ok"}));});\nserver.listen(Number(process.env.PORT),process.env.HOST);\n`); + const run=await prepareManual({repositoryRoot:repo,skipBuild:true}),owned=await readManualOwnership({repositoryRoot:repo}),replacement=join(repo,"dep-replacement.js"); + await writeFile(replacement,`import {writeFileSync} from "node:fs";writeFileSync(${JSON.stringify(marker)},"executed");export function start(){}\n`); + await assert.rejects(serveManual({repositoryRoot:repo,beforeSpawn:async()=>rename(replacement,join(repo,"backend/dist/dep.js"))}),/identity|changed|refused|distribution|module|readiness|failed/i); + await assert.rejects(lstat(marker)); await assert.rejects(lstat(join(run.root,"backend.pid"))); + assert.deepEqual(await matchingManualServerPids(run.root,owned.nonce),[]); assert.deepEqual(await listenerPids(),[]); +}); + +test("immutable loader serves verified cached dependency bytes after a same-path regular replacement", { concurrency: false }, async () => { + const repo=await fakeRepo(),marker=join(repo,"dep-replacement-executed"); + await writeFile(join(repo,"backend/dist/dep.js"),`export function mark(){ globalThis.__depSource = "original"; }\n`); + await writeFile(join(repo,"backend/dist/server.js"),`import http from "node:http";\nlet n = 0;\nconst server=http.createServer(async (req,res)=>{ if(req.url==="/load"){ await import(\`./dep.js?v=\${++n}\`).then(m=>m.mark()); } res.setHeader("content-type","application/json"); res.end(JSON.stringify({status:"ok",dep:globalThis.__depSource??"unset"})); });\nserver.listen(Number(process.env.PORT),process.env.HOST);\n`); + const run=await prepareManual({repositoryRoot:repo,skipBuild:true}),owned=await readManualOwnership({repositoryRoot:repo}); + const pid=await serveManual({repositoryRoot:repo}); + try { + const health=async()=>(await (await fetch("http://127.0.0.1:8791/health")).json()); + const load=async()=>{ await fetch("http://127.0.0.1:8791/load"); return (await health()).dep; }; + for(let n=0;n<60;n++){try{if((await health()).status==="ok")break;}catch{}await new Promise(r=>setTimeout(r,50));} + assert.equal(await load(),"original"); + await writeFile(join(repo,"backend/dist/dep.js"),`import {writeFileSync} from "node:fs";writeFileSync(${JSON.stringify(marker)},"executed");export function mark(){ globalThis.__depSource = "replaced"; }\n`); + assert.equal(await load(),"original"); await assert.rejects(lstat(marker)); + await writeFile(join(repo,"backend/dist/dep.js"),`export function mark(){ globalThis.__depSource = "original"; }\n`); + } finally { + try { await stopManual({repositoryRoot:repo}); } catch { try { process.kill(pid,"SIGTERM"); } catch {} } + } + await cleanupManual({repositoryRoot:repo}); +}); + test("opened production FD prevents deterministic check-spawn replacement execution", { concurrency: false }, async () => { const repo=await fakeRepo(),safe=join(repo,"safe-executed"),malicious=join(repo,"malicious-executed"); await installFakeServer(repo,{marker:safe}); - const run=await prepareManual({repositoryRoot:repo,skipBuild:true}),script=join(repo,"backend/dist/server.js"),replacement=join(repo,"replacement-server.js"); + const run=await prepareManual({repositoryRoot:repo,skipBuild:true}),script=join(repo,"backend/dist/server.js"),replacement=join(repo,"replacement-server.js"),owned=await readManualOwnership({repositoryRoot:repo}); await writeFile(replacement,`import {writeFileSync} from "node:fs";writeFileSync(${JSON.stringify(malicious)},"bad");setInterval(()=>{},1000);`); - await assert.rejects(serveManual({repositoryRoot:repo,beforeSpawn:async()=>rename(replacement,script)}),/entrypoint|identity|changed/i); + await assert.rejects(serveManual({repositoryRoot:repo,beforeSpawn:async()=>rename(replacement,script)}),/entrypoint|identity|changed|readiness|failed|distribution|module/i); await assert.rejects(lstat(malicious)); await assert.rejects(lstat(join(run.root,"backend.pid"))); + assert.deepEqual(await matchingManualServerPids(run.root,owned.nonce),[]); }); test("foreign 8791 health can never make a delayed authenticated child RUNNING", { concurrency: false }, async () => { diff --git a/backend/scripts/p1-render-snapshot.mjs b/backend/scripts/p1-render-snapshot.mjs index 7d9aa0a9..d423d696 100755 --- a/backend/scripts/p1-render-snapshot.mjs +++ b/backend/scripts/p1-render-snapshot.mjs @@ -1,8 +1,8 @@ #!/usr/bin/env node import { spawnSync } from "node:child_process"; import { createHash } from "node:crypto"; -import { lstatSync, realpathSync } from "node:fs"; -import { lstat, mkdir, readFile, realpath } from "node:fs/promises"; +import { constants, lstatSync, realpathSync } from "node:fs"; +import { lstat, mkdir, open, readFile, realpath } from "node:fs/promises"; import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import { fileURLToPath } from "node:url"; @@ -81,7 +81,41 @@ async function atomicCopy(source,output) { if(result.error||result.status!==0)throw new Error("anchored output publication refused; rendered parent identity changed or output is unsafe"); } -export async function renderOwnedSnapshot({ repositoryRoot = defaultRepositoryRoot, ownershipPath, snapshotPath, outputPath, env = process.env, beforePublish }) { +function sameEntry(actual, expected) { return actual.dev === expected.dev && actual.ino === expected.ino; } +async function readBounded(path, max, label) { + let handle; + try { + handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW); + const before = await handle.stat(), pathEntry = await lstat(path); + if (!before.isFile() || pathEntry.isSymbolicLink() || !pathEntry.isFile() || !sameEntry(before, pathEntry)) throw new Error(`${label} is unsafe`); + if (before.size < 1 || before.size > max) throw new Error(`${label} is unbounded`); + const bytes = Buffer.alloc(before.size); let offset = 0; + while (offset < bytes.length) { + const { bytesRead } = await handle.read(bytes, offset, bytes.length - offset, offset); + if (bytesRead < 1) throw new Error(`${label} changed while reading`); + offset += bytesRead; + } + const after = await handle.stat(); + if (!sameEntry(before, after) || after.size !== before.size) throw new Error(`${label} changed while reading`); + return bytes; + } finally { + if (handle) await handle.close().catch(() => {}); + } +} +async function readSnapshotManifest(root, manifestPath, commit, yamlName, expectedDigest) { + let manifestEntry; + try { assertNoSymlinks(root, manifestPath); manifestEntry = await lstat(manifestPath); } + catch (error) { if (error?.code === "ENOENT") throw new Error("snapshot manifest is missing or unbounded"); throw error; } + if (!manifestEntry.isFile() || manifestEntry.isSymbolicLink() || await realpath(manifestPath) !== manifestPath) throw new Error("snapshot manifest is unsafe"); + const bytes = await readBounded(manifestPath, 1048576, "snapshot manifest"); + let manifest; try { manifest = JSON.parse(bytes.toString("utf8")); } catch { throw new Error("snapshot manifest is malformed"); } + const files = manifest?.files; + if (manifest?.head !== commit || !files || typeof files !== "object" || Array.isArray(files)) throw new Error("snapshot manifest identity is unsafe"); + if (!HEX64.test(files[yamlName] ?? "") || files[yamlName] !== expectedDigest) throw new Error("snapshot manifest digest is unsafe"); + return manifest; +} + +export async function renderOwnedSnapshot({ repositoryRoot = defaultRepositoryRoot, ownershipPath, snapshotPath, outputPath, snapshotSha256, env = process.env, beforePublish }) { const repo = realpathSync(repositoryRoot); const { root } = await ownership(repo, resolve(repo, ownershipPath)); const snapshot = resolve(repo, snapshotPath); const output = resolve(repo, outputPath); const snapshotsRoot = join(root, "installation", "registry", "snapshots"); @@ -89,18 +123,14 @@ export async function renderOwnedSnapshot({ repositoryRoot = defaultRepositoryRo if (!isAbsolute(snapshotPath) || !below(snapshotsRoot, snapshot)) throw new Error("snapshot is not an owned absolute path"); const match = /^([0-9a-f]{40})\/([a-z][a-z0-9-]{2,62})\.yaml$/.exec(relative(snapshotsRoot, snapshot).split(sep).join("/")); if (!match || !HEX40.test(match[1])) throw new Error("snapshot is not commit addressed"); + if (!HEX64.test(snapshotSha256 ?? "")) throw new Error("snapshot digest identity is unsafe"); assertNoSymlinks(root, snapshot); const snapshotEntry = await lstat(snapshot); if (!snapshotEntry.isFile() || snapshotEntry.isSymbolicLink() || await realpath(snapshot) !== snapshot) throw new Error("snapshot is unsafe"); - const snapshotBytes = await readFile(snapshot); - const snapshotDigest = createHash("sha256").update(snapshotBytes).digest("hex"); - const manifestPath = join(dirname(snapshot), "snapshot.json"); - assertNoSymlinks(root, manifestPath); - const manifestEntry = await lstat(manifestPath); - if (!manifestEntry.isFile() || manifestEntry.isSymbolicLink() || await realpath(manifestPath) !== manifestPath) throw new Error("snapshot manifest is unsafe"); - let manifest; - try { manifest = JSON.parse(await readFile(manifestPath, "utf8")); } catch { throw new Error("snapshot manifest is malformed"); } const yamlName = `${match[2]}.yaml`; - if (manifest?.head !== match[1] || manifest?.files?.[yamlName] !== snapshotDigest) throw new Error("snapshot content does not match its immutable manifest"); + const manifestPath = join(snapshotsRoot, match[1], "snapshot.json"); + await readSnapshotManifest(root, manifestPath, match[1], yamlName, snapshotSha256); + const snapshotBytes = await readBounded(snapshot, 1048576, "snapshot"); + if (createHash("sha256").update(snapshotBytes).digest("hex") !== snapshotSha256) throw new Error("snapshot bytes changed"); if (!below(renderedRoot, output) || dirname(output) !== renderedRoot || !output.endsWith(".yaml")) throw new Error("output is not an owned rendered path"); assertNoSymlinks(root, dirname(output)); try { if ((await lstat(output)).isSymbolicLink()) throw new Error("output is unsafe"); } catch (error) { if (error.code !== "ENOENT") throw error; } @@ -117,8 +147,8 @@ export async function renderOwnedSnapshot({ repositoryRoot = defaultRepositoryRo try { lease = runner.acquireWorkspaceRuntime(snapshot); const verifySnapshot = async () => { - const current = await readFile(snapshot); - if (createHash("sha256").update(current).digest("hex") !== snapshotDigest) throw new Error("snapshot content changed during rendering"); + const current = await readBounded(snapshot, 1048576, "snapshot"); + if (createHash("sha256").update(current).digest("hex") !== snapshotSha256) throw new Error("snapshot content changed during rendering"); }; await verifySnapshot(); if(beforePublish)await beforePublish({output,renderedRoot}); @@ -132,11 +162,11 @@ export async function renderOwnedSnapshot({ repositoryRoot = defaultRepositoryRo return output; } function parseArgs(argv) { - if (argv.length !== 6) throw new Error("usage: p1-render-snapshot.mjs --ownership PATH --snapshot ABSOLUTE_PATH --output PATH"); - const result = {}; for (let i=0;icreateHash("sha256").update(bytes).digest("hex"); async function fixture() { const repo=await realpath(await mkdtemp(join(tmpdir(),"p1-render-repo-"))); roots.push(repo); const root=join(repo,".artifacts/manual-acceptance/p1"); const commit="a".repeat(40); const snapshot=join(root,"installation/registry/snapshots",commit,"p1-filesystem.yaml"); @@ -33,23 +34,37 @@ evidence: source: {type: filesystem, uri: workspace-content/p1-filesystem/evidence, patterns: ["**/*.md"], max_bytes: 10485760} policy: {max_chunk_chars: 4000, retain_published_generations: 3} `); - const snapshotBytes=await readFile(snapshot); await writeFile(join(dirname(snapshot),"snapshot.json"),JSON.stringify({head:commit,revisions:[{id:"p1-filesystem",commit,blob:"0".repeat(40),snapshotPath:snapshot,state:"operational"}],files:{"p1-filesystem.yaml":createHash("sha256").update(snapshotBytes).digest("hex")}})); + const snapshotBytes=await readFile(snapshot); const snapshotSha256=sha256(snapshotBytes); + const manifestPath=join(dirname(snapshot),"snapshot.json"); + await writeFile(manifestPath,JSON.stringify({head:commit,revisions:[{id:"p1-filesystem",commit,blob:"0".repeat(40),snapshotPath:snapshot,state:"operational"}],files:{"p1-filesystem.yaml":snapshotSha256}})); const env={THT_WS_P1_FILESYSTEM_DWH_TRANSPORT:"postgres_direct",THT_WS_P1_FILESYSTEM_DWH_HOST:"dwh.invalid",THT_WS_P1_FILESYSTEM_DWH_PORT:"5432",THT_WS_P1_FILESYSTEM_DWH_USER:"reader",THT_WS_P1_FILESYSTEM_DWH_PASSWORD_FILE:secret}; - return {repo,root,snapshot,env}; + return {repo,root,snapshot,snapshotSha256,manifestPath,env}; } test.afterEach(async()=>Promise.all(roots.splice(0).map(r=>rm(r,{recursive:true,force:true})))); -test("renderer copies a production lease deterministically with mode 0600 and no leases",async()=>{ const f=await fixture(); const one=join(f.root,"rendered/one.yaml"),two=join(f.root,"rendered/two.yaml"); await renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:one,env:{...process.env,...f.env}}); await renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:two,env:{...process.env,...f.env}}); assert.deepEqual(await readFile(one),await readFile(two)); assert.equal((await lstat(one)).mode&0o777,0o600); assert.deepEqual(await (await import("node:fs/promises")).readdir(join(f.root,"installation/registry/snapshots/runtime")),[]); }); +const call=(f,extra={})=>renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,snapshotSha256:f.snapshotSha256,env:{...process.env,...f.env},...extra}); +const runtimeLeases=async f=>await (await import("node:fs/promises")).readdir(join(f.root,"installation/registry/snapshots/runtime")); -test("renderer rejects unowned, symlink, and out-of-root paths",async()=>{ const f=await fixture(); const outside=join(f.repo,"outside.yaml"); await writeFile(outside,"x"); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:outside,outputPath:join(f.root,"rendered/x.yaml"),env:f.env}),/owned|snapshot/); const link=join(dirname(f.snapshot),"linked.yaml"); await symlink(f.snapshot,link); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:link,outputPath:join(f.root,"rendered/x.yaml"),env:f.env}),/snapshot|symlink/); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:outside,env:f.env}),/output/); }); +test("renderer copies a production lease deterministically with mode 0600 and no leases",async()=>{ const f=await fixture(); const one=join(f.root,"rendered/one.yaml"),two=join(f.root,"rendered/two.yaml"); await call(f,{outputPath:one}); await call(f,{outputPath:two}); assert.deepEqual(await readFile(one),await readFile(two)); assert.equal((await lstat(one)).mode&0o777,0o600); assert.deepEqual(await runtimeLeases(f),[]); }); -test("renderer releases its acquired lease when atomic output copy fails",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/existing.yaml"); await mkdir(output); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:output,env:{...process.env,...f.env}}),/anchored|publication|unsafe/); assert.deepEqual(await (await import("node:fs/promises")).readdir(join(f.root,"installation/registry/snapshots/runtime")),[]); }); +test("renderer rejects unowned, symlink, out-of-root and missing-digest paths",async()=>{ const f=await fixture(); const outside=join(f.repo,"outside.yaml"); await writeFile(outside,"x"); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:outside,outputPath:join(f.root,"rendered/x.yaml"),snapshotSha256:f.snapshotSha256,env:f.env}),/owned|snapshot/); const link=join(dirname(f.snapshot),"linked.yaml"); await symlink(f.snapshot,link); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:link,outputPath:join(f.root,"rendered/x.yaml"),snapshotSha256:f.snapshotSha256,env:f.env}),/snapshot|symlink/); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:outside,snapshotSha256:f.snapshotSha256,env:f.env}),/output/); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:join(f.root,"rendered/x.yaml"),env:f.env}),/snapshot digest identity/); }); +test("renderer releases its acquired lease when atomic output copy fails",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/existing.yaml"); await mkdir(output); await assert.rejects(call(f,{outputPath:output}),/anchored|publication|unsafe/); assert.deepEqual(await runtimeLeases(f),[]); }); -test("renderer rejects a regular snapshot replacement against its manifest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/replaced.yaml"); await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:output,env:{...process.env,...f.env},beforePublish:async()=>{await writeFile(f.snapshot,"workspace:\n schema_version: 3\n id: p1-filesystem\n name: replaced\n")}}),/snapshot content changed/); await assert.rejects(lstat(output)); }); +test("renderer refuses a same-path regular snapshot byte replacement against manifest and expected digest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/replaced.yaml"); const replaced=(await readFile(f.snapshot,"utf8")).replace("max_chunk_chars: 4000","max_chunk_chars: 3999"); await writeFile(f.snapshot,replaced); await assert.rejects(call(f,{outputPath:output}),/snapshot bytes changed/); await assert.rejects(lstat(output)); assert.deepEqual(await runtimeLeases(f),[]); }); + +test("renderer refuses snapshot manifest head, digest, and expected-digest tampering",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/tampered.yaml"); const manifest=JSON.parse(await readFile(f.manifestPath,"utf8")); + await writeFile(f.manifestPath,JSON.stringify({...manifest,head:"c".repeat(40)})); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest identity/); + await writeFile(f.manifestPath,JSON.stringify({...manifest,files:{"p1-filesystem.yaml":"d".repeat(64)}})); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest digest/); + await writeFile(f.manifestPath,JSON.stringify(manifest)); await assert.rejects(call(f,{outputPath:output,snapshotSha256:"e".repeat(64)}),/snapshot manifest digest/); + await assert.rejects(lstat(output)); assert.deepEqual(await runtimeLeases(f),[]); }); + +test("renderer refuses a missing or malformed snapshot manifest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/nomanifest.yaml"); await rm(f.manifestPath); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest.*(missing|unbounded|unsafe)/); await writeFile(f.manifestPath,"{not json"); await assert.rejects(call(f,{outputPath:output}),/snapshot manifest.*malformed/); await assert.rejects(lstat(output)); assert.deepEqual(await runtimeLeases(f),[]); }); + +test("renderer rejects a regular snapshot replacement against its manifest",async()=>{ const f=await fixture(); const output=join(f.root,"rendered/replaced.yaml"); await assert.rejects(call(f,{outputPath:output,beforePublish:async()=>{await writeFile(f.snapshot,"workspace:\n schema_version: 3\n id: p1-filesystem\n name: replaced\n")}}),/snapshot content changed/); await assert.rejects(lstat(output)); }); test("renderer anchors publication when rendered parent is concurrently swapped", async()=>{ const f=await fixture(),output=join(f.root,"rendered/raced.yaml"),moved=join(f.root,"rendered-moved"),outside=join(f.repo,"outside-rendered"); await mkdir(outside); - await assert.rejects(renderOwnedSnapshot({repositoryRoot:f.repo,ownershipPath:join(f.root,"ownership.json"),snapshotPath:f.snapshot,outputPath:output,env:{...process.env,...f.env},beforePublish:async()=>{await rename(join(f.root,"rendered"),moved);await symlink(outside,join(f.root,"rendered"));}}),/identity|changed|unsafe|publication/i); + await assert.rejects(call(f,{outputPath:output,beforePublish:async()=>{await rename(join(f.root,"rendered"),moved);await symlink(outside,join(f.root,"rendered"));}}),/identity|changed|unsafe|publication/i); assert.deepEqual(await (await import("node:fs/promises")).readdir(outside),[]); }); diff --git a/docs/testing/p1-manual-acceptance.md b/docs/testing/p1-manual-acceptance.md index ea8eea61..14b2b748 100644 --- a/docs/testing/p1-manual-acceptance.md +++ b/docs/testing/p1-manual-acceptance.md @@ -37,15 +37,20 @@ path/device/inode/size/SHA-256, creates no supervisor or readiness-status file, `PENDING` and the server stopped, and refuses an existing root. Use guarded `stop` and `cleanup` rather than deleting or reusing state manually. -`serve` revalidates the bound `backend/dist/server.js` identity and bytes, every owned -root/runtime/log ancestor, the absence of a legacy supervisor, and the original log identity before -spawning. The log and production entrypoint are opened with no-follow semantics and their descriptors -are passed directly to the child; an immutable preload makes Node load the already verified -entrypoint bytes rather than a later pathname replacement. The child remains the production Node -entrypoint itself: `node --import data:text/javascript;base64, -backend/dist/server.js` followed by six ownership, control, and entrypoint-identity arguments. The -preload owns the authenticated fixed `127.0.0.1:8792` control channel and bounded watchdog, and tracks -the HTTP server that this same process successfully binds to `127.0.0.1:8791`. Before publishing the +`serve` revalidates the bound `backend/dist/server.js` identity and bytes, the immutable +post-build manifest of every regular `backend/dist` file (path, size, SHA-256, device, inode), +every owned root/runtime/log ancestor, the absence of a legacy supervisor, and the original log +identity before spawning. The log, the production entrypoint, and the distribution manifest are +opened with no-follow semantics; the entrypoint and manifest descriptors are passed directly to the +child, and an immutable preload makes Node load the already verified entrypoint bytes and the +complete verified `backend/dist` module graph rather than a later pathname replacement. At startup +the preload hash-verifies every manifest file and serves only those cached verified bytes for any +import below `backend/dist`, so a same-path regular replacement is refused (before or during +serving) and can never execute. The child remains the production Node entrypoint itself: +`node --import data:text/javascript;base64, backend/dist/server.js` followed by +six ownership, control, and entrypoint-identity arguments (plus the manifest descriptor on fd 4). +The preload owns the authenticated fixed `127.0.0.1:8792` control channel and bounded watchdog, and +tracks the HTTP server that this same process successfully binds to `127.0.0.1:8791`. Before publishing the `RUNNING` PID record, the parent requires exact nonce-bound control acknowledgements that identify that owned listener, a 2xx `GET /health`, stable listener generation and entrypoint identity, and a final authenticated status check. A foreign health listener cannot satisfy readiness. A startup or @@ -61,11 +66,16 @@ no-follow directory identities to rename and remove only the exact stopped owned siblings and automated integration artifacts are outside its cleanup boundary. After `prepare`, follow the 14 ordered steps in the generated absolute-path `GUIDE.md`. Personally run each generated `http-01` through `http-14` curl script in numeric order; they save the exact status, three validation, three sequential publication, pull, three read responses, and three ZIP exports. Each publication derives its current base commit with a bounded parser from the preceding saved API response, with no placeholder base. Run the five numbered negative validation scripts separately at checklist step 10. The render commands validate the bounded saved read response, -its commit-addressed owned snapshot path, the saved publish commit, and the installed Git HEAD before -calling the acceptance-only production renderer. The renderer imports the built `ThtRunner`, resolves -bindings from environment paths, copies one lease with mode `0600` through an opened no-follow -`rendered` directory descriptor, rejects an output-parent identity swap, and releases the lease in -`finally`. For each exported ZIP, invoke the generated extractor with the exact expected workspace ID +its commit-addressed owned snapshot path, the saved publish commit, the installed Git HEAD, and the +bounded `snapshot.json` manifest of that commit: they bind the snapshot bytes to the manifest digest, +the saved revision blob to the manifest revision, and the manifest blob to the installed Git commit +(`git rev-parse :workspaces/.yaml` plus `git hash-object` of the snapshot bytes) before +calling the acceptance-only production renderer with the expected `--snapshot-sha256`. The renderer +revalidates the bounded `snapshot.json` (`head`, `files[.yaml]`) and reads the snapshot exactly +once with no-follow semantics, rendering only the digest-verified bytes. It imports the built +`ThtRunner`, resolves bindings from environment paths, copies one lease with mode `0600` through an +opened no-follow `rendered` directory descriptor, rejects an output-parent identity swap, and +releases the lease in `finally`. For each exported ZIP, invoke the generated extractor with the exact expected workspace ID (`p1-filesystem`, `p1-http`, or `p1-s3`); its `python3` helper opens the source once, stages and revalidates its SHA-256, anchors every extraction and cleanup operation to an opened no-follow `exports/extracted` directory descriptor, and binds both the manifest and parsed descriptor identity