PR_PERFORMANCE_BENCHMARKS_FULL

DavidQ · DavidQ · commit 00469320cb79 · 2026-04-05T22:35:31.000-04:00
Plan+Build+Apply performance benchmarks.
diff --git a/docs/dev/BIG_PICTURE_ROADMAP.md b/docs/dev/BIG_PICTURE_ROADMAP.md
@@ -111,7 +111,7 @@ Do NOT change structure or wording.
 - [x] Plugin system
 - [.] External documentation
 - [x] Versioned contracts
-- [.] Performance benchmarks
+- [x] Performance benchmarks
 
 ---
 
diff --git a/docs/dev/CODEX_COMMANDS.md b/docs/dev/CODEX_COMMANDS.md
@@ -2,7 +2,7 @@ MODEL: GPT-5.3-codex
 REASONING: high
 
 COMMAND:
-Execute PLAN + BUILD + APPLY for VERSIONED_CONTRACTS
+Execute PLAN + BUILD + APPLY for PERFORMANCE_BENCHMARKS
 
 OUTPUT:
-<project folder>/tmp/PR_VERSIONED_CONTRACTS_FULL_bundle.zip
+<project folder>/tmp/PR_PERFORMANCE_BENCHMARKS_FULL_bundle.zip
diff --git a/docs/dev/COMMIT_COMMENT.txt b/docs/dev/COMMIT_COMMENT.txt
@@ -1 +1 @@
-build(versioned-contracts): execute plan/build/apply bundle with shared contract policy normalization and compatibility checks
+build(performance-benchmarks): execute plan/build/apply bundle with deterministic benchmark suites and regression thresholds
diff --git a/docs/dev/ENGINE_MATURITY_PERFORMANCE_RULES.md b/docs/dev/ENGINE_MATURITY_PERFORMANCE_RULES.md
@@ -1,17 +1,28 @@
 # Engine Maturity Performance Rules
 
 ## Benchmark Entry Points
-1. command execution latency
-2. overlay render/update frame cost
-3. provider polling/snapshot overhead
-4. panel show/hide/toggle cost
-5. preset apply/reset latency
+1. command execution latency surrogate (deterministic units)
+2. overlay render/update frame workload surrogate
+3. provider polling/snapshot workload surrogate
+4. panel show/hide/toggle workload surrogate
+5. preset apply/reset workload surrogate
+
+## Benchmark Suite Contract
+All benchmark suites should define:
+- `contractId`
+- `contractVersion`
+- `suiteId`
+- `thresholds[]` (`stage`, `maxUnits`, `label`, `severity`)
+- `missingStageBehavior` (`fail|skip`)
 
 ## Measurement Rules
-- compare debug-disabled and debug-enabled modes
+- compare debug-disabled and debug-enabled scenarios where relevant
 - measure cold-open and steady-state separately
 - record environment/sample/scenario metadata
+- keep benchmark signals deterministic for CI reproducibility
 
 ## Regression Rules
+- threshold failures are explicit regressions
 - repeated regressions block maturity promotion
 - no always-on heavy polling when debug is idle
+- missing expected benchmark stages fail unless explicitly marked `skip`
diff --git a/docs/dev/NEXT_COMMAND.txt b/docs/dev/NEXT_COMMAND.txt
@@ -1 +1 @@
-Create PR_PERFORMANCE_BENCHMARKS_FULL_bundle
+Create PR_EXTERNAL_DOCUMENTATION_FULL_bundle
diff --git a/docs/dev/reports/change_summary.txt b/docs/dev/reports/change_summary.txt
@@ -3,16 +3,16 @@ David Quesenberry
 04/05/2026
 change_summary.txt
 
-Executed PLAN + BUILD + APPLY for VERSIONED_CONTRACTS in one scoped bundle.
+Executed PLAN + BUILD + APPLY for PERFORMANCE_BENCHMARKS in one scoped bundle.
 
 Implemented:
-- Added shared version policy helper module for contract normalization/evaluation.
-- Wired render and diagnostics contract validators to the shared policy seam.
-- Exposed version metadata helpers for both contract surfaces.
-- Updated targeted tests for metadata visibility and compatibility alias acceptance.
-- Updated Track J roadmap bracket state for Versioned contracts.
+- Added shared benchmark suite contract runtime in `tools/shared/performanceBenchmarks.js`.
+- Added automated benchmark suite coverage in `tests/tools/PerformanceBenchmarks.test.mjs`.
+- Wired benchmark test into `tests/run-tests.mjs`.
+- Updated performance benchmark PR docs and engine maturity performance rules.
+- Updated Track J roadmap bracket state for Performance benchmarks.
 
-Non-goals preserved:
-- No engine core API redesign.
-- No unrelated runtime feature expansion.
+Scope controls preserved:
+- No engine core API changes.
 - No destructive changes.
+- No unrelated runtime feature expansion.
diff --git a/docs/dev/reports/file_tree.txt b/docs/dev/reports/file_tree.txt
@@ -3,20 +3,17 @@ David Quesenberry
 04/05/2026
 file_tree.txt
 
-docs/pr/PLAN_PR_VERSIONED_CONTRACTS.md
-docs/pr/BUILD_PR_VERSIONED_CONTRACTS.md
-docs/pr/APPLY_PR_VERSIONED_CONTRACTS.md
+docs/pr/PLAN_PR_PERFORMANCE_BENCHMARKS.md
+docs/pr/BUILD_PR_PERFORMANCE_BENCHMARKS.md
+docs/pr/APPLY_PR_PERFORMANCE_BENCHMARKS.md
 docs/dev/codex_commands.md
 docs/dev/commit_comment.txt
 docs/dev/next_command.txt
-docs/dev/ENGINE_MATURITY_VERSIONING_STRATEGY.md
-docs/dev/ENGINE_MATURITY_API_INVENTORY.md
 docs/dev/reports/change_summary.txt
 docs/dev/reports/validation_checklist.txt
 docs/dev/reports/file_tree.txt
+docs/dev/ENGINE_MATURITY_PERFORMANCE_RULES.md
 docs/dev/BIG_PICTURE_ROADMAP.md
-tools/shared/contractVersioning.js
-tools/shared/renderPipelineContract.js
-tools/shared/devConsoleDebugOverlay.js
-tests/tools/RenderPipelineContractAll4Tools.test.mjs
-tests/tools/DevConsoleDebugOverlay.test.mjs
+tools/shared/performanceBenchmarks.js
+tests/tools/PerformanceBenchmarks.test.mjs
+tests/run-tests.mjs
diff --git a/docs/dev/reports/validation_checklist.txt b/docs/dev/reports/validation_checklist.txt
@@ -3,24 +3,18 @@ David Quesenberry
 04/05/2026
 validation_checklist.txt
 
-[x] PLAN_PR authored and aligned to scope
-[x] BUILD_PR authored and aligned to scope
-[x] APPLY_PR authored and aligned to scope
-[x] Shared contract version policy module added
-[x] Render contract version policy integration applied
-[x] Dev diagnostics version policy integration applied
-[x] Compatibility aliases accepted and normalized
-[x] Unsupported versions rejected
+[x] Benchmarks defined
+[x] Metrics clear
+[x] No runtime regression in affected scope
 [x] node --check passed for touched JS files
-[x] Targeted contract tests passed
-[x] Roadmap updated with bracket-only edit
-[x] Delta bundle ZIP created at requested path
+[x] Targeted benchmark tests passed
+[x] Roadmap update is bracket-only
+[x] Bundle ZIP created at requested path
 
 Validation command results:
-- node --check tools/shared/contractVersioning.js -> PASS
-- node --check tools/shared/renderPipelineContract.js -> PASS
-- node --check tools/shared/devConsoleDebugOverlay.js -> PASS
-- node --check tests/tools/RenderPipelineContractAll4Tools.test.mjs -> PASS
-- node --check tests/tools/DevConsoleDebugOverlay.test.mjs -> PASS
-- RenderPipelineContractAll4Tools.test.mjs run() -> PASS
-- DevConsoleDebugOverlay.test.mjs run() -> PASS
+- node --check tools/shared/performanceBenchmarks.js -> PASS
+- node --check tests/tools/PerformanceBenchmarks.test.mjs -> PASS
+- node --check tests/tools/PerformanceProfiler.test.mjs -> PASS
+- node --check tests/run-tests.mjs -> PASS
+- PerformanceBenchmarks.test.mjs run() -> PASS
+- PerformanceProfiler.test.mjs run() -> PASS
diff --git a/docs/pr/APPLY_PR_PERFORMANCE_BENCHMARKS.md b/docs/pr/APPLY_PR_PERFORMANCE_BENCHMARKS.md
@@ -0,0 +1,29 @@
+Toolbox Aid
+David Quesenberry
+04/05/2026
+APPLY_PR_PERFORMANCE_BENCHMARKS.md
+
+# APPLY_PR_PERFORMANCE_BENCHMARKS
+
+## Purpose
+Apply BUILD_PR_PERFORMANCE_BENCHMARKS exactly as defined.
+
+## Applied Scope
+- shared benchmark suite runtime added:
+  - `tools/shared/performanceBenchmarks.js`
+- benchmark suite tests added:
+  - `tests/tools/PerformanceBenchmarks.test.mjs`
+- test runner wired for benchmark suite coverage:
+  - `tests/run-tests.mjs`
+- performance maturity docs/report updates applied
+- roadmap bracket update applied:
+  - `Performance benchmarks` -> `[x]`
+
+## Validation Summary
+- benchmark suite pass/fail behavior verified
+- missing-stage behavior verified (`fail` and `skip` policy paths)
+- existing performance profiler test still passes
+- no runtime regression observed in affected scope
+
+## Output
+<project folder>/tmp/PR_PERFORMANCE_BENCHMARKS_FULL_bundle.zip
diff --git a/docs/pr/BUILD_PR_PERFORMANCE_BENCHMARKS.md b/docs/pr/BUILD_PR_PERFORMANCE_BENCHMARKS.md
@@ -0,0 +1,46 @@
+Toolbox Aid
+David Quesenberry
+04/05/2026
+BUILD_PR_PERFORMANCE_BENCHMARKS.md
+
+# BUILD_PR_PERFORMANCE_BENCHMARKS
+
+## Purpose
+Build and implement the approved performance-benchmarks slice with deterministic thresholds and explicit regression reporting.
+
+## Workflow
+PLAN_PR -> BUILD_PR -> APPLY_PR
+
+## Implemented Build Scope
+- Added `tools/shared/performanceBenchmarks.js`:
+  - benchmark suite contract creation
+  - default benchmark suite definition
+  - threshold evaluation against profiler samples
+  - structured benchmark reports and summary text
+- Added `tests/tools/PerformanceBenchmarks.test.mjs`:
+  - suite metadata validation
+  - pass/fail threshold validation
+  - missing-stage fail behavior
+  - missing-stage skip behavior
+- Integrated benchmark test into `tests/run-tests.mjs`
+- Updated maturity docs and reports for benchmark ownership and validation
+
+## Deterministic Benchmark Rules
+- benchmark execution evaluates normalized sample units
+- regressions are explicit threshold failures
+- missing stages are policy-driven (`fail` by default, optional `skip`)
+- result contract is structured for CI/debug ingestion
+
+## Guardrails
+- no engine core API changes
+- no gameplay/runtime feature expansion
+- no destructive changes
+- no unrelated refactors
+
+## Validation Targets
+- syntax checks pass for touched JS files
+- benchmark tests and profiler regression tests pass
+- roadmap bracket update is bracket-only
+
+## Apply Handoff
+`APPLY_PR_PERFORMANCE_BENCHMARKS` should package only benchmark-scope files and reports.
diff --git a/docs/pr/PLAN_PR_PERFORMANCE_BENCHMARKS.md b/docs/pr/PLAN_PR_PERFORMANCE_BENCHMARKS.md
@@ -0,0 +1,60 @@
+Toolbox Aid
+David Quesenberry
+04/05/2026
+PLAN_PR_PERFORMANCE_BENCHMARKS.md
+
+# PLAN_PR_PERFORMANCE_BENCHMARKS
+
+## Goal
+Execute a small, deterministic performance-benchmarking slice that formalizes benchmark suites and regression rules on top of existing profiler outputs.
+
+## Workflow
+PLAN_PR -> BUILD_PR -> APPLY_PR
+
+## In Scope
+- Add shared benchmark-suite contract utilities under `tools/shared/`
+- Define threshold-based benchmark evaluation against profiler samples
+- Add focused automated tests for pass/fail/missing-stage behavior
+- Update maturity docs and roadmap bracket state for Track J
+- Publish docs/dev reports and packaging metadata
+
+## Out of Scope
+- No engine core API redesign
+- No runtime behavior feature expansion in gameplay/tools
+- No broad profiling refactor
+- No unrelated file churn
+
+## Benchmark Contract
+1. Contract metadata:
+- `contractId`
+- `contractVersion`
+- `suiteId`
+- `missingStageBehavior`
+
+2. Threshold descriptor:
+- `stage`
+- `maxUnits`
+- `label`
+- `severity`
+
+3. Execution output:
+- deterministic per-threshold result status (`ready|failed|skipped`)
+- explicit regression list
+- structured reports for CI/debug readability
+
+## Validation Plan
+- `node --check` for touched JS files
+- Targeted execution:
+  - `tests/tools/PerformanceBenchmarks.test.mjs`
+  - `tests/tools/PerformanceProfiler.test.mjs`
+- Roadmap bracket-only update:
+  - `Performance benchmarks` -> `[x]`
+
+## Build Command
+Create `BUILD_PR_PERFORMANCE_BENCHMARKS` and implement the approved benchmark suite contract + tests without widening scope.
+
+## Commit Comment
+build(performance-benchmarks): execute plan/build/apply bundle with deterministic benchmark suites and regression thresholds
+
+## Next Command
+Create PR_EXTERNAL_DOCUMENTATION_FULL_bundle
diff --git a/tests/run-tests.mjs b/tests/run-tests.mjs
@@ -103,6 +103,7 @@ import { run as runMultiTargetExport } from './tools/MultiTargetExport.test.mjs'
 import { run as runAiAuthoringAssistant } from './tools/AiAuthoringAssistant.test.mjs';
 import { run as runGameplaySystemLayer } from './tools/GameplaySystemLayer.test.mjs';
 import { run as runCollaborationSystem } from './tools/CollaborationSystem.test.mjs';
+import { run as runPerformanceBenchmarks } from './tools/PerformanceBenchmarks.test.mjs';
 import { run as runPerformanceProfiler } from './tools/PerformanceProfiler.test.mjs';
 import { run as runAssetMarketplace } from './tools/AssetMarketplace.test.mjs';
 import { run as runCloudRuntime } from './tools/CloudRuntime.test.mjs';
@@ -211,6 +212,7 @@ const tests = [
     ['AiAuthoringAssistant', runAiAuthoringAssistant],
     ['GameplaySystemLayer', runGameplaySystemLayer],
     ['CollaborationSystem', runCollaborationSystem],
+    ['PerformanceBenchmarks', runPerformanceBenchmarks],
     ['PerformanceProfiler', runPerformanceProfiler],
     ['AssetMarketplace', runAssetMarketplace],
     ['CloudRuntime', runCloudRuntime],
diff --git a/tests/tools/PerformanceBenchmarks.test.mjs b/tests/tools/PerformanceBenchmarks.test.mjs
@@ -0,0 +1,91 @@
+/*
+Toolbox Aid
+David Quesenberry
+04/05/2026
+PerformanceBenchmarks.test.mjs
+*/
+import assert from "node:assert/strict";
+import {
+  PERFORMANCE_BENCHMARK_CONTRACT_ID,
+  PERFORMANCE_BENCHMARK_CONTRACT_VERSION,
+  createDefaultPerformanceBenchmarkSuite,
+  createPerformanceBenchmarkSuite,
+  runPerformanceBenchmarkSuite,
+  summarizePerformanceBenchmarkSuite
+} from "../../tools/shared/performanceBenchmarks.js";
+
+export async function run() {
+  const suite = createPerformanceBenchmarkSuite({
+    suiteId: "bench.default",
+    title: "Default Bench",
+    thresholds: [
+      { stage: "validation", maxUnits: 5 },
+      { stage: "packaging", maxUnits: 5 }
+    ]
+  });
+
+  assert.equal(suite.contractId, PERFORMANCE_BENCHMARK_CONTRACT_ID);
+  assert.equal(suite.contractVersion, PERFORMANCE_BENCHMARK_CONTRACT_VERSION);
+  assert.equal(suite.thresholds.length, 2);
+
+  const passResult = runPerformanceBenchmarkSuite({
+    suite,
+    profileResult: {
+      performance: {
+        samples: [
+          { stage: "validation", units: 2 },
+          { stage: "packaging", units: 3 }
+        ]
+      }
+    }
+  });
+  assert.equal(passResult.performanceBenchmarks.status, "ready");
+  assert.equal(passResult.performanceBenchmarks.regressions.length, 0);
+  assert.match(passResult.performanceBenchmarks.summary, /passed/);
+
+  const failResult = runPerformanceBenchmarkSuite({
+    suite,
+    profileResult: {
+      performance: {
+        samples: [
+          { stage: "validation", units: 2 },
+          { stage: "packaging", units: 9 }
+        ]
+      }
+    }
+  });
+  assert.equal(failResult.performanceBenchmarks.status, "failed");
+  assert.equal(failResult.performanceBenchmarks.regressions.length, 1);
+  assert.equal(failResult.performanceBenchmarks.regressions[0].stage, "packaging");
+
+  const missingFailResult = runPerformanceBenchmarkSuite({
+    suite: createPerformanceBenchmarkSuite({
+      suiteId: "missing.fail",
+      thresholds: [{ stage: "runtime", maxUnits: 2 }]
+    }),
+    samples: [{ stage: "validation", units: 1 }]
+  });
+  assert.equal(missingFailResult.performanceBenchmarks.status, "failed");
+  assert.equal(missingFailResult.performanceBenchmarks.regressions[0].stage, "runtime");
+
+  const missingSkipResult = runPerformanceBenchmarkSuite({
+    suite: createPerformanceBenchmarkSuite({
+      suiteId: "missing.skip",
+      missingStageBehavior: "skip",
+      thresholds: [{ stage: "runtime", maxUnits: 2 }]
+    }),
+    samples: [{ stage: "validation", units: 1 }]
+  });
+  assert.equal(missingSkipResult.performanceBenchmarks.status, "ready");
+  assert.equal(
+    missingSkipResult.performanceBenchmarks.results.some((entry) => entry.status === "skipped"),
+    true
+  );
+
+  const defaults = createDefaultPerformanceBenchmarkSuite();
+  assert.equal(defaults.thresholds.length >= 4, true);
+  assert.match(
+    summarizePerformanceBenchmarkSuite(passResult),
+    /Performance benchmark suite bench\.default passed/
+  );
+}
diff --git a/tools/shared/performanceBenchmarks.js b/tools/shared/performanceBenchmarks.js

Original file line number	Diff line number	Diff line change
`@@ -1 +1 @@`
`1`		`-build(versioned-contracts): execute plan/build/apply bundle with shared contract policy normalization and compatibility checks`
	`1`	`+build(performance-benchmarks): execute plan/build/apply bundle with deterministic benchmark suites and regression thresholds`
Original file line number	Diff line number	Diff line change
`@@ -1 +1 @@`
`1`		`-Create PR_PERFORMANCE_BENCHMARKS_FULL_bundle`
	`1`	`+Create PR_EXTERNAL_DOCUMENTATION_FULL_bundle`