feat: add strategy-aware benchmark artifacts#36
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 339e2e9655
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const updatedSession: GenerationSession = { | ||
| ...sessionForHandoff, | ||
| }; | ||
| writeDeterministicJsonSync(paths.sessionPath, updatedSession); |
There was a problem hiding this comment.
Keep session strategy immutable when preparing handoff
This command persists sessionForHandoff back to session.json, which includes the override values for guidanceStrategy and baseHandoffPath; if prepare-generation-handoff is run after attempts exist, it can retroactively relabel the session strategy and handoff path for all later summaries/comparisons even though earlier attempts were produced under a different strategy. Because buildGenerationSessionSummary reports the session-level strategy, this mutates benchmark evidence instead of just emitting an alternate handoff artifact.
Useful? React with 👍 / 👎.
| .requiredOption("--workspace-root <path>", "Workspace root for emitted run artifacts") | ||
| .option("--tool <tool>", "Generation tool identifier (codex|cursor)") | ||
| .option("--guidance-mode <mode>", "Session guidance mode (prepared|unguided)") | ||
| .option("--tool <tool>", "Generation tool identifier (codex|cursor|local-llm)") |
There was a problem hiding this comment.
Align advertised init tool values with runtime validation
init-generation-session help now claims --tool accepts local-llm, but ensureSessionTool still only allows codex|cursor, so using the documented local-llm value fails with an input error. This creates a broken CLI contract for users who rely on --help output.
Useful? React with 👍 / 👎.
|
Holding this PR as superseded-in-place for now. The current branch is stale and the repo is still receiving a standalone failing |
339e2e9 to
fc31496
Compare
|
Refreshed this PR against current Local verification on the refreshed branch:
Current blocker is still external to the branch content: GitHub is attaching a failing plain |
Summary
Testing