feat: add self-reflection agent, js-fact-checking and simulation skills#184
Merged
YunchuWang merged 2 commits intomainfrom Apr 28, 2026
Merged
feat: add self-reflection agent, js-fact-checking and simulation skills#184YunchuWang merged 2 commits intomainfrom
YunchuWang merged 2 commits intomainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new “self-reflection” automation agent and supporting “fact-checking”/“simulation” skills to improve the quality and durability of Copilot-authored changes and documented lessons in this repo.
Changes:
- Introduces a self-reflection agent spec and a manual GitHub Actions workflow to run it against recently merged PRs.
- Adds two new skills docs: JS/TS/Node.js fact-checking and empirical simulation.
- Updates existing agent prompts (daily-code-review, pr-verification) to require fact-checking/simulation for runtime-behavior assumptions.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/self-reflection.yaml | New workflow to gather merged PR context and run the self-reflection agent in CI. |
| .github/agents/self-reflection.agent.md | New agent definition and end-to-end process for extracting/deduping lessons into copilot-instructions.md. |
| .github/skills/js-fact-checking/SKILL.md | New skill documentation for verifying JS/TS/Node behavior claims against official sources. |
| .github/skills/simulation/SKILL.md | New skill documentation for running minimal repro scripts when docs are inconclusive. |
| .github/agents/daily-code-review.agent.md | Adds requirements to fact-check/simulate runtime-behavior assumptions before committing fixes. |
| .github/agents/pr-verification.agent.md | Adds requirements to fact-check/simulate runtime-behavior assumptions in verification samples. |
You can also share your feedback on Copilot code review. Take the survey.
YunchuWang
force-pushed
the
bill/self-reflection-agent
branch
from
01baaa6 to
0b4f618
Compare
Member
Author
|
This PR is behind main by 3 commits. Please rebase on latest main before review. |
- Add self-reflection agent that extracts lessons from merged PRs and proposes updates to copilot-instructions.md - Add GitHub Action workflow (workflow_dispatch) to trigger self-reflection with configurable scan window - Add js-fact-checking skill for verifying JS/TS/Node.js behavioral claims against official documentation before committing them - Add simulation skill for empirically verifying runtime behavior when fact-checking is inconclusive (writes and runs minimal reproduction scripts) - Update daily-code-review agent to use both skills when making fixes that depend on specific runtime behavior - Update pr-verification agent to fact-check behavioral assumptions in verification samples
YunchuWang
force-pushed
the
bill/self-reflection-agent
branch
from
0b4f618 to
5187ecf
Compare
- Use scan_labels input in gh pr list query (was declared but unused) - Add copilot CLI install step to workflow (prevents failure on fresh runner) - Change simulation template from ESM import to CommonJS require (matches repo's module system) - Replace ts-node recommendation with plain JS default and Node.js --experimental-strip-types for TS (no external tooling dependency) - Replace non-existent fetch_webpage tool reference with runTerminal + curl Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
kaibocai
approved these changes
Apr 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds three new Copilot agent/skill capabilities and a GitHub Action workflow:
1. Self-Reflection Agent (.github/agents/self-reflection.agent.md)
An autonomous agent that mines merged PRs and their review conversations for high-value lessons learned, then proposes surgical updates to .github/copilot-instructions.md.
Key features:
2. Self-Reflection GitHub Action (.github/workflows/self-reflection.yaml)
A \workflow_dispatch\ workflow that triggers the self-reflection agent on demand with:
3. JS Fact-Checking Skill (.github/skills/js-fact-checking/SKILL.md)
A skill that agents invoke when making claims about JS/TS/Node.js runtime behavior.
4. Simulation Skill (.github/skills/simulation/SKILL.md)
A fallback skill when fact-checking can't provide high confidence. Writes and runs a minimal reproduction script in a temporary directory to empirically verify runtime behavior.
5. Updated Existing Agents
Testing