feat(agent): guide agents to use --no-agent when authoring user-run scripts#502
Open
platinummonkey wants to merge 1 commit into
Open
feat(agent): guide agents to use --no-agent when authoring user-run scripts#502platinummonkey wants to merge 1 commit into
platinummonkey wants to merge 1 commit into
Conversation
…cripts
Pup auto-detects AI agents and wraps JSON output in {status, data, metadata}.
When an agent writes a script for the user to run outside the agent session,
the envelope vanishes and the script breaks — leaving the agent insisting the
script "works" because it tested under agent mode.
Two complementary signals are added so agents pick the right flag:
- Runtime: every agent-mode envelope now includes `metadata.note` reminding
the agent to append --no-agent when authoring scripts the user will run.
Always present regardless of whether the caller supplied Metadata.
- Discovery: both the full and per-domain agent schemas now expose a
`script_authoring` block (summary/rule/examples/detection), and the
anti_patterns array references it so agents that scan anti-patterns first
are still led to the full guidance.
LLM_GUIDE.md gains a parallel "Authoring scripts the user will run" section
so `pup agent guide` carries the same warning.
- src/formatter.rs: new AGENT_ENVELOPE_NOTE constant; extracted
build_agent_envelope() for unit-testability; format_and_print() delegates
to it. 3 new tests (with-meta, without-meta, data-hoist interaction).
- src/main.rs: new build_script_authoring_guidance() inserted into
build_agent_schema and build_agent_schema_scoped. 4 new tests + shared
assert_script_authoring_contract helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pup auto-detects AI agents and wraps JSON output in
{status, data, metadata}. When an agent writes a script for the user to run later (outside the agent session), the envelope vanishes and the script breaks — and the agent confidently insists the script "works" because it tested it under agent mode.This adds two complementary signals so agents reliably append
--no-agentwhen authoring user-run scripts:metadata.notewith the rule and a pointer to--no-agent. Present on every response, regardless of whether the caller suppliedMetadata.pup --help/pup logs --help/pup agent schema) now expose ascript_authoringblock (summary/rule/examples/detection). Theanti_patternsarray in both schemas points at it so agents that scan anti-patterns first are still led to the full guidance.Changes
src/formatter.rs:24— newpub const AGENT_ENVELOPE_NOTE.src/formatter.rs:56— extractedbuild_agent_envelope()for unit-testability;format_and_print()delegates to it. Every agent-mode envelope now always hasmetadata.note.src/main.rs:9199— newbuild_script_authoring_guidance()helper.src/main.rs:9346, 9419— anti-pattern reference appended in both schema builders.src/main.rs:9349, 9422—script_authoringinserted into both schemas.docs/LLM_GUIDE.md:247— example envelope JSON updated to shownote; new "Authoring scripts the user will run" subsection.Testing
8 new tests, all passing:
formatter::tests::test_agent_envelope_injects_script_authoring_note_with_meta— with caller-supplied Metadata, assertsmetadata.note == AGENT_ENVELOPE_NOTEand contains--no-agent.formatter::tests::test_agent_envelope_injects_script_authoring_note_without_meta— withmeta = None, asserts the note still appears and the constant itself contains--no-agent.formatter::tests::test_agent_envelope_hoists_inner_data_and_keeps_note— pins the interaction between the existingdatahoist and the new injection.test_agent_schema::schema_includes_script_authoring_guidance+scoped_schema_includes_script_authoring_guidance— assert full contract (summary/rule/examples/detection) via sharedassert_script_authoring_contracthelper.test_agent_schema::schema_anti_patterns_reference_script_authoring+scoped_schema_anti_patterns_reference_script_authoring— both schemas referencescript_authoringinanti_patterns.Gates:
cargo fmt --check: cleancargo clippy --all-targets -- -D warnings: cleancargo test: all new tests pass🤖 Generated with Claude Code