Skip to content

feat(agent): guide agents to use --no-agent when authoring user-run scripts#502

Open
platinummonkey wants to merge 1 commit into
mainfrom
feat/agent-script-authoring-guidance
Open

feat(agent): guide agents to use --no-agent when authoring user-run scripts#502
platinummonkey wants to merge 1 commit into
mainfrom
feat/agent-script-authoring-guidance

Conversation

@platinummonkey
Copy link
Copy Markdown
Collaborator

@platinummonkey platinummonkey commented May 13, 2026

Summary

Pup auto-detects AI agents and wraps JSON output in {status, data, metadata}. When an agent writes a script for the user to run later (outside the agent session), the envelope vanishes and the script breaks — and the agent confidently insists the script "works" because it tested it under agent mode.

This adds two complementary signals so agents reliably append --no-agent when authoring user-run scripts:

  • Runtime: every agent-mode envelope now includes metadata.note with the rule and a pointer to --no-agent. Present on every response, regardless of whether the caller supplied Metadata.
  • Discovery: both the full and per-domain agent schemas (pup --help / pup logs --help / pup agent schema) now expose a script_authoring block (summary / rule / examples / detection). The anti_patterns array in both schemas points at it so agents that scan anti-patterns first are still led to the full guidance.

Changes

  • src/formatter.rs:24 — new pub const AGENT_ENVELOPE_NOTE.
  • src/formatter.rs:56 — extracted build_agent_envelope() for unit-testability; format_and_print() delegates to it. Every agent-mode envelope now always has metadata.note.
  • src/main.rs:9199 — new build_script_authoring_guidance() helper.
  • src/main.rs:9346, 9419 — anti-pattern reference appended in both schema builders.
  • src/main.rs:9349, 9422script_authoring inserted into both schemas.
  • docs/LLM_GUIDE.md:247 — example envelope JSON updated to show note; new "Authoring scripts the user will run" subsection.

Testing

8 new tests, all passing:

  • formatter::tests::test_agent_envelope_injects_script_authoring_note_with_meta — with caller-supplied Metadata, asserts metadata.note == AGENT_ENVELOPE_NOTE and contains --no-agent.
  • formatter::tests::test_agent_envelope_injects_script_authoring_note_without_meta — with meta = None, asserts the note still appears and the constant itself contains --no-agent.
  • formatter::tests::test_agent_envelope_hoists_inner_data_and_keeps_note — pins the interaction between the existing data hoist and the new injection.
  • test_agent_schema::schema_includes_script_authoring_guidance + scoped_schema_includes_script_authoring_guidance — assert full contract (summary/rule/examples/detection) via shared assert_script_authoring_contract helper.
  • test_agent_schema::schema_anti_patterns_reference_script_authoring + scoped_schema_anti_patterns_reference_script_authoring — both schemas reference script_authoring in anti_patterns.

Gates:

  • cargo fmt --check: clean
  • cargo clippy --all-targets -- -D warnings: clean
  • cargo test: all new tests pass

🤖 Generated with Claude Code

…cripts

Pup auto-detects AI agents and wraps JSON output in {status, data, metadata}.
When an agent writes a script for the user to run outside the agent session,
the envelope vanishes and the script breaks — leaving the agent insisting the
script "works" because it tested under agent mode.

Two complementary signals are added so agents pick the right flag:

- Runtime: every agent-mode envelope now includes `metadata.note` reminding
  the agent to append --no-agent when authoring scripts the user will run.
  Always present regardless of whether the caller supplied Metadata.
- Discovery: both the full and per-domain agent schemas now expose a
  `script_authoring` block (summary/rule/examples/detection), and the
  anti_patterns array references it so agents that scan anti-patterns first
  are still led to the full guidance.

LLM_GUIDE.md gains a parallel "Authoring scripts the user will run" section
so `pup agent guide` carries the same warning.

- src/formatter.rs: new AGENT_ENVELOPE_NOTE constant; extracted
  build_agent_envelope() for unit-testability; format_and_print() delegates
  to it. 3 new tests (with-meta, without-meta, data-hoist interaction).
- src/main.rs: new build_script_authoring_guidance() inserted into
  build_agent_schema and build_agent_schema_scoped. 4 new tests + shared
  assert_script_authoring_contract helper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@platinummonkey platinummonkey requested a review from a team as a code owner May 13, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant