Skip to content

Preserve user content in agent config files on coldbox ai refresh#43

Merged
lmajano merged 2 commits intodevelopmentfrom
copilot/fix-claude-file-overwrite-issue
Mar 28, 2026
Merged

Preserve user content in agent config files on coldbox ai refresh#43
lmajano merged 2 commits intodevelopmentfrom
copilot/fix-claude-file-overwrite-issue

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 28, 2026

coldbox ai refresh blindly overwrites agent config files (e.g. CLAUDE.md / AGENTS.md), destroying any user or AI-agent customizations on every run.

Approach

Introduce <!-- COLDBOX-CLI:START --> / <!-- COLDBOX-CLI:END --> demarcation markers in the generated files. On refresh, only the content between those markers is replaced; everything after the end marker is preserved.

Changes

Templates (agent-modern-instructions.md, agent-flat-instructions.md)

  • Wrap all framework-managed content in COLDBOX-CLI:START / COLDBOX-CLI:END markers
  • Move "Custom Application Details" user section to after the end marker with an explanatory comment
  • Restructure modern template so all managed content is contiguous above the user section

AgentRegistry.cfc

  • Add MANAGED_SECTION_START / MANAGED_SECTION_END static constants
  • Add mergeUserContent(filePath, newContent) — extracts the user section from an existing file and re-appends it after the refreshed managed block:
    • No existing file → write as-is (first install)
    • Existing file without end marker → overwrite (old format, no user section to preserve)
    • Existing file with end marker → replace managed section, keep everything after it
  • Wire mergeUserContent() into configureAgent() for all agent types (including Claude's AGENTS.md)

Example of resulting file structure:

<!-- COLDBOX-CLI:START -->
<!-- ⚡ Managed by ColdBox CLI — refreshed on `coldbox ai refresh`. Do not edit between markers. -->

# MyApp - AI Agent Instructions
... framework-generated content ...

<!-- COLDBOX-CLI:END -->

<!-- ℹ️ YOUR PROJECT DOCUMENTATION — ColdBox CLI will NOT overwrite this section. -->

## Custom Application Details
... user/AI-authored content survives refresh ...

⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.

Copilot AI linked an issue Mar 28, 2026 that may be closed by this pull request
@lmajano lmajano marked this pull request as ready for review March 28, 2026 11:59
Copilot AI review requested due to automatic review settings March 28, 2026 11:59
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.

Add COLDBOX-CLI:START/END markers to agent instruction templates so
that `coldbox ai refresh` only updates the managed framework section
while leaving user-authored documentation intact.

- Add <!-- COLDBOX-CLI:START --> and <!-- COLDBOX-CLI:END --> markers to
  both agent-modern-instructions.md and agent-flat-instructions.md
- Restructure templates: all managed content (guidelines, skills, MCP,
  project info) sits inside the markers; Custom Application Details
  moves to after the end marker with a note that it won't be overwritten
- Add MANAGED_SECTION_START/END static constants to AgentRegistry.cfc
- Add private mergeUserContent() helper that extracts the user section
  from an existing file and appends it to the refreshed managed section
- Update configureAgent() to call mergeUserContent() for all agents
  (including AGENTS.md for Claude) instead of blindly overwriting

Agent-Logs-Url: https://github.com/ColdBox/coldbox-cli/sessions/2b2dc668-9136-4fc3-9cec-b1d19209c61b

Co-authored-by: lmajano <137111+lmajano@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix CLAUDE.md file overwritten when AI refreshed Preserve user content in agent config files on coldbox ai refresh Mar 28, 2026
Copilot AI requested a review from lmajano March 28, 2026 12:02
@lmajano lmajano merged commit ce91d29 into development Mar 28, 2026
1 check passed
@lmajano lmajano deleted the copilot/fix-claude-file-overwrite-issue branch March 28, 2026 12:12
@homestar9
Copy link
Copy Markdown
Contributor

homestar9 commented Apr 14, 2026

@lmajano it appears to be working as expected. However, you might consider a more succinct default Agents.md file. I'm concerned that with such a large file, it's going to eat up people's context windows. I know this is is still an emerging standard, but in my research online I've seen recommendations averaging around ~300 lines.

I've been solving this problem in my new apps by providing references to additional Agents.md files elsewhere in my application. For example, I have a section in my root Agents.md file that gives the LLM instructions on where to find more context on testing guidelines:

# Testing

## Running Tests

\```bash
# Start the server
box server start

# Run all tests via browser (will take a very long time)
http://localhost:60830/tests/runner.cfm

# Run specific test file
http://localhost:60830/tests/runner.cfm?directory=tests.specs.unit&testBundles=GatewayTest

# Run tests from the CLI using the `box testbox run` command.
# Use the `bundles` argument with dot-delimited paths (no .cfc extension).
# You can run multiple test bundles at once by comma-separating them:
box testbox run runner="http://localhost:60830/tests/runner.cfm" bundles="tests.specs.unit.ScraperGatewayTest,tests.specs.unit.ClientTest" verbose=false
\```

## Writing Tests

Detailed test guidelines for AI agents can be found in the `tests/AGENTS.md` file. Reference as needed when writing tests.

Note how at the end, I point the LLM to additional information. You might consider something like this instead of listing out the entire manifest in a single file. For example:

## AI Guidelines & Skills (On-Demand)

This project uses the ColdBox AI integration (`.ai/` directory). The full manifest file for skills and guidelines is located at `.ai/manifest.json`. When you need to implement a feature or fix a bug, query the manifest for relevant guidelines and skills by name.

Just some food for thought.

@lmajano
Copy link
Copy Markdown
Member

lmajano commented Apr 15, 2026

can you give me an example on our current approach. I believe I had left it lazy loaded, so it would read the other skils when needed

@homestar9
Copy link
Copy Markdown
Contributor

homestar9 commented Apr 24, 2026

Here's what a default Agents.md file looks like when you run coldbox ai install today:

https://pastebin.com/MsxKMPxu

Note that it's about 1000 lines long. If this were my template, I would consider trimming some of the generated reference/index sections, particularly:

  • Manifest

  • Using Guidelines & Skills

  • Available Guidelines

  • Available Skills

Instead, I would replace it with a short section like this:

## AI Guidelines & Skills (On-Demand)

When needed, read `.ai/manifest.json` to discover available guidelines, skills, agents, and MCP servers. Load only the guideline or skill resources relevant to the current task instead of eagerly loading the entire AI resource catalog into context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CLAUDE.md File Overwritten When AI Refreshed

4 participants