Skip to content

Add e2e lifecycle coverage for existing installs#183

Open
markcallen wants to merge 5 commits into
mainfrom
fix/claude-skill-patch-fallback
Open

Add e2e lifecycle coverage for existing installs#183
markcallen wants to merge 5 commits into
mainfrom
fix/claude-skill-patch-fallback

Conversation

@markcallen
Copy link
Copy Markdown
Contributor

@markcallen markcallen commented May 15, 2026

Summary

  • add standalone scripts/e2e-*.sh coverage for existing-install lifecycle and upgrade flows
  • reconcile stale managed agent/skill artifacts and keep support-file sections in sync during wrapper refreshes
  • extend doctor output to report taskSystem and update backend coverage for skill refresh and unreadable Claude archive recovery

Testing

  • ./scripts/run-unit-tests-pre-push.sh except the final CLI go test step needed GOCACHE=/tmp/go-build-cli because the default cache path was read-only in this environment
  • env GOCACHE=/tmp/go-build-cli go test ./... in cli/ballast
  • go test ./... in packages/ballast-go
  • uv run --python 3.12 python -m unittest tests.test_cli in packages/ballast-python
  • pnpm --filter @everydaydevopsio/ballast run test -- doctor.test.ts
  • for script in ./scripts/e2e-*.sh; do "$script"; done

Follow-up

Copilot AI review requested due to automatic review settings May 15, 2026 15:16
@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

❌ Patch coverage is 75.71885% with 76 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cli/ballast/main.go 68.69% 48 Missing and 19 partials ⚠️
packages/ballast-typescript/src/install.ts 88.57% 8 Missing ⚠️
packages/ballast-typescript/src/doctor.ts 66.66% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates install patch behavior for Claude .skill archives so unreadable existing archives can be replaced with canonical packaged skill content instead of failing the install.

Changes:

  • Adds Claude skill patch fallback logic in TypeScript and Python installers.
  • Adds regression tests for unreadable Claude .skill archives in TypeScript and Python.
  • Documents the recovery behavior in PRD.md.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
PRD.md Adds the unreadable Claude .skill archive recovery requirement and acceptance criterion.
packages/ballast-typescript/src/install.ts Introduces TypeScript fallback handling for Claude skill archive patching.
packages/ballast-typescript/src/install.test.ts Adds TypeScript regression coverage for invalid Claude .skill archive replacement.
packages/ballast-python/ballast/cli.py Adds Python fallback handling for Claude skill archive patching and changes common agent registration.
packages/ballast-python/tests/test_cli.py Adds Python regression coverage for invalid Claude .skill archive replacement.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/ballast-python/ballast/cli.py Outdated
Comment thread PRD.md
Comment thread packages/ballast-typescript/src/install.ts Outdated
Comment thread packages/ballast-typescript/src/install.ts
@markcallen markcallen changed the title Handle unreadable Claude skill archives during patch Add e2e lifecycle coverage for existing installs May 15, 2026
@markcallen markcallen requested a review from Copilot May 15, 2026 21:16
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 5 comments.

Comment thread packages/ballast-go/cmd/ballast-go/main.go
Comment thread packages/ballast-python/ballast/cli.py Outdated
Comment thread cli/ballast/main.go Outdated
Comment thread cli/ballast/main.go
Comment thread cli/ballast/main.go Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 6 comments.

Comment thread cli/ballast/main.go
Comment thread packages/ballast-typescript/src/install.ts
Comment thread packages/ballast-go/cmd/ballast-go/main.go
Comment thread cli/ballast/main.go
Comment thread packages/ballast-typescript/src/install.ts Outdated
Comment thread packages/ballast-typescript/src/install.test.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (3)

cli/ballast/main.go:2282

  • For cursor/opencode, this condition allows deletion solely because the target matches, without checking that the stale skill was present in the previous .rulesrc.json or otherwise owned by Ballast. Since removeStaleManagedFiles builds candidates from every supported skill name, a user-authored .cursor/rules/<supported-skill>.mdc or .opencode/skills/<supported-skill>.md that was never managed by Ballast can be removed during refresh, violating the requirement to avoid deleting unrelated user files.
	for _, file := range stringDifference(allManagedSkillPaths(root, target), managedSkillPaths(root, target, next.Skills)) {
		if !ballastOwnsManagedFile(file) && !trackedPaths[file] && !allowConfigBackedStaleSkillRemoval(target, file) {
			continue

cli/ballast/main.go:2730

  • Claude .skill archives generated by the backends currently contain the raw skill SKILL.md without the new Created by [Ballast] marker, so this ownership check returns false unless the path also appears in a managed support-file section. If a support file was disabled, skipped, deleted, or preserved as unmanaged, refresh reconciliation will leave stale Claude skill archives behind even though they were Ballast-installed.
	if strings.HasSuffix(path, ".skill") {
		reader, err := zip.NewReader(bytes.NewReader(content), int64(len(content)))
		if err != nil {
			return false
		}
		for _, file := range reader.File {
			if file.Name != "SKILL.md" {
				continue
			}
			rc, err := file.Open()
			if err != nil {
				return false
			}
			data, readErr := io.ReadAll(rc)
			_ = rc.Close()
			if readErr != nil {
				return false
			}
			return strings.Contains(string(data), ballastManagedMarker)

cli/ballast/main.go:2273

  • Stale rule cleanup cannot remove cursor/opencode agent rules because those targets have no support-file tracking and generated agent rule files do not include the Created by [Ballast] marker used by ballastOwnsManagedFile. As a result, removing a saved agent and refreshing a cursor/opencode install leaves the old managed rule files in place instead of reconciling the retained target.
	for _, file := range stringDifference(allManagedRulePaths(root, target), managedRulePaths(root, target, next)) {
		if !ballastOwnsManagedFile(file) && !trackedPaths[file] {
			continue

Comment thread cli/ballast/main.go Outdated
Comment thread scripts/e2e-doctor-after-lifecycle-change.sh Outdated
Comment thread cli/ballast/main.go
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 4 comments.

Comment thread cli/ballast/main.go
Comment on lines +2348 to +2350
for _, file := range stringDifference(allManagedSkillPaths(root, target), managedSkillPaths(root, target, next.Skills)) {
if !ballastOwnsManagedFile(file) && !trackedPaths[file] && !allowConfigBackedStaleSkillRemoval(target, file) {
continue

PROJECT="${WORKDIR}/doctor-after-lifecycle-change"
create_monorepo_fixture "${PROJECT}"
add_typescript_profile "${PROJECT}"
script: ./scripts/e2e-add-task-system.sh
needs_typescript: true
- name: doctor-after-lifecycle-change-e2e
script: ./scripts/e2e-doctor-after-lifecycle-change.sh

(
cd "${PROJECT}"
ballast install --target codex --agent linting --yes >/dev/null
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Audit smoke and e2e tests for naming, scope, best practices, and trigger policy

2 participants