[ML] Automate patch version bump in CI pipeline#3030
Draft
edsavage wants to merge 8 commits intoelastic:mainfrom
Draft
[ML] Automate patch version bump in CI pipeline#3030edsavage wants to merge 8 commits intoelastic:mainfrom
edsavage wants to merge 8 commits intoelastic:mainfrom
Conversation
Replace the version-bump pipeline stub with a patch-only flow: Slack notification, Wolfi step running dev-tools/bump_version.sh to bump elasticsearchVersion on BRANCH (and .backportrc.json on main), then json-watcher polling for staging and snapshot artifact versions. Supports DRY_RUN=true to skip git push. Minor-branch automation will follow in a separate change. Made-with: Cursor
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
- Add send_slack_version_bump_notification.sh (version-bump pipeline only) with ML_CPP_VERSION_BUMP_TEST_MODE banner and optional channel override. - Wire job-version-bump.json.py to use it; support ML_CPP_VERSION_BUMP_SKIP_DRA_WAIT for short smoke runs without json-watcher polling. Made-with: Cursor
Widen ml-cpp-version-bump branch_configuration for PR elastic#3030 smoke tests; revert to main-only after validation. Made-with: Cursor
The temporary ml-cpp-version-bump branch_configuration change is proposed separately so PR elastic#3030 stays focused on the bump automation. Made-with: Cursor
There was a problem hiding this comment.
Pull request overview
This PR updates the ML CI “version bump” pipeline from a stubbed/manual flow to an automated patch-version bump workflow that (1) notifies Slack, (2) bumps elasticsearchVersion on a target branch, and (3) optionally polls published artifact JSON until the expected versions appear.
Changes:
- Add
dev-tools/bump_version.shto bumpelasticsearchVersion(and update.backportrc.jsonwhen targetingmain) with optionalDRY_RUN=true. - Add a dedicated Buildkite pipeline YAML generator for Slack notifications in the version-bump pipeline.
- Update
.buildkite/job-version-bump.json.pyto run the bump step and then poll staging/snapshot artifact JSON viaelastic/json-watcher.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
dev-tools/bump_version.sh |
New automation script to bump elasticsearchVersion, commit, and push (with DRY_RUN support). |
.buildkite/pipelines/send_slack_version_bump_notification.sh |
New pipeline snippet generator to configure Slack notifications specifically for version bump runs. |
.buildkite/job-version-bump.json.py |
Replaces the old stub/block flow with bump + (optional) artifact polling steps. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+92
to
+93
| if v == 'main' and re.match(r'\^v\d+\.\d+\.\d+\\\$', k): | ||
| new_mapping['^v${target_version}\$'] = 'main' |
There was a problem hiding this comment.
The .backportrc.json update logic writes an incorrect mapping key: the regex checks for keys ending in \$, and the new key is written as ^v${target_version}\$. In this repo .backportrc.json uses keys like ^v9.5.0$ (no backslash), so the match will never trigger and (if it did) the script would write a key that backportr likely won’t match. Update the regex to match literal ^/$ correctly and write the new key without a backslash before $.
Suggested change
| if v == 'main' and re.match(r'\^v\d+\.\d+\.\d+\\\$', k): | |
| new_mapping['^v${target_version}\$'] = 'main' | |
| if v == 'main' and re.match(r'^\^v\d+\.\d+\.\d+\$$', k): | |
| new_mapping['^v${target_version}$'] = 'main' |
Patch-only version bumps update gradle.properties only; main/minor backport mapping stays out of this PR. Made-with: Cursor
Remove ML_CPP_VERSION_BUMP_TEST_MODE / TEST banner and ML_CPP_VERSION_BUMP_SKIP_DRA_WAIT (DRA wait steps always run). DRY_RUN remains a normal Buildkite env for bump_version.sh. Optional ML_CPP_VERSION_BUMP_SLACK_CHANNEL retained for routing. Made-with: Cursor
Runs before Slack queue and bump so CI credentials are validated without creating or updating remote refs. Made-with: Cursor
workflow_dispatch with branch/new_version/dry_run; GitHub App token for checkout and git push (addresses Buildkite Vault bot 403 class of issues). Document suggested secrets in workflow header. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the version-bump pipeline stub with a patch-only flow: Slack notification, Wolfi step running
dev-tools/bump_version.shto bumpelasticsearchVersionon BRANCH, then json-watcher polling for staging and snapshot artifact versions.Supports DRY_RUN=true to skip git push. Minor-branch automation will follow in a separate change.
Made-with: Cursor