dra: trigger bump-logstash workflow from version bump pipeline

[fr4nc1sc0-r4m0n]

Summary

Continues the work from #18886, which cannot be updated directly because it was opened from a fork. This branch applies the original intent plus review feedback from that PR.

• Wire .buildkite/version_bump_pipeline.yml to trigger bump-logstash.yml via the elastic/gh-cli Buildkite plugin for minor and patch release-day bumps
• Use upstream trigger parameters only: NEW_VERSION, BRANCH, and WORKFLOW (no RELEASE_TRACK; release-eng does not supply it)
• Make logstash_release_track optional in bump-logstash.yml and skip updating versions.yml when it is not provided (per @donoghuc review on dra: support bumping releases minor/patch using the existing tools #18886)

Changes from #18886

Original PR	This PR
Triggered version_bumps.yml (gem dependency bumps)	Triggers bump-logstash.yml (stack version bumps)
Removed team-member safeguard from version_bumps.yml	No change to version_bumps.yml
Required logstash_release_track	Optional; omitted by automated pipeline

Test plan

• Verify Buildkite pipeline YAML is valid
• Confirm bump-logstash.yml accepts dispatch with only logstash_version and logstash_branch
• Dry-run version bump pipeline in Buildkite with WORKFLOW=patch and WORKFLOW=minor test inputs

Related issue: https://github.com/elastic/observability-robots/issues/3404

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• run exhaustive tests : Run the exhaustive tests Buildkite pipeline.

[mergify]

This pull request does not have a backport label. Could you fix it @fr4nc1sc0-r4m0n? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• If no backport is necessary, please add the backport-skip label

[github-actions]

TL;DR

The x-pack integration step appears to have been terminated by Buildkite infrastructure (exit status: -1) while :logstash-xpack:rubyIntegrationTests was still running. I don’t see evidence tying this to the PR changes, which only touch the version-bump Buildkite/GitHub Actions/updatecli wiring.

Remediation

• Retry the failed Buildkite x-pack integration step/build.
• If it fails again, capture the complete Buildkite job log or agent termination reason; the available log ends before RSpec/Gradle reports a terminal failure.

Investigation details

Root Cause

This is classified as Infrastructure / inconclusive termination rather than a code bug. The failed job summary reports Exit status: -1, and the only available log file stops abruptly at 97% EXECUTING while org.logstash.xpack.test.RSpecIntegrationTests is still running.

The repeated expected: > 0 / got: 0 messages in the log are from the monitoring shared example retry loop, not the final job failure. In x-pack/qa/integration/support/shared_examples.rb:19-22, the Elasticsearch monitoring-doc assertion is wrapped in Stud.try(max_retry.times, retryable_errors), and the log later moves on to the next example/context.

The PR diff changes these files only:

• .buildkite/version_bump_pipeline.yml
• .ci/updatecli/bump-logstash-version.yml
• .github/workflows/bump-logstash.yml

Those changes do not overlap the x-pack monitoring integration specs shown in the failing log.

Evidence

• Build: https://buildkite.com/elastic/logstash-pull-request-pipeline/builds/5028
• Job/step: :lab_coat: x-pack integration
• Failed job summary: State: failed Exit status: -1
• Key log excerpt:

:logstash-xpack:rubyIntegrationTests > Executing test org.logstash.xpack.test.RSpecIntegrationTests
Failed (expected: > 0
     got:   0). Sleeping for 0.32
        records metrics on es

Monitoring TLS hot-reload
  config.reload.automatic: false
Running logstash with bin/logstash -f ... timeout: 60
<============-> 97% EXECUTING [4m 32s]

Verification

• Checked PR files/diff and the referenced x-pack monitoring specs.
• Searched for an existing flaky-test issue matching this symptom; none found.
• Did not run the full x-pack integration suite locally because it requires the Buildkite Elasticsearch integration environment and the captured failure is an infra-style termination rather than a reproducible assertion failure.

Follow-up

If the retry produces a complete RSpec failure summary instead of another -1 termination, the next investigation should start from that final failure block.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[fr4nc1sc0-r4m0n]

buildkite build this

[infra-vault-gh-plugin-prod]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 62a6743

Failed CI Steps

• 🥼 IT Persistent Queues / part 4-of-6
• 🥼 x-pack integration

History

• 💔 Build #5028 failed 62a6743

cc @fr4nc1sc0-r4m0n