Skip to content

Fix duplicate metric registration during concurrent pipeline state builds #26187

Open
patrickmann wants to merge 2 commits into
masterfrom
fix/pipeline-duplicate-metric-registration
Open

Fix duplicate metric registration during concurrent pipeline state builds #26187
patrickmann wants to merge 2 commits into
masterfrom
fix/pipeline-duplicate-metric-registration

Conversation

@patrickmann

@patrickmann patrickmann commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Closes #26080

PipelineInterpreter.State registers cache metrics via a non-atomic removeMatching + safelyRegisterAll sequence. Two independent code paths (PipelineInterpreterStateUpdater and PipelineMetadataUpdateJob) can construct State concurrently, causing Thread B to register metrics between Thread A's remove and register. This results in noisy ERROR logs: Duplicate metric set registered.

The fix adds a static lock to synchronize the compound remove+register operation. This is acceptable since updates are relatively infrequent.

How Tested

Added concurrent regression test StateTest.concurrentStateConstructionDoesNotCauseDuplicateMetricErrors

Manual test: deploy to a test environment, trigger rapid pipeline config changes (e.g. bulk rule updates), verify no "Duplicate metric set registered" ERROR in server logs

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Refactoring (non-breaking change)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have requested a documentation update.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.

…ilds (#26080)

Synchronize the remove+register metric sequence in PipelineInterpreter.State
to prevent concurrent State constructions from racing on the shared MetricRegistry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses noisy Duplicate metric set registered ERROR logs caused by a race during concurrent construction of PipelineInterpreter.State, where cache metrics were removed and re-registered via a non-atomic sequence.

Changes:

  • Add a static lock in PipelineInterpreter.State to synchronize the metric removeMatching + safelyRegisterAll compound operation.
  • Add a concurrent regression test intended to exercise concurrent State construction.
  • Add an unreleased changelog entry for issue #26080.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
graylog2-server/src/main/java/org/graylog/plugins/pipelineprocessor/processors/PipelineInterpreter.java Serializes cache metric remove+register to prevent concurrent duplicate registrations/log spam.
graylog2-server/src/test/java/org/graylog/plugins/pipelineprocessor/processors/StateTest.java Adds a concurrent construction regression test and switches to MetricRegistry.
changelog/unreleased/issue-26080.toml Records the fix in the unreleased changelog.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@patrickmann patrickmann marked this pull request as ready for review June 1, 2026 14:20
@patrickmann patrickmann requested a review from a team June 1, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Duplicate metric registration error during pipeline metadata updates

2 participants