[server][pubsub] PubSub health OTel metric entity - #2930
Open
misyel wants to merge 5 commits into
Open
Conversation
…etric dimension First of a stacked series splitting the PubSub Health Monitoring feature (linkedin#2534) into reviewable chunks. Introduces only the foundational, behavior-free building blocks — no wiring, all default-off. - New pubsub health types (venice-common): - PubSubHealthStatus (HEALTHY/UNHEALTHY) - PubSubHealthCategory (BROKER/METADATA_SERVICE) as a metric dimension - PubSubHealthSignalProvider (pluggable detection interface) - PubSubHealthChangeListener (state-transition callback interface) - New config keys (all default-off) + VeniceServerConfig parsing: - server.pubsub.health.monitor.enabled - server.pubsub.partition.pause.enabled - server.pubsub.health.probe.interval.seconds - server.pubsub.health.probe.topic - New OTel dimension VENICE_PUBSUB_HEALTH_CATEGORY (required by PubSubHealthCategory) + dimension unit tests. No call sites yet; compiles and tests independently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Second of the stacked series splitting the PubSub Health Monitoring feature (linkedin#2534). Stacks on the foundation PR. - PubSubHealthOtelMetricEntity: defines the OTel metric entities for pubsub health (unhealthy count, paused partition count, probe success/failure, state transitions), keyed by the VENICE_PUBSUB_HEALTH_CATEGORY dimension from the foundation PR. - Registers PubSubHealthOtelMetricEntity in ServerMetricEntity so the entities are emitted with the rest of the server metrics. Metric entities only — the monitor that populates them lands in a later PR. Compiles independently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds foundational PubSub health types/configs and introduces the server-side OpenTelemetry metric entity definitions for PubSub health monitoring, registering them so they are emitted with existing server metrics.
Changes:
- Introduces PubSub health core types (
PubSubHealthStatus,PubSubHealthCategory, signal provider, change listener) and config keys + server config parsing. - Adds a new OTel dimension
VENICE_PUBSUB_HEALTH_CATEGORYwith unit tests. - Adds
PubSubHealthOtelMetricEntityand registers it inServerMetricEntity.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| internal/venice-common/src/main/java/com/linkedin/venice/pubsub/PubSubHealthStatus.java | Adds enum for PubSub target health state. |
| internal/venice-common/src/main/java/com/linkedin/venice/pubsub/PubSubHealthSignalProvider.java | Adds pluggable health-signal provider interface (doc wording issue noted). |
| internal/venice-common/src/main/java/com/linkedin/venice/pubsub/PubSubHealthChangeListener.java | Adds callback interface for health state transitions. |
| internal/venice-common/src/main/java/com/linkedin/venice/pubsub/PubSubHealthCategory.java | Adds category enum and binds it to the new OTel dimension. |
| internal/venice-common/src/main/java/com/linkedin/venice/ConfigKeys.java | Adds PubSub health monitor/pause/probe config keys. |
| internal/venice-client-common/src/main/java/com/linkedin/venice/stats/dimensions/VeniceMetricsDimensions.java | Adds VENICE_PUBSUB_HEALTH_CATEGORY dimension constant. |
| internal/venice-client-common/src/test/java/com/linkedin/venice/stats/dimensions/VeniceMetricsDimensionsTest.java | Extends dimension naming-format tests for the new dimension. |
| internal/venice-client-common/src/test/java/com/linkedin/venice/stats/dimensions/PubSubHealthCategoryTest.java | Adds fixture-based tests for the new dimension enum. |
| clients/da-vinci-client/src/main/java/com/linkedin/davinci/stats/ServerMetricEntity.java | Registers the new PubSub health metric entity with server metrics aggregation. |
| clients/da-vinci-client/src/main/java/com/linkedin/davinci/stats/PubSubHealthOtelMetricEntity.java | Defines PubSub health OTel metric entities (gauges/counters/latency). |
| clients/da-vinci-client/src/main/java/com/linkedin/davinci/config/VeniceServerConfig.java | Parses and exposes PubSub health monitor-related configs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+4
to
+5
| * A pluggable provider of health signals for PubSub targets. The PubSub health monitor | ||
| * aggregates signals from all registered providers to determine overall broker health. |
configs Covers the diffCoverage gate for da-vinci-client: exercises the new pubsub-health config getters and both sides of the `pubSubHealthMonitorEnabled && ...` short-circuit in partition-pause resolution (monitor-on honors pause, monitor-off gates it off), plus the default values. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
internal/venice-common/src/main/java/com/linkedin/venice/pubsub/PubSubHealthSignalProvider.java:6
- This Javadoc says the monitor aggregates signals to determine overall "broker" health, but the API supports multiple categories (broker and metadata service). The comment should describe "target" health to avoid misleading implementers.
* A pluggable provider of health signals for PubSub targets. The PubSub health monitor
* aggregates signals from all registered providers to determine overall broker health.
*
Comment on lines
+14
to
+15
|
|
||
| public enum PubSubHealthOtelMetricEntity implements ModuleMetricEntityInterface { |
…health PubSubHealthOtelMetricEntity adds 7 new entities (185 -> 192), so the hardcoded count assertion in ServerMetricEntityTest must be bumped. The registration/no-duplicate/no-null tests already pass, confirming all 7 are legitimately registered. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
internal/venice-common/src/main/java/com/linkedin/venice/pubsub/PubSubHealthSignalProvider.java:6
- The Javadoc says the monitor aggregates signals to determine overall "broker health", but this interface models health for a PubSub target address across categories (BROKER vs METADATA_SERVICE). The wording is misleading for readers/implementers and should be generalized.
* A pluggable provider of health signals for PubSub targets. The PubSub health monitor
* aggregates signals from all registered providers to determine overall broker health.
*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second of the stacked series splitting the PubSub Health Monitoring feature (#2534). Stacks on #2929 (foundation).
What's included
PubSubHealthOtelMetricEntity— defines the OTel metric entities for pubsub health (unhealthy count, paused partition count, probe success/failure, state transitions), keyed by theVENICE_PUBSUB_HEALTH_CATEGORYdimension introduced in [server][pubsub] PubSub health foundation: core types, config keys, metric dimension #2929.PubSubHealthOtelMetricEntityinServerMetricEntityso the entities are emitted with the rest of the server metrics.Metric entities only — the monitor that populates them lands in a later PR.
Stack
Decomposition of #2534:
PubSubHealthMonitor+ signal provider + stats)How was this PR tested?
da-vinci-client).Does this PR introduce any user-facing or breaking changes?