Skip to content

Make the compaction group graceful period configurable per block range - #7916

Merged
yeya24 merged 5 commits into
cortexproject:masterfrom
yeya24:compactor-per-range-compaction-delay
Nov 6, 2026
Merged

yeya24 merged 5 commits into
cortexproject:masterfrom
yeya24:compactor-per-range-compaction-delay

Conversation

@yeya24

@yeya24 yeya24 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does:

Makes the compaction group graceful period added in #7182 configurable per block range, via a new -compactor.block-ranges-compaction-delay.

#7182 holds a compaction group back until its time range ended more than twice -compactor.cleanup-interval ago. That period is the same for every entry of -compactor.block-ranges, and the gate only looks at rangeEnd, so the group for the last sub-range of a larger range expires at the same moment as the larger range's own group. With the default 2h,12h,24h, the 2h group for [10:00, 12:00) and the 12h group for [00:00, 12:00) become eligible together.

A larger range can therefore start compacting while its last sub-range is still collecting level-1 blocks. The blocks that arrived in time are compacted into the larger range's result and marked for deletion; the ones that arrived afterwards can no longer join it and end up in a second, independent group covering the same range. Both results are valid blocks for the same time range, which is fine for querying but does mean a consumer that aggregates blocks per time range sees the same range represented twice.

The new flag is a list matched to -compactor.block-ranges by position, so a larger range can be configured to wait for its sub-ranges to settle first.

Behaviour and compatibility:

  • Leaving the flag empty keeps today's behaviour exactly: every range falls back to 2 * -compactor.cleanup-interval. That fallback is also what any range without a configured entry uses, so the coupling to the cleanup interval (and therefore to the bucket index refresh rate, which is the second reason given in Add graceful period for compaction groups #7182) is preserved when the flag is not set.
  • A configured 0 means no delay, and is distinct from leaving the list empty.
  • Validate rejects a list that is non-empty and does not have one entry per block range, and rejects negative entries.
  • The delay is a lower bound. Groups only become eligible on a compaction run, so the effective wait can be up to -compactor.compaction-interval longer. The flag description says so.
  • Only the partitioning compaction strategy is affected; ShuffleShardingGrouper has no such gate.

Which issue(s) this PR fixes:

NA

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Per GENAI_POLICY.md: this change was drafted with an AI coding assistant. I have reviewed every line, verified the behaviour against the tests below, and I take responsibility for it under the DCO.

Tests added: TestPartitionCompactionGrouper_CompactionDelayForRange (per-range lookup, empty-list fallback, configured zero, unknown range, partially configured list), TestPartitionCompactionGrouper_GroupBlocksByRange_CompactionDelay and ..._CompactionDelayOfNonFirstRange (the gate end to end for the 2h and 12h ranges, including a group 1ms past its delay), TestPartitionCompactionGrouper_GroupBlocksByRange_DefaultDelayFollowsCleanupInterval (unconfigured still tracks the cleanup interval), and TestConfig_Validate cases for the length and negative-value checks.

The partitioning grouper holds a compaction group back until its time range
ended more than twice the cleanup interval ago. That graceful period is the
same for every block range, so the group for the last sub-range of a larger
range expires at the same moment as the larger range's own group. A larger
range can therefore start compacting while its last sub-range is still
collecting blocks, which splits the result into a group compacted early and a
second group for the blocks that arrived afterwards.

Add -compactor.block-ranges-compaction-delay, a list matched to
-compactor.block-ranges by position, so a larger range can be made to wait for
its sub-ranges to settle. Leaving it empty keeps the existing behaviour: every
range falls back to twice the cleanup interval, which is also what any range
without a configured entry uses.

The delay is a lower bound. Groups only become eligible on a compaction run,
so the effective wait can be up to the compaction interval longer.

Signed-off-by: Ben Ye <benye@amazon.com>
@yeya24
yeya24 marked this pull request as ready for review November 5, 2026 23:56
@yeya24
yeya24 requested a review from a team as a code owner November 5, 2026 23:56
@yeya24
yeya24 requested a review from danielblando November 5, 2026 23:56
@yeya24
yeya24 merged commit 1a21b8e into cortexproject:master Nov 6, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants