Skip to content

Conversation

@danielblando
Copy link
Contributor

@danielblando danielblando commented Dec 11, 2025

What this PR does:
This is yet another case of double compaction which can create an unwanted group.

The fetcher when using bucketIndex strategy have a limit visibility of activeBlocks (only the blocks visible when creating the index) and already does filter the deleteMarkers which are marked in the index. If we additionally use the filter, it will also filter out recently deleted blocks which were not updated on the index yet. The cleaner try to maintan a stable version of the index to avoid causing unwanted compaction, which this filter override the index state.

Block 1 to 8 are result blocks of partitionGroup 1
How the issue can happen
T0: blockId1 to blockId7 were created
T1: Index was created
T2: blokcId8 was created completing the partitionGroup 1
T2: Cleaner started deleting parent blocks from partitionGroup 1
T3: Compactor job check the deletionMark for blocks in index
T4: Compactor create a new partitionGroup with just blocks 1 to 7 as 8 was not yet in the index

Which issue(s) this PR fixes:
Related to #7129

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Daniel Deluiggi <[email protected]>
@danielblando danielblando marked this pull request as ready for review December 11, 2025 17:19
@danielblando
Copy link
Contributor Author

L1:
b1 b2 b3

PG 1 {2h}. 8-10
parent blocks: b1 b2 b3
Result blocks: c1 c2

PG 2 {12h} 0-12
parent blocks c0 c1 c2
Result block: d1 d2

New c3

PG 2 {12h}
parent d1 d2 c3

T0: b1 b2 b3 c1 c2

T1: cleaner dleete b1 b2 b3

T2: compaction run
c0(6-8) b1(8-10) b2(8-10) b3(8-10) c1(8-10) c2(8-10)

T3: New index

Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@SungJin1212 SungJin1212 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/compactor lgtm This PR has been approved by a maintainer size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants