Do not use ignoreDeletionMarker filter for bucketindex lister #7156
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
This is yet another case of double compaction which can create an unwanted group.
The fetcher when using bucketIndex strategy have a limit visibility of activeBlocks (only the blocks visible when creating the index) and already does filter the deleteMarkers which are marked in the index. If we additionally use the filter, it will also filter out recently deleted blocks which were not updated on the index yet. The cleaner try to maintan a stable version of the index to avoid causing unwanted compaction, which this filter override the index state.
Block 1 to 8 are result blocks of partitionGroup 1
How the issue can happen
T0: blockId1 to blockId7 were created
T1: Index was created
T2: blokcId8 was created completing the partitionGroup 1
T2: Cleaner started deleting parent blocks from partitionGroup 1
T3: Compactor job check the deletionMark for blocks in index
T4: Compactor create a new partitionGroup with just blocks 1 to 7 as 8 was not yet in the index
Which issue(s) this PR fixes:
Related to #7129
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]