pageserver: fix image layer skip while maintaing the one visit per layer property #9025
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
VladLazar
changed the title
5 tasks
4968 tests run: 4800 passed, 4 failed, 164 skipped (full report)Failures on Postgres 17
Failures on Postgres 14
Flaky tests (12)Postgres 17
Postgres 16
Postgres 14
Test coverage report is not availableThe comment gets automatically updated with the latest test results
84d1af7 at 2024-09-19T10:45:16.312Z :recycle: |
Problem Different keyspaces may require different floor LSNs in vectored delta layer visits. This patch adds support for such cases. Summary of Changes * Rework layer visit collection. Each layer type has a separate visit type which is aware of the requirements. For delta layers we track the floor LSN of keyspaces and merge only when that's matching. Other layer types do not have this requirement so they merge everything. * Thread the new visit types into the `get_values_reconstruct_data` calls. For delta layers, the code was adapted such that it may merge reads across keyspaces with different LSN floor requirements. * Tweak the fringe update code in `get_vectored_reconstruct_data_timeline` to handle different cont LSNs for different keyspaces. In practice, we will only update the fringe from one keyspace, since this "keyspace split" only happens when an image layer overlaps a delta layer (and image layers always complete all their keys). * Update tests with the new interfaces
VladLazar
force-pushed
the
vlad/fix-vec-read-img-layer-skip-take-2
branch
from
d6e102d
to
f6780a5
Compare
`ValuesReconstructState::consume_done_keys` may only be called once after a layer visit. The code in the previous commit called it for each keyspace, resulting in keys not being marked done in the split keyspace by floor LSN scenario.
VladLazar
force-pushed
the
vlad/fix-vec-read-img-layer-skip-take-2
branch
from
f6780a5
to
84d1af7
Compare
VladLazar
added a commit
that referenced
this pull request
Sep 19, 2024
## Problem Different keyspaces may require different floor LSNs in vectored delta layer visits. This patch adds support for such cases. ## Summary of changes Different keyspaces wishing to read the same layer might require different stop lsns (or lsn floor). The start LSN of the read (or the lsn ceil) will always be the same. With this observation, we fix skipping of image layers by indexing the fringe by layer id plus lsn floor. This is very simple, but means that we can visit delta layers twice in certain cases. Still, I think it's very unlikely for any extra merging to have taken place in this case, so perhaps it makes sense to go with the simpler patch. Fixes #9012 Alternative to #9025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Different keyspaces may require different floor LSNs in vectored
delta layer visits. This patch adds support for such cases.
Summary of Changes
visit type which is aware of the requirements. For delta layers
we track the floor LSN of keyspaces and merge only when that's
matching. Other layer types do not have this requirement so they
merge everything.
get_values_reconstruct_datacalls. For delta layers, the code was adapted such that it may
merge reads across keyspaces with different LSN floor requirements.
get_vectored_reconstruct_data_timelineto handle different cont LSNs for different keyspaces. In practice,
we will only update the fringe from one keyspace, since this "keyspace
split" only happens when an image layer overlaps a delta layer (and
image layers always complete all their keys).
Fixes #9012
Checklist before requesting a review
Checklist before merging