Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: fix image layer skip while maintaing the one visit per layer property #9025

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Commits on Sep 18, 2024

  1. Configuration menu
    Copy the full SHA
    a8ad678 View commit details
    Browse the repository at this point in the history
  2. pageserver: handle reading up to different floor LSNs in deltas

    Problem
    
    Different keyspaces may require different floor LSNs in vectored
    delta layer visits. This patch adds support for such cases.
    
    Summary of Changes
    
    * Rework layer visit collection. Each layer type has a separate
    visit type which is aware of the requirements. For delta layers
    we track the floor LSN of keyspaces and merge only when that's
    matching. Other layer types do not have this requirement so they
    merge everything.
    * Thread the new visit types into the `get_values_reconstruct_data`
    calls. For delta layers, the code was adapted such that it may
    merge reads across keyspaces with different LSN floor requirements.
    * Tweak the fringe update code in `get_vectored_reconstruct_data_timeline`
    to handle different cont LSNs for different keyspaces. In practice,
    we will only update the fringe from one keyspace, since this "keyspace
    split" only happens when an image layer overlaps a delta layer (and
    image layers always complete all their keys).
    * Update tests with the new interfaces
    VladLazar committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    f0ad90f View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Configuration menu
    Copy the full SHA
    3a218d7 View commit details
    Browse the repository at this point in the history
  2. fixup: update external loop state before handling keyspaces

    `ValuesReconstructState::consume_done_keys` may only be called once
    after a layer visit. The code in the previous commit called it for
    each keyspace, resulting in keys not being marked done in the split
    keyspace by floor LSN scenario.
    VladLazar committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    84d1af7 View commit details
    Browse the repository at this point in the history