fix(pageserver): make delta split layer writer finish atomic

[skyzh]

Problem

similar to #8841, we make the delta layer writer atomic when finishing the layers.

Summary of changes

• put_value not taking discard fn anymore
• finish decides what layers to keep

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

4968 tests run: 4804 passed, 0 failed, 164 skipped (full report)

---

Flaky tests (9)

Postgres 17

• test_scrubber_physical_gc[4]: release-x86-64, release-arm64
• test_subscriber_restart: release-x86-64, debug-x86-64

Postgres 16

• test_location_conf_churn[1]: release-arm64
• test_scrubber_physical_gc[4]: release-x86-64, release-arm64

Postgres 14

• test_ondemand_wal_download_in_replication_slot_funcs: release-arm64
• test_subscriber_restart: release-x86-64

Code coverage* (full report)

• functions: 31.8% (7404 of 23299 functions)
• lines: 49.7% (59541 of 119722 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
1378e85 at 2024-09-18T16:35:42.664Z :recycle:}