Allocate pages using a mult-tenant lock-free pool #2587
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Centril
commented
Apr 11, 2025
crates/core/src/db/datastore/locking_tx_datastore/committed_state.rs
Outdated
Show resolved
Hide resolved
Centril
force-pushed
the
centril/page-pool
branch
2 times, most recently
from
9bf77a5 to
925547e
Compare
Centril
commented
Apr 11, 2025
| @@ -433,6 +433,7 @@ impl<P: BlobProvider> SnapshotFetcher<P> { | |||
| } | |||
|
|
|||
| fn verify_page(&self, expected_hash: blake3::Hash, buf: &[u8]) -> Result<()> { | |||
| // TODO(centril, kim): consider whether we want to use the page pool here. | |||
gefjon
reviewed
Apr 11, 2025
crates/core/src/db/datastore/locking_tx_datastore/committed_state.rs
Outdated
Show resolved
Hide resolved
|
benchmarks please |
Criterion benchmark resultsError when comparing benchmarks: Couldn't find AWS credentials in environment, credentials file, or IAM role. Caused by: |
|
Callgrind benchmark in progress... |
1 similar comment
|
Callgrind benchmark in progress... |
jsdt
reviewed
Apr 11, 2025
crates/core/src/db/datastore/locking_tx_datastore/committed_state.rs
Outdated
Show resolved
Hide resolved
Criterion benchmark resultsError when comparing benchmarks: Couldn't find AWS credentials in environment, credentials file, or IAM role. Caused by: |
jsdt
reviewed
Apr 14, 2025
Centril
force-pushed
the
centril/page-pool
branch
from
1e09780 to
37d500f
Compare
jsdt
approved these changes
Apr 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
Fixes #2016.
This PR implements page allocation through a lock-free multi-tenant pool,
using
crossbeam_queue::ArrayQueueas the basis of the pool.This lock-free queue is bounded. And this PR sets that bound to to 32 GiB. After that, pages will be dropped.
API and ABI breaking changes
None
Expected complexity level and risk
2
Testing
Most of the testing is done by adjusting existing tests.