RDMA FLUSH and ATOMIC WRITE with ODP #1580
Draft
+166
−64
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The kernel-side enablement was merged to for-next. |
rleon
reviewed
Apr 10, 2025
libibverbs/verbs.h
Outdated
| @@ -235,6 +235,8 @@ enum ibv_odp_transport_cap_bits { | |||
| IBV_ODP_SUPPORT_READ = 1 << 3, | |||
| IBV_ODP_SUPPORT_ATOMIC = 1 << 4, | |||
| IBV_ODP_SUPPORT_SRQ_RECV = 1 << 5, | |||
| IBV_ODP_SUPPORT_FLUSH = 1 << 6, | |||
There was a problem hiding this comment.
Why aren't these bits coming from UAPI?
There was a problem hiding this comment.
Okay. I will send a kernel patch to add IB_UVERBS_ODP_* flags in include/uapi/rdma/ib_user_verbs.h and update this PR accordingly.
|
General comment to all patches. Please write commit messages for all commits. Thanks |
To commit: ?? ("RDMA/core: Move ODP capability definitions to uapi").
Signed-off-by: Daisuke Matsuda <[email protected]>
The bits are used from both kernel space and userland, so the definitions in libibverbs should depend on those in UAPI. Signed-off-by: Daisuke Matsuda <[email protected]>
Add ODP capabilities for RDMA FLUSH and ATOMIC WRITE from UAPI definitions so that the flags can be queried via ibv_query_device_ex(). Signed-off-by: Daisuke Matsuda <[email protected]>
Let pyverbs use the new bits for ODP capabilities. Signed-off-by: Daisuke Matsuda <[email protected]>
Add a test of ATOMIC WRITE sequences with ODP. MR is invalidated at the beginning of this workload so that page fault is triggered at least once. The test was implemented in reference to test_qp_ex_rc_atomic_write() in test_qpex.py. create_qp_ex() is moved to utils.py so that it can be used from other tests. Signed-off-by: Daisuke Matsuda <[email protected]>
Add a test of RDMA FLUSH sequences with ODP. MR is invalidated at the beginning of this workload so that page fault is triggered at least once. The test was implemented in reference to test_qp_ex_rc_flush() in test_qpex.py Signed-off-by: Daisuke Matsuda <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RDMA FLUSH[1] and ATOMIC WRITE[2] were added to rxe, but they cannot run
in the ODP mode as of now. This series is for the userspace enablement.
The former two patches add definitions of ODP capabilities for these operations.
The rest add rdma-core testcases for these capabilities
For kernel-side enablement in rxe, please see:
(Updated v3) https://lore.kernel.org/linux-rdma/[email protected]/T/#t
[1] [for-next PATCH 00/10] RDMA/rxe: Add RDMA FLUSH operation
https://lore.kernel.org/lkml/[email protected]/
[2] [PATCH v7 0/8] RDMA/rxe: Add atomic write operation
https://lore.kernel.org/linux-rdma/[email protected]/