Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mpi #1100

Draft
wants to merge 69 commits into
base: master
Choose a base branch
from
Draft

Mpi #1100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
956f893
make hypre_MPI_Comm a struct
liruipeng Nov 17, 2023
5eba421
fix GPU build
liruipeng Nov 19, 2023
6483d53
fix consistent mpi
liruipeng Nov 19, 2023
c2546c1
change orders of headers
liruipeng Nov 22, 2023
a791d17
add new mpi interface
liruipeng Nov 22, 2023
e0b077c
use new mpi interface
liruipeng Nov 22, 2023
ac9e0d5
bug fix
liruipeng Dec 8, 2023
feb5005
restruct par_csr_communication.c
liruipeng Dec 8, 2023
2bbc23d
make hypre_MPI_Request a struct
liruipeng Dec 8, 2023
538cf07
changes in euclid/parasails from hypre_mpi_request
liruipeng Dec 9, 2023
646b513
bug fix
liruipeng Dec 9, 2023
8d44472
update par_csr_communication
liruipeng Dec 11, 2023
85f274b
add lower level Tmemcpy
liruipeng Dec 11, 2023
48caed5
add actions in mpi_requests
liruipeng Dec 11, 2023
a2d56d9
update util header
liruipeng Dec 11, 2023
6b87015
make gpu aware mpi runtime option
liruipeng Dec 12, 2023
dd212da
make gpu aware mpi runtime option
liruipeng Dec 12, 2023
9a7dc2f
bug fix and add test options
liruipeng Dec 12, 2023
d69a8a8
bug fix and add test options
liruipeng Dec 12, 2023
43f4ca8
run header
liruipeng Dec 12, 2023
98260a1
Update src/utilities/handle.c
liruipeng Dec 12, 2023
ed55acc
bug fix
liruipeng Dec 12, 2023
2d8708a
Merge branch 'gpu_mpi' of github.com:hypre-space/hypre into gpu_mpi
liruipeng Dec 12, 2023
0bd9778
Merge branch 'master' of github.com:hypre-space/hypre into mpi
liruipeng Dec 12, 2023
bc5be69
Merge branch 'mpi' of github.com:hypre-space/hypre into mpi
liruipeng Dec 12, 2023
655aac9
run header
liruipeng Dec 12, 2023
3897f33
add gpu_mpi to the macro
liruipeng Dec 12, 2023
e46a2b5
Merge branch 'mpi' of github.com:hypre-space/hypre into mpi
liruipeng Dec 12, 2023
5e7373b
fix compile errors
liruipeng Dec 13, 2023
4f2e63f
update headers
liruipeng Dec 13, 2023
bbce604
Merge branch 'gpu_mpi' of github.com:hypre-space/hypre into mpi
liruipeng Dec 13, 2023
01b2abf
run header
liruipeng Dec 13, 2023
4845d70
persistent mpi
liruipeng Dec 14, 2023
3a5f2ae
persistent mpi
liruipeng Dec 15, 2023
4392afe
add persistent buffer in par_csr_comm
liruipeng Dec 15, 2023
f8ebeb3
fix memory leaks in persist mpi
liruipeng Dec 15, 2023
b5dbb02
unifying persistent
liruipeng Dec 17, 2023
d7c2188
refactoring parcsr comm persistent
liruipeng Dec 17, 2023
9e5d0c9
fix memory leaks
liruipeng Dec 18, 2023
df37bf9
revert hypre_MPI_comm struct
liruipeng Jun 22, 2024
56a4f73
revert more hypre MPI comm
liruipeng Jun 22, 2024
fe6b841
minor changes
liruipeng Jun 22, 2024
cba68d1
Merge branch 'master' of github.com:hypre-space/hypre into mpi
liruipeng Jun 22, 2024
82f1b60
fix after merge
liruipeng Jun 22, 2024
09eb22f
minor changes
liruipeng Jun 22, 2024
2085e3d
a working version
liruipeng Jun 25, 2024
d0a769c
another way to implement
liruipeng Jun 25, 2024
5c2567a
add comm in comm_handle
liruipeng Jun 26, 2024
1cedf13
debug for non-mpi (not finished)
liruipeng Jun 26, 2024
499547e
change mpi request back
liruipeng Jun 27, 2024
c364769
code refactor
liruipeng Jun 27, 2024
bd23e90
minor changes
liruipeng Jun 27, 2024
84fc444
minor changes
liruipeng Jun 27, 2024
67678d4
a small change
liruipeng Jun 27, 2024
7ecf8ef
fixed some issues; add hypre_ to function names
liruipeng Jun 27, 2024
a972b79
put Grequest in mpi_multi_recv
liruipeng Jun 30, 2024
e280057
reworked persistent mpi
liruipeng Jul 1, 2024
6ef031b
clean up code
liruipeng Jul 1, 2024
4924b31
fix for nonmpi
liruipeng Jul 1, 2024
19d830c
fix memory leaks
liruipeng Jul 2, 2024
12483cc
free key val
liruipeng Jul 2, 2024
46309e8
code clean up
liruipeng Jul 3, 2024
7e25b17
minor change
liruipeng Jul 3, 2024
2d9235a
minor changes
liruipeng Jul 3, 2024
4e76819
minor change
liruipeng Jul 3, 2024
d3ec50a
minor changes
liruipeng Jul 3, 2024
11cb8aa
minor change
liruipeng Jul 3, 2024
e687ab4
turn off debug code
liruipeng Jul 3, 2024
68c4b54
add persistent mpi to GPU matvec
liruipeng Jul 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions src/parcsr_ls/par_relax.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -787,9 +787,9 @@ hypre_BoomerAMGRelaxHybridGaussSeidel_core( hypre_ParCSRMatrix *A,
num_sends = hypre_ParCSRCommPkgNumSends(comm_pkg);

#if defined(HYPRE_USING_PERSISTENT_COMM)
persistent_comm_handle = hypre_ParCSRCommPkgGetPersistentCommHandle(1, comm_pkg);
v_buf_data = (HYPRE_Real *) hypre_ParCSRCommHandleSendDataBuffer(persistent_comm_handle);
v_ext_data = (HYPRE_Real *) hypre_ParCSRCommHandleRecvDataBuffer(persistent_comm_handle);
persistent_comm_handle = hypre_ParCSRCommPkgGetPersistentCommHandle(1, comm_pkg, HYPRE_MEMORY_HOST, HYPRE_MEMORY_HOST);
v_buf_data = (HYPRE_Real *) hypre_ParCSRCommHandleSendData(persistent_comm_handle);
v_ext_data = (HYPRE_Real *) hypre_ParCSRCommHandleRecvData(persistent_comm_handle);
#else
v_buf_data = hypre_CTAlloc(HYPRE_Real,
hypre_ParCSRCommPkgSendMapStart(comm_pkg, num_sends),
Expand All @@ -813,13 +813,13 @@ hypre_BoomerAMGRelaxHybridGaussSeidel_core( hypre_ParCSRMatrix *A,
#endif

#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandleStart(persistent_comm_handle, HYPRE_MEMORY_HOST, v_buf_data);
hypre_ParCSRPersistentCommHandleStart(persistent_comm_handle);
#else
comm_handle = hypre_ParCSRCommHandleCreate(1, comm_pkg, v_buf_data, v_ext_data);
#endif

#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandleWait(persistent_comm_handle, HYPRE_MEMORY_HOST, v_ext_data);
hypre_ParCSRCommHandleWait(persistent_comm_handle);
#else
hypre_ParCSRCommHandleDestroy(comm_handle);
#endif
Expand Down
202 changes: 139 additions & 63 deletions src/parcsr_mv/_hypre_parcsr_mv.h
View file
Open in desktop

Large diffs are not rendered by default.

867 changes: 299 additions & 568 deletions src/parcsr_mv/par_csr_communication.c
View file
Open in desktop

Large diffs are not rendered by default.

173 changes: 119 additions & 54 deletions src/parcsr_mv/par_csr_communication.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
* Structure containing information for doing communications
*--------------------------------------------------------------------------*/

#ifdef HYPRE_USING_PERSISTENT_COMM
typedef enum CommPkgJobType
{
HYPRE_COMM_PKG_JOB_COMPLEX = 0,
Expand All @@ -22,9 +21,63 @@ typedef enum CommPkgJobType
HYPRE_COMM_PKG_JOB_INT_TRANSPOSE,
HYPRE_COMM_PKG_JOB_BIGINT,
HYPRE_COMM_PKG_JOB_BIGINT_TRANSPOSE,
NUM_OF_COMM_PKG_JOB_TYPE,
NUM_OF_COMM_PKG_JOB_TYPE
} CommPkgJobType;
#endif

static inline CommPkgJobType
hypre_ParCSRCommHandleGetJobType(HYPRE_Int job)
{
CommPkgJobType job_type = HYPRE_COMM_PKG_JOB_COMPLEX;
switch (job)
{
case 1:
job_type = HYPRE_COMM_PKG_JOB_COMPLEX;
break;
case 2:
job_type = HYPRE_COMM_PKG_JOB_COMPLEX_TRANSPOSE;
break;
case 11:
job_type = HYPRE_COMM_PKG_JOB_INT;
break;
case 12:
job_type = HYPRE_COMM_PKG_JOB_INT_TRANSPOSE;
break;
case 21:
job_type = HYPRE_COMM_PKG_JOB_BIGINT;
break;
case 22:
job_type = HYPRE_COMM_PKG_JOB_BIGINT_TRANSPOSE;
break;
}

return job_type;
}

static inline hypre_MPI_Datatype
hypre_ParCSRCommHandleGetMPIDataType(HYPRE_Int job)
{
hypre_MPI_Datatype dtype = HYPRE_MPI_COMPLEX;

switch (hypre_ParCSRCommHandleGetJobType(job))
{
case HYPRE_COMM_PKG_JOB_COMPLEX:
case HYPRE_COMM_PKG_JOB_COMPLEX_TRANSPOSE:
dtype = HYPRE_MPI_COMPLEX;
break;
case HYPRE_COMM_PKG_JOB_INT:
case HYPRE_COMM_PKG_JOB_INT_TRANSPOSE:
dtype = HYPRE_MPI_INT;
break;
case HYPRE_COMM_PKG_JOB_BIGINT:
case HYPRE_COMM_PKG_JOB_BIGINT_TRANSPOSE:
dtype = HYPRE_MPI_BIG_INT;
break;
default:
break;
}

return dtype;
}

/*--------------------------------------------------------------------------
* hypre_ParCSRCommHandle, hypre_ParCSRPersistentCommHandle
Expand All @@ -34,18 +87,45 @@ struct _hypre_ParCSRCommPkg;
typedef struct
{
struct _hypre_ParCSRCommPkg *comm_pkg;
HYPRE_MemoryLocation send_memory_location;
HYPRE_MemoryLocation recv_memory_location;
HYPRE_Int num_send_bytes;
HYPRE_Int num_recv_bytes;
void *send_data;
void *recv_data;
void *send_data_buffer;
void *recv_data_buffer;
HYPRE_Int num_requests;
hypre_MPI_Request *requests;
HYPRE_Int persistent;
void *send_data;
void *recv_data;
/* send/recv buffers to copy to/from */
void *send_buffer;
void *recv_buffer;
HYPRE_MemoryLocation send_location;
HYPRE_MemoryLocation recv_location;
hypre_MemoryLocation send_buffer_location;
hypre_MemoryLocation recv_buffer_location;
HYPRE_Int num_requests;
hypre_MPI_Request *requests;
HYPRE_Int num_extra_requests;
hypre_MPI_Request *extra_requests;
MPI_Comm comm;
} hypre_ParCSRCommHandle;

/*--------------------------------------------------------------------------
* Accessor macros: hypre_ParCSRCommHandle
*--------------------------------------------------------------------------*/

#define hypre_ParCSRCommHandleCommPkg(comm_handle) (comm_handle -> comm_pkg)
#define hypre_ParCSRCommHandlePersistent(comm_handle) (comm_handle -> persistent)
#define hypre_ParCSRCommHandleSendData(comm_handle) (comm_handle -> send_data)
#define hypre_ParCSRCommHandleRecvData(comm_handle) (comm_handle -> recv_data)
#define hypre_ParCSRCommHandleSendBuffer(comm_handle) (comm_handle -> send_buffer)
#define hypre_ParCSRCommHandleRecvBuffer(comm_handle) (comm_handle -> recv_buffer)
#define hypre_ParCSRCommHandleSendLocation(comm_handle) (comm_handle -> send_location)
#define hypre_ParCSRCommHandleRecvLocation(comm_handle) (comm_handle -> recv_location)
#define hypre_ParCSRCommHandleSendBufferLocation(comm_handle) (comm_handle -> send_buffer_location)
#define hypre_ParCSRCommHandleRecvBufferLocation(comm_handle) (comm_handle -> recv_buffer_location)
#define hypre_ParCSRCommHandleNumRequests(comm_handle) (comm_handle -> num_requests)
#define hypre_ParCSRCommHandleRequests(comm_handle) (comm_handle -> requests)
#define hypre_ParCSRCommHandleRequest(comm_handle, i) (comm_handle -> requests[i])
#define hypre_ParCSRCommHandleNumExtraRequests(comm_handle) (comm_handle -> num_extra_requests)
#define hypre_ParCSRCommHandleExtraRequests(comm_handle) (comm_handle -> extra_requests)
#define hypre_ParCSRCommHandleExtraRequest(comm_handle, i) (comm_handle -> extra_requests[i])
#define hypre_ParCSRCommHandleComm(comm_handle) (comm_handle -> comm)

typedef hypre_ParCSRCommHandle hypre_ParCSRPersistentCommHandle;

typedef struct _hypre_ParCSRCommPkg
Expand All @@ -63,7 +143,7 @@ typedef struct _hypre_ParCSRCommPkg
/* remote communication information */
hypre_MPI_Datatype *send_mpi_types;
hypre_MPI_Datatype *recv_mpi_types;
#ifdef HYPRE_USING_PERSISTENT_COMM
#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandle *persistent_comm_handles[NUM_OF_COMM_PKG_JOB_TYPE];
#endif
#if defined(HYPRE_USING_GPU) || defined(HYPRE_USING_DEVICE_OPENMP)
Expand All @@ -78,31 +158,33 @@ typedef struct _hypre_ParCSRCommPkg
* Accessor macros: hypre_ParCSRCommPkg
*--------------------------------------------------------------------------*/

#define hypre_ParCSRCommPkgComm(comm_pkg) (comm_pkg -> comm)
#define hypre_ParCSRCommPkgNumComponents(comm_pkg) (comm_pkg -> num_components)
#define hypre_ParCSRCommPkgNumSends(comm_pkg) (comm_pkg -> num_sends)
#define hypre_ParCSRCommPkgSendProcs(comm_pkg) (comm_pkg -> send_procs)
#define hypre_ParCSRCommPkgSendProc(comm_pkg, i) (comm_pkg -> send_procs[i])
#define hypre_ParCSRCommPkgSendMapStarts(comm_pkg) (comm_pkg -> send_map_starts)
#define hypre_ParCSRCommPkgSendMapStart(comm_pkg,i) (comm_pkg -> send_map_starts[i])
#define hypre_ParCSRCommPkgSendMapElmts(comm_pkg) (comm_pkg -> send_map_elmts)
#define hypre_ParCSRCommPkgDeviceSendMapElmts(comm_pkg) (comm_pkg -> device_send_map_elmts)
#define hypre_ParCSRCommPkgSendMapElmt(comm_pkg,i) (comm_pkg -> send_map_elmts[i])
#define hypre_ParCSRCommPkgDeviceSendMapElmt(comm_pkg,i) (comm_pkg -> device_send_map_elmts[i])
#define hypre_ParCSRCommPkgNumRecvs(comm_pkg) (comm_pkg -> num_recvs)
#define hypre_ParCSRCommPkgRecvProcs(comm_pkg) (comm_pkg -> recv_procs)
#define hypre_ParCSRCommPkgRecvProc(comm_pkg, i) (comm_pkg -> recv_procs[i])
#define hypre_ParCSRCommPkgRecvVecStarts(comm_pkg) (comm_pkg -> recv_vec_starts)
#define hypre_ParCSRCommPkgRecvVecStart(comm_pkg,i) (comm_pkg -> recv_vec_starts[i])
#define hypre_ParCSRCommPkgSendMPITypes(comm_pkg) (comm_pkg -> send_mpi_types)
#define hypre_ParCSRCommPkgSendMPIType(comm_pkg,i) (comm_pkg -> send_mpi_types[i])
#define hypre_ParCSRCommPkgRecvMPITypes(comm_pkg) (comm_pkg -> recv_mpi_types)
#define hypre_ParCSRCommPkgRecvMPIType(comm_pkg,i) (comm_pkg -> recv_mpi_types[i])
#define hypre_ParCSRCommPkgComm(comm_pkg) (comm_pkg -> comm)
#define hypre_ParCSRCommPkgNumComponents(comm_pkg) (comm_pkg -> num_components)
#define hypre_ParCSRCommPkgNumSends(comm_pkg) (comm_pkg -> num_sends)
#define hypre_ParCSRCommPkgSendProcs(comm_pkg) (comm_pkg -> send_procs)
#define hypre_ParCSRCommPkgSendProc(comm_pkg, i) (comm_pkg -> send_procs[i])
#define hypre_ParCSRCommPkgSendMapStarts(comm_pkg) (comm_pkg -> send_map_starts)
#define hypre_ParCSRCommPkgSendMapStart(comm_pkg,i) (comm_pkg -> send_map_starts[i])
#define hypre_ParCSRCommPkgSendMapElmts(comm_pkg) (comm_pkg -> send_map_elmts)
#define hypre_ParCSRCommPkgDeviceSendMapElmts(comm_pkg) (comm_pkg -> device_send_map_elmts)
#define hypre_ParCSRCommPkgSendMapElmt(comm_pkg,i) (comm_pkg -> send_map_elmts[i])
#define hypre_ParCSRCommPkgDeviceSendMapElmt(comm_pkg,i) (comm_pkg -> device_send_map_elmts[i])
#define hypre_ParCSRCommPkgNumRecvs(comm_pkg) (comm_pkg -> num_recvs)
#define hypre_ParCSRCommPkgRecvProcs(comm_pkg) (comm_pkg -> recv_procs)
#define hypre_ParCSRCommPkgRecvProc(comm_pkg, i) (comm_pkg -> recv_procs[i])
#define hypre_ParCSRCommPkgRecvVecStarts(comm_pkg) (comm_pkg -> recv_vec_starts)
#define hypre_ParCSRCommPkgRecvVecStart(comm_pkg,i) (comm_pkg -> recv_vec_starts[i])
#define hypre_ParCSRCommPkgSendMPITypes(comm_pkg) (comm_pkg -> send_mpi_types)
#define hypre_ParCSRCommPkgSendMPIType(comm_pkg,i) (comm_pkg -> send_mpi_types[i])
#define hypre_ParCSRCommPkgRecvMPITypes(comm_pkg) (comm_pkg -> recv_mpi_types)
#define hypre_ParCSRCommPkgRecvMPIType(comm_pkg,i) (comm_pkg -> recv_mpi_types[i])
#define hypre_ParCSRCommPkgPersistentCommHandles(comm_pkg) (comm_pkg -> persistent_comm_handles)
#define hypre_ParCSRCommPkgPersistentCommHandle(comm_pkg,i) (comm_pkg -> persistent_comm_handles[i])

#if defined(HYPRE_USING_GPU) || defined(HYPRE_USING_DEVICE_OPENMP)
#define hypre_ParCSRCommPkgTmpData(comm_pkg) ((comm_pkg) -> tmp_data)
#define hypre_ParCSRCommPkgBufData(comm_pkg) ((comm_pkg) -> buf_data)
#define hypre_ParCSRCommPkgMatrixE(comm_pkg) ((comm_pkg) -> matrix_E)
#define hypre_ParCSRCommPkgTmpData(comm_pkg) ((comm_pkg) -> tmp_data)
#define hypre_ParCSRCommPkgBufData(comm_pkg) ((comm_pkg) -> buf_data)
#define hypre_ParCSRCommPkgMatrixE(comm_pkg) ((comm_pkg) -> matrix_E)
#endif

static inline void
Expand Down Expand Up @@ -130,21 +212,4 @@ hypre_ParCSRCommPkgCopySendMapElmtsToDevice(hypre_ParCSRCommPkg *comm_pkg)
#endif
}

/*--------------------------------------------------------------------------
* Accessor macros: hypre_ParCSRCommHandle
*--------------------------------------------------------------------------*/

#define hypre_ParCSRCommHandleCommPkg(comm_handle) (comm_handle -> comm_pkg)
#define hypre_ParCSRCommHandleSendMemoryLocation(comm_handle) (comm_handle -> send_memory_location)
#define hypre_ParCSRCommHandleRecvMemoryLocation(comm_handle) (comm_handle -> recv_memory_location)
#define hypre_ParCSRCommHandleNumSendBytes(comm_handle) (comm_handle -> num_send_bytes)
#define hypre_ParCSRCommHandleNumRecvBytes(comm_handle) (comm_handle -> num_recv_bytes)
#define hypre_ParCSRCommHandleSendData(comm_handle) (comm_handle -> send_data)
#define hypre_ParCSRCommHandleRecvData(comm_handle) (comm_handle -> recv_data)
#define hypre_ParCSRCommHandleSendDataBuffer(comm_handle) (comm_handle -> send_data_buffer)
#define hypre_ParCSRCommHandleRecvDataBuffer(comm_handle) (comm_handle -> recv_data_buffer)
#define hypre_ParCSRCommHandleNumRequests(comm_handle) (comm_handle -> num_requests)
#define hypre_ParCSRCommHandleRequests(comm_handle) (comm_handle -> requests)
#define hypre_ParCSRCommHandleRequest(comm_handle, i) (comm_handle -> requests[i])

#endif /* HYPRE_PAR_CSR_COMMUNICATION_HEADER */
39 changes: 19 additions & 20 deletions src/parcsr_mv/par_csr_matop.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6285,7 +6285,6 @@ hypre_ParCSRMatrixDiagScaleHost( hypre_ParCSRMatrix *par_A,
/* Local variables */
HYPRE_Int i;
hypre_Vector *rdbuf;
HYPRE_Complex *recv_rdbuf_data;
HYPRE_Complex *send_rdbuf_data;

/*---------------------------------------------------------------------
Expand All @@ -6307,23 +6306,21 @@ hypre_ParCSRMatrixDiagScaleHost( hypre_ParCSRMatrix *par_A,

#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandle *comm_handle =
hypre_ParCSRCommPkgGetPersistentCommHandle(1, comm_pkg);
hypre_ParCSRCommPkgGetPersistentCommHandle(1, comm_pkg, HYPRE_MEMORY_HOST, HYPRE_MEMORY_HOST);

hypre_VectorData(rdbuf) = (HYPRE_Complex *)
hypre_ParCSRCommHandleRecvDataBuffer(comm_handle);
hypre_ParCSRCommHandleRecvData(comm_handle);
hypre_SeqVectorSetDataOwner(rdbuf, 0);

#else
hypre_ParCSRCommHandle *comm_handle;
#endif

/* Initialize rdbuf */
hypre_SeqVectorInitialize_v2(rdbuf, HYPRE_MEMORY_HOST);
recv_rdbuf_data = hypre_VectorData(rdbuf);
HYPRE_Complex *recv_rdbuf_data = hypre_VectorData(rdbuf);
#endif

/* Allocate send buffer for rdbuf */
#if defined(HYPRE_USING_PERSISTENT_COMM)
send_rdbuf_data = (HYPRE_Complex *) hypre_ParCSRCommHandleSendDataBuffer(comm_handle);
send_rdbuf_data = (HYPRE_Complex *) hypre_ParCSRCommHandleSendData(comm_handle);
#else
send_rdbuf_data = hypre_TAlloc(HYPRE_Complex, send_map_starts[num_sends], HYPRE_MEMORY_HOST);
#endif
Expand All @@ -6339,7 +6336,7 @@ hypre_ParCSRMatrixDiagScaleHost( hypre_ParCSRMatrix *par_A,

/* Non-blocking communication starts */
#ifdef HYPRE_USING_PERSISTENT_COMM
hypre_ParCSRPersistentCommHandleStart(comm_handle, HYPRE_MEMORY_HOST, send_rdbuf_data);
hypre_ParCSRPersistentCommHandleStart(comm_handle);

#else
comm_handle = hypre_ParCSRCommHandleCreate_v2(1, comm_pkg,
Expand All @@ -6356,7 +6353,7 @@ hypre_ParCSRMatrixDiagScaleHost( hypre_ParCSRMatrix *par_A,

/* Non-blocking communication ends */
#ifdef HYPRE_USING_PERSISTENT_COMM
hypre_ParCSRPersistentCommHandleWait(comm_handle, HYPRE_MEMORY_HOST, recv_rdbuf_data);
hypre_ParCSRPersistentCommHandleWait(comm_handle);
#else
hypre_ParCSRCommHandleDestroy(comm_handle);
#endif
Expand Down Expand Up @@ -6710,9 +6707,10 @@ hypre_ParCSRMatrixBlockColSumHost( hypre_ParCSRMatrix *A,

/* Allocate the recv and send buffers */
#if defined(HYPRE_USING_PERSISTENT_COMM)
comm_handle = hypre_ParCSRCommPkgGetPersistentCommHandle(HYPRE_COMM_PKG_JOB_COMPLEX, comm_pkg);
recv_data = (HYPRE_Complex *) hypre_ParCSRCommHandleRecvDataBuffer(comm_handle);
send_data = (HYPRE_Complex *) hypre_ParCSRCommHandleSendDataBuffer(comm_handle);
comm_handle = hypre_ParCSRCommPkgGetPersistentCommHandle(HYPRE_COMM_PKG_JOB_COMPLEX, comm_pkg,
memory_location, memory_location);
recv_data = (HYPRE_Complex *) hypre_ParCSRCommHandleRecvData(comm_handle);
send_data = (HYPRE_Complex *) hypre_ParCSRCommHandleSendData(comm_handle);
send_data = hypre_Memset((void *) send_data, 0,
(size_t) (num_cols_offd_A) * sizeof(HYPRE_Complex),
memory_location);
Expand All @@ -6733,7 +6731,7 @@ hypre_ParCSRMatrixBlockColSumHost( hypre_ParCSRMatrix *A,

/* Non-blocking communication starts */
#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandleStart(comm_handle, memory_location, send_data);
hypre_ParCSRPersistentCommHandleStart(comm_handle);

#else
comm_handle = hypre_ParCSRCommHandleCreate_v2(2, comm_pkg,
Expand All @@ -6757,7 +6755,7 @@ hypre_ParCSRMatrixBlockColSumHost( hypre_ParCSRMatrix *A,

/* Non-blocking communication ends */
#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandleWait(comm_handle, memory_location, recv_data);
hypre_ParCSRPersistentCommHandleWait(comm_handle);
#else
hypre_ParCSRCommHandleDestroy(comm_handle);
#endif
Expand Down Expand Up @@ -6919,9 +6917,10 @@ hypre_ParCSRMatrixColSumHost( hypre_ParCSRMatrix *A,

/* Allocate the recv and send buffers */
#if defined(HYPRE_USING_PERSISTENT_COMM)
comm_handle = hypre_ParCSRCommPkgGetPersistentCommHandle(HYPRE_COMM_PKG_JOB_COMPLEX, comm_pkg);
recv_data = (HYPRE_Complex *) hypre_ParCSRCommHandleRecvDataBuffer(comm_handle);
send_data = (HYPRE_Complex *) hypre_ParCSRCommHandleSendDataBuffer(comm_handle);
comm_handle = hypre_ParCSRCommPkgGetPersistentCommHandle(HYPRE_COMM_PKG_JOB_COMPLEX, comm_pkg,
memory_location, memory_location);
recv_data = (HYPRE_Complex *) hypre_ParCSRCommHandleRecvData(comm_handle);
send_data = (HYPRE_Complex *) hypre_ParCSRCommHandleSendData(comm_handle);
send_data = hypre_Memset((void *) send_data, 0,
(size_t) (num_cols_offd_A) * sizeof(HYPRE_Complex),
memory_location);
Expand All @@ -6942,7 +6941,7 @@ hypre_ParCSRMatrixColSumHost( hypre_ParCSRMatrix *A,

/* Non-blocking communication starts */
#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandleStart(comm_handle, memory_location, send_data);
hypre_ParCSRPersistentCommHandleStart(comm_handle);

#else
comm_handle = hypre_ParCSRCommHandleCreate_v2(2, comm_pkg,
Expand All @@ -6962,7 +6961,7 @@ hypre_ParCSRMatrixColSumHost( hypre_ParCSRMatrix *A,

/* Non-blocking communication ends */
#if defined(HYPRE_USING_PERSISTENT_COMM)
hypre_ParCSRPersistentCommHandleWait(comm_handle, memory_location, recv_data);
hypre_ParCSRPersistentCommHandleWait(comm_handle);
#else
hypre_ParCSRCommHandleDestroy(comm_handle);
#endif
Expand Down
Loading