Fully-coupled with OpenMP

[LucaFurnari]

Dear,

I was wondering if it is possible to compile and run WRF+WRF-Hydro in fully-coupled mode with a hybrid option (dmpar+smpar with MPI+OpenMP) to speed-up computation. I've checked that the previous version of WRF-Hydro was not supporting such a feature.

If not, will it be possible in the future?

Thank you

[scrasmussen]

Hi, WRF-Hydro only supports parallelism through MPI at the moment. In the future there has been talk about using GPUs, possible offloading using Do Concurrent. Do Concurrent would, depending upon the compiler and flags, essentially give the user dmpar+smpar. There is no timeline for the work at the moment, just discussions and some exploratory work.

Do you have a specific use case or machine that you are trying to optimize performance for?

[samcom12]

Following for GPU version of WRF.

[LucaFurnari]

Hi,
Thank you very much for your previous info.

After the last message we posted in July, we understood that it was not possible to run WRF+WRF_HYDRO in hybrid mode. We ran a series of tests to try to improve efficiency (i.e. time calculation), and we achieved the following:

1. We tried tiling, but it doesn't seem to work for WRF_HYDRO;
2. Compilation options: we compiled with mpiifort (Intel compiler) but saw no improvements;
3. Quilty option to reduce output file writing time, but this also doesn't work with unpaired WRF;
4. By subdividing N tasks, with NtaskX << NtaskY, we obtained a reduction in CPU time;
5. When submitting a job, not using all the cores of a node, but only a part of them (32 instead of 36) leads to a sensible reduction of time;
   All these tests reported here were compiled by activating the NetCDF4 option, which allowed us to reduce the output size but, at the same time, increased writing times.

In conclusion, for the domain we are analyzing, we managed to reduce the calculation times from approximately 9 min per day to 6 min per day. The goal would be to halve these times in view of a coupled climate simulation of around 100 years.

We attach the configure.wrf we are using, the Job submission script and some info on the supercomputer we are using.
configure.wrf.txt
Job_WRF_Hydro_EXP_1.sh.txt

If you have any ideas to suggest on how to reduce calculation times, this would be of great help to us.
Finally, is there any update on WRFHydro by any chance?

Thanks in advance and sorry for the long message!