Hook up drydep interface. #376

overfelt · 2024-11-17T14:20:08Z

No description provided.

codecov · 2024-11-17T14:41:41Z

Codecov Report

Attention: Patch coverage is 93.93939% with 4 lines in your changes missing coverage. Please review.

Project coverage is 95.87%. Comparing base (edaa167) to head (77f56d3).

Files with missing lines	Patch %	Lines
src/mam4xx/aero_model.hpp	83.33%	4 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #376   +/-   ##
=======================================
  Coverage   95.86%   95.87%           
=======================================
  Files          45       45           
  Lines        9342     9352   +10     
=======================================
+ Hits         8956     8966   +10     
  Misses        386      386

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

odiazib · 2024-11-17T18:27:47Z

src/mam4xx/mo_gas_phase_chemdr.hpp

-    const Real adv_mass_kg_per_moles[gas_pcnst],
+    const Real adv_mass_kg_per_moles[gas_pcnst], const Real mmr[gas_pcnst],
+    const Real fraction_landuse[mam4::mo_drydep::n_land_type],
+    const int col_index_season[mam4::mo_drydep::n_land_type],


@overfelt For col_index_season, please see PR 3108. This PR was merged into the master branch in the scream repo. However, I do not see these changes in the master branch of E3SM. We may need to wait or request for these branches to be synced.

mingxuanwupnnl

Thank you @overfelt. I just left some comments (may edit them later).

mingxuanwupnnl · 2024-11-19T19:29:14Z

src/mam4xx/seq_drydep.hpp

+  for (int i = 0; i < NSeas; ++i) {
+    for (int j = 0; j < NLUse; ++j) {
+        data.rac(i, j) = rac_a[i][j];
+    }
+  }  


The way of initialization works on CPU but probably not for GPU. Need to fix it. @odiazib @singhbalwinder and I discussed it before, I @odiazib has ways to fix it.

mingxuanwupnnl · 2024-11-19T19:31:08Z

src/mam4xx/mo_gas_phase_chemdr.hpp

+    mam4::mo_drydep::drydep_xactive(
+        drydep_data,
+        fraction_landuse, // fraction of land use for column by land type
+        month,            // month
+        col_index_season, // column-specific mapping of month indices to
+                          // seasonal land-type indices [-]
+        sfc_temp,         // surface temperature [K]
+        air_temp,         // surface air temperature [K]
+        tv,               // potential temperature [K]
+        pressure_sfc,     // surface pressure [Pa]
+        pressure_10m,     // 10-meter pressure [Pa]
+        spec_hum,         // specific humidity [kg/kg]
+        wind_speed,       // 10-meter wind spped [m/s]
+        rain,             // rain content [??]
+        snow,             // snow height [m]
+        solar_flux,       // direct shortwave surface radiation [W/m^2]
+        mmr,              // constituent MMRs [kg/kg]
+        dvel,             // deposition velocity [1/cm/s]
+        dflx              // deposition flux [1/cm^2/s]
+    );


need to add if (kk == nlev) for drydep_xactive. Only gas dry dep fluxes and velocity are only calculated at bottom layer.

Some input vars need calculation. For example, wind_speed, it is calculated as sqrt(ubot^2 + vbot^2). Not sure which is prefered, I guess it would be better to calculations here in MAMxx code.

singhbalwinder · 2024-11-23T00:30:06Z

I should have tested it before pushing. I will test it and fix the build errors.

singhbalwinder · 2024-11-23T06:35:27Z

src/mam4xx/mo_sethet.hpp

+  }
+  Kokkos::fence();
+  for (int mm = 0; mm < gas_wetdep_cnt; mm++) {
+    int mm2 = wetdep_map[mm];


@overfelt @odiazib @mjs271 : I was getting an error (sethet: het_rates (wet dep) not set for het reaction number) [see line 498 of this file] and I added this fence. Please let me know if there is a better way to fix it. I believe the error is due to a race condition where abort becomes > 0.

We should use team.team_barrier(), as this part of the code is within a parallel_for.

Since you are modifying the code anyway, doing a ThreadVectorRange inside a TeamThreadRange as on line 298 is the way to go. And, yes, team_barrier() is the right call in this case.

A team_barrier() right before the abort check on line 496 might be overkill, but there are already five team_barrier()s in this function, maybe one more would not hurt.

mingxuanwupnnl · 2024-12-05T21:32:30Z

src/mam4xx/mo_gas_phase_chemdr.hpp

+    mam4::mo_drydep::drydep_xactive(
+        drydep_data,
+        fraction_landuse, // fraction of land use for column by land type
+        month,            // month
+        col_index_season, // column-specific mapping of month indices to
+                          // seasonal land-type indices [-]
+        sfc_temp,         // surface temperature [K]
+        air_temp,         // surface air temperature [K]
+        tv,               // potential temperature [K]
+        pressure_sfc,     // surface pressure [Pa]
+        pressure_10m,     // 10-meter pressure [Pa]
+        spec_hum,         // specific humidity [kg/kg]
+        wind_speed,       // 10-meter wind spped [m/s]
+        rain,             // rain content [??]
+        snow,             // snow height [m]
+        solar_flux,       // direct shortwave surface radiation [W/m^2]
+        qq,               // constituent MMRs [kg/kg]
+        dvel,             // deposition velocity [1/cm/s]
+        dflx              // deposition flux [1/cm^2/s]
+    );


gas dry dep is only executed for bottom layer. I think we need to add if condition here. Some of the input vars for dry dep needs calculation. Currently it is done in EAMxx branch. We might consider move them to MAMxx code if it is the preferred practice.

* Hook up drydep interface. * Inequality needs to be >= to account for fortran -> C++ indexing. * Only call mam4::mo_drydep::drydep_xactive on the bottom level: nlev-1 --------- Co-authored-by: Balwinder Singh <[email protected]>

overfelt requested a review from singhbalwinder November 17, 2024 14:26

odiazib reviewed Nov 17, 2024

View reviewed changes

mingxuanwupnnl requested changes Nov 19, 2024

View reviewed changes

overfelt and others added 9 commits November 22, 2024 16:19

Hook up drydep interface.

4b3fcf6

Fiees for Cuda build.

8cae3b0

Cast to void to avoid compiler warning.

c53551b

Fix incorrect loop bounds.

93bd746

Add const specifier to declaration.

e6330ea

Fix Cuda build. Need to deep_copy data to device.

9ea1e9f

Fix to speed up wet scavaging process.

cc04416

Fix validation tests and Clang format.

80a9013

Adds assert statements and removes mmr variable

29c29a5

singhbalwinder force-pushed the jroverf/drydep_for_eamxx branch from 8652ecb to 29c29a5 Compare November 23, 2024 00:23

Changes after rebase

65f1211

singhbalwinder added 3 commits November 22, 2024 21:14

Fixes build error

77f56d3

Fixes code after rebase and fixes a possible race condition

39213c5

Removes codes forgot to remove in the last commit

7a4d3cb

singhbalwinder reviewed Nov 23, 2024

View reviewed changes

mingxuanwupnnl requested changes Dec 5, 2024

View reviewed changes

overfelt and others added 4 commits December 9, 2024 13:24

Hook up drydep interface.

ed255c3

Changes after rebase

5117547

Fixes code after rebase and fixes a possible race condition

134eff8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hook up drydep interface. #376

Hook up drydep interface. #376

overfelt commented Nov 17, 2024

codecov bot commented Nov 17, 2024 •

edited

Loading

odiazib Nov 17, 2024 •

edited

Loading

mingxuanwupnnl left a comment

mingxuanwupnnl Nov 19, 2024

mingxuanwupnnl Nov 19, 2024 •

edited

Loading

singhbalwinder commented Nov 23, 2024

singhbalwinder Nov 23, 2024

odiazib Nov 24, 2024 •

edited

Loading

overfelt Nov 25, 2024

mingxuanwupnnl Dec 5, 2024

Hook up drydep interface. #376

Are you sure you want to change the base?

Hook up drydep interface. #376

Conversation

overfelt commented Nov 17, 2024

codecov bot commented Nov 17, 2024 • edited Loading

Codecov Report

odiazib Nov 17, 2024 • edited Loading

Choose a reason for hiding this comment

mingxuanwupnnl left a comment

Choose a reason for hiding this comment

mingxuanwupnnl Nov 19, 2024

Choose a reason for hiding this comment

mingxuanwupnnl Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

singhbalwinder commented Nov 23, 2024

singhbalwinder Nov 23, 2024

Choose a reason for hiding this comment

odiazib Nov 24, 2024 • edited Loading

Choose a reason for hiding this comment

overfelt Nov 25, 2024

Choose a reason for hiding this comment

mingxuanwupnnl Dec 5, 2024

Choose a reason for hiding this comment

codecov bot commented Nov 17, 2024 •

edited

Loading

odiazib Nov 17, 2024 •

edited

Loading

mingxuanwupnnl Nov 19, 2024 •

edited

Loading

odiazib Nov 24, 2024 •

edited

Loading