New historical emission data

[WalterKolczynski-NOAA]

Type of static data

Experimental data

Target directory

Exp Data: GOCART emissions

Type of change

• Update existing files (Version update needed)
• Remove files (Version update needed)
• Add new files to existing directory
• Add new directory (Version update needed if fix)

Any workflow changes needed?

Fix/IC version update only

Please list all related issues. If this request requires a workflow update (including a fix version update), please open a separate global-workflow issue to track the change.

Required for #2883

Pickup location

/scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/GEFS_ExtData

Size

156672

Additional information

gocart_emissions isn't currently versioned, so we need to figure out what we're going to do there.

[KateFriedman-NOAA]

@bbakernoaa Just getting to this, sorry! So your set looks like a subset of the existing set, is that correct? Or is your set complete and does not need what is in the current set but not in your set? Below is the existing set and your set from the top-level view. Is there a need to retain the current set as-is? I'm considering my options for this since the datasets are so large (at least the current one is). Thoughts?

/scratch1/NCEPDEV/global/glopara/data/gocart_emissions:

19:00:00 gocart_emissions/$ll
total 1544
drwxr-sr-x  2 role.glopara global    4096 Jul 14  2022 Dust
drwxr-sr-x  2 role.glopara global    4096 May 19  2022 GBBEPx
drwxr-sr-x  3 role.glopara global    4096 May 19  2022 GEFS_PRODUCTION
drwxr-sr-x  3 role.glopara global    4096 May 19  2022 MERRA2
drwxr-sr-x  3 role.glopara global    4096 May 19  2022 NASA
drwxr-sr-x  6 role.glopara global    4096 May 19  2022 PIESA
-rw-r--r--  1 role.glopara global   16669 Sep 16  2022 dump
drwxr-sr-x  2 role.glopara global   16384 May 19  2022 monochromatic
drwxrwsr-x 31 role.glopara global    4096 Aug 23  2024 nexus
drwxr-sr-x  2 role.glopara global    4096 May 19  2022 optics
drwxr-sr-x  2 role.glopara global 1507328 Aug  4  2023 volcanic
19:00:00 gocart_emissions/$ll /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/GEFS_ExtData
total 32
drwxr-sr-x 3 Barry.Baker rda-arl-gpu  4096 Feb 20 20:51 MERRA2
drwxr-sr-x 5 Barry.Baker rda-arl-gpu  4096 Sep  9 19:20 PIESA
drwxr-sr-x 2 Barry.Baker rda-arl-gpu 16384 Feb 20 20:47 monochromatic
drwxr-sr-x 9 Barry.Baker rda-arl-gpu  4096 Feb 21 16:26 nexus
drwxr-sr-x 2 Barry.Baker rda-arl-gpu  4096 Feb 20 20:35 optics

[bbakernoaa]

@KateFriedman-NOAA We shouldn't need what is in the current set. The current set actually includes all kinds of other data for not just GEFS but for the regional system too and science datasets. This is a much slimmer version specifically targeting the GDAS/GEFS-AEROv13/GCAFS

[KateFriedman-NOAA]

@bbakernoaa Gotcha, sounds great to have this slimmer version! Another question, do you anticipate further updates to this dataset in the future? I'm mainly concerned about updates that would overwrite existing files in your new set down the road. I'm considering versioning if so. Thanks!

[bbakernoaa]

There may be some updates as we progress towards GCAFS. Versioning would likely be the way to go

[KateFriedman-NOAA]

@bbakernoaa I have copied your new set here on Hera (with new versioning introduced):

/scratch1/NCEPDEV/global/glopara/data/GEFS_ExtData/20250310

I have left the /scratch1/NCEPDEV/global/glopara/data/gocart_emissions set as-is for now and can plan to retire it in the near future.

Are you good with it being called "GEFS_ExtData"? Please confirm that ../GEFS_ExtData/20250310 set looks ok and I will sync it to the other platforms.

[bbakernoaa]

We might want to use the use the name GCAFS as I believe the Aerosols are being pulled out of GEFS with a plan for a standalone application. It isn't a big deal at this point but @yangfanglin may have an opinion

[yangfanglin]

We can keep the data on the GEFS side for now, to maintain the existing capability. Once GCAFS becomes more mature, we can start to reconfigure the data directories as need.

[KateFriedman-NOAA]

Thanks for the feedback @bbakernoaa and @yangfanglin ! I'll leave the new folder called "GEFS_ExtData".

[KateFriedman-NOAA]

@bbakernoaa To incorporate a new version variable for the emissions data some workflow changes are needed. Do you have an open or upcoming PR that I could send those changes into? Perhaps #2883? Let me know, thanks!