Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package #143

Open
wants to merge 44 commits into
base: package
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
dc6240a
moved original CWARHM summa scripts to cwarhm-summa folder
bvanosnabrugge Feb 21, 2022
15fe313
added original cwarhm commit to cwarhm-summa
bvanosnabrugge Feb 21, 2022
d1562b6
package structure and wrappers to cwarhm-summa
bvanosnabrugge Feb 22, 2022
122c8f5
add .vscode to .gitignore
bartvanosnabrugge Feb 22, 2022
ca86987
deleted .vscode from repo
bvanosnabrugge Feb 22, 2022
f0c21ac
set reset_test to True
bvanosnabrugge Feb 22, 2022
52dbfc5
Update README.rst with how to run the test case
bartvanosnabrugge Feb 22, 2022
5997647
Merge branch 'package' of https://github.com/bartvanosnabrugge/summaW…
bvanosnabrugge Feb 22, 2022
2ebf9e4
Update README.rst
bartvanosnabrugge Mar 1, 2022
1a017f5
Update README.rst
bartvanosnabrugge Mar 1, 2022
27b63ad
added function to read control file including expanding default paths
bvanosnabrugge Mar 2, 2022
da5e66b
updated sphinx to reference cwarhm
bvanosnabrugge Mar 5, 2022
bebe235
Update README.rst
bartvanosnabrugge Mar 7, 2022
3ae2c0e
control file path now set from cwarhm_summa_folder variable
bvanosnabrugge Mar 7, 2022
acf53c7
Merge branch 'package' of https://github.com/bartvanosnabrugge/summaW…
bvanosnabrugge Mar 7, 2022
a9b8305
Update README.rst
bartvanosnabrugge Mar 7, 2022
d392964
added placeholder files for functions
bvanosnabrugge Mar 8, 2022
e6325bf
added placeholder files for functions
bvanosnabrugge Mar 8, 2022
1360d98
Merge branch 'package' of https://github.com/bartvanosnabrugge/summaW…
bvanosnabrugge Mar 8, 2022
28719cc
removed qgs.exitQgis() line to prevent segmentation faults when runni…
bvanosnabrugge Mar 8, 2022
9dad562
era5 merging and default path change
guoqiang-tang Mar 9, 2022
2bbb922
small modification
guoqiang-tang Mar 10, 2022
2dc2825
Merge pull request #42 from tgq14-fork/package
bartvanosnabrugge Mar 11, 2022
715727a
#11 explicitly add rasterio
bartvanosnabrugge Mar 17, 2022
6ba97f3
added rasterstats to environment
bvanosnabrugge Mar 18, 2022
3758130
added functions for mesh specific processing, and needed model agnost…
bvanosnabrugge Mar 18, 2022
1778707
added some notes to test file
bvanosnabrugge Mar 18, 2022
599215a
added test sample data
bvanosnabrugge Mar 18, 2022
3e34c86
added two classes to mesh.py to write CLASS and RUN OPTIONS ini files
bvanosnabrugge Mar 31, 2022
39bb144
updated docstrings of MeshClassIniFile
bvanosnabrugge Apr 4, 2022
b556938
added MeshHydrologyIniFile class to write MESH_parameters_hydrology.i…
bvanosnabrugge Apr 4, 2022
0cf811b
classes for soil_layers file and reservoir.txt file
bvanosnabrugge Apr 5, 2022
444047a
added docstring to easymore wrapper map_forcing_data
bvanosnabrugge Apr 7, 2022
8726e3b
comments to reindex_forcing_file
bvanosnabrugge Apr 7, 2022
880b18d
in reindex_forcing_file ensured lat lon coordinates are reindexed
bvanosnabrugge Apr 7, 2022
441858f
changed default run option flag OUTFILESFLAG to off and removed .nc f…
bvanosnabrugge Apr 7, 2022
505c967
file extension is not included when setting FNAME in run_options file
bvanosnabrugge Apr 7, 2022
71bfcd0
added preprocessing when opening forcing files to read lat lon as coo…
bvanosnabrugge Apr 7, 2022
4a79b26
changed generate_mizuroute_topology to package version
bvanosnabrugge Apr 7, 2022
f3de148
merge mesh development from package-dev-mesh branch
bvanosnabrugge Apr 7, 2022
c273669
resolved conflict - two different imports on the same line
bvanosnabrugge Apr 7, 2022
9bfb869
Added how to test the mesh model_specific part
bartvanosnabrugge Apr 7, 2022
68575b7
era5 download script in python
bvanosnabrugge Apr 14, 2022
89921ef
update to era5 download and split download and further steps in test …
bvanosnabrugge May 9, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 32 additions & 11 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,14 +1,35 @@
# Logs and databases #
######################
slurm-*.out
output*.out
ERA5_createForcing_CONUS*.sh
core.*
# Byte-compiled / optimized / DLL files
__pycache__/
*/__pycache__/
*.py[cod]
*$py.class

# notebook stuff
.ipynb_checkpoints
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
*.pyc

# rtd
rtd/build
# Sphinx documentation
docs/_build/

rtd/source/*.md
# test suite
tests/*/results

# code editors
.vscode
49 changes: 49 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
CWARHM
=======

**CWARHM** is a Python library to organize workflows to build hydrological models.
The Package branch is to develop the original SUMMA-CWARHM (currently on main, in this branch moved under dependencies/summa-cwarhm) into a python package where the code exectuting workflow steps are callable functions, and the actual workflows (here a workflow are all the steps involved in setting up an hydrological model) are scripts calling the available functions in a certain order.

Installation
--------------
The dependencies of the CWARHM package (or CWARHM-Assembler - CWARHMA) can be installed using conda:

``cd /path/to/summaWorkflow_public``

``conda env create -f environment.yml``

``conda activate cwarhm-env``


Additionally the package can be installed in the environment in developer mode (not needed for the test):

``cd /path/to/summaWorkflow_public``

``conda activate cwarhm-env``

``pip install -e .``

Test Bow at Banff SUMMA
----------
As test case workflows/summa_bowatbanff/ is available. Before running the test script test_bow_at_banff.py, some path modifications have to be made:

1.) In workflows/summa_bowatbanff/control_Bow_at_Banff_test.txt, change the **root_path** modeling domain setting to a local folder where the test results should be saved.

2.) In workflows/summa_bowatbanff/test_bow_at_banff.py, change the paths:
- cwarhm_summa_folder; to point to the `./summaWorkflow_public/dependencies/cwarhm-summa` folder
- results_folder_path , to match the root_path in the control file
- test_data_path , path to the test data, if you want to skip the data download workflow steps (by default)
- reset_test to True (default), this is a flag that starts a new run (and deletes all data from any results folder), and restarts by copying the test data to the results folder or False: continue with existing data in the results folder.

3.) The test data is not part of this repo due to its size (30+GB). For those in the comphyd group it can be found here:

**copernicus** /project/gwf/gwf_cmt/cwarhm_test_data
**GRAHAM** /project/6008034/CompHydCore/cwarhm_test_data

If this is not accessible, you can download the data with the CWARHM functions (wrapped from the original CWARHM). You can also use a results directory from an earlier test run.

Test Bow at Banff MESH
---------
The Bow at Banff MESH test, performs a part of a complete workflow (with data specific parts and most of the model agnostic parts processed by the above workflow). to have a look at the relevant functions have a look at tests/test_mesh_bowatbanff.py
Note that for it to run some input data is needed that is now included in the test folder and will be extracted automatically (21MB).
Only one path has to adjusted, but in two(!) places (see test/test_mesh_bowatbanff.py).
Empty file added cwarhm/__init__.py
Empty file.
Empty file.
Loading