Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GFS Data Sampling #50

Merged
merged 3 commits into from
Jan 30, 2025
Merged

GFS Data Sampling #50

merged 3 commits into from
Jan 30, 2025

Conversation

siddharth7113
Copy link
Contributor

@siddharth7113 siddharth7113 commented Jan 29, 2025 •
edited
Loading

Description

This PR introduces the GFSDataSampler by improving dataset handling, normalization, and logging.

  • Code for better readability and maintainability.
  • Improved NaN handling with a dedicated handle_nan_values function.
  • Updated gfs_config.yaml to include new channels and ensure consistency.

Fixes: #1

How Has This Been Tested?

  • Manually tested dataset loading, NaN handling, and normalization.
  • Plotted data before and after normalization to ensure correctness.
  • Tested expected dimensions and structure.

✅ Sanity check (plotted data changes) → Yes

Checklist:

@siddharth7113
feat(gfs_data_sampler): dataset handling, normalization, and logging
bb193ee 
- `GFSDataSampler` for improved readability and maintainability.
- Added comprehensive docstrings and comments for documentation.
- Enhanced NaN handling with `handle_nan_values` function.
- Updated dataset configuration in `gfs_config.yaml`:
  - Defined appropriate time resolution and forecast step intervals.
  - Included new channels for GFS data processing.

- Reintroduce optimized test cases with improved efficiency.
@siddharth7113
Copy link
Contributor Author

@peterdudfield
Sorry to open a new PR but today I faced this https://xkcd.com/1597 :)

@siddharth7113
Copy link
Contributor Author

I think this is ready now

@peterdudfield
Copy link
Contributor

Looks great, thank you for this

@peterdudfield
Copy link
Contributor

Shallwe merge?

@siddharth7113
Copy link
Contributor Author

Shallwe merge?

Yes ,please

@peterdudfield peterdudfield merged commit 2cfc1f2 into openclimatefix:main Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@peterdudfield peterdudfield peterdudfield approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@siddharth7113 @peterdudfield