Batch / Sample class - draft #125
Conversation
|
To mention, any point in updating type hints to be effectively more robust? I.e. using TypeVar: T = TypeVar('T', bound='SampleBase')
ArrayType = Union[np.ndarray, torch.Tensor, xr.DataArray]
class SampleBase(ABC, Generic[T]):
def to_torch(self) -> T:Suppose important for early and type related errors, particularly with multiple refactors. |
|
Updates implemented @Sukh-P |
|
Further update @Sukh-P - purely updating uk_regional.py - numbered file saving and loading similar to the reference logic provided: Previous function before this specific update (also pre-change init) left in uk_regional.py - just to check this is all in line with overall requirements here. |
| REQUIRED_KEYS = { | ||
| 'nwp', | ||
| GSPSampleKey.gsp, | ||
| SatelliteSampleKey.satellite_actual, |
There was a problem hiding this comment.
This could be optional
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
There was a problem hiding this comment.
Could be duplicate, to check
|
|
||
| # Fixture define | ||
| @pytest.fixture | ||
| def pvnet_config_filename(tmp_path): |
There was a problem hiding this comment.
move to json, check not in conftest already
|
|
||
| def plot(self, **kwargs) -> None: | ||
| """ Sample visualisation definition """ | ||
| logger.debug("Creating PVNetSample visualisation") |
There was a problem hiding this comment.
Could update later using ocf-datapipes visulation.
| logger.error(f"Invalid key type: {type(key)}") | ||
| raise TypeError(f"Key must be str or int, got {type(key)}") | ||
|
|
||
| # REFERENCE |
| GSPSampleKey.solar_elevation | ||
| } | ||
|
|
||
| # REFERENCE |
Overview
First stages of a standardised sample handling system, with unified interface across different dataset types - refer to issue #71. Initial focus regarding Dataset type is PVNet UK Regional. As of the moment, base.py and uk_regional.py function as effective parent and child classes, respectively.
Main Implementation
base.py
uk_regional.py