Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support unit conversion within Friendly data #24

Open
suvayu opened this issue Jan 26, 2022 · 2 comments
Open

Support unit conversion within Friendly data #24

suvayu opened this issue Jan 26, 2022 · 2 comments
Labels
API enhancement New feature or request

Comments

@suvayu
Copy link
Member

suvayu commented Jan 26, 2022

There are two kinds of units to deal with: physical units, and economic units. Domain specific physical units can be defined and maintained within friendly data using pint. For economic units, salamanca can be used.

Since units are often added as columns (it's also in the registry), adopting that is acceptable, although it inflates file size. An alternative would be to add the unit to the metadata of all relevant columns, similar to alias. However, that is essentially a "fork" of the frictionless spec, and would add a maintenance burden.

As for behaviour during conversions, when doing it in-memory using the Python API, it is simple, the unit conversion functions can do the conversion while reading a dataset into memory. There is no reason why the API cannot also work on dataframes/data arrays directly. When using the CLI, we can adopt the --export/--inplace modes of operation. The functionality can be a separate module, like tseries.

@suvayu suvayu added enhancement New feature or request API labels Jan 26, 2022
@danielhuppmann
Copy link

Let me also point you to the iam-units package (https://github.com/IAMconsortium/units), which is pint with a additional definitions for units common in energy systems modeling, like "ton of coal equivalent" and redefining kt to be "kiloton" instead of "knot". And iam-units also has a conversion module between emissions species (greenhouse gases) according to several global-warming-potential (GWP) metrics.

cc @khaeru @gidden

@suvayu
Copy link
Member Author

suvayu commented Jan 26, 2022

@danielhuppmann thanks for pointing to iam-units, it slipped my mind :(. Friendly data can of course build on top of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants