d3m_outputsCLI renamed todval. Still compatible withd3m_outputs- valid8 package added as submodule
- Added DSE and D3M templates for template-based validation with valid8
- Support for MXE metric added
- CLI-only times improved x30
- Predictions file index now reads from problem schema
- General reformatting & code quality changes
- Validation of categorical data no longer fails with unknown values
- Columns and rows being in the target's order is not required anymore
- Add subpipelines support
- Remove a noisy logging print when validating non-numerical prediction files
- New implementation of l2_avg
- Fix bugs occuring when running unittest discover
- Fix division by zero in normalize method
- Add an option in pipelines validation to skip the rank validation
- objectDetectionAP metric now supports str parsed arrays as an input
- Fix Pipelines schema full validation
- Fix categorical targets validation
- Handle the case where test data and ground truth have different shapes
- Allow duplicate primitives in steps
- Add TA3 Task1 validation
- Fix some bugs
- Add transformed and normalized scores and baseline scores
First release for Summer 2018 Evaluation Validation and Scoring
- Add a CHANGELOG.md file tracking version changes.
- Support for MIT-LL data specification v3.1
- Support for v3.1 metrics and metric parameters
- f1 parameter
- score on all targets
- Added new metrics precision and recall (object metric still TBD)
- Task 2 scoring - summer 2018 specification
- Support for metalearning format pipelines
Last release according to February 2018 Evaluation standards.
Released on April 4th 2018
- Added post-search validation
- Added test script generation