v8.0.1

Changes from v8.0.0:

• Add internal stream iterator method to sidestep Safari ReadableStream limitations. (Fixes #389)

Full Changelog: v8.0.0...v8.0.1

v8.0.0

Changes from v7.2.1:

• Breaking: Drop custom Arquero JSON format with embedded schema. It provides little if any additional value (as the schema does not include type information), and I don't have any evidence that people use it.
• Add support to load data from input readable streams.
• Add full streaming readers for CSV, fixed width, and NDJSON formats.
• Add support for gzip and deflate decompression of input streams.
• Add fromArrowStream, fromCSVStream, fromFixedStream, and fromJSONStream methods for async stream parsing.
• Expand fromJSON and toJSON to support a JSON type property, for control over row-oriented, column-oriented, and newline-delimited (NDJSON) formats.
• Update build, including package.json use of the browser property, for more targeted node/web separation.
• Update test cases to reflect changes.

Full Changelog: v7.2.1...v8.0.0

v7.2.1

Changes from v7.2.0:

• Update dependencies, including Arrow IPC encoding fixes via flechette.
• Minor test and code hygiene fixes.

Full Changelog: v7.2.0...v7.2.1

v7.2.0

Changes from v7.1.0:

• Add default key and value determination for lookup verb. The default keys are now all columns with matching names. The default values are all columns in the secondary table whose names do no match any columns in the primary table. (#379)
• Tighten table expression type signature.
• Update lookup verb types and documentation.
• Update dependencies.

Full Changelog: v7.1.0...v7.2.0

v7.1.0

Changes from v7.0.0:

• Add collate helper for custom sort orders.
• Add toCSV header option.
• Update toCSV to include a trailing newline.
• Update comparator parser/generator to support collation comparators.
• Update tests and documentation.

Full Changelog: v7.0.0...v7.1.0

v7.0.0

Changes from v6.0.1:

• Breaking: Change the default handler of Apache Arrow data from the apache-arrow JS reference implementation to the smaller, faster, and more flexible @uwdata/flechette.

  This change improves performance and simplifies Arquero internals, as it obviates the need for internal custom Arrow data conversion steps and a variety of earlier performance and correctness workarounds. Arrow IPC binary decoding (in fromArrow) and building/encoding (in toArrow and toArrowIPC) now use Flechette.

  If passing an already instantiated Arrow table to fromArrow, Arquero supports both apache-arrow and @uwdata/flechette tables, including optimizations for groupby and filter operations over dictionary types. However, special extraction of decimals, dates, and nested types (lists, structs) is no longer provided by Arquero; instead we default to whatever the Arrow implementation provides. For Flechette, this includes customizable extraction options and efficient serialization to native JS array and object types. For Arrow-JS, decimals are not well-supported and lists are returned as Arrow Vector types and structs as slow Proxy row objects.

  If loading data directly from IPC bytes, Arquero results should be consistent. Moreover, as Flechette is small, it is bundled directly with Arquero, so (unlike before) no additional imports are needed to process Arrow data. However, when passing-in already-parsed apache-arrow tables, results will differ for some data types.

Full Changelog: v6.0.1...v7.0.0

v6.0.1

• Update package.json, use main again to resolve CDN issues.
• Update dependencies.

v6.0.0

• Breaking: Switch project to type module, update imports for node compatibility.
• Breaking: Remove serialized Query functionality; it does not seem to be used at all.
• Breaking: Limit extensibility methods to op functions only, remove table method and package registration. New table methods can instead be added by directly assigning them to ColumnTable.prototype.
• Breaking: Remove Table.columns() method. Use Table.data() instead.
• Breaking: Remove Table.columnArray() method. Use Table.array() instead.
• Breaking: Rename toArrowBuffer() to toArrowIPC().
• Breaking: Refactor module exports to be flat, including internal classes and toXXX format methods.
• Add lightweight base Table representation, ColumnTable subclass adds transformations and format methods.
• Add array type checks for columns to optimize access.
• Add more Arrow conversions (Date, Decimal, etc.) and corresponding arrowColumn() option flags.
• Add object freeze to table column object.
• Remove internal column wrapper interface, use array-like objects directly.
• Refactor verb definitions to a single location, remove previous engine folder.
• Refactor verb argument parsing to consolidate in top-level verb methods.
• Refactor all Apache Arrow functionality to the arrow folder.
• Use Mocha as the test runner.
• Improve TypeScript types and testing.
• Update build process, drop node bundle, retain UMD builds.
• Update dependencies.

v5.4.1

Changes from v5.4.0:

• Update dependencies and dev dependencies (thanks @domoritz)
• Fix errors in JSDoc TypeScript annotations (thanks @FilipBac)
• Update links in documentation (thanks @HQFOX)

v5.4.0

Changes from v5.3.0:

• Chore: Drop yarn, use npm only. Update docs accordingly.
• CI: Update CI settings.
• Deps: Bump dependencies.