Skip to content

v3.0.0
Compare
Choose a tag to compare
@RamazanKara RamazanKara released this 06 Jan 23:55
· 12 commits to main since this release
v3.0.0
22293f9

v3.0.0

Key Enhancements

  • Parallel File Processing: Implemented an advanced parallel processing approach where the input file is divided into chunks and processed independently in parallel, drastically reducing I/O bottleneck.
  • Optimized Memory Management: Refined memory usage by processing data in chunks and employing local maps for data aggregation to reduce memory overhead.
  • Improved Data Aggregation: Enhanced the efficiency of data aggregation through the use of sharded data structures, minimizing lock contention.

Processing Time: 1m3s. Tested with a Ryzen 5800x3d and 32 gigs Ram

Assets 2
Loading