Run PySpark jobs in EMR for Workbench
- User-friendly CLI to assist in configuring, submitting, tracking EMR jobs
- Support EMR Serverless
Adds Exception handling in when no-logs are available to display
Add APP_GEN1 tag in every run to assist in tracking for OneAI platform team
- Alpha release of EMR EKS support
- Logs are now printed periodically
- Add option for ping_duration when tracking the job status
- Add option to add tags for the jobs
- Add option to add execution timeout for a job
- During code package have the ability to add location path
- Removed EMR EKS support (temporarily)
- Refactored code
- Add more functionality to build dependencies and run
- Ability to only show appended logs
- Rename arguments of package dependencies
- functions will have a return response (useful when using library as an API)
- Fix minor bugs
- fix minor bugs
- add isort and pre-commit in dev
- remove unused imports
- add more configurations to build package
- remove dependency of implicit tags
- refactor code
- udate docstrings
- update compiling dependency package to remove hidden folders
- added test cases
- fix minor bugs
- updated version of aiohttp > 3.9.2
- Added doc-strings
- updated build and run command configurations
- refactored redundant code and library requirements
- update requirements
- remove dev dependencies