Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚡️ Speed up function _attach_resources_to_jobs_and_instigator_jobs by 45% in python_modules/dagster/dagster/_core/definitions/definitions_class.py #65

Open
wants to merge 1 commit into
base: codeflash/optimize-remove_none_recursively-2024-06-26T09.20.53
Choose a base branch
from

Commits on Jul 25, 2024

  1. ⚡️ Speed up function _attach_resources_to_jobs_and_instigator_jobs

    …by 45%
    
    Certainly! Here's an optimized version of the provided program. I've made improvements to minimize redundant looping and dictionary look-up operations.
    
    
    
    ### Key Optimizations.
    1. **Combine Job Collections**: Collect all jobs from the primary `jobs` list, `schedules`, and `sensors` into a single dictionary first to eliminate redundant duplicates.
    2. **Single Pass for Jobs**: Handle unsatisfied jobs and prepare resource-bound versions in a single pass over the `jobs` list.
    3. **Single Pass for Updates**: Update all `schedules` and `sensors` to use the resource-bound job versions in one loop.
    codeflash-ai[bot] authored Jul 25, 2024
    Configuration menu
    Copy the full SHA
    57e08de View commit details
    Browse the repository at this point in the history