Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make all manager methods async #273

Merged
merged 2 commits into from
Feb 24, 2025
Merged

make all manager methods async #273

merged 2 commits into from
Feb 24, 2025

Conversation

minrk
Copy link
Contributor

@minrk minrk commented Feb 18, 2025

consistent API makes addressing #204 easier. I ran into issues because my approach to list_clusters relies on making a request to the Gateway to list clusters, which should be async, instead of relying on in-memory state which may be out-of-date. I also found it simpler to avoid holding open connections to all clusters, instead only connecting when information is requested, so the Manager is essentially stateless when working with a Gateway.
Gateway clusters also have a 'pending' state, which local clusters don't seem to, which is important to consider, since connecting to a pending cluster will not succeed until/unless the cluster starts.

Also moves the global manager state from module global to initialized at extension load time, which works better when instantiating an asyncio task, which may not connect to the right event loop.

allows easier, more consistent override

- defer initialization to `await manager` instead of immediately (avoids invoking asyncio at import time)
- instantiate default manager as part of extension loading, not at import time
Copy link
Member

@jacobtomlinson jacobtomlinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a good improvement to me! Thanks!

@jacobtomlinson jacobtomlinson merged commit 19d5dac into dask:main Feb 24, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants