Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update loky to 3.5.1. #11341

Merged
merged 15 commits into from
Mar 19, 2025
Merged

Update loky to 3.5.1. #11341

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
294529e
[Don't merge] Test macos hang.
trivialfis Mar 15, 2025
d428977
test rabit.
trivialfis Mar 15, 2025
92a3e7a
Disable other tests for now.
trivialfis Mar 15, 2025
d9d8f35
full test suite.
trivialfis Mar 15, 2025
3b94fab
Fail fast.
trivialfis Mar 15, 2025
4b14886
freeze loky.
trivialfis Mar 15, 2025
6e5aba4
Revert.
trivialfis Mar 15, 2025
5982ac9
remove pyspark.
trivialfis Mar 15, 2025
8d2ca09
Freeze loky on Linux.
trivialfis Mar 17, 2025
722c99a
Linux.
trivialfis Mar 18, 2025
04330f7
Freeze.
trivialfis Mar 18, 2025
733caef
assert loky version.
trivialfis Mar 18, 2025
bad7028
revert assert.
trivialfis Mar 18, 2025
5d854c2
3.5.1
trivialfis Mar 19, 2025
36d104b
Return to main.
trivialfis Mar 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ops/conda_env/aarch64_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ dependencies:
- awscli
- numba
- llvmlite
- loky
- loky>=3.5.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of requiring latest, we could simply disallow the problematic version

This could also be done in setup.py and other environment files

Suggested change
- loky>=3.5.1
- loky!=3.5.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the suggestion! I think I would like to keep it as the latest for the CI, it's only used by the tests and we don't want to deal with more than one version in the CI.

- pyarrow
- pyspark>=3.4.0
- cloudpickle
Expand Down
2 changes: 1 addition & 1 deletion ops/conda_env/linux_cpu_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ dependencies:
- boto3
- awscli
- py-ubjson
- loky
- loky>=3.5.1
- pyarrow
- protobuf
- cloudpickle
Expand Down
11 changes: 2 additions & 9 deletions ops/conda_env/macos_cpu_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ dependencies:
- pip
- wheel
- pyyaml
- cpplint
- pylint
- numpy
- scipy
- llvm-openmp
Expand All @@ -20,22 +18,17 @@ dependencies:
- python-graphviz
- hypothesis
- astroid
- sphinx
- sh
- recommonmark
- mock
- breathe
- pytest
- pytest-cov
- pytest-timeout
- python-kubernetes
- urllib3
- jsonschema
- boto3
- awscli
- loky
- loky>=3.5.1
- pyarrow
- pyspark>=3.4.0
- cloudpickle
- pip:
- setuptools
- sphinx_rtd_theme
2 changes: 1 addition & 1 deletion ops/conda_env/win64_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ dependencies:
- python-graphviz
- pip
- py-ubjson
- loky
- loky>=3.5.1
- pyarrow
6 changes: 5 additions & 1 deletion tests/python/test_collective.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

import numpy as np
import pytest
from loky import get_reusable_executor

import xgboost as xgb
from xgboost import RabitTracker, build_info, federated
Expand All @@ -25,10 +24,13 @@ def run_rabit_worker(rabit_env: dict, world_size: int) -> int:

@pytest.mark.skipif(**tm.no_loky())
def test_rabit_communicator() -> None:
from loky import get_reusable_executor

world_size = 2
tracker = RabitTracker(host_ip="127.0.0.1", n_workers=world_size)
tracker.start()
workers = []

with get_reusable_executor(max_workers=world_size) as pool:
for _ in range(world_size):
worker = pool.submit(
Expand Down Expand Up @@ -60,6 +62,8 @@ def run_federated_worker(port: int, world_size: int, rank: int) -> int:
@pytest.mark.skipif(**tm.skip_win())
@pytest.mark.skipif(**tm.no_loky())
def test_federated_communicator() -> None:
from loky import get_reusable_executor

if not build_info()["USE_FEDERATED"]:
pytest.skip("XGBoost not built with federated learning enabled")

Expand Down
Loading