Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test suite uses unsafe multiprocessing start method fork on Linux #1274

Closed
jakob-keller opened this issue Jan 22, 2025 · 0 comments · Fixed by #1275
Closed

Test suite uses unsafe multiprocessing start method fork on Linux #1274

jakob-keller opened this issue Jan 22, 2025 · 0 comments · Fixed by #1275
Assignees

Comments

@jakob-keller
Copy link
Collaborator

Describe the bug
Our test suite uses a combination of multiprocessing and multithreading. This can be dangerous when the multiprocessing start method fork is used. Start method spawn would be the safe choice in this context.

By default, current Python releases default to spawn on Windows and macOS, but use fork on Linux, which is what we use in CI/CD. Since Python 3.12, this triggers DeprecationWarnings, such as here:

tests/test_stubber.py::test_add_client_error
/usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=2749) is multi-threaded, use of fork() may lead to deadlocks in the child.
self.pid = os.fork()

We should ensure that multiprocessing always uses spawn in the context of our test suite. I will propose a fix.

This could actually also explain the flakiness experienced in #1266. Fingers crossed...

@jakob-keller jakob-keller self-assigned this Jan 22, 2025
@jakob-keller jakob-keller changed the title Enforce multi Test suite uses unsafe multiprocessing start method fork on Linux Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant