[3.12] gh-100228: Document the os.fork threads DeprecationWarning. (GH-109767) (#109773)

* gh-100228: Document the os.fork threads DeprecationWarning. (GH-109767)

Document the `os.fork` posix threads detected `DeprecationWarning` in 3.12 What's New, os, multiprocessing, and concurrent.futures docs.

Many reviews and doc cleanup edits by Adam & Hugo. 🥳

(cherry picked from commit 5e7ea95)

Co-authored-by: Gregory P. Smith <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>

* link to the discussion thread from whatsnew

Include the link to the discussion in the what's new text per @malemberg's comment on. #109767

(i'll follow up with a PR to main to include this edit there as well)

---------

Co-authored-by: Gregory P. Smith <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>

Author: 4 people

Doc/library/concurrent.futures.rst

@@ -293,6 +293,14 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
       The *max_tasks_per_child* argument was added to allow users to
       control the lifetime of workers in the pool.
 
+   .. versionchanged:: 3.12
+      On POSIX systems, if your application has multiple threads and the
+      :mod:`multiprocessing` context uses the ``"fork"`` start method:
+      The :func:`os.fork` function called internally to spawn workers may raise a
+      :exc:`DeprecationWarning`. Pass a *mp_context* configured to use a
+      different start method. See the :func:`os.fork` documentation for
+      further explanation.
+
 .. _processpoolexecutor-example:
 
 ProcessPoolExecutor Example

Doc/library/multiprocessing.rst

@@ -131,6 +131,12 @@ to start a process.  These *start methods* are
        Code that requires *fork* should explicitly specify that via
        :func:`get_context` or :func:`set_start_method`.
 
+    .. versionchanged:: 3.12
+       If Python is able to detect that your process has multiple threads, the
+       :func:`os.fork` function that this start method calls internally will
+       raise a :exc:`DeprecationWarning`. Use a different start method.
+       See the :func:`os.fork` documentation for further explanation.
+
   *forkserver*
     When the program starts and selects the *forkserver* start method,
     a server process is spawned.  From then on, whenever a new process

Doc/library/os.rst

@@ -4157,15 +4157,38 @@ written in Python, such as a mail server's external command delivery program.
 
    .. audit-event:: os.fork "" os.fork
 
+   .. warning::
+
+      If you use TLS sockets in an application calling ``fork()``, see
+      the warning in the :mod:`ssl` documentation.
+
    .. versionchanged:: 3.8
       Calling ``fork()`` in a subinterpreter is no longer supported
       (:exc:`RuntimeError` is raised).
 
-   .. warning::
-
-      See :mod:`ssl` for applications that use the SSL module with fork().
+   .. versionchanged:: 3.12
+      If Python is able to detect that your process has multiple
+      threads, :func:`os.fork` now raises a :exc:`DeprecationWarning`.
+
+      We chose to surface this as a warning, when detectable, to better
+      inform developers of a design problem that the POSIX platform
+      specifically notes as not supported. Even in code that
+      *appears* to work, it has never been safe to mix threading with
+      :func:`os.fork` on POSIX platforms. The CPython runtime itself has
+      always made API calls that are not safe for use in the child
+      process when threads existed in the parent (such as ``malloc`` and
+      ``free``).
+
+      Users of macOS or users of libc or malloc implementations other
+      than those typically found in glibc to date are among those
+      already more likely to experience deadlocks running such code.
+
+      See `this discussion on fork being incompatible with threads
+      <https://discuss.python.org/t/33555>`_
+      for technical details of why we're surfacing this longstanding
+      platform compatibility problem to developers.
 
-   .. availability:: Unix, not Emscripten, not WASI.
+   .. availability:: POSIX, not Emscripten, not WASI.
 
 
 .. function:: forkpty()
@@ -4178,6 +4201,11 @@ written in Python, such as a mail server's external command delivery program.
 
    .. audit-event:: os.forkpty "" os.forkpty
 
+   .. versionchanged:: 3.12
+      If Python is able to detect that your process has multiple
+      threads, this now raises a :exc:`DeprecationWarning`. See the
+      longer explanation on :func:`os.fork`.
+
    .. versionchanged:: 3.8
       Calling ``forkpty()`` in a subinterpreter is no longer supported
       (:exc:`RuntimeError` is raised).

Doc/whatsnew/3.12.rst

@@ -1065,6 +1065,20 @@ Deprecated
   contain the creation time, which is also available in the new ``st_birthtime``
   field. (Contributed by Steve Dower in :gh:`99726`.)
 
+* :mod:`os`: On POSIX platforms, :func:`os.fork` can now raise a
+  :exc:`DeprecationWarning` when it can detect being called from a
+  multithreaded process. There has always been a fundamental incompatibility
+  with the POSIX platform when doing so. Even if such code *appeared* to work.
+  We added the warning to to raise awareness as issues encounted by code doing
+  this are becoming more frequent. See the :func:`os.fork` documentation for
+  more details along with `this discussion on fork being incompatible with threads
+  <https://discuss.python.org/t/33555>`_ for *why* we're now surfacing this
+  longstanding platform compatibility problem to developers.
+
+  When this warning appears due to usage of :mod:`multiprocessing` or
+  :mod:`concurrent.futures` the fix is to use a different
+  :mod:`multiprocessing` start method such as ``"spawn"`` or ``"forkserver"``.
+
 * :mod:`shutil`: The *onerror* argument of :func:`shutil.rmtree` is deprecated as will be removed
   in Python 3.14. Use *onexc* instead. (Contributed by Irit Katriel in :gh:`102828`.)