vendored zlib 1.3 doesn't work with macOS SDK 11 + clang19 and later (upgrade to 1.3.1)

[malt3]

Description of the bug:

The version of zlib currently vendored in Bazel (third_party/zlib) contains zlib 1.3.
Newer versions of clang and the macOS SDK cannot compile it, due to fdopen not being defined.
Upgrading to zlib 1.3.1 helps (I tested it by patching remote_java_tools).

Which category does this issue belong to?

Java Rules

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Use rules_java and set the macOS SDK to 11 or later. Build any java_library, java_binary or java_test.

Which operating system are you running Bazel on?

macOS

What is the output of bazel info release?

release 6.5.0 (tested with later versions up to 8.0.0)

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

N/A

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

Other projects also reported issues with zlib 1.3 and newer macOS SDKs: pocoproject/poco#4586

Any other information, logs, or outputs that you want to share?

Output when trying to build zlib as part of java_tools:

  ERROR: /private/var/tmp/_bazel_runner/e658bb24d378780e146cce538fc6c71c/external/remote_java_tools/java_tools/zlib/BUILD.bazel:18:11: Compiling java_tools/zlib/zutil.c [for tool] failed: (Exit 1): cc_wrapper.sh failed: error executing command (from target @remote_java_tools//java_tools/zlib:zlib) 
    (cd /private/var/tmp/_bazel_runner/e658bb24d378780e146cce538fc6c71c/sandbox/processwrapper-sandbox/3/execroot/rules_nixpkgs_java_testing && \
    exec env - \
      PATH=/nix/store/igfwnnvi05rz0mxdwh058v6n2h49zxi3-bash-5.2p37/bin:/nix/store/5r1vifkx3r6frjv7ry294wr5ripyfcq8-coreutils-9.5/bin:/nix/store/5akj0i2678z9rw7hfhld0sb5pil6x0ac-diffutils-3.10/bin:/nix/store/92rhvhbznl1hr3s9nzjlnh8inwkj5jjx-file-5.46/bin:/nix/store/y8saag3bdrpmm9lg77s4zlf228c2p5av-findutils-4.10.0/bin:/nix/store/5gk32d118di010z5sagdn3fn369mn1h7-gawk-5.3.1/bin:/nix/store/81bbk94chb8bx151i0ggr0p6kd92vkfy-gnugrep-3.11/bin:/nix/store/mnqf31z6laz34i1aw5s7kn1fxcv9sx2l-patch-2.7.6/bin:/nix/store/40j8br4hnzpfx92swmrvsmgia0ia1vgg-gnused-4.9/bin:/nix/store/rm81cyl4bpr5yi3l9vway0yzkm74s0bb-gnutar-1.35/bin:/nix/store/pnydkl0h7s8qx0nni5albjnbidfvc2vs-gzip-1.13/bin:/nix/store/45g1q2ama4j033d5qi2lgfzy1xmagnmf-python3-3.12.8/bin:/nix/store/j7mnxx3g72q1b4fz9m8vyb8ij7dac24c-unzip-6.0/bin:/nix/store/rjalkk4jk9ybac657n132zvlf4mrlaa7-which-2.21/bin:/nix/store/8yn92rjaymp0jg56s2ga5qspil0912vd-zip-3.0/bin:/nix/store/hw96mj4xvmnql853miy56ksyd240lixc-bash-interactive-5.2p37/bin:/nix/store/xm705bmbpmq0clffq2crznf1dqfs8v3p-bazel-6.5.0/bin:/nix/store/smhxyrqzvgd1dq6rybisaclqlh027da8-bazel-buildtools-7.3.1/bin:/nix/store/3jaczspfki3262h5wljq3hs7y3d1y8fv-gcc-wrapper-14-20241116/bin:/nix/store/mg083f4hcmzxs06yssw3zgcbxj9zkw9c-gcc-14-20241116/bin:/nix/store/5r1vifkx3r6frjv7ry294wr5ripyfcq8-coreutils-9.5/bin:/nix/store/2pwca1kspfil3r2pna1xiv8s1nakqlqq-cctools-binutils-darwin-wrapper-1010.6/bin:/nix/store/fqdszn5v7xm6ks1vckhipagvy2v4jn9k-cctools-binutils-darwin-1010.6/bin:/nix/store/g4yb78f3k17hycwh6dk76lg34q165b50-nix-2.24.12/bin:/nix/store/y9ilj7w794c9il2s04xx5ikg9kf7zmm4-git-2.47.1/bin:/nix/store/5i0yzl0ll8f6q9lwg8shhqj0xx87zrgx-openssh-9.9p1/bin:/nix/store/05lm9x40bab10mw9fpabkrqz9zmkwlbp-clang-wrapper-19.1.6/bin:/nix/store/fj9lmfyr96ciyx63bf036dg1g8qn4yx8-clang-19.1.6/bin:/nix/store/qf3w254mvzn5id5hwhv9dr2cpiq533lc-cctools-binutils-darwin-wrapper-1010.6/bin:/nix/store/s1wfd12p4r86gnsjwkmkc9yx1xjj043p-xcbuild-0.1.1-unstable-2019-11-20-xcrun/bin:/nix/store/yx78p8xhl4m1838qnal0q8yqrxphzc91-cctools-1010.6/bin:/nix/store/lr7yd2fg1b6ciff3i2m9jbrj24nhizfq-libiconv-107/bin:/nix/store/5r1vifkx3r6frjv7ry294wr5ripyfcq8-coreutils-9.5/bin:/nix/store/y8saag3bdrpmm9lg77s4zlf228c2p5av-findutils-4.10.0/bin:/nix/store/5akj0i2678z9rw7hfhld0sb5pil6x0ac-diffutils-3.10/bin:/nix/store/40j8br4hnzpfx92swmrvsmgia0ia1vgg-gnused-4.9/bin:/nix/store/81bbk94chb8bx151i0ggr0p6kd92vkfy-gnugrep-3.11/bin:/nix/store/5gk32d118di010z5sagdn3fn369mn1h7-gawk-5.3.1/bin:/nix/store/rm81cyl4bpr5yi3l9vway0yzkm74s0bb-gnutar-1.35/bin:/nix/store/pnydkl0h7s8qx0nni5albjnbidfvc2vs-gzip-1.13/bin:/nix/store/gzsznkc3sxhbr9v2fpmg3b4b7h7qzvjv-bzip2-1.0.8-bin/bin:/nix/store/85rlha3lfj2l14c9d0fz3k0crvps3baw-gnumake-4.4.1/bin:/nix/store/igfwnnvi05rz0mxdwh058v6n2h49zxi3-bash-5.2p37/bin:/nix/store/5d20l54js1snzsbkmhz88mz7ranfgr9b-patch-2.7.6/bin:/nix/store/9gcv4rzvxv1jl0i5pgsigcn93c08628w-xz-5.6.3-bin/bin:/nix/store/92rhvhbznl1hr3s9nzjlnh8inwkj5jjx-file-5.46/bin:/nix/store/mnqf31z6laz34i1aw5s7kn1fxcv9sx2l-patch-2.7.6/bin:/nix/store/45g1q2ama4j033d5qi2lgfzy1xmagnmf-python3-3.12.8/bin:/nix/store/j7mnxx3g72q1b4fz9m8vyb8ij7dac24c-unzip-6.0/bin:/nix/store/rjalkk4jk9ybac657n132zvlf4mrlaa7-which-2.21/bin:/nix/store/8yn92rjaymp0jg56s2ga5qspil0912vd-zip-3.0/bin \
      *** \
    external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections -MD -MF bazel-out/darwin-opt-exec-C7777A24/bin/external/remote_java_tools/java_tools/zlib/_objs/zlib/zutil.d '-frandom-seed=bazel-out/darwin-opt-exec-C7777A24/bin/external/remote_java_tools/java_tools/zlib/_objs/zlib/zutil.o' '-DBAZEL_CURRENT_REPOSITORY="remote_java_tools"' -iquote external/remote_java_tools -iquote bazel-out/darwin-opt-exec-C7777A24/bin/external/remote_java_tools -isystem external/remote_java_tools/java_tools/zlib -isystem bazel-out/darwin-opt-exec-C7777A24/bin/external/remote_java_tools/java_tools/zlib -g0 -Wno-deprecated-non-prototype -Wno-unused-variable -Wno-implicit-function-declaration -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/remote_java_tools/java_tools/zlib/zutil.c -o bazel-out/darwin-opt-exec-C7777A24/bin/external/remote_java_tools/java_tools/zlib/_objs/zlib/zutil.o)
  # Configuration: ae22cc30df5a1eb6750de571901cf25455ee37480e4ad39595563e94345ffa8c
  # Execution platform: @rules_nixpkgs_core//platforms:host
  
  Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
  In file included from external/remote_java_tools/java_tools/zlib/zutil.c:8:
  external/remote_java_tools/java_tools/zlib/zutil.h:170:11: warning: 'OS_CODE' macro redefined [-Wmacro-redefined]
    170 | #  define OS_CODE 19
        |           ^
  external/remote_java_tools/java_tools/zlib/zutil.h:141:11: note: previous definition is here
    141 | #  define OS_CODE  7
        |           ^
  In file included from external/remote_java_tools/java_tools/zlib/zutil.c:10:
  In file included from external/remote_java_tools/java_tools/zlib/gzguts.h:20:
  /nix/store/pmk5kk8a0dgcai69bvx4799pl5nm4nbk-apple-sdk-11.3/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:210:7: error: expected identifier or '('
    210 | FILE    *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
        |          ^
  external/remote_java_tools/java_tools/zlib/zutil.h:147:33: note: expanded from macro 'fdopen'
    147 | #        define fdopen(fd,mode) NULL /* No fdopen() */
        |                                 ^
  /nix/store/05lm9x40bab10mw9fpabkrqz9zmkwlbp-clang-wrapper-19.1.6/resource-root/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
     26 | #define NULL ((void*)0)
        |                ^
  In file included from external/remote_java_tools/java_tools/zlib/zutil.c:10:
  In file included from external/remote_java_tools/java_tools/zlib/gzguts.h:20:
  /nix/store/pmk5kk8a0dgcai69bvx4799pl5nm4nbk-apple-sdk-11.3/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:210:7: error: expected ')'
  external/remote_java_tools/java_tools/zlib/zutil.h:147:33: note: expanded from macro 'fdopen'
    147 | #        define fdopen(fd,mode) NULL /* No fdopen() */
        |                                 ^
  /nix/store/05lm9x40bab10mw9fpabkrqz9zmkwlbp-clang-wrapper-19.1.6/resource-root/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
     26 | #define NULL ((void*)0)
        |                ^
  /nix/store/pmk5kk8a0dgcai69bvx4799pl5nm4nbk-apple-sdk-11.3/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:210:7: note: to match this '('
  external/remote_java_tools/java_tools/zlib/zutil.h:147:33: note: expanded from macro 'fdopen'
    147 | #        define fdopen(fd,mode) NULL /* No fdopen() */
        |                                 ^
  /nix/store/05lm9x40bab10mw9fpabkrqz9zmkwlbp-clang-wrapper-19.1.6/resource-root/include/__stddef_null.h:26:15: note: expanded from macro 'NULL'
     26 | #define NULL ((void*)0)
        |               ^
  In file included from external/remote_java_tools/java_tools/zlib/zutil.c:10:
  In file included from external/remote_java_tools/java_tools/zlib/gzguts.h:20:
  /nix/store/pmk5kk8a0dgcai69bvx4799pl5nm4nbk-apple-sdk-11.3/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:210:7: error: expected ')'
    210 | FILE    *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
        |          ^
  external/remote_java_tools/java_tools/zlib/zutil.h:147:33: note: expanded from macro 'fdopen'
    147 | #        define fdopen(fd,mode) NULL /* No fdopen() */
        |                                 ^
  /nix/store/05lm9x40bab10mw9fpabkrqz9zmkwlbp-clang-wrapper-19.1.6/resource-root/include/__stddef_null.h:26:22: note: expanded from macro 'NULL'
     26 | #define NULL ((void*)0)
        |                      ^
  /nix/store/pmk5kk8a0dgcai69bvx4799pl5nm4nbk-apple-sdk-11.3/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:210:7: note: to match this '('
  external/remote_java_tools/java_tools/zlib/zutil.h:147:33: note: expanded from macro 'fdopen'
    147 | #        define fdopen(fd,mode) NULL /* No fdopen() */
        |                                 ^
  /nix/store/05lm9x40bab10mw9fpabkrqz9zmkwlbp-clang-wrapper-19.1.6/resource-root/include/__stddef_null.h:26:14: note: expanded from macro 'NULL'
     26 | #define NULL ((void*)0)
        |              ^
  1 warning and 3 errors generated.
  INFO: Elapsed time: 34.013s, Critical Path: 0.88s
  INFO: 36 processes: 34 internal, 2 processwrapper-sandbox.
  FAILED: Build did NOT complete successfully
  //tests:java-test                                               FAILED TO BUILD

[fmeum]

@hvadehra for java_tools

[Neustradamus]

To follow.

[meteorcloudy]

Have your project migrated to Bzlmod or have you tried a newer Bazel version?

In Bazel 8, zlib 1.3.1.bcr.1 should be used.
https://cs.opensource.google/bazel/bazel/+/master:src/MODULE.tools;l=11?q=MODULE.tools&ss=bazel

[meteorcloudy]

But yes, we should get rid of the vendored zlib in Bazel source.