Fix library lookup path in the vendored OpenSSL

[mislav]

When a vendored OpenSSL is needed for compiling Ruby, that OpenSSL installation ends up with its bin/openssl executable broken due to not finding "libssl.so" and "libcrypto.so" in the global load path for libraries. This doesn't seem to negatively affect the Ruby "openssl" extension at runtime, but does constitute a broken OpenSSL installation nevertheless.

This change causes the bin/openssl executable and related shared libraries to be built with an "RPATH" pointing to the "lib" directory of the vendored OpenSSL. See https://wiki.openssl.org/index.php/Compilation_and_Installation#Using_RPATHs

Fixes #2488

[eregon]

I checked locally and it works, thank you for the quick fix!

[jeremy]

clang -fPIC -arch arm64 -O3 -Wall -L. -dynamiclib -current_version 1.1 -compatibility_version 1.1 -Wl,-search_paths_first -L/opt/homebrew/opt/ruby/lib -Wl,-rpath=/Users/dorin/.local/share/mise/installs/ruby/2.7.8/openssl/lib -install_name /Users/dorin/.local/share/mise/installs/ruby/2.7.8/openssl/lib/libcrypto.1.1.dylib -o …
                  
ld: unknown options: -rpath=/Users/dorin/.local/share/mise/installs/ruby/2.7.8/openssl/lib 
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Looks like this needs to use -Wl,-rpath,"$OPENSSL_PREFIX_PATH/lib" for Clang (comma rather than equals).

[jeremy]

RPATH is automatically set up on BSDs, so this is only needed on Linux. Also, comma syntax appears to be preferred.

Suggested change

	package_option openssl configure --openssldir="$OPENSSLDIR" --libdir="lib" -Wl,-rpath="$OPENSSL_PREFIX_PATH/lib" zlib-dynamic no-ssl3 shared ${nokerberos:+no-ssl2 no-krb5}
	# Add RPATH for Linux/GCC
	case "$(uname -s)" in
	Linux)
	if cc -v 2>&1 | grep -q "gcc"; then
	EXTRA_RPATH_FLAGS="-Wl,-rpath,${OPENSSL_PREFIX_PATH}/lib -Wl,--enable-new-dtags"
	fi
	;;
	esac
	package_option openssl configure --openssldir="$OPENSSLDIR" --libdir="lib" zlib-dynamic no-ssl3 shared ${nokerberos:+no-ssl2 no-krb5}

[eregon]

Could you make a PR?
BTW clang on Linux would also need the RPATH, so we shouldn't do this only with gcc.
Re --enable-new-dtags I don't know if necessary, I got a RUNPATH anyway without it, it seems recent Linux changed the default anyway.
(BTW I wonder why OpenSSL only sets up RPATH automatically on BSD (according to docs), where it's probably more complicated even)

[jeremy]

Can't shepherd through a PR or test on all these platforms at the moment. Recommend reverting this PR for now so macOS users can install Ruby.

Note that --enable-new-dtags is set by default on some distros but not others. If you have a RUNPATH, it's because it's on by default for you (Gentoo, Debian, … others?). Behavior is subtly different with RUNPATH taking precedence and respecting LD_LIBRARY_PATH thereafter.

[mislav]

Thanks @jeremy. I've undone rpath argument passing on non-Linux platforms, and added clang detection.

I've also expedited a release thinking that the previous change already made it into a release, but it turns out I've just released both changes together, so fingers crossed that this is the right incantation. https://github.com/rbenv/ruby-build/releases/tag/v20250114

If more regressions are discovered, I think I would also revert the whole rpath thing since it's more trouble than it's worth. I must admit I don't know much about RPATH vs RUNPATH, but openssl should not need this many extra compiler flags just to produce a working build in a custom location.

[jeremy]

Ah! Looks like mise installs ruby-build from git which gave us an early preview of the forthcoming release. Nice to catch this early, then.

[eregon]

I filed two openssl issues upstream for the gotchas mentioned here: openssl/openssl#26422 openssl/openssl#26423